Bluemini.comBluemini.com

Headless IE rendering

posted: 01 Nov 2013

If you've ever used or heard of phantom.js, you'll know the advantages that a tool like this can bring to testing web front ends. However, until recently, there was no way to capture rendered output from other rending engines besides webkit/blink. Now, however, a new headless rendering engine has joined the scene, TrifleJS. Trifle still uses the V8 JavaScript engine, but uses IE's Trident rendering engine to accomplish the --render output.

If you know the Phantom API, you shouldn't find Trifle too daunting as it is a port of Phantom. Now we can at least verify renderings, if not JS compatibility, across both Chrome and IE.

http://triflejs.org/

keywords: testing phantomjs

ActiveMQ and JRun4's JMS collision

posted: 18 Sep 2013

It seems that the latest versions of Apache ActiveMQ (5.*) are now using JMS 1.1 rather than the earlier (4.*) versions that used JMS 1.0. This means that when enabling ActiveMQ 5.* in our CF applications, we were getting some inconsistent behavior when trying to send messages, with the page either halting or receiving Java error messages relating to the ActiveMQ sendMessage() method. The issue seemed to revolve around the fact that ActiveMQ relies on the javax.jms package and if it was finding an older version of this package (from a previous incompatible version of JMS) things go south. However, if you can force JRun to load the 1.1 javax.jms package before the native JRun version, you can get things to work.

You can do this by first creating a new directory in { application.home }/lib/ named after the version of ActiveMQ I was using, eg { application.home }/lib/activemq520, and putting the Active MQ jar in there (use the jar file found in the root of the ActiveMQ install, usually called activemq-all-5.?.?.jar. If you then create a custom jvm-config for any JRun/CF instance that you need to work with JMS 1.1 you can ensure that the right version of the javax.jms package is loaded.

Simply duplicate the existing jvm.config file found in { application.home }/bin and call it something like jvm-amq520.config and edit the line that starts 'java.class.path'. What I've found to work is to simply ensure that the activemq folder we created above is listed BEFORE the /lib folder (I think it's the jrun.jar file that we need to beat to loading).

My java.class.path now ends:

...,{application.home}/lib/activemq520,{application.home}/lib

Then when running JRun specify the new config jrun -config jvm-amq520.config -start default Or on Windows, to install the instance as a Windows service jrunsvc -install default CFDefault CFDefault CFDefault -config jvm-amq52.config you can swap out the 'default' for the name of the instance and CFDefault for the text you want to display in the Windows services manager.

keywords: coldfusion

Python/Django error more meaningful on 2.7 than 3.

posted: 18 Sep 2013

Had a strange one today trying to start one of my Django sites after manually (i.e. not using manage.py) adding a new app. I had added the views.py and the urls.py but forgot the __init__.py. Trying to start that on Django under Python 3, I got this rather obscure (at least to me) error

AttributeError: 'module' object has no attribute '__file__'

Now that had me a little confused, until I tried running the exact same application using Python 2.7 and the error came back

ImportError: No module named tricurve

tricurve, being the name of the application I'd added was a great big clue here. I realized that I had simply not indicated that I wanted Python to consider tricurve as a module, which is done by creating a __init__.py file. After doing that, it started fine under Python 2.7 and then, after switching back to Python 3, it started fine too.

HTML

keywords: python django

Bluemini on fluid/responsive grid

posted: 03 Feb 2012

Last night I launched my first attempt at a fluid/responsive grid layout for Bluemini.com. Using CSS3 media queries to apply different styles based on client width and a fallback basic fluid design for IE8 and lower that don't support CSS3, the outcome is very acceptable. I chose to only support 3+1 width configurations.

1. A max-width of 990px where anything above that width will get side margins and a background color 1b Below 990px the grid becomes fluid and columns will scale as percentages of the overall width
2. Below 768px the right hand column is moved below the main content of the blog.
3. Below 400px we get an effective single column layout with the blog at the top, followed by the about and the nav I move the nav/about sections around by having my basic HTML render with the content first, then about, then nav.

I didn't design it to be 'mobile first', but essentially, that's what it ended up doing. I render the HTML so that the natural order is how I want the content to display on the smallest screen. The basics of the layout come down to this

Wrapping the content and about with their own div means that it can be styled as its own entity and separately from the nav. Full screen: the wrapper gets floated right and the nav floated left (within wrapper content floated left, about floated right). Page has a max-width of old fixed with (990px) Middle: change the divs inside wrapper to both go full width so the content is above the about Narrow: essentially remove all float and the contents renders in natural order.

keywords: responsive

Query time bar chart in CF debug

posted: 27 Jul 2011

One of the most useful (although not necessarily beautiful) parts of ColdFusion's toolset it the debug feature. This allows you to view the timings for all the parts that contribute to the final rendering of a page. It shows you a break down of all the queries that were run; their execution time, records returned and the full query body passed to the db.

However, getting an overview of the all queries, particularly comparing each query with the others, is difficult. Enter another great feature of CF, they provide the code they use to generate the debug pages, for you to tweak :)

So here is a simple piece of code that renders an easy to see comparison of the different query times running on the page. Add it to {jrun.home}/servers/{server.name}/cfusion-ear/cfusion-war/WEB-INF/debug/classic.cfm. I stuck it in at line 567, just after the template timings and exception summaries and before the full SQL Queries output.

<!--- Query Performance Graph --->
<cfoutput>
  <cfif bFoundSQLQueries>
    <cftry>
      <p class="cfdebug">
        <hr/>
        <strong class="cfdebuglge">
          <a name="cfdebug_querygraph">SQL Query Graph</a>
        </strong>
      </p>
      <cfset maxQueryTime = 0>
      <cfloop query="cfdebug_queries">
        <cfset maxQueryTime = Max(cfdebug_queries.executionTime, maxQueryTime)>
      </cfloop>
      <table>
        <cfloop query="cfdebug_queries">
          <tr>
            <td>#cfdebug_queries.name#</td>
            <td>#Max(cfdebug_queries.executionTime, 0)#ms</td>
            <td>
              <div style="width: #Int(Max(cfdebug_queries.executionTime, 0)/maxQueryTime*100)#px; background-color: red; height: 10px"></div>
            </td>
            <td>#cfdebug_queries.template#</td>
          </tr>
        </cfloop>
      </table>
      <cfcatch>#cfcatch.message#</cfcatch>
    </cftry>
  </cfif>
</cfoutput>

You end up with something like this:

Update: http POSTs in ColdFusion and Apache Common

posted: 30 Jun 2011

I mentioned in this post that I had been having issues with cfhttp and uploading files. Well, the solution that I proposed in that post used a String object, into which the file contents were read, before assembling the response. This works well, but it does impose the overhead that the file is read into memory before being passed to the destination. This obviously starts to cause problems when we get large files being uploaded and so I started looking for another alternative.

I'm not sure what underpins the implementation of cfhttp in ColdFusion (I presumed in was Apache commons httpclient) but I thought I'd give that a go regardless, since it's already available to CF as it comes already inside the cfusion/libs directory. The 3.1 version has now been superseded, but I managed to locate the Javadocs and some example code, so thought I'd give it a try.

Here's what I came up with:

<cfset objFile = CreateObject("java", "java.io.File")>
<cfset objFile.init(filepathandname)>
<cfset uploadUrl = request.rhino.getLocal("docapiurl", true)>

<cfset commonsHttp = CreateObject("java", "org.apache.commons.httpclient.HttpClient")>

<cfset commonsHttpPost = CreateObject("java", "org.apache.commons.httpclient.methods.PostMethod")>
<cfset commonsHttpPost.init(uploadUrl)>

<cfset commonsHttpFilePart = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.FilePart")>
<cfset fileName = GetFileFromPath(filepathandname)>
<cfset commonsHttpFilePart.init("file", filename, objFile)>

<cfset commonsHttpMeta = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.StringPart")>
<cfset commonsHttpMeta.init("meta", xmlString)>

<cfset commonsHttpAction = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.StringPart")>
<cfset commonsHttpAction.init("action", "upload")>

<cfset httpParts = [commonsHttpFilePart, commonsHttpMeta, commonsHttpAction]>
<cfset commonsHttpMultipartRequest = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity")>
<cfset commonsHttpMultipartRequest.init(httpParts, commonsHttpPost.getParams())>
<cfset commonsHttpPost.setRequestEntity(commonsHttpMultipartRequest)>

<cfset commonsHttp.getHttpConnectionManager().getParams().setConnectionTimeout(5000)>
<cfset status = commonsHttp.executeMethod(commonsHttpPost)>
<cfset uploadResponse = commonsHttpPost.getResponseBodyAsString()>
 

It seems to be working well, so in case you run in to cfhttp problems, give some Java a try.

Javadocs: http://hc.apache.org/httpclient-3.x/apidocs/index.html

Samples: http://svn.apache.org/viewvc/httpcomponents/oac.hc3x/trunk/src/examples/ (MultipartFileUploadApp.java

keywords: http coldfusion

Go California - On track to ban styrofoam

posted: 14 Jun 2011

I lived in California for two years, and whilst it didn't turn out to be the 'place for me' I was always impressed by their leadership on environmental issues. I'm sure these changes weren't always for altruistic ends, but hey, it made things a little better for the ordinary man in the street. Now they are looking to ban Styrofoam state wide! Fingers crossed, and I hope the trend catches on to other states (Washington?)

CFHttp issues with POST ing file content

posted: 07 Jun 2011

As part of some expansion on our corporate intranet, I have begun building out a Documents API, similar to that from Google. The XML includes nodes that are more focused on our own particular business and internal arrangements, but pulls from the Google counterpart in many other ways. I wanted the API to be RESTful as far as possible and therefore needed to accept files directly via HTTP POST. Integrating this into an application seemed pretty straight forward, since CFHTTP allows you to specify a cfhttpparam with type="file", everything seemed sweet.

Unfortunately, POSTing files through CFHTTP has given me a number of headaches, particularly with .docx files. The resulting file, saved via a pretty conventional CFFILE action="upload" process increases by 2 bytes! Which is somewhat annoying. Here's the code I was using:

I've not done many parametric tests and haven't been able to identify if the 'Accept-Encoding' and 'TE' headers have caused the problems, but they are necessary to prevent any compression of the response which CFHttp doesn't like. So I took a delve into the HTTP spec and in the end, wrote my own HTTP body to achieve the result. I read the binary file in to a byte array 'myfile' using standard CFFile and then use this to initialize a Java String object using ISO-8859-1 character encoding. The divider can be any string that's not found anywhere else in the body, I don't scan for it, so there is a potential for conflict here, but the length of the hash makes it 'pretty' unlikely. The important thing to note here is that the Content-Disposition declaration comes immediately after the divider, that a single blank line separates the Content-Disposition declaration and the content of that POST field and that a single line follows this content before the next divider. The final thing to note is that dividers are all prefixed with '--' except for the terminal one, which is also suffixed with '--'.

There may be a nicer way to achieve the 'string' response of the actual file data, HTTP being a text based protocol, the body has to be sent as a string, but for now, this approach is working. The most important thing during testing and getting this to work was that the character encoding was consistent and I found that 'ISO-8859-1' proved the best. UTF-8 didn't work so well.

Maintaining Cache Validity over a Multi Server Clu

posted: 06 May 2011

Overview

Increasingly, data is being held in memory caches on individual servers over a cluster of multiple machines, to reduce the load and latency associated with repeated lookups in a master data store. These caches are essentially key/value stores, held in application memory. Validity of the data in each cache is of greater importance than synchronicity so whilst notification of updates must be reliably cascaded out to all other nodes, the underlying data is not necessary. We have proposed and implemented a lightweight, difference based, approach to keeping cached data valid at the expense of keeping it synchronized using JMS message services.

Validity vs Synchronicity

A cache is only as valuable as the data that it holds. If that data is out of date (stale or invalid) then this can affect the users of systems that use the cache and ultimately leads to further problems with data integrity later on.

Synchronicity, on the other hand, was less of an issue. If one cache contained data that another cache did not, then the corresponding application could fetch the data from the master data store and save it to the cache. This relies on all servers in the cluster having appropriate access to the underlying data store.

One important aspect of this deferred synchronicity is that whenever any cache is updated, all other cluster members must be notified of the action and be able to validate their own copies of the data, if such a copy exists.

Hash Exchange

As discussed, it is not important that all caches contain the same data, rather it is important that the data that any cache contains is up to date. Hence we needed a mechanism to validate data whenever there was potential for it to change. To achieve this, the cache stores alongside each data node, a hash of the stored value. It uses this hash when both informing other caches of activity connected to a node and also when receiving messages related to a node, to discern whether cache related actions are required.

The JMS Message

For current versions no data is passed over the JMS, only notifications that data has been somehow affected. When a node is inserted or modified, the key name that stores the data and the hash value are communicated in a message to all participants of the cache cluster.

1. When data is pushed into the cache an MD5 hash value for the data is calculated. If data already exists in the node, the hash of the incoming data is checked with that of the existing data. If the hashes match no further action is taken. However, if the hashes do not match, or the data doesn't exist, then the data is stored in the cache and a message is transmitted to the other members of the cluster.
2. On receipt of a message, each cache will check the referenced key and hash value. If the hashes do not match, then the data is immediately flushed. If no matching key is found or the hashes match, no action is taken. Note, data is not refreshed, rather it is dropped. The application is tasked with checking the cache and inserting new data if the cache doesn't contain valid data.
3. An application may also invalidate a cache key on demand. This can be done to control the size of the cache or for other housekeeping reasons, however, it is not necessary to message the other cluster members in this case.

Process Flow: A data modification example..

Consider a two server setup where both servers have been recently started.

• SERVER ONE - cache = empty
• SERVER TWO - cache = empty

1. A user logs in to server one and requests a record, the cache is checked and found to be empty, the data is therefore retrieved from the database and inserted into the cache and a new hash created {name:nick, hash:1234567890}. According to the rules, a new message is generated with the hash value {1234567890} and transmitted to the cluster.
2. Both servers receive the message and both check their caches. Server one's cache contains the key and the hash values match and no further action is taken. Server two's cache is empty so again, no further action is taken.
   • SERVER ONE - cache = {data={name=nick}, hash=1234567890}
   • SERVER TWO - cache = empty
3. An editor visits server two, because they are performing an edit on the same data viewed in step 1, the application checks its cache but finds no data so retrieves the data from the database.
4. On saving the edit, the application invalidates the cache and then displays the record. Checking the cache, it finds it empty and therefore retrieves the data from the database and stores it in cache. This generates a message, this time creating a hash based on the newly edited data, eg {abcdefghij}
5. Both servers again receive the message and both check their caches. Server two's cache contains the key and the hash values match, so no further action is taken. Server one's cache contains the key, but the hashes don't match so it invalidates the key.
6. The next time the data is viewed on server 1, the cache will be checked and found empty, causing a final round of insertion and messaging, after which both caches will be primed and up to date.
<

FunctionalCF making progress

posted: 02 May 2011

I've been working quite hard recently on implementing a functional syntax style of programming that's implemented in CFML (

Anyway, I reached a pretty significant point in my development of FunctionalCF so I wanted to post about it to both record it and do a little advertisement for it. I've completely re-written the parser to now build a nested object hierarchy at function definition time, so that's made subsequent running of the function much more elegant. The parser is rather sweet too, with zero look ahead, it starts with a list object and then feeds that object character by character. The list object, is itself aware of how to interpret the incoming stream and as it sees characters such as (. [ and single quotes, it instantiates child objects to handle the stream. If there is a child object, the parent simply passes the data stream on and just takes note of the returned value. If the child object encounters the end of its own data stream, it will return a value to the parent to indicate such. One minor complication of this was that the character used to determine the termination of an object was in some cases required by the parent and in others not.

The second big improvement is that the core now binds var placeholders, specified during the function definition stage, with the values passed in during execution. This is done by creating a bindMap structure in pure CFML and then passing this through all objects in the hierarchy. This data structure is also now defined in the IRunnable interface.

If you care to check out this project, take a look at