Bluemini.comBluemini.com

Update: http POSTs in ColdFusion and Apache Common

posted: 30 Jun 2011

I mentioned in this post that I had been having issues with cfhttp and uploading files. Well, the solution that I proposed in that post used a String object, into which the file contents were read, before assembling the response. This works well, but it does impose the overhead that the file is read into memory before being passed to the destination. This obviously starts to cause problems when we get large files being uploaded and so I started looking for another alternative.

I'm not sure what underpins the implementation of cfhttp in ColdFusion (I presumed in was Apache commons httpclient) but I thought I'd give that a go regardless, since it's already available to CF as it comes already inside the cfusion/libs directory. The 3.1 version has now been superseded, but I managed to locate the Javadocs and some example code, so thought I'd give it a try.

Here's what I came up with:

<cfset objFile = CreateObject("java", "java.io.File")>
<cfset objFile.init(filepathandname)>
<cfset uploadUrl = request.rhino.getLocal("docapiurl", true)>

<cfset commonsHttp = CreateObject("java", "org.apache.commons.httpclient.HttpClient")>

<cfset commonsHttpPost = CreateObject("java", "org.apache.commons.httpclient.methods.PostMethod")>
<cfset commonsHttpPost.init(uploadUrl)>

<cfset commonsHttpFilePart = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.FilePart")>
<cfset fileName = GetFileFromPath(filepathandname)>
<cfset commonsHttpFilePart.init("file", filename, objFile)>

<cfset commonsHttpMeta = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.StringPart")>
<cfset commonsHttpMeta.init("meta", xmlString)>

<cfset commonsHttpAction = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.StringPart")>
<cfset commonsHttpAction.init("action", "upload")>

<cfset httpParts = [commonsHttpFilePart, commonsHttpMeta, commonsHttpAction]>
<cfset commonsHttpMultipartRequest = CreateObject("java", "org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity")>
<cfset commonsHttpMultipartRequest.init(httpParts, commonsHttpPost.getParams())>
<cfset commonsHttpPost.setRequestEntity(commonsHttpMultipartRequest)>

<cfset commonsHttp.getHttpConnectionManager().getParams().setConnectionTimeout(5000)>
<cfset status = commonsHttp.executeMethod(commonsHttpPost)>
<cfset uploadResponse = commonsHttpPost.getResponseBodyAsString()>
 

It seems to be working well, so in case you run in to cfhttp problems, give some Java a try.

Javadocs: http://hc.apache.org/httpclient-3.x/apidocs/index.html

Samples: http://svn.apache.org/viewvc/httpcomponents/oac.hc3x/trunk/src/examples/ (MultipartFileUploadApp.java

keywords:

Go California - On track to ban styrofoam

posted: 14 Jun 2011

I lived in California for two years, and whilst it didn't turn out to be the 'place for me' I was always impressed by their leadership on environmental issues. I'm sure these changes weren't always for altruistic ends, but hey, it made things a little better for the ordinary man in the street. Now they are looking to ban Styrofoam state wide! Fingers crossed, and I hope the trend catches on to other states (Washington?)

CFHttp issues with POST ing file content

posted: 07 Jun 2011

As part of some expansion on our corporate intranet, I have begun building out a Documents API, similar to that from Google. The XML includes nodes that are more focused on our own particular business and internal arrangements, but pulls from the Google counterpart in many other ways. I wanted the API to be RESTful as far as possible and therefore needed to accept files directly via HTTP POST. Integrating this into an application seemed pretty straight forward, since CFHTTP allows you to specify a cfhttpparam with type="file", everything seemed sweet.

Unfortunately, POSTing files through CFHTTP has given me a number of headaches, particularly with .docx files. The resulting file, saved via a pretty conventional CFFILE action="upload" process increases by 2 bytes! Which is somewhat annoying. Here's the code I was using:

<cfhttp method="POST" url="#request.rhino.getLocal("docapiurl", true)#" result="uploadResponse">
   <!--- to protect from web servers that compress the response --->
   <cfhttpparam type="Header" name="Accept-Encoding" value="deflate;q=0">
   <cfhttpparam type="Header" name="TE" value="deflate;q=0">
   <cfhttpparam type="formfield" name="meta" value="#xmlString#">
   <cfhttpparam type="formfield" name="action" value="upload">
   <cfhttpparam type="file" name="file" file="#filepathandname#">
</cfhttp>

I've not done many parametric tests and haven't been able to identify if the 'Accept-Encoding' and 'TE' headers have caused the problems, but they are necessary to prevent any compression of the response which CFHttp doesn't like. So I took a delve into the HTTP spec and in the end, wrote my own HTTP body to achieve the result. I read the binary file in to a byte array 'myfile' using standard CFFile and then use this to initialize a Java String object using ISO-8859-1 character encoding. The divider can be any string that's not found anywhere else in the body, I don't scan for it, so there is a potential for conflict here, but the length of the hash makes it 'pretty' unlikely. The important thing to note here is that the Content-Disposition declaration comes immediately after the divider, that a single blank line separates the Content-Disposition declaration and the content of that POST field and that a single line follows this content before the next divider. The final thing to note is that dividers are all prefixed with '--' except for the terminal one, which is also suffixed with '--'.

<cffile action="readbinary" file="#filepathandname#" variable="myfile">
<cfset by="CreateObject("java","java.lang.String">
<cfset by.init(myfile, "iso-8859-1")>

<cfset contentdivider=Hash("this is my divider for this")>

<cfhttp method="POST" url="#uploadURL#" result="uploadResponse" charset="iso-8859-1">
    <cfhttpparam type="header" name="Content-Type" value="multipart/form-data; boundary=#contentDivider#">
    <cfhttpparam type="Header" name="Accept-Encoding" value="deflate;q=0">
    <cfhttpparam type="Header" name="TE" value="deflate;q=0">
    <cfhttpparam type="body" value="--#contentDivider#
Content-Disposition: form-data; name=""file""; filename=""#fileName#""
Content-Type: #mimeType#

#by.toString()#

--#contentDivider#
Content-Disposition: form-data; name=""meta""

#xmlString#

--#contentDivider#
Content-Disposition: form-data; name=""action""

upload
--#contentDivider#--">
</cfhttp>

There may be a nicer way to achieve the 'string' response of the actual file data, HTTP being a text based protocol, the body has to be sent as a string, but for now, this approach is working. The most important thing during testing and getting this to work was that the character encoding was consistent and I found that 'ISO-8859-1' proved the best. UTF-8 didn't work so well.