I just finished the first version of a new pure JavaScript library for AMF. I’ve wanted to do this for a while but didn’t think it was possible since JavaScript doesn’t have a ByteArray. But then I came across this: “High Performance JavaScript Port of ActionScript’s ByteArray“. That became the basis for amf.js. Before I get into the gory details of how this works, check out some developer eye candy:
http://www.jamesward.com/demos/JSAMF/censusTest.html
Ok, hopefully that worked for you. I’ve tested this in the latest Chrome, Firefox, Safari, and IE and they all seem to work. It should also work on your iPad, iPhone, or Android device.
Now for those gory details… AMF is a protocol initially created in Flash Player as a way to serialize data for storage on disk or transfer over a network. Typically in web apps we use text-based serialization protocols (like JSON or RESTful XML) for data transfer. But there are some advantages to using binary protocols – primarily much better performance. There are two versions of the AMF protocol, AMF0 and AMF3. Both are publicly documented by Adobe and numerous server-side implementations of AMF exist. AMF is just a serialization technology, not a transport. So you can put AMF encoded data into any transport (like HTTP / HTTPS). Typically Flash Player is the client that reads / writes AMF data.
I recently had a conversation with Stephan Janssen who runs Parleys.com (an amazing Flex app), which started me on this fun project. The Parleys.com PC-profile web client and the Adobe AIR desktop client both use BlazeDS and AMF as the primary serialization protocol for moving data between client and server. This is a great choice for those clients because it makes the apps snappy. But for the HTML5 client Stephan wants to reuse his AMF endpoints. This is where amf.js comes in.
Flash Player has a ByteArray API that can be used for a lot of amazing things. One of those things is to read and write AMF. If you have an object in Flash Player and you create a new ByteArray and then call “byteArray.writeObject(myObject)” you will get a ByteArray with the AMF representation of that object. Likewise if you get some AMF and you call “byteArray.readObject()” you get the object(s) from the AMF. In Flex there are high level APIs (like RemoteObject, Consumer, etc.) that use this native AMF support in Flash Player.
To create a pure JavaScript AMF library the first thing that is needed is a pure JavaScript ByteArray library since JavaScript doesn’t natively have one. I used one from adamia.com since it was similar to the ByteArray in Flash Player, seemed fast, and seemed to parse floats correctly. This ByteArray has some of the basic functions like readByte, readFloat, etc. But what about that cool readObject function? Well, that has to be built from scratch. And it should support both AMF0 and AMF3.
Using the AMF specs and code from BlazeDS & pyamf as a reference I was able to add the other functions to the ByteArray. But there was a problem. Using XMLHttpRequest as the method of getting the AMF was not working right. Some bytes were incorrect. It turns out XMLHttpRequest uses UTF-8 and that screws up some of the bytes above 128. I tried other charsets and each one would change some range of bytes. That is not good because I need the bytes to be exactly what the server sent. Then I came across this gem:
//XHR binary charset opt by Marcus Granado 2006 [http://mgran.blogspot.com] req.overrideMimeType('text/plain; charset=x-user-defined');
Using the “x-user-defined” charset left the bytes alone. Perfect! Except that IE doesn’t support the req.overrideMimeType function. But IE does actually have a real ByteArray available in req.responseBody via VBScript. For now in IE I just change the ByteArray into a string (like req.responseText in the other browsers) although a lot of optimization could be done to just use the VBScript ByteArray directly.
Right now amf.js is just a basic JavaScript library for reading AMF data. It doesn’t support using a BlazeDS MessageBrokerServlet yet because I need to be able to assemble a AMF object in JavaScript and send that in the HTTP request to the servlet. But it works fine with a custom servlet that uses BlazeDS’s AMF library to just write AMF into the HTTP response. It should also work with pyamf, AMFPHP, and other AMF server libraries.
To use amf.js start by dumping some AMF into an HTTP response. In Java with BlazeDS I did this:
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.setHeader("Content-Type", "application/x-amf;charset=x-user-defined"); ServletOutputStream out = response.getOutputStream(); ActionMessage requestMessage = new ActionMessage(MessageIOConstants.AMF3); MessageBody amfMessage = new MessageBody(); amfMessage.setData(someSerializableObject); requestMessage.addBody(amfMessage); AmfMessageSerializer amfMessageSerializer = new AmfMessageSerializer(); amfMessageSerializer.initialize(SerializationContext.getSerializationContext(), out, new AmfTrace()); amfMessageSerializer.writeMessage(requestMessage); out.close(); }
In a HTML web page add the amf.js script:
<script type="text/javascript" src="amf.js"></script>In JavaScript make a XHR request for some data:
var url = "TestServlet"; var req; function getAMF() { if (window.ActiveXObject) { req = new ActiveXObject("Microsoft.XMLHTTP"); } else if (window.XMLHttpRequest) { req = new XMLHttpRequest(); //XHR binary charset opt by Marcus Granado 2006 [http://mgran.blogspot.com] req.overrideMimeType('text/plain; charset=x-user-defined'); } req.onreadystatechange = processReqChange; req.open("GET", url, true); req.send(null); }
And when the response comes back decode the AMF:
function processReqChange() { if (req.readyState == 4) { if (req.status == 200) { var o = decodeAMF(req.responseText).messages[0].body; } else { alert("There was a problem retrieving the data:\n" + req.statusText); } } }
For details on how to support IE, check out the source code for censusTest.html.
While amf.js works in my tests there is more work to be done. I need to add the write functions so that AMF can be sent to the server. Then supporting BlazeDS’s MessageBrokerServlet should be pretty straightforward. I’d also like to create pure JavaScript implementations of Flex’s RemoteObject, Consumer, and Producer APIs. Also, I need people to test amf.js with their AMF to make sure that I’ve implemented things correctly. All of the code is on github.com so go ahead and fork it! Let me know what you think.
38 Comments
Very very cool. I soooo want to see this brought to completion. :)
Could this utlimately lead to another means for FlashJS communication, and could JS now understand SharedObject files?
SharedObjects are stored in a special place on the users filesystem in the AMF format. Since JavaScript doesn’t have access to read these files from the filesystem (unless you were using Adobe AIR), I don’t think it would be possible to communicate between Flash and JavaScript through a SharedObject.
If you want to send typed objects to javascript you could encode your Flash AMF ByteArray to a string (base64 or whatever) and send it over the ExternalInterface, then decode with Javascript AMF decoder . (You’ll maybe hit some argument size limit if you send big structures, but you could send the string in chunks)
Not sure if this makes sense, but it would re-use the mapping of data-types between each language, and commonality is nice.
Having this in BlazeDS supporting server push with long http-polling would be a top notch feature !
How would this compare with APE?
What is APE?
APE Push engine – http://www.ape-project.org (and http://github.com/APE-Project)
Your AMF deserializer doesn’t support cyclic data structures :) – it will just crash with an index-out-of-bounds exception. In all seriousness though, I would strongly advise that you take a look at RocketAMF (http://github.com/warhammerkid/rocket-amf), which is very similar in terms of code structure to amf.js and it is fully tested against Flash Player’s implementation, so it’s very unlikely there are any major bugs remaining. In addition, if you have any questions about implementation, please feel free to ask me, as I’m the developer of RocketAMF and know quite a bit about AMF serialization and deserialization.
Awesome! Thanks Stephen! I’ll take a look. Can you explain more about how a cyclic data structure is encoded in AMF?
The problem is this code here:
var o = new Object();
var ref = this.readUInt29();
if ((ref & 1) == 0)
return this.objectTable[(ref >> 1)];
var ti = this.readTraits(ref);
var className = ti.className;
var externalizable = ti.externalizable;
if (externalizable)
{
o = this.readExternalizable(className);
}
else
{
var len = ti.properties.length;
for (var i = 0; i < len; i++)
{
var propName = ti.properties[i];
var value = this.readObject();
o[propName] = value;
}
if (ti.dynamic)
{
for (; ;)
{
var name = this.readString();
if (name == null || name.length == 0) break;
var value = this.readObject();
o[propName] = value;
}
}
}
this.objectTable.push(o); // PROBLEM HERE - Should be up right below "var o = new Object();"
return o;
If you add it to the object table after deserializing the object, then if that object is referenced anywhere in the object graph, it won’t be in the objectTable yet. Consequently you’ll either get an index-out-of-bounds or a totally different object. The solution is to add it to the object table before starting to deserialize the contents. This is true for arrays as well, as they can reference themselves as one of the elements.
Awesome! Makes sense. Thanks!
This is a link to the APE web site.
This is AJAX Push Engine. In some ways (not all) it is an alternative to BlazeDS. It stands for AJAX Push Engine.
Very cool stuff. I wrote a JavaScript AMF deserializer as part of my AMF Explorer Firebug extension (http://amfexplorer.riaforge.org), but it used the binary input stream available in Mozilla (https://developer.mozilla.org/en/nsIBinaryInputStream). I was planning on writing the serializer as well, just haven’t found the time yet. Anyway, it is very interesting to see a pure JavaScript implementation.
Sweet! Wish I would have known about that a few days ago. :)
Very neat technically, but I’m finding it hard to imagine a good use for this. I guess it depends on the performance & size of AMF decoded by pure JS, versus (gzipped) JSON decoded natively by the browser. Currently I use AMFPHP which very easily lets you access the same exact backend PHP method via an AMF gateway (for Flash, or anything with an AMF library) or via a JSON gateway (for JS, or anything else that speaks JSON, which is pretty much everything these days). I use AMF for Flash, and JSON for ajax calls, and it works pretty well.
when u say iPad/iPhone, is there a way to work with these native objects in a stand alone objective c app?
This is pretty awesome btw.
There might already be an Objective C AMF implementation. If not then someone could pretty easily write one.
Yes, there’s an Objective-C AMF implementation: http://github.com/nesium/cocoa-amf
Ha! Apparenlty everyone is writing their own AMF parser these days. I have an AMF0 and AMF3 writer/reader (for my SharedObject AIR app .minerva) and ported that to an AMF0/ AMF3 reader for my Firefox extension Flashbug. Like Nathan it uses the nsIbinaryInputStream since I didn’t see of any other way of grabbing the binary data. Really cool to see a pure JS version. If you need any help with a writer lemme know.
Gabriel, does WebKit give you a binary object that you were able to use to build the AMF serializer/deserializer for AIR?
Couldn’t you just use ByteArray through the Flash-JS bridge in Air? Or is that not supported?
My AIR app is built in AS3, Flashbug is my only foray into AMF parsing in Javascript. But to answer your question I think the only thing in the specs that allow binary data will be the FileReader and FileWriter APIs being developed for HTML5. But it’s so far down the line they only have FileReader in FireFox. But that’s why I found this post interesting. They are doing binary parsing with just strings.
As for the Flash-JS bridge, I suppose that would work as well. You do have access to the ByteArray class in HTML air apps and most of the Flash API.
If your AIR app is AS3, is there a reason you chose not to use ByteArray to read/write AMF in your AIR app? (Just wondering if there are some caveats that other developers of AMF libraries should be aware of.)
Given that WebSockets support text or binary frames I am hoping that HTML5 will have some way to easily work with binary data. I’ll have to look into the FileReader/Writer APIs to see if they fit the bill.
really cool library and very useful, but James could you update your wiki on GitHub with information above and does anyone tried this with AMFPHP? I presume I have to create a custom receiver for the AMFPHP gateway.php or can I use the same setup?
:$
Awesome. Can’t wait to try this. Well done.
Hi guys, there is a project called merapi which lacks of a javascript based amf ser/deser. Actually its acts a bridge between 2 languages eg. Flex to Java, C# to java in a very lightweight way. In my current project i need such a ser/deser for the javascript side. I think its a good practice to integragte your code with merapi.
regards gurkerl
How i can use encoding JS object to AMF from NodeJS?
I’m sure it’s possible but I don’t know much about NodeJS. I’ll look into this when I add the writeObject implementation.
James,
Really impressive and exciting!
How about open sourcing on code.google.com so others can contribute…
Really looking forward to using with a BlazeDS MessageBrokerServlet and
getting the cyclic data structure fixed we can use this in our projects.
Would love to hook this into GWT and/or ext-js.
Nice job!!!!
Thanks Mike. It’s open source on github.com so anyone can contribute!
Hi James,
Would it be possible to store and retrieve large ByteArray’s from Flex into the JS engine through amf.js and the Flex Ajax Bridge?
Cheers,
= Dirk
I’m not sure what the limits of ExternalInterface are but theoretically, yes.
I Googled for exactly this not long ago and came up with nothing.
I was in fact working in NodeJS – So, I went ahead and wrote node-amf
http://github.com/timwhitlock/node-amf
Hi,
What if compare this with The Ajax client library?
http://livedocs.adobe.com/blazeds/1/blazeds_devguide/help.html?content=ajaxds_3.html
That actually uses a hidden SWF so it’s not independent from Flash. amf.js will work in environments where Flash isn’t.
Good news for iPhone owners! I actually found this page with google it it looks quite interesting.
I’ll definitely give it a try.
Does it allow cross/sub domain requests?
It could but cross-domain requests in JavaScript aren’t supported by very many browsers yet.