amf.js – A Pure JavaScript AMF Implementation

I just finished the first version of a new pure JavaScript library for AMF. I’ve wanted to do this for a while but didn’t think it was possible since JavaScript doesn’t have a ByteArray. But then I came across this: “High Performance JavaScript Port of ActionScript’s ByteArray“. That became the basis for amf.js. Before I get into the gory details of how this works, check out some developer eye candy:
http://www.jamesward.com/demos/JSAMF/censusTest.html

Ok, hopefully that worked for you. I’ve tested this in the latest Chrome, Firefox, Safari, and IE and they all seem to work. It should also work on your iPad, iPhone, or Android device.

Now for those gory details… AMF is a protocol initially created in Flash Player as a way to serialize data for storage on disk or transfer over a network. Typically in web apps we use text-based serialization protocols (like JSON or RESTful XML) for data transfer. But there are some advantages to using binary protocols – primarily much better performance. There are two versions of the AMF protocol, AMF0 and AMF3. Both are publicly documented by Adobe and numerous server-side implementations of AMF exist. AMF is just a serialization technology, not a transport. So you can put AMF encoded data into any transport (like HTTP / HTTPS). Typically Flash Player is the client that reads / writes AMF data.

I recently had a conversation with Stephan Janssen who runs Parleys.com (an amazing Flex app), which started me on this fun project. The Parleys.com PC-profile web client and the Adobe AIR desktop client both use BlazeDS and AMF as the primary serialization protocol for moving data between client and server. This is a great choice for those clients because it makes the apps snappy. But for the HTML5 client Stephan wants to reuse his AMF endpoints. This is where amf.js comes in.

Flash Player has a ByteArray API that can be used for a lot of amazing things. One of those things is to read and write AMF. If you have an object in Flash Player and you create a new ByteArray and then call “byteArray.writeObject(myObject)” you will get a ByteArray with the AMF representation of that object. Likewise if you get some AMF and you call “byteArray.readObject()” you get the object(s) from the AMF. In Flex there are high level APIs (like RemoteObject, Consumer, etc.) that use this native AMF support in Flash Player.

To create a pure JavaScript AMF library the first thing that is needed is a pure JavaScript ByteArray library since JavaScript doesn’t natively have one. I used one from adamia.com since it was similar to the ByteArray in Flash Player, seemed fast, and seemed to parse floats correctly. This ByteArray has some of the basic functions like readByte, readFloat, etc. But what about that cool readObject function? Well, that has to be built from scratch. And it should support both AMF0 and AMF3.

Using the AMF specs and code from BlazeDS & pyamf as a reference I was able to add the other functions to the ByteArray. But there was a problem. Using XMLHttpRequest as the method of getting the AMF was not working right. Some bytes were incorrect. It turns out XMLHttpRequest uses UTF-8 and that screws up some of the bytes above 128. I tried other charsets and each one would change some range of bytes. That is not good because I need the bytes to be exactly what the server sent. Then I came across this gem:

//XHR binary charset opt by Marcus Granado 2006 [http://mgran.blogspot.com]
req.overrideMimeType('text/plain; charset=x-user-defined');

Using the “x-user-defined” charset left the bytes alone. Perfect! Except that IE doesn’t support the req.overrideMimeType function. But IE does actually have a real ByteArray available in req.responseBody via VBScript. For now in IE I just change the ByteArray into a string (like req.responseText in the other browsers) although a lot of optimization could be done to just use the VBScript ByteArray directly.

Right now amf.js is just a basic JavaScript library for reading AMF data. It doesn’t support using a BlazeDS MessageBrokerServlet yet because I need to be able to assemble a AMF object in JavaScript and send that in the HTTP request to the servlet. But it works fine with a custom servlet that uses BlazeDS’s AMF library to just write AMF into the HTTP response. It should also work with pyamf, AMFPHP, and other AMF server libraries.

To use amf.js start by dumping some AMF into an HTTP response. In Java with BlazeDS I did this:

public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
{
    response.setHeader("Content-Type", "application/x-amf;charset=x-user-defined");
    ServletOutputStream out = response.getOutputStream();
    ActionMessage requestMessage = new ActionMessage(MessageIOConstants.AMF3);
    MessageBody amfMessage = new MessageBody();
    amfMessage.setData(someSerializableObject);
    requestMessage.addBody(amfMessage);
    AmfMessageSerializer amfMessageSerializer = new AmfMessageSerializer();
    amfMessageSerializer.initialize(SerializationContext.getSerializationContext(), out, new AmfTrace());
    amfMessageSerializer.writeMessage(requestMessage);
    out.close();
}

In a HTML web page add the amf.js script:

<script type="text/javascript" src="amf.js"></script>

In JavaScript make a XHR request for some data:

var url = "TestServlet";
var req;
 
function getAMF()
{
    if (window.ActiveXObject)
    {
        req = new ActiveXObject("Microsoft.XMLHTTP");
    }
    else if (window.XMLHttpRequest)
    {
        req = new XMLHttpRequest();
        //XHR binary charset opt by Marcus Granado 2006 [http://mgran.blogspot.com]
        req.overrideMimeType('text/plain; charset=x-user-defined');
    }
    req.onreadystatechange = processReqChange;
    req.open("GET", url, true);
    req.send(null);
}

And when the response comes back decode the AMF:

function processReqChange()
{
    if (req.readyState == 4)
    {
        if (req.status == 200)
        {
            var o = decodeAMF(req.responseText).messages[0].body;
        }
        else
        {
            alert("There was a problem retrieving the data:\n" + req.statusText);
        }
    }
}

For details on how to support IE, check out the source code for censusTest.html.

While amf.js works in my tests there is more work to be done. I need to add the write functions so that AMF can be sent to the server. Then supporting BlazeDS’s MessageBrokerServlet should be pretty straightforward. I’d also like to create pure JavaScript implementations of Flex’s RemoteObject, Consumer, and Producer APIs. Also, I need people to test amf.js with their AMF to make sure that I’ve implemented things correctly. All of the code is on github.com so go ahead and fork it! Let me know what you think.

  • John Farrar

    Very very cool. I soooo want to see this brought to completion. :)

  • Could this utlimately lead to another means for FlashJS communication, and could JS now understand SharedObject files?

    • SharedObjects are stored in a special place on the users filesystem in the AMF format. Since JavaScript doesn’t have access to read these files from the filesystem (unless you were using Adobe AIR), I don’t think it would be possible to communicate between Flash and JavaScript through a SharedObject.

      • Bart

        If you want to send typed objects to javascript you could encode your Flash AMF ByteArray to a string (base64 or whatever) and send it over the ExternalInterface, then decode with Javascript AMF decoder . (You’ll maybe hit some argument size limit if you send big structures, but you could send the string in chunks)

        Not sure if this makes sense, but it would re-use the mapping of data-types between each language, and commonality is nice.

  • Having this in BlazeDS supporting server push with long http-polling would be a top notch feature !

  • John Farrar

    How would this compare with APE?

  • Your AMF deserializer doesn’t support cyclic data structures :) – it will just crash with an index-out-of-bounds exception. In all seriousness though, I would strongly advise that you take a look at RocketAMF (http://github.com/warhammerkid/rocket-amf), which is very similar in terms of code structure to amf.js and it is fully tested against Flash Player’s implementation, so it’s very unlikely there are any major bugs remaining. In addition, if you have any questions about implementation, please feel free to ask me, as I’m the developer of RocketAMF and know quite a bit about AMF serialization and deserialization.

    • Awesome! Thanks Stephen! I’ll take a look. Can you explain more about how a cyclic data structure is encoded in AMF?

    • The problem is this code here:

      var o = new Object();

      var ref = this.readUInt29();

      if ((ref & 1) == 0)
      return this.objectTable[(ref >> 1)];

      var ti = this.readTraits(ref);
      var className = ti.className;
      var externalizable = ti.externalizable;

      if (externalizable)
      {
      o = this.readExternalizable(className);
      }
      else
      {
      var len = ti.properties.length;

      for (var i = 0; i < len; i++)
      {
      var propName = ti.properties[i];

      var value = this.readObject();

      o[propName] = value;
      }

      if (ti.dynamic)
      {
      for (; ;)
      {
      var name = this.readString();
      if (name == null || name.length == 0) break;

      var value = this.readObject();
      o[propName] = value;
      }
      }
      }

      this.objectTable.push(o); // PROBLEM HERE - Should be up right below "var o = new Object();"

      return o;

      If you add it to the object table after deserializing the object, then if that object is referenced anywhere in the object graph, it won’t be in the objectTable yet. Consequently you’ll either get an index-out-of-bounds or a totally different object. The solution is to add it to the object table before starting to deserialize the contents. This is true for arrays as well, as they can reference themselves as one of the elements.

  • John Farrar

    This is a link to the APE web site.
    This is AJAX Push Engine. In some ways (not all) it is an alternative to BlazeDS. It stands for AJAX Push Engine.

  • Very cool stuff. I wrote a JavaScript AMF deserializer as part of my AMF Explorer Firebug extension (http://amfexplorer.riaforge.org), but it used the binary input stream available in Mozilla (https://developer.mozilla.org/en/nsIBinaryInputStream). I was planning on writing the serializer as well, just haven’t found the time yet. Anyway, it is very interesting to see a pure JavaScript implementation.

    • Sweet! Wish I would have known about that a few days ago. :)

  • Very neat technically, but I’m finding it hard to imagine a good use for this. I guess it depends on the performance & size of AMF decoded by pure JS, versus (gzipped) JSON decoded natively by the browser. Currently I use AMFPHP which very easily lets you access the same exact backend PHP method via an AMF gateway (for Flash, or anything with an AMF library) or via a JSON gateway (for JS, or anything else that speaks JSON, which is pretty much everything these days). I use AMF for Flash, and JSON for ajax calls, and it works pretty well.

  • when u say iPad/iPhone, is there a way to work with these native objects in a stand alone objective c app?

    This is pretty awesome btw.

  • Ha! Apparenlty everyone is writing their own AMF parser these days. I have an AMF0 and AMF3 writer/reader (for my SharedObject AIR app .minerva) and ported that to an AMF0/ AMF3 reader for my Firefox extension Flashbug. Like Nathan it uses the nsIbinaryInputStream since I didn’t see of any other way of grabbing the binary data. Really cool to see a pure JS version. If you need any help with a writer lemme know.

    • Gabriel, does WebKit give you a binary object that you were able to use to build the AMF serializer/deserializer for AIR?

      • Couldn’t you just use ByteArray through the Flash-JS bridge in Air? Or is that not supported?

        • My AIR app is built in AS3, Flashbug is my only foray into AMF parsing in Javascript. But to answer your question I think the only thing in the specs that allow binary data will be the FileReader and FileWriter APIs being developed for HTML5. But it’s so far down the line they only have FileReader in FireFox. But that’s why I found this post interesting. They are doing binary parsing with just strings.

          As for the Flash-JS bridge, I suppose that would work as well. You do have access to the ByteArray class in HTML air apps and most of the Flash API.

          • If your AIR app is AS3, is there a reason you chose not to use ByteArray to read/write AMF in your AIR app? (Just wondering if there are some caveats that other developers of AMF libraries should be aware of.)

            Given that WebSockets support text or binary frames I am hoping that HTML5 will have some way to easily work with binary data. I’ll have to look into the FileReader/Writer APIs to see if they fit the bill.

  • really cool library and very useful, but James could you update your wiki on GitHub with information above and does anyone tried this with AMFPHP? I presume I have to create a custom receiver for the AMFPHP gateway.php or can I use the same setup?

    :$

  • Jeffrey Gilbert

    Awesome. Can’t wait to try this. Well done.

  • Hi guys, there is a project called merapi which lacks of a javascript based amf ser/deser. Actually its acts a bridge between 2 languages eg. Flex to Java, C# to java in a very lightweight way. In my current project i need such a ser/deser for the javascript side. I think its a good practice to integragte your code with merapi.

    regards gurkerl

  • How i can use encoding JS object to AMF from NodeJS?

    • I’m sure it’s possible but I don’t know much about NodeJS. I’ll look into this when I add the writeObject implementation.

  • James,
    Really impressive and exciting!

    How about open sourcing on code.google.com so others can contribute…
    Really looking forward to using with a BlazeDS MessageBrokerServlet and
    getting the cyclic data structure fixed we can use this in our projects.
    Would love to hook this into GWT and/or ext-js.

    Nice job!!!!

    • Thanks Mike. It’s open source on github.com so anyone can contribute!

  • Hi James,

    Would it be possible to store and retrieve large ByteArray’s from Flex into the JS engine through amf.js and the Flex Ajax Bridge?

    Cheers,

    = Dirk

    • I’m not sure what the limits of ExternalInterface are but theoretically, yes.

  • I Googled for exactly this not long ago and came up with nothing.
    I was in fact working in NodeJS – So, I went ahead and wrote node-amf
    http://github.com/timwhitlock/node-amf

  • Serge

    Hi,
    What if compare this with The Ajax client library?
    http://livedocs.adobe.com/blazeds/1/blazeds_devguide/help.html?content=ajaxds_3.html

    • That actually uses a hidden SWF so it’s not independent from Flash. amf.js will work in environments where Flash isn’t.

      • Serge

        Good news for iPhone owners! I actually found this page with google it it looks quite interesting.
        I’ll definitely give it a try.

        Does it allow cross/sub domain requests?

        • It could but cross-domain requests in JavaScript aren’t supported by very many browsers yet.

  • mrb

    Intresting indeed.

    Do you think we will be able to do amf- remoting from a pure/descent javascript web browser without Flash?

    Not that I don’t like flash but if you have a lcds, weborb or amfphp already deployed and you need to consume their services without flash this would very helpful.

    cheers

    • That is the goal! It’s certainly possible. Just needs a bit more work.

  • Mrb

    James, that’s great,

    I think we could give you a hand with that but will certainly need your help.

    Do you think you can define 3-4 things that should be done to improve it and your approach to it and we could code it, test it and use-it!

    cheers

    • Ideally we would have JavaScript versions of RemoteObject, Consumer, and Producer. To do that we’d need some of the underlying pieces in place. All of the code for these objects (in AS3) is already available so it should be too hard to port them. One piece which I didn’t address yet is the AMF encoder. That will be an essential piece since BlazeDS / LCDS use AMF encoded messages in order to determine what to respond with. So that would be a great start! Also the JS code needs to be refactored into a more object-friendly way.

  • In Flashbug I have a JavaScript AMF, SOL and SWF reader. In my app .minerva I have an AMF writer (in AS3) but it would be easy enough to port to JavaScript. If anyone would be interested I could post it somewhere.

    http://blog.coursevector.com/flashbug
    http://blog.coursevector.com/minerva

  • Mrb

    Thanks Gabriel that will be very helpful! let me know where it ends

  • Has anyone used a reliable, full-featured, cross-browser, binary output stream in JavaScript? I’ve been looking at this off and on for several months now and I’ve yet to find one. Some browsers have native implementations, for example Mozilla has nsIBinaryOutputStream, IE has a way to do this via VBScript, but in general JavaScript doesn’t play well with binary data.

    If the goal is to use remoting via XHR a better approach may be to add JSON endpoints to BlazeDS. There was a bug for this a couple of years ago but it doesn’t look like there was any interest (https://bugs.adobe.com/jira/browse/BLZ-72). Thoughts?

    • Adding a JSON channel to BlazeDS would be cool. It’s certainly possible since BlazeDS can already do a AMF-X (AMF over XML).

  • mrb

    Hi Gabriel!

    I checked http://code.google.com/p/cv-minerva/source/browse/#svn/ but there is no code there!.. would you kindly post the amf writer to check it out or maybe send it to mrbarletta -> gmail-com?

    Thanks!

  • Sorry about that. I currently use the project specific pages just to keep track of issues. I’ve uploaded all my reader and writers of various formats to my lib project:

    http://code.google.com/p/cvlib/

    There you can see JS versions for AMF0, AMF3, SWF (about 80%), CFF (just to read the font name), Zip (zlib deflate), SOL (shared objects), WAV (to create a valid WAV file from SWF sound), and AMF (the AMF0/3 wrapper).

    For the writers you’ll have to look into the AS3 folder. There I also have more font types from my merlin app which it’s important for this discussion but just an FYI.

    • I meant isn’t, not it’s

  • The issue with trying to port Gabriel’s library to JavaScript is there is no equivalent of AS3’s ByteArray in JavaScript.

  • Did you look at my library? =)

  • Mrb

    What about what james said about…. But then I came across this: “High Performance JavaScript Port of ActionScript’s ByteArray“. ?

  • http://code.google.com/p/cvlib/source/browse/trunk/js/ByteArrayString.js

    You pass it a string, then use it like ByteArray as normal.

  • Over at amfphp we’re looking at making it as easy as possible to call the same services with amf or json. So I’m interested to see that amf might be used instead of json for a flashless app. I don’t have much experience building web apps without Flash, so when do you think which approach would make more sense?

    • If people want to build a browser app that doesn’t require Flash (for iPad / iPhone) then using an AMF JavaScript library would be a good way to reuse existing AMF endpoints.

  • Hey James, I haven’t used it yet, but thanks for making this lib. I plan on using it extensively over the next year to expose my existing services to mobile devices. I’ve been watching the project for a few months, and I think it’s awesome you’ve built this!

  • Patrick Whittingham

    Instead of Java how can I get this to work with CFMX +7.x. Since Remoting is totally integrated how would I do it when it is sent by AJAX.

    • Good question. I’m really not sure. My goal is to eventually create a full JS client library for Remoting (with BlazeDS, LCDS, and CF). But I haven’t had time to work on this lately. Fortunately there is progress on that front with cvlib:
      http://code.google.com/p/cvlib/

  • I.S.G

    This is great, but how to encodeAMF? so that a AMF client can be created with amf.js?

    • Check out CVLib:
      http://code.google.com/p/cvlib/

      It has the encoding libraries.

      • I.S.G

        Thanks. That is great too. But I can not find documents about it (The “importScript” in AMFWorker.js hints that they are still in the original context of Firefox.). It would be appreciated if someone can give the link to the document, or tutorial about how to create an AMF client in pure javascript with amf.js and CVLib.

  • Pingback: Toaster Lite – HTTP/AMF monitoring tool at Space of Flex/AIR technologies()

  • sonal

    Hi,

    i am working on a flex j2ee project. we have a lot of data being sent from java to flex. we are using the granaiteds, also applying AMFMessageFilter to serialise and deserialise. we want to futher reduce the size of the amf message. is it possible to reduce the size of amf message futher? Thanks in advance

    • You can use gzip on top of AMF to further compact the transfer of the data.

  • Tim

    Anyone know what’s up with github? I keep getting a 404 using : https://github.com/jlward4th/JSAMF

  • Pingback: Data Paging in Flex 4()

  • Marcus Baffa

    Hi James,

    I have a LOB application developed in Flex and I need to make it work in HTML. My services were written in C# with FluorineFX.
    This is a full application so I need to send and receive AMF packets using Value Objects. Can I use your amf.js to do the job. Do you think that Javascript can to the same job that did ActionsScript ???
    Thanks

    • That is possible but I haven’t finished JSAMF to be a full drop-in replacement for Flex’s RemoteObject (yet). However I think there is a project on Google Code somewhere that has gotten further with this.

  • I have my reader/writer up here http://code.google.com/p/cvlib/source/browse/#svn%2Ftrunk%2Fjs it also uses webworkers to read through the files.

    • Fantastic stuff! Thanks Gabriel!

    • rooskie_in_nz

      Hey Gabriel, thanks for your library mate! Is there an example on how to use it? Cheers.

  • Pingback: Remoting Technologies for javascript | SwhistleSoft Blog()

  • Cooool!

  • Haven’t tried it but assuming it works, impressive! I’ve often shied away from using AMF in a Flash/Flex project. I know it’s often superior but I often think, what if my Web service needs to be used by non-Flash clients, or the Flash client gets recreated in HTML5 one day, then we’ll be stuck with AMF. Guess not!

  • fox

    Could this be somehow used to play audio trough amf to create a ‘secure’ audio playback solution ?
    Currently it is way to easy for anyone to download the source files :/

  • To view the contents amf file in human-readable form:
    http://amfview.org

  • Woog Lae Kim

    Thank you for your code. 
    But your code has unicode problem.
    I modify some part of your code.

    readString: function (len) {
    var str = new Array();
    var code,code2,code3,code4,j = 0;

    for (var i=0; i<len; i++) {
    var cc = (this.data[this.pos++] & 0xFF);
    if( cc < 128 ) {
    str.push(String.fromCharCode(cc));
    } else if( cc < 224 ) {
    var cc1 = (this.data[this.pos++] & 0xFF);
    i++;
    str.push(String.fromCharCode(((cc-192)<<6) + (cc1-128)));
    } else if( cc < 240 ) {
    var cc1 = (this.data[this.pos++] & 0xFF);
    var cc2 = (this.data[this.pos++] & 0xFF);
    i=i+2;
    str.push(String.fromCharCode(((cc-224)<<12) + ((cc1-128)<<6) + (cc2-128)));
    } else {
    var cc1 = (this.data[this.pos++] & 0xFF);
    var cc2 = (this.data[this.pos++] & 0xFF);
    var cc3 = (this.data[this.pos++] & 0xFF);
    i=i+3;
    str.push(String.fromCharCode(((cc-240)<<18) + ((cc1-128)<<12) + ((cc2-128)<<6) + (cc3-128)));
    }
    }
    return str.join('');
    },

  • Mike

    great work!.. a jsProducer + jsConsumer is definitely something to watch for

  • شات عراقنا
  • Narendra

    Hi James… Great work. I have same kind of requirement now…
    Can you please let me know if you have made any update to amf.js to send AMF to server as well.

    Thanks in advance.

    naren.aries@gmail.com

  • Jai

    With the new Adobe Data Services 4.6 Ajax Client Library, do we still need this library. Can we connect to BlazeDS using Ajax Client Library without having any Flash/Flex on client?

  • Madan Singh

    Hello James,

    I am experiencing a problem while load testing flex based apps with Jmeter tool.Hope you will b e a bit ware of this tool.

    Actually what my problem is..in jmeter I am getting http response as AMF or text.
    In AMF option, I am able to view text in readable state but when i change it to text mode…data appears inb form of binary.

    So what I here wants to do is I simply wants to extract some text value from AMF response that is in readable form.But Jmeter Response text extractor works only for text option not for AMF response text.So can you help me out how one can get binary data converted into text or any other way to resolve this.

    Thanks in advance!