Census RIA Benchmark Updated with GZip and Laszlo

By James Ward | Published: August 15, 2007

I’ve just posted the newest version of the Census RIA Benchmark application which compares data loading via various methods in Ajax, Flex, and now Laszlo. When I first began talking about the results of these benchmarks and heavily advocating AMF for large data sets some people suggested that my results were flawed because I wasn’t using gzip compression on the text streams. I have always wondered how many people actually use gzip but these critics seemed to indicate that everyone was using it. Based on some past experiments I postulated that AMF would still be significantly faster than using gzip. There is a trade off when you use gzip. The amount of data that has to cross the wire is significantly smaller but there is increased latency on the server to do the gzip and on the client to ungzip. Sometimes this trade off is worth it. Unless you are using AMF. AMF uses a very basic form of compression that is extremely fast, faster even than just creating XML or JSON strings. And of course much faster than creating big strings and then compressing/decompressing them. Now there is definitive proof of this. AMF is still by far the fastest method for loading large data sets in RIAs. Not only is it faster for loading the data, Tamarin makes client-side operations like sorts and filters extremely fast. All of this is evident if you spend some time with the Census application. Check it out and let me know what you think.

For this version I’ve added a Laszlo test. Sometimes people ask what the differences are between Flex and Laszlo. This was my first time building anything with Laszlo so I won’t comment on the differences in programming models since my views are probably skewed after spending three years with Flex and only a few hours with Laszlo. But two significant differences between Flex and Laszlo can easily be seen in the benchmark. First Laszlo doesn’t use AMF3 that I know of. Let me know if I am wrong about this so I can update the test. Also Laszlo does not yet compile to Flash 9 bytecode so Laszlo applications are not able to take advantage of the Tamarin virtual machine. Beyond that I really don’t know much about the differences. If anyone that reads this has used both it would be great to hear about your experiences. Just please don’t tell me that Laszlo is better because it has multiple output formats. Tell me how this or other features have made a difference in what you have built. Also since this was my first attempt at building a Laszlo application I may not have done everything correctly. So please take a look at the source code and help me make it better and more accurate.

I hope that you find the Census RIA Benchmark application helpful. Please let me know if you have any feedback about it.

This entry was posted in Ajax, Flex, Laszlo, RIA. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.
  • Hakon

    We are currently using WCF in .Net to produce XML Web Services for a Flex Client. In WCF it is possible to write Custom Encoders (http://msdn2.microsoft.com/en-us/library/ms735115.aspx). Do you think it would be possible to write such an encoder for AMF and call it from Flex? I think this would be a great possibility for connecting Flex to .Net backends.

  • http://www.jamesward.org James Ward

    Hi Hakon,

    I believe that WebOrb has a .Net AMF3/remoting server. There may be other options that I’m not aware of. Please let me know what you discover.

    Thanks.

    -James

  • http://www.bnowak.com Bnowak

    There is also something called Fluorine (fluorine.thesilentgroup.com)

  • http://www.mattking.org Matt King

    Why not post tests with real world result sets? I’m just wondering how many sites will actually download 5000 rows of data at a time. Typically searching, filtering and paging will reduce data sets to at most 100 results at a time being brought down from a server.

  • Pingback: Ajaxian » RIA Data Loading Benchmarks

  • http://www.jamesward.org James Ward

    Matt,

    Everyone has different requirements and use cases. This benchmark was created specifically to show how RIAs deal with large data sets. I meet with many people building RIAs that need to deal with thousands and sometimes hundreds of thousands of rows. For them these benchmarks are very relevant. For you they may not be.

    -James

  • http://jroller.com/page/tedgoddard Ted Goddard

    It would be interesting to measure the performance of ICEfaces (http://www.icefaces.org) under similar conditions. Is the source code to the benchmark available?

  • http://www.jamesward.org James Ward

    Hi Ted,

    That would be great! You can check out the “census” module from cvs on sourceforge: http://sf.net/projects/flexapps

    Email me if you would like to work with me on this james jamesward org

    -James

  • http://dylanschiemann.com/ Dylan Schiemann

    Which version of each toolkit are you using? That certainly doesn’t look like Dojo 0.9′s performance metrics, nor does it look like you are using a Dojo build…

  • http://www.jamesward.org James Ward

    Hi Dylan,

    I’m using Dojo 0.4.3. According to a conversation I had with Alex Russell I can’t use Dojo 0.9 because there isn’t a sortable datagrid/table yet. I’ll be happy to update to Dojo 0.9 when there is.

    -James

  • http://www.kalbfus.com Brian Kalbfus

    I noticed that your census app is taking 8.9 seconds to transfer 47.2k of data with Laszlo and 1.0 second to transfer 29.1k of data with Flex AMF3. Are you compiling the Laszlo portion to a SOLO executable or running it in interpreted mode? Did you try version 4.0 with its DHTML compile target? I’d imagine that would be similar to the AJAX performance you test here.

    In any event, your census app looks real cool and informative – this discrepancy jumped out at me!

    Thanks!
    Brian

  • http://www.jamesward.org James Ward

    Hi Brian,

    Did you get these results multiple times? How many rows were you using? Perhaps the way I’m capturing the transfer time is not accurate. Right-click on the app to view the source. Let me know if you know of a better way to capture that information. Also I’m compiling the Laszlo version to a SWF with LPS 4.0.3. I do not have a DHTML version of Laszlo but it would be interesting to add one.

    -James

  • http://www.kalbfus.com Brian Kalbfus

    Hi James,

    I got the same results multiple times and at two times of the day. To use DHTML, I believe you add the URL parameter: lzr=dhtml

    I don’t know if you can specify an lzr parameter to the canvas like you can with debug, but it may be worth a try.

    -Brian

  • http://www.kalbfus.com Brian Kalbfus

    James,

    Compiling to dhtml is a simple matter of opening the xml in Laszlo Explorer and clicking the SOLO button to get a zipped directory of the current application config.

    -Brian

  • Tom Lane

    Hey James, it looks like the Flex SOAP AS test is transferring double the data of the E4X version, which seems wrong. The only difference should be in the client deserialization/render times.

  • http://www.jamesward.org James Ward

    Hi Tom,

    Yes. That is strange. I’ll investigate.

    -James

  • Chris Shepperd

    Hi James,

    First of all, great app. Second, not sure why but it makes my Firefox 3.0.1 with Flash 9.0.115 crash when I run it. IE works good.

    Another question… If you already have web services created and you don’t have access to make those web services into remote objects, do you think it’s better performance wise to have a translation on an intermediate java server from web services to remote objects (close in proximity to the web services server) and then to the client, or just hit the web service directly. Thanks.

  • http://www.jamesward.org James Ward

    Hi Chris,

    Firefox 3 broke some stuff that used to work with IFrame’s. I’m working on a new version of Census that will fix this.

    I think in most circumstances it would be faster to do the translation on the server from Web Services to AMF because:
    1) Servers are usually faster than clients
    2) Server to server is usually a shorter, faster network route than server to client

    I hope that helps.

    -James

  • Lakshmi

    Hi James,

    Very nice app! I have some questions.

    1) Why is there a significant difference between the transfer time for Soap AS and Soap e4x? Similarly for the flex xml AS and flex xml e4x. Isn’t it the same data that is getting transferred in the two cases?
    2) Also, the difference between soap and xml is significant. Why so? Is it just due to the soap envelope overhead or some thing else?
    3) When you say xml (as in flex xml AS), would REST using HTTPService fall in this category?
    4) The difference between e4x and AS is interesting! The parse time is high for AS but render time is much lower. If you render many times, it looks like you are better off using AS. Is that an appropriate conclusion?

    Thanks,
    Lakshmi

  • http://www.jamesward.com James Ward

    Hi Lakshmi,

    1) My best guess is that in the case of E4X something is happening at a lower level in Flash Player than I can get to with my tests. So the ~100ms extra on the E4X tests should probably be in the parse category.
    2) In general SOAP has a lot more parsing overhead than RESTful XML.
    3) The XML tests are RESTful XML using HTTPService in Flex.
    4) E4X has lower up front costs because it doesn’t parse the XML. But you take a it when you actually query and interact with the XML data. Also E4X is a different language than ActionScript so many times the decision is made depending on which language you want to use. I always prefer AMF to XML.

    I hope that helps.

    -James

  • Jeff

    Hey James, great looking tool but it is not working for me with FF3.0.6 and FP 10.0.12.36 Debug. Any ideas why?

  • http://www.jamesward.com James Ward

    @Jeff

    There is a bug in FF’s iframe handling. To get around it you need to click on the results pane which then launches the test. I’m still working on a new version that will fix this.

    -James

  • Pingback: Flex Data Service Benchmarks « Flex Flash ActionScript Developer