Dev411 Blog: Category json

Catalyst - Customizing the view to stop IE from caching JSON

2006-07-25T10:49:00-05:00

Often times you'll want to customize your response specific to the View being used, e.g. setting extra headers. This can be done directly in your View class by creating a process method. All View base classes have a process method defined in Catalyst::View that gets called at rendering time. By adding a process method in your subclass and redispatching to the parent you can do some preliminary processing.

One use is preventing IE from caching JSON responses by setting the Cache-Control and Pragma HTTP response headers. You can do this in the application's JSON view as follows:

package MyApp::View::JSON;
use strict; use warnings;
use base qw/Catalyst::View::JSON/;

sub process {
  my $self = shift;
  my $c = $_[0];
  $c->res->header( 'Cache-Control' =>
    'no-store, no-cache, must-revalidate,'.
    'post-check=0, pre-check=0, max-age=0'
  );
  $c->res->header( 'Pragma' => 'no-cache' );
  $c->res->header( 'Expires' => 'Thu, 01 Jan 1970 00:00:00 GMT' );
  $self->NEXT::process(@_);
}

1;

Your process method will get called, set it's response headers and then use NEXT to call C::V::JSON's process method.

JSON XSS exploit: don't use text/html

2006-07-24T20:40:00-05:00

Jim Ley reports on the Google JSON XSS exploit with example code and screen shots of stealing information from the AdSense site. The moral of the story is don't use text/html for the MIME type when returning JSON, use application/json which is an IETF standard (RFC 4627) now. Most browsers should handle application/json fine, however Opera may have problems and you may want to use application/x-javascript for that. Something to remember even if your AJAX code/library doesn't care about the MIME type returned by the server, e.g. Dojo.

If you are using Catalyst and Catalyst::View::JSON, your JSON response will automatically be set to application/json for all user agents except Opera (which gets application/x-javascript) so you're already safe(r).

Dojo - Using dojo.io.bind

2006-06-03T23:15:00-05:00

I recently looked into Dojo Toolkit 0.3.0's dojo.io.bind to perform AJAX requests because of it's good handling of memory leaks. I checked out the dojotoolkit.org intro page which was okay but didn't cover how to pass parameters or how to process a JSON response so I thought I'd put some info together here. Here are a few things worth knowing:

A dojo response has a type that is typically set to load or error, though sometimes it may be neither. With dojo, type=="load" means success.
Dojo.io.bind has a few handlers to process the response. The load handler is executed when the response type=="load" and the error handler is executed when the response type=="error". There is also a catch-all handle handler that can accept all response types so you can process a response that has a type that's neither load nor error.
If request parameters are set as a hashref using content, dojo.io.bind will automatically turn them into form post parameters
The response mimetype (or content-type) is set in the client JavaScript. Dojo uses this to determine how to process the response. Dojo ignores content-types set by the server

Here are two examples to help put the above together. The examples cover a plain HTML fragment response as well as a JSON response. Both examples submit a hashref. Dojo can also submit an entire form automatically. This is covered on the Dojo.IO Intro link above and isn't covered here, for now. Thanks to Toby Corkindale who posted his saveHandler function to the catalyst mailing list.

Example 1: HTML processed by load
This first example retrieves a single fragment of HTML in the response body and processes it using the load handler.

dojo.io.bind({
    url: "/binduri",
    method: "post",
    /* optional content. If the content is in the
     * form of a hashref they are converted to
     * post paramters */
    content: {
        username: "George",
        password: "curious"
    },
    load: function(type,data,evt) {
        if (data) alert('Response Data: '+data);
    },
    mimetype:'text/html'
});

Example 2: JSON processed by handle
This second example processes a JSON response using the all-in-one handle handler. The handle is typically used in special situations where load and error are not appropriate such as a progress bar. It is shown here just to provide the syntax. To process a JSON response, set the dojo.io.bind mimetype to text/json and return the JSON object in the response body.

dojo.io.bind({
    url: "/binduri",
    method: "post",
    /* optional content. If the content is in the
     * form of a hashref they are converted to
     * post paramters */
    content: {
        username: "George",
        password: "curious"
    },
    handle: function(type,data,evt) {
        if (type=='load') {
            alert( 'Successful response' );
            if (data.myItem)
                alert('Response JSON Item: '
                +data.myItem);
        } else if (type=='error') {
            alert('A error has occurred');
        } else {
            alert('Some other event has occurred');
        }
    },
    mimetype:'text/json'
});

Conclusion
Dojo.io has a nice interface that has a few conveniences including auto-conversion of a hashref to request form parameters and auto-eval of JSON set in the response body. Along with the good memory leak handling for IE, a very nice implementation.

Prototype - X-JSON fails on long value in IE

2006-06-01T15:14:00-05:00

Unlike Dojo Toolkit and other client-side AJAX libraries that read a JSON object from the response body, prototype.js reads it from the custom X-JSON header. I didn't run into any problems with prototype's approach developing on Firefox but I soon discovered IE 6 has a max header size that will cause prototype 1.5.0 rc0's evalJSON method to silently fail with an unreported [object Error] error. In my case, IE 6 would handle an X-JSON string with length of 138 bytes but would fail with a length of 1659 bytes. Firefox 1.5.0.3 and 1.0.7 worked fine with both strings. I'm not sure what IE 6's exact limit is but the fact there's a limit at all is discouraging IMO.

The argument I've seen for putting the JSON object in X-JSON is that you can put a separate large quanity of HTML in the response body. This design works well when you only have one or zero large quantities of HTML. As soon as you have two, you need to either put them both in the response body using another serialized data notation or put them both in the JSON object and put the JSON object in the response body instead of the X-JSON header. It seems better to simply expect the JSON object in the response body in all cases than in cases when there's only one or zero large return values.

Let's take a look at an example. In the following the one Ajax JSON response can return meta data about the overall response ("Result":"ok") as well as a list of "Posts", say when updating a post list on something like Digg's homepage. Further more a list of "Hot Stories" can simultaneously updated in the same request. Each one of the stories may have a description or overall information that exceeds IE 6's max header length.

{
  "Result":"ok",
  "Posts":[
    {
      "time":1149349580,
      "id":100001,
      "title":"Article 11 title",
      "desc":"Article 11 desc
         (longer than IE header length)",
      "votes":120,
      "comments":5,
      "submitter":"garfield"
    },
    {
      "time":1149349583,
      "id":100002,
      "title":"Article 12 title",
      "desc":"Article 12 desc
         (longer than IE header length)",
      "votes":125,
      "comments":15,
      "submitter":"garfield"
    }
  ],
  "Hot Stories":[
    {
      "time":1149348000,
      "id":1001,
      "title":"Hot Article 1 title",
      "desc":"Hot Article 1 desc
         (longer than IE header length)",
      "votes":1120,
      "comments":115,
      "submitter":"garfield"
    },
    {
      "time":1149349000,
      "id":1002,
      "title":"Hot Article 2 title",
      "desc":"Hot Article 1 desc
         (longer than IE header length)",
      "votes":1120,
      "comments":115,
      "submitter":"garfield"
    }
  ]
}

I understand where having a proprietary X-JSON header and a separate response body is useful however I view those situations as a subset of a more generic, and powerful, use. With more complex return values, putting the JSON object in the response body is ultimately a more flexible and extensible design.

I'll continue to use prototype for now but I'm going to set all my JSON objects in the response body and eval it myself given IE 6's short comings. A bigger concern for me is that I feel Prototype's use of putting the JSON object in X-JSON will encourage inefficient AJAX implementations that end up using more HTTP request/response cycles than necessary. I really hope prototype.js will move their auto-eval feature to operate on the response body due to this problem and to make it more compatible with other JS AJAX libraries at the same time.

Prototype, JSON and Catalyst

2006-05-31T19:42:00-05:00

I use the prototype.js library (aka prototype) for AJAX and recently moved to using JSON to return multiple elements instead of a single one. I had to rearchitect my client request code and my server code to handle JSON but it's worth it to minimize the number of HTTP requests. I use Catalyst and JSON::Syck which make generating the JSON response on the server-side very easy.

Prototype.js is interesting in that it auto-evals a JSON object that's placed in the X-JSON header instead of the response body like other libaries such as Dojo Toolkit. This allows you to send HTML in the response body but I'm not convinced this is a good implementation since it supports only one HTML fragment in the response body. I think it would be better to simply put the JSON object in the response body and have all the HTML fragments in the JSON object. I just seems cleaner to put all the HTML fragments in JSON instead of fragment 1 in the response body and fragments 2-n in JSON. Although I have the X-JSON header working, I may just move to the more standard JSON in response body approach and eval the JSON string myself. To read the auto-evaled JSON response, the following example expects a JSON associative array and displays the value of the myItem key (more code is available in the wiki link below):

onComplete: function( request, json ) {
    alert( json.myItem );
}

On the server-side, I use the Catalyst framework to set the header and JSON::Syck to transform Perl data structures to JSON. JSON::Syck is a wrapper around the C libsyck library and very fast. There is a view, Catalyst::View::JSON, however it puts the JSON string in the response body because that is what Dojo and other libraries expect. Since AJAX libraries don't idnetify themselves in anyway to the server (e.g. user agent), the Catalyst View would probably need a patch to with a configuration switch to populate X-JSON instead of the response body. For now, I'm using JSON::Syck directly. The following are the minimal calls to use. If you have UTF-8 output, see C::V::JSON for UTF-8 encoding.

$c->res->header(
    'X-JSON' => JSON::Syck::Dump( \%args );
);

I've put together some example code that I thought would be better placed on the wiki. It can be found at: Prototype.js, JSON and Catalyst.

UPDATE: After running into the IE max header length issue, I've decided to put the JSON object in the response body and manually eval request.responseText on the client.