JSON – JavaScript Object Notation

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition – December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

JSON is built on two structures:

• A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.

• An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

These are universal data structures. Virtually all modern programming languages support them in one form or another. It makes sense that a data format that is interchangable with programming languages also be based on these structures.

In JSON, they take on these forms:

An object is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace). Each name is followed by : (colon) and the name/value pairs are separated by , (comma).

An array is an ordered collection of values. An array begins with [ (left bracket) and ends with ] (right bracket). Values are separated by , (comma).

A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested.

A string is a collection of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. A character is represented as a single character string. A string is very much like a C or Java string.

A number is very much like a C or Java number, except that the octal and hexadecimal formats are not used.

Whitespace can be inserted between any pair of tokens. Excepting a few encoding details, that completely describes the language.

JavaScript is a general purpose programming language that was introduced as the page scripting language for Netscape Navigator. It is still widely believed to be a subset of Java, but it is not. It is a Scheme-like language with C-like syntax and soft objects. JavaScript was standardized in the ECMAScript Language Specification, Third Edition.

The Object: An Introduction

Behold, an Object…

var myFirstObject = {};

It may not look like much, but those squiggly braces have the potential to record every bit of information humanity has ever gathered, and express the most complex programs computer scientists can dream up. In fact, Javascript itself is stored inside a set of squiggly braces just like that, as are all of its primitive data types — strings, numbers, arrays, dates, regular expressions, they’re all objects and they all started out just like myFirstObject.

Creating A New Object

The old way to create a new object was to use the “new” keyword.

var myJSON = new Object();

This method has been deprecated now in favor of simply defining an empty object with squigly braces…

var myJSON = {};

Objects as Data

At its most base level a Javascript Object is a very flexible and robust data format expressed as “name/value pairs”. That is, an object holds a name which is an object’s property — think of it as a plain old variable name that’s attached to the object name. And the object holds the value of that name. Here’s an example…

var myFirstJSON = { "firstName" : "John",
"lastName"  : "Doe",
"age"       : 23 };
 
document.writeln(myFirstJSON.firstName);  // Outputs John
document.writeln(myFirstJSON.lastName);   // Outputs Doe
document.writeln(myFirstJSON.age);        // Outputs 23

This object has 3 properties or name/value pairs. The name is a string — in our example, firstName, lastName, and age. The value can be any Javascript object (and remember everything in Javascript is an object so the value can be a string, number, array, function, even other Objects) — In this example our values are John, Doe, and 23. John and Doe are strings but age is a number and as you can see this is not a problem.

This data format is called JSON for JavaScript Object Notation. What makes it particularly powerful is that since the value can be any data type, you can store other arrays and other objects, nesting them as deeply as you need. Here is an example of a somewhat complex JSON structure…

var employees = { "accounting" : [   // accounting is an array in employees.
{ "firstName" : "John",  // First element
"lastName"  : "Doe",
"age"       : 23 },
 
{ "firstName" : "Mary",  // Second Element
"lastName"  : "Smith",
"age"       : 32 }
], // End "accounting" array.
"sales"       : [ // Sales is another array in employees.
{ "firstName" : "Sally", // First Element
"lastName"  : "Green",
"age"       : 27 },
 
{ "firstName" : "Jim",   // Second Element
"lastName"  : "Galley",
"age"       : 41 }
] // End "sales" Array.
} // End Employees

Here employees is an object. That object has two properties or name/value pairs. Accounting is an array which holds two JSON objects showing the names and age of 2 employees. Likewise sales is also an array which holds two JSON objects showing the name and ago of the two employees who work in sales. All of this data exists within the employees object. There are several different ways to access this data.

Accessing Data In JSON

The most common way to access JSON data is through dot notation. This is simply the object name followed by a period and then followed by the name/property you would like to access.

var myObject = { ‘color’ : ‘blue’ };

document.writeln(myObject.color); // outputs blue.

If your object contains an object then just add another period and name…

var myObject = { 'color' : 'blue',
'animal' : {'dog' : 'friendly' }
};
 
document.writeln(myObject.animal.dog); // outputs friendly

Using the “employee” example above, if we wanted to access the first person who worked in sales…

document.writeln(employees.sales[0].firstName + ' ' + employees.sales[0].lastName);

We can also access the second person who works in “accounting”.

document.writeln(employees.accounting[1].firstName + ' ' + employees.accounting[1].lastName);

To recap, the “employee” example is an object which holds two arrays each of which holds two additional objects. The only limits to the structure are the amount of storage and memory available to it. Because JSON can store objects within objects within objects and arrays within arrays that can also store objects, there is no virtual limit to what a JSON object can store. Given enough memory and storage requirement, a simple JSON data structure can store, and properly index, all the information ever generated by humanity.

Simulating An Associative Array

You can also access JSON data as if it were an Associative Array.

var myFirstJSON = { "firstName" : "John",
"lastName"  : "Doe",
"age"       : 23 };
 
document.writeln(myFirstJSON["firstName"]);  // Outputs John
document.writeln(myFirstJSON["lastName"]);   // Outputs Doe
document.writeln(myFirstJSON["age"]);        // Outputs 23

Be aware that this is NOT an associative array, however it appears. If you attempt to loop through myFirstObject you will get, in addition to the three properties above, any methods or prototypes assigned to the object, so while you’re more than free to use this method of addressing JSON data, just treat it for what it is (Object Properties) and not for what it is not (Associative Array).

Receiving JSON via AJAX

There are three seperate ways to receive JSON data via AJAX. Assignment, Callback, and Parse.

JSON Via Assignment

There’s no standard naming convention for these methods, however “assignment method” is a good descriptive name because the file comming in from the server creates a javascript expression which will assign the JSON object to a variable. When the responseText from the server is passed through eval, someVar will be loaded with the JSON object and you can access it from there.

var JSONFile = “someVar = { ‘color’ : ‘blue’ }”; // example of what is received from the server.

eval(JSONFile); // Execute the javascript code contained in JSONFile.

document.writeln(someVar.color); // Outputs ‘blue’

JSON Via Callback

The second method calls a pre-defined function and passes the JSON data to that function as the first argument. A good name for this method is the “callback method”. This approach is used extensively when dealing with third party JSON files (IE, JSON data from domains you do not control).

function processData(incommingJSON) {
document.writeln(incommingJSON.color); // Outputs 'blue'
}
 
// example of what is received from the server...
var JSONFile = "processData( { 'color' : 'blue' } )";
 
eval(JSONFile);

JSON Via Parse

The third and final method sends a raw object which must be parsed by a function. This could be referred to as the “parse method”. This is, by far, the safest and most secure way to transfer JSON data and it will be a part of the next version of Javascript due to be released in 2008. For now, it is, unfortunately, limited only to domains which you control.

// The following block implements the string.parseJSON method
(function (s) {
// This prototype has been released into the Public Domain, 2007-03-20
// Original Authorship: Douglas Crockford
// Originating Website: http://www.JSON.org
// Originating URL    : http://www.JSON.org/JSON.js
 
// Augment String.prototype. We do this in an immediate anonymous function to
// avoid defining global variables.
 
// m is a table of character substitutions.
 
var m = {
'\b': '\\b',
'\t': '\\t',
'\n': '\\n',
'\f': '\\f',
'\r': '\\r',
'"' : '\\"',
'\\': '\\\\'
};
 
s.parseJSON = function (filter) {
 
// Parsing happens in three stages. In the first stage, we run the text against
// a regular expression which looks for non-JSON characters. We are especially
// concerned with '()' and 'new' because they can cause invocation, and '='
// because it can cause mutation. But just to be safe, we will reject all
// unexpected characters.
 
try {
if (/^("(\\.|[^"\\\n\r])*?"|[,:{}\[\]0-9.\-+Eaeflnr-u \n\r\t])+?$/.
test(this)) {
 
// In the second stage we use the eval function to compile the text into a
// JavaScript structure. The '{' operator is subject to a syntactic ambiguity
// in JavaScript: it can begin a block or an object literal. We wrap the text
// in parens to eliminate the ambiguity.
 
var j = eval('(' + this + ')');
 
// In the optional third stage, we recursively walk the new structure, passing
// each name/value pair to a filter function for possible transformation.
 
if (typeof filter === 'function') {
 
function walk(k, v) {
if (v && typeof v === 'object') {
for (var i in v) {
if (v.hasOwnProperty(i)) {
v[i] = walk(i, v[i]);
}
}
}
return filter(k, v);
}
 
j = walk('', j);
}
return j;
}
} catch (e) {
 
// Fall through if the regexp test fails.
 
}
throw new SyntaxError("parseJSON");
};
}
) (String.prototype);
// End public domain parseJSON block
 
// begin sample code (still public domain tho)
JSONData = '{"color" : "green"}';   // Example of what is received from the server.
testObject=JSONData.parseJSON();
document.writeln(testObject.color); // Outputs: Green.

As you can see, you’ll need to include the public domain prototype which will parse the JSON data, however once it’s included, processing JSON data is as simple as it looks in the last three lines of the above example. Of the three extraction methods, the “parse method” is the most secure and exposes your code to the fewest problems. You should use this method wherever possible in all of your JSON requests via AJAX.

Retrieving JSON Data Via Ajax

For these examples we’ll be using the Ajax framework from the article: The Ultimate Ajax Object.

When you are communicating with your own servers, AJAX is one of the better ways to transfer data from the server to the browser. Provided the server is under your control you can be reasonably assured this method of data transfer is safe. Here is an example of a simple AJAX request which communicates with the server, retrieves some data, and passes it along to a processing function. We’ll be using the callback method here where the JSON object will execute a pre-defined function after it’s loaded, in this case — processData(JSONData);

function processData(JSONData) {
alert(JSONData.color);
}
 
var ajaxRequest = new ajaxObject('http://www.somedomain.com/getdata.php');
ajaxRequest.callback = function (responseText) {
eval(responseText);
}
ajaxRequest.update();
 
// In this example we assume the server sends back the following data file
// (which the ajax routine places in responseText)
//
// processData( { "color" : "green" } )

In this example our data file, because it’s actually javascript code, when passed through the eval statement will execute processData, passing our actual JSON data to the function as the first argument.

The next example will use the parse method and assumes you have the parseJSON prototype located elsewhere in your code.

function processData(JSONData) {
alert(JSONData.color);
}
 
var ajaxRequest = new ajaxObject('http://www.somedomain.com/getdata.php');
ajaxRequest.callback = function (responseText) {
JSONData = responseText.parseJSON();
processData(JSONData);
}
ajaxRequest.update();
 
// In this example we assume the server sends back the following data file
// (which the ajax routine places in responseText)
//
// { "color" : "green" }

Now when the the server returns the JSON file, it’s parsed on the line which reads “JSONData = responseText.parseJSON();” and then JSONData is passed to processData. The end result is the same as the first example, but if, for some reason, the JSON data contained malicious or invalid data then the second example would be more likely to securely handle any problems.

Howto Send JSON Data To The Server

Because AJAX data is sent as an encoded string, some preparation of JSON data must be made before it can be sent to the server. Fortunately, Douglas Crockford at JSON.org has released a set of very useful routines which will convert any Javascript data type into a JSON string which can be easily sent to the server.

The source for this library can be obtained at http://www.JSON.org/JSON.js. The code is public domain and is as easy to use as clipping the data and pasting it into your toolbox.

The following example defines a JSON object then uses the toJSONString() method to convert the object into a string which is ready to be sent to the server.

var employees = { "accounting" : [   // accounting is an array in employees.
{ "firstName" : "John",  // First element
"lastName"  : "Doe",
"age"       : 23 },
 
{ "firstName" : "Mary",  // Second Element
"lastName"  : "Smith",
"age"       : 32 }
], // End "accounting" array.
"sales"       : [ // Sales is another array in employees.
{ "firstName" : "Sally", // First Element
"lastName"  : "Green",
"age"       : 27 },
 
{ "firstName" : "Jim", // Second Element
"lastName"  : "Galley",
"age"       : 41 }
] // End "sales" Array.
} // End Employees
 
var toServer = employees.toJSONString();
 
document.writeln(toServer);

This outputs:

//Outputs:
//{"accounting":[{"firstName":"John","lastName":"Doe","age":23},{"firstName":"Mary","lastName":"Smith","age":32}],"sales":[{"firstName":"Sally","lastName":"Green","age":27},{"firstName":"Jim","lastName":"Galley","age":41}]}

Retreiving JSON From Third Party Servers

Yahoo Pipes

Almost all of the web is still hidden behind XML (rss) which means you need a server to actually do anything at all with that data. With Yahoo Pipes however you can transform any RSS/XML feed into JSON. To do this, simply find the URL of the RSS feed you would like to use and append it to the end of the following line…

http://pipes.yahoo.com/pipes/9oyONQzA2xGOkM4FqGIyXQ/run?&_render=JSON&_callback=piper&feed=

This is a yahoo pipe which will accept a RSS/XML feed and convert it to JSON. Using this little tool your web pages can DIRECTLY deal with ANY XML/RSS data on the web without the need for a server side script.

Using our example above, modified now to read dzone’s rss feed:

function loadJSON(url) {
var yahooPipe = 'http://pipes.yahoo.com/pipes/9oyONQzA2xGOkM4FqGIyXQ/run?&_render=JSON&_callback=processJSON&feed=';
var headID = document.getElementsByTagName("head")[0];
var newScript = document.createElement('script');
newScript.type = 'text/javascript';
newScript.src = yahooPipe+url;
headID.appendChild(newScript);
}
 
function processJSON(feed){
document.writeln(feed.value.title+'<BR>');
for(i=0; i<feed.value.items.length; i++) {
document.write("<a href='"+feed.value.items[i].link+"'>");
document.writeln(feed.value.items[i].title+"</a><BR>");
}
}
 
loadJSON('http://feeds.dzone.com/dzone/frontpage');

JSON Security

Most of the hysteria over JSON security involves very far-fetched scenarios. One scenario, for instance, assumes a user can be “tricked” to visiting a web page and that web page initiates a third party <script> JSON-request to a secure site and “sniffs” for sensitive data it gets back. This attack attempts to trick the server into thinking the user is on the server’s site and requesting JSON data. It assumes the victim (server) is serving JSON as a callback or assignment — if the server is sending a raw JSON file meant to be parsed, this attack is ineffective because a <script> can’t initiate the parse only AJAX can parse responseText and AJAX can not be used across different domains.

This is the reason this article (and most security professionals) recommend you serve sensitive JSON data as a raw JSON file which must be passed through a parseJSON method.

And of course, it never hurts to mention again that if your web pages contain ANY sensitive data what-so-ever, they should never attempt to use <script> tags to load third-party JSON data.

A fairly decent overview of JSON security vulnerabilities can be found here: JavaScript Hijacking.pdf

An overview of VERY exotic AJAX security vulnerabilities via prototype attacks can be found here: Subverting Ajax.pdf

JSON Best Practices

In the wake of the above security report, Douglas Crockford (Senior JavaScript Architect at Yahoo Inc) wrote a blog entry which provided very simple and elegant methods to avoid most every JSON security problem. Those methods boiled down to…

• Never Trust The Browser
• Keep Data Clean (Don’t embed functions in your JSON data)
• Avoid Third Party JSON
• Authenticate on the server side all JSON data requests (make sure they’re on your site)
• Use SSL (Browser Encryption) for sensitive data.

The future of JSON

Right now JSON is in its infancy, however it’s a pretty big baby. PHP supports JSON in version 5. And the first cracks in XML’s domination of RSS are just starting to appear.

In 2008 most browsers will be able to parse JSON and to convert any variable into JSON. There is also an upcoming standard (also from Douglas Crockford) which will enable a browser to securely request a JSONfile from outside the current domain. When both of these technologies have been implemented, we will be entering the world of Web 3.0 where any browser can request any JSON published data anywhere in the world and manipulate it to its own ends without the need to involve any proxy servers.

XML when it was introduced made waves, but JSON, when it is fully supported by the browser, will make tsunamis and if you’re not already preparing for that day, and the new applications browser-supported JSON will enable then you’re very much cheating yourself out of a front-row seat at the next revolution.

JSON and PHP

• php-json is an extremely fast PHP C extension for JSON (JavaScript Object Notation) serialisation. It conforms to the JSON specification.

• Zend_Json provides convenience methods for serializing native PHP to JSON and decoding JSON to native PHP. For more information on JSON, visit the JSON project site.

• XML-RPC for PHP. A PHP implementation of the XML-RPC web RPC protocol. Extra modules provide support for the JSON and JSONRPC protocols. A javascript version of the library is also available.

JSON and JAVA

• SOJO stands for Simplify your Old Java Objects or, in noun form, Simplified Old Java Objects.

  The intention for this project is a Java framework, that convert JavaBeans in a simplified representation. So it is easy and uniform to handle and control the access to JavaBeans properties.

• google-gson is a Java library that can be used to convert Java Objects into its JSON representation. It can also be used to convert a JSON string to an equivalent Java object. Gson can work with arbitrary Java objects including pre-existing objects that you do not have source-code of.
• SON-lib is a java library for transforming beans, maps, collections, java arrays and XML to JSON and back again to beans and DynaBeans.

  It is based on the work by Douglas Crockford in http://www.json.org/java .

JSON and C++

• TinyJSON - a minimalistic yet powerful JSON parser that makes extensive use of several Boost classes, such as the great Boost.Spirit parsing framework, Boost.Any, and Boost.Smart_Ptr.
• jsoncpp is an implementation of a JSON (http://json.org) reader and writer in C++. JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate.
• JOST, also uses Boost Spirit! There are, however, considerable differences between JSON Spirit and JOST in both design and implementation.

Hunlock & Mauriziostorani Blog