JSON is a handy serialization format for passing data between servers and browsers and between independent, cooperating web pages. It’s increasingly the format of choice for website APIs.

ECMAScript 5 (the standard underlying JavaScript) includes built-in support for producing and parsing JSON. SpiderMonkey has included such support since before ES5 added it.

SpiderMonkey’s support, because it predated ES5, hasn’t always agreed with ES5. Also, because JSON support was added before it became ubiquitous on the web, it wasn’t written with raw speed in mind.

Improving JSON.parse

We’ve now improved JSON parsing in Firefox 5 to be fast and fully conformant with ES5. For awhile we’ve made improvements to JSON by piecemeal change. This worked for small bug fixes, and it probably would have worked to fix the remaining conformance bugs. But performance is different: to improve performance we needed to parse in a fundamentally different way. It was time for a rewrite.

What parsing bugs got fixed?

The bugs the new parser fixes are quite small and generally shouldn’t affect sites, in part because other browsers overwhelmingly don’t have these bugs. We’ve had no compatibility reports for these fixes in the month and a half they’ve been in the tree:

• The number syntax is properly stricter:
  • Octal numbers are now syntax errors.
  • Numbers containing a decimal point must now include a fractional component (i.e. 1. is no longer accepted).
• JSON.parse("this") now throws a SyntaxError rather than evaluate to true, due to a mistake reusing our keyword parser. (Hysterically, because we used our JSON parser to optimize eval in certain cases, this change means that eval("(this)") will no longer evaluate to true.)
• Strings can’t contain tab characters: JSON.parse('"\t"') now properly throws a SyntaxError.

This list of changes should be complete, but it’s possible I’ve missed others. Parsing might be a solved problem in the compiler literature, but it’s still pretty complicated. I could have missed lurking bugs in the old parser, and it’s possible (although I think less likely) that I’ve introduced bugs in the new parser.

What about speed?

The new parser is much faster than the old one. Exactly how fast depends on the data you’re parsing. For example, on Opera’s simple parse test, I get around 156000 times/second in Firefox 4, but in Firefox 5 with the new JSON parser I get around 339000 times/second (bigger is better). On a second testcase, Kraken’s JSON.parse test (json-parse-financial, to be precise), I get a 4.0 time of around 140ms and a 5.0 time of around 100ms (smaller is better). (In both cases I’m comparing builds containing far more JavaScript changes than just the new parser, to be sure. But I’m pretty sure the bulk of the performance improvements in these two cases are due to the new parser.) The new JSON parser puts us solidly in the center of the browser pack.

It’ll only get better in the future as we wring even more speed out of SpiderMonkey. After all, on the same system used to generate the above numbers, IE gets around 510000 times/second. I expect further speedup will happen during more generalized performance improvements: improving the speed of defining new properties, improving the speed with which objects are allocated, improving the speed of creating a property name from a string, and so on. As we perform such streamlining, we’ll parse JSON even faster.

Side benefit: better error messages

The parser rewrite also gives JSON.parse better error messages. With the old parser it would have been difficult to provide useful feedback, but in the new parser it’s easy to briefly describe the reason for syntax errors.

js> JSON.parse('{ foo: 17 }'); // unquoted property name
(old) typein:1: SyntaxError: JSON.parse
(new) typein:1: SyntaxError: JSON.parse: expected property name or '}'

We can definitely do more here, perhaps by including context for the error from the provided string, but this is nevertheless a marked improvement over the old parser’s error messages.

Bottom line

JSON.parse in Firefox 5 is faster, follows the spec, and tells you what went wrong if you give it bad data. ’nuff said.

JSON.parse('[1, 2, 3, ]');
JSON.parse('{ "1": 2, }");

We did so because the JSON RFC permitted implementations to accept extensions, and trailing commas are nice for humans to be able to use. The down side is that accepting extra syntax like this makes interoperability harder: implementations which don’t implement the same extension, for reasons every bit as valid as those of implementations allowing the extension, are disadvantaged. ES5 weighed these concerns and chose to precisely specify permissible JSON syntax, putting everyone on the same page: trailing commas are not permitted. Therefore, the examples above should throw a SyntaxError per ES5.

SpiderMonkey has now been changed to conform to ES5 on this point: trailing commas are syntax errors. If you still need to accept trailing commas, you should use a custom implementation that accepts them — but best would be for you to adjust the processes that produce JSON strings including trailing commas to not include them. (If you are an extension or are in privileged code, for the moment you can use nsIJSON.legacyDecode to continue to accept trailing commas. However, note that we have added it only to accommodate legacy code in the process of being updated to no longer generate faulty input, and it will be removed sometime in the future.)

You can experiment with a version of Firefox with this change by downloading a TraceMonkey branch nightly; this change should also make its way into mozilla-central nightlies shortly, if you’d rather stick to trunk builds. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.)

Don’t try to use the arguments or caller properties of functions created in strict mode code. Don’t try to use the callee or caller properties of arguments objects corresponding to invocations of functions in strict mode code. Don’t try to use the caller property of a function if it might be called from strict mode code. You are in for an unpleasant surprise (a thrown TypeError) if you do.

function strict() { "use strict"; return arguments; }
function outer() { "use strict"; return inner(); }
function inner() { return inner.caller; }

strict.caller;    // !!! BAD IDEA
strict.arguments; // !!! BAD IDEA
strict().caller;  // !!! BAD IDEA
strict().callee;  // !!! BAD IDEA
outer();          // !!! BAD IDEA

Really, it’s best not to access the caller of a function, the current function (except by naming it), or the arguments for a given function (except via arguments or by use of the named parameter) at all.

ES5 strict mode: self-limitation, not wish fulfillment

ES5 introduces the curious concept of strict mode. Strict mode, whose name and concept derive from the similar feature in Perl, is a new feature in ES5 whose purpose is to deliberately reduce the things you can do in JavaScript. Instead of a feature, it’s really more the absence of several features, within the scope of the strict-annotated code: with, particularly intrusive forms of eval, silent failure of writes to non-writable properties, silent failure of deletions of non-configurable properties, implicit creation of global-object properties, and so on. The goal of these removals is to simplify both the reasoning necessary to understand such code and the implementation of it in JavaScript engines: to sand away some rough edges in the language.

Magical stack-introspective properties of functions and arguments

Consider this code, and note the expected behavior (expected per the web, but not as part of ES3):

function getSelf() { return arguments.callee; }
assertEq(getSelf(), getSelf); // arguments.callee is the enclosing function

function outer()
{
  function inner() { return arguments.callee.caller; } // inner.caller === outer
  return inner();
}
assertEq(outer(), outer); // fun.caller === nearest function in stack that called fun, or null

function args2()
{
  return args2.arguments;
}
assertEq(args2(17)[0], 17); // fun.arguments === arguments object for nearest call to fun in stack, or null

Real-world JavaScript implementations take many shortcuts for top performance. These shortcuts are not (supposed to be) observable, except by timing the relevant functionality. Two common optimizations are function inlining and avoiding creating an arguments object. The above “features” play havoc with both of these optimizations (as well as others, one of which will be the subject of a forthcoming post).

Inlining a function should conceptually be equivalent to splatting the function’s contents in that location in the calling function and doing some α-renaming to ensure no names in the splatted body conflict with the surrounding code. The ability to access the calling function defeats this: there’s no function being invoked any more, so what does it even mean to ask for the function’s caller? (Don’t simply say you’d hard-code the surrounding function: how do you know which property lookups in the inlined code will occur upon precisely the function being called, looking for precisely the caller property?) It is also possible to access a function’s arguments through fun.arguments. While the “proper” behavior here is more obvious, implementing it would be a large hassle: either the arguments would have to be created when the function was inlined (in the general case where you can’t be sure the function will never be used this way), or you’d have to inline code in such a way as to be able to “work backward” to the argument values.

Speaking of arguments, in offering access to the corresponding function via arguments.callee it has the same problems as fun.caller. It also presents one further problem: in (some, mostly old) engines, arguments.caller provides access to the variables declared within that function when it was most recently called. (If you’re thinking security/integrity/optimization hazard, well, you now know why engines no longer support it.)

In sum these features are bad for optimization. Further, since they’re a form of dynamic scoping, they’re basically bad style in many other languages already.

Per ES5, SpiderMonkey no longer supports this stack-inspecting magic when it interacts with strict mode

As of the most recent Firefox nightly, SpiderMonkey now rejects code like that given above when it occurs in strict mode (more or less). (The properties in question are now generally implemented through a so-called “poison pill” mechanism, an immutable accessor property which throws a TypeError when retrieved or set.) The specific scenarios which we reject are as follows.

First, attempts to access the caller or arguments (except by directly naming the object) of a strict mode function throw a TypeError, because these properties are poison pills:

function strict()
{
  "use strict";
  strict.caller;    // !!! TypeError
  strict.arguments; // !!! TypeError
  return arguments; // direct name: perfectly cromulent
}
strict();

Second, attempts to access the enclosing function or caller variables via the arguments of a strict mode function throw a TypeError. These properties too are poison pills:

function strict()
{
  "use strict";
  arguments.callee; // !!! TypeError
  arguments.caller; // !!! TypeError
}
strict();

Third (and most trickily, because non-strict code is affected), attempts to access a function’s caller when that caller is in strict mode will throw a TypeError. This isn’t a poison pill, because if the "use strict" directive weren’t there it would still “work”:

function outer()
{
  "use strict";
  return inner();
}
function inner()
{
  return inner.caller; // !!! TypeError
}
outer();

But if there’s no strict mode in sight, nothing will throw exceptions, and what worked before will still work:

function fun()
{
  assertEq(fun.caller, null); // global scope
  assertEq(fun.arguments, arguments);
  assertEq(arguments.callee, fun);
  arguments.caller; // won't throw, won't do anything special
  return arguments;
}
fun();

Conclusion

With these changes, which are required by ES5, stack inspection is slowly going the way of the dodo. Don’t use it or rely on it! Even if you never use strict mode, beware the third change, for it still might affect you if you provide methods for other code to call. (But don’t expect to be able to avoid strict mode forever: I expect all JavaScript libraries will adopt strict mode in short order, given its benefits.)

(For those curious about new Error().stack, we’re still considering what to do about it. Regrettably, we may need to kill it for information-privacy reasons too, or at least censor it to something less useful. Nothing’s certain yet; stay tuned for further announcements should we make changes.)

You can experiment with a version of Firefox with these changes by downloading a TraceMonkey branch nightly; these changes should also make their way into mozilla-central nightlies shortly, if you’d rather stick to trunk builds. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.)

var property = 42;
var obj =
  {
    property: 17,
    method: function() { return this.property; }
  };

var bound = obj.method.bind(obj);
assertEq(bound(), 17);

…or with pre-specified leading arguments:

function multiply()
{
  var product = 1;
  for (var i = 0, sz = arguments.length; i < sz; i++)
    product *= arguments[i];
  return product;
}

var productTimesFive = multiply.bind(null /* this */, 5);
assertEq(productTimesFive(4, 3, 2, 1), 120);

…and, in a feature present only in the ES5 bind implementation (and not in any of the numerous precursors), they even work with new:

function Vector()
{
  var args = arguments;
  this.length = arguments.length;
  this.get = function(i) { return args[i]; };
  this.set = function(i, v) { args[i] = v; };
}

var PartialVector = Vector.bind(null /* this, ignored with new */, 3, 7);

var threeSevenTen = new PartialVector(10); // new Vector(3, 7, 10)

For more information, see the article on Function.prototype.bind on MDC. For the utmost information, see the ES5 specification for the method.

As always, you can experiment with a version of Firefox with Function.prototype.bind by downloading a nightly from nightly.mozilla.org. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.)

For quite some time SpiderMonkey and Mozilla-based browsers have supported user-defined getter and setter functions (collectively, accessors), both programmatically and syntactically. The syntaxes for accessors were once legion, but SpiderMonkey has pared them back almost to the syntax recently codified in ES5 (and added new syntax where required by ES5).

// All valid in ES5
var a = { get x() { } };
var b = { get "y"() { } };
var c = { get 2() { } };

var e = { set x(v) { } };
var f = { set "y"(v) { } };
var g = { set 2(v) { } };

SpiderMonkey has historically parsed literal accessors using a slightly-tweaked version of its function parsing code. Therefore, as previously explained SpiderMonkey would accept essentially anything which could follow function in a function expression as valid accessor syntax in object literals.

ES5 requires accessors have exact numbers of arguments

A consequence of parsing accessors using generalized function parsing is that SpiderMonkey accepted some nonsensicalities, such as no-argument setters or multiple-argument getters or setters:

var o1 = { get p(a, b, c, d, e, f, g) { /* why have any arguments? */ } };
var o2 = { set p() { /* to what value? */ } };
var o3 = { set p(a, b, c) { /* why more than one? */ } };

ES5 accessor syntax sensibly deems such constructs errors: a conforming ES5 implementation would reject all of the above statements.

SpiderMonkey is changing to follow ES5: getters require no arguments, setters require one argument

SpiderMonkey has now been changed to follow ES5. There seemed little to no gain in continuing to support bizarre numbers of arguments when the spec counseled otherwise, and any code which does end up broken is easily fixed.

As always, you can experiment with a version of Firefox with these changes to accessor syntax by downloading a nightly from nightly.mozilla.org. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.)

The special __parent__ property has been removed from SpiderMonkey and is no longer in nightly builds. If you don’t use __parent__ or don’t know what the property does, you didn’t miss much.

If you use __parent__, you have a couple replacements. If you were using it to determine the global object for another object, use Components.utils.getGlobalForObject instead. If you were using it only to test its value against an expected value, use nsIDOMWindowUtils.getParent instead (but do note that its semantics are not absolutely identical to those of __parent__). If you were using it from unprivileged web scripts as a potential vector for security exploits, I feel your pain and will take no steps to assuage it. If you were using it some other way, comment and we’ll figure something out for your use case.

If you think you understood the __parent__ property (you probably don’t), or if you’re interested in the nitty-gritty details of JavaScript semantics, read on for the details of this esoteric property and the reasons for its removal.

Scoping in JavaScript

In the following example code, when the function g is invoked, what stores the v variable?

function f(a)
{
  var v = a;
  function g() { return v; }
  return g;
}

var fun = f(2);
fun();

The variable is accessed in an enclosing scope, created when the enclosing function was invoked. In the ECMAScript standard, the location of such variables is an object, stored as an internal [[Scope]] property of the function being called. In ES3 this object was a standard JavaScript object; ES5 tightens semantics slightly and uses a simpler structure, but the idea’s the same.

Meet the __parent__

It’s not possible to access the object stored as [[Scope]] in ECMAScript proper, but it has been possible to access it in SpiderMonkey and Mozilla-based browsers. The magical __parent__ property can often provide access to this value:

this.toString = function() { return "global"; }
var q = 17;
function foo() { return q; }
print(foo.__parent__); // prints "global"

“Often”? When does __parent__ not expose this value?

__parent__ doesn’t always reflect [[Scope]] because ECMAScript requires that certain objects not be made available to scripts. Among such objects are what SpiderMonkey refers to as With objects, Call objects, and Block objects.

With objects

SpiderMonkey creates With objects to handle the esoteric, non-lexical name lookup required by the semantics of the with statement in JavaScript. These semantics require that a name, depending on the runtime object used in the with, refer to a property of that object or to a variable in an enclosing scope. It’s impossible to know in general which will be the case before runtime, and therefore it’s impossible to speed up a lot of script that runs inside a with. (Incidentally, this is why you should never ever ever [ever] use with in performance-critical code. Use a two-letter abbreviation variable to save typing, if verbosity is your concern.) We can’t simply use the with block’s object as the scope, because if we miss on a lookup there we want to fall back to the normal scope; therefore we introduce a With object. Here’s an example of a situation where a With object is created:

with ({ toString: function() { return "with object"; } })
{
  print(function() { return toString; }.__parent__); // prints "with object"
}

__parent__ doesn’t actually give you the With object, because it’s a carefully-tuned internal value. With objects have behavior and functionality optimized for their particular purpose, and if we simply exposed them to scripts it would probably be possible to do Very Bad Things. Consequently, if you try to access __parent__ in a situation where you “should” get a With object, we instead give you back the object where the With object begins its search: the object used for the with block. This may be what a developer superficially familiar with __parent__ might expect, but it is nevertheless a lie.

Call objects

SpiderMonkey’s Call objects represent the local variables of a function call. Therefore, Call objects correspond to the [[Scope]] of functions. Returning to the original example, reproduced below, v is stored in a Call object, which is the [[Scope]] of the function g:

function f(a)
{
  var v = a;
  function g() { return v; }
  return g;
}

var fun = f(2);
print(fun.__parent__);

When you attempt to access g‘s __parent__ property, SpiderMonkey censors the [[Scope]] and returns null, because ECMAScript requires that Call objects not be exposed. This case, which is far more common than the with case, completely prevents access to [[Scope]] at all (rather than exposing a half-representation of the value). This situation is even more pervasive in light of the modern JavaScript encapsulation practice of enclosing libraries in closures for pseudo-namespacing purposes. In many libraries __parent__ on many functions of interest gives no value whatsoever.

Block objects

One of the first things every JavaScript developer learns is that JavaScript variables are not scoped to blocks. Language tutorials usually emphasize this point, because it differs from most other languages (and in particular, it differs from the C-ish family of languages, the syntax of which JavaScript borrows to a fair degree). The conventional wisdom is that names in functions are always scoped to enclosing functions or to the global scope. If you want a locally-defined name, you have to wrap it up in a function.

The conventional wisdom is wrong.

Here’s proof: what does this example print?

var v = "global";
function test(frob)
{
  try
  {
    if (frob === 0)
      return v + " try";
    if (frob === 1)
      throw "local";
  }
  catch (v)
  {
    return v + " catch";
  }
  finally
  {
    if (frob === 2)
      return v + " finally";
  }
  return v + " after finally";
}

print("not throwing, in try:          " + test(0));
print("throwing, in catch:            " + test(1));
print("not throwing, in finally:      " + test(2));
print("not throwing, after finally:   " + test(3));

Behavior depends on whether the binding for v as introduced by the catch is scoped to the function or to something else. JavaScript specifies that v, referring to the value potentially thrown while executing the try block, is scoped only to the catch block. Thus v in the catch refers to the thrown value; all other uses of v are outside the catch block and refer to the global variable v. Therefore the output is this:

not throwing, in try:          global try
throwing, in catch:            local catch
not throwing, in finally:      global finally
not throwing, after finally:   global after finally

Any function which referred to v inside the test function would have a Call object as its [[Scope]] — except for a function defined inside the catch block. Such a function at that location would have to capture the thrown value, so there must be something else on the scope chain before the Call object. SpiderMonkey refers to these objects as Block objects, because they implement traditional block-level scoping.

Block objects have the same issues as Call objects, so SpiderMonkey censors them to null as well. For optimization purposes we’d like to “boil them away” whenever possible, so as not to create an obnoxious little one-property object when we catch an exception and define a function that captures it. Exposing such an object directly prevents these optimizations, and would likely expose some security vulnerabilities.

But __parent__‘s issues don’t stop merely at those induced by complex language semantics. Even perfectly simple code — simpler even than nested functions — has potential for unpredictability.

If certain optimizations are implemented, __parent__ may be a lie

Consider this script in the global scope:

function fun() { return 17; }

print(fun.__parent__);

Does fun‘s [[Scope]] need to be the global object?

It’s OK to cheat if you never get caught.

ECMAScript provides no way for script to access [[Scope]]. It’s a construct used to ease specification, and it need not even exist in implementations. fun never uses any values from its enclosing scope, so there’s no reason that [[Scope]] must be global object. An implementation that carried around [[Scope]] with each function might simply make [[Scope]] be null, which could potentially speed up some garbage collection algorithms. Similarly, a function which refers only to never-modified variables in enclosing scopes might simply copy those variables’ values and, again, make [[Scope]] lie. Should a specification-artifact internal property constrain implementations looking to improve performance, or code size, or any number of other measures of desirability? Given where JavaScript use is heading now, the answer must be no.

For certain types of functions, there are very good reasons, even in simple cases, to make [[Scope]] a lie. SpiderMonkey implements some optimizations along these lines, but those optimizations don’t affect the value we store for [[Scope]] at the present. If it were advantageous to change that, we would do so in a heartbeat. Therefore, while __parent__ may have a reliable value now, it’s possible it would not in the future.

__parent__ is available even where you think it isn’t

Oddly enough, __parent__, despite its representing [[Scope]], isn’t just applicable to function objects. Instead, its value is generalized to all objects, in a way even more difficult to describe than that above. (Very roughly, __parent__ on non-function objects corresponds to the object which would be the __parent__ of a function created in the same context. I think. Mostly the value is used for security checks; improper setting of __parent__ is a common cause of implementation-caused XSS vulnerabilities.) This generalization and its semantics have even less support in the specification, so its meaning is even less clear than for functions.

__parent__ doesn’t expose anything useful

__parent__, beyond having unintuitive behavior, doesn’t tell the developer anything he’d care to know. Why would you need to access the enclosing scope of a function, as an object, in the first place? The very definition of the problem is obscure; it is almost a prerequisite to have read the ECMA-262 specification to be able to ask the question.

In the searching of various codebases that we’ve done, we’ve found only these use cases for __parent__:

Testing

A small number of Mozilla tests use __parent__ to verify proper scoping of objects. Since the value as we use it has security implications, this use case is reasonable — but it has extremely limited applicability. This use case can be supported by creating an alternate means of accessing an object’s parent, a means exposed through XPCOM, not through JavaScript itself, and certainly not by polluting every object with the functionality. We have implemented nsIDOMWindowUtils.getParent(obj) to support this testing-oriented use case. Beware: this method doesn’t censor like __parent__ does! Tread cautiously when using it where a With, Call, or Block object might be exposed, and don’t use the returned object except in immediate strict equality (===) checks. (NB: we’ve implemented it this way out of expediency; if it turns out to be too great a responsibility for typical testers, we’ll likely change it to censor.)

Determining the global context (global object) where an object was created

Since __parent__ often (for any function not nested in another function) is simply the global object, some people have used __parent__ to retrieve the global object associated with an arbitrary object. By walking from __parent__ to __parent__ you can often reach the global object this way. However, since the __parent__ value is sometimes a lie, it’s not as simple as var global = obj; while (global.__parent__) global = global.__parent__;. Instead, you have to be careful to start from an object with a known __parent__, one guaranteed not to be a nested function or similar. The easiest way to do this is to first walk up the prototype chain to eventually bottom out at Object.prototype, then to walk the parent chain. This method is baroque and non-obvious, and it requires careful tiptoeing around incorrect __parent__ values. If you know the object whose global you want to retrieve is a DOM node, the fully-standard node.ownerDocument.defaultView property also provides global access. Otherwise, the use case would be better addressed by exposing a method somewhere to directly retrieve the corresponding global object. We have recently added such a method to support this use case: Components.utils.getGlobalForObject(obj).

Being evil to Firefox

A fair number of security bugs reported against SpiderMonkey in recent years have worked by successfully confusing the engine into assigning a bad parent to an object. This incorrect security context can then be used as a vector to run code in the context of another website (resulting in an XSS hole, possibly usable against any website), or in some cases, in the context of the browser “chrome” itself (providing the capability for arbitrary code execution — the worst-case scenario as far as exploits are concerned). __parent__ is immutable and can’t itself be used to confuse the engine (at least in the absence of bugs, and we have no way to demonstrate such). However, it provides greater visibility into the engine, and it can sometimes expose values to script which wouldn’t (and shouldn’t) be accessible any other way.

Being evil to sandboxing mechanisms

__parent__ exposes security holes in some websites as well as in the browser directly. Such websites are those which purport to “safely” execute scripts provided by other users, through sandboxing or other mechanisms. For example, Facebook uses FBJS, a script-rewriting plus runtime-check mechanism, to allow applications to include interactivity. Google‘s Caja system provides a rigorous capabilities-providing system (also through script-rewriting plus runtime checks) to do the same, and it too is used in public sites these days. One requirement of such systems is that the global object must never be directly exposed: if you have the global object, you have unfettered access to the document, you have eval, you have Function, and you are utterly and thoroughly hosed. __parent__ provides easy access to this, so all these systems must censor access to __parent__, both statically (obj.__parent__) and dynamically (var p = "__parent__"; obj[p]). Not everyone knows, or remembers, that this is necessary. Removing __parent__ reduces attack surface in JavaScript sandboxes.

The former two use cases are better addressed in other ways. The latter two are holes best removed entirely.

__parent__ is being removed

__parent__ is being removed from SpiderMonkey. The value it purports to expose is esoteric and has semantics defined by a specification which JavaScript developers shouldn’t really have to read to understand it. The identification of that value is non-intuitive. Sometimes the value it exposes is a lie. There are plausible reasons why it might be made to lie if potential performance-improving ideas were implemented. It’s exposed even in places where it makes little sense. The use cases for it can be and are better served in other ways — or explicitly not served, when it serves as a vector for security vulnerabilities. In sum __parent__ doesn’t pass muster, so it’s being removed.

You can experiment with a version of Firefox without support for __parent__ by downloading a nightly from nightly.mozilla.org. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.) The next release is many months away, which should provide plenty of time for extension developers to update their extensions to not use __parent__, if by chance they had managed to discover it and use it.

We’ve removed support for a handful of obsolete getter/setter syntaxes in SpiderMonkey and Mozilla. This does not include { get property() { return "value"; }, set property(v) { } }, which is widely used and which is part of the latest standard. If you don’t get any syntax errors with your code, you don’t need to worry about this. If you do, skip to the end for details on how to adjust your code to cope. But really, you should read it all for the sheer joy of learning about all sorts of awful syntax you didn’t even know existed before it went away. [Or at least empathize with us liberated SpiderMonkey hackers. ])

Properties in JavaScript and ECMAScript 3

The fundamental data structure in JavaScript is the object: a container mapping names to values through properties. You can add, remove, or change the value associated with any property, so long as the property may be modified. All user-defined properties are infinitely modifiable in any of these ways; only a few properties defined by ECMAScript (the standard on which JavaScript is based) are not fully modifiable.

var obj1 = {};
obj.property = 17; // add
var obj2 = { property: 42 }; // add from birth
obj2.property = 17; // change it
delete obj2.property; // remove it

Properties which store values are useful, but what if you want properties which can do things when you interact with them? What if you want to have properties which map strings to lazily-computed values? Or what if you want setting a property to have side effects (as, for example, setting an array’s length to 0 removes all elements in it)?

Properties with getters and setters in JavaScript

If you want properties which have functionality beyond just holding a value, you need getters and setters, stored within accessor properties. (Properties which hold values are called data properties.) JavaScript has long included extensions to ECMAScript to create accessor properties, both syntactic:

var o1 =
  {
    get property() { print("gotten!"); return "get"; },
    set property(v) { print("sotten!  " + v); }
  };
var v1 = o1.property; // prints "gotten!", v1 === "get"
o1.property = "new"; // prints "sotten!  new"

…and programmatic:

var o2 = {};
o2.__defineGetter__("property", function() { print("gotten!"); return "get"; });
o2.__defineSetter__("property", function(v) { print("sotten!  " + v); });
var v2 = o2.property; // prints "gotten!", v2 === "get"
o2.property = "new"; // prints "sotten!  new"

Getters and setters are now part of ES5. The syntax demonstrated above is valid ES5; a different API, Object.defineProperty, provides more flexible support for specifying getters and setters dynamically. Developers using the old-school APIs should begin updating to use the new API as browsers make new releases supporting it. Firefox in particular will include support for Object.defineProperty in its next major release, likely to occur in the latter half of the year.

Examining antediluvian accessor syntax

Unbeknownst to the vast majority of web developers, extension developers, and even Mozilla developers, in the past JavaScript has included other getter and setter syntaxes.

Named ES5-like getters and setters

If you look up the function that acts as the getter given ES5-standard getter syntax, what’s the name of that function?

var o = { get property() { return "get"; } };
print(Object.getOwnPropertyDescriptor(o, "property").get.name);

There are a couple plausible answers here: “anonymous“, “property” (the property name), or “” (the empty string) might be reasonable. JavaScript and ES5 arbitrarily create the getter or setter as a function whose name is the empty string. What if you wanted to name that function? (Bear with me for a moment and pretend this is a compelling need, and that adding a named getter or setter programmatically is absolutely unacceptable.)

Solely by accident of implementation, in the past SpiderMonkey has parsed the following syntax to assign names to getter and setter functions:

var o = { get property getter() { return "get"; } };
// Prints "Name: getter" in past versions of SpiderMonkey (or would if
// Object.getOwnPropertyDescriptor existed; __lookupGetter__ is a
// simple workaround); previous line is syntax error elsewhere
print("Name: " + Object.getOwnPropertyDescriptor(o, "property").get.name);

SpiderMonkey internally implemented the parsing of literal getters and setters by parsing them as though the start of a function expression had just been parsed:

// Faked-up parser state when parsing normal getters/setters
var o = { get property () { } };
                       ↑
function () { }
         ↑

Function expressions may be named or unnamed, but this wasn’t originally considered, so in the above example getter is treated as the name of the function created to correspond to the getter:

// Faked-up parser state when parsing named-getter-function syntax
var o = { get property getter() { } };
                       ↑
function getter() { }
         ↑

No other JS engine accepts this unintentional accessor-method name token.

Getters and setters in object literals

Possibly the best-known additional syntax is for specifying getters and setters in object literals. This syntax was the original Netscape invention for getters and setters; in practice it was superseded by the newer, more function-looking syntax. SpiderMonkey is again the only engine to implement it.

function g() { print("gotten!"); return "get"; }
var o1 =
  {
    property getter: g,
    property setter: function(v) { print("sotten!  " + v); }
  };
var v1 = o1.property; // prints "gotten!", v1 === "get"
o1.property = "new"; // prints "sotten!"

This accessor property syntax has one large advantage over the more-common syntax previously demonstrated (and even over the unintentional named-accessor mistake shown in the previous section). Where you see property in the object literal above, you could instead see a numeric literal, or a string literal — just as you might see either in any object literal without getters or setters, e.g. { 1: "value", "o": "hai" }. Historically, in get property() { ... }, property was required to be an identifier, thus excluding numbers and non-identifier accessor properties from representation. The syntax here had the further advantage of allowing serialization to “source” (more accurately, a reasonable but not always equivalent facsimile) of objects containing non-identifier-named accessor properties, through another Netscape extension in JavaScript.

This syntax also has a few disadvantages. Since the getter and setter contextual keywords follow the property name, the eye must scan past the property name to determine whether a portion of a literal represents a data property or an accessor property. This special-case check also complicates parsing, because now the parser has to check for something beyond just a colon at such locations. (To be sure, this problem exists with get foo() { }, but it’s restricted to the single leading token get, not to all leading tokens.) Since the value assigned to the getter is parsed as an arbitrary expression, there’s no guarantee the value must be a function — that must be checked at runtime.

Assigning getters and setters to properties

This accessor syntax provides the same functionality as Object.defineProperty(obj, propname, { get: fun, enumerable: true, configurable: true }) (mutatis mutandis for setters), except as part of the language syntax rather than as part of its standard library. Again, no other engine has implemented this syntax.

var o = {};
o.property getter = function() { print("gotten!"); return "get" };
o.property setter = function() { print("sotten!"); };
var v = o.property; // prints "gotten!", v === "get"
o.property = "new"; // prints "sotten!"

This syntax is also obscure: outside SpiderMonkey source and test files, only a single file in the Mozilla source code uses it. Strangely, a trawl through AMO shows half a dozen extensions have managed to discover this syntax, despite its near-complete disuse in Mozilla itself.

Assigning getters and setters to names rather than properties

Syntactically, this is just a different flavor of the previous example:

varname getter = function() { return "get"; };
var q = varname; // "get"

Semantically, however, it’s a rather different beast. The problem is that not all names are alike in SpiderMonkey. While ECMAScript specifies all name accesses in terms of objects (pure-JS objects in ES3, tighter spec-internal artifacts in ES5), most if not all JS engines out there optimize name access based on the type of the name. Local and enclosing variable access may be some number of pointer jumps, comparisons, and an offset, rather than some sort of hash table lookup in a more general case. Global variable access can in many circumstances skip lookups in enclosing scopes, going to the global object directly. (Last and certainly least, variable access inside with almost necessarily must be essentially unoptimized and dog-slow. Friends don’t let friends use with!) These sorts of optimizations rely on names always being plain old values, not accessors (except in the global case, where the type of optimizations implemented are qualitatively quite different). Slowing down local or enclosing variable accesses just to support this very rare case would be insane.

SpiderMonkey actually hasn’t supported this syntax for awhile. I only mention it because SpiderMonkey includes code specifically to exclude it. If this syntax is seen and varname can be resolved to a var, it’s a compile-time syntax error. Otherwise, if varname resolves to a var at runtime (possible in the presence of with or eval), it’s a runtime TypeError. Last, if it doesn’t, it “works” — and you are most likely Jesse, combining syntax and features in obscure and evil ways solely to make SpiderMonkey developers’ lives hard. In sufficiently old versions of Firefox where these restrictions weren’t in place, it’s entirely conceivable this syntax may have resulted in security vulnerabilities (one large factor in its removal from SpiderMonkey).

Prefixed function expressions

Perhaps the most bizarre getter/setter syntax is a modification of the syntax for function expressions and statements. As with all the others, this syntax has only been implemented by SpiderMonkey.

getter function foo() { return "foo getter"; };
var v = foo; // "foo getter"
var q = setter function bar(v) { };

When the prefixed function is a statement in the global scope, the syntax is equivalent to Object.defineProperty(globalObject, "foo", { get: function foo() { /* ... */ }, enumerable: true, configurable: true }) (mutatis mutandis for setter). If it’s a statement in a function scope or an expression that’s not a statement, the prefix serves no purpose that I can discern, except that it affects Function.prototype.toString()‘s behavior by including the prefix in the returned string.

None of these old getter/setter syntaxes provide value

Now that ES5 has codified The One True Syntax and The One True Programmatic API, these older syntaxes bring little to the table.

• The mistaken ES5-like named accessor get property funname() { } syntax doesn’t satisfy a compelling need.
• property getter: in object literals provides one compelling feature: the ability to have non-identifier-named properties. As ES5′s get property() { } syntax includes these further extensions beyond what engines have already implemented, this advantage no longer exists:

  var o =
    {
      get name() { return "names valid"; },
      set break() { this.x = "keywords too"; },
      set 1() { this.y = "numeric literals also accepted"; },
      get "custom string"() { return "arbitrary string literals too!"; }
    };
  

  (property getter: has a final advantage with respect to an ancient Netscape extension, but given that extension’s dubious future I will omit the details. Suffice it to say the use case is highly esoteric, and reasonably graceful degradation is possible without property getter:.)

• getter = and getter function are fully subsumed by Object.defineProperty.
• varname getter = was already gone.

In sum: these syntaxes make some things slightly easier, but they don’t provide anything you can’t do with ES5′s standardized accessor support.

These syntaxes were the source of numerous bugs

In addition to not being particularly useful, these syntaxes imposed notable costs on development. Supporting so many different getter and setter syntaxes isn’t easy, and the relevant code paths are quite complicated, attempting to decide when which syntax is correct and when not (particularly as far as object serialization is concerned). This has resulted in a multitude of accessor bugs usually found by Jesse‘s fuzz-testing and almost never by real-world scripts: bugs which, in C or C++, can often lead to memory-unsafety and, in the extreme, arbitrary code execution. By my count SpiderMonkey has sixteen separate tests (corresponding to the same number of bugs) dedicated to edge cases and corner behaviors with these syntaxes: syntaxes no one uses, syntaxes superseded by newer and better ones, and syntaxes which no other JS engine currently supports, nor ever will support.

These syntaxes continue to impose costs on development. Not all related bugs have been fixed, and changes to nearby code do have to take account of this syntax. We have had at least one long-standing (but believed “mostly harmless”, in that a sanity-check fails but surrounding defensive code completely contains the problem) bug involving this syntax, which due to its relative harmlessness has gone unfixed for nearly three years (and, almost as bad, undiscovered for two of them). Recent implementation work on ES5′s strict mode support required adjustments to the area of parsing object literals (for ES5′s strict mode rejection of duplicate property names), adjustments required to work around support for these syntaxes.

In short, TANSTAAFL. We’ve paid a large cost to keep these syntaxes around, and we continue to pay to keep them around — sometimes directly, sometimes indirectly, but unavoidably if support is worthwhile.

Support for all non-ES5 accessor syntaxes has been removed from SpiderMonkey

But for the many reasons previously given, support for these obsolete syntaxes is not worthwhile, so we have removed them from SpiderMonkey. get property funname() { } was an error from the start that no one will miss. SpiderMonkey has recently implemented support for ES5 numeric- and string-literal accessor property names (support for keyword names already exists), so the remaining important use case for property getter: has been eliminated. The getter = and getter function syntaxes never provided extra value, so they too have been removed without qualms.

To give an idea of the complexity eliminated by removing these syntaxes, the patch to remove them added 116 lines of code but removed 313 lines of code. Outside of code changes (that is, adjusting or removing tests which used these features), it added 133 lines but removed 1213 lines. It’s always great deleting code like this.

Updating existing code to adapt to these removals

One nice feature of removing syntax is that the failure mode when that syntax is encountered is blindingly obvious: the script will fail to parse. Parse errors show up in the JavaScript console, so it’s easy to tell when this is the problem; SpiderMonkey’s excellent error messages should point directly at the offending location.

If by chance you do actually use any of these syntaxes, the necessary fixes are simple. Suppose the existence of these helper functions:

function accessorDescriptor(field, fun)
{
  var desc = { enumerable: true, configurable: true };
  desc[field] = fun;
  return desc;
}

function defineGetter(obj, prop, get)
{
  if (Object.defineProperty)
    return Object.defineProperty(obj, prop, accessorDescriptor("get", get));
  if (Object.prototype.__defineGetter__)
    return obj.__defineGetter__(prop, get);

  throw new Error("browser does not support getters");
}

function defineSetter(obj, prop, set)
{
  if (Object.defineProperty)
    return Object.defineProperty(obj, prop, accessorDescriptor("set", set));
  if (Object.prototype.__defineSetter__)
    return obj.__defineSetter__(prop, set);

  throw new Error("browser does not support setters");
}

Here’s how you can update each old syntax to work again:

get property funname() { }

var o = defineGetter({}, "property", function funname() { });

property setter: fun

var o = defineSetter({}, "property", fun);

obj.prop getter = fun

defineGetter(obj, "prop", fun);

setter function prop() { } (when at global scope; otherwise just remove the setter prefix)

defineSetter(obj, "prop", fun);

You can experiment with a version of Firefox with support for these obsolete syntaxes removed by downloading a nightly from nightly.mozilla.org. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.)

A brief word on __defineGetter__ and __defineSetter__

As you may have noticed, all examples here use Object.defineProperty in preference to either __defineGetter__ or __defineSetter__, using the latter two only as fallback when the former is absent. While many browsers support these methods, not all do. Object.defineProperty is the future, and it is the standard; Microsoft has even gone on the record to say that they will not implement __defineGetter__ or __defineSetter__ in IE given the existence of the standardized method (props to them for that choice, by the way). For greatest forward compatibility with all browsers, you should use Object.defineProperty if it exists, and only fall back to __define{G,S}etter__ if it does not.

In a distant future we would like to remove support for __defineGetter__ and __defineSetter__, after ES5 adoption has taken off, so as not to distract from the standardized support. The less new web developers have to know about legacy extensions superseded by standardized alternatives, the better. This action is at least several years in the future, likely longer; being able to make the change will require preparation and adjustment in anticipation of that time. Given upcoming releases of browsers supporting ES5 functionality, there’s no better time than the present to start gradually, and gracefully, adopting standardized methods over legacy alternatives.

If comparefn is not undefined and is not a consistent comparison function for the elements of this array (see below), the behaviour of sort is implementation-defined.

[...]

A function comparefn is a consistent comparison function for a set of values S if all of the requirements below are met for all values a, b, and c (possibly the same value) in the set S: The notation a <_CF b means comparefn(a,b) < 0; a =_CF b means comparefn(a,b) = 0 (of either sign); and a >_CF b means comparefn(a,b) > 0.

• Calling comparefn(a,b) always returns the same value v when given a specific pair of values a and b as its two arguments. Furthermore, Type(v) is Number, and v is not NaN. Note that this implies that exactly one of a <_CF b, a =_CF b, and a >_CF b will be true for a given pair of a and b.
• Calling comparefn(a,b) does not modify the this object.
• a =_CF a (reflexivity)
• If a =_CF b, then b =_CF a (symmetry)
• If a =_CF b and b =_CF c, then a =_CF c (transitivity of =_CF)
• If a <_CF b and b <_CF c, then a <_CF c (transitivity of <_CF)
• If a >_CF b and b >_CF c, then a >_CF c (transitivity of >_CF)

Disjunction: is /foo/ the same object, or a new object, each time it’s evaluated in ES3?

According to ECMA-262 3rd edition, what should this code print?

function getRegEx() { return /regex/; }
print("getRegEx() === getRegEx(): " + (getRegEx() === getRegEx()));

The answer depends upon this question: when a JavaScript regular expression literal is evaluated, does it create a new RegExp object each time, or does it evaluate to the exact same RegExp object each time it’s evaluated? Let’s look at a few examples and make a guess.

I sense a pattern

var tests =
  [
   function getNull() { return null; },
   function getNumber() { return 1; },
   function getString() { return "a"; },
   function getBoolean() { return false; },
   function getObject() { return {}; },
   function getArray() { return []; },
   function getFunction() { return function f(){}; },
  ];

for (var i = 0, sz = tests.length; i < sz; i++)
{
  var t = tests[i];
  print(t.name + "() === " + t.name + "(): " + (t() === t()));
}

If you test that code, you’ll see that the first four results are true, and the rest are false, all per ECMA-262 3rd edition. (Okay, technically, and bizarrely, ES3 permitted either result for the function case, but no browser ever implemented a result of true; ES5 acknowledges reality and mandates that the result be false.) The first four functions return primitive values; the last three return objects. There’s only a single instance of any primitive value — or, alternately, you might say, equality doesn’t distinguish between different instances of the same primitive. Therefore it doesn’t really matter whether primitive literals evaluate to new instances or the same instance. On the other hand, objects compare equal only if they’re the same object. Since the object cases didn’t compare identically, they must be new objects each time. This makes sense: if this were not the case, what would happen in the following example?

function makePoint(x, y)
{
  var pt = {};
  pt.x = x;
  pt.y = y;
  return pt;
}

var pt1 = makePoint(1, 2);
var pt2 = makePoint(3, 4);

It would be complete nonsense if the object literal above evaluated to the same object every time it were encountered; the next two lines would blow away the previous point, and we would have pt1.x ===3 && pt1.y === 4.

Plausible assertion: regular expression literals evaluate to new objects when encountered?

Returning to the original question, then, what does ES3 say this code should print?

function getRegEx() { return /regex/; }
print("getRegEx() === getRegEx(): " + (getRegEx() === getRegEx()));

A regular expression is an object. If you don’t want to get weird property-poisoning of the sort just suggested, regular expression literals must evaluate to different objects each time they’re encountered, right?

Alternative: ES3 says /foo/ is the same object every time

Wrong. According to ES3, there’s only a single object for each regular expression literal that’s returned each time the literal is encountered:

A regular expression literal is an input element that is converted to a RegExp object (section 15.10) when it is scanned. The object is created before evaluation of the containing program or function begins. Evaluation of the literal produces a reference to that object; it does not create a new object.

This was originally a dubious optimization in the standard to avoid the “costly” creation of a regular expression object every time a literal would be encountered. It’s perhaps a little surprising that the same object is returned each time, but does it make a difference in real programs not written to demonstrate the quirk? Often it doesn’t matter. As a simple example, if (/^\d+$/.test(str)) { /* ... */ } executes identically either way, assuming RegExp.prototype.test is unmodified. The RegExp never escapes, and its use doesn’t depend on mutable state, so creating new objects each time doesn’t make a difference (other than negligibly, in speed).

Sometimes, however, the shared-object misoptimization does matter meaningfully: when a RegExp with mutable state is used in ways that depend on that state. Most regular expressions don’t store any state, so if the same RegExp object is used twice it’s no big deal. However, it can matter a lot for regular expressions specified with the global flag:

var s = "abcddeeefffffgggggggghhhhhhhhhhhhh";
function next(s)
{
  var r = /(.)\1*/g;
  r.exec(s);
  return r.lastIndex;
}

var r = [];
for (var i =0; i < 8; i++)
  r.push(next(s));
print(r.join(", "));

Each time a regular expression with the global flag is used, its lastIndex property is updated with the index of the location in the matched string where matching should resume when the regular expression is next used. Thus, in this example we have mutable state, and if next is called multiple times we have uses which will depend on that mutable state. Let’s see what happens in engines which implemented regular expression literals per ES3. If you download the Firefox 3.6 release candidate and test the above code in it (adjusting the implied print to alert), the printed result will be this:

1, 2, 3, 5, 8, 13, 21, 34

ES5: an escape to sanity

Is ES3′s behavior what you’d expect? No, it isn’t. In fact, ES3′s behavior, which Mozilla and SpiderMonkey implement, is the second-most duplicated bug filed against Mozilla’s JavaScript engine. SpiderMonkey and (strangely enough) v8 are the only notable JavaScript engines out there that implement ES3′s behavior. ES3′s behavior is rarely what web developers expect, and it doesn’t provide any real value, so ES5 is changing to the behavior you’d expect: evaluating a regular expression literal creates a new object every time.

Starting with Firefox 3.7, Firefox will implement what ES5 specifies. Download a Firefox nightly from nightly.mozilla.org and test it out as above (use the profile manager if you want to keep your current Firefox settings and install untouched). Instead of the Fibonacci sequence you’ll get this:

1, 1, 1, 1, 1, 1, 1, 1

The bottom line

Starting with Firefox 3.7, evaluating a regular expression literal like /foo/ will create a new RegExp object, just as evaluating {} or [] currently creates a new object or array. The optimization ES3 specified has resulted in clear developer confusion and was misguided and inconsistent with respect to other object literal syntax in JavaScript.

Again, as with my previous post, we doubt this change will affect many scripts (in this case, except for the better). The fact that few browsers implemented ES3′s semantics means that most sites have to cope with either choice of semantics, so the semantics in ES5, implemented by Mozilla for Firefox 3.7, are likely already handled. Still, it’s possible that this change might break some sites (particularly those which include browser-specific code), so we’re giving a heads-up as early as possible.