25.05.10

20.05.10

A brief note on web video formats, Mozilla, Theora, and H.264 in light of WebM

In a dispassionate comparison of the two major web video formats as of a few days ago, H.264 is basically stronger than Theora. H.264 has numerous and varied hardware decoders. Theora does not. H.264 has numerous software decoders. Theora has this too. Windows 7 and OS X natively support H.264. Neither natively supports Theora. Flash supports H.264. It does not support Theora. Numerous video cameras natively generate H.264. The same is not true of Theora. H.264 produces high quality for its compression. It’s disputable whether this is true of Theora. Licensing H.264 is burdensome but reasonably risk-free. Licensing Theora is painless but disputably risky.

Still, H.264 has weaknesses. H.264 is quite complex. Theora is relatively simple. H.264 is subject to actively enforced patents. Theora is, from all public appearances and from inferred verdicts of legal reviews by numerous organizations using or implementing Theora, subject to none. To use H.264, you must be sufficiently small as to not be worth the trouble of monetizing, you must find someone willing to acquire a license permitting them to extend to you the privileges you desire, or you must pay the piper his demand. To use, modify, or redistribute Theora requires no permission.

Network effects, unusually strong in technology, ceded the advantage to H.264. The smart money was always on H.264 enjoying continued success and Theora enjoying comparative irrelevance. But for Mozilla “logic” could not be dispositive. The loss of freedom of redistribution and grant of near-unlimited reuse and modification was a dealbreaker for H.264, absent a situation where refusing to deal with the devil (metaphorically speaking) caused greater harm than doing so.

Where faith in a fact can help create the fact, that would be an insane logic which should say that faith running ahead of scientific evidence is the ‘lowest kind of immorality’ into which a thinking being can fall. Yet such is the logic by which our scientific absolutists pretend to regulate our lives!

William James, The Will to Believe

Mozilla (and Opera, I should note) believed it was crucial for implementing, producing, and consuming video on the web to be legal and free to all without restriction. We had faith that it was possible to solve the problem of video on the web in the same way other problems at the foundation of the web were solved: through freely usable standards as happened with TCP/IP, HTTP, DNS, HTML, CSS, ECMAScript, PNG, SVG, JPEG, and many others. Absent this faith and a willingness to translate it into action by not implementing H.264, I doubt even a plausible free solution would have materialized. Google’s motivations surely are not exactly those of Mozilla or Opera, but I believe our actions strongly motivated Google to spend over $130 million to give away a video format.

Google’s release of VP8 and WebM cannot, I think, be attributed to logic alone; it must also be attributed to faith that freely usable video could be reality while evidence counseled otherwise.

18.05.10

Esoteric Supreme Court trivia question

Mystery images!

A couple months ago various travels took me within a mile or two of the location where an important Supreme Court case concerning freedom of speech originated. Naturally, I took the opportunity to do a little side trip to see the place and take a few pictures:

A shot with a building in the background and parked cars in foreground
First view...
Panned right, cars in the foreground with low building spanning width of picture in distance
Panning right a little...
Panned right further, building continues in background with parked cars in foreground
Panned right further

The question

Which Supreme Court case was this? (Or, where were these pictures taken?) I’ll give readers a little time before I make a new post giving the answer, unless someone knows or manages to guess the answer.

A few hints

The first hint is barely a hint if you know much about me, but it seems worth mentioning for the benefit of readers who don’t. The second hint may be quite helpful if you’re familiar with well-known First Amendment cases and their holdings. If you’re not, I doubt it will be of much use. The third arguably trivializes the problem of answering the question — but only if you choose to use the business names in the picture while trying to figure out the answer.

07.05.10

SpiderMonkey change du jour: the special __parent__ property has been removed

tl;dr

The special __parent__ property has been removed from SpiderMonkey and is no longer in nightly builds. If you don’t use __parent__ or don’t know what the property does, you didn’t miss much.

If you use __parent__, you have a couple replacements. If you were using it to determine the global object for another object, use Components.utils.getGlobalForObject instead. If you were using it only to test its value against an expected value, use nsIDOMWindowUtils.getParent instead (but do note that its semantics are not absolutely identical to those of __parent__). If you were using it from unprivileged web scripts as a potential vector for security exploits, I feel your pain and will take no steps to assuage it. If you were using it some other way, comment and we’ll figure something out for your use case.

If you think you understood the __parent__ property (you probably don’t), or if you’re interested in the nitty-gritty details of JavaScript semantics, read on for the details of this esoteric property and the reasons for its removal.

Scoping in JavaScript

In the following example code, when the function g is invoked, what stores the v variable?

function f(a)
{
  var v = a;
  function g() { return v; }
  return g;
}

var fun = f(2);
fun();

The variable is accessed in an enclosing scope, created when the enclosing function was invoked. In the ECMAScript standard, the location of such variables is an object, stored as an internal [[Scope]] property of the function being called. In ES3 this object was a standard JavaScript object; ES5 tightens semantics slightly and uses a simpler structure, but the idea’s the same.

Meet the __parent__

It’s not possible to access the object stored as [[Scope]] in ECMAScript proper, but it has been possible to access it in SpiderMonkey and Mozilla-based browsers. The magical __parent__ property can often provide access to this value:

this.toString = function() { return "global"; }
var q = 17;
function foo() { return q; }
print(foo.__parent__); // prints "global"

“Often”? When does __parent__ not expose this value?

__parent__ doesn’t always reflect [[Scope]] because ECMAScript requires that certain objects not be made available to scripts. Among such objects are what SpiderMonkey refers to as With objects, Call objects, and Block objects.

With objects

SpiderMonkey creates With objects to handle the esoteric, non-lexical name lookup required by the semantics of the with statement in JavaScript. These semantics require that a name, depending on the runtime object used in the with, refer to a property of that object or to a variable in an enclosing scope. It’s impossible to know in general which will be the case before runtime, and therefore it’s impossible to speed up a lot of script that runs inside a with. (Incidentally, this is why you should never ever ever [ever] use with in performance-critical code. Use a two-letter abbreviation variable to save typing, if verbosity is your concern.) We can’t simply use the with block’s object as the scope, because if we miss on a lookup there we want to fall back to the normal scope; therefore we introduce a With object. Here’s an example of a situation where a With object is created:

with ({ toString: function() { return "with object"; } })
{
  print(function() { return toString; }.__parent__); // prints "with object"
}

__parent__ doesn’t actually give you the With object, because it’s a carefully-tuned internal value. With objects have behavior and functionality optimized for their particular purpose, and if we simply exposed them to scripts it would probably be possible to do Very Bad Things. Consequently, if you try to access __parent__ in a situation where you “should” get a With object, we instead give you back the object where the With object begins its search: the object used for the with block. This may be what a developer superficially familiar with __parent__ might expect, but it is nevertheless a lie.

Call objects

SpiderMonkey’s Call objects represent the local variables of a function call. Therefore, Call objects correspond to the [[Scope]] of functions. Returning to the original example, reproduced below, v is stored in a Call object, which is the [[Scope]] of the function g:

function f(a)
{
  var v = a;
  function g() { return v; }
  return g;
}

var fun = f(2);
print(fun.__parent__);

When you attempt to access g‘s __parent__ property, SpiderMonkey censors the [[Scope]] and returns null, because ECMAScript requires that Call objects not be exposed. This case, which is far more common than the with case, completely prevents access to [[Scope]] at all (rather than exposing a half-representation of the value). This situation is even more pervasive in light of the modern JavaScript encapsulation practice of enclosing libraries in closures for pseudo-namespacing purposes. In many libraries __parent__ on many functions of interest gives no value whatsoever.

Block objects

One of the first things every JavaScript developer learns is that JavaScript variables are not scoped to blocks. Language tutorials usually emphasize this point, because it differs from most other languages (and in particular, it differs from the C-ish family of languages, the syntax of which JavaScript borrows to a fair degree). The conventional wisdom is that names in functions are always scoped to enclosing functions or to the global scope. If you want a locally-defined name, you have to wrap it up in a function.

The conventional wisdom is wrong.

Here’s proof: what does this example print?

var v = "global";
function test(frob)
{
  try
  {
    if (frob === 0)
      return v + " try";
    if (frob === 1)
      throw "local";
  }
  catch (v)
  {
    return v + " catch";
  }
  finally
  {
    if (frob === 2)
      return v + " finally";
  }
  return v + " after finally";
}

print("not throwing, in try:          " + test(0));
print("throwing, in catch:            " + test(1));
print("not throwing, in finally:      " + test(2));
print("not throwing, after finally:   " + test(3));

Behavior depends on whether the binding for v as introduced by the catch is scoped to the function or to something else. JavaScript specifies that v, referring to the value potentially thrown while executing the try block, is scoped only to the catch block. Thus v in the catch refers to the thrown value; all other uses of v are outside the catch block and refer to the global variable v. Therefore the output is this:

not throwing, in try:          global try
throwing, in catch:            local catch
not throwing, in finally:      global finally
not throwing, after finally:   global after finally

Any function which referred to v inside the test function would have a Call object as its [[Scope]] — except for a function defined inside the catch block. Such a function at that location would have to capture the thrown value, so there must be something else on the scope chain before the Call object. SpiderMonkey refers to these objects as Block objects, because they implement traditional block-level scoping.

Block objects have the same issues as Call objects, so SpiderMonkey censors them to null as well. For optimization purposes we’d like to “boil them away” whenever possible, so as not to create an obnoxious little one-property object when we catch an exception and define a function that captures it. Exposing such an object directly prevents these optimizations, and would likely expose some security vulnerabilities.

But __parent__‘s issues don’t stop merely at those induced by complex language semantics. Even perfectly simple code — simpler even than nested functions — has potential for unpredictability.

If certain optimizations are implemented, __parent__ may be a lie

Consider this script in the global scope:

function fun() { return 17; }

print(fun.__parent__);

Does fun‘s [[Scope]] need to be the global object?

It’s OK to cheat if you never get caught.

Smalltalk implementer maxim

ECMAScript provides no way for script to access [[Scope]]. It’s a construct used to ease specification, and it need not even exist in implementations. fun never uses any values from its enclosing scope, so there’s no reason that [[Scope]] must be global object. An implementation that carried around [[Scope]] with each function might simply make [[Scope]] be null, which could potentially speed up some garbage collection algorithms. Similarly, a function which refers only to never-modified variables in enclosing scopes might simply copy those variables’ values and, again, make [[Scope]] lie. Should a specification-artifact internal property constrain implementations looking to improve performance, or code size, or any number of other measures of desirability? Given where JavaScript use is heading now, the answer must be no.

For certain types of functions, there are very good reasons, even in simple cases, to make [[Scope]] a lie. SpiderMonkey implements some optimizations along these lines, but those optimizations don’t affect the value we store for [[Scope]] at the present. If it were advantageous to change that, we would do so in a heartbeat. Therefore, while __parent__ may have a reliable value now, it’s possible it would not in the future.

__parent__ is available even where you think it isn’t

Oddly enough, __parent__, despite its representing [[Scope]], isn’t just applicable to function objects. Instead, its value is generalized to all objects, in a way even more difficult to describe than that above. (Very roughly, __parent__ on non-function objects corresponds to the object which would be the __parent__ of a function created in the same context. I think. Mostly the value is used for security checks; improper setting of __parent__ is a common cause of implementation-caused XSS vulnerabilities.) This generalization and its semantics have even less support in the specification, so its meaning is even less clear than for functions.

__parent__ doesn’t expose anything useful

__parent__, beyond having unintuitive behavior, doesn’t tell the developer anything he’d care to know. Why would you need to access the enclosing scope of a function, as an object, in the first place? The very definition of the problem is obscure; it is almost a prerequisite to have read the ECMA-262 specification to be able to ask the question.

In the searching of various codebases that we’ve done, we’ve found only these use cases for __parent__:

Testing
A small number of Mozilla tests use __parent__ to verify proper scoping of objects. Since the value as we use it has security implications, this use case is reasonable — but it has extremely limited applicability. This use case can be supported by creating an alternate means of accessing an object’s parent, a means exposed through XPCOM, not through JavaScript itself, and certainly not by polluting every object with the functionality. We have implemented nsIDOMWindowUtils.getParent(obj) to support this testing-oriented use case. Beware: this method doesn’t censor like __parent__ does! Tread cautiously when using it where a With, Call, or Block object might be exposed, and don’t use the returned object except in immediate strict equality (===) checks. (NB: we’ve implemented it this way out of expediency; if it turns out to be too great a responsibility for typical testers, we’ll likely change it to censor.)
Determining the global context (global object) where an object was created
Since __parent__ often (for any function not nested in another function) is simply the global object, some people have used __parent__ to retrieve the global object associated with an arbitrary object. By walking from __parent__ to __parent__ you can often reach the global object this way. However, since the __parent__ value is sometimes a lie, it’s not as simple as var global = obj; while (global.__parent__) global = global.__parent__;. Instead, you have to be careful to start from an object with a known __parent__, one guaranteed not to be a nested function or similar. The easiest way to do this is to first walk up the prototype chain to eventually bottom out at Object.prototype, then to walk the parent chain. This method is baroque and non-obvious, and it requires careful tiptoeing around incorrect __parent__ values. If you know the object whose global you want to retrieve is a DOM node, the fully-standard node.ownerDocument.defaultView property also provides global access. Otherwise, the use case would be better addressed by exposing a method somewhere to directly retrieve the corresponding global object. We have recently added such a method to support this use case: Components.utils.getGlobalForObject(obj).
Being evil to Firefox
A fair number of security bugs reported against SpiderMonkey in recent years have worked by successfully confusing the engine into assigning a bad parent to an object. This incorrect security context can then be used as a vector to run code in the context of another website (resulting in an XSS hole, possibly usable against any website), or in some cases, in the context of the browser “chrome” itself (providing the capability for arbitrary code execution — the worst-case scenario as far as exploits are concerned). __parent__ is immutable and can’t itself be used to confuse the engine (at least in the absence of bugs, and we have no way to demonstrate such). However, it provides greater visibility into the engine, and it can sometimes expose values to script which wouldn’t (and shouldn’t) be accessible any other way.
Being evil to sandboxing mechanisms
__parent__ exposes security holes in some websites as well as in the browser directly. Such websites are those which purport to “safely” execute scripts provided by other users, through sandboxing or other mechanisms. For example, Facebook uses FBJS, a script-rewriting plus runtime-check mechanism, to allow applications to include interactivity. Google‘s Caja system provides a rigorous capabilities-providing system (also through script-rewriting plus runtime checks) to do the same, and it too is used in public sites these days. One requirement of such systems is that the global object must never be directly exposed: if you have the global object, you have unfettered access to the document, you have eval, you have Function, and you are utterly and thoroughly hosed. __parent__ provides easy access to this, so all these systems must censor access to __parent__, both statically (obj.__parent__) and dynamically (var p = "__parent__"; obj[p]). Not everyone knows, or remembers, that this is necessary. Removing __parent__ reduces attack surface in JavaScript sandboxes.

The former two use cases are better addressed in other ways. The latter two are holes best removed entirely.

__parent__ is being removed

__parent__ is being removed from SpiderMonkey. The value it purports to expose is esoteric and has semantics defined by a specification which JavaScript developers shouldn’t really have to read to understand it. The identification of that value is non-intuitive. Sometimes the value it exposes is a lie. There are plausible reasons why it might be made to lie if potential performance-improving ideas were implemented. It’s exposed even in places where it makes little sense. The use cases for it can be and are better served in other ways — or explicitly not served, when it serves as a vector for security vulnerabilities. In sum __parent__ doesn’t pass muster, so it’s being removed.

You can experiment with a version of Firefox without support for __parent__ by downloading a nightly from nightly.mozilla.org. (Don’t forget to use the profile manager if you want to keep the settings you use with your primary Firefox installation pristine.) The next release is many months away, which should provide plenty of time for extension developers to update their extensions to not use __parent__, if by chance they had managed to discover it and use it.

06.05.10

Dear Lawzyweb

(Joining the recent stream of lazyweb posts on p.m.o…)

If you’ve paid even the slightest attention to tech news, you know Apple lost an iPhone prototype in a bar in the Bay Area. The finder sold it to Gizmodo for $5000, and Jason Chen of Gizmodo published a story with photos and details of it (and numerous followups) — a juicy tech story. More recently, San Mateo police, pursuant to a warrant, searched Jason Chen’s house, seizing numerous pieces of technology hardware. It thus becomes a juicy law story: trade secrets, protection of journalists’ sources, freedom of speech and the First Amendment, handling of lost or stolen property, lots of possible angles. In a number of them it approaches the clearly-defined boundaries of state and federal laws. Great popcorn fodder all around.

There are enough legal questions to satisfy anyone looking to argue them. There are correct answers and incorrect answers, but for a legal novice like me for whom the unknown unknowns are considerable, it’s far more productive to read others’ arguments than to hazard speculation. Also, some parts are matters of fact potentially for a jury to decide, further imperiling predictions.

Every so often, however, it’s possible to pass into realms where my knowledge is less patchy. One commentator, Peter Scheer of the First Amendment Coalition, thinks the police should have obtained a subpoena rather than a warrant, thereby according a journalist what one might claim is his due “delicacy”. Scheer closes an argument for this course of action by speculating as to why it was not taken:

Perhaps there is a more mundane explanation for the failure to use a subpoena in this case: The DA [district attorney] may have been under intense pressure (from whom? Apple, which reported the phone was stolen?) to act even before he could convene a grand jury to issue a subpoena.

If so, the DA may come to regret his haste: If a court rules he shouldn’t have used a warrant, the DA’s possession of evidence seized from Chen’s home may undermine any possible prosecution of other, more culpable, parties.

Assume arguendo that a court does indeed at some point rule the DA shouldn’t have used a warrant. Scheer then claims the seized evidence “may undermine any possible prosecution” of other parties (most likely referring to the original finder, as there is some question of whether the finder actually made a good-faith effort to return the iPhone prototype to its owner, potentially falling afoul of California law). Is this correct? The exclusionary rule forbids admissibility of evidence gained through unreasonable search or seizure in court, following straightforwardly from Weeks v. United States, 232 U. S. 383 (1914), and the Fourth Amendment. The exclusionary rule is then applicable to the states (and to local government such as San Mateo County) under Mapp v. Ohio, 367 U. S. 643 (1961). If case law stopped here it seems to me Scheer would be right — but it doesn’t. Prior to Mapp the Supreme Court held that:

In order to qualify as a “person aggrieved by an unlawful search and seizure,” [for whom evidence from an illegal search or seizure could be suppressed] one must have been a victim of a search or seizure, one against whom the search was directed, as distinguished from one who claims prejudice only through the use of evidence gathered as a consequence of a search or seizure directed at someone else.

Frankfurter, J. in Jones v. United States, 362 U. S. 257, 261 (1960)

It seems to me that, were the warrant declared invalid, evidence from the search would be suppressed in any potential prosecution of Jason Chen (and maybe Gizmodo — but in Alderman v. United States, 394 U. S. 165 (1968), the Court explicitly declined to apply the exclusionary rule with respect to evidence gained through illegal search of a “coconspirator”; Gizmodo or its other employees might or might not be such, maybe depending on whom a case targeted). However, I don’t see how evidence would be suppressed in the prosecution of anyone else — most particularly of the finder of the prototype.

The question for the la[w]zyweb: would evidence from Jason Chen’s computers, pursuant to an illegal search and seizure, be admissible in court against the original finder of the iPhone prototype? I think it would be admissible, and I think Peter Scheer is mistaken if he is suggesting that it wouldn’t.

Speculation’s fine, but as I already provide the less-educated kind I’d prefer if comments consisted of the more-educated kind. :-)

« NewerOlder »