Another nasty bug — and an idea

I spent about two hours tracking down a DOM node sharing bug — nodes were being put into a new structure outside of the document before the salient data had been read out. While there was no information in these nodes, the lens system insisted that they still be there. (More on that — eventually.)

After finally tracking it down and writing a version of cloneNode that also copies event handlers, everything worked. Between this and the last prototype aliasing bug I had, I got an idea. A programmer could keep a “bug journal”, a list of bugs found and described first by their behavior, then by their solution (and, if those two aren’t descriptive enough, the underlying problem should be described as well). For example, two days ago I ran into my first genuine typing bug in JavaScript — a type checker would have rejected my program, and from the errors generated it wasn’t obvious where the problem was.

This practice could be useful in a few ways. First, the process of writing down the description can help the programmer find the solution. Tedious, but perhaps worthwhile. Surely some bugs would end up being described post facto, since it’s not worth the time when the fix is fairly clear. Second, the solution may add to a ‘bag of tricks’ at the programmer’s/team’s disposal. Third, the bug and solution tease out invariants in the program, and so the bug journal could be gleaned for inter-module and system-level documentation.

Fourth, and dearest to my heart, I think it’s an interesting way to evaluate programming languages. The bug log of a programmer writing an e-mail interface in Java and that of one writing such an interface in JavaScript would provide for some interesting contrasts. To provide more than anecdotal evidence, you’d need to use a much larger sample size of programmers and kinds of programs being written.

I think the bug journal would differ profoundly from examination of bug tracking logs. Only the truly mysterious bugs and the large, architectural shortcomings make it into the tracker. On a daily basis, programmers struggle with making buggy code workable — before it ever hits version control. So bug tracking logs can highlight difficulties in design and with the team, but a bug journal shows exactly what a programmer has to deal with.

More on this

So I showed my confusing problem detailed in this last post to Dave Herman, who after an initial surprise, said that this was probably due to ES4 expansions method binding — that is, o.f sometimes closes over this, but sometimes not.

It doesn’t work in Opera or IE — they return false for both function calls — but it seems that if they implement ES4, the first call will eventually return true. That seems terrible to me — it was so surprising! It also makes it a little difficult to use JavaScript itself as a compiler representation, since you can’t use A-normal form if let-abstraction doesn’t work. Never mind that programmers (read: I) use let-abstraction to break up complicated expressions when debugging, and so this change in behavior will only confuse matters more.

Dave pointed out that in trade-offs between consistency and convenience, the latter sometimes wins, particularly when changes affect thousands and millions of people. But it’s not clear to me how convenient this is; it’s a tiny shortcut for those who know about it, but it’s very fragile. I’d liken it to operator precedences: in only a few cases do people take advantage of the ordering, so arithmetic expressions are generally just written out with parentheses for clarity.

Capture this!

JavaScript woes:

function C() {
    var _this = this;
    this.foo = function () { return _this === this; }; 
    this.bar = function () { return this.foo; };
    return this;
}
var c = new C();

What is the value of c.foo()? What is the value of c.bar()()? Both are true. But let:

var foo = c.bar()

What is the value of foo()?

If you thought it was true, you’re wrong. This doesn’t make much sense at all. Either this is captured by a dot, or it isn’t. Looking at the inner call to c.bar, it’s syntactically obvious that the call to bar has this = c. But why is that preserved when evaluating it within an outer expression, but not when we split the expression into two statements?

I’m sure this has been seen before, but I’ll probably show it to Dave Herman anyway. If I remember correctly, Python gets this right. Between this and a prototype aliasing bug I had yesterday, I’m irritated.

As an aside, I was speaking with Arjun Guha last night, and none of the conventional type theory would have helped me with these problems. In fact, I get almost no type errors in JavaScript — only grotesque object model problems.

XSugar

I was referred to XSugar as another practical example of bidirectional programming; I read the paper, Dual Syntax for XML Languages and played with the tool (a bit).

The basic idea is pretty clever. You specify a single grammar, showing at the nonterminal level how the XML syntax relates to the non-XML syntax. For example:

person : [Name name] __ "(" [Email email] ")" _ [Id id] = 
         
              [Name name]  
              [Email email]  
         

Relates the line Michael Greenberg (email hidden; JavaScript is required) 00000000 to the XML <student sid="00000000"> <name>Michael Greenberg</name> <email>email hidden; JavaScript is required</email> </student>. (Note that Id, Name, and Email are all regular expressions defined at the top, lex-style.) Since there’s a relation between the textual format and the XML format at the grammar level, the transformation in either direction is achieved by a transformation of the parse tree generated — they call (a derivation of) it the UST, or unifying syntax tree. This parse tree is first translated to the more abstract UST, which is then “unparsed” into the other format.

To guarantee a unique unparsing, they perform ambiguity checks on the grammar. The perform can perform an additional check against an XML Schema (or DTD, or whatever) to ensure that all syntactic non-XML generates validating XML. They do this by extracting an “XML graph” — a flowchart for the tree structure — from the XSugar specification, which can then be checked against an XML Schema. This check boils down to “encoding content models and schema definitions as finite string automata over a common vocabulary and checking inclusion of their languages” (see XML Graphs in Program Analysis, which seems a little light on detail).

The work is good — it solves a problem, and simply so. (Much more simply than in the Kawanaka and Hosoya paper, biXid: a bidirectional transformation language for XML. They have slightly looser ambiguity requirements, but with a weird effect.) The error messages they give are okay, but they don’t seem so marvelous that they merit inclusion in the paper. Writing wise, the description of the XML graph could be clearer, and their error messages aren’t entirely clear to me. But it’s hard to criticize their result — 2372 lines of ad hoc Python to 119 lines of XSugar!

I was disappointed by only one thing. I wondered whether I might use XSugar for automatic conversion between SXML and XML; unfortunately, what I came up with was:

WS = \r|\n|\t|" "
AT = "@"
TOP = "*TOP*"
LP = "("
RP = ")"

node : LP [Element e] 
	LP AT [attriblist as] RP
	[nodelist ns] RP
	= <[Element e] [attriblist as]>[nodelist ns] ;

attrib : LP [Attribute a] [Text v] RP =

But then I was stuck — I realized that one can’t abstract over attributes. It doesn’t seem like this sort of abstraction is impossible, but the feature seems to be missing. It would require another style of syntax to make it clear which context nonterminals were in — attribute or node. Or I may just be missing something — the journal paper is the only documentation, short of JavaDoc.

CLR Hegemony

A well-shaven Jim Miller gave a talk at Brown today on introducing dynamic languages into the CLR. I went because (a) they were giving away a free camera (which I didn’t get), and (b) the talk seemed actually interesting. Much more interesting than my last Microsoft talk, which was some interminably boring thing about webservices.

The talk was essentially an overview of the CLR, of the form, “Do you think this is cool? Work for us!” The focus was on IronPython‘s genesis and growth. Those kids love their Python.

I tried to ask him the following question: “You’ve talked a lot about getting basic interoperability between programs written in different languages — object-oriented, procedural, functional — but you’ve said nothing about heterogeneity of styles. How does the CLR cope with that?” I asked because he said that an interface of an object included a way to create it, modify its insides, and so on — not necessarily how I define my datatypes. But then he turned it around, and asked me for an example — the best I could come up with was Erlang’s processes and single-assignment (a red herring) or Haskell’s purity (an academic joke).

Not what I was looking for. The mismatch is more serious, but I couldn’t adequately get at it on the spot. I asked him a follow-up, about GUI programming style. Would a GUI in IronPython need to be written in some bogus Windows Forms style, all callbacks? I didn’t want to compare the callback-GUI system to Flapjax‘s, since that’d be a bit wankerish of me. I didn’t handle it well at all.

So: what did I mean to ask? On the one hand, my problem is this: the mismatch between the CLR and some paradigms is pretty clear. If I wanted to work with Windows Forms in, say, F#, I would have to write in an all-ref style. In other words, the CLR lets me program C# in the language of my choice.

On the other hand, how else would I write a GUI? Flapjax is the most viable FRP implementation I’ve seen, but it’s still extremely nascent. When interfacing with the user, what is better than direct, mutative style? Python uses callback-based Tkinter and WxPython; DrScheme uses the MrEd callback-based framework. So what gives?

I may be more comfortable in the FP world, and happier programming Scheme in Scheme, rather than C# in Scheme, or even worse, Scheme in C# — but it’s becoming increasingly clear to me that my preference may be more related to my own experience than actual advantages.

Google E-Mail Masking

As an alternative to the Enkoder, Google Groups turns e-mail addresses in headers and in document text into links to CAPTCHAs, viz. the documentation. They’re not using the hardest distortions I’ve seen, but there seems to be some good contact between the letters.

I wonder what the space looks like between the fully automatic but very fragile Enkoder system and the many-click and -keypress CAPTCHA system. A system using Javascript combined with an easy Turing test would give the best of both worlds. With the CAPTCHA Turing test reduced to, say, a single click, the self-evaluating Javascript could combine that weaker test with a computational payment.

So I see two possible directions for the Enkoder system. One is a user-initiated computational payment: a “decode data” link, when clicked, changes to show “loading…” until the computation produces whatever was enkoded — an e-mail link, data, whatever. The other is a user-initiated puzzle: click the e-mail, a popup div or window presents an intuitive puzzle of moderate difficulty that is generated via the Enkoder-self-evaluating Javascript method; solving it correctly should then deliver the data.

The first is just an engineering issue. First I’d need to collect data on how long dekoding takes on different platforms, and to try to maximize my control while preventing the explosion of the enkoded data in size. (Modular multiplication? I knew MA158 Cryptography would be good for something!) The second direction needs some serious thought about puzzle design. It’s important to be acultural and not too intellectually challenging, but still hard for a machine to figure out. Secondly, it’s not clear how to prevent replay attacks if the Turing test is all client-side, so it has to be difficult to brute-force.