Determinism and market madness

The past few months of misery in the stock market have been so obviously driven by fundamentals that there’s something a little bit lightheaded and silly about saying a rebound is on the way because of technical factors. Consider the following chart (click to enlarge), which doesn’t even take into account the political and military complexities in North Africa:

Yes, I know, technical analysis is supposed to (depending whose explanation you read about why it works) operate independent of events external to the price of the stock, or perhaps to incorporate those factors in such a way that there’s no need to look at anything but the price; but there are philosophical holes in that explanation the size and stickiness of the La Brea Tar Pits. Leaving aside whether those holes can be patched (and, also, my apologies for the layers of mixed metaphor I seem to be building up here like a stale rhetorical napoleon), it’d be nice if we could look at this market death spiral we’re in and see it as something regular and predictable (and finite!). People operate in the everyday world by matching patterns and proceeding as if those patterns will continue, what Sherlock Holmes always called “deductive logic” but is really inductive; I don’t think they do it by following anything remotely like the rules of formal logic (I think modeling the brain as a computer running a program puts the cart pretty far out in front of the horse), but they do do it in a fairly consistent way. By making that distinction, though, I'm at odds with a lot of people in cognitive science, philosophy, and artificial intelligence.

Three hundred years ago, around the same time that Adam Smith was laying the ground work for those researchers by asserting that we were all members of the supremely logical species Homo economicus, his empiricist contemporaries elsewhere in Western Europe were coming out pretty strong in favor of a pure determinism: the world as giant clock. It’s a seductive way to think, and I admit to having the following quote from the Marquis de Laplace on my fridge:

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at any given moment knew all of the forces that animate nature and the mutual positions of the beings that compose it, if this intellect were vast enough to submit the data to analysis, could condense into a single formula the movement of the greatest bodies of the universe and that of the lightest atom; for such an intellect nothing could be uncertain and the future just like the past would be present before its eyes.

The thing about Laplace’s brand of determinism is that it’s compelling on a really fundamental level because it’s how we live our lives day-to-day; if it weren’t so compelling, and weren’t so familiar, then why would the head-scratching, insomnia-inducing philosophical discontinuity between it and the concept of free will even be worth talking about? The steady encroachment, warranted or not, by scientists onto the traditional grounds of metaphysics has muddied the popular conception of what the free-will debate is about, by introducing the idea of quantum particles with unknowable and/or unpredictable states. But, woo-woo stuff like that aside, where determinism impacts our lives isn’t in the question of whether you or I, as conscious beings riding around in our heads and looking out of our eye sockets, act predictably to satisfy our needs, but whether all those other eye-socketed things around us act in a way that we can predict well enough to function. And it seems that they do. (This is a finer-grained version of the truism that we all, like it or not, are in practice practicing Rousseauians: without the Social Contract, how would you ever make it through five minutes on the freeway alive?)

In 1980 I was an avid reader of Creative Computing magazine, which, like other great magazines of the early home-computer culture of the late 1970s and early ’80s, published in each issue complete programs (not just illustrative snippets of code) that you could type into your computer, save, and run. Most were games, but many were serious applications, such as primitive databases; Compute! even published a full-featured word processor. Believe it or not, this was until 1990 or so (I think PC World was the last holdout, or maybe one of the Atari- or Amiga-specific magazines) one of the chief ways that non-commercial software was distributed (the other being BBSs). So, late in 1980 Creative Computing printed a program, with accompanying explanatory article, that was supposed to predict the outcome of that year's Presidential election. As a kid I thought this was magic. Via simulation games that had you playing a bullfighter or king of Sumer, I had gained an intuitive grasp of the way that software could model a small part of the real world; but I couldn't make the leap of generalization from those toy models to one that would, accurately enough to be interesting, simulate the behavior of 100 million adults. (Never mind that like most people under the age of 25 or so, I found the behavior of even a single adult totally incomprehensible, period.)

I'm reconstructing this from memory, of course, and I'd love to see a copy of the magazine again, or have someone email me saying that they wrote that program. I imagine that the algorithms used were predicated on some sort of deterministic optimism about voter behavior in the 1980 race, i.e., an assumption that similar factors were present as in prior elections. (Or were there? Carter’s single term is now remembered mostly for an unusually heated level of strife in the Middle East and an economy so screwed-up that a relatively new term of art from economics became common currency just so people could grasp what was happening.) Probably there was also a dash of statistics added in, since if I remember right the program was supposed to narrow in on its best guess as you entered numbers from the earlier-reporting states, just like the TV networks do. (“As X goes, so goes the nation.”) It’s this latter algorithm, or group of algorithms, that most intrigues me. It’s pattern matching of a sort; and pattern matching, no more and no less, is what technical analysis of capital markets is all about. When we see that A usually precedes B, after a while we begin to expect B when we see A; if we get what we expect, our predictive model is strengthened. It’s a little bit circular, but so is all inductive reasoning.

That circularity becomes a problem only when we mistake an A-prime, or a C, or even a Z, for an A, and have become complacent enough that we mistake the B-prime or D or unrecognizable squiggle that follows for a B—and then act as if a B is guaranteed. On that path lies madness, and something worse than madness: losing money.

Applications: Tools, or Just Fancy Data?

[Originally published April 6, 2007.]

In my previous post in this blog I challenged the feasibility, from the interface-design point of view, of running applications in a browser window—on the grounds that applications and data are two different things, and the browser is inherently a tool for viewing data.

Let me add a couple of thoughts to this. First, a general point: it’s important to note that this isn’t just a linguistic or even an epistemological issue, but an ontological one. That is, it’s not just a matter of what kinds of arrangements of bits we call “data” versus “applications” and what kind of tool we use to manipulate them. It’s a question of what that tool is, what it does, to what, and for whom. Think about physical tools, the kind you buy at a hardware store.¹ They’re classified according to a huge variety of schemata:

Some are classified according to the raw material on which they are designed to operate: a crosscut saw for wood, versus a hacksaw for metal.

Some, according to the physical shape of the artifact they manipulate: an Allen wrench for bolts or screws with a concave hexagonal impression in the head, a Robertson driver for those with a similar but square impression.

Some, according to the purpose of the artifact they manipulate: a flare wrench for fittings on hydraulic lines.

Some, according to the operation to be performed, largely independent of the context of the object of the operation: a screw extractor for rotating fasteners whose heads are damaged.

And often there is an overlapping schema wherein tools are classified according to the general circumstances in which you would use them, hierarchically, with groups and subgroups: there are mechanics’ tools, and there are metric tools, and then there are wrenches with built-in tubing, for opening hydraulic bleed screws without a mess in brake or clutch systems with metric fasteners.²

The point is that these classification schemes aren’t something imposed from outside, as biologists impose the Linnaean taxonomy on the ever-changing and ever-being-discovered sloppiness of the natural world in order to make it a little bit more manageable. The epistemology of a tool guides its ontology: that brake-bleeding wrench was designed specifically for the task, very likely by some mechanic fed up with the inadequate tools he or she had available to do a brake job, and a crosscut saw acquired the form it has not by chance but because generations of woodworkers refined the design to cut certain pieces of lumber in a way that was useful to them. So tools evolve not only with their objects but also with the circumstances of their use, and it’s an oversimplification to say that there is a straightforward correspondence between the tool and its object and thus a clear-cut division of tools by what object they act upon. This isn’t an excuse for the browser as application platform as currently understood, though. Exactly the opposite! The tool that is used to run online applications and explore online databases must be one that is tailored to its job, rather than the clumsy square-peg-in-a-round-hole hack of the browser-as-it-stands.

And this brings us to the other point I want to add to the previous entry. Douglas Hofstadter said of the supposed form/content distinction³ that “content is just fancy form.”⁴ Are applications, then, just fancy data? It’s tempting to state the question and its counterpoint as opposing theses:

• [First,] Applications are just fancy data: the more complex a data set becomes, the more operations its inherent properties suggest, until some “tipping point” of complexity is reached, at which those operations can be abstracted from that data set and ones with similar structure. [vs.]
• [Second,] Applications are closely analogous to tools; data, to raw materials and the workpieces made from them: though they may evolve together, the two are fundamentally different.

I don’t think I can, or need to, disprove the second, though I think in the paragraphs above I’ve pointed the way towards some problems with the thesis that make it less appealing than it might at first be.

The first is also intuitively appealing, but it too is problematic. I think there’s a implied argument there, a flawed one: there’s a leap in logic between the premise “data sets inspire operations” and the conclusion “those operations comprise the application.” While the premise is true, valuable and significant operations often emerge from the users of data; and through a feedback process, these operations become commonplace in ways that the data alone never could have suggested. People made tabular data easier to understand by making graphs of it for hundreds of years before some mad genius at Microsoft came up with PivotTables, and now they’re indispensable. But they sure aren’t inherent in a ledger of handwritten numbers. Nicholson Baker made a perceptive point in one of his semi-autobiographical novels that the designers of sugar packets and windshield wipers didn’t anticipate that people would centrifuge the first to better control the release of the contents and use the second to keep advertising flyers from blowing off of parked cars in the wind, but those behaviors have become integral facets of the use and therefore cultural significance of those artifacts.

And yet not everything you can do with a particular kind of data is something you should do. Word processors replace typewriters, in that they let you do things with paper. You can put various semantically significant symbols on a piece of paper, and you can also make an airplane out of it. Should a word processing program, then, contain a paper-airplane-design feature? Probably not.

The appeal of flawed thesis #1 above when I first started thinking hard about it a few years ago led me to embrace document-centric user interface design. For instance, I mentioned the idea approvingly in a review of Alan Cooper’s book The Inmates Are Running the Asylum in 2001 (an essay that now seems somewhat embarrassingly snarky and strident, but I’m archiving it here as-is anyway rather than trust Amazon to hold on to it for me forever). I still like the idea in theory, but I have serious doubts about the viability of the implementation. As I note in the Cooper review, Jef Raskin’s work in UI design exhibits the most extreme form of document-centricity—no applications at all. In the characteristic systems Raskin pioneered, the screen is a single window into one big document containing text, numbers, pictures, whatever; in theory, any operation can be performed at any point in the document at any time. Not only does the user not need to open a spreadsheet application to total up a column of numbers in the middle of a word-processing document, but he or she simply can’t, because there is no spreadsheet and no word processor; there’s just the numbers and the text. To add up those numbers you’d just select them and invoke a “TOTAL” command of some kind. Don't believe me? Read the description of the Canon Cat interface.

This is supposed to make life with the computer easier, because it does away with modes, the most-feared bugbear of interface design since the early days of the Macintosh. That is, you never have to worry about whether you’re in the “typing mode” or the “calculating mode” (for example), because (again) there is no spreadsheet and no word processor to switch between. But just a few sentences ago I said that there are problems with the implementation of this notion. Here’s the issue: Not all operations can be performed on all types of data. What happens when you try to invoke that “TOTAL” command after selecting a column of words? Will the computer do nothing? Will it spit back at you the total of the ASCII values of the letters in the selected words? (Let’s hope not!) Will it beep? (Ditto.) There’s no good answer. If you’re allowed to perform any operation on any type of data, cases where user input doesn’t make sense are going to be plentiful. (And any interface that makes it easier to make mistakes is obviously not an improvement.)

The alternative is to allow only those operations that make sense for the particular type of data in question. And that’s just modes again! Suddenly our noble and inspired designers of document-centric interfaces find themselves impaled on the horns of the elephant in the room.⁵ (Ouch!⁶)

1. The definitive discussion of this is probably somewhere in Wittgenstein, if one were to look hard enough. [back]
2. This one is going on my birthday list for sure. [back]
3. ”Supposed” to other people than just me and Hofstadter. See, for example, this whitepaper. [back]
4. In Metamagical Themas. And perhaps the design of a tool is an emergent property of its use—just fancy use! [back]
5. The designers of archy, the latest system inspired by Raskin’s work, imply that they solve these problems by making the system smart enough to detect the kind of data being acted upon and perform the correct action. Way to beg the question, guys. [back]
6. Sorry about the mixed metaphor and cliche. On the third hand, though, why exactly is modality so dreaded? I’m not the first person to notice that life itself is modal. If you pick up a pencil you’re constrained to a certain, even if fairly large, set of actions: you can write, pick your nose, stab your enemies, or stir a pot of soup with that pencil, but you can’t loosen a bolt, or do a thousand other things for which only other tools are appropriate. (And here we are back at the flawed but oh-so-seductive analogy between physical and virtual tools.) [back]