Java and the growth of the software business: The more things change...

I’m at a position career-wise where it’s to my advantage to broaden my technical skills before I deepen them any further, so Java is in the to-be-learned queue for real this time (as opposed to in previous posts in which I wrote “I really ought to learn Java” and then went back to banging out backwards ifs in Perl). I’ve been balking at putting Java on the front burner for a while, and for a while I didn’t know why.

Now the reason has finally bubbled to the surface: it’s that Java seemed like a toy back in 1998 or so when I was first programming professionally. Its speed was an issue; the programmers I hung around with were hardcore old-school bare-metal types, and our attitude was that you wrote systems code in C or assembler, applications code in C or C++ (using the Win32 or Microsoft Foundation Classes libraries), and anything else was too slow for serious work. Java’s promise of platform agnosticism seemed like marketing smoke and mirrors, and we couldn’t get past the fact that it was interpreted. (Interpreted? Interpreters were for scripting languages, and you couldn’t write a real application in those, either—more words to eat as Perl and Python matured!)

I’ll get back to the speed issue in a bit. What’s really changed my mind the most about Java is the sheer amount of support for the language in the form of third-party libraries, on top of the staggeringly large, well-thought-out platform that Sun provided from the get-go and Oracle has continued to enlarge and refine. I say “well-thought-out” deliberately; the Microsoft Foundation Classes sounded like a good idea in 1999, but in practice their functionality overlapped with the C++ standard library and the older APIs that Windows programmers were already familiar with, and the typical MFC program ended up being a mashup of all three. (Which is why MS did a good and brave thing when they scrapped all the old cruft of MFC and Win32 and built .NET from scratch.)

The particular example that tipped the balance isn’t so important as such, except that it illustrates the fine-grained libraries that are out there: it’s the jzy3d package, which provides the kind of three-dimensional data visualization that a year ago I was trying to build by hand. And this brings us back to the question of speed: jzy3d is built on top of JOGL, a Java API for OpenGL; and one of the example programs linked from the JOGL site is a port of the game Quake2 that runs in the browser. I was flabbergasted to see that. Quake, for you young-uns, was a ground-breaker in immersive 3-D graphics on its release in 1994 or so; to run it smoothly in high resolution took a high-end CPU (I had a 100 MHz Pentium, pretty good for the time) and a $300 graphics card; it ran best in DOS, needing to hammer on the metal directly for maximum speed; and it’s still a benchmark for low-powered systems (is there a port to the iPhone? I don’t know, but I’d be impressed if there were). As far as I’m concerned, if Java is fast enough to run Quake, then it’s fast enough to run any typical desktop application.

And I think that’s what the majority of the software business thinks, too. Department of Labor stats and the Wikipedia page on software engineering show a doubling in SE jobs since 2004, and that’s despite outsourcing and the 2008 economic crash; what’s more, that statistic doesn’t account for the number of programmers who don’t have the title of software engineer. I’d suspect the true level of growth to be quite a bit greater.

And though I can’t find any information as to what language the majority of these people are coding in most of the time (and if I could I’d suspect it of having been rigged by someone with a vested interest, like Oracle or MS... though here is an attempt from 2006—look at the numbers at the end of each section), I bet Java accounts for much of that growth. The language is just that good, and that effective, and that well-supported. It's enabled many more programming jobs to be created, because it's lowered the barrier to entry. Let me stress that I don’t mean that as a euphemism for “Now you don't have to be as smart to be a programmer, so a lot of stupid people are getting good white-collar jobs that they wouldn't have otherwise.” Rather, a flatter learning curve for mainstream software development is a universal win: it means more jobs, better-working software, and a more robust world economy, thanks to, for one, the continuing adoption of general-purpose hardware running custom software in industries that previously relied on custom and/or archaic hardware for process control. The more things change, the more they change!

Ten years ago people were calling Java “C++ for dummies.” Well, guess what? That isn’t an insult any more. People are coming to realize that C++ can be a real pain in the butt. It’s hard to write maintainable, reliable code in C++. You can do it; I can do it, and I have done so in the past. And I want to make clear that where speed is of the absolute essence, C++ is obviously a better choice (although I think the complexity-vs.-features tradeoff of using C++ vs. good ol’ C ought to be be carefully considered in those cases). There's at least one excellent API for user-level software in general use that works best with C++, Qt, and if there's one thing you can say about Qt programs it's that they are snappy! And, undoubtedly no coincidence, the API is a delight, very clean and lightweight—it reminds me of programming BeOS. (With the speed and memory of the average consumer-level computer being what they are these days, it’s ridiculous that we still have to put up with sluggish “click-and-wait” UIs... but that's a topic for another post.)

In practice, most C++ code, I would bet, is written using a subset of the language with which the programmers on any given project feel more comfortable, ignoring the really tricky features (no multiple inheritance, for example; maybe no operator overloading) and using some sort of integrated garbage collection scheme (overloaded new, for example) rather than relying on the programmer to manage dynamically allocated memory. I didn’t realize until I took a good look at Java (if you must know, I’ve been reading the first 200 pages or so of Java in a Nutshell a bit at a time before bed for a while now) that that comes very close to describing what Java syntax is: a slightly more verbose, less complicated version of C++. (And of course it has garbage collection, although not tunable for the needs of real-time programming, as far as I know.) QED: There’s nothing wrong with that. (Really, who could possibly take offense at replacing =0 with virtual, or : with extends?) Unless you need that last bit of speed that only a true compiled-to-assembler language can give, I can’t see anything wrong with Java as a baseline for applications programming.

Not reinventing the wheel (again)

As I mentioned in two previous posts, I’ve been doing some 3-D graphics programming using the Perl interface to SDL. SDL is quite fast for 2-D graphics, but it doesn’t do 3-D on its own; so there’s a stack on my desk over a foot high of used textbooks with titles like Introduction to Computer Graphics Algorithms from which I’ve extracted the matrix transforms for various kinds of projections of a given pixel from threespace to twospace (orthographically onto a planar "window" given by a certain equation, seen in perspective by an eye located at given points, and so on). It’s been educational, but the results I’m getting are unusably slow, and I think they would still be too slow even if I were using a faster language. So, it’s time to start using a graphics library that supports 3-D—that is, one where someone else has already done the hard part of writing optimized code to do those transforms. It’s one of the hardest lessons to learn as a new software engineer transitioning into the workforce from school: it’s OK to use other people’s code; it’s better, in fact, to use other people’s code in the form of using existing libraries, rather than reinventing the wheel. (Jeff Atwood wrote an interesting contrarian blog post on why reinventing the wheel is sometimes, in special cases, the right thing to do: in general, the preference for reinvention is proportional to the need for performance; but note that there’s no consideration of cost there. Ironically, reinventing the wheel in the software sense is more likely to be favored by the ill-informed or non-technical manager, who doesn’t understand [the accepted wisdom] that it’s "always" better to buy than to build).

The obvious choice, one that’s been around for years with lots of support in the form of books, tutorials, and sample code online, is OpenGL with the GLUT libraries. The GLUT sample code I’ve skimmed seems to have (at a very high level) a two-part structure—you first set up the parameters for a 3-D rendering (the minimal case being an object and a view, with no light sources, texturemapping, or other complicated/cool stuff), and then you spawn a thread or another process to do the actual rendering on the fly until told to terminate. I’m not sure yet how this will square with the fact that I want the rendering to refresh in real time per the new data I’m collecting from the server. Let’s pull some numbers out of a hat: say I have a Perl program that runs all day long, sleeping most of the time, but waking up every five minutes to web-scrape and munge some derivatives-pricing data; say the latter takes about a minute, allowing for the bottleneck of my mediocre broadband connection. It sends that data on to the graphics code, which creates an interesting 3-D graph or visualization¹ in n minutes. Then that 3-D image pops up in a window, rotating slowly so that you can see it well enough to interpret it. If n<4, then the whole thing can repeat ad infinitum. If n>4, then the rendering is going to lag behind (and more behind, and more behind...) the data collection, and become useless. Depending on how GLUT works, I might be able to kill two birds with one stone—minimize n, and minimize CPU load—by avoiding the real-time render entirely: I’m hoping that I can actually have OpenGL render the frames of a smooth 360-degree rotation of the graph from start to finish as image files. Then these could be converted to an animated GIF (if they aren’t already) and shown on a web page containing other useful data (which itself could get updated every five minutes, continuing the example, or more often). Meanwhile, the Perl code sleeps again, and then wakes up and continues the cycle...

I want to have the possibility of running this code on OS X, where I spend most of my time, or on Linux, or on Windows 2000 or Windows XP (which I run in virtual machines here and there for various reasons). Toward that end, in turn, I’m requiring myself to write the most portable code possible: since OpenGL and GLUT are (intended to be) entirely platform-independent within the limits of the hardware they’re run on, there’s no sense in doing anything else. To refamiliarize myself with the Microsoft Visual Studio IDE² I’ll be coding my GLUT app as a Win32 application in C++, if necessary using some simple but not too kludgy wrapper classes around C code full of function pointer callbacks that’s sadly inherent to the Win32 libraries (well, I guess I can give them a break... it was the ‘90s.). However, if the instructions I’ve found for writing cross-platform-safe GLUT code for Linux and Windows using VS 2005 are accurate, that shouldn’t be a problem.

But I’m also going to try writing the whole thing in POGL, the Perl OpenGL/GLUT bindings, which are claimed to be just as fast as using OpenGL via C or C++ or <insert compiled language here> That makes the parsing that I referred to here easier since I won’t have to call Perl from a compiled C++ program (obviously a performance hit to be avoided) and I won’t have to find a regular expression library for C++ (of which I'm sure there are plenty, but the differences in regexp syntax between them and Perl would probably be unpleasant).

1. The form of which is a secret—I can’t give away all my tricks, can I? However, one of the books in that big pile is a collection of academic papers called something like Topics in Scientific Data Visualization, so you can guess I’m headed into some pretty wild territory. (Here be dragons!)[back]


2. One of the few truly great pieces of programming to come out of Redmond, it pains me somewhat to admit, along with Excel 2003 and the old QuickBASIC compiler (the distant MS-DOS ancestor of Visual BASIC, for you young whippersnappers). [back]