Monday, October 10, 2011

You’ve got a strong back—get a job!

Here’s the current roster of personal projects I’m working on in an effort to become more employable:
  • A realtime, multithreaded Scala program which I'm building in stages but which I envision will probably end up at 10-20,000 lines of code--involving realtime computations against a NoSQL database and interaction with a SOAP service.
  • A build-your-own content-management website á là mywedding.com—I’m tinkering with the Django, Drupal, and Catalyst MVC frameworks, trying to decide which will work best while getting some exposure to PHP 5, Python 3, and Moose in the bargain. This will include a MySQL backend, so I’m brushing up on SQL as well.
  • Some Arduino toys including a couple of retro, 8-bit-style Gameduino games to polish my rusty C skills.
  • A slightly more ambitious microcontroller project, this one using an ARM board and the mbed platform to do some real-world data acquisition and control of a mechanical system (that’s deliberately vague: more details will come when I start to get an idea whether it’ll actually work!).

(A gold star for those who know what classic ’90s movie the title of this post is a quote from.)

Tuesday, October 4, 2011

Software engineering methodology: my, how you’ve grown!

“Hey, is anyone working on foo.c in the Whatsit build? I need to make some changes in it.”

That’s what source control consisted of at my first job as a software engineer, fifteen years ago. Bug tracking took place somewhere between the brain of the lead developer for each project and his email inbox. Code reviews consisted of the smirking face of another programmer appearing over the top of my cube, saying, “I found a place where you dereferenced a null pointer. Want me to show you?”

I learned about the classic academic work in software engineering in school as part of my computer science degree: Fred Brooks, the waterfall model, unit testing, and so on. But as far as I knew in 1996, outside of large companies with an established research culture (Intel, IBM, Xerox) none of those methods were being applied. Neither was the general idea behind them, that software development needed planning, structure, and organization, things that could not to be trusted to individual programmers working in isolation.

It’s amazing how much things have changed in the past fifteen years. The classic book The Psychology of Computer Programming was published in the '70s but its effect was purely academic. It took years for the lessons described there to filter down to the level of actual practice, and it wouldn’t have happened without Steve McConnell’s Code Complete. From the latter book grew the agile development movement, a reaction both against the waterfall model (which makes it so easy to postpone testing until it gets pushed off the bottom of the release schedule) and against cowboy development culture (which makes for a host of problems—poor bug tracking, poor source control, reinvention of the wheel as every programmer builds his own libraries, and on and on.).

And I never would have expected the massive growth that has taken place in what I’ll call the collaborative programmer culture. Just as the free exchange of information in general has exploded due to the Internet, so has in particular the exchange of information on how to program better. There’s something going on that’s greater than just more message boards and more acceptance of the open-source “more eyes mean fewer bugs” idea. I remember when the entire O’Reilly library fit on just one of those old-fashioned revolving wire book stands at the best technical bookstore in the world; now it takes four or five stands, not just because there are more programming languages and software technologies to cover but also because the coverage of each is deeper (e.g. compare the matched set of Llama and Camel Books that used to make up the entire printed documentation of Perl, to the dozens of books now published on the language). Code Complete spawned an entire genre of “software as a craft” books like the Pragmatic Programmer series.

All these factors mean, I hope, that a new generation of software developers is growing up with the idea that agile development methodology is not just the correct way to write software but the natural, comfortable one. This is probably the biggest change ever, with the greatest lasting effect, in the way software is created—bigger than the widespread adoption of OOP over straight procedural programming, bigger than the shift to dynamic languages and automatic garbage collection, bigger than Linux making a professional development environment available to students. It’ll be interesting to see its long-term effect... has the looming dragon of the Software Crisis finally been slain?