This is another one of those posts that has taken me a while to write - a little over a month, in fact. Today I added a few final sentences and feel it is finally good to go. Whew! I feel the subject was important enough to be worth the effort. I hope you feel that way too.
Back at the end of March, I read an online article by Bill Snyder at the Infoworld site called Multi-core to leave developers in dust?. It concerned the looming challenge of leveraging the parallel processing resources being made possible by the emergence of inexpensive, soon-to-be-utterly-ubiquitous multiple-core CPUs. Snyder feels the IT industry is not ready for the challenge. I agree. I first wrote about the problem last December in a post entitled The Landscape of Parallel Computing Research: the NY Times and Berkeley EECS. In many ways this post is a continuation of that thread.
Here's a bit of what Snyder has to say:
That's not to say that the IT industry is scoffing at the potential benefits of multi-core processing. But the mountain between IT and some future multi-core promise land -- namely, the task of developing parallelized apps that keep pace with continual core advances -- is huge, says David Patterson, the Pardee Professor of Computing Science at UC Berkeley and director of the parallel computing lab. "It's the biggest challenge in 50 years of computing. If we do this, it's a chance to reset the foundation of computing."
In the short run, Patterson says, we can parallelize legacy software and gamble on getting value out of eight cores. But that would be only an interim solution, as such apps would not scale to 32 or 64 cores, he adds.
What is frustrating is that this problem didn't exactly sneak up on the industry. Chip development cycles are very long, and key software developers are well aware of what's moving through the pipeline. Sure, software always lags hardware. Many of us complained that we didn't have software that would take advantage of 500MHz back in the '90s. But what Patterson and others call the multi-core revolution poses problems for developers that are qualitatively different than the problems of the past. Why wait so long to get serious about solving them?
The answer to the last question is, in my opinion, a personality characteristic common among IT personnel, which is the always-incorrect belief that my current technology - architectural model, operating system, hardware platform, programming language, integrated development environment, application framework, or what-have-you - is the be-all and end-all of IT evolution.
Case in point: Not long ago I was indirectly involved in the port of an application from Ingres on a Vax to a Java/J2EE platform on Windows 2003 Server (web container) and JBoss/Oracle on Linux (EJB container and DBMS). A single programmer had worked on the system for almost 20 years, was not far from retirement, and had much fear of and no interest in learning the new technologies. What was sad about the situation is not the programmer's mindset, but the fact that her managers had allowed the situation to devolve into the state it was in at the time of the need for conversion. If the leaders don't lead, the followers oftentimes don't follow. This was by no means the only time I have encountered this mindset malaise.