I have been a huge fan of Gentoo Linux for several years. Why? Is it the elitist appeal of compiling your operating system from scratch? Could it be performance gains achieved by compiler and package optimizations for a particular piece of hardware? The excellent forums? Nope, none of the above. The Gentoo community instills a culture of looking under the hood, of customizing and tweaking, of not being afraid of greasy hands. I get laughed at in the office sometimes because of this and I have to spend more time installing, but I think it's worth it. Deep customization seems to be the wave of the future, and here is why I think so.
Enterprise Application Servers: great idea! Provide a uniform platform optimized for a specific type of application. Abstract the hardware layer with some operating system and a JVM. Serve up Ebay! But why stop there? Why can't the app server run on bare metal? What is the use of all that OS overhead/complexity/variance underneath? In software design, we constantly make compromises between optimization and generality. What if we didn't have to? What if we could develop and deploy an entire operating system for a specific application or job position or organization? Well of course you could but it would be incredibly expensive and take forever. And who's gonna write all those device drivers?
At SCALE in LA a few months back I had the opportunity to see a presentation by and briefly talk with Bruno Gonçalves de Albuquerque of the Haiku project. The concepts behind the operating system are fantastic. The level of optimization at the user experience level when you control the entire OS (read: filesystem) is really amazing, and clearly blows the doors off any desktop OS I've ever used. Why am I not using Haiku right now? Well apart from it not being quite done yet, hardware support. Developing device drivers is a constant and thankless battle, typically undertaken by Microsofts and OEMs. Where is the incentive for a Taiwanese OEM to provide a device driver for Haiku? While listening to Bruno Gonçalves de Albuquerque it occurred to me that this problem may just go away if Haiku had binary compatibility with the Linux driver model. All of a sudden my graphics card works, and my wifi (gasp) too! Linux, besides being an operating system, provides support for a very large number of devices and systems. Fresh off the presss, 2.6.25 supports a new hardware platform. The fact that all of this work is open source presents an opportunity for other OS producers to leverage this work. It lowers the cost and increases the potential user base by supporting a greater variety of hardware. But who cares? We have OS X and Ubuntu and XP. Do we really need another operating system?
On the hardware front things are changing radically. Eight cores today, 64 tomorrow. Specialized (audio, physics, credit card validators) cores on the same chip. There is talk of doing away with the graphics card altogether and putting that in the CPU too. The list goes on... But where is the software? Where are the operating systems and languages and runtime environments to fully take advantage of this latest round of hardware innovation? And who decides where virtualization occurs? Is it KVM...or perhaps some JVMs? Do I have to run one master OS that virtualizes all the others? Can I partition, version, and manage OSs per core? Dynamically? Can I have a custom OS for Crysis III that gives me a few more frames per second because my machine isn't looking for security patches? Do I really have one machine or many?
There is a shift happening in software caused by new hardware platforms. Component based design, concurrency, functional programming, and virtualization are all, in part, results of trying to utilize new hardware designs. This change is similar, but hopefully more fundamental to the shift from single-task operating systems to multitasking ones. From TSRs to Windows. It took so much time for that earlier change to manifest*. What's the difference between then and now? How long do we have to wait for MS Windows 95 part II?
Well I think the difference is Linux and open source. The difference is an open driver model that allows OS developers to focus on new ideas rather than supporting 3000 network cards. The ability to see solutions in the open, and the ability to collaborate. No more silos, no more ivory towers. The demand for radical OS customization will increase as divergence grows between hardware potential and what our existing operating systems, languages, and runtimes are capable of. And as this demand increases I hope to see new, radical, highly-custom new operating systems and runtimes emerge. Exciting times!
* And yes, I had an Amiga too back in 1989. And yes, it was sweet.