Required Reading: Ultra Large-Scale Systems

At Divmod, we're always talking about the future of computing, software, and the network. This usually focuses on our work with Twisted or the Divmod platform. But we have also spent considerable time assessing research in the area of what is called "ultra large-scale systems." Our primary business interest with this revolves around development, deployment, and management. However, there is a great deal of work that needs to be done to make ULS systems a reality.

I have a series of blog posts planned to discuss ULS systems in the areas where I have a vested interest. Despite the fact that a popular study on the matter was funded by the U.S. Department of Defense (conducted by the Carnegie Mellon Software Engineering Institute), I will not be discussing this technology in the context of war efforts nor national defense. Instead, I will be engaging in this discussion within the context of the medical/health services field, per the example given by Richard Gabriel in his presentation (PDF) to the Chinese University of Hong Kong.

This is especially pertinent today, as Google prepares for its Google Health announcement. Even though there are by some estimates (optimistic ones, in my view) 25 years of research ahead of us, it is generally agreed that ULS systems will consist aggregate sub-systems, built incrementally over time. I don't believe anyone is under the illusion that Google does not want to produce the first ULS system in history, and they are making rapid progress towards this goal. Google Health brings this point home very clearly, in the context of the research that has been done in this area.

The world is rapidly changing. The most important issues in technology and business are not who is going to create the next catchy "Web 2.0" application, or what mega corp games are being played in the Great Silicon Valley Soap Opera. The important issues are how the systems of the next 100-200 years are being built now, who they are being built by, and who as access to that technology.

We are a powerful and creative community. We are concerned about distributing power to the people, education, privacy, and freedom -- open source is built upon these principles. If we want those who are building ULS systems to build them fairly and with our concerns in mind, then we must get involved now. We must start building the necessary tools.

As for the reading, a couple of these were linked to in this blog post. Here is a list that should get your brain revved up and ready to roll (these are all PDF files):

• Design Beyond Human Abilities
• Ultra-Large-Scale Systems: The Software Challenge of the Future
• Scale Changes Everything
• ULS Systems Research Roadmap

I will have more specifics about the sorts of things we're exploring in future blog posts, but until then -- don't get left behind!

Update: Allen Short just pointed out some additional material that predates the Carnegie Mellon research: the Agoric Papers, co-authored by Mark Miller who is a co-creator of the E programming language.

Update 2: Glyph Lefkowitz just sent me a link to the Big Ball of Mud that discusses similar concepts.

Technorati Tags: technology, business, community , open source, uls