Let a thousand servers bloom

Posted by Christophe Bisciglia, Senior Software Engineer

Just as people are social animals, computers are social machines—the more, the merrier. Twenty or thirty years ago, large, centralized mainframes sat alone in sheltered bunkers in computer science departments and government offices alike, choking for hours on mere megabytes of data. Even with recent advances in server technology, large, centralized machines are still struggling to cope with today’s modern computational challenges, which now involve terabytes of data and processing requirements well beyond a single CPU (or two, or four, or eight). One computer just won’t hack it; these days, to support a new paradigm of massively parallel systems architecture, we need to break the machine out of its bunker and give it some friends.

In this age of “Internet-scale” computing, the new, evolving problems faced by computer science students and researchers require a new, evolving set of skills. It’s no longer enough to program one machine well; to tackle tomorrow’s challenges, students need to be able to program thousands of machines to manage massive amounts of data in the blink of an eye. This is how I, along with my good friend and mentor Ed Lazowska of the University of Washington’s CSE department, started to think about CS curricula and the obstacles to teaching a practical and authentic approach to massively parallel computing.

It's no easy feat. Teaching these methods effectively requires access to huge clusters and innovative new approaches to curricula. That's why we are pleased to announce the successful implementation of our Academic Cluster Computing Initiative pilot program at a handful of schools, including the University of Washington, Carnegie-Mellon University, Massachusetts Institute of Technology, Stanford University, the University of California at Berkeley and the University of Maryland. This pilot extends our expertise in large scale systems to strong undergraduate programs at the pilot schools, allowing individual students to take advantage of the hundreds of processors being made available. As the pilot progresses, we'll work with our technology partner IBM to shake the bugs out of the system so that we can expand the program to include more educators and academic researchers.

The future of computing is already taking shape on campuses today, and Google and IBM are thrilled to help inspire a new generation of computer scientists to think big. All of the course material developed by UW as well as other tools and resources to facilitate teaching this cutting- edge technology is available at http://code.google.com/edu. If you're a student wondering just what this sort of thing means for you, check out the five-part video lecture series (originally offered to Google Engineering interns) that introduces some of the fundamental concepts of large-scale cluster computing.