The European Union (EU) funded DataGrid project passed its first-year review at the beginning of March. In a one-day exercise, external experts appointed by the EU watched as a grid testbed was put through its paces. Jobs submitted from several institutes across Europe used grid technology to make the best use of distributed network resources.
The DataGrid project brings together five European institutions engaged in particle physics research: CERN; the French CNRS; Italy’s INFN; NIKHEF in the Netherlands; and PPARC in the UK, along with the European Space Agency as principal contractors. A total of 17 institutions are involved in developing the so-called “middleware” software to analyse distributed computing, storage and data resources and to determine the best place to run a job. Middleware is also responsible for distributing the computation, as well as handling all necessary logging and bookkeeping, and providing fast, secure data transfer and cataloguing.
For the first-year review, computing centres run by the five principal particle physics contractors took part, and 15 jobs were submitted. These came mostly from particle physics experiments, with a smaller number being submitted by the Earth observation and computational biology communities. The jobs were efficiently distributed across the available resources on the testbed network, providing a strong demonstration that the middleware was doing its job correctly. A few of the jobs did not finish as expected, but the reviewers accepted this, saying that they would not have believed that the demonstration was live if there had been no glitches.
The main motivation behind the DataGrid project is storing and processing the enormous amount of data that will be produced by experiments at CERN’S Large Hadron Collider. A data flow of a few petabytes per year is anticipated, and more than 50,000 workstations will be needed for analysis. Although the jobs in the first-year review consumed a total of less than one hour of computing time, they demonstrated the principle that such a task can be handled using a grid approach. In its current state, the testbed can provide 8 months of CPU time in a single day on a total of 242 machines. The next step is to expand testbed use to more users and to conduct more challenging tests.
