Les grilles nordiques appuient la Grille mondiale LHC
Les pays nordiques participent au développement de la Grille au CERN depuis le tout début, bien qu'ils aient dès le départ opté pour le développement de leur propre intergiciel. La plupart des projets nordiques utilisent aujourd'hui l'ARC (Connecteur de ressources avancé), élaboré par NorduGrid, en plus de l'intergiciel gLite du projet EGEE (réalisation de grilles pour la science en ligne). En effet, plusieurs sites participent à la fois à NorduGrid et à des projets tels qu'EGEE et font donc appel aussi bien à gLite qu'à ARC. L'article replace ces nombreuses activités de grille nordiques dans leur contexte.
Nordic countries have participated in Grid developments at CERN since the beginning, although at the early stages they already opted for developing their own middleware. Most Nordic Grid projects now use the Advanced Resource Connector (ARC), developed by NorduGrid, while some also use the gLite middleware deployed by the Enabling Grids for E-sciencE (EGEE) project. Several sites are involved both in NorduGrid and in projects such as EGEE, and therefore deploy both ARC and gLite. But let's start from the beginning.
The EU DataGrid (EDG) project began in 2001, but with no partners in the Nordic high-energy physics (HEP) community. The community therefore decided to create a satellite project collaborating with EDG, and applied for Nordic funding through the NORDUnet2 programme. This project was later named NorduGrid and involved a dozen Nordic universities and high-performance computing centres, representing four Nordic countries, Denmark, Finland, Norway and Sweden. Funding covered test-bed hardware and three postdoctoral research positions.
The project spent a few months testing and installing the then embryonic EDG middleware, and communicated proposals for improvements back to EDG and Globus. It also participated in the EDG integration and testing efforts. However, the planned contribution to ATLAS Data Challenges in 2002, to test the emerging computing infrastructure, seemed likely to be impossible using Grid tools because of many shortcomings, such as problematic job-submission or the information system and monitoring. The decision was therefore taken to write a new solution. NorduGrid thus implemented new versions of the information system, the Grid front-end and other essential components. The EDG project encountered the same problems and later introduced the BDII and Condor-G tools to tackle the difficulties, albeit in a different way. This was the first major divergence between the two middleware solutions, and it still remains today. EDG became the middleware of choice for the LHC Computing Grid (LCG) project and has been further developed into the gLite middleware stack by EGEE.
In mid-2002, NorduGrid emerged as one of the first production-level Grids and continued to grow until it became one of the world's largest Grids in 2004. (For more details on the use and achievements of the NorduGrid collaboration see CERN Courier April 2005 p13).
When the funding for NORDUnet2 ceased in 2003, NorduGrid was converted into a collaboration dedicated only to middleware support and development, while deployment of the ARC middleware was taken over by the Nordic DataGrid Facility (NDGF). April 2004 saw the first stable release of the ARC middleware, and since then middleware development has concentrated on adding new features and functionalities, leading to the next stable release, scheduled for August 2006. In parallel, efforts were made to extend ARC to new sites, supporting users and improving relations with other Grids with possible future interoperation in mind.
In 2005, several new projects supporting further development of the Grid infrastructure and the ARC middleware were conceived: KnowARC, NDGF (both starting in June 2006) and a NorduGrid continuation project as part of the NORDUnet3 programme, which will start in October 2006.
What then is the relationship between these different projects, and how do NorduGrid and the ARC middleware fit into this scheme? ARC is the open-source middleware coordinated by the NorduGrid collaboration, which is based on a memorandum of understanding between the universities of Copenhagen, Lund, Oslo and Uppsala, plus the Helsinki Institute of Physics. This collaboration provides major contributions to most of the projects. KnowARC and NDGF will be the major contributors to the ARC code base in the coming years.
The KnowARC project will improve and extend the ARC middleware to become next-generation Grid middleware conforming to community-based standard interfaces. It will also address interoperability with existing, well-deployed middlewares. By getting ARC included in the standard Linux distributions, KnowARC will contribute to Grid technology, enabling all kinds of users, from industry to education and research, to set up and use this standards-based resource-sharing platform easily. More than half of the contributors to KnowARC are from outside of the Nordic countries and more than half are not related to HEP, making KnowARC a true all-purpose Grid project.
The NDGF project aims to create a seamless computing infrastructure for all Nordic researchers, leveraging existing, national computational resources and Grid infrastructures, and in addition includes middleware developers. NDGF is not a resource provider and does not force researchers to use the Grid, but rather helps researchers to share their resources if this is advantageous for Nordic or international collaborations. In this scheme, HEP fits perfectly, but bio-informatics is expected to become heavily involved as well. After the initial pilot phase, the project has been substantially expanded to include middleware developers.
The first big goal for the NDGF HEP activities is to provide the NDGF Tier-1 centre for the Worldwide LHC Computing Grid (WLCG) infrastructure. As NDGF does not possess any resources, the actual Tier-1 service will be distributed between the seven largest computer centres in the Nordic countries, utilizing the fact that up-time requirements can be relaxed by redundancy. The middleware used for the computing and storage infrastructure will be ARC, introducing another Grid flavour into the WLCG, together with gLite and OSG.
Organizationally, NDGF is co-hosted with the Nordic Regional Research and Educational Network NORDUnet, following the vision that the Grid and access to computing resources should be no different to other services on the network. Hence, a close coupling between the network for the distributed Tier-1 centres and the computer sharing services is guaranteed.
Further activities
The Nordic countries are also quite heavily involved in the EGEE project in operation and support, applications, training, testing and certification. One of the tasks undertaken by the Nordic countries is interoperability between ARC and the EGEE middleware gLite. Job submission from ARC to gLite is a part of the work programme for the KnowARC project. Training and outreach are primarily targeted by the Nordic Grid Neighbourhood network, which arranges conferences and student exchanges within the Nordic countries, the Baltic states and Russia.
Another Nordic project that deserves to be mentioned is the BalticGrid project. Baltic states traditionally have strong links with the Nordic countries and collaborate closely with NorduGrid, providing both hardware resources and middleware developers. This Grid experience helped to create the BalticGrid project, which introduces gLite services and tools in addition to ARC services that already exist.