Globus Toolkit upgrade aids the Grid community

Résumé

Globus Toolkit est développé depuis la fin des années 90 pour fournir des infrastructures et des applications de calcul décentralisées, axées sur le service. Ce progiciel joue un rôle clé pour la mise en œuvre et les applications de la grille de calcul pour le LHC (LCG) et de grilles associées. Dans cet article, Ian Foster commente la version 4 de Globus Toolkit (GT4), le dernier né de ce progiciel de Globus en libre accès pour le calcul sur grille. Disponible depuis avril 2005, GT4 présente d'importantes améliorations, que ce soit en matière de robustesse, performance, convivialité, documentation, conformité aux normes ou fonctionnalités.

Globus software has long played a significant role in the LHC Computing Grid (LCG) and related Grid deployments and applications. In 2005 the Globus community achieved the following significant milestones: the release of Globus Toolkit version 4; the establishment of a new community development process; major new funding from the US National Science Foundation and the UK Engineering and Physical Science Research Council; and the establishment of companies focused on supporting commercial use of the software.

Globus software has been developed since the late 1990s to support service-oriented distributed-computing applications and infrastructures. Core components of the Globus Toolkit (GT) address, within a common framework, basic issues relating to security, resource access, resource management, data movement, resource discovery and so forth. These components enable a broader "Globus ecosystem" of tools and components that build on, or operate with, core GT functionality to provide a wide range of useful application-level functions. These tools have in turn been used to develop a wide range of Grid infrastructures and applications.

A major upgrade

GT4, released on 30 April 2005, provides significant improvements in terms of robustness, performance, usability, documentation, standards compliance and functionality (I Foster 2005). For example, the GRAM job submission service now easily manages 32,000 jobs, up from a few hundred in GT3. A UK e-Science evaluation speaks favourably of the quality, usability and performance of the GT4 code and its documentation (T Harmer et al. 2005). It notes, for example, that "GT4 installation was straightforward", "GT4 services demonstrated significant improvements in performance and reliability over their GT3 versions" and "GT4 package descriptions were of a high quality, well structured, and accurate".

GT4 makes extensive use of Web Services mechanisms to define its interfaces and structure its components. Web Services provide flexible, extensible and widely adopted XML-based mechanisms used for describing, discovering and invoking network services. Furthermore, its document-oriented protocols very much suit the loosely coupled interactions that many argue are preferable for robust distributed systems. These help the development of service-oriented architectures: systems and applications structured as communicating services, in which service interfaces are described, operations are invoked, access is secured and so on - all happening in uniform ways.

GT4 schematic

The figure illustrates the principal GT4 components. A set of service implementations (the bottom half of the figure) implement useful infrastructure services. These address such concerns as execution management (GRAM), data access and movement (GridFTP for data movement, reliable file transfer, OGSA-DAI for database access), replica management (replica location service, data replication service), monitoring and discovery (Index, Trigger, WebMDS), credential management (MyProxy, Delegation, SimpleCA) and instrument management (grid telecontrol protocol). Most are Java Web Services but some (indicated at the bottom right) are implemented in other languages and use other protocols. A powerful authorization framework enables the implementation of diverse authorization policies.

In addition, three containers can be used to host user-developed services in Java, Python and C. These containers provide implementations of security, management, discovery, state management and other mechanisms frequently required when building services. They extend open-source service-hosting environments with support for a range of useful Web Service specifications, including WS Resource Framework (WSRF), WS-Notification and WS-Security.

Finally, client libraries allow client programs in Java, C and Python to invoke operations on both GT4 and user-developed services (top of the figure). In many cases, multiple interfaces provide different levels of control: for example, in the case of GridFTP, there is not only a simple command-line client (globus-url-copy) but also control and data channel libraries for use in programs - and the XIO library allowing for the integration of alternative transports.

GT4's use of uniform abstractions and mechanisms across these different components means that clients can interact with different services in similar ways, which facilitates the construction of complex, interoperable systems, and encourages code reuse.

Globus is more than software: it is also a community of users and developers, and the infrastructure that supports this community - code repositories, e-mail lists, a problem-tracking system and so forth. To facilitate external contributions, the Globus team recently introduced a community governance and development process, GlobDev, hosted at http://dev.globus.org. A growing number of Grid software projects are hosted and developed there.

Further reading

I Foster 2005 IFIP International Conference on Network and Parallel Computing LNCS 3779 2.
T Harmer et al. 2005 UK e-Science Technical Report Series UkeS-2005-03.

This report describes the work of many talented colleagues and collaborators (see www.globus.org). Work on Globus has been supported in part by the Mathematical, Information, and Computational Sciences Division subprogram of the Office of Advanced Scientific Computing Research, US Department of Energy, by the National Science Foundation (NSF) under its NSF Middleware Initiative and other programmes, and by IBM, DARPA, NASA, Microsoft, the UK Engineering and Physical Sciences Research Council and Department of Trade and Industry, and the Swedish Research Council. The author is also co-founder and chief open-source strategist at Univa Corporation.