Workshop identifies steps to reap benefits from multicore and virtualization technologies

Driven by advances in two of its research and development projects (Parallelization of Software Frameworks to Exploit Multicore Processors and Portable Analysis Environment using Virtualization Technology) CERN organized a workshop on 24–26 June about adapting applications and computing services to multicore and virtualization technologies. The workshop brought together experts from industry, developers using these technologies and IT service providers at CERN. It provided an understanding of what can be achieved and identified a set of actions required for physics applications to further exploit multicore and virtualization technologies.

The workshop was structured into sessions on technology, application requirements, computing services and Grid services. Although they had a CERN focus, presentations and discussions were enhanced by contributions from representatives of institutes elsewhere in Europe and from North America.

From a technology viewpoint, the number of cores per machine will continue to increase in the near future and the challenge becomes implementing software in ways that can efficiently exploit them. The increased number of cores per machine has helped to drive the rapid adoption of virtualization. In addition to its benefits for resource consolidation, virtualization creates opportunities for a more flexible approach to offering computing services. Both technologies are rapidly maturing, particularly in terms of performance and management tools. Physics applications can benefit from these advances but computing services need to adapt to support them.

Application requirements were presented in terms of the promising experience so far using multicore and virtualization together with requests for support beyond test environments. As expressed by Paolo Calafiura: "ATLAS is ready to transition from prototypes to production." The presentations on IT and Grid services completed the picture with information about available and planned services.

From the fruitful discussions that ensued, the follow-up actions below were identified. These, together with increased understanding and collaboration, represent the results of the workshop.

Actions at CERN:
• provide infrastructure in CERN's computer centre for the preparation of CernVM images and the Virtual Organization's application software delivery to them. CernVM images are generated by tools of the CernVM project, http://cern.ch/CernVM, which provides a virtual software appliance for developing and running LHC data analysis.
• include the capability to run CernVM images in CERN's virtualized batch initiative.
• test scheduling options for parallel jobs in mixed workload environments.

Actions requiring Grid-wide collaboration:
• establish procedures for creating images that can be trusted and run at Grid sites. This is needed for Virtual Organizations to be able to run their images at Grid sites.
• investigate scenarios for reducing the need for public IPv4 addresses on Worker Nodes. Virtualization is increasing IP address usage and given the IPv4 address limitations (www.ipv6actnow.org) public IPv4 addresses need to be used wisely.
• deploy multicore performance and monitoring tools (e.g. KSM, PERFMON) at CERN and at other Grid sites.
• provide input to initiatives for running multicore jobs Grid-wide, e.g. EGEE's MPI (Message Passing Interface) Working Group recommendations.

Interoperability with clouds:
• prototype a solution to run Grid jobs on cloud resources.

Further information can be obtained from the slides and an executive summary of the workshop, which are linked from the agenda page: http://indico.cern.ch/conferenceDisplay.py?confId=56353.