Control systems for big physics reach maturity

Control systems are a huge feature of the operation of particle accelerators and other large-scale physics projects. They allow completely integrated operation, including the continuous monitoring of subsystems; display of statuses and alarms to operators; preparation and automation of scheduled operations; archiving data; and making all of the experimental data available to operators and system experts. The latest news from projects around the world formed the main focus of the 11th International Conference on Accelerator and Large Experimental Physics Control Systems (ICALEPCS), which took place on 13–19 October in Knoxville, Tennessee. More than 360 people from 22 countries attended the meeting hosted by the Oak Ridge National Laboratory (ORNL) and the Thomas Jefferson National Accelerator Facility at the Knoxville Conference Center. The 260 presentations, including 71 talks, confirmed the use of established technologies and reviewed their consolidation. Excellent poster sessions also provided plenty of opportunity for discussions with the authors during the coffee breaks.

The weekend prior to the conference saw three related meetings. Almost 50 people attended the Control System Cyber-Security workshop, where eight major laboratories presented and discussed current implementations and future prospects for securing control systems. All have acknowledged the risk and all follow a “defence-in-depth” approach, focusing on network protection and segregation, authorization and authentication, centralized PC installation schemes and collaboration between information-technology and controls experts.

Approaches to control systems

In parallel, 200 people attended meetings of the collaborations developing the open-source toolkits EPICS and TANGO. The EPICS collaboration in particular has grown since previous ICALEPCS meetings. The contributions presented at the conference showed that these two toolkits are the most widely used and are the predominant choice for many facilities. For example, EPICS has recently been selected for use at the Spallation Neutron Source (SNS) at ORNL, while the control system of the ALBA light source in Spain will be based on TANGO.

Alternative solutions employ commercial supervisory control and data acquisition (SCADA) products for control systems. This is the case, for example, at CERN, the Laser Mégajoule project and the SOLEIL synchrotron. At CERN, the cryogenics system for the LHC and the LHC experiments, among others, make extensive use of commercial SCADA systems. The combination of their use with appropriate software frameworks developed in common has largely facilitated the design and construction of these control systems. They are currently being scaled up to their final operational size – a task that has gone smoothly so far (Under control: keeping the LHC beams on track and Detector controls for LHC experiments).

Independent of the approach adopted, the controls community has focused strongly on the software-development process, taking an increasing interest in risk reduction, improved productivity and quality assurance, as well as outsourcing. The conference learned of many efforts for standardization and best practice, from the management of requirements to development, implementation and testing. Speakers from CERN, for example, explained the benefits of the adoption of the Agile design and programming methodology in the context of control-system development.

The ITER tokamak project in Cadarache, France, has taken an approach that uses a unified design to deal with the static and dynamic behaviour of subsystems. The operation of ITER requires the orchestration of up to 120 control systems, including all technical and plasma diagnostic systems. ITER will outsource a large fraction of these control systems, which will be procured “in kind” from the participating teams. Outsourcing also played a major role in the Australian Synchrotron and it involved co-operation between research institutions and industrial companies to enhance and optimize the functionality of their control-system products. Such collaboration needs the definition of strict acceptance criteria and deadlines, but it also allows outsourcing of the risk. The Mégajoule project tested its subcontracting process within a small “vertical slice”, before adapting all of the outsourcing and the integration process to the full-scale system. The Atacama Large Millimetric and Submillimetric Array has provided further lessons about the successful organization of a distributed team and integration of different objects. The project enforced a common software framework on all participating teams, and the integration process focused on functionality rather than on the subsystems.

In addition to the software frameworks for control systems, there are many plug-ins, tools and utilities under development, using, in particular, the Java language. For example, EPICS employs Java at all levels from the front-end Java input/output (I/O) controllers to the supervision layer. Java is now a top candidate for new developments, owing mostly to its productivity and portability, not only for graphical user interfaces (GUIs) and utilities but also for applications that are calculation intensive. The accelerator domain has integrated more advanced Java-related techniques successfully. SLAC, for example, has benefited from the open-source Eclipse technologies, and the Java-based open-source Spring is being deployed in the LHC accelerator control systems at CERN (Under control: keeping the LHC beams on track). However, somewhat contrarily to these common efforts, individual projects have also developed a variety of new electronic logbooks and custom GUI builders.

The flexibility and portability of Java are becoming increasingly combined with the extensible markup language XML. With interoperability in mind, the growing (and correct) usage of XML and associated technologies provides a good basis for openness, data exchange and automation, rather than simply for configuration.

An example of this openness is the adoption of industrial solutions for data management. Modern control systems have seen a rapid growth in data to be archived, together with rising expectations for performance and scalability. File-based or dedicated solutions for data management are reaching their limits, so these techniques are now being replaced by high-performance databases, such as Oracle and PostgreSQL. These databases not only record the parameters of control systems but also are used for administration, documentation management and equipment management. In addition to these well established technologies, some users have chosen ingenious approaches. For example, SPring-8 in Japan has a geographic information system integrated into its accelerator management (figure 1). The Google Maps-like system allows localizing, visualizing and monitoring of equipment in real time, and it has opened up interesting perspectives for the control systems community.

Hardware becomes soft

On the hardware side, VME equipment shows an increased use of embedded controllers, such as digital signal processors and field-programmable gate arrays. Their flexibility brings the controls software directly onto the front end, for example, as cross-compiled EPICS I/O controllers. The development of radiation hard front-ends, for example, for the Compact Linear Collider study and the LHC at CERN, have presented other challenges. Timing systems have also had to face new challenges: the LHC requires independent and asynchronous timing cycles of arbitrary duration; timing distributions, for the accelerators at SOLEIL or the Los Alamos Neutron Science Center, for example, are based on common networks with broadcasting clocks and event-driven data; and modern free-electron lasers (FELs), such as at SPring-8, depend on timing accuracies of femtoseconds to achieve stable laser beams.

FELs and light sources were the main focus of several status reports at the conference. The X-ray FEL project at SPring-8 has implemented its control system in MADOCA, a framework that follows a three-tier control model. The layers consist of an interface layer based on DeviceNet programmable logic controllers and VME crates; communication middleware based on remote procedure calls; and Linux consoles for the GUIs. The control system for the Free-electron Laser in Hamburg (FLASH) at DESY provides bunch-synchronized data recording using a novel integration of a fast DAQ system. The FLASH collaboration carried out an evaluation of the front-end crates used in the telecoms industry, which suggested that they had more reliable operation and integrated management compared with VME crates. The collaboration for the ALICE experiment at the LHC reported on progress with its control system, which is currently being installed, commissioned and prepared for operation, due to start later this year. Other status reports came from the Facility for Antiproton and Ion Research at GSI and the Diamond Light Source in the UK.

The conference concluded with presentations about the new developments and future steps in the evolution of some of the major controls frameworks. These underlined that the ICALEPCS conference not only confirmed the use of established technologies and designs, in particular EPICS and TANGO, but also showed the success of commercial SCADA solutions. Control systems have become highly developed and the conference reviewed consolidation efforts and extensions thoroughly. The social programme included a dinner with bluegrass music and an excellent tour of the SNS, the world’s most intense pulsed accelerator-based neutron source, which rounded off the meeting nicely. Now the controls community awaits the 12th ICALEPCS conference, to be held in Kobe, Japan, in autumn 2009.