A look at the vast amount of work done during the long shudown, and some of the challenges that arose.
Image credit: Michael Hoch.
For the past two years, teams from the CMS collaboration, many from distant countries, have been hard at work at LHC point 5 at Cessy in France. Their goal – to ensure that the CMS detector will be able to handle the improved performance of the LHC when it starts operations at higher energy and luminosity. More than 60,000 visitors to the CMS underground experimental cavern during the first long shutdown (LS1) witnessed scenes of intense and spectacular activity – from movements of the 1500-tonne endcap modules to the installation of the delicate pixel tracker, only the size of a portable toolbox but containing almost 70-million active sensors.
This endeavour involved planning for a huge programme of work (CERN Courier April 2013 p17). Since LS1 began, more than 1000 separate work packages have been carried out, ranging from the repairs and maintenance required after three years of operation during the LHC’s Run 1, through consolidation work for a long-term future, to the installation of completely new detector systems as well as the extension of existing ones. In addition to the many CMS teams involved, the programme relied on the strong general support and substantial direct contributions from physics and technical departments at CERN. This article, by no means exhaustive, aims to provide some insight into LS1 as it happened at point 5.
An early start
Vital contributions started as early as 2009, well before LS1 began. One example is the refurbishment by CERN’s General Services and Physics Departments of building 904 on the Prévessin site, to provide 2000 m2 of detector-assembly laboratories, which were used for the new parts of the muon detector. Another is the creation by CMS (mainly through contracts managed by CERN’s Engineering Department) of the Operational Support Centre in the surface-assembly building at point 5. This centre incorporates work areas for all of the CMS systems that had to be brought to the surface during LS1, and includes a cold-storage, cold-maintenance facility where the pixel tracker was kept until the new beampipe was fitted. There is also a workshop area suitable for modifying elements activated by collision products, which, as the LS1 story progressed, provided useful flexibility for dealing with unexpected work.
The highest-priority objective for CMS during LS1 was to operate the tracker cold
The highest-priority objective for CMS during LS1 was to operate the tracker cold. The silicon sensors of this innermost subdetector, which surrounds the LHC beampipe, must endure more than 109 particles a second passing through it, and cannot be completely replaced until about a decade from now. The damaging effects of this particle flux, sustained over many years of operation, can be mitigated by operating the sensor system at a temperature that is 20–30 °C lower than the few degrees above zero used so far. Alongside modifications to allow delivery of the coolant at much lower temperatures, a new system of humidity control had to be introduced to prevent condensation and icing. This involved sealing the tracker envelope, while making provision for a flow of up to 400 m3/h of dry gas. The system installed by CMS is a novel one at CERN: it dries air and then optionally removes oxygen via filtering membranes. The first full-scale tests took place at the end of 2013, and there was great satisfaction when an operating temperature of –20 °C was achieved stably.
However, as one challenge faded, a new one emerged immediately. On warming up, tell-tale drips of water were visible coming from the insulated bundles of pipework carrying the coolant into the detector – indication that air at room temperature and humidity had been reaching the cold pipes inside the system and forming ice. Fortunately, tests soon showed that an additional flow of dry air, injected separately into the pipework bundles, would suppress this problem. Responding to CMS’s request for help, the Engineering Department recently delivered a new dry-air plant that will make humidity suppression in the cooling distribution feasible on a routine basis, with a comfortable margin in capacity.
Another high-priority project for LS1 involved the muon detectors. A fourth triggering and measurement station in each of the endcaps was incorporated into the original CMS design, but it was not considered essential for initial operation. These stations are now needed to increase the power to discriminate between interesting low-momentum muons originating from the collision (e.g. potentially from a Higgs-boson decay) and fake muon signatures caused by backgrounds. Seventy-two new cathode-strip chambers (CSCs) and 144 new resistive-plate chambers (RPCs) were assembled across a three-year period by a typical CMS multinational team from institutes in Belgium, Bulgaria, China, Colombia, Egypt, Georgia, India, Italy, Korea, Mexico, Pakistan, Russia and the US, as well as from CERN. They were then installed as superposed layers of CSCs and RPCs on the two existing discs at the ends of the steel yoke that forms the structural backbone of CMS. Teams worked on the installation and commissioning in two major bursts of activity, matching the periods when the required detector configuration was available, and completing the job in late spring 2014.
Image credit: Michael Hoch.
A further improvement of the endcap muon system was achieved by installing new on-chamber electronics boards in the first, innermost layer of the CSCs to withstand the higher luminosity, while reusing the older electronics in one of the new fourth layers, where it is easier to cope with the collision rate. Here again, the unexpected had to be dealt with. One of the two layers had just been re-installed after months of re-fitting work, when tests revealed a potential instability caused by the accidental omission of a tiny passive electronic component. It was considered significantly risky to leave this uncorrected, so the installation teams had to go into full reverse. Working late into the evenings and at weekends to avoid interfering with previously scheduled activities, they partially extracted all 36 chambers, corrected the fault, put them back in place and re-commissioned them.
No part of the detector escaped the attention of the upgrade and maintenance teams. The modular structure of CMS, which can be separated into 13 major slices, was fully exploited to allow simultaneous activity, with as many as eight mobile work platforms frequently in use to give access to different slices and different parts of their 14 m diameter. Multiple maintenance interventions on the five barrel-yoke wheels restored the number of working channels to 99.7% – a figure not seen since 2009, just after installation. Similar interventions on the CSC and RPC stations on the endcap disks were also successful, with the few per cent that had degraded over the past few years restored completely. In addition, to improve maintainability, some key on-board electronics from the barrel part of the muon system was moved from the underground experimental cavern to the neighbouring service cavern, where it will now remain accessible during LHC operation. All of the photo-transducers and much of the on-detector electronics of the hadron calorimeter (HCAL) are to be replaced over the next few years, and a substantial part of this work was completed during LS1. In particular, photo-transducers of a new type were installed in the outer barrel and forward parts of the sytem, which will lead to an immediate improvement in performance.
The rate of proton–proton collisions will be five times higher
The need for some work streams was completely unforeseen until revealed by routine inspection. The most notable example was the discovery of a charred feed-through connector serving the environmental-screen heaters of one of the two preshower systems for the electromagnetic calorimeter (ECAL). Full diagnosis (under-rated capacitors) and subsequent repair of both preshower systems required their removal to the surface, where a semi-clean lab was created at short notice within the Operational Support Centre. The repairs and re-installation were a complete success, and the preshower system has been re-commissioned recently at its planned operating temperature of –8 °C.
The CMS consolidation programme had also to prepare the infrastructure of the experiment – originally designed for a 10-year operating lifetime – for running well into the 2030s. LHC operating periods lasting around three years will be interleaved with substantial shutdowns of one to two years in length. Moreover, the rate of proton–proton collisions will be five times higher, and the integrated number of collisions (ultimately) 10 times higher, than the original design goal.
Key adaptations were made during LS1 to address redundancy in the power and cryogenics systems, to extend the predicted lifetime of the one-of-a-kind CMS magnet. Further measures for protection against power glitches were implemented through an extension of the detector’s short-term uninterruptible power supply. Changes to the detector cooling included modifications for greater capacity and redundancy, as well as the addition of a new system in preparation for the upcoming upgrade of the pixel tracker, based on two-phase (evaporating liquid) carbon dioxide. This technology, new for CMS, involved the installation of precision-built concentric vacuum-insulated feed and return lines – difficult-to-modify structures that have to be made extremely accurately to ensure proper integration with the constricted channels that feed services into the apparatus. These changes presented challenges for the CMS Integration Office, where the “compact” in CMS was defended vigorously every day in computer models and then in the caverns.
The most massive change to the structure of the experiment was the addition of the new 125-tonne shielding discs
New detectors were not the only large-scale additions to CMS. The most massive change to the structure of the experiment was the addition of the new 125-tonne shielding discs – yoke endcap disc four (YE4) – installed outside of the fourth endcap muon station at either end of the detector. Each shielding disc, 14 m in diameter but only 125 mm thick, was made of 12 iron sector casings. Following manufacture and pre-assembly tests in Pakistan, these discs, whose design and preparation took five years, were disassembled for shipping to CERN and then re-assembled on the Meyrin site, where they were filled with a special dense (haemetite) shielding concrete, mixed for this specific application by CERN’s civil engineers. Loaded with a small percentage of boron, this concoction will act as a “sponge” to soak up many of the low-energy neutrons that give unwanted hits in the detector, and whose numbers will increase as the LHC beam intensities get higher.
The YE4 discs, transported in sectors to point 5, were the first slices of CMS to be assembled underground – all of the existing major elements had been pre-assembled on the surface and lowered into the underground cavern in sequence (CERN Courier July/August 2006 p28). In the original concept, the YE4 discs could be separated from the supporting YE3 only by driving the whole endcap system back to the cavern headwall, where YE4 could be unhooked and supported. Because all of the other slices of the CMS “swiss roll” can be displaced from one another to give access to the detectors sandwiched in between, it was decided late in the project – in fact, after assembly had already started – to equip each YE4 shielding disc with air pads and a system of electric screw-jacks. This would allow the YE4 disc to separate from the supporting neighbour disc (YE3) by up to 3.7 m without the necessity to move it to the headwall – a major operation. In fact, one so-called “push-back system” was used immediately after assembly of the YE4 disc, to permit installation of RPCs with the endcaps partially closed. This maintained the rapid-access modularity that was a core feature of the CMS design (CERN Courier October 2008 p48).
The final change was at the heart of CMS, in preparation for the installation during the LHC’s year-end technical stop of 2016–2017 of an upgraded pixel tracker – the closest physics detector to the collision point. The 0.8-mm-thick central beampipe used during Run 1, with an outer diameter of 59.6 mm, was replaced by a similar one of 45-mm outer diameter and, like the first one, made of beryllium, to be as transparent as possible to particles emanating from the LHC collisions. The narrower beampipe will allow the first layer of the new pixel tracker to be closer to the collision point than before. This geometrical improvement, combined with an additional fourth layer of sensors, will upgrade the tracker’s ability to resolve where a charged particle originated. When running under conditions of high pile-up in Run 2 and Run 3 – that is, with many more protons colliding every time counter-rotating bunches meet at the centre of CMS – the disentangling of which tracks belong to which collision vertices will be crucial for most physics analyses.
Image credit: Michael Hoch.
The delicate operations of removing and replacing the beampipe – requiring the detector to be open fully – are possible only in a long shutdown. The new beampipe, designed jointly with CERN’s Technology Department, which procured and prepared it on behalf of CMS, was installed in June 2014. Its installation was followed immediately by vacuum pumping, combined with heating (“bake-out”) to more than 200 ºC, to expel gas molecules attached to the chamber walls. This ensured that the operating pressure of around 10–10 mbar would be possible – and achieved eventually. Following the bake-out of the new central beampipe, several mechanical tests were made to ensure that the upgraded pixel tracker can be installed in the limited time window that will be available in 2016–2017.
It is probable that a proverb exists in every language and culture involved in CMS, warning against relaxing before the job is finished. In mid-August 2014, the end of the LS1 project seemed to be on the horizon. The beampipe bake-out was being completed and preparations for the pixel tracker’s re-installation were underway, so many team members took the opportunity for a quick summer holiday. Then, their mobile phones began to buzz with reports of the first indications of a severe fault found in pre-installation tests of the barrel pixel system, which had been removed only to allow the change of beampipe. About 25% (around 50) of the modules in one quadrant were not responding. By the end of August, the half-shell containing the faulty quadrant had been transported to its makers at the Paul Scherrer Institute (PSI) for detailed investigation.
On 5 September, the diagnostics revealed that the reason for failure was electro-migration-induced shorts between adjacent bond pads of the high-density interconnect – a flexible, complex, multilayer printed circuit used to extract the signals. An investigation showed that the most likely origin was a brief and inadvertent lapse in humidity-control protocols in the course of routine calibration exercises many months earlier, when the pixel system was up in the surface laboratory. By 18 September, a comprehensive strategy of replacement and repair had been worked out by the PSI team. Because this required purchasing new components and restarting the production of detector modules, the revised schedule foresaw the detector being back at CERN by the end of November, with installation planned for around 8 December, almost exactly two months later than intended originally.
A new end game
At this late stage, with insufficient contingency remaining in the baseline schedule to accommodate the delay, it was decided to change radically the end-game sequence of the shutdown. Instead of waiting for the repair of the pixel tracker, CMS was closed immediately to conduct a short magnet-test, to identify any problems that otherwise would not have appeared until the final closure for beam. After finishing the remaining work on the bulkhead seal that allows the tracker to be operated cold, this sequence of closing the detector, testing the magnet and then re-opening CMS became the critical path for two months, with the remaining upgrade activity being postponed or re-arranged around the new schedule. The new sequence implied unexpected tight deadlines for several teams – particularly those working on the magnet and the forward region – and a massive extra workload for the heavy-engineering team. The additional closing and opening sequence required 36 single movements of heavy discs, and 16 insertions and removals of the heavy-raiser platforms that support the forward calorimeters at beam height. A concerted and exceptional effort resulted in the magnet yoke being closed by mid-October, and both forward regions being closed and ready for magnetic field by 6 November.
The closing of the experiment, just in time for first beam in 2015, brought the saga of LS1 to a happy ending.
The following day, the magnet was ramped to 1 T and then discharged. This sequence allowed yoke elements to settle, and also verified that the control and safety systems performed as expected. By 10 November, enough liquid helium had been accumulated for 36 hours of operation at full field, and the test programme resumed. However, at 2.4 T, the main elevator providing underground access stopped working, owing to some field-sensitive floor-level sensors having been installed mistakenly during routine maintenance. After reducing the field temporarily to allow personnel to leave the underground areas, the ramp-up continued, reaching the working value of 3.8 T at around 7.00 p.m., demonstrating that the magnet’s upgraded power and cryogenics system worked well. Despite the rapid endcap-yoke closure with only approximate axial alignment, the movements under the magnetic forces of the endcap discs (including the new YE4s) and the forward systems were well within the ranges observed previously, although specific movements occurred at different field values. The new beampipe support system and the new phototransducers of the HCAL and beam-halo monitors were shown to be tolerant to the magnetic field. Most importantly, the environmental seal around the tracker and the new dry-gas injection system functioned well enough in the magnetic field to allow tracker operation at –20 °C. The top-priority task of LS1 could therefore be declared a success.
Following this, the opening of the detector was a race against time to meet the target of installing the barrel and forward pixel trackers, and enclosing them in a stable environment before CERN’s 2014 end-of-year closure. This was achieved successfully, providing a fortuitous “dry run” of what will have to be done during the year-end stop of 2016–2017, when the new pixel tracker will be installed. Following a thorough check and pre-calibration of the pixel system, the last new elements of CMS in the LS1 project – upgraded beam monitors and the innovative pixel luminosity telescope (CERN Courier March 2015 p6) – were installed by the end of the first week of February 2015.
The closing of the experiment, just in time for first beam in 2015, brought the saga of LS1 to a happy ending. It is time to celebrate with the collaboration teams, contractors and CERN technical groups, who have all contributed to the successful outcome. The imminent start of Run 2 now raises the exciting prospect of new physics, but behind the scenes preparations for the next CMS shutdown adventure have already begun.
