Most particle-physics conferences emphasise the results of physics analyses. The PHYSTAT series is different: speakers are told not to bother about the actual results, but are reminded that the main topics of interest are the statistical techniques used, the resulting uncertainty on measurements, and how systematics are incorporated. What makes good statistical practice so important is that particle-physics experiments are expensive in human effort, time and money. It is thus very worthwhile to use reliable statistical techniques to extract the maximum information from data (but no more).
Origins
Late in 1999, I had the idea of a meeting devoted solely to statistical issues, and in particular to confidence intervals and upper limits for parameters of interest. With the help of CERN’s statistics guru Fred James, a meeting was organised at CERN in January 2000 and attracted 180 participants. It was quickly followed by a similar one at Fermilab in the US, and further meetings took place at Durham (2002), SLAC (2003) and Oxford (2005). These workshops dealt with general statistical issues in particle physics, such as: multivariate methods for separating signal from background; comparisons between Bayesian and frequentist approaches; blind analyses; treatment of systematics; p-values or likelihood ratios for hypothesis testing; goodness-of-fit techniques; the “look elsewhere” effect; and how to combine results from different analyses.
Subsequent meetings were devoted to topics in specific areas within high-energy physics. Thus, in 2007 and 2011, CERN hosted two more meetings focusing on issues relevant for data analysis at the Large Hadron Collider (LHC), and particularly on searches for new physics. At the 2011 meeting, a whole day was devoted to unfolding, that is, correcting observed data for detector smearing effects. More recently, two PHYSTAT-ν workshops took place at the Institute for Physics and Mathematics of the Universe in Japan (2016) and at Fermilab (2017). They concentrated on issues that arise in analysing data from neutrino experiments, which are now reaching exciting levels of precision. In between these events, there were two smaller workshops at the Banff International Research Station in Canada, which featured the “Banff Challenges” – in which participants were asked to decide which of many simulated data sets contained a possible signal of new physics.
The PHYSTAT workshops have largely avoided having parallel sessions so that participants have the opportunity to hear all of the talks. From the very first meetings, the atmosphere has been enhanced by the presence of statisticians; more than 50 have participated in the various meetings over the years. Most of the workshops start with a statistics lecture at an introductory level to help people with less experience in this field understand the subsequent talks and discussions. The final pair of summary talks are then traditionally given by a statistician and a particle physicist.
A key role
PHYSTAT has played a role in the evolution of the way particle physicists employ statistical methods in their research, and has also had a real influence on specific topics. For instance, at the SLAC meeting in 2003, Jerry Friedman (a SLAC statistician who was previously a particle physicist) spoke about boosted decision trees for separating signal from background; such algorithms are now very commonly used for event selection in particle physics. Another example is unfolding, which was discussed at the 2011 meeting at CERN; the Lausanne statistician Victor Panaretos spoke about theoretical aspects, and subsequently his then student Mikael Kuusela became part of the CMS experiment, and has provided much valuable input to analyses involving unfolding. PHYSTAT is also one of the factors that has helped in raising the level of respectability with which statistics is regarded by particle physicists. Thus, graduate summer schools (such as those organised by CERN) now have lecture courses on statistics, some conferences include plenary talks, and books on particle-physics methodology have chapters devoted to statistics. With the growth in size and complexity of data in this field, a thorough grounding in statistics is going to become even more important.
Recently, Olaf Behnke of DESY in Hamburg has taken over the organisation and planning of the PHYSTAT programme and already there are ideas regarding having a monthly lecture series, a further PHYSTAT-ν workshop at CERN in January 2019 and a PHYSTAT-LHC meeting in autumn 2019, and possibly one devoted to statistical issues in dark-matter experiments. In all probability, the future of PHYSTAT is bright.
