In all experiments for detection of Weakly Interacting Massive Particle (WIMP) dark matter, it is essential to develop a function that can distinguish events caused by WIMP candidates from those caused by background radiation. Manually developing such a classifier is challenging, time-consuming, and necessitates detailed physical modeling.
Machine learning has the potential to automate this task and accelerate experimentation, in addition to detecting patterns that humans cannot. However, impure calibration data adversely affects training of models, and unusual detector topologies make data challenging to process.
I have developed novel machine learning algorithms that perform significantly better at this task than previous methods, in the PICO-60 and DEAP-3600 experiments. These results should allow accelerated iteration for teams working on these experiments, while improving accuracy and retaining reliability. Additionally, they promise to generalize to future WIMP experiments.
I approached the 10% calibration data impurity present in the PICO-60 bubble chamber experiment by developing semi-supervised learning algorithms that synthesize new labels for training data, improving accuracy from 80.7% to 99.2%. Additionally, I investigated previously unexplored input data formats and neural network architectures.
DEAP-3600 is a spherical detector with light sensors on the surface. I present new algorithms that can process this unusual topology: a cylindrical projection system, and a new type of CNN that processes arbitrary geometric data. This reduced the rate of false positives from 91.0% to 75.7%.
I have lead-authored an academic paper on my PICO-60 research, which the PICO collaboration has reviewed and approved. It will shortly undergo peer review.
Can I develop novel machine learning techniques that are more accurate than existing methods at classifying background radiation in the PICO-60 and DEAP-3600 experiments for WIMP dark matter detection?
One of the most significant fields of research in physics today is dark matter research. A conclusive answer either way would have the potential to revolutionize our understanding of the universe at a fundamental level.
However, developing a detector for dark matter, in the form of Weakly Interacting Massive Particles (WIMPs), is difficult. Even if one develops a highly sensitive apparatus such as a bubble chamber or photon detector, there is still background radiation, such as alpha particles, that have properties similar to expected dark matter particles.
Developing a conventional classifier to separate dark matter from background radiation is possible. However, it usually relies on detailed physical modeling of the detector, and manual optimization, both of which are time-consuming and must be reworked whenever the experiment changes.
Machine learning is a potential solution to this; calibration data can be used to train a model to separate particle types. However, calibration data is frequently impure, which often leads to overfitting and poor accuracy. Additionally, in many experiments, unusual detector formats make it challenging to find an appropriate machine learning model. I hope to resolve these challenges.
Based on my research, I expect to be able to develop a machine learning model that classifies more accurately than current methods, by applying original semi-supervised learning techniques and new systems based on convolutional neural networks.
There have been many high-profile, well-documented dark matter experiments in the past several years; PICO-60 and DEAP-3600 are two experiments for which I was able to get access to data, by reaching out to research physicists.
The PICO-60 experiment (Amole et al.) is a bubble chamber containing superheated C3F8. When a particle strikes an atom in the liquid, a disturbance is created and a bubble forms. The primary form of background radiation is alpha particles, emitted by nuclear decays inside the detector. A conventional classifier for alpha particles, known as the Acoustic Parameter (AP), was developed by the PICO-60 collaboration and verified based on physical modeling and empirical modification to distinguish accurately between alpha particles and nuclear recoils, which means it can be used to verify machine learning algorithms.
Because of the benefits of automation, preliminary experimentation has previously been done by Amole et al. with machine learning for discrimination of WIMP-like particles from alpha particles, yielding 80.7% accuracy. However, 10% of calibration data consists of impurities, likely causing overfitting and thus reducing accuracy. The input to AP, as well as the neural network, is an 8-band Fourier transform of audio captured by piezoelectric microphones in the detector.
The DEAP-3600 detector (Amaudruz et al.) contains 3.3 tonnes of liquid argon, which emits photons when struck by a particle. The photons are detected by 255 extremely sensitive light detectors (photomultipliers or PMTs) placed around the acrylic vessel containing the liquid argon. Based on the counts and timings of photons that reach each of the PMTs, it is possible to determine the energy and location of any event that occurs in the body of the detector. However, alpha events that occur in the neck are very difficult to isolate, because they overlap with the apparent characteristics of expected WIMP candidates.
It is impractical to create significant amounts of clean calibration data in the DEAP-3600 detector. Thus, data from a Monte Carlo simulation (which was benchmarked using real-world calibration data) is used instead.
A conventional classifier was previously developed by DEAP-3600 physicists; it was able to remove 99.6% of neck events, at the cost of 91.0% of hypothetical (simulated) WIMP events. Machine learning has not been applied in the past. Based on its applications in PICO-60, I am hopeful I can make a similar difference for DEAP-3600.
I expect I can make a difference to these two experiments, in part, because of the precedent set by previous work using machine learning in experimental physics; it is instrumental to efficiently classifying the Higgs boson in the Large Hadron Collider, an effort summarized well by Guest et al.
I believe this work is beneficial to society because it helps enable experiments that explore the fundamental nature of the universe. Not only does it help humanity understand who we are and what we are made of, but previously obscure physics research has proven instrumental to life-saving inventions many times in the past: for instance, according to Minervini et al., superconductors are essential to magnetic resonance imaging.
For PICO-60, I developed and compared two sets of classification algorithms:
In DEAP-3600, the key challenge is the aforementioned unusual detector format: a sphere tiled with a hexagonal lattice of PMTs. While it is fundamentally an image, conventional CNNs are intended only for flat rectangular images. I attempted to solve the problem in three different ways:
I ran grid searches (exhaustive searches of network hyperparameters) to optimize each algorithm. These took up to 10 days of compute time.
I implemented all of my algorithms in Python, using Keras, TensorFlow, and NumPy. I used Matplotlib for data visualization.
My code is public at https://github.com/brendon-ai/dark-matter.
I worked independently with minimal support at SNOLAB and my home.
All performance statistics cited refer to performance on a randomly selected validation set composed of examples not used for training. Each set of network hyperparameters tested in a grid search was trained and tested multiple times (see below), with a differently randomized split between training and validation data. Performance statistics are averaged over the multiple training runs.
The below spreadsheet quantifies the performance of every configuration tested.
Summary of the PICO analysis:
The three architectures I tested for neck alpha identification in DEAP-3600 were evaluated based on their ability to reduce the rate of false positives (potential WIMP candidates misidentified as neck alphas) compared to previous results from conventional methods. Only models with the same (or lower) 0.4% false negative rate were considered.
Below are the predictions of the cylindrical projection CNN.
The below table quantifies every configuration tested.
Summary of the DEAP analysis:
Novel machine learning algorithms were developed for particle classification in the PICO-60 and DEAP-3600 dark matter experiments, and found to exceed the performance of previous research.
In PICO-60, a new semi-supervised learning algorithm called gravitational differentiation was found to improve classification accuracy from the 80.7% reached in previous machine learning studies, to 99.2%. Other supervised and semi-supervised learning algorithms were also explored.
In DEAP-3600, application of a cylindrical projection with a 2D CNN to process the detector's spherical topology was found to reduce the proportion of false positives from the 91.0% previously reached with a conventional classifier, to 75.7%, while keeping the rate of false negatives at 0.4%.
These results answer my hypothesis in the affirmative. Indeed, semi-supervised learning for PICO-60 performed better than both the previous best result with machine learning, and my own supervised learning work. This significantly improves the accuracy that can be obtained quickly and without any manual optimization, allowing the team working on the experiment to iterate more quickly without the overhead of developing a conventional classifier.
The reduction in the false positive rate for DEAP-3600 provides evidence that machine learning is capable of improving the efficiency of this experiment. This may reduce the operation time required to collect sufficient data pointing towards the existence or non-existence of WIMP dark matter.
In general, the classifiers developed during this study demonstrate great promise for machine learning in dark matter detection. Fundamentally, the problems I have solved with respect to PICO-60 and DEAP-3600 are not specific to these two experiments; they are common. Thus, my algorithms should be applicable in the broader field of dark matter detection. I hope to explore more experiments in the future!
My semi-supervised and supervised classifiers for PICO-60 are immediately applicable, because they were optimized using real-world data collected from calibration sources and background radiation. In future iterations of PICO, application should be straightforward once calibration data has been collected.
One current limitation relates to a so-called "position correction", in which the amplitude of audio data is normalized based on the position of the bubble. It is not currently possible to apply this correction to any audio format other than the 8-band Fourier transform. This is a possible reason for the weaker performance of models trained on the high-resolution Fourier transform and the raw waveform. Future work for PICO-60 should thus focus on generalizing this correction to other audio formats, to confirm or refute this conjecture.
At the moment, data from a simulated DEAP-3600 detector is used for validation of all machine learning and conventional particle classifiers. A Monte Carlo simulation is always an approximation of real-world behavior, and not an absolute proof. I have not yet used real-world calibration data because it is currently too limited in quantity (approximately 30 usable events). Long-term future work should thus focus on evaluating how well machine learning classifiers generalize to real-world calibration data.
I am a 15-year-old high school student in Sudbury, Ontario. Ever since I grew up reading about astronomy and physics, I have always dreamed of contributing to the quest to understand the nature of the universe we live in!
My interests in machine learning have been sparked by two of my greatest inspirations: Andrej Karpathy and Geoffrey Hinton. I greatly respect not only their tremendous contributions to the field, but also their dedication to ethical practices and improving the lives of others.
I have been working on machine learning projects since 2016. Before exploring dark matter research, I developed a real-world autonomous vehicle based on an electric go-kart. With this project, I was incredibly honored to win Best Project at the Canada-Wide Science Fair, and most recently First Prize at the European Union Contest for Young Scientists 2018.
After high school, I hope to go to university for software engineering. My long-term aspiration is to work in the artificial intelligence/machine learning industry of Silicon Valley. I am inspired by the cutting-edge AI research, and I would love to be a part of that someday.
Winning a prize at Google Science Fair would be an incredible inspiration to work hard and dig deeper into the fascinating field of dark matter research! I would absolutely love the opportunity to meet a diverse group of researchers in physics and artificial intelligence; it would be tremendously valuable in allowing me to explore places where I can make a difference in other experiments and laboratories.
During the summer, I worked at the SNOLAB neutrino/dark matter observatory in Sudbury, Ontario. I did not enter the underground laboratory; I stayed in the surface building.
As this is a software project, at no point did I put myself or anyone else at any risk. All contact with radioactive sources was done by professionals years prior to this project.
My supervisor was Dr. Nigel Smith: firstname.lastname@example.org
During the summer of 2018, I worked at SNOLAB, a dark matter and neutrino observatory in Sudbury, Ontario. Thanks very much to Dr. Nigel Smith for generously providing me this opportunity! Also, thanks to Ken Clark, Carsten Krauss, Scott Fallows, and Pierre Gorel for introducing me to the PICO-60 and DEAP-3600 experiments.
While I was at SNOLAB, I received minor guidance from the above research physicists on the following subjects:
I did not receive any guidance on the following subjects, completing them entirely independently:
Of course, thanks to my family and friends for supporting me throughout this challenging, at times frustrating, but certainly worthwhile endeavor.
While at SNOLAB, I did not interact with any physics equipment. I was granted access to the Graham compute cluster located at the University of Waterloo. This helped to make my computationally expensive grid searches practical to run within a reasonable period of time, especially for complex architectures such as the topological CNN.
J. Kiefer and J. Wolfowitz. “Stochastic Estimation of the Maximum of a Regression Function”. In: Ann. Math. Statist. 23.3 (Sept. 1952), pp. 462–466. doi: 10.1214/aoms/1177729392. url: https://doi.org/10.1214/aoms/1177729392.
J. M. Hollander, I. Perlman, and G. T. Seaborg. “Table of Isotopes”. In: Rev. Mod. Phys. 25 (2 1953), pp. 469–651. doi: 10.1103/RevModPhys.25.469. url: https://link.aps.org/doi/ 10.1103/RevModPhys.25.469.
Rene Brun and Fons Rademakers. “ROOT - An Object Oriented Data Analysis Framework”. In: AIHENP’96 Workshop, Lausanne. Vol. 389. 1996, pp. 81–86.
Gerard Jungman, Marc Kamionkowski, and Kim Griest. “Supersymmetric dark matter”. In: Phys. Rept. 267 (1996), pp. 195–373. doi: 10 . 1016 / 0370 - 1573(95 ) 00058 - 5. arXiv: hep - ph/9506380 [hep-ph].
Eric Jones, Travis Oliphant, Pearu Peterson, et al. SciPy: Open source scientific tools for Python. 2001–. url: http://www.scipy.org/.
Travis Oliphant. A guide to NumPy. 2006.
J. D. Hunter. “Matplotlib: A 2D graphics environment”. In: Computing In Science & Engineering 9.3 (2007), pp. 90–95. doi: 10.1109/MCSE.2007.55.
F. Pedregosa et al. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research 12 (2011), pp. 2825–2830.
Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: CoRR abs/1412.6980 (2014). arXiv: 1412.6980. url: http://arxiv.org/abs/1412.6980.
Stefan van der Walt et al. “scikit-image: image processing in Python”. In: PeerJ 2 (June 2014), e453. issn: 2167-8359. doi: 10.7717/peerj.453. url: http://dx.doi.org/10.7717/peerj.453.
Martin Abadi et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. 2015. url: http://tensorflow.org/.
Francois Chollet et al. Keras. https://keras.io. 2015.
Wei Dai et al. “Very Deep Convolutional Neural Networks for Raw Waveforms”. In: CoRR abs/1610.00087 (2016). arXiv: 1610.00087. url: http://arxiv.org/abs/1610.00087.
C. Amole. PhD thesis. Queen’s University, 2017.
C. Amole et al. “Dark Matter Search Results from the PICO−60 C3F8 Bubble Chamber”. In: Phys. Rev. Lett. 118 (25 2017), p. 251301. doi: 10 . 1103 / PhysRevLett . 118 . 251301. url: https://link.aps.org/doi/10.1103/PhysRevLett.118.251301.
J. Minervini et al. "Recent advances in superconducting magnets for MRI and hadron radiotherapy: an introduction to 'Focus on superconducting magnets for hadron therapy and MRI'." In: Superconductor Science and Technology. url: http://iopscience.iop.org/article/10.1088/1361-6668/aaa826.
D. Guest et al. "Deep Learning and its Application to LHC Physics." arXiv: 1806.11484. url: https://arxiv.org/abs/1806.11484.
P.-A. Amaudruz et al. "First results from the DEAP-3600 dark matter search with argon at SNOLAB." In: Phys. Rev. Lett. 121, 071801 (2018). doi: 10.1103/PhysRevLett.121.071801. arXiv: 1707.08042. url: https://arxiv.org/abs/1707.08042.