Using unsupervised learning to detect broken symmetries, with relevance to searches for parity violation in nature.
Authors: Christopher Gorham Lester
Venue: Accepted by TMLR ()

Abstract: Testing whether data breaks symmetries of interest can be important to many fields. This paper describes a simple way that machine learning algorithms (whose outputs have been appropriately symmetrised) can be used to detect symmetry breaking. The original motivation for the paper was an important question in Particle Physics: "Is parity violated at the LHC in some way that no-one has anticipated?" and so we illustrate the main idea with an example strongly related to that question. However, in order that the key ideas be accessible to readers who are not particle physicists but who are interesting in symmetry breaking, we choose to illustrate the method/approach with a 'toy' example which places a simple discrete source of symmetry breaking (the handedness of human handwriting) within a idealised particle-physics-like context. Readers interested in seeing extensions to continuous symmetries, non-ideal environments or more realistic particle-physics contexts are provided with links to separate papers which delve into such details.

Published in Transactions on Machine Learning Research (10/2022)
Using unsupervised learning to detect broken symmetries, with relevance to searches for parity violation in nature.
Christopher G. Lester' and Rupert Tombs'
Cavendish Laboratory, University of Cambridge, CB3 OHE, United Kingdom
Reviewed on OpenReview:
Abstract
Testing whether data breaks symmetries of interest can be important to many fields. This paper describes simple way that machine learning algorithms (whose outputs have been appropriately symmetrised) can be used to detect symmetry breaking. The original motiva- tion for the paper was an important question in Particle Physics: "Is parity violated at the LHC in some way that no-one has anticipated?" and so we illustrate the main idea with an example strongly related to that question. However, in order that the key ideas be accessible to readers who are not particle physicists but who are interesting symmetry breaking, we choose illustrate the hatbho//ppprachh with 'toy' example which places a simple discrete source of symmetry breaking (the handedness of human handwriting) within ide- context. Readers interested in seeing extensions to continuous symmetries, non-ideal environments more realistic particle-physics contexts are provided with links to separate papers which delve into such details.
alised
1 Introduction
On the 18th June 2021, after more than a docade studying collision data from the Large Hadron Collider (LHC), the ATLAS Collaboration (The ATLAS Collaboration 2008) submitted for publication its 1000 paper. Despite that huge body of work, not single of those papers (and not single one of the papers of any other LHC collaboration then since) has yet looked for new mechanisms of overt' parity violation.
The main reason for this omission is that there are few theoretical models which predict forms of overt parily violation that the LHC could see given its unpolarised collisions, and so without a steer from theory to guide the construction of specific analysis strategies, view seems to have formed that progress in this area is either waste of time or simply impossible to perform in an efficient manner.
The purpose of this paper is to dispel that myth but to do in toy/idealised environment so that the paper is more accessible to readers in other fields who might also be looking to detect symmetry violation. By using the handedness of human handwriting as our source of symmetry breaking we intend to emphasise that the method does not. require potential sources of symmetry violation to have be known about in advance and simulated - the method processes only recorded data without additional simulation input. Readers who would like to see the framework put into use in 'real' particle physics context are encouraged to read a more detailed sister paper Lester et al. (2022b). Extensions to continuous symmetries and non-ideal environments be found in different sister paper Tombs & Lester (022).. Eschewing such details, the present paper hopes that its decision to motivate the key idea via test for discrete broken symmetry (parity) in an idealised environment will enlarge the potential readership.
The Standard Model of Particle Physics has been known violate parity since the 1950s. The discovery by Wu et al, 1957 relied the observation that beta particles emitted from certain nuclci were found to be preferentially emitted auti-parally to the initial state nuclear spins of those nuclel observation which only admits explanations which violate parity. It was not necessary the underlying (parity violating) mechanism leading those observations have been known understood for

---

Published in Transactions on Machine Learning Research (10/2022)
(1)
2
R
R
D
Q 2
REPIN 1IUTT
Zarving MaNT 2-.03
Figure 1: This figure aims to show how transformations of a 'real' three- dinensiimensoonll collider calorimeter (top row) correspond to induced transformations in the unrolled two-dimensional (n, d)-space in which those deposits are often represented. The first column shows deposits that would be typical of five jet event. The second column shows easier-to-visualise set of deposits resembling letter 'R' The other three colunns show how the deposits of the first 'R' would move around if affected by (i) an arbitrary rotation about the z-axis, (ii) a rotation of 180 degrees about the r-axis and (iii) a parity inversion.
2 Method and Principles
The symmetry we use to illustrate the method is 'parity 2 the symmelry which is its own inverse and which interchanges any object x with its mirror image Px. In physics and chemistry parity is usually thought of as a spatial operation acting on three dimensional structures æ such as collider events or molecules. Parities can, however, act on many other things In our illustration each object will be a small image containing information about collision. A function f mapping the input space to one on which concept of negation exists will be termed 'parity odd' if and only if f(z) - (Pxz) for all
The core principles underlying the method which this paper proposes for making for general parity-violation detections are: (i) that (by definition!) a parity-odd function f(x) can only have an asymmetric distribution under parity if the dataset on which it has been evaluated contains some elements 21, I2, which differ in frequency from their parity-flipped partners Pæi, Pæz, ...; and that therefore (ii) the observation of any statistically significant asymmetry in any parity-odd variable when evaluated on real dataset is bona fide evidence for parity violation in that dataset; and so (iii) the goal of the LHC collaborations [glossing over some important real-world considerations which are mentioned later] should be to create parity-odd functions f(x) which, after optimisation for asymmetry on a 'training' part of a real dataset, are then tested for asymmetry on other independent portions of the same dataset.
Any function g(x) can be trivially made into a parity-odd function f(x by antisymmetrisatton::: f(x) = g(x) - (Pxx), and every parity-odd function (xx can be so-constructed. Thus goal of 'principle (iil) above can be met by using any of the latest tools of machine learning provide diverse functions g(x) which can be optimised for the asymmetries they evoke in fx) on the training portion of a dataset of interest. The power in the method comes from the marriage between (a) the capacity of machine learning algorithms to
ekc-out the best g(z) for given domain or datasct, and (b) the symmetry used to build f(x) from g(z) (imposed by hand rather than learned!) which guarantees that any asymmetry seen in a test-set evaluation is interesting.
that discovery of parity violation to have been declared. It is signs such those signs which can only be explained away parity violating (as opposed to being effects which could be parity violating but which could also, in principle, be accounted for by alteruative parity conserving explanations) which call signs of 'overt' parity violation.
"More general symmetrics (including continuous and nou-binary symmetrics) dealt with in more complex sister paper
Tombs & Lester 2022
"Arithmelic negation on the line perhaps one of the simplest parities imaginable.
2

---

Published in Transactions on Machine Learning Research (10/2022)
For tests of parity in particle physics this process is genera 4 avoids all first-order use of Monte Carlo, and (like the original discovery of parity violation by Wu et al. (1957)) requires no knowledge of the parity violation mechanism to reach its conclusion.
We later choose to illustrate the above principle in the simplest way we can imagine:
by creating a neural-ne
5 g(x) which acts on collisions T,
by (enti)symmetrising the output of ((x) into parity-odd function f(x) using f(x) = - ((t)) and then
training the net g(x) to maximise the significance of the distance between the mean of f (() (when evaluated on test dataset) and zero
This illustration relics on the fact that a symmetric distribution with a mean will have a mean of zero, and so statistically significant non-zeroness in the mean is measure of asymmetry""
Unusual features of this core idea (as distinct from its implementation in our particular illustration)
It must be emphasised how different this proposed use of machine learning is that most frequently seen in particle physics. There are many ways one could illustrate this, but perhaps the most casy one to highlight is that the training step is (lechnically, though not in any practical sense!) optional. If the training step were entirely omitted, the net g(x) would be in a random state but the resulting f(x) would still be parity-odd. If the evaluation of this f(x) on the test set were to yield a statistically significant asymmetry by a measure fixed in advance (e.g. a non-zero mean) then that asymmetry would be valid evidence for parity violation! In practice, of course, the training step is very important. Training cannot be ignored. Without, training there is almost no chance that the net g(x) will have the right structure to generate an asymmetry in f(x), even on parity-violating dataset. But the point we wish to stress is that the training process exists not to make the discovery of carity-violation possible, but only to make it much more likely that discovery of parity-violation can be made rather than missed.
For related reasons 'over-training does not have the same negalive overlones it would normally have at least for experimental physicists whose greatest concern is avoiding a false positive (a claimed discovery of a symmetry violation when it does not really exist). The possibility of over-training still exists but it cannot lead to an accidental dis-viscovery of parity-violation as there is simply no concept of 'training that can mislead', only of 'more useful' or 'eess useful' training. For discovery of parity violation all that matters is that the appropriate net symmetrisation (f(z)) manages to show statistically significant asymmetry the test data. If it docs, it's good net, however it got there. The worst that can happen is that in training the net aends to entirely the wrong things (all unrelated to parity) and so does not know how to generate an asymmetry in the test-set (or a mean away from zero in our illustrative example) even if a better-trained net could have done so. For example: if the training resulted in g(x) evaluating to a constant 's' on every element of the training set, and ended up evaluating to the constant 't' on everything else, then the possible output of f(x) would be in the set. 5 {t- 8, 0, s f). In the case that L, this set would contain only single element (S {(}) and so f(x) would be incapable of showing asymmetry any dataset, making false positives impossible. In the case that 8 + I, the set S would contain three elements: (- 8, 0 and -.
A lengthy proof of sufficiency regarding the use of parity-odd variahles is presented in a sister paper [Lester 2021ap.
PIt is a shame that we have not used a generative adversarial network, for if we had we could have called our paper a tabloid newspaper title 'Stressed GANs snag desserts' Such a title would have been ideal given its symmetry properties.
"As will be seen later, this is a mean which has been suitably normalised so as to prevent the network gaining by (say) simply multiplying the net final output by number greater than one.
"Note that our decision to use a neural-net rather than some other function approximator, our decision to optimise for deviations of the mean of that parity-odd variable from zero rather than to optimise for some fancier loss function, are all unashamedly likely to be non-optimal. The sole aim of the paper is not to show which fancy loss function or network is best, but is rather to demoustrate that general searches for parity violation
be accomplished at all and with relative ease. *During review it was suggested that 'over-fitting' may have be more standard term in the ML field for what we have referred to 'over-training' As non-spocialists we would happily use either term.
9 " common reaction the authors have experienced when describing their proposal newcomers 'yy gosh! Are you not proposing a mechanism maximising over-training problems!?"
3

---

Published in Transactions on Machine Learning Research (10/2022)
In such case f() could show an asymmetry test set and (as mentioned already) any statistically significant asymmetry in the test set is valid evidence for symmetry-violation.
We note that the proposal is also unusual in that the test and training datasets are identically distributed, consist entirely of data (no Monte Carlo), and are unlabelled - and so are immediately usable in high luminosity and intense environments where simulation can be both complex and costly.
Benefits for particle physicists, even if nature does not violate parity in new way
No detector is perfectly aligned or calibrated or without regions where acceptance can vary. Consequently evidence of what we have called 'parity violation' cannot be pinned immediately at nature's door. On the contrary, it is quite possible that the biggest sources of parity violation could be mis-alignments and mis- calibrations in the detectors used. This is not really a disadvantage of the method, though. Instead it is a sign that if you can get a significant asymmetry on the test-sample, then you know immediately that cither you don't. understand something about. your detector, or you have interesting discovery. Both are equally important things to know, a signal is a win-win outcome even if the source of the parity-violation is not immediately obvious. Whether the same win-win situation occurs in other fields is less clear. For example, a biologist using a net to detect parity violation in pictures of snails (shells are handed) may not be pleased if the parity violation which has been detected in those images is later found to have been caused by things in the background of each image (e.g. road signs, leaves, cars) rather than being due to the snails themselves.
Removing real-world complications
Real detectors and experiments are, of course, imperfect. Not all parts work with perfect efficiency, and acceptance does not extend uniformly into all directions. A more complex sister paper (Tombs & Lester 2022) address issues arising from those concerns, including: how to spot symmetry violation in detector which itself violates that symmetry, and how to generalise the symmetry-violatee symmetries (including continuous symmetries) rather than the narrow case of parity discussed herein. The current paper, however, deliberately eschews those details as the intention here is to provide a simple and pedagogically useful illustration of the key message, free from other distractions.
discovery process to general
3 Data for an illustrative example
Visualising deposits in the 'MNIST Calorimeter'
Calorimetric deposits in idealised versions of detectors like ATLAS or CMS are commonly represented as functions on a cylinder about the bean axis as depicted in the upper left corner of Figure [ This cylinder may be opened out into rectangle with pseudorapidity 1) on one axis and azimuthal angle on the other as depicted in the lower left corner of the same figure. In this flattened view the upper and lower edges are identified with each other.
Real calorimetric deposits are blotchy and messy, and this makes it hard for humans to picture what they might look like when rotated or mirrored. Since the goals of this paper are pedagogical, we choose to imagine a strange world in which calorimetric deposits look just like hand-written letters of the alphabet. For example, the second column of Figure [ imagines what would happen if calorimeter deposits happened to have landed in locations which (once unfolded) would resemble the capital letter 'R'. Making use of this visual simplification, the other columns of Figure illlustrate the effect that different transformations would have on the first 'R!
The LHC datasets
A real analysis would use real data not Monte Carlo simulations. Unfortunately, as there is no real detector that produces energy deposits which look like letters, we must generate some toy data of our own for our pedagogical purposes. There is a ready supply of trittwritten numerals in the MNIST dataset (LeCun & Cortes 2010 we implement generator that draws its calorimeter deposits from there. We call these
4

---

Published in Transactions on Machine Learning Research (10/2022)
5
1.
9
6
7
0
6
1
9
r
7
A
9
0
144
5
4
0
2
4
3
7
1
6
2
0
#
8
/
4
4
8
6
$
3
4
0
3
1
8
4
3
4
3
H
2
H
1
P
7
3
0
3
4
8
8
8
2
9
0
4
4
5
7
7
5
L
9
2
J
8
4
8 1
7
O
4
9
2
8
0
6
P
O
9
1
0
\
6
6
8
4
7
4
9
5
Z
8
O
8
A
>
8
(a) A mode- 1-chiral LHC dataset
(b)
A
LHC dataset
4
ko
7
$
5
B
p
2
D
6
7
8
q
8
5
+2
5
6
g
q
3
3
&
3
$
Φ
P
P
3
le
0
3
&
&
5
3
2
1
9
&
9
7
a
2
8
8
$
K
3
$
4
8
&
$
3
@
K
9
3
4
4
B
5
&
0
3
9
2
66
4
Z
g
4
7
i
(c) mode-2-chiral LHC dataset
(d) mode-2-achiral LHC dataset
Figure 2: Four lypes of LHC dataset shown. Further descriptions be found in the body-lext.
Large datasets of Handuritten Calorimeter digits or 'LHC datasets' for short. We produce LHC datasets in four modes shown in Figure and defined follows:
mode-1-chiral These are normal MNIST digits. These samples are chiral because humans write hand-written digits with handed-appendagge called hands. The word chiral even come from the ancient Greek word for hand: Xetp;
mode-l-achiral These are MNIST digits which have been reversed left-to-right with probability 50%; The symmetrisation process makes these samples achiral.
mode-2-chiral These are random pairs of normal MNIST digits superimposed on each other. They inherit their chirality from their constituent digits.
mode-2-achiral These are random pairs of normal MNIST superimposed on each other with one member of the pair forwards and one reversod. The symmetrisation process makes these samples achiral.
Mode-1-achiral acts as control for mode-1-chiral, and mode-2-achiral acts a control for mode-2-chiral. The only purpose of mode-2 is to illustrate that the analysis is not dependent on there being only small number of 'tppes' of deposit in this case ten as there are ten sorts of digits. Both types of Mode-2 therefore act as a form control against the possibility that mode-1 is too simple to be a good proxy for real and highly variable calorimeter data.
Removing remaining differences between real calorimeter data and MNIST derived calorimeter analogues
MNIST digits are 'oriented' because the humans who originally generated them were instructed to write them in boxes the conventional way up.

---

Published in Transactions on Machine Learning Research (10/2022)
Real jet data, in contrast, is not oriented: if energy deposits to appear in configuration resembling capital 'R'' then the deposits would be likely to arrive in any of the configurations shown in columns two, three and four of Figure ! provided that the collider has symmetric beams and no reason to choose any @-orientation more than any other.
To avoid the MNTST-based dataset. having an unfair handle (orientability) not possessed by real jet data, this discrepancy needs to be removed. One way of removing it would be augment our dataset by adding g-translations of uniformly random magnitude to every image in an LHC dataset (being sure to wrap pixels lost from the top back onto the bottom) before then giving every image 180 degree rotation with probability 50%.
Although one could augment the dataset in that way, it is more elficient to simply make the neural net invariant with respect to those transformations. Doing so is as easy as making the net parity-odd, the only difference being that one seeks symmetry rather than antisymmetry and so one must combine sub- nets with symmetric operations (like maxpooling summing) rather than antisymmetric operalions (like subtractions).
4
An example parity-odd axisymmetric network
Recall that the necessary axisymmetry comes from establishing (i) an invariance with respect to y-translations of 28x28-pixel images corresponding to d-invariance in the real world, and (ii) invariance with respect to 180 degree rotations of the 28x28 images corresponding to interchange of beams and isotropy.
Our final network f (x) gains its parity-oddness and one-half of its axisymmetry (the part requiring invariance with respect to 180 degree rotations) by being defined terms of sub-network g(x) as follows:
f(x) g(x) + ((R100z) - g(Pz) (0000gg(BB) ((RssoPr).
(1)
The sur-network g(x) (implemented in pytorch by Paszke et al. (2019) therefore only needs respect
the d-rotation (or y-translation) invariance. The g(x) sub-network starts by having two 2D-convolutional network layers (one with kernel size 4 x 4 in 17 Φ, and the other with size 14 x 6 in v x é, always stride 1) to provided basic image processing of the 28x28 pixel images fed to it. These two layers bring the number of features up to 32 and then to 64, and have the appropriate over-run regions to allow them to implement the required periodic boundary conditions. They are then followed by a max-pooling over the direction to finally achieve the desired d-invariance. g(x) then concludes with two fully connected layers reducing 640 features to 128 and thence to a single feature so that f() can then make used of g(x) in the manner outlined in To describe the structure of our network, annotate its printout (as pytorch Module) as follows:
Net
# phi-cyclic pad (conv1): Conv2d(1, 32, ternel_size=(e 4), stride=(1, 1))
# relu
# phi-cyclic pad
(conv2): : (32, 64, kernel_size=(11 6), stride=(1, 1))
# relu
mmax_pool): MaxPool2d
(28, 2), stride= (28, 2), ...)
# flatten
(fc1): (in_ out_ featffuuuuees112,, bias=True)
# relu
(fc2): Linear (inear(nn_eatureee out fut_features=1, bias=True)
The network is trained on images provided in batches. Each batch
B (01, T2,,, . [

---

Published in Transactions on Machine Learning Research (10/2022)
contains [B] = -44 images. For each batch the mean net score Pa is calculated:
1
E(E). JEB
f(z).
PB
B
The standard deviation of the scores of the images in the batch is also recorded as dg. The net is trained to maximise Un/0g. This process encourages the net to do its best lo give every image a common constant positive value, and eschew negative values if possible. The net is trained maximise pB/OB rather than Ps alone since the former (unlike the latter) cannot be trivially made larger by replacing f(x) with Xf(x) for some constant >> 1. The complete code for the network may be found in Lester (2021b).
Though it has already been emphasised, we repeat the message that the model chosen above for g(x) and the choice made for the loss function μB are almost certainly non-optimal. Indeed, that is one reason why the more detailed paper exploring realistic particle physics events (Lester et al. (2022b)) tests a variety of very different models and very different loss functions. None of that lack of optimality matters for the purposes of the current paper, however, whose role is pedagogical. The only things that matters in this paper are the way that g(z) has been (amti)symmetrised into fa), and (as will be seen quantitatively in the results section) that the mechanism works despite its simplicity.
5 Results
Quantitative results are presented in Table and confirm that parity violation is not detected where it should be absent (the most important result) and yet is discovered where it is present (the bonus).
The same results are presented qualitatively and graphically in Figure g in which the preponderance of green in plots (a) and (c) within Figure shows that the networks trained on the chiral modes are able to label the majority of their inputs with positive scores according to J, meaning that they have identified that these datasets chiral (i.e. violate parity). In contrast the nets have not managed to achieve this for the achiral controls ((b) and (d) in the same figure), despite having been trained on achiral control data. It is notable and pleasing that the networks are not flummoxed by the differences between mode- 1 and mode-2.
Mode
Pchiral
Pachiral
mode-1 1
9011 %
5112 %
mode-2
89 %
42+6 %
Table 1: When tested on groups of 1024 images in the relevant test dataset, the above numbers show the typical fraction pt of images per batch which were assigned positive value by the function f after training. As desired, the chiral LHC datasets are flagged as chiral at a statistically significant level, while those LHC datasets which are achiral are not, with their fractions remaining consistent with the 50% expected by chance alone.
Figure 4 demonstrates that our network function is indeed parity-ode odd and axisymmetric as desired.
6 Discussion.
The strategy has achieved its goals. What is perhaps more interesting to reflect on is what this buys in comparison to other methods for analysing multi-jet events for evidence of parity violation in colliders (such as the Large Hadron Collider) without polarised beams.
Only one work (Lester et al.) 2022a) has previously tried to tackle this task in a systematic way. To give a chastening example: when that work considered collisions of the form Sab (three important jets, and perhaps some other stuff X the final state, resulting from the collision of an 'a' with a 'b") was found that 19-dimensional continuous ideal vector parity (to use the nomenclature of Lester (2021a)) would, in principle, detect any parity-violating signature. However: (i) over 60 pages of algebra were necessary to
7

---

Published in Transactions on Machine Learning Research (10/2022)
6
8
2
0
6
8
9
3
6
0
8
4
8
6
3
5
O
4
8
9
8
9
8
2
5
0
2 1
7
50
50
0
4
7
4
8
6
4
7
8
8
f
O
r
6
100
9
6
0
7
4
9
4
100
5
2
8
6
8
8
6
9
3
2
1
O
9
9
7
1
&
2
2
\
O
150
150
0
4
8
7
4
9
1
8
6
6
6
8
O
S
6
/
5
8
O
8
2
5
Y
S
0 5
\
2
H
200
200
6
6
5
4
/
3
0
4
5
6
2
7
2
3
50
100
150
200
50
100
150
200
(a) A mode- I-chiral LHC dataset
(b) A mode- mooe--Lachiee 1-achiral LHC dataset
0
3
2
9
4
@
4
3
3
5
a
X
0
2
3
3
5
&
4
4
4
1
2
1
3
50
50
6
3
8
5
4
9
3
4
2
4
X
7
100
9
&
0
8
4
4
100
8
*
3
3
4
0
8
2
0
4
Ø
2
9
3
2
2
2
7
3
150
150
gr
5
9
4
8
5
6
8
A
d
*
+
5
/
8
7
X
D
2
8
+
8
200
200
C/
8
4
7
3
9
g
3
9
46
50
100
150
200
50
100
150
200
(c) A mode-2-chiral LHC dataset
(d) A mode-2-acchiral LHC dataset
Figure 3: Four types of LHC dataset from the test dataset are shown after having been coloured according to the output of the net. Bright green represents increasingly positive numbers. Shades of increasingly vibrant red represent increasingly negative numbers. Yellow represents zero.
generate that result, and (ii) there seems little prospect of generalising that result to four final-state particles (due to factorial increases in computation complexity) let alone to n final-state particles.
Why do we mention these limitations of Lester et al. (2022a)? The reason that the present work can be considered to be doing a very similar job to Lester et al. (2022a) but for events with 'up to' 784 28x28 jets in the final state, yet in a way that scales much better
10 intensity of each pixel can, in principle, represent very narrow and so image with (say) only three black pixels
could represent three-jet event. Moreover, the permutation symmetry modelled between jets in Lester al. (2022a) carries
to images, since the pixels to somchow list jets in particular order. The major difference between Lester et. 2022a and the present work is, however, that Lester et 2022a provides guarantees of ideality (see definition this (2021a)) which the present work never can.
8

---

Published in Transactions on Machine Learning Research (10/2022)
2
4
8
4
4
5
0
5
t
4
9
7
9
3
2
6
1
4
5
5
8
8
0
3
1
6
6
8
8
4
3
8
5
3
9
4
7
4
7
4
3
5
9
O
8
/
3
8
5
&
7
8
8
6
1
4
4
49
7
9
5
3
2
8
6 2 1 8
3
6 C
8 +
3
4
3 4
P
2 S
9
4
7
4
5 3
2
8
9
2
9 3
h
8
h
S
1
4
6
6
6
5
2
9
1
5
S
5
8
8 8 8
0
2 9 & 2
1 1
9
9
8
8
4
3
8
5
6
4
4
4
L
4
3
9
6
O
8
&
8
S
7 0 3
2
1
6
H
6
8
2
P
5
8
8
2
2 8
8 2
D
8
8
P
&
8 2 8
+
E
8
&
3
8
R
p
2 8
0 3 1
2
8
2
b
H
3
+
8
2 8
2
3
2
2
8
8
3
1
Q
&
8
3
2
3
8 3
R
0
8
3
8
2
8
8
1
Figure 4: Here the same 64 letters from mode-1-chiral LHC dataset are evaluated by the same symmetrised network (x) in six different ways. In the first pair of rows (the control rows) the images are evaluated without any changes at all. Most of the controls are coloured green (as one might expect) but the fact that this happens is irrelevant. for the purposes of the check being performed here. What matters is how the colours in those two rows relate to the colours in the remaining rows. In the next pair of rows the nets are re-evaluated but after a shift (or roll) down of cach image by 10 pixels. In the next two rows the transformation prior to evaluation is instead 180 degree rotation. The key take home message from these first six rows is that (as the colours do not change!) all of these transformations do not affect the network score, and so the network is axisymmetric as desired. This property is not learned, it is guaranteed by construction. Nonetheless, it is nice to see a graphical check that the implementation does what was expected. The lower six rows repeat the upper six rows, but after an initial left-right parity flip of each image. In this case we see that there is (as desired) an immediate flip in the output from positive to negative (or in rare cases vice versa) because the network is, by construction, parity-odd.
7 Conclusion
This paper has proposed a framework for conducting tests of symmetry violation at the LHC. A koy feature of our proposed methodology is that it can be applied to recorded data: a discovery would be made by comparing collision events to themselves rather than to theoretical predictions (of which there are few). The proposal therefore plugs a gap in the literature by making discoveries possible which hitherto could easily have been missed on account of the lack of thcoretical models resulting in lack of dedicated searches. This is important: the LHC is exploring centre of mass energies which are orders of magnitude higher than those in which parity violation was first discovered, and there is therefore no knowing what mysterious signals could be hiding in LHC datasets created in the last decade. Important signals could be hiding in plain sight.
This paper is intended to serve a primarily pedagogical and rwwrennss-raising purposes. It is also intended to approachable to readers outside of particle physics. Accordingly it was decided to demonstrate the viability of proposed methodology by applying it to loy problem wherein the hidden parity violating signature was the handedness of human handwriting. This is simple target symmetry (parity) in idealised detector possessing equally simple 'unimportant" intrinsic symmetries (axisymmetry and beam interchange symmetry) which are reminiscent of symmetries found in detectors at particle physics colliders.
A more complex sister paper (Tombs & Lester 2022) describes how the framework presented here could casily be extended to consider continuous symmetrics and shows how to deal with important issues (notably acceptance and inefficiency) which were ignored here for reasons of accessibility. A more 'realistic' particle physics demonstration is developed in a sister paper Lester et al. (002b))
9

---

Published in Transactions on Machine Learning Research (10/2022)
8 Acknowledgements
The authors greatly acknowledge discussions with Ben Nachman related to the present work and to Collins et al. 2018), and with Shikma Bressler related to the present work and to Volkovich et al. (2022). Useful discussions with Radha Mastandrea, Daniel Noel and Vidhi Lalchand are all very much appreciated, as has been the patience of Tom Gillam in assisting with numerous questions about pytorch. Finally we note that the paper was considerably improved as result of open-review discussions with three anonymous referees and editor in the TMLR journal. We thank them for their time and attention.
References
Jack Collins, Kiel Howe, and Benjamin Nachman. Anomaly detection for resonant. new physics with machine learning. Phys. Rev. Lett., 121:241803, 12 2018. doi: 00.103//llysRevLeet........
URL https:
7/link. wtw.appp...oooo/oooo...ooooooo do0/10. 1ww./1100haaae www.///yyseeeeet....mmom.mm 121. 241803
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL http://yann. lecun
com/exdb/mnist/
Christopher G. Lester. Chiral measurements. 11 2021a. doi: 10.48550/acXivv221111oo
URL
https:
//doi.org/10. 211.0/488850///1.1111111.1
Christopher G. Lester. http:////hhhhhoooo/////o///////a //github.com http:///eeeeeeeeeeee/o//ooaaaaaa
11 2021b. URL https
Christopher G. Lester, Ward Haddadin, and Ben Gripaios. Torentz and permutation invariants of particles 11l: constraining non-standard sources of parity violation. International Journal of Modern Physics, 37, 8 2022a. doi: 10.1122500277551X2250533
URL https:://oi..orrggteeee /tt.////..000/0007///25555559990
Christopher G. Lester, Radha Mastandrea, Daniel Noel, and Rupert Tombs. Hunting for vampires and other unlikely forms of parily violation the Large Hadron Collider. JHEP, 08:231, 2022b. doi: 10.1007/ JJEEPSS(002))33L URL https://doi. org/10 10. 1007/ (2022)
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performanc deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, F. Fox, and R. Garnett (eds.), Advances in Neural Informa- tion Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019. URL http:/// neurips. -ddf
The ATLAS Collaboration. The ATLAS Experiment the CERN Large Hadron Collider. JINST, 3:S08003, 2008. doi: 10.1088/1748. /3/08//0000/00/// URL http::/e//oaaarooooooooooo/////2
Rupert Tombs and Christopher G. Lester A method to challenge symmetries in data with self-supervised learning. JINST, P000044 2022. doi: 10.1088/1745 12444-0221/1100//P00000 URL
10. 1011108811433 00088/1748-0211111 1ww100//7778-002111770/0000000
Sergey Volkovich, Federico De Vito Halevy, and Shikma Bressler. The Dire-Directed Paradigm for BSM searches. Eur. Phys. J. C, (2(3))265, 2022. doi: 24L110//4pp///s00022222211211111 URL https://doi.
C. S. Wu, E. Ambler, R. W. Hayward, D. D. Hoppes, and R. P. Hudson. Experimental test of parily conservation in beta decay. Phys. Rev., 105:1413 -111111--115,, Feb 1957. doi: https://link. aps httpp/or///oo//.1103/////0 1tt.://...110///hhe 105.1413'
URL
10


Reviews:
[{"review_id": "VTukIF6ia8", "reviewer": "Reviewer_7yco", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- idea on hypothesis testing with model training without any prior on the parity violation source / type / theory, working with unlabeled **real** data\n- nice analogues and examples in paper for people outside the LHC and physics community.\n- probe the method on toy MNIST modified data\n\n**Weaknesses**\n- The main weak point is that no any test performed on real LHC data, even for known decays / events to demonstrate how it is working in real scenario. Or at least on simulated data with idealized detector or with detector with some asymmetries which we know. \n- It is discussed a bit in the paper, but from a different view: we can detect the problem with detectors with this hypothesis testing. It is true, but then we have problem in real LHC experiments that we cannot detect and recognize if parity violation comes from new discovery or from detector systematic errors (which is more likely) and the hypothesis testing itself is not useful at all. We can spend years on proving that there is no any other systematic error but maybe we again forgot to account something. There is no any discussion or solution proposed in the paper on this.\n- With both above points there is no any investigation how we depend on these detector asymmetries, how different scenarios of parity violation level and systematic error will affect each other for proposed method to claim discovery. There is no any evaluation of sensitivity of this hypothesis testing and which level we could expect to capture with it.\n- I remember that at LHCb for example we had only 9x9 images of calorimeter (so very small compared to what discussed in the paper). It could be that for this level of images the method is not gonna to work. Also we have a huge amount of other information about the events which we can use and here it is ignored and not discussed how to use.\n- The method proposes to maximize mean/std of output model values on a batch. In this case model can simply learn the constant (say 1, and then mean is 1 and std 0), e.g. either memorizing all the data or just learning to predict constant everywhere. In both case this function is useless and will not do the proper job on test set. How can we prevent from this and make sure we are doing the right job? ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Parity violation is one of the phenomena which Large Hadron Collider is targeting to resolve. Particularly the main methodology right now for scientific discovery is to analyse jets and investigate particular decays and search for particular theory for parity violation. With that it is very hard to make progress quickly as we need to test and prove every theory and family of decays. Current paper propose hypothesis testing on violation of parity without making any prior on source / type of violation by training parity-odd approximation with maximization mean / std value computed on the batch and checking if this trained function can produce same-mean on test data. In case the predicted mean is shifted then the parity violation happens and we can claim discovery / problems with detectors symmetry. The paper consider pedagogical example on MNIST data modified with incorporating similar symmetry properties as in calorimeters and demostrating that we can use proposed method of hypothesis testing to detect which data has parity violation.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Overall idea itself is cool, that probably we can use simple unsupervised real data training and check if we have any parity violation. Some application may be if we just need to test that our simulated data for example have necessary properties. Or in case of real data (some known decay) when we know that it should be without parity violation but we see it - then we can recalibrate detector or estimate necessary systematic errors for future use. But this should be clearly stated in the paper. Right now it seems authors try to tackle more general problem of working with both detector and data parity violation. And moreover very toy, not physics related example is used (but data are very-very different in MNIST type and what we have in physics from my experience on working with both of them).\n\nFirst, I would like to see authors to clarify the weak point I made above, discuss them and point to either my misunderstanding or uncover more precisely the procedure and more experiments.\n\nSecond I have several questions which would be nice to clarify in the text (especially for the part where I am wrong so that it is clear for future readers):\n- maxpool and conv layers to have proper symmetry properties should have proper cyclic padding. Probably authors did this as I see in comments on code similar mentioning. But could you confirm that?\n- how with proposed method we can distinguish / detect between parity violation in detector or in physics events as in real applications we will have definitely detector parity violation and maybe parity violation for physics? \n- parity violation effect could be very small (correct me here, maybe I am wrong).  E.g. B-meson oscillations have very little probability and the methods should be very sensitive to this. So what is the limits of the proposed method with respect to this sensitivity? Could be that for real case scenario it will not work? In the past my team tried to use Mann-Whitney test (ROC curve) to detect if we have very rare decay in the real data - and we fail to use it as it was giving huge std, and effect we are searching is very little / very rare to be caught by models and this test.\n- as I pointed above systematic errors from detector could be large and the reason why we have all these complicated not e2e methods is because we need to estimate properly all systematic errors. And exactly this is a root of problems. We use for example control channels to be able to calibrate ourselves and estimate these errors. If we have particular theory and particular decay study / energy band we better know what factors can affect so we can estimate them. In general case we cannot estimate all systematic errors. How do authors deal with this in proposed method? Also most probable we want to exactly estimate the violation level - and the proposed method even with detecting there is violation will not help to detect exact violation or help somehow to approximate it (correct me if I am wrong) \n- what is the pdf of learnt values on train and test set for model output? how do we depend on data set size in train and test to be able to detect parity violation?\n- different particles could probably have different level of parity violation - again question to sensitivity - for what kind of data method could be sensitive enough?\n- the example considered in the paper is very simple and has 28x28 images. What is the actual calorimeter images we expect in different LHC experiments? How do we use other information from the event (e.g. full even reconstruction and jet properties estimation, like momentum)? As I mentioned in LHCb calorimeter images are very small in terms on number of pixels, so it could be that no any helpful information can be on that level to see parity violation.\n- title is very misleading. I would prefer not to put GAN in title as it is creating more hype around them in physics. Right now GANs are tried actively to use for Monte-Carlo simulation which I don't trust much honestly. Also it is  better not to force people by title and abstract believe that GANs are able to solve problems of parity violation.\n", "broader_impact_concerns": "No any concerns as it is about particle physics which involve only nature questions and not related to any user sensitive information or user related predictions.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yY8HJJuenR", "reviewer": "Reviewer_jRVr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "First, I want to stress that I do not know much about machine learning for high energy physics, and my review should be weighted accordingly.\n\n**Strengths:** The paper is written in a lightweight and simple way.\n\n**Weaknesses:**\n\n1. Are the claims made in the submission supported by accurate, convincing and clear evidence?\n\nClaims in the submission are mostly qualitative and nontechnical. For any more rigorous treatment, or theoretical results, the authors refer the reader to multiple \"sister papers\". Key concepts are never formally defined. The presented experiments are mostly qualitative. \n\nTake for example section 2: _\"In practice, of course, the training step is very important. Without training there is little chance that the net will actually be attending to any useful features of the data [...] the training process exists [...] only to increase the probability of being able to claim discovery\"_. No further elaboration on this paragraph is presented anywhere in the paper: In what sense should it be important for the neural network (or other function approximator) to _\"attend to any useful features of the data\"_? How does the _\"probability to be able to claim discovery\"_ change when the deployed neural network is trained, as opposed to when it is not? Can you show, in theory or experiments, this difference in performance of trained vs. untrained models?\n\nThe authors do not provide any formal definitions in the paper. For example, in section 1, the authors present one of their main ideas _``that a parity-odd function $f(x)$ can only have an asymmetric distribution if the dataset on which it has been evaluated contains some elements $x_1, x_2, \\ldots$ which differ in frequency from their parity-flipped partners $Px_1, Px_2, \\ldots$\"_ While this sentence is probably meant as a nontechnical exposition of the main idea, no technical exposition follows. The authors do not explain what is the domain of the random variables, whether they are real-valued or categorical, what is $P$ and how are the parity-flipped partners defined etc---all of this is left to the imagination and intuition of the reader.\n\nThe assessment of statistical significance of the discovered parity violation (and how and whether it differs between trained and untrained networks) is not discussed. The only reported quantitative output of the experiments is table 1.\n\n2. Would some individuals in TMLR's audience be interested in the findings of this paper?\n\nA strong motivation of this work seems to be the analysis of LHC data. At the same time, according to the authors, the manuscript aim to expose some very simple ideas, abstracting away the complexities of the high energy physics application as much as possible. In the words of the authors, their aim is to _\"provide a simple and pedagogically useful illustration of the key message, free from other distractions\"_. For any theoretical results, discussions of complexities coming from applying the method to LHC data and extensions to symmetry other than parity readers are referred to multiple \"sister papers\". \n\nThe question is whether what is left is interesting enough, and whether such pedagogical aim makes this work appropriate for publication in TMLR. Overall, my impression is that the effort towards abstracting away details of the LHC application might have gone too far, ultimately leaving too little technical content. \n\nLet me further elaborate. This paper is clearly not about GANs, as the authors candidly admit in footnote 4. The authors also argue (footnote 6) that the paper is not really about neural networks either; and it could even be said that it is not about (machine) learning, based on section 2, paragraph \"Unusual Features of This Proposal\". Ultimately, it is unclear what aspects of the paper should be interesting for the TMLR readers.\n\nIt is well possible that someone interested in the high-energy physics applications might immediately recognise the value of the work (I myself am admittedly not, therefore I am unable to answer the question:_\"Is there something to be learned by some researchers in their area from their work?\"_): in which case, one wonders whether this paper is at all necessary, and whether it wouldn't instead be better to present this ideas to the high energy physics community in a dedicated, subject-specific workshop or venue.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a method to spot parity violations using parity-odd functions. The motivation comes from high energy physics.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "For the paper to be fit for publication in TMLR, some possible suggestions could be to try to incorporate more technical details; investigate (theory, experiments) the difference between trained and untrained networks and discuss assessment of statistical significance for both cases; extend the work to symmetries different from parity; discuss aspects of practical applicability of their method to LHC data or present any potentially interesting application to any other data. It may well be that a paper with the aforementioned changes ends up being very close to one of the \"sister papers\".\n\n**Minor:**\n\nThe Pytorch pseudocode for the neural network appears unnecessary.\n\nIn section 2, the authors use the term \"overtraining\": presumably they mean \"overfitting\" (the latter term is more standard in machine learning)?\n\nI would consider modifying the title: the joke is a bit of a stretch given no serious references to GANs in the paper.", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "lgnh5REzL5", "reviewer": "Reviewer_ghxN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The writing is exceptionally clear. Figure 1 is excellent. The paper was a joy to read!\n- This is a clever idea and could prove very powerful. I am not a physicist and don't know all the details of the application, but I see no flaws in the argument here. The approach has many advantages over the existing approaches to testing for parity violations (namely, looking for unexpected \"bumps\" in histograms, or coming up with alternative hypotheses and performing Monte Carlo simulations to test each one). \n- The discussion makes a good argument for why this ML approach is valuable.\n- While the motivation comes from HEP, this is an interesting ML strategy for data analysis, and could potentially be useful for data analysis in other fields. It makes sense to publish this work in an ML venue.\n- The authors use a toy dataset to demonstrate their method, and refer to a separate paper which applies the method to particular application in HEP. I think this decision makes sense. \n\n\nWeaknesses\n- None.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a machine learning method for detecting symmetry violations in data. The approach is general, but motivated by an imporant problem in high-energy physics. The method is demonstrated on a toy problem. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "None.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I would like to point out that I perfectly understand the reservations of the two reviewers who suggested a lean reject (but who acknowledged the scientific value of the paper), and I thank them for their careful work which has lead to these improvements.  I think that appreciating the style of this paper is really person-dependent. But through my own reading, as explained before, I think that the conceptual content overweight the lack of technicality in this case.
The discussions have lead to great improvements of the paper but I would like to ask the authors to implement this change of title and, if possible, take another look at the comments of reviewers and potentially improve further, in particular concerning the relations to "real applications". Once these revisions have been implemented, the paper should be in a position to be published.
Decision: Accept (poster)