File size: 2,607 Bytes
adb02cc 07b1bb7 15d0c76 07b1bb7 15d0c76 07b1bb7 15d0c76 b9a420c 15d0c76 07b1bb7 15d0c76 07b1bb7 adb02cc 07b1bb7 adb02cc 4b11d0a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
---
license: apache-2.0
language:
- en
library_name: transformers
---
# AnalysisObjectTransformer Model
This repository contains the implementation of the AnalysisObjectTransformer model, a deep learning architecture designed for event classification with data from the CERN LHC.
The model operates reconstructed-object and event-level features. MultiHeadAttention is used to extract the correlation between reconstructed objects such as jets (hadrons) or leptons in the final state, while event-level features capture the event summary, such as total hadronic energy or missing transverse energy.
Achieves state-of-the-art performance on final states which can be summarized as jets accompanied by missing transverse energy.
## Model Overview
The AnalysisObjectTransformer model is structured to process object-level features, in the case of jets: energy, mass, area, btag score, in any order (permutation invariance) and event-level features (HT, MET) to classify signal from background processes to enhance the sensitivity to rare BSM signatures.
### Components
See [**here**](https://excalidraw.com/#json=tCXGu1s6Az9wh4md45JU6,A3ezTIoqB10HVxOt4hhRSA) for complete architecure.
- **Embedding Layers**: Transform input data into a higher-dimensional space for subsequent processing.
- **Attention Blocks (AttBlock)**: Utilize multi-head attention to capture dependencies between different elements of the input data.
- **Class Blocks (ClassBlock)**: Extend attention mechanisms to incorporate class tokens, enabling the model to focus on class-relevant features. Implementation based on "Going deeper with transformers": https://arxiv.org/abs/2103.17239
- **MLP Head**: A sequence of fully connected layers that maps the output of the transformer blocks to the final prediction targets.
## Usage
Firstly, clone the repository:
```bash
git clone https://huggingface.co/maciek-g/AnalysisObjectTransformer
```
You can then import the model object and use within standard PyTorch and PyTorch lightning training workflows.
```python
from particle_transformer import AnalysisObjectTransformer
model = AnalysisObjectTransformer(input_dim_obj=..., input_dim_event=..., embed_dims=..., linear_dims1=..., linear_dims2=..., mlp_hidden_1=..., mlp_hidden_2=..., num_heads=...)
```
## Parameter definitions
`input_dim_obj`: The number of features associated with each event object (features per jet, lepton etc..)
`input_dim_events`: The number of features associated with the event (Number of jets, total hadronic energy, total missing transverse energy etc..)
`embed_dims`: Sequence embedding dims
|