|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
library_name: transformers |
|
--- |
|
# AnalysisObjectTransformer Model |
|
|
|
This repository contains the implementation of the AnalysisObjectTransformer model, a deep learning architecture designed for event classification with data from the CERN LHC. |
|
The model operates reconstructed-object and event-level features. MultiHeadAttention is used to extract the correlation between reconstructed objects such as jets (hadrons) or leptons in the final state, while event-level features capture the event summary, such as total hadronic energy or missing transverse energy. |
|
Achieves state-of-the-art performance on final states which can be summarized as jets accompanied by missing transverse energy. |
|
|
|
## Model Overview |
|
|
|
The AnalysisObjectTransformer model is structured to process object-level features, in the case of jets: energy, mass, area, btag score, in any order (permutation invariance) and event-level features (HT, MET) to classify signal from background processes to enhance the sensitivity to rare BSM signatures. |
|
|
|
### Components |
|
|
|
|
|
See [**here**](https://excalidraw.com/#json=tCXGu1s6Az9wh4md45JU6,A3ezTIoqB10HVxOt4hhRSA) for complete architecure. |
|
|
|
- **Embedding Layers**: Transform input data into a higher-dimensional space for subsequent processing. |
|
- **Attention Blocks (AttBlock)**: Utilize multi-head attention to capture dependencies between different elements of the input data. |
|
- **Class Blocks (ClassBlock)**: Extend attention mechanisms to incorporate class tokens, enabling the model to focus on class-relevant features. Implementation based on "Going deeper with transformers": https://arxiv.org/abs/2103.17239 |
|
- **MLP Head**: A sequence of fully connected layers that maps the output of the transformer blocks to the final prediction targets. |
|
|
|
|
|
## Usage |
|
|
|
Firstly, clone the repository: |
|
```bash |
|
git clone https://huggingface.co/maciek-g/AnalysisObjectTransformer |
|
``` |
|
You can then import the model object and use within standard PyTorch and PyTorch lightning training workflows. |
|
|
|
```python |
|
from particle_transformer import AnalysisObjectTransformer |
|
|
|
model = AnalysisObjectTransformer(input_dim_obj=..., input_dim_event=..., embed_dims=..., linear_dims1=..., linear_dims2=..., mlp_hidden_1=..., mlp_hidden_2=..., num_heads=...) |
|
``` |
|
|
|
## Parameter definitions |
|
|
|
`input_dim_obj`: The number of features associated with each event object (features per jet, lepton etc..) |
|
|
|
|
|
`input_dim_events`: The number of features associated with the event (Number of jets, total hadronic energy, total missing transverse energy etc..) |
|
|
|
`embed_dims`: Sequence embedding dims |
|
|
|
|
