## 3 Materials and Methods
### Action Detection
#### 3.2.1 Action Recognition with MAROON  
Here, we explain in detail the architecture and idea of our action recognition network MAROON--mask-guided action recognition. In Figure 4, we show the architecture of MAROON. We highlight in blue the pathways and lateral connections that we newly introduce in comparison to the base model of SlowFast [22]. The two most important innovations in our network are (1) the use of masked input frames and (2) the triple-stream approach. As described previously, the input frames of the action recognition network are first cut out from the overall video frame using the bounding boxes detected by SWIFT and then masked by the instance mask of the animal. This allows MAROON to focus entirely on the actor during the feature extraction and not learn unnecessary background information. Figure 4: Our action recognition network MAROON uses three streams and masked input frames. The blue arrows represent the new third path and the new lateral connections in the MAROON architecture compared to SlowFast. Figure 3: Our action detection system, a combination of SWIFT and MAROON. Brown boxes represent data, blue boxes describe neural networks and algorithms and yellow boxes represent algorithmic instructions. Our network architecture builds on the idea of SlowFast [22]. SlowFast introduces a two-stream network, where two pathways exist, a slow pathway and a fast pathway. The task of the fast pathway is to extract motion features (for example, the type of movement and speed of the animal), and the slow pathway should concentrate on the appearance features (for example, the color and the pose of the animal). To achieve this, they modify the number of frames \(T\) that are fed into each of the two streams and the number of feature channels \(C\) that the convolutional layers have in the respective streams. For this, the parameters \(\alpha\) and \(\beta\) are introduced. For MAROON, we extend this idea from a two-stream approach to a triple-stream approach to extract motion and appearance features with different granularities. In Figure 4, we show the architecture of MAROON. Accordingly, we name the three pathways the MAROON slow pathway, MAROON medium pathway, and MAROON fast pathway. We newly introduce the parameters \(\gamma\) and \(\delta\). With \(\alpha\) and \(\delta\) (\(\alpha,\delta>1\)), we steer the amount of frames for each of the three pathways. The MAROON fast pathway receives the most densely sampled \(\alpha T\) frames. The MAROON medium pathway receives \(T\) frames, and the MAROON slow pathway receives only \(T/\delta\) frames. A typical distribution of frames, for example, if you consider a 16-frame input sequence, would be 16 frames for the fast pathway, 4 frames for the medium pathway, and 2 frames for the slow pathway. The second aspect in which the three pathways differ is the channel capacity \(C\), which is controlled by the two parameters \(\beta\) and \(\gamma\). A higher amount of channels allows for the extraction of more detailed features. This is the case when the appearance of the animal is in focus. The medium pathway has \(C\) channels. The parameters \(\beta\) and \(\gamma\) (\(\beta<1,\gamma>1\)) steer the amount of channels for the other two pathways. With \(\beta C\) channels, the fast pathway has less channels than the medium pathway to concentrate on the motion feature extraction. The slow pathway has with \(\gamma C\) channels the highest number of channels of all three pathways. To summarize, the higher the number of frames fed into the pathway, the lower the channel capacity and vice versa. In addition to the two-stream architecture, the connection of both streams by lateral connections is one of the crucial innovations of SlowFast [22]. Thus, the features extracted from the fast pathway are regularly combined with the features of the other pathway. We extend this idea to our third pathway by combining the features of the fast pathway with the features of our slow pathway in the same way.