# ChaMTeC: CHAnnel Mixing and Temporal Convolution Network for Time-Series Anomaly Detection
## 3 Methodology
### Encoder Layer for Channel and Temporal Processing  
The encoder consists of multiple layers, each leveraging fully connected (FC) layers for feature-wise transformations and convolutional layers for temporal dependencies, as represented in Figure 2. This design enables the encoder to perform channel and temporal processing in a unified manner. Channel processing focuses on capturing the interactions between features (or sensor channels) at a given time step through fully connected layers, allowing the model to learn complex relationships such as correlations between pump flow, tank levels, and operational statuses. In parallel, temporal processing aims to model the evolution of each feature across time. To achieve this, the encoder integrates multi-head self-attention for learning long-range dependencies and dilated convolutional layers for capturing localized short-term trends. While attention enables the model to dynamically focus on relevant past time steps, convolutions ensure the efficient extraction of patterns across fixed-length receptive fields. Together, these mechanisms allow the encoder to simultaneously learn spatial (channel-wise) and temporal representations, making the framework robust against both point anomalies and context-dependent deviations in multivariate time-series data. Multivariate time-series data in industrial systems exhibit dependencies both across features (channels) at each time step and along the temporal axis across sequences. Capturing these dependencies is essential for accurate anomaly detection. Therefore, the encoder is designed to jointly process. To achieve this, we implement a hybrid encoder that consists of two main stages:  
1. Channel Processing: Using fully connected (FC) layers applied to each time step independently, the encoder learns nonlinear transformations and inter-feature interactions in the input space. Feature embeddings are processed through a two-layer FC network as follows: \[\mathbf{h}=\sigma(\mathbf{z}W_{1}+b_{1})\] (2) \[\mathbf{x}^{\prime}=\text{Dropout}(\mathbf{h}W_{2}+b_{2})\] (3)where \(W_{1},W_{2}\) are the weight matrices, \(b_{1},b_{2}\) are biases, and \(\sigma\) is the ReLU activation function. The dropout layer improves generalization. 2. Temporal Processing: The encoder employs two complementary approaches for temporal modeling. Attention-based Temporal Processing:Multi-head self-attention captures global temporal dependencies. For input \(\mathbf{x}^{\prime}\),  
\[ \begin{split}\text{MultiHead}(\mathbf{x}^{\prime})& =(\text{head}_{1}\|,\ldots,\|\text{head}_{h})\,W_{\text{O}}\\ \text{head}_{i}&=\text{softmax}\Bigg{(}\frac{( \mathbf{x}^{\prime}W_{\text{O}}^{i})(\mathbf{x}^{\prime}W_{\text{K}}^{i})^{ \top}}{\sqrt{d_{k}}}\Bigg{)}(\mathbf{x}W_{V}^{i})\\ \mathbf{x}_{\text{attn}}&=\text{LayerNorm}(\mathbf{x }+\text{Dropout}(\text{MultiHead}(\mathbf{x})))\end{split}  \tag{4}\]  
Convolutional Temporal Processing:Local temporal patterns are captured through dilated convolutions as follows:  
\[ \begin{split}\mathbf{y}&=\text{ReLU}(\text{Conv1D}( \text{LayerNorm}(\mathbf{x}_{\text{attn}}),W_{c}^{1}))\\ \mathbf{x}_{\text{conv}}&=\text{LayerNorm}( \mathbf{x}_{\text{attn}}+\text{Conv1D}(\mathbf{y},W_{c}^{2}))\end{split}  \tag{6}\]  
where \(W_{c}^{1}\) and \(W_{c}^{2}\) are convolutional kernel weights. The normalization layer stabilizes training. The full encoder processes the input embedding through the following transformations:  
\[ \begin{split}\text{(a)}&\text{Feature processing to the }d_{model}\text{ dimension;}\\ \text{(b)}&\text{Multi-head temporal attention;}\\ \text{(c)}&\text{Position-wise convolutional processing;}\\ \text{(d)}&\text{Layer normalization and residual connections.}\\ \end{split}  \tag{7}\]  
where \(f_{attn}\) and \(f_{conv}\) denote the attention and convolutional operations, respectively. This hybrid architecture combines the following:  
* Global receptive field via attention;
* Local feature extraction via convolutions;
* Stable training through residual connections.