# Faster R-CNN-LSTM Construction Site Unsafe Behavior Recognition Model
## 2 Method
### Ts-Lstm
Assuming that \(OX_{t}^{l}\) is the output of the \(l\)th unit layer at time \(t\), the output and three gates of the LSTM unit can be expressed as  
\[i_{t}^{l,n}=\sigma\Big{(}\mathbf{W}_{ix}^{l,n}OX_{t}^{l,n}+\mathbf{W}_{i,h}^{l,n}h_{t-1}^{l,n}+b_{i}^{l,n}\Big{)} \tag{5}\]  
\[f_{t}^{l,n}=\sigma\Big{(}\mathbf{W}_{fx}^{l,n}OX_{t}^{l,n}+\mathbf{W}_{fh}^{l,n}h_{t-1}^{l,n}+b_{f}^{l,n}\Big{)} \tag{6}\]  
\[c_{t}^{l,n}=f_{t}^{l,n}c_{t-1}^{l,n}+i_{t}^{l,n}\text{tanh}\Big{(}\mathbf{W}_ {c}^{l,n}\mathbf{OX}_{t}^{l,n}+\mathbf{W}_{ch}^{l,n}h_{t-1}^{l,n}-b_{c}^{l,n} \Big{)} \tag{7}\]  
\[\mathbf{o}_{t}^{l,n}=\sigma\Big{(}\mathbf{W}_{ox}^{l,n}\mathbf{OX}_{t}^{l,n} +\mathbf{W}_{ch}^{l,n}h_{t-1}^{l,n}+b_{o}^{l,n}\Big{)} \tag{8}\]  
In the formula, \(i_{t}^{l,n}\) is the input gate, \(f_{t}^{l,n}\) is the forgetting gate, \(\mathbf{o}_{t}^{l,n}\) is the output gate, \(c_{t}^{l,n}\) is the activation unit, \(h_{t-1}^{l,n}\) is the output of the unit, \(W_{mn}^{l,n}\) is the connection weight matrix of the \(n\)th unit to the \(m\)th unit in the \(l\)-layer TS LSTM, and the update formula of each LSTM unit is:  
\[h_{t}^{l,n}=o_{i}^{l,n}\text{tanh}\Big{(}c_{i}^{l,n}\Big{)} \tag{9}\]  
Next, let the \(m\)th time series input be \(\mathbf{X}_{t}^{l}\), which is expressed as \(\mathbf{X}_{t}^{l,m}\) and substituted into Formulas (6)-(9). Similarly, \(h_{t}^{l,n}\) is expressed as \(h_{t}^{l,n,m}\) and substituted into Formula (10). \[q_{\text{S}}^{l,m}=\sum\limits_{t=0}^{w_{l}-1}concat\bigg{(}\Big{[}h_{n}^{l,n,m}\Big{]}_{n=0}^{n=N_{l}-1},0\bigg{)};((\cdot))_{n=0}^{n=N_{l}-1}=\Big{[} \Big{(}\cdot h_{0}(\cdot)_{1}\cdots(\cdot)_{N_{l}-1}\Big{)} \tag{10}\]  
Figure 3.0  
Expressions (11) and (12) represent the MeanPool value and the SumPool value of the \(n\)th unit in the \(l\) layer under the n time sequence, respectively. Then, it is connected in a series to obtain  
\[ \begin{cases}r_{\text{S}}^{m}=\left[q_{\text{S}}^{0,m}\right]^{\text{T}}\\ r_{\text{M}}^{m}=\Big{[}concat\Big{(}\left[q_{\text{S}}^{1,m},q_{\text{S}}^{ 2,m}\right],1\Big{)}\Big{]}^{\text{T}}\\ r_{\text{L}}^{m}=\Big{[}concat\Big{(}\left[q_{\text{S}}^{3,m},q_{\text{S}}^{4, m},q_{\text{S}}^{5,m}\right],1\Big{)}\Big{]}^{\text{T}}\end{cases}  \tag{12}\]  
The linear activation algorithm of the TS-LSTM module is given by Expressions (13)-(15). \[a_{\text{S}}^{m}=w_{\text{S}}\cdot r_{\text{S}}^{m}+b_{\text{S}} \tag{13}\]  
\[a_{\text{M}}^{m}=w_{\text{M}}\cdot r_{\text{M}}^{m}+b_{\text{M}} \tag{14}\]  
\[a_{\text{L}}^{m}=w_{\text{L}}\cdot r_{\text{L}}^{m}+b_{\text{L}} \tag{15}\]  
In the formula, \(\mathbf{w}\) and \(\mathbf{b}\) represent the weight and deviation of the linear layer, respectively. After activation, the \(k\)th action values of \(a_{\text{S}}^{m}\), \(a_{\text{M}}^{m}\), and \(a_{\text{L}}^{m}\) are set to \(a_{\text{S}}^{m,k}\), \(a_{\text{M}}^{m,k}\), and \(a_{\text{L}}^{m,k}\), respectively, and finally, they are normalized. \[\Pr(c\mid a_{\text{S}}^{m})=\frac{\exp\big{(}a_{\text{S}}^{m,c}\big{)}}{\sum \limits_{k=0}^{N_{\text{S}}-1}\exp\big{(}a_{\text{S}}^{m,k}\big{)}} \tag{16}\]\[\Pr(c\mid a_{\mathrm{M}}^{m})=\frac{\exp\left(a_{\mathrm{M}}^{m,c}\right)}{\sum \limits_{k=0}^{N_{c}-1}\exp\left(a_{\mathrm{M}}^{m,k}\right)} \tag{17}\]  
\[\Pr(c\mid a_{\mathrm{L}}^{m})=\frac{\exp\left(a_{\mathrm{L}}^{m,c}\right)}{ \sum\limits_{k=0}^{N_{c}-1}\exp\left(a_{\mathrm{L}}^{m,k}\right)} \tag{18}\]  
\(N_{c}\) and \(c\) represent the number of action types and the index of the corresponding class, respectively. The maximum likelihood estimation of the sample using the cross-entropy function is shown in Equation (20). \[e=-\sum\limits_{m=0}^{N_{M}-1}\sum\limits_{c=0}^{N_{c}-1}y_{c}^{m}\ln\{Pr\} \tag{19}\]  
In the formula, \(N_{M-1}\) and \(y_{c}^{m}\) represent the total number of training samples and the actual action labels, respectively.