# Human Activity Recognition by the Image Type Encoding Method of 3-Axial Sensor Data
## 3 Image Type Encoding Method of the 3-Axial Sensor Data
### Brightness Intensity Distribution Pattern Transformation  
Image type expansion was performed for the increased data obtained through raw sensor data segmentation. The accelerometer and gyroscope sensor data segmented into identical sizes can express 2D image patterns using the raw values that correspond to the amplitude of the continuous data, and these pattern data can be used to analyze physical activities. Figure 1 shows an example of the raw accelerometer sensor data expressed as a brightness value. The raw time series data in Figure 1a are mapped to a brightness value and visualized according to that value. In the case of transforming each point A-E of the time series graph into a brightness value, the brightness intensity distribution pattern (BIDP) for each physical activity data can be obtained, as shown in Figure 2b. Each point is expressed as a distinct brightness value according to the measured value. In the case of transformation into a 256-grayscale image, a brightness value of 128 is assigned to point A as it is located at the center between the maximum and minimum amplitudes. Point B, which has the minimum amplitude, is assigned a brightness value of 0, while point C, which has the maximum amplitude, is assigned a brightness value of 255. Points D and E are assigned brightness values of 0 and 128, respectively. First, to represent the consistent pattern of physical activity in an image format, the BIDP was transformed into a \(DSS\times DSS\) matrix by applying Equation (2) after expressing the raw data from the accelerometer and gyroscope sensors as brightness values in a \(1\times DSS\) matrix. \[BIDP=A^{T}G= \begin{bmatrix}a_{1}\\ \vdots\\ a_{DSS}\end{bmatrix}\begin{bmatrix}g_{1}&\cdots&g_{DSS}\end{bmatrix} = \begin{bmatrix}a_{1}g_{1}&\cdots&a_{1}g_{DSS}\\ \vdots&\ddots&\vdots\\ a_{DSS}g_{1}&\cdots&a_{DSS}g_{DSS}\end{bmatrix}  \tag{2}\]  
where, \(A= \begin{bmatrix}a_{1}&a_{2}&a_{3}&\cdots&a_{DSS}\end{bmatrix} \), \(G= \begin{bmatrix}g_{1}&g_{2}&g_{3}&\cdots&g_{DSS}\end{bmatrix} \). Figure 1: Brightness distribution transformation of raw sensor data: (**a**) raw data graph, and (**b**) brightness intensity distribution pattern (\(BIDP\)). Figure 2: Example of \(BIDP\) visualization. In Equation (2), \(A\) and \(G\) represent the \(1\times DSS\) size \(BDIP\) matrices of the accelerometer and gyroscope sensors that correspond to one DSS, respectively. They are transformed into images of \(DSS\times DSS\) size by taking the dot product with the transposed matrix of matrix \(A\), denoted as \(A^{T}\). Figure 2 shows the results of the \(BDIP\) dimension expansion over time. The spatial characteristics of physical activities can be obtained by discerning brightness intensity within patterns, which can be observed based on the raw sensor values. Figure 2a shows a strong area of brightness distributed at the beginning of the \(BDIP\), and the resulting image is characterized by an emphasized space at the upper left corner. Figure 2b shows a dot pattern with a strong brightness area distributed between light brightness intensities, which is emphasized at the center. Figure 2c also shows a dot pattern, but the strong brightness area is emphasized at the upper left corner instead of the center. Figure 2d is similar to Figure 2a, but the location of the brightness area differs. In short, distinct spatial characteristics can be obtained depending on the location of the strong brightness intensities, which can be used to emphasize the properties of the sensor data. However, Figure 2 shows experimental results that did not consider the 3-axial nature of the raw data. The raw accelerometer data, which comprises three axes, does not exhibit a standardized form as shown in Figure 3. Therefore, this study serializes the 3-axial sensor data to apply Equation (2) above and express the spatial characteristics of physical activities in a more accurate form. Serializing the 3-axial sensor data, as shown in Figure 3, differentiates the sensor values for each axis and expresses a more complex geometric spatial pattern.