# Activity Recognition Using Different Sensor Modalities and Deep Learning
## 2 Materials and Methods  
In our previous work, we built a modular and expandable WSN, which consists of a master device and IMU slave devices [27]. The master device establishes and manages the wireless network, as well as collects and records incoming information from the slave devices. Slave devices connect to the managed network by the master device via Wi-Fi communication and send acceleration and gyroscope outputs to the master. The software on the master device interactively accepts various management commands that can be sent via a web interface on the same wireless network. Management commands range from starting the data flow of slave devices to viewing the status of the connected devices on the network. Moreover, the user can access and download recorded files from the master device from the web interface. For this study, the existing WSN was expanded by adding slave smart insoles. The block diagram of the updated WSN is shown in Figure 1.  
Wireless IMU devices consist of a 3-axis accelerometer, a 3-axis gyroscope, and Wi-Fi modules. The built-in controller of the Wi-Fi module is utilized to collect the raw data from the accelerometer and the gyroscope. Then, these datasets are sent to the master device via Wi-Fi. The designed IMU devices are shielded by a box to isolate the circuit from users. Figure 1: The block diagram of the designed wireless sensor network. The enclosure box has an attached elastic rope and stopper. The rope and the stopper allow the user to comfortably mount and secure the sensor on the human body. Wireless smart insoles were designed to collect foot contact states during activities. The electronic parts of the insole consist of FSRs, a small-size and low-capacity battery, and a development board supporting Wi-Fi communication. Four FSRs were used in each insole design. These sensors were placed on the heel, medial forefoot (MF), lateral forefoot (LF), and toe area of the foot by examining many studies in the literature [28; 29; 30; 31; 32]. FSRs transmit their outputs via the analog interface to the development board, which processes the incoming FSR outputs and transfers the master device via Wi-Fi connectivity. The system also has a two-way switch. When the switch is positioned at \"on\", the designed smart insole connects to the sensor network and waits for broadcast commands from the master device; these commands are transmitted to all slaves. Then, the master device continuously and sequentially requests data transmission from the slave devices, which prompts them to read and send sensor data. Moreover, these requests also allow the slave devices to work synchronously with each other. When the switch is positioned at \"off\", the battery can easily charge without removing it from the insole. The designed smart insole can be seen in Figure 2.  
We have used a memory foam sole in the insole design. It has advantages including flexibility, cushioning, and moisture absorption. This type of sole helps to reduce shocks and vibration and prevents perspiration, as well as providing optimal walking comfort. The obtained FSR outputs were converted into foot contact states as \"0\" or \"1\" by using a threshold value. In other words, weight measurements were accepted as \"No contact\" when they were less than a threshold value but were otherwise accepted as \"Contact\". There are many types of FSRs. We have preferred using circular and small-size FSRs for the toe, LF, and MF areas, while square and wider-size FSRs are used for the heel area in our insole design. These sensors have the same specifications except for size. In this study, we have used a total of four slave sensors, namely, two smart insoles and two IMU sensors. Smart insoles were placed inside of the shoe sole for both the right and left feet, and IMU sensors were attached on the thigh part of each leg, as seen in Figure 3. The sampling frequency of all slave nodes was configured as 40 Hz.