# Design of Smart Home Control System Based on Wireless Voice Sensor

Authors: Changchun Yang
Journal: Journal of Sensors
 (2021)

---

## Abstract

With the rapid development of economy, science, and technology, people have higher and higher requirements for the quality of home life, and smart home systems are required to have higher convenience and comfort. The continuous development of embedded technology, short-range wireless communication technology, and microcontroller technology provides a new development direction for the research and development of smart home systems. Under the premise of fully studying the research status and development trend of smart home systems, this paper combines embedded technology, ZigBee wireless communication technology, voice recognition technology, GSM communication technology, etc., and proposes a ZigBee-based smart voice home appliance control system. The system adopts ZigBee wireless communication to form a comprehensive management system for household electrical appliances, electronic and electrical equipment, etc., for household electrical control, intelligent security, and environmental monitoring. This paper designs the overall scheme of a smart home control system based on voice recognition technology. According to this scheme, the hardware and software of the system are designed, and the wireless communication method combining Wi-Fi and RF (radio frequency) of the system is also determined. In order to make the system more humane, two sets of control schemes, voice control and button control, are designed according to the system’s functional requirements. In the voice control solution, the user first reads the control commands set in the voice library on the smart terminal, and then, the system automatically recognizes the commands read by the user. After the recognition is successful, the user can control the home appliances through the command. In the key control scheme, users can control home appliances through keys on the smart terminal. The upper computer transplants the audio library and the algorithm studied in this paper to realize the audio collection and storage, the confirmation of family members, and the recognition of voice commands; the lower computer is used for real-time collection of environmental information and wireless control of home facilities. After the system is started, the family members are first confirmed by voice, and then, the voice command recognition result and real-time environmental information jointly decide to form a control command. The upper computer sends the control command to the lower computer through the serial port to control the corresponding home equipment. The system is tested and analyzed under experimental conditions. The test results show that the system can control home facilities through voice commands combined with real-time environmental information while confirming family members.


---

## Body

## 1. Introduction

People’s increasing demand for living conditions has catalyzed the emergence of smart homes. Smart homes realize the interconnection of smart devices in the home, register each device on the cloud, and implement remote control through mobile terminals [1]. From a technical point of view, from the development to the present, the smart home has gone through three stages. At the beginning, the device was connected to a smart home system through the home bus technology to complete the control. The wired access method made the installation and maintenance of the smart home become cumbersome [2]. Compared with the former, this method has become more mature and more convenient to use, but it is still in the stage of “weak intelligence” and cannot meet the needs of intelligence and humanization. In recent years, related technologies have made huge breakthroughs, which have rapidly improved the accuracy of speech recognition [3]. Using voice recognition technology in smart homes, people can communicate and control in natural language with devices, which greatly improves experience and meets people’s consumer needs. From this perspective, using voice recognition in smart homes has a very broad prospect.Smart home is based on the residential environment, using wireless network technology, automated control, and voice recognition technology to integrate home appliances and facilities to establish an efficient home management system to enhance the convenience and comfort of home life [4]. Smart home connects various household facilities into a wireless network and provides multiple functions such as home appliance control, lighting control, and indoor and outdoor remote control. Smart home not only has the functions of traditional home but also has the characteristics of automatic and intelligent and can even save energy and money [5]. With the advent of the information technology era, computers have become more and more indispensable daily tools for people, and language is the most direct way of expressing human thoughts, opinions, and emotions. As a human voice interaction object, let it have the same ability to understand words as humans, which will make a huge change in people’s lives and working methods, and a voice human-computer interaction system based on voice recognition technology will emerge as the times require. It is the desire for dialogue between humans and machines to use language to inspire people’s desire for innovation, and it is the needs of social life that have promoted the rapid development of technologies such as speech recognition [6]. Embedded systems have the characteristics of portability, high performance, and low price. Combining it with speech recognition technology to realize the function of speech recognition on the embedded platform has great research and practical application significance.The purpose of smart home is to provide users with a comfortable, safe, and convenient home life experience and overcome the shortcomings of traditional electrical control solutions. Traditional home appliance control has disadvantages such as cumbersome wiring, difficult maintenance, poor scalability, and high cost. At the same time, there are problems such as aging wiring that threaten the safety of family property. The traditional electric appliance control usually adopts the push switch to control the electric appliance. Generally, the long distance between the electric appliance and the control switch requires complicated wiring, which is inconvenient to control and poor in scalability. Compared with the traditional electrical control scheme, ZigBee wireless communication is adopted between the remote control and the receiving end in this design. The system is highly integrated, and the remote control can be controlled by “one-to-many” and “many-to-one” by programming related programs. This article introduces the software design part of the smart home control system based on speech recognition technology. First, according to the function of the system, the overall design of the system software and the wireless communication method among the client, server, and controller are introduced. Secondly, it introduces the construction of the software development environment. Then, a radio frequency communication protocol was designed to make multinode control more convenient. Finally, it focuses on the software design of the main control center, control node, and intelligent terminal. In this paper, the research and design of the smart home embedded control system based on voice perception conducts an experimental test. A corpus was constructed, and experimental tests were carried out on each module. The system has been functionally tested, and the results show that the system can realize the functions of audio collection, family member confirmation, wireless control, voice command recognition, and environmental information collection and control system.


## 2. Related Work

Internationally, protocols such as HomePlug, HAVi, HomePNA, DLNA, ECHONET, and PLC are dominant in smart home and home networks. Among them, HomePlug is one of the wire communication standard interfaces defined by Panasonic, Intel, Hewlett-Packard, and other 13 companies. The standard is mainly dedicated to the establishment of open power line Internet access specifications for various information home appliances. At the same time, the standard also includes PLC technical specifications and thus establishes a complete PLC technical standard system, so that power line communication can develop in the direction of smart homes. The DLNA Digital Life Network Alliance was jointly established by Sony, Intel, Microsoft, and other companies. Based on existing mature standards, such as IP and Wi-Fi, it is aimed at enabling different brands, types of electrical appliances, PCs, and mobile devices to communicate with each other, collaborate and interconnect, and establish an interoperable platform based on open industrial standards. The bus is an important part of the smart home. It is used to connect the electrical and communication devices in the home to form a complete home network. It is a fully distributed intelligent control network technology. The current mainstream bus standards include LonWorks, ElB, BACNET, CAN, PROFIBUS, CEBuS, and APBus. The LonWorks protocol was proposed by the American Echelon Company and approved by the American National Standards Institute as a standard for controlling networks. Now, it is widely used in building automation, industrial control, etc. Finally, there is the definition of communication technology standards. The main wireless communication methods are the 802.11 series standards, 802.15.1 standards, and 802.15.4 standards formulated by IEEE. Among them, the 802.15.4 standard is the basis of ZigBee and other specifications. In addition, the M2M standard has also been widely used.In foreign countries, the research level of smart homes in western developed countries is the highest [7]. And no matter from the concept of smart home to the formulation of industry standards and the actual state of development, foreign countries are far ahead of China. Smart home is an emerging industry, and its market was regarded as a sweet and delicious cake from the beginning. Major companies have joined the research and development of smart homes, which has also promoted the rapid and vigorous development of smart homes abroad [8]. Some large multinational companies, such as Intel, Motorola, Cisco, and Panasonic, even when smart homes were just proposed and just started to develop, quickly entered this smart home market battle and created their own R&D units related to smart home appliances. The world software giant Bill Gates spent more than 40 million US dollars to build a modern digital mansion in Washington State, USA, which not only makes countless people envy but also expands their imagination. Smart homes have gradually received more attention from all walks of life and have become a comfortable and modern living environment [9]. However, many professional organizations cannot accurately define what a smart home should be. Many people equate a smart home with the Internet access function of various electrical devices, which is obviously not comprehensive. Some developers even use the concept of smart home to promote that users can fully digitize their lives just through the networking of home appliances [10]. In fact, both modern and future smart homes are far more than these.In order to make the collection of training samples easier and more efficient, relevant scholars have adopted a method of artificially adding noise to playback and recording; that is, the source voice and noise are played manually and in loudspeaker at each position and then recorded at different distances and directions [11]. Although this method speeds up the construction of the database to a certain extent, it still has a certain distance from the real situation. The noise source in the real environment is unpredictable, so it cannot fundamentally solve the shortcomings of insufficient database. The researchers adopted a spectral subtraction speech enhancement method based on stochastic resonance theory, using subsampling stochastic resonance to preprocess low signal-to-noise ratio speech signals, so that weak speech, noise, and nonlinear systems have a synergistic effect to reduce noise [12]. It can amplify the weak voice signal and then use spectral subtraction to reduce the noise for the second time on the voice signal processed by stochastic resonance. Related scholars have proposed a distributed microphone array, that is, the layout of the microphone array to a larger range: install several mono microphones on the ceiling, each microphone is 1 meter apart, and then use the algorithm to select the “best” sound source, and finally, ASR decoding and voice command matching are used to improve the recognition rate in noisy environments [13].Compared with the vigorous development of foreign speech recognition technology, the domestic start is relatively late. In the 1980s, due to the popularization of computer technology in China, the conditions for research on voice technology were available, and many scientific research units began to study voice recognition technology [14]. With the development of the “863” project, speech recognition has been listed as a special research topic and has developed rapidly. Representatives include iFLYTEK, Baidu Voice, and Ali. Xunfei uses the original deep full-sequence convolutional neural network algorithm for speech recognition model training, which makes the recognition accuracy reach more than 98% [15]. It has won multiple championships in international speech recognition competitions, and Baidu adopts streaming multilevel truncation attention. The force model SMLTA and Ali use the LFR-DFSMN model to present a scene where a hundred flowers bloom [16–18]. At this stage, China has come from behind in the field of speech recognition and has great influence [19, 20].


## 3. System Overall Structure Design and Hardware Module Design

### 3.1. The Overall Structure Design of the System

The design of the ZigBee-based smart voice home appliance control system is mainly composed of four parts: remote control, receiver (security device), mobile phone, and household appliances. Each part forms a household electrical appliance control system through the ZigBee wireless network and GSM network. The specific structure is shown in Figure1. The system realizes wireless networking through ZigBee technology and realizes the communication between the remote control and the receiving end through the wireless network, so as to send corresponding instructions to control household appliances, and users can realize remote control of home appliances through the mobile phone GSM network. At the same time, the receiving end uses the ZigBee module to receive electrical control instructions and send electrical operating status and electrical fault information to the remote control. The smart security device detects the intrusion of thieves through the infrared sensor module, and the buzzer alarms. Its ZigBee module sends information to the remote control, and the remote control sends alarm messages to the user’s mobile phone through the GSM network, realizing smart security to protect the user’s home safety.Figure 1
Overall scheme design of smart home control.


### 3.2. Design of Speech Recognition Module

The purpose of smart home is to improve the quality of home life. Reliable language recognition technology brings great convenience to life. This design adds a voice recognition module to the system remote control to achieve the effect of voice control of the status of home appliances. Voice recognition technology is widely used, such as GPS navigation, voice-activated smart toys, and automatic vending systems. Speech recognition technology is an important way to realize man-machine dialogue, and it has become an important direction of research today. Through speech recognition technology, it is possible to use speech recognition as a device for man-machine dialogue, and the operation is more convenient, intelligent, and flexible. The voice module design uses LD3320 as the main control chip. The chip contains 48 pins, which can be divided into power pins, volume control and output pins, control pins, and clock input pins according to their functions. Through the external corresponding peripheral circuit, the voice recognition function is realized. The P0~P7, MD, RDB, CSB, INTB, WRB, RSTB, A0, and other control pins of the speech recognition module LD3320 chip are connected with 1K/10K pull-up resistors to ensure the signal stability of the chip control port and assist the system to work stably; Pin 12 (MBS) is the microphone bias. To ensure that a floating voltage can be output to the microphone, this pin is connected to an RC circuit. EQ1/EQ2/EQ3 speaker volume is connected to an external auxiliary circuit to achieve the effect of adjusting the speaker volume. The LD3320 chip must be connected to an external clock. It comes with a PLL frequency synthesizer that can generate the required frequency inside the chip. The chip can accept an external clock frequency range of 4-48 MHz. The voice recognition module is connected to the single-chip microcomputer through SPI. The STM32 single-chip microcomputer PA8 provides the clock for the LD3320, SDCK receives the SPI clock signal of the single-chip microcomputer, and SCS connects the single-chip microcomputer PB5. The RSTB reset pin is connected to the PB12 pin of the microcontroller, and the INTB is connected to the PC1 pin as an interrupt output. MICP and MICN are used as input pins to input voice signals.


### 3.3. Design of Power Supply and Charging Module

In order to enhance the portability of the system, the power supply uses a combination of a rechargeable lithium battery and an ordinary power supply. This makes the remote controller of the system more flexible and more convenient. The power supply module design uses a positive low-dropout regulator ASM1117 for voltage stabilization. The chip integrates overheating and current limiting circuits, which can realize the functions of current limiting and overheating cut-off, and the stable range is -40°C~125°C. The power module can provide two voltages: one is 5.0 V; the other is a stable 3.3 V power supply provided by the ASM1117 voltage regulator chip. Two capacitors are connected to the output of ASM1117 to improve voltage stability through filtering. Connecting a diode to the output of the battery can effectively prevent the battery from being charged directly in the reverse current direction.The power supply is designed with a rechargeable lithium battery, so a charging module must be added to the circuit to charge the battery. The system charging module uses the TP4056 chip, which includes battery temperature detection, undervoltage lockout, automatic charging, soft start, and dual battery status features such as output LED pins.In this design, the charging module can work normally when the CE pin of the TP4056 chip in the charging module is connected to a high level. Pins 6 and 7 are connected to LED5 and LED4, respectively. When LED5 is on, it means that the phone is charging. When the battery is fully charged, the two capacitors C9 and C10 achieve filtering and R7, R8, and R9 complete current limiting. The charging module design can be excellently adapted to this design and improve the portability and simplicity of the smart home system.


## 3.1. The Overall Structure Design of the System

The design of the ZigBee-based smart voice home appliance control system is mainly composed of four parts: remote control, receiver (security device), mobile phone, and household appliances. Each part forms a household electrical appliance control system through the ZigBee wireless network and GSM network. The specific structure is shown in Figure1. The system realizes wireless networking through ZigBee technology and realizes the communication between the remote control and the receiving end through the wireless network, so as to send corresponding instructions to control household appliances, and users can realize remote control of home appliances through the mobile phone GSM network. At the same time, the receiving end uses the ZigBee module to receive electrical control instructions and send electrical operating status and electrical fault information to the remote control. The smart security device detects the intrusion of thieves through the infrared sensor module, and the buzzer alarms. Its ZigBee module sends information to the remote control, and the remote control sends alarm messages to the user’s mobile phone through the GSM network, realizing smart security to protect the user’s home safety.Figure 1
Overall scheme design of smart home control.


## 3.2. Design of Speech Recognition Module

The purpose of smart home is to improve the quality of home life. Reliable language recognition technology brings great convenience to life. This design adds a voice recognition module to the system remote control to achieve the effect of voice control of the status of home appliances. Voice recognition technology is widely used, such as GPS navigation, voice-activated smart toys, and automatic vending systems. Speech recognition technology is an important way to realize man-machine dialogue, and it has become an important direction of research today. Through speech recognition technology, it is possible to use speech recognition as a device for man-machine dialogue, and the operation is more convenient, intelligent, and flexible. The voice module design uses LD3320 as the main control chip. The chip contains 48 pins, which can be divided into power pins, volume control and output pins, control pins, and clock input pins according to their functions. Through the external corresponding peripheral circuit, the voice recognition function is realized. The P0~P7, MD, RDB, CSB, INTB, WRB, RSTB, A0, and other control pins of the speech recognition module LD3320 chip are connected with 1K/10K pull-up resistors to ensure the signal stability of the chip control port and assist the system to work stably; Pin 12 (MBS) is the microphone bias. To ensure that a floating voltage can be output to the microphone, this pin is connected to an RC circuit. EQ1/EQ2/EQ3 speaker volume is connected to an external auxiliary circuit to achieve the effect of adjusting the speaker volume. The LD3320 chip must be connected to an external clock. It comes with a PLL frequency synthesizer that can generate the required frequency inside the chip. The chip can accept an external clock frequency range of 4-48 MHz. The voice recognition module is connected to the single-chip microcomputer through SPI. The STM32 single-chip microcomputer PA8 provides the clock for the LD3320, SDCK receives the SPI clock signal of the single-chip microcomputer, and SCS connects the single-chip microcomputer PB5. The RSTB reset pin is connected to the PB12 pin of the microcontroller, and the INTB is connected to the PC1 pin as an interrupt output. MICP and MICN are used as input pins to input voice signals.


## 3.3. Design of Power Supply and Charging Module

In order to enhance the portability of the system, the power supply uses a combination of a rechargeable lithium battery and an ordinary power supply. This makes the remote controller of the system more flexible and more convenient. The power supply module design uses a positive low-dropout regulator ASM1117 for voltage stabilization. The chip integrates overheating and current limiting circuits, which can realize the functions of current limiting and overheating cut-off, and the stable range is -40°C~125°C. The power module can provide two voltages: one is 5.0 V; the other is a stable 3.3 V power supply provided by the ASM1117 voltage regulator chip. Two capacitors are connected to the output of ASM1117 to improve voltage stability through filtering. Connecting a diode to the output of the battery can effectively prevent the battery from being charged directly in the reverse current direction.The power supply is designed with a rechargeable lithium battery, so a charging module must be added to the circuit to charge the battery. The system charging module uses the TP4056 chip, which includes battery temperature detection, undervoltage lockout, automatic charging, soft start, and dual battery status features such as output LED pins.In this design, the charging module can work normally when the CE pin of the TP4056 chip in the charging module is connected to a high level. Pins 6 and 7 are connected to LED5 and LED4, respectively. When LED5 is on, it means that the phone is charging. When the battery is fully charged, the two capacitors C9 and C10 achieve filtering and R7, R8, and R9 complete current limiting. The charging module design can be excellently adapted to this design and improve the portability and simplicity of the smart home system.


## 4. Software Design of Smart Home Control System

### 4.1. Hidden Markov Speech Recognition Model

The hidden Markov model is a kind of mathematical statistical model, which is used to describe the process of transition from one state to another. Because of this, we need to find various state parameters in the hidden Markov model from the observed transition state and apply these parameters to the speech recognition system. Compared with the Markov model, the state of the hidden Markov model is hidden. We cannot observe it intuitively. We can only infer its state based on the visible state and the probability of its hidden state. The more difficult part is to determine the hidden parameters in the process from the observable parameters and then use these parameters for the next step of analysis.To use the hidden Markov model in speech recognition, the first step is to determine the topological form of the hidden Markov model. The topological form of the hidden Markov model is distinguished according to the appearance of the Markov chain, which can be divided into each state ergodic type, left-right type with spanning and left-right type without spanning.The left-right model is widely used in speech recognition, especially in the recognition of isolated words for unspecified persons. As shown in Figure2, this is a left-right hidden Markov model without spanning. It can be seen from the figure that the four states from S1 to S4 can jump over time or stay in this state.Figure 2
Hidden Markov model state transition diagram.“A” indicates the probability of a set of state transitions:
(1)A=aij,aij=Pqt+1=Sj−1,qt−1=Si+1,i,j∈0,N−1.“B” represents the distribution of observed symbol probability in the Sj state:
(2)B=bj+1Ok.“Π” represents the probability of initial state distribution:
(3)πi=Pq1=Si+1,i∈0,N+1.The Baum-Welch algorithm can solve the template training problem of the hidden Markov model, that is, the problem of data estimation of the hidden Markov model. Given a sequence of observationsO=O1,O2,⋯,OT, using the Baum-Welch algorithm, we can determine λ=π,A,B, so that PQ,O丨λ is the maximum. From front to back variables, there are
(4)POλ=∏i=0N−1∏j=0N−1aijbiOt−1αtiβtj+1,t∈0,T+1.In the formula,αti represents the output observation sequence O=O1,O2,⋯,OT, and the probability of reaching the state θi, and βt+1j represents the number sequence starting from the state Sj. Find λ so that PQ,O丨λ is the maximum value. This is a problem of extreme values of functional functions. Since the template training sequence is limited, there is no best estimation method to estimate λ. The Baum-Welch algorithm uses a recursive method to keep PQ,O丨λ locally maximized, and finally, the model parameters λ=π,A,B can be obtained.We defineξti,j as the probability that given the training sequence O and the model λ, the Markov chain at time t is in the θi state and t+1 is in the θj state, namely,
(5)ξti,j=Pqt=θi,qt+1=θjλ.At timet, the mathematical probability ξti of the Markov chain in the θi state is defined as
(6)ξti=Pqt=θiλ=∏j=0N−1ξti,j.In this way, we have obtained the revaluation formula of the Baum-Welch algorithm:(7)aij′=∏t=0T−1ξti,j∏t=0T−1λi,bjk′=∏t=0T−1λj∏t=0T−1λi.


### 4.2. Software Development Environment Construction

The construction of the software development environment mainly includes the development environment of the PC terminal and the JZ2440 development board. The PC side is relatively simple; we only need to install QT software editor, keil5 software tool, and Linux operating system. Utilizing the Linux system has the advantages of open source code, easy cutting, convenient transplantation, security, and stability. The development environment of the JZ2440 development board is built with an embedded Linux system. The construction process includes the transplantation of Bootloader, the transplantation of the kernel, and the production of the root file system.The version of U-boot used in this topic is U-boot-2012.04.01, and the kernel version is Linux-3.4.2. We build the required root file system and then burn the U-boot, kernel, and root file system. At this point, the development environment of the JZ2440 development board is completed. In order for the software written on the embedded operating system to run on the ARM development board, you need to install the arm-linux-gcc cross-compiler in the embedded system before setting up the ARM development environment. The cross-compiler version used in this topic is arm-linux-gcc-4.3.2 compiler. All the source code compressed packages mentioned above can be downloaded from the Internet.


### 4.3. Main Control Center Software Design

As the server, the main control center is the main part of the smart home control system and the hub that connects smart terminals and control nodes. On the one hand, the main control center is responsible for parsing the instructions sent from the smart terminal, and the instructions can be parsed into control commands or monitoring commands. If it is a control command, the main control center will send the command to the control node for processing; if it is a monitoring command, the main control center will process the stored video-play, close or forward. On the other hand, the main control center receives the response information fed back by the control node and feeds the response information back to the smart terminal. The main control center software design flow chart is shown in Figure3.Figure 3
Flow chart of the software design of the main control center.


### 4.4. RF Communication Protocol Design

Radio frequency is a communication method for wireless communication between the main control center and each control node. As the nRF905 radio frequency module itself does not have networking capabilities, a communication protocol is designed to make the system have networking capabilities.In order to ensure orderly and reliable communication between the main control center and each node, the address of the main control center and each node is unique. The main control center can communicate with all nodes in both directions, but it can only communicate with one node at a time. The communication protocol data packet includes preamble, source address, destination address, control type, control command, and CRC check code. The preamble and CRC check code are automatically generated by nRF905.In the communication protocol data packet, the source address, destination address, and control type all occupy 1 byte, and the control command occupies 2 bytes. Since the default address of the radio frequency module is 0x00, in order to ensure the reliability of the system, the source address and destination are not 0. In this system, the main control center address is 0x01, the infrared node address range is 0x01~0x8F, and the relay node address range is 0x90~0xFF. Therefore, the system can support up to 143 infrared nodes and 112 relay nodes, and each node can support up to 65536 control commands, which can meet the needs of most ordinary houses. The control type represents the type of home appliances. The control types designed in this article include 6 types of air conditioners, TVs, set-top boxes, fans, projectors, and sockets. When the main control center sends data, the source address is set to 0x01, and the destination address is set to the destination node address. When a node sends a response message to the master control center, the source address is the address of each node itself, and the destination address is the master control center address 0x01. When sending a radio frequency signal, the processor packs the source address, destination address, and control command together and sends it to nRF905. The nRF905 will automatically load the preamble and CRC check code. After the load is successful, the signal can be sent out. When receiving a radio frequency signal, nRF905 first verifies the preamble and CRC check code, and then, the program verifies the source address and destination address to check whether the signal is sent to itself; if not, ignore it; if it is, then receive the signal.


### 4.5. Intelligent Terminal Voice Recognition Control Scheme Design

The user first enters the login interface of the smart home voice control system. In this interface, the user first needs to log in to the home gateway, and only under the home gateway can the home appliances be controlled. This system can only log in users who have already registered. Those who have not registered need to register before they can log in. The user name and password registered by the user are saved in the sqlite database of the main control center. A “super user login” is also designed under this interface. In this mode, users can add or delete control nodes.When the “air conditioning” button is pressed, it enters the voice air conditioning control interface; when the “TV” button is pressed, it enters the voice TV control interface. The user only needs to press and hold the voice input button. If the voice input by the user can match the voice in the voice library, the voice control function can be realized. If the match fails, the user will be prompted. The voice control function of other infrared home appliances is similar to this.


## 4.1. Hidden Markov Speech Recognition Model

The hidden Markov model is a kind of mathematical statistical model, which is used to describe the process of transition from one state to another. Because of this, we need to find various state parameters in the hidden Markov model from the observed transition state and apply these parameters to the speech recognition system. Compared with the Markov model, the state of the hidden Markov model is hidden. We cannot observe it intuitively. We can only infer its state based on the visible state and the probability of its hidden state. The more difficult part is to determine the hidden parameters in the process from the observable parameters and then use these parameters for the next step of analysis.To use the hidden Markov model in speech recognition, the first step is to determine the topological form of the hidden Markov model. The topological form of the hidden Markov model is distinguished according to the appearance of the Markov chain, which can be divided into each state ergodic type, left-right type with spanning and left-right type without spanning.The left-right model is widely used in speech recognition, especially in the recognition of isolated words for unspecified persons. As shown in Figure2, this is a left-right hidden Markov model without spanning. It can be seen from the figure that the four states from S1 to S4 can jump over time or stay in this state.Figure 2
Hidden Markov model state transition diagram.“A” indicates the probability of a set of state transitions:
(1)A=aij,aij=Pqt+1=Sj−1,qt−1=Si+1,i,j∈0,N−1.“B” represents the distribution of observed symbol probability in the Sj state:
(2)B=bj+1Ok.“Π” represents the probability of initial state distribution:
(3)πi=Pq1=Si+1,i∈0,N+1.The Baum-Welch algorithm can solve the template training problem of the hidden Markov model, that is, the problem of data estimation of the hidden Markov model. Given a sequence of observationsO=O1,O2,⋯,OT, using the Baum-Welch algorithm, we can determine λ=π,A,B, so that PQ,O丨λ is the maximum. From front to back variables, there are
(4)POλ=∏i=0N−1∏j=0N−1aijbiOt−1αtiβtj+1,t∈0,T+1.In the formula,αti represents the output observation sequence O=O1,O2,⋯,OT, and the probability of reaching the state θi, and βt+1j represents the number sequence starting from the state Sj. Find λ so that PQ,O丨λ is the maximum value. This is a problem of extreme values of functional functions. Since the template training sequence is limited, there is no best estimation method to estimate λ. The Baum-Welch algorithm uses a recursive method to keep PQ,O丨λ locally maximized, and finally, the model parameters λ=π,A,B can be obtained.We defineξti,j as the probability that given the training sequence O and the model λ, the Markov chain at time t is in the θi state and t+1 is in the θj state, namely,
(5)ξti,j=Pqt=θi,qt+1=θjλ.At timet, the mathematical probability ξti of the Markov chain in the θi state is defined as
(6)ξti=Pqt=θiλ=∏j=0N−1ξti,j.In this way, we have obtained the revaluation formula of the Baum-Welch algorithm:(7)aij′=∏t=0T−1ξti,j∏t=0T−1λi,bjk′=∏t=0T−1λj∏t=0T−1λi.


## 4.2. Software Development Environment Construction

The construction of the software development environment mainly includes the development environment of the PC terminal and the JZ2440 development board. The PC side is relatively simple; we only need to install QT software editor, keil5 software tool, and Linux operating system. Utilizing the Linux system has the advantages of open source code, easy cutting, convenient transplantation, security, and stability. The development environment of the JZ2440 development board is built with an embedded Linux system. The construction process includes the transplantation of Bootloader, the transplantation of the kernel, and the production of the root file system.The version of U-boot used in this topic is U-boot-2012.04.01, and the kernel version is Linux-3.4.2. We build the required root file system and then burn the U-boot, kernel, and root file system. At this point, the development environment of the JZ2440 development board is completed. In order for the software written on the embedded operating system to run on the ARM development board, you need to install the arm-linux-gcc cross-compiler in the embedded system before setting up the ARM development environment. The cross-compiler version used in this topic is arm-linux-gcc-4.3.2 compiler. All the source code compressed packages mentioned above can be downloaded from the Internet.


## 4.3. Main Control Center Software Design

As the server, the main control center is the main part of the smart home control system and the hub that connects smart terminals and control nodes. On the one hand, the main control center is responsible for parsing the instructions sent from the smart terminal, and the instructions can be parsed into control commands or monitoring commands. If it is a control command, the main control center will send the command to the control node for processing; if it is a monitoring command, the main control center will process the stored video-play, close or forward. On the other hand, the main control center receives the response information fed back by the control node and feeds the response information back to the smart terminal. The main control center software design flow chart is shown in Figure3.Figure 3
Flow chart of the software design of the main control center.


## 4.4. RF Communication Protocol Design

Radio frequency is a communication method for wireless communication between the main control center and each control node. As the nRF905 radio frequency module itself does not have networking capabilities, a communication protocol is designed to make the system have networking capabilities.In order to ensure orderly and reliable communication between the main control center and each node, the address of the main control center and each node is unique. The main control center can communicate with all nodes in both directions, but it can only communicate with one node at a time. The communication protocol data packet includes preamble, source address, destination address, control type, control command, and CRC check code. The preamble and CRC check code are automatically generated by nRF905.In the communication protocol data packet, the source address, destination address, and control type all occupy 1 byte, and the control command occupies 2 bytes. Since the default address of the radio frequency module is 0x00, in order to ensure the reliability of the system, the source address and destination are not 0. In this system, the main control center address is 0x01, the infrared node address range is 0x01~0x8F, and the relay node address range is 0x90~0xFF. Therefore, the system can support up to 143 infrared nodes and 112 relay nodes, and each node can support up to 65536 control commands, which can meet the needs of most ordinary houses. The control type represents the type of home appliances. The control types designed in this article include 6 types of air conditioners, TVs, set-top boxes, fans, projectors, and sockets. When the main control center sends data, the source address is set to 0x01, and the destination address is set to the destination node address. When a node sends a response message to the master control center, the source address is the address of each node itself, and the destination address is the master control center address 0x01. When sending a radio frequency signal, the processor packs the source address, destination address, and control command together and sends it to nRF905. The nRF905 will automatically load the preamble and CRC check code. After the load is successful, the signal can be sent out. When receiving a radio frequency signal, nRF905 first verifies the preamble and CRC check code, and then, the program verifies the source address and destination address to check whether the signal is sent to itself; if not, ignore it; if it is, then receive the signal.


## 4.5. Intelligent Terminal Voice Recognition Control Scheme Design

The user first enters the login interface of the smart home voice control system. In this interface, the user first needs to log in to the home gateway, and only under the home gateway can the home appliances be controlled. This system can only log in users who have already registered. Those who have not registered need to register before they can log in. The user name and password registered by the user are saved in the sqlite database of the main control center. A “super user login” is also designed under this interface. In this mode, users can add or delete control nodes.When the “air conditioning” button is pressed, it enters the voice air conditioning control interface; when the “TV” button is pressed, it enters the voice TV control interface. The user only needs to press and hold the voice input button. If the voice input by the user can match the voice in the voice library, the voice control function can be realized. If the match fails, the user will be prompted. The voice control function of other infrared home appliances is similar to this.


## 5. System Function Test and Analysis

### 5.1. Experimental Program

According to the composition of the personnel making the speech library, the experimental site simulates the real home environment, and the key laboratory of ordinary noise perception technology and intelligent system is selected. The mini model is chosen to be installed in the laboratory to simulate the entire home environment. The light, temperature, and humidity of natural conditions are used to simulate the changes of home environment parameters. The curtains, bedroom lights, and living room lights are replaced by mini curtains and lamp tubes, and two mini fans are used to simulate air conditioning cooling and heating command signals. The TV control signal is replaced by indicator lights, and the remaining indicator lights are used to show that it is convenient to add home facilities at any time.A simple model is used to simulate the home environment, and a smart home embedded control system model based on voice perception is used to conduct experiments to test the confirmation effect of real family members and test the control effect of the fusion of home environment information with the voice perception recognition result on the simulated home facilities.The different member thresholds shown in Figure4 are used in the experiment. After the system is started, the audio codec chip collects the voices of the testers for 5 s from 100 members, calculates the likelihood, and obtains the corresponding correct rate. In the actual environment, environmental conditions are artificially changed. In order to achieve better and more obvious effect, this paper sets the environmental parameter threshold as shown in Figure 5. Except for the system wake-up command, the experimental tests are all tested.Figure 4
Normalized threshold for 100 members.Figure 5
Normalized threshold experiment of environmental parameters.


### 5.2. Environmental Information Collection Control Test

The communication between the host computer and the coordinator is completed through the serial port. The environmental information collection and the test of the control module in this article are carried out through the serial port debugging assistant on the PC. It is equivalent to the control command sent by the upper computer to the lower computer, which makes the test process more clear. First, insert the designed lower-level computer coordinator into the USB interface of the PC; then, check the serial port number; open the serial debugging assistant software, strictly follow the serial port structure in the serial communication program design to set the serial port, and finally open the serial port.Turn on the coordinator switch, and power on the acquisition node and the control node, and wait 3 seconds to complete the networking and network access. Finally, the query and control commands are sent to the coordinator through the serial port. You perform environmental information acquisition test according to the query command. You can get a single command to control each device as shown in Table1. After testing, the coordinator is correctly networked after power-on, and each node is correctly connected to the network; when the configuration of the serial port structure is consistent, the command is sent in hexadecimal, and the data transmission and control are error-free, and the functional design fully meets the control requirements. The control of the scene mode is equivalent to the control of all devices together, and the system selects two modes: sleeping mode and away from home mode. The time-consuming result of the scenario mode test is shown in Figure 6.Table 1
Control device command data format.
Smart home control equipmentControl command (open)Control command (closed)Living room lights1A 00 01 06 01 1E 211A 00 01 06 00 1F 21Bedroom lights1A 00 01 05 01 1D 211A 00 01 05 00 1C 21Air conditioning and refrigeration1A 00 01 01 01 1B 211A 00 01 01 00 1A 21Air conditioning and heating1A 00 01 08 00 10 211A 00 01 09 01 10121Curtain1A 00 01 04 01 1C 211A 00 01 04 00 1D 21Television1A 00 01 07 01 1F 211A 00 01 07 00 1E 21Figure 6
Time consumption of scenario mode testing.


### 5.3. Test Results of Overall System Control Effect

After the collected audio features are extracted, the likelihood is calculated, and the maximum likelihood is obtained, and it is judged whether the maximum is within the threshold range of the corresponding number of the maximum. If it is within the threshold, it is confirmed as a family member. The statistical results of confirmation of family members are shown in Figure7. It can be seen that the confirmation accuracy rates of family members under different thresholds are all above 90%.Figure 7
Confirmation accuracy rate of family members under different thresholds.After actual tests in different environments, the system can work as designed. Figure8 is the summary statistical results of speech recognition accuracy corresponding to the number of different sentences.Figure 8
Speech recognition accuracy rate under different numbers of sentences.If you want to observe the specific execution of the system, you can connect the serial port of the host computer to the securtCRT terminal and set the corresponding protocol. After the system is turned on, you can see the execution of the system in the terminal.In the confirmation and recognition of family members, the selection of the threshold directly affects the recognition rate, and the threshold setting is to prevent people who are not family members from controlling the smart home system. If the threshold is set too high, the effect of improving system security can be achieved, but it will cause a drop in the correct recognition rate. Therefore, the threshold setting should be determined based on actual measurement. The reason why the log likelihood result is negative is that the probability is a number less than 1. If these probabilities are multiplied, it is close to 0. In order to reduce the error, we take the logarithm of the probability to a negative value, and then add it. In this way, it can be judged by the maximum log likelihood that it corresponds to that model, and then, it can be judged whether it is a family member by the threshold of the log likelihood of the model. The threshold is relaxed (that is, the environment is between the thresholds at the time), so it is convenient to experiment. After the voice command is issued, the system can correctly execute the predetermined operation after judging according to the environmental information, which proves the correctness of the system.


## 5.1. Experimental Program

According to the composition of the personnel making the speech library, the experimental site simulates the real home environment, and the key laboratory of ordinary noise perception technology and intelligent system is selected. The mini model is chosen to be installed in the laboratory to simulate the entire home environment. The light, temperature, and humidity of natural conditions are used to simulate the changes of home environment parameters. The curtains, bedroom lights, and living room lights are replaced by mini curtains and lamp tubes, and two mini fans are used to simulate air conditioning cooling and heating command signals. The TV control signal is replaced by indicator lights, and the remaining indicator lights are used to show that it is convenient to add home facilities at any time.A simple model is used to simulate the home environment, and a smart home embedded control system model based on voice perception is used to conduct experiments to test the confirmation effect of real family members and test the control effect of the fusion of home environment information with the voice perception recognition result on the simulated home facilities.The different member thresholds shown in Figure4 are used in the experiment. After the system is started, the audio codec chip collects the voices of the testers for 5 s from 100 members, calculates the likelihood, and obtains the corresponding correct rate. In the actual environment, environmental conditions are artificially changed. In order to achieve better and more obvious effect, this paper sets the environmental parameter threshold as shown in Figure 5. Except for the system wake-up command, the experimental tests are all tested.Figure 4
Normalized threshold for 100 members.Figure 5
Normalized threshold experiment of environmental parameters.


## 5.2. Environmental Information Collection Control Test

The communication between the host computer and the coordinator is completed through the serial port. The environmental information collection and the test of the control module in this article are carried out through the serial port debugging assistant on the PC. It is equivalent to the control command sent by the upper computer to the lower computer, which makes the test process more clear. First, insert the designed lower-level computer coordinator into the USB interface of the PC; then, check the serial port number; open the serial debugging assistant software, strictly follow the serial port structure in the serial communication program design to set the serial port, and finally open the serial port.Turn on the coordinator switch, and power on the acquisition node and the control node, and wait 3 seconds to complete the networking and network access. Finally, the query and control commands are sent to the coordinator through the serial port. You perform environmental information acquisition test according to the query command. You can get a single command to control each device as shown in Table1. After testing, the coordinator is correctly networked after power-on, and each node is correctly connected to the network; when the configuration of the serial port structure is consistent, the command is sent in hexadecimal, and the data transmission and control are error-free, and the functional design fully meets the control requirements. The control of the scene mode is equivalent to the control of all devices together, and the system selects two modes: sleeping mode and away from home mode. The time-consuming result of the scenario mode test is shown in Figure 6.Table 1
Control device command data format.
Smart home control equipmentControl command (open)Control command (closed)Living room lights1A 00 01 06 01 1E 211A 00 01 06 00 1F 21Bedroom lights1A 00 01 05 01 1D 211A 00 01 05 00 1C 21Air conditioning and refrigeration1A 00 01 01 01 1B 211A 00 01 01 00 1A 21Air conditioning and heating1A 00 01 08 00 10 211A 00 01 09 01 10121Curtain1A 00 01 04 01 1C 211A 00 01 04 00 1D 21Television1A 00 01 07 01 1F 211A 00 01 07 00 1E 21Figure 6
Time consumption of scenario mode testing.


## 5.3. Test Results of Overall System Control Effect

After the collected audio features are extracted, the likelihood is calculated, and the maximum likelihood is obtained, and it is judged whether the maximum is within the threshold range of the corresponding number of the maximum. If it is within the threshold, it is confirmed as a family member. The statistical results of confirmation of family members are shown in Figure7. It can be seen that the confirmation accuracy rates of family members under different thresholds are all above 90%.Figure 7
Confirmation accuracy rate of family members under different thresholds.After actual tests in different environments, the system can work as designed. Figure8 is the summary statistical results of speech recognition accuracy corresponding to the number of different sentences.Figure 8
Speech recognition accuracy rate under different numbers of sentences.If you want to observe the specific execution of the system, you can connect the serial port of the host computer to the securtCRT terminal and set the corresponding protocol. After the system is turned on, you can see the execution of the system in the terminal.In the confirmation and recognition of family members, the selection of the threshold directly affects the recognition rate, and the threshold setting is to prevent people who are not family members from controlling the smart home system. If the threshold is set too high, the effect of improving system security can be achieved, but it will cause a drop in the correct recognition rate. Therefore, the threshold setting should be determined based on actual measurement. The reason why the log likelihood result is negative is that the probability is a number less than 1. If these probabilities are multiplied, it is close to 0. In order to reduce the error, we take the logarithm of the probability to a negative value, and then add it. In this way, it can be judged by the maximum log likelihood that it corresponds to that model, and then, it can be judged whether it is a family member by the threshold of the log likelihood of the model. The threshold is relaxed (that is, the environment is between the thresholds at the time), so it is convenient to experiment. After the voice command is issued, the system can correctly execute the predetermined operation after judging according to the environmental information, which proves the correctness of the system.


## 6. Conclusion

As a product of the new era, smart home has its unique mission. How to meet people’s life requirements to a greater extent is the ultimate direction of smart home development. In order to continue to develop in this direction, research on embedded voice control systems for smart homes is necessary. In the context of the research on smart homes, this paper combines voice control, ZigBee technology, and the most popular operating system and, after considering the cost, proposes a set of feasible voice control system solutions. The intelligent terminal recognizes the voice signal as a specific voice command through the process of preprocessing, feature extraction, pattern matching, postprocessing, etc., and sends it to the main control center for analysis. The radio frequency communication sends the voice command to the corresponding control node, and finally, the node finds the control command through the voice command and sends the command to control the home appliance and, at the same time, responds and feedbacks to the main control center. The software structure of the system is based on the C/S (Client/Server) structure; that is, the intelligent terminal is the client, the main control center is the server, and the control node is the controller. The client and the server communicate through the TCP protocol, and the server and the controller communicate wirelessly through the radio frequency protocol. This article implements functions such as audio collection, confirmation of family members, recognition of voice commands, real-time environmental information collection, and sending of control commands and conducts experimental tests on various functional modules, and the test results are normal. The test results show that family members can correctly control the system and achieve the goal of correct control of home facilities through voice commands combined with family environment information. However, the star-shaped network topology currently adopted by the system is simple and easy to use, but when there are a large number of sensors, the system needs a large number of information collection nodes, which will greatly increase the system cost. At this time, the star structure is not as cost-effective as the tree and mesh structure. Therefore, at this point, the system needs to be improved, depending on the specific situation. In addition, since the system needs to communicate with the cloud frequently when it is working, this process is very time-consuming, and wireless communication also takes time, making the response of the entire system a bit slow. If the response speed of the entire system can be improved, the effect will be even better.



---

*Source: 8254478-2021-10-26.xml*