SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(4); March (2023), pp. 390–395
ISSN (Online):2352-6386; ISSN (Print):2405-9021
https://alife-robotics.org/jrnal.html

Zilong Liu1, Haoyang Gui1, Luqi Shen1, Longyu Gao1, Fengzhi Dai1,2
1

Tianjin University of Science and Technology, China
Tianjin Tianke Intelligent and Manufacture Technology CO., LTD, China

2

Received 22 November 2021
Accepted 16 April 2023

Smart window
New structure
Machine vision

With the vigorous advancement of intelligent manufacturing, smart home has gradually enteredpeople's daily life. At present, smart window always have the problem of low intelligence and afew functions. And almost all kinds of traditional windows have the problem of inconvenient
cleaning outside. The paper proposed a new mechanical structure design of the window, so thatthe window can be turned by nearly 160 degrees, which is convenient for cleaning the both sidesof the window. In terms of intelligent systems, a variety of sensors combined with Internet
modules are used to realize the remote intelligent control (turning with the wind, andautomatically closing windows when there is no one at home when bad weather). Window adds
a child mode, which is based on machine vision. In this mode, when the window detects that there
is a child nearby, the window will be automatically closed to effectively avoid the accident ofchildren falling from the building caused by climbing the window.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

the window, which will affect the light transmission andaesthetics of the window for a long time.
Manual control: Traditional windows still require
manual opening or closing.
There are potential safety hazards: Children may fallfrom buildings because the windows do not haveprotective equipment. The window structure is simple
and easy to be damaged, which leads to the problem ofburglary and brings economic losses to users.

1. Introduction
With the development of science and technology and theimprovement of people's living standards, the smart
window industry has developed rapidly around the world.
Thanks to the development of the construction industry,
China's smart window market has grown steadily year byyear, occupying an important market position. It can beseen that smart windows are the industry and the future
development trend.
On this basis, the team first conducted research on
existing window products. Fig. 1 shows the mainstream
traditional windows currently on the market. It is notdifficult to see that traditional windows mainly have thefollowing disadvantages:
Inconvenient cleaning: limited by its structural design,
it is difficult for users to clean the glass from outside ofFig.1. Traditional window structure
Corresponding author’s E-mail: daifz@tust.edu.cn URL: www.tust.edu.cn390

eIn the research on smart windows, it is found that thedefects of existing smart windows are mainly manifested
in the following aspects:
Low degree of intelligence: all kinds of intelligent
windows have single functions. For example, smart
dimming glass windows study the light transmission ofglass; smart security windows are developed in terms ofautomatic alarms.
Inconvenient installation: Fig. 2 shows the control
principle of a mainstream smart window product on themarket. By adding a slider to the window or the wall next
to the window, a controller is installed on the slider, andthe controller is connected to the window. Through thecontroller’s movement on the slider controls the opening
and closing of the window. This puts higher demands on
the strength of the window frame and the space reserved
for window installation. But this has not changed theinconvenience caused by the traditional window structure,and the disadvantage of inconvenient cleaning still exists.

The control system is composed of core controller, sensor,
display, motor and Bluetooth module.
2.1. Hardware Design of Control System
The control system is composed of Arduino UNO andESP8266, which is used to process the data and control
commands received by the sensor and Bluetooth. Sensors
include compass module, temperature and humidity
sensor, air dust sensor, rain sensor, harmful gas sensor,
and combustible gas sensor [1], which are responsible forsensing changes in the surrounding environment. For
example, the harmful gas sensor will automatically open
the window when harmful gas is detected.
A 1.3-inch OLED screen is used to display the datacollected by the sensors installed on the window. Stepper
motors and stepper motor drivers are used to control themechanical structure movement of the window. The mainelements of the window are shown in Fig. 3.

Fig.3. The main elements of the window

2.2. Hardware connection of Control System

Fig.2. Traditional smart windows

Aiming at the problems of existing traditional
windows and smart windows, the team designed a newsmart window. First of all, a new design is made on themechanical structure of the window, so that the windowcan be turned 160°, which solves the problem of cleaning
the outside. At the same time, the structure is integrated
with the window frame for easy installation.
At the same time, adding a variety of sensors enables
the window to adapt to more usage scenarios. The child
identification and alarm module is also added to improvesafety.

The core controller receives and analyzes the data sentby the sensors connected to it through the Arduino
UNO. With the processed data, the core controller candrive the stepper motor to control the movement of thewindow's mechanical structure, thereby changing thestate of the window.
The 1.3-inch OLED screen is connected to thecontroller via the I2C bus. The frequency of screen
drawing and refreshing is controlled by the Arduino
UNO timer, refreshing once every certain time, so thatthe controller can process the received information or
instructions in time [2].

2. Control System Design391

e

The Bluetooth module on the smart window isconnected with the controller through the USART bus torealize the wireless transceiver function in close range.
The connection mode of each sensor and the one-chipcomputer is shown in Fig.4.

move in the horizontal direction through the movementof the lead screw, and the window is rotated by theconnecting rod during the movement of the slide), only
one motor can realize the window flip nearly 160 degrees.
The connecting rod structure and the window mechanicalstructure are shown in Fig. 6 and 7 respectively.

Fig.6. Connection rod structure
Fig.4. Nearly 160° flipped windows

A stepping motor is connected between the controller
and the mechanical structure of the smart window, andthe stepping motor is controlled by sending a pulse signal
to the motor driver to open or close the window.
The Bluetooth module mounted on the smart window
is connected to the controller through the USART bus torealize the short-distance wireless transceiver function.
The circuit connection diagram of the control system isshown in Fig.5.

Fig.7. Mechanical structure drawing

Fig.8 shows the three typical open states of the window,
namely 30°, 90° and 160°. Compared with the existing
windows, it can effectively solve the problem of difficult
cleaning from outside. Clean the outside of the window
is shown in Fig.9.

Fig.8. Mechanical structure drawing
Fig.5. Control system connection diagram

3. Mechanical Structure Design
Smart design of the mechanical structure is the key to thelarge-angle opening and closing of the smart window.
The structure is designed by adding the connecting rod(the motor drives the lead screw and makes the slider392

e

Fig.9. Clean the outside of the window

Fig.11. Process of App control

4. App Design

5. Algorithm Design

The team developed a mobile App for smart windows,
and the App interface is shown in Fig. 10. It is connectedto the control system through Bluetooth on the mobilephone, so that the user can control the window throughthe App. The main functions of the App include querying,displaying environmental data information, searching fornearby windows that can be paired, and controlling
windows. The process of App controlling windows isshown in Fig. 11.

5.1. Control algorithm design
In a general program cycle, the controller obtains
external information by polling and displays it on thedisplay. However, due to the presence of rain outside thewindow, indoor toxic gas, combustible gas and otheremergencies, higher priority countermeasures arerequired. Therefore, various sensors can send interrupt
signals to the controller through digital output pins,
which ensures priority deal with emergencies. The work
flow chart is shown in Fig.12, which can realize thefollowing functions: When the weather is bad and there is no one in thehouse, the windows will be closed automatically [3].
 Automatically open windows when the indoor air is
bad or contains harmful gas.
 Adjust the opening and closing angle by itself
according to the external wind direction.
•Fig.10. App interface

Fig.12. Work flow chart

393

e

5.2. The algorithm of detecting and protecting
childclosing multifunctional smart window will have broad
market prospects.

When a person is within the shooting range of the camera,
the camera will automatically find and detect the user's
face image.
A fixed-length feature vector is extracted from theconvolution feature map, and then the feature vector is
input into two fully connected layer networks. The age
and gender classification is training from face features.
In order to effectively prevent the occurrence of falling
down caused by children climbing windows, when thecamera detects that the child is close to the window, thewindow will automatically be closed. The result of childface detection and recognition is shown in Fig. 13.

References
1. Khandelwal, Hitesh, Albertus PHJ Schenning, andMichael G. Debije. "Infrared regulating smart window
based on organic materials." Advanced Energy Materials
7.14 (2017): 1602209.
2. Wang, Kai, et al. "Integrated energy storage andelectrochromic function in one flexible device: an energy
storage smart window." Energy & Environmental Science
5.8 (2012): 8384-8389.
3. Zhao Sun, Shuyi Wei, Xiuxia Zhang, Smart window
Control System Design Based on Single Chip
Microcomputer, 2020 IEEE International Conference onProgress in Informatics and Computing (PIC), 2020 Dec:
pp. 300-303.

Authors Introduction
Mr. Zilong Liu
He majored in Machinery and electronics
engineering in College of MechanicalEngineering, Tianjin University ofScience and Technology. He studied
mechanical control engineering andother related knowledge, and achieved
certain results in related professional
competitions.

Fig.13. Children's face recognition results

In the training stage, the system recognized 136 of the150 samples of children's photos, with an accuracy of90.6%. And in the testing stage, 150 of the 186 samplesof children's photos is recognized, with an accuracy of80.6%. Anyway, the recognition rate should be improved.

Mr. Haoyang Gui
He majored in Internet of Things
Engineering at the School of ArtificialIntelligence, Tianjin University of
Science and Technology. Mainlyengaged in related software testing anddevelopment.

6. Conclusion
Aiming at the shortcomings of traditional window in
intelligence and mechanical structure, this paper
proposes a new multi-scene smart window with wideangle opening and closing mechanical structure. The
structure is based on the original traditional casement
window design, and a linkage mechanism is added,
which enables the window to be turned over by nearly
160 degrees, effectively solving the problem of difficult
cleaning of the outer side of the window.
At the same time, the window can adjust the opening
and closing angle according to the wind direction toensure the maximum air intake. When the camera detectsthat there is a child's activity near the window, it willsound an alarm and automatically close the window toprevent accidents caused by children climbing thewindow. In summary, this wide-angle opening andMr. Luqi Shen
He majored in industrial design at theSchool of Mechanical Engineering,Tianjin University of Science andTechnology. He learned some computer
programming and related knowledge .

Mr. Longyu Gao
He majored in automation in Collegeof Electronic Information andAutomation, Tianjin University ofScience and Technology. He learned
SCM and data crawler and otherrelated knowledge, and achieved
certain results in relevant professional
competitions.

394

eDr. Fengzhi Dai
He received an M.E. and Doctor ofEngineering (PhD) from the Beijing
Institute of Technology, China in 1998
and Oita University, Japan in 2004
respectively. His main research interests
are artificial intelligence, patternrecognition and robotics.

395