365

Sensors and Materials, Vol. 35, No. 2 (2023) 365–376
MYU Tokyo
S & M 3173

Multilayer-perceptron-based Slip Detection Algorithm
Using Normal Force Sensor Arrays
Hamid Bamshad, Sangwon Lee, Kyungchan Son,
Hyemi Jeong, Geonwoo Kwon, and Hyunseok Yang*
Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea(Received October 25, 2022; accepted December 5, 2022; online published December 13, 2022)

Keywords: tactile sensor, robot grasping, slip detection, artificial intelligence

Slip detection is an essential technology for robotic grippers to autonomously grasp unknown
objects and can be achieved using a tactile sensor. In this paper, we propose a high-performance
multilayer-perceptron-based slip detection algorithm that utilizes only normal force dataobtained by frequency selective surface(FSS) sensor arrays. This is achieved in three stages inthis study. First, slip and no-slip training data are aggregated such that the data closely resemble
those of the real world. Second, the most suitable means of preprocessing the raw sensor output
is identified. Third, the classification method with the highest performance is chosen on thebasis of a performance comparison among various classification techniques. The online
performance of the algorithm is evaluated by conducting two tasks: a simple pick and place task
and a task of maintaining a stable grasp of an object whose weight is changing.

1.

Introduction

With the progress in technology and the increasing demand for the facilitation of human
lifestyles, the paradigm in the research field of robotics has shifted from developing automatedindustrial robots to advanced human-assistant robots. This has made autonomous grasping by
robotic grippers an increasingly important area of research. Robots that are capable of grasping
and manipulating unknown objects autonomously could provide humans with valuableassistance. Autonomous grasping technology has been examined from a variety of perspectives.
For example, compliant grip mechanisms have been developed in which a robotic gripper canpassively adapt to the shape of the target object.(1–3) Despite the simple structure of thesemechanisms and their capability of holding objects with a good grip, they cannot grasp thin or
sharp objects. There have also been studies on reconfigurable grippers, such as vacuum suction
grippers, which can be adapted to the shape of different objects to exert a suction force on them.
(4) In addition, the concept of adjustable grippers has been advanced, where the gripper is able tochange both its shape and orientation in accordance with the object.(5) Reconfigurable grippers
and adjustable grippers are generally beneficial in grasping objects of various shapes, but they*Corresponding author: e-mail: hsyang@yonsei.ac.kr

https://doi.org/10.18494/SAM4184

ISSN 0914-4935 © MYU K.K.
https://myukk.org/

366

Sensors and Materials, Vol. 35, No. 2 (2023)

do not possess powerful grasping capabilities. These mechanisms are unlikely to provide an
ideal solution to the problem of autonomous grasp techniques, since such techniques require thecapability of not only grasping objects of many shapes, but also exerting a considerable grasp
force. It is possible to achieve both characteristics using rigid robotic grippers, such as robotic
hands or parallel grippers, equipped with a high-performance slip detection algorithm.(6,7)
This type of mechanism can grasp a variety of objects while exerting sufficient grasp forces
to ensure a solid grasp. Accordingly, most studies carried out on autonomous grasping have been
concerned with developing algorithms for detecting slip. The research on slip detection
algorithms for robotic grippers can be divided into four major phases: the development ofsensors, the method of data collection, the method of data processing, and the classification
method. Previous studies have generally focused on only some of these topics. The mostcommonly proposed or investigated types of sensors are electrode-based sensors such as
BioTac,(8–11) center-of-pressure sensors,(12) and force-sensitive-resistor sensors.(13) Other
commonly investigated sensors include the photodiode-based OptoForce sensor,(14) laser
Doppler velocimeter sensors,(15,16) the vision-data-based GelSight sensor,(17) and TacTip
sensors.(18) The output of these sensors can be force, torque, vibration, pressure, or point ofcontact. As part of the data collection process, in addition to acquiring sensor data for the control
system, various approaches have also been applied to collect sensor data for training purposes.
When a robotic gripper grasping a target object reduces its grasping force, slip is induced.(18) As
slip occurs, by using a fixed camera to detect an ArUco marker on the object, it is possible todetect both the amount and the time of slip. Other methods of collecting sensor data involve
dragging a sensor along the surface of the target object(11,19) or simply pulling it down.(9)The acquisition of raw data from the sensor is followed by a preprocessing phase in which
input data for the classification phase is produced. This preprocessing phase has been a focus ofstudy for researchers and various methods have been implemented. The raw data from the sensor
was converted into data that represents the change in force per unit of time by Su et al.(8)Johansson and Flanagan analyzed frequency-domain data and concluded that vibration
amplitudes at certain frequencies can respond to contact, transient motion, and the release of an
object.(20) Similarly, other studies utilized this method in which the sensor output is first
converted into frequency-domain data then divided into predetermined frequency ranges.(8,11,21)
James and Lepora suggested using the velocities of the moving contact points for classification.
(18) Finally, Stachowsky et al. utilized the sensor data covariance to detect slip.(13) Despite theavailability of these methods, some researchers did not preprocess the raw sensor data and used
it directly for the next phase.(9,11,12,22,23)
As mentioned above, the final phase in slip detection is classification, which has been
investigated by many researchers, and a variety of classification methods have been developed to
improve the detection of slip. Here, the support vector machine (SVM) and logistic regression(LR) are the most common methods of classification,(18) while the use of empirical data as a
threshold for slip detection is one of the naive approaches.(8,13,18,21)
Furthermore, owing to advances in computational power, many studies implemented deepneural networks. A long-short-term memory (LSTM) neural network was used by Van Wyk andFalco to develop robust slip detectors.(11) Later, Deng et al. implemented a multilayer perceptron

Sensors and Materials, Vol. 35, No. 2 (2023)

367(MLP) network on the hidden layers of an LSTM-based encoder–decoder.(9) Gaussian mixturesand convolutional neural networks have also been used in several studies.(9,23)
Although previous studies have achieved significant improvements in slip detection
performance, some limitations must be considered. The data collection methods used in thesestudies often obtain data that is unrepresentative of real-life slip and no-slip conditions.
Moreover, the preprocessing and classification methods used have not often been compared withother possible approaches.
In contrast to previous studies, in this study, we aim to construct a high-performance slip
detection algorithm based entirely on normal force data. We first use sensor arrays comprisingfrequency selective surface (FSS) normal force sensors, which are piezoresistive sensors
manufactured by Honeywell Inc., to collect slip and no-slip data. The full hardware setup is
shown in Fig. 1. To collect the slip and no-slip data using these sensors, we use mini steel balls togradually increase the object’s weight and, consequently, induce slip. As a result of thismethodology, we aggregate both linear and rotational slip data that are also distinguishable andcontinuous, which are characteristics of real-world slip data. Regarding the type of preprocesseddata and classification method, unlike in previous studies, we thoroughly examine andinvestigate the available methods and select the one with the highest performance. After
constructing the detection algorithm, we demonstrate its performance in executing two tasks: a
simple pick and place task, and a task of maintaining a stable grasp of an object whose weight ischanging.

2.

Hardware Setup

2.1

Sensor array

We use two sensor arrays, each containing five FSS sensors, which are mounted on acustomized circuit board, as shown in Fig. 1. In addition, a customized silicon cover is attached
to encapsulate the entire array. A maximum operating force of 14.7 N can be applied to each FSS

Fig. 1. (Color online) Developed hardware setup for experiment, including 3-DOF arm with gripper and sensor
arrays.

368

Sensors and Materials, Vol. 35, No. 2 (2023)

sensor, and the sampling rate for the sensor array is 80 Hz. An HX711 analog-to-digital converter(JK Electronics Co.) and an Arduino Mega-2560 microcontroller board are used to transmit thedata from the sensor array to the PC.
2.2

Gripper and manipulator

As shown in Fig. 1, the Robotis RM-X52-TNM gripper has a simple parallel gripper
mechanism that is operated by two links and a single crank. A Robotis Dynamixel actuator(XM430-W350-T) is implemented on both the gripper and manipulator. Without a load, themotor rotates at 45 rpm and has a stall torque of 4.2 Nm. The maximum distance between thegripper jaws is 75 mm without a sensor and 60 mm with a sensor. Using the Kinetic Robot
Operating System, the gripper and manipulator are integrated with the sensor arrays on the PC.

3.

Collection of Training Data

To select the most appropriate data processing and ultimately train the classification
structure, we require raw sensor outputs that have been labeled as slip or no-slip. To achieve this,we examined two data collection methods used in previous research. In one method, the sensor
is dragged over the surface of the target object,(11,19) and in the other method, slip is produced by
reducing the gripping force of the gripper while holding an object.(9) Following the examinationof the results, we concluded that neither method is suitable for the collection of training data, andwe proposed an alternative novel approach.
3.1

Methods from previous research

Figure 2(a) shows the output of the 10-channel sensor arrays when the sensor drag method isapplied. As shown in this figure, when the gripper is dragged, there is a clear distinction between
slip and no-slip data. The advantage of this method is that it is very simple to implement and canbe used to gather a large amount of data. However, this method does not allow the aggregatedanalysis of data for mild slips. Moreover, although slip is usually composed of both linear androtational slip, only linear slip data is collected by this method. Figure 2(b) shows the normalized
output signal when the force reduction method is applied.

Fig. 2.

(Color online) FSS sensor array signal output for (a) sensor drag method and (b) force reduction method.

Sensors and Materials, Vol. 35, No. 2 (2023)

369

With the reduction of the grasp force, natural slip is induced, leading to both linear androtational slip. However, as can be seen in the figure, the data show very discrete changes and,more importantly, there is no clear distinction between the times when slip does and does notoccur. In summary, neither method is suitable for the collection of training data.
3.2

Proposed method

We propose the following alternative method of collecting data. The gripper grasps a target
object that has been marked with an ArUco marker. Four objects in the form of open boxes madeof different materials on each side are provided (40 mm by 60 mm by 60 mm). Miniature steel
balls are poured into the object to gradually increase its weight, causing slip, while a camera
detects the pitch and height of the object, as shown in Fig. 3. Using the moving-average
information of these two types of data, the sensor signal is automatically classified as slip or noslip. As can be seen in Fig. 4, the information from the marker identifies the time of slip and noslip; then the sensor signal outputs are labeled accordingly. The proposed method has theadvantages of being able to differentiate between slip and no-slip, gather mild-slip data, avoid
discrete data changes, and include both linear and rotational slip data.

4.

Means of Processing Raw Sensor Output

In this section, four candidate methods are compared to determine the most suitable method
for processing the raw sensor output. The selected method for processing data is then enhanced
through the adjustment of parameters.
4.1

Selection of suitable processed data

Following the collection of slip and no-slip data, we examine which type of data is mostsuitable for input to the classification phase. In general, there are four potential options: the sumof the FSS signals, the difference between the sums per unit time, the covariance between theFig. 3.

(Color online) Schematic of the proposed data collection method.

370

Sensors and Materials, Vol. 35, No. 2 (2023)

Fig. 4. (Color online) ArUco marker data and FSS signal data obtained by the proposed method. (a) Data acquiredwith ArUco marker (pitch and height). (b) 10-channel output signal from FSS arrays.

channels of the sensor array, and the Fourier transformed sum of the signals. The candidates arecompared on the basis of the obtained data and the results of the Kolmogorov–Smirnov (KS)test. The results of each processed data type for a sample data set are shown in Fig. 5. As can beseen from the figure, the force sum is not applicable since the baseline itself changes constantly.
Next, the force derivative shows inadequate distinction between mild-slip and no-slip data. Also,
the p-value for the force derivative is relatively high. Covariance and frequency are advantageous
because they can distinguish between mild-slip and no-slip regions, and their p-values are also
low, although even these data contain some false negatives. To reduce the likelihood of suchoccurrences, we selected a sequence that includes both covariance and frequency information.
4.2

Parameter adjustment: covarianceFirst, the algorithm for computing the covariance is arranged such that the difference between
consecutive FSS signal outputs is calculated first:.
dˆ=
i (tk ) Fi (tk ) − Fi (tk −1 )Then, normalization is conducted:(1)Sensors and Materials, Vol. 35, No. 2 (2023)

371

Fig. 5. (Color online) Four candidates for suitable processed data. (a) Force sum (p-value: N/A). (b) Forcederivative (p-value: 8.35 × 10 −29). (c) Covariance (p-value: 1.37 × 10 −117). (d) Frequency (p-value: 3.89 × 10 −111).

di (tk ) =dˆi (tk )
.
d max

(2)


Given that the jth entry of D is di (tk−j), X is defined as X (tk ) =  D1 ; D2 ;...; Dn .

(3)

Covariance is finally calculated as
Cov(tk ) = X (tk ) X (tk )T .

(4)

In this algorithm, the window size n determines the distinction between the patterns of slip
data and no-slip data; the larger the value, the larger the disparity between the patterns of thetwo groups. As shown by the results in Fig. 6(a), with increasing n, the likelihood of false
negatives is reduced. By experimentation, we found that the optimal value of n was 10.
4.3

Parameter adjustment: frequency

The frequency is computed using the fast Fourier transform after the 10-channel data of the
FSS are added together. In the case of the frequency, the number of data time steps included in
the transform determines the number of sample frequencies. Similar to covariance, the larger the372

Sensors and Materials, Vol. 35, No. 2 (2023)

Fig. 6. (Color online) (a) Behavior of covariance data for window sizes of 1, 5, and 10. (b) Behavior of frequency
data for window sizes of 10, 20, and 40.

window size n, the larger the distinction between the slip and no-slip patterns, as can be seen in
Fig. 6(b). However, for the frequency, a larger n means a longer decay time (the time for the slip
pattern to fade away after the slip is over). The decay time is also a significant factor since it may
induce false positives once the slip is no longer occurring. To resolve this issue, the p-values anddecay time were computed and examined for different values of n. We set a window size of 40
based on the fact that it is more important for the algorithm to be able to classify mild slips thanto have short decay times.

5.

Classification Method

After choosing the most suitable input data for the classification method, we compared theperformance of five potential classification methods. Next, the algorithm was evaluated offline
and further improvements were made to enhance its performance.
5.1

Selection of appropriate classification method

The potential classification methods were an SVM, LR, an LSTM network autoencoder
combined with an MLP in the hidden layers, an LSTM network with an MLP, and lastly, aSensors and Materials, Vol. 35, No. 2 (2023)

373

simple MLP. The test was conducted using the training data that had been collected with the fourmaterials and the results are shown in Fig. 7. According to the results, the MLP network had thehighest average accuracy of 98% and was thus selected as the classification method.
5.2

Offline performance evaluation

Having chosen the classification method, we conducted a simple offline performance
evaluation. A validation data set was given to the trained network and the results were observed.
Out of 2800 time steps of data, 162 corresponded to false negatives and 47 corresponded to false
positives. The false negatives are due to the insensitivity of the network, and the false positives
occur because of the long decay time. To improve these results, the algorithm was modified sothat there would be a queue of lookback sequences. The idea was to incorporate past data into
the algorithm and thus improve the classification performance. After the modification, for thesame validation data, the numbers of false negatives and false positives decreased to 103 and 36,
respectively, while the other performance evaluation parameters remained almost the same. In
short, the network was further improved by the modification.
5.3

Proposed network

The proposed network, including the lookback queue, is shown in Fig. 8. As can be seen, thealgorithm computes the covariance and frequency separately, then combines them into a single

Fig. 7.

(Color online) Accuracy of the potential classification methods for training the data set.

Fig. 8.

(Color online) Schematic of the proposed slip classification network.

374

Sensors and Materials, Vol. 35, No. 2 (2023)

Fig. 9.

(Color online) Number of successes of the two tasks for each material.

sequence. After that, the sequence is made into a lookback queue and finally applied to the MLP
network. The average training accuracy of this network for the four chosen materials was 98.4%
and the online classification rate was 75 Hz.

6.

Online Performance Evaluation

An online performance evaluation was conducted with the proposed network. Two tasks—apick and place task and a task of maintaining a stable grasp of an object whose weight waschanging—were conducted 10 times each for all four materials. The former task involved
picking up the target object from the ground and placing it back down, and the latter task
involved maintaining the grasp of the target object as mini steel balls were poured into it. As
shown in Fig. 9, performance was excellent for plastic and paper but lower for aluminum.

7.

Conclusion

In this study, we investigated and compared various combinations of methods that can beused to implement slip detection for robot grippers equipped with tactile sensors. By using our
newly proposed collection method, selecting the most appropriate processed data, and choosing
the classification method with the highest performance, we developed a highly accurate slip
detection algorithm based only on normal force data. The results of both offline and online
evaluations indicate that the selected methods are sufficiently effective and have highperformance. Even though the setup in this study cannot be precisely replicated for other objects
and environments, or even in industry, the methodology outlined in this study can serve as areference for selecting methods to detect slip. Our future work will be focused on two areas:
improving the detection accuracy of the algorithm by collecting more training data, andconducting online evaluations that take into consideration the deformation of the target object.

Sensors and Materials, Vol. 35, No. 2 (2023)

375

Acknowledgments
This work was supported in part by the Yonsei University Research Fund (Post Doc.
Researcher Supporting Program) of 2021 (Project No. 2021-12-0151).

References
1 R. Deimel and O. Brock: Proc. 2013 IEEE Int. Conf. Robotics and Automation (IEEE, 2013) 2047–2053. https://doi.org/10.1109/ICRA.2013.6630851
2 Y. She, C. Li, J. Cleary, and H. J. Su: J. Mech. Rob. 7 (2015) 021007-1. https://doi.org/10.1115/1.4029497
3 N. Feng, Q. Shi, H. Wang, J. Gong, C. Liu, and Z. Lu: Int. J. Adv. Manuf. Technol. 97 (2018) 319. https://doi.
org/10.1007/s00170-018-1949-2
4 Z. Zhakypov, F. Heremans, A. Billard, and J. Paik: IEEE Rob. Autom. Lett. 3 (2018) 2894. https://doi.
org/10.1109/LRA.2018.2847403
5 M. Riedel, B. Corves, M. Nefzi, and M. Hüsing: Proc. 2nd Int. Workshop on Fundamental Issues and Future
Research Directions for Parallel Mechanisms and Manipulators: LIRMM (2008) RWTH-CONV-005939.
https://publications.rwth-aachen.de/record/48835/files/4023.pdf
6 S. Y. Jung, S. K. Kang, M. J. Lee, and I. Moon: Proc. 2007 Int. Conf. Control, Automation and Systems. (IEEE,
2007) 83–86. https://doi.org/10.1109/ICCAS.2007.4406884
7 X. Zhang and Q. Xu: Precis. Eng. 56 (2019) 53. https://doi.org/10.1016/j.precisioneng.2018.09.004
8 Z. Su, K. Hausman, Y. Chebotar, A. Molchanov, G. E. Loeb, G. S. Sukhatme, and S. Schaal: Proc. 2015 IEEERAS 15th Int. Conf. Humanoid Robots (IEEE, 2015) 297–303. https://doi.org/10.3390/s20041050
9 Z. Deng, Y. Jonetzko, L. Zhang, and J. Zhang: Sensors 20 (2020) 1050. https://doi.org/10.3390/s20041050
10 Y. Gao, L. A. Hendricks, K. J. Kuchenbecker, and T. Darrell: Proc. 2016 IEEE Int. Conf. Robotics andAutomation (IEEE, 2016) 536–543. https://doi.org/10.1109/ICRA.2016.7487176
11 K. Van Wyk and J. Falco: Proc. 2018 IEEE Int. Conf. Robotics and Automation (2018) 2744–2751. https://doi.
org/10.1109/ICRA.2018.8461117
12 Y. Suzuki: IEEE Rob. Autom. Lett. 2 (2017) 2180. https://doi.org/10.1109/LRA.2017.2723469.
13 M. Stachowsky, T. Hummel, M. Moussa, and H. A. Abdullah: IEEE/ASME Trans. Mechatron. 21 (2016) 2214.
https://doi.org/10.1109/TMECH.2016.2551557
14 M. Kaboli, K. Yao, and G. Cheng: Proc. 2016 IEEE-RAS 16th Int. Conf. Humanoid Robots (2016) 752–757.
https://doi.org/10.1109/HUMANOIDS.2016.7803358
15 B. Choi, H. R. Choi, and S. Kang: Proc. 2005 IEEE/RSJ Int. Conf. Intelligent Robots and Systems (2005)
2638–2643. https://doi.org/10.1109/IROS.2005.1545267
16 N. Morita, H. Nogami, E. Higurashi, and R. Sawada: Sensors 18 (2018) 326. https://doi.org/10.3390/s18020326
17 S. Dong, W. Yuan, and E. H. Adelson: Proc. 2017 IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS)(2017) 137–144. https://doi.org/10.1109/IROS.2017.8202149
18 J. W. James and N. F. Lepora: IEEE Trans. Rob. 37 (2020) 506. https://doi.org/10.1109/TRO.2020.3031245
19 M. Schöpfer, C. Schürmann, M. Pardowitz, and H. Ritter: Proc. ISR 41st Int. Symp. Robotics and ROBOTIK
6th German Conf. Robotics (2010) 1–7. https://ieeexplore.ieee.org/abstract/document/5756770
20 R. S. Johansson and J. R. Flanagan: Nat. Rev. Neurosci. 10 (2009) 345. https://doi.org/10.1038/nrn2621
21 J. M. Romano, K. Hsiao, G. Niemeyer, S. Chitta, and K. J. Kuchenbecker: IEEE Trans. Rob. 27 (2011) 1067.
https://doi.org/10.1109/TRO.2011.2162271
22 M. A. Abd, I. J. Gonzalez, T. C. Colestock, B. A. Kent, and E. D. Engeberg: Proc. IEEE/ASME Int. Conf.
Advanced Intelligent Mechatronics (IEEE, 2018) 21–27. https://doi.org/10.1109/AIM.2018.8452704
23 Y. Zhang, Z. Kan, Y. A. Tse, Y. Yang, and M. Y. Wang: arXiv preprint arXiv:1810.02653 (2018). https://doi.
org/10.48550/arXiv.1810.02653

376

Sensors and Materials, Vol. 35, No. 2 (2023)

About the Authors
Hamid Bamshad received his B.Sc. degree in mechanical engineering from Ferdowsi Universityof Mashhad, Iran, in 2011 and his master’s degree in mechanical engineering from Yonsei
University in 2015. Currently, he is conducting doctoral research in mechanical engineering at
Yonsei University. His main research interests are in the fields of automation, robotics,
dynamics, and control as well as artificial intelligence. (hamidbamshad@yonsei.ac.kr)
Sangwon Lee received his B.S. and M.S. degrees in mechanical engineering from Yonsei
University in 2020 and 2022, respectively. He is currently working at Samsung Electronics Co.,
Ltd., as a factory automation engineer. His research interests include robot automation, learningbased online classification, and readjustment-related control. (sangwon2659@yonsei.ac.kr)
Kyungchan Son received his B.S. degree in mechanical engineering from Yonsei University,
Seoul, Korea, in 2014 and his Ph.D. degree in mechanical engineering from Yonsei University in
2021. His research interests include robotics, control theory, and machine learning for robotics
applications. (skc0313@yonsei.ac.kr)
Hyemi Jeong received her B.S. degree in mechanical engineering in 2016 from Yonsei
University, Seoul, South Korea, and is currently working toward a Ph.D. degree in the field of
robotics and control systems at Yonsei University. Their research interests include robotics andthe optimal planning/control algorithms of manipulators. (stree104@yonsei.ac.kr)
Geonwoo Kwon received his B.S. degree in mechanical engineering from Yonsei University,
South Korea, in 2018, where he is currently pursuing a Ph.D. degree in robotics and control
engineering. His main research interests include adaptive control, trajectory planning, sensor
fusion, and visual inertia systems. (kay1216@yonsei.ac.kr)
Hyunseok Yang received his B.S. degree from the Department of Mechanical Engineering,
Yonsei University, Seoul in 1984 and his M.S. and Ph.D. degrees from the Department of
Mechanical Engineering, Massachusetts Institute of Technology (MIT) in 1988 and 1993,
respectively. He is currently a professor in the Department of Mechanical Engineering, Yonsei
University and coordinates research on motion control and robotics. (hsyang@yonsei.ac.kr)