ReSkin: versatile, replaceable, lasting tactile skins
Anonymous Author(s)
Afﬁliation email
Abstract: Soft sensors have continued growing interest because they enable both passive conformal contact and provide active contact data from the sensor properties.
However, the same properties of conformal contact result in faster deterioration of soft sensors and larger variations in their response characteristics overtime and across samples, inhibiting their ability to be long-lasting and replace- able.
ReSkin is a tactile soft sensor that leverages machine learning and magnetic sensing to offer a low-cost, diverse and compact solution for long-term use.
Mag- netic sensing separates the electronic circuitry from the passive interface, makingit easier to replace interfaces as they wear out while allowing for a wide variety ofform factors.
Machine learning allows us to learn sensor response models that are robust to variations across fabrication and time, and our self-supervised learning algorithm enables ﬁner performance enhancement with small, inexpensive datacollection procedures.
We believe that ReSkin opens the door to more versatile,scalable and inexpensive tactile sensation modules than existing alternatives.
Introduction
In recent years, AI has advanced signiﬁcantly from large-scale recognition to defeating human players in games.
But surprisingly current approaches still struggle at one task: dexterous manipulation.
While babies, from a young age can perform several challenging manipulation tasks, robots continue to struggle even with simple tasks.
Why is that?
We believe a signiﬁcant bottleneck in dexterous manipulation is the lack of practical solutions to tactile sensing.
From collecting large-scale richcontact data in the wild for learning models to building individual tactile sensors for robot ﬁngersand hand surfaces, current tactile sensing solutions lack on multiple dimensions and fail to scale up.
D) Contact localization on a new ReSkin sensor using our self-supervised adaptation procedure.
E) Contact localization on a ReSkin curated into a fabric sleeve as a 2in x 4in contiguous skin.
F) ReSkin sensor as a ﬁngertip sensor to record forces and contacts while folding a dumpling
In the context of robotics and AI, good tactile skins aim to provide: (a) conformal contact for stable grasping/manipulation; (b) accurate compression and shear force measurements; (c) high force( < 0.1 N) and temporal resolution ( > 100 Hz); and (d) large surface area coverage ( > 4 cm 2 ) withgood spatial resolution for sensing at all contact points.
For practical usage, good tactile sensorsshould also prioritize being (e) compact and versatile, (f) inexpensive, and (g) long-lasting.
Currentsolutions for tactile sensing have not been able to address all of these needs.
For example, visionbased tactile sensors are often bulky, expensive, and slow to respond (30-60 Hz) [1, 2, 3].
Resistiveand capacitive soft sensors require many connections that lead to early failure and integration challenges [4, 5].
Commercial sensing options, such as BioTac, are expensive ( > $1000) and available inlimited form factors.
Rigid tactile sensors, such as force-sensitive resistors, lack the soft, deformablesurface that is advantageous for object/environment interaction.
Above all, while there has been aplethora of work focused on ﬁngertip sensing, all-over sensing skins are much less studied.
There are two primary reasons why sensing skins have not been practical solutions for tactile sensing: (a) ﬁrst, there is a direct trade-off between the soft materials that enable conformal contact andtheir ability to perform well over time.
The exact properties that make soft sensors ideal for dexterous manipulation, make them degrade easily during robotic tasks; (b) but more importantly, evenskins with durable lasting materials require data-driven modeling which generally fails to generalizefrom one sensor to another.
Therefore, any replacement of skin requires relearning the model whichis impractical (hence, limiting experiments to one sensor only [6]).
We propose ReSkin – an inexpensive ( < $30), replaceable, compact, versatile and long-lasting tactile soft skin.
ReSkin is composed of soft magnetized skin and a ﬂexible magnetometer-based sensing mechanism.
Any deformation of the skin caused by normal/shear forces isread via distortions in magnetic ﬁelds.
These distortionscan be mapped back to estimate the contact points andforces on the original skin using a learned machine learning model.
The ReSkin design is compact (2-3mm thick)and long-lasting (our ML models perform accurate predictions even beyond 50K interactions.
ReSkin is versatile – the skin and the sensor mechanism can be used anywhere from robot hands to objects to gloves, arm sleevesand even dog paws.
ReSkin has high temporal (up to400Hz) and spatial resolution (1mm with 90% accuracy).
But what makes ReSkin the ideal tactile skin is the ability to replace an old skin with a new skin as ifyou are peeling off an old band-aid and putting a new one on.
Our learned models perform stronglyeven on new skins out-of-the-box but can be further adapted to high precision and resolution usinga self-supervised calibration technique.
We believe ReSkin has the ability to collect contact data inthe wild, provide robust tactile perception capabilities to our robot (See Figure 1) and effectivelymake tactile perception a ﬁrst-class citizen among its peers (pixels and sound).
Background
Soft sensing skins provide tactile or proprioceptive information without affecting the underlyingmechanics of the system.
Machine learning approaches have recently been shown to effectivelyparse soft sensor data for robotic grasping [7, 8, 9], proprioception [10], and object classiﬁcation[11], among many others [12, 13, 14].
Recent work in this area has relied on all types of sensing principles to collect and infer information in all types of form factors.
For example, resistivenetworks patterned onto a knit glove have been used to collect tactile grasping dataset and classifyobjects using neural networks [15].
Capacitive soft skins can be scaled up to larger areas by sampling at different interrogation frequencies to detect position via SVMs with just one interface [16].
Multi-modal sensing skins have also been shown to improve the ability to discern between 8 typesof applied deformation using neural networks [17] and static or dynamic inputs [18].
Data-drivenapproaches are becoming more common over traditional modeling due to the unpredictable materialproperties that introduce both non-stationary responses and non-linear behaviors over time, especially considering the dynamic interactions and unconstrained environments robots may encounter.
Unlike capacitive[16], resistive[15], and piezoelectric[19, 20] soft sensing, magnetic and optical softsensors do not require direct electrical connections between the circuitry and elastomer.
This is ideal to keep cost down, as the elastomeric interface degrades much faster than the accessory electronics.
It also simpliﬁes the replacement process by not requiring the user to disconnect and reconnect individual wires.
While optical sensors provide high spatial resolution data, they also require a clearline of sight between the camera and elastomer to observe deformations[1, 21, 22].
The camera’sdepth-of-focus puts a hard limit on the minimum distance to the elastomer surface leading to relatively bulky sensor modules.
In contrast, magnetic sensing beneﬁts greatly from minimizing thedistance between sensor and elastomer, allowing for a much more compact tactile sensor.
In addition, the small form factor of magnetometers, as compared to cameras, enables compatibility acrossmore diverse form factors for the tactile sensor.
We demonstrate these key beneﬁts by integrating themagnetic skin onto an arm sleeve, glove, dog shoe and a robot end-effector.
In each case, the elastomer is removable while the circuitry stays in place.
While there have been a number of attemptstowards developing a large area skin - piezoresistive fabrics [19, 20, 23], rigid taxels [24] as wellas optical sensors[25], they often lack shear sensing capabilities[24, 20, 23], conformal contact[24]and/or a scalable fabrication process[20].
ReSkin, however, is uniquely positioned to satisfy all ofthese requirements, and has the potential to be a scalable solution for all-over sensing skin.
[Table]
Cost
[Table]
Easily replaceable
[Table]
coverage
[Table]
Durable ( > 50k contacts)
[Table]
The underlying principle for magnetic sensing is that an applied deformation is measured as a changein magnetic ﬂux readings by nearby magnetometers.
However, we still need to learn or estimate themapping function that decodes change in magnetic ﬂux into contact force position and magnitude.
Several works on soft sensors have used neural networks for sensor characterizations [27, 28], butthese models are often trained on single sensor prototypes, and do not necessarily transfer to newcopies of the sensor.
Then, the end-user is required to collect and sometimes label their own data foreach sensor, which additionally requires access expensive, specialized equipment [28, 29].
In this paper, we systematically perform an experimental analysis of the proposed magnetic tactilesensor.
First, we extensively study the characterization of a single sensor over time.
We demonstratethat one can learn a quite accurate data-driven model to map magnetic ﬂux changes to contact forcelocation and magnitude.
We also demonstrate that the skins are long-lasting.
However, modelstrained on one sensor fail to generalize to other sensors or to different circuit board designs.
Ourﬁrst insight is to exploit multi-sensor learning: learn a more generalizable model by using data froma larger number of sensors.
While this leads to signiﬁcant improvement, it still falls well short oftraining and testing on the same sensor.
Inspired by recent work in self-supervised learning, we alsopresent a simple self-supervised calibration procedure which learns to adapt the multi-sensor modelto a particular sensor using just a couple of hundred pokes on the skin.
Our self-supervised approachis inspired from several works in slow feature learning [30, 31] and contrastive learning [32, 33, 34].
Design and Fabrication
The sensing principle for ReSkin relies on relative distance changes between embedded magneticmicroparticles in an elastomer matrix and a nearby magnetometer.
The use of magnetic microparticles allows the skin to be molded into many shapes and thicknesses.
When the magnetic compositeis deformed by applied force, the magnetometer reports changes in magnetic ﬂux in its X-,Y-, and
Z- coordinate system [35].
For an overall sensing area of 20mm x 20mm (Figure 3), we measuremagnetic ﬂux changes using 5 magnetometers.
Four magnetometers (MLX90393; Melexis) arespaced 7mm apart around a central magnetometer.
All 3D-printed molds, circuit board ﬁles, bill ofmaterials, and libraries used have been publicly released and opensourced at Github-Link .
Magnetic Elastomer Fabrication.
Our fabrication technique takes advantage of strong edge-effectsof permanent magnets.
We magnetize the composite over a 4x4 grid of much smaller cube magnets
(0.25in; AppliedMagnets).
Additionally, we apply a more uniform magnetic ﬁeld during curing byplacing the grid of magnets both above and below the sample (Figure 3B).
Magnetic microparticles(MQP-15-7; Magnequench) and 2-part polymer (Dragonskin 10 NV; Smooth-On) are mixed in a
2:1:1 ratio and poured into a 3D-printed mold
The mold is in a vacuum chamber for 3minutes air bubbles before placing magnets above and below the sample cures at room temperature (approx 24 ◦ C) for at least 3 hours before being removed
Circuit Board.
We use two variants of the circuit board for the experiments and demos presented inthis paper – a rigid board and a ﬂexible board illustrated in Figure 3C.
The circuit includes a 4-pinconnector (JST-SH; Molex) that transfers 20 values of magnetometer data (Temp, B X , B Y , B Z for5 chips) to a microcontroller (Trinket M0; Adafruit) at approximately 400 Hz.
The microcontrollerprocesses and transmits this data over USB to be read over serial from a central computer.
Toallow for easy replacement without damaging the board, we avoid the use of permanent adhesivefor attaching the sensing skin.
Skins are secured using four screws (M2-6; for the rigid boards), orusing four sewable snaps (size 4/0, Dritz; for the ﬂexible boards).
The bottoms of the rigid boardsare ﬁrst insulated by applying a thin layer of polymer (nitrocellulose).
Experimental Setup
Our goal is to perform experimental analysis of the proposed sensor, ReSkin, and the learnedmapping between the magnetic ﬁeld measurements from the sensor ( B ) and the planar location( x = ( x, y ) ) and magnitude of the applied force ( F ).
We want to analyze how ReSkin performson the different desired attributes – the accuracy of contact force estimation, spatial resolution ofcontact prediction, robustness to wear and tear and how model performance varies across differentskin instances.
For all the experiments, we use the data collection apparatus shown in Figure 3.
The circuit board along with the skin is ﬁxed to a 3D printed mount and streams 4 values, (Temp,
B X , B Y , B Z ), measurements for each of the ﬁve magnetometers at 400 Hz.
A hemispherical indenter fastened to the end of a Dobot Magician robot is used to apply forces at different locations on theskin.
The indenter also encases a 3-axis F/T sensor (Nano17; ATI) that streams force data at 1kHz.
We restrict ourselves to quasi-static measurements and analysis for the results presented in the following sections, unless stated otherwise.
The world coordinate frame used in these experiments isdeﬁned such that the xy-plane is aligned with the base of the robot as shown in Figure 3.
To collectdata, we ﬁrst specify a location for the robot to move to such that the indenter makes contact withthe skin.
We then record ﬁve measurements from the sensor board.
The speciﬁed xy-location is usedas the ground truth label for the location of the force.
The normal force measured by the Nano17sensor is used as the ground truth label for the magnitude of the applied force.
The indentations are made in a snake-like pattern along a 9x9 grid (excluding 4 points per corner;a total of 65 indentations) of size 16cm x 16cm shown in Figure 3.
During each iteration, we do asingle pass at each of 6 depths from 0.2mm to 1.2mm, for a total of 390 indentations.
We collectdata over multiple iterations.
Single Sensor Model – Decoding Magnetic Flux to Contact Characteristics
Our ﬁrst experiment is to evaluate the accuracy of the mapping from magnetic ﬂux B to contact force location and magnitude prediction.
Our ﬁve-layer multilayer perceptron (MLP) archi- tecture for the mapping function is: B(15) → MLP+ReLU(200) → MLP(200) → MLP(40) →
MLP+ReLU(200) → MLP+ReLU(200) → xyF(3) .
The change in magnetic ﬁeld resulting fromdeformation is used as the input to our model.
The third activation layer is the bottleneck featurelayer.
We use feat ( · ) to represent this 3-layer feature extraction network.
Our loss function is L2loss on ( x, y, F ) .
We deﬁne the accuracy of contact localization as the fraction of points whose xand y predictions are both within ± 1 mm of their true label.
We collect a total of 50K samples anduse a random 45K for training and 5K for test.
On this simple experiment, we get MSE for locationand force is 0 .
037 ± 0 .
014 mm 2 and 0 .
005 ± 0 .
002 N 2 respectively, with a contact localizationaccuracy of 99 .
58 ± 0 .
To demonstrate the shear sensing capability of the sensor, we perform another experiment.
Insteadof the quasi-static setup explained in Sec. 4, data is collected dynamically by indenting the skin to acertain depth and dragging it along the length of the sensor.
We move in straight lines along x and ydirections at intervals of 2 mm to cover the entire area of the sensor.
The network architecture is thesame as described in the previous paragraph, predicting ( x, y, F x , F y , F z ) instead of just ( x, y, F z ) .
On this experiment, we get MSE for F xy : 0 . 0011 ± 0 . N 2 , without compromising on predictionof normal force (MSE: 0 . 003 ± 0 . 001 N 2 ) or contact location (MSE: 0 .
085 ± 0 .
006 mm 2 ). Of course, the above setup is not realistic since training and test data is unlikely be sampled randomly from the same distribution.
Instead, we use a more practical setting, training on an initial Ksamples and testing on the samples that come after.
As the elastomer goes through multiple cyclesof compression and retraction, we see a drift in the properties of the elastomer.
This is evidenced bythe variation in the recorded magnetic ﬁeld shown in Figure 4a.
Therefore, it is critical to analyzehow the learned model behaves with respect to time.
[Figure]
Since we learn a sensor model that uses the change in magnetic ﬁeld as input, we would need torecord the magnetic ﬁeld before and after contact occurs.
Depending on the application scenarioin which the sensor is deployed, it may often be easier to collect calibrating no-load magnetic ﬁeldmeasurements at regular intervals.
Here, we design an experiment to quantify the effect of thefrequency of this measurement on learned sensor models.
We collect data from 50,000 indentationson a single sensor.
We train a neural network to predict contact location and force, using the ﬁrst5,000 indentations as the training set.
This model is then evaluated on test sets comprising 1000indentations after every 5000 indentations, to understand the degree and rate of domain shift.
Theresults of this experiment are shown in Figure 5.
Based on Figure 5a, we observe that prediction errors are higher and increase faster when we onlymake a single no-load measurement at the start.
This can be attributed to the drift in the elastomerresponse over time, which can be offset to a certain extent by more frequent no-load measurements.
Errors are also higher when no-load measurements are updated before every contact.
This could bea result of overﬁtting to the training data, since later experiments were seen to signiﬁcantly beneﬁtfrom updating zero measurements just before every contact.
Furthermore, Figure 5b indicates thatthe model, on average, overestimates the applied force.
This overestimation can be attributed to thesoftening of the elastomer as the number of interactions increases.
[Figure]
Adapting to New Sensors – MultiSensor Model + Self-supervised Learning
Our goal is to provide a simple, replaceable tactile sensor.
To achieve this, it is imperative thatany learned models acting on sensor measurements generalize to new sensor boards and skins.
Wedemonstrate the generalizability of our learned sensor response model in the following sections.
Ourmodels predict the contact normal force ( F ) and location( x = ( x, y ) ) using the change in magneticﬁeld measured by the magnetometers( B ).
However, the cheap and easy fabrication method for
ReSkin comes with a signiﬁcant degree of variability in the sensor response.
Figure 4b demonstratesthe variation in raw magnetic ﬁeld resulting from an identical indentation across different sensors.
So, how can we learn a model that generalizes to new sensors and even new circuit boards?
We use two techniques to help improve generalization for new skins and PCBs.
First, instead ofusing data from a single sensor, we use data from multiple sensors to train our mapping function.
This allows the model to see more diverse data in training and learn a more generalizable mappingfunction.
Additionally, we apply a feature regularization component (self-supervised loss) to ourloss function.
This component is a triplet loss computed in feature space as follows:
[Equation] [Equation]
) where B a , B p and B n are three datapoints with corresponding contact locations x a , x p and x n ,such that || x a − x p
|| <
|| x a − x n || , ie.
x a is closer to x p than x n .
Subscripts a , p and n refer toanchor, positive and negative samples respectively.
This loss encourages points that are closer onthe skin to be closer to each other in feature space.
It acts as a regularizer while also enabling us touse the self-supervised adaptation procedure described in the following paragraph.
Note that this self-supervised loss does not require ground-truth contact location or force readingsand therefore can be leveraged to further improve performance on new sensor boards and skins.
Anew user can collect their own unlabeled dataset, which can be indexed without requiring explicitlabels.
For instance, the user can use the tip of a pen to indent the sensor skin in a straight line andincrementally index these points as they move along the line.
Triplets of points can now be sampledalong this line, and the indices can be used to order the pairs within each triplet by distance.
Ourmulti-sensor learned model can then be ﬁne-tuned using these triplets to minimize the triplet loss.
At every training step, we sample a batch from the original training data, and an equal-sized batchof triplets (sampled with replacement) from the unlabeled dataset.
The former is used to minimizethe original loss function, while the latter is only used to minimize the triplet loss.
6.1 Results
We compare four model approaches.
Our baseline model is trained on one sensor and tested on adifferent sensor.
The other three are multi-sensor models – (a) trained without the triplet loss, (b)trained with the triplet loss, and (c) trained with the triplet loss along with self-supervised adaptation.
For the following comparisons, we collect data of 10,000 indentations each from 18 different skins.
We use a set of 6 sensor boards and 18 skins: each board appearing thrice in the dataset, eachtime with a different skin on top.
For the multi-sensor models, we perform 6-fold cross-validation,with the held out test set corresponding to three different skins on a particular sensor board eachtime.
For the self-supervised adaptation, we use different subsets of the unseen sensor data in theadaptation step, to qualitatively illustrate the effect of the quantity of data used for adaptation.
Forthe single-sensor model, we individually train on 3 different sensors and test on 9 other sensors.
[Table]
Our self-supervised adaptation signiﬁcantly improves prediction accuracy as well as MSE in xy, F
Based on Table 2, we see that the multi-sensor models do signiﬁcantly better than the single-sensormodel.
Training on a larger set of sensors allows the neural network model to generalize better.
Further, we see that adding the triplet loss slightly affects performance.
This small drop could be attributed to the additional constraint on the feature space resulting from the triplet loss.
However, theself-supervised adaptation gives us a sizeable improvement over the model predicting without adaptation.
Note that the adaptation procedure also results in improved force prediction performance.
[Figure]
To further investigate the effectiveness of our self-supervised adaptation procedure, we look at howincreasing the quantity of data used for adaptation correlates with model performance.
As can beseen from Figure 6a, there is a signiﬁcant improvement in performance between no self-supervisionand using 780 points for self-supervision.
However, performance seems to plateau as we increasethe data beyond this point, which indicates that our model adapts quickly with small amounts ofunlabeled data.
Finally, Figure 6b shows the effect of the number of sensors in the training data onmodel performance.
We use data from three unseen sensors as the test set.
As expected, performanceimproves as the amount of training data increases.
Further, we observe that the self-supervisedadaptation, while always doing better than the other models, offers a signiﬁcant improvement inperformance when using fewer sensors in training.
Generalizing models with manual indentations Since the analysis of our self-supervised adaptation technique is performed in a very controlled scenario we perform another, less controlled,experiment which is likely to be closer to the end user’s application setting.
We manually indent thesensor 325 times and run self-supervised adaptation using the triplet loss.
To test the effectivenessof the adaptation, we collect test data using the original experimental setup to evaluate the adaptedmodels.
After adaptation we see improvements in accuracy, from 79.86% to 84.84%, MSE xy from0.676 mm 2 to 0.489 mm 2 and MSE F from 0.268 N 2 to 0.192 N 2 , clearly demonstrating the effectiveness of our proposed method even outside a controlled environment.
Generalizing models to a different sensor board type In order to demonstrate the effectiveness ofour self-supervised adaptation scheme, we now adapt a model learned using rigid sensor boards ona ﬂexible sensor board.
Note that this is a signiﬁcantly harder adaptation problem since the distancebetween skin and circuit board is 80% lesser in case of the ﬂexible board.
We see an average contactlocalization accuracy of 75% with MSE error on location and force as 0.72 mm 2 and 0.54 N 2respectively.
The relatively larger force errors can be attributed to an overestimation bias resultingfrom signals appearing stronger due to the reduced thickness of the ﬂexible board.
ReSkin in Action
We have demonstrated that ReSkin is a sensor capable of high resolution contact localization andforce prediction.
The performance does not deteriorate signiﬁcantly with wear and tear.
But most importantly, learned models generalize to new skins with a simple self-supervised adaptationscheme.
We now highlight how ReSkin’s compact design allows it to be used in diverse applicationswith different form factors.
In this section, we use ReSkin in different settings to emphasize itseffectiveness in a range of application scenarios.
These demos are for proof-of-concept only.
Forthe following, we fabricated both ﬂexible and rigid circuit boards of the exact same design: ﬂexibleboards are more comfortable and thinner, while rigid boards can withstand larger applied forces.
Force Sensitivity: Water in shot glass To visually illustrate the force sensitivity of our sensor, wedo a pouring demo where we place a shot glass on top of a ReSkin sensor.
As the water ﬁlls up,we see monotonically increasing sensor measurements indicating the sensor’s ability to distinguishforces as small as the weight of less than 20 mL ( < 0 .
2 N) of water (Supplementary Video S1).
Robot Gripper.
Next, we show that ReSkin can be a useful tactile sensor for robotics applicationssuch as grasping delicate objects such as blueberries and grapes (See Figure 1B).
Two ReSkin sensors with ﬂexible circuit boards are placed on either side of a parallel jaw gripper (Robotiq Hand-E
Gripper on Sawyer Arm).
Grasping soft and squishy objects requires force feedback – applyingtoo much force will squish blueberry and the grape.
We show that the built-in force sensing (30Nminimum) is insufﬁcient for the task, whereas ReSkin does an excellent job of using force feedbackto control grasping.
Furthermore, we demonstrate that the grasping continues to work well, with notuning required, when we replace one of the skins with a new skin (Supplementary Videos S2, S3).
Location Sensitivity: Poking To visually illustrate the location sensitivity of our sensor, we do asimple poking task on a new sensor and show the resolution of real-time contact location estimation(See Figure 1D and Supplementary Video S4).
Dog Shoe.
Our next application demonstrates how ReSkin’s compact design makes it non-obtrusiveand useful for measuring tactile forces in the wild.
One magnetic skin and ﬂexible circuit board isplaced inside the sole of a dog shoe (size: 1.75in).
A 1/16in layer of urethane foam is added on topfor comfort.
The data is collected on-board and logged to an SD card at 250 Hz.
The sensorizedshoe is worn on the front right leg of a small dog (17 lb).
The sensor tracks magnitude and directionof applied force while resting, walking, and running (See Figure 1C and Supplementary Video S5).
Glove.
We also demonstrate how ReSkin can be used to measure forces during natural humanobject interactions.
A magnetic skin and rigid circuit board is placed on the right-hand index ﬁnger.
A nitrile glove was placed over to hold the board in place, and keep the objects clean.
The data wascollected on-board and logged to an SD card at 250 Hz.
We demonstrate sensor output during thesealing of dough (See Figure 1F and Supplementary Video S6).
Arm Sleeve.
Finally, we want to demonstrate that ReSkin is a surface sensor and can be used forwide coverage tactile sensing.
Speciﬁcally, we connected 8 ﬂexible boards in two rows of four andfabricated a larger, continuous skin (2in x 4in).
All 8 boards are connected to a microcontroller (QT
Py; Adafruit) that samples all 40 magnetometers at 133 Hz.
We show how ReSkin can be scaled upfor contact localization across larger surface areas (See Figure 1E Supplementary Video S7).
Conclusion
We present ReSkin: a low-cost, compact and long-lasting surface tactile sensor with high localization accuracy and force sensitivity.
ReSkin combines soft sensing with recent advances in machinelearning to develop models that generalize across time and individual skins.
More speciﬁcally, weuse multi-sensor learning combined with self-supervised triplet loss for slow feature changes.
Wealso present an SSL adaptation procedure to further reﬁne the models for new skins.
Therefore,
ReSkin sensors have easily replaceable skin (as easy as peeling and putting new band-aid) that canbe used right away.
We demonstrate that the compact form of ReSkin makes it an ideal candidate fordiverse applications: from grasping delicate objects to measuring forces exerted by dog feet; frombuilding wide-coverage contiguous skin to measuring contact forces in the wild.
Limitations and Future Work: While we have shown promising results on contact localization andforce prediction, there is enormous untapped potential for ReSkin at this stage.
Experiments in thispaper are based on single point contact, and we aim to further investigate multi-point contact.
Aninteresting direction for future work is to analyze the effect of external magnetic ﬁelds and metallicobjects on ReSkin’s sensing ability.
ReSkin can stream data up to 400 Hz and we aim to leveragethis capability to train better models using dynamic time-series data.
We believe that ReSkin (andits desirable properties) will make tactile perception far more accessible for real-world use.