Original: Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
Revised: Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear

Original: Abstract: Robotic touch, particularly when using soft optical tactile sensors,suffers from distortion caused by motion-dependent shear.
Revised: Abstract: Robotic touch, particularly when using soft optical tactile sensors,suffers from distortion caused by motion-dependent shear.

Original: The manner in whichthe sensor contacts a stimulus is entangled with the tactile information about thegeometry of the stimulus.
Revised: The manner in whichthe sensor contacts a stimulus is entangled with the tactile information about thegeometry of the stimulus.

Original: In this work, we propose a supervised convolutionaldeep neural network model that learns to disentangle, in the latent space, thecomponents of sensor deformations caused by contact geometry from those dueto sliding-induced shear.
Revised: In this work, we propose a supervised convolutionaldeep neural network model that learns to disentangle, in the latent space, thecomponents of sensor deformations caused by contact geometry from those dueto sliding-induced shear.

Original: The approach is validated by reconstructing unshearedtactile images from sheared images and showing they match unsheared tactileimages collected with no sliding motion.
Revised: The approach is validated by reconstructing unshearedtactile images from sheared images and showing they match unsheared tactileimages collected with no sliding motion.

Original: In addition, the unsheared tactile imagesgive a faithful reconstruction of the contact geometry that is not possible from the sheared data, and robust estimation of the contact pose that can be used forservo control sliding around various 2D shapes.
Revised: In addition, the unsheared tactile imagesgive a faithful reconstruction of the contact geometry that is not possible from the sheared data, and robust estimation of the contact pose that can be used forservo control sliding around various 2D shapes.

Original: Finally, the contact geometryreconstruction in conjunction with servo control sliding were used for faithful full object reconstruction of various 2D shapes.
Revised: Finally, the contact geometryreconstruction in conjunction with servo control sliding were used for faithful full object reconstruction of various 2D shapes.

Original: The methods have broad applicabilityto deep learning models for robots with a shear-sensitive sense of touch.
Revised: The methods have broad applicabilityto deep learning models for robots with a shear-sensitive sense of touch.

Original: [Figure]
Revised: [Figure]

Original:  Introduction
Revised:  Introduction

Original: A fundamental aspect of tactile sensing is that it measures the environment via physical contact.
Revised: A fundamental aspect of tactile sensing is that it measures the environment via physical contact.

Original: Therefore, the manner in which the sensor contacts a stimulus is entangled with the tactile information about that stimulus.
Revised: Therefore, the manner in which the sensor contacts a stimulus is entangled with the tactile information about that stimulus.

Original: For example, when sliding or rubbing ﬁngers across tactile stimuli, soft tactile sensors are inevitably distorted by motion-dependent shear, making the sensor response history- dependent and generalization hard in tactile tasks.
Revised: For example, when sliding or rubbing ﬁngers across tactile stimuli, soft tactile sensors are inevitably distorted by motion-dependent shear, making the sensor response history- dependent and generalization hard in tactile tasks.

Original: This issue is prevalent in camera-based optical tactile sensors that use the lateral motion of markers to infer contact geometry, such as the TacTip [1],
Revised: This issue is prevalent in camera-based optical tactile sensors that use the lateral motion of markers to infer contact geometry, such as the TacTip [1],

Original: ChromaTouch [2], GelForce [3] and the optical tactile sensor described in [4], but may occur also in other shear-sensitive tactile sensors.
Revised: ChromaTouch [2], GelForce [3] and the optical tactile sensor described in [4], but may occur also in other shear-sensitive tactile sensors.

Original: Generally, this shear distortion of the sensor response when sliding along surfaces would make it difﬁcult to perform tactile-dependent tasks such as shape reconstruction and continuous object exploration or manipulation
Revised: Generally, this shear distortion of the sensor response when sliding along surfaces would make it difﬁcult to perform tactile-dependent tasks such as shape reconstruction and continuous object exploration or manipulation

Original: Therefore, an important capability is to separate the deformation due to the manner of contact fromthat due to the stimulus geometry.
Revised: Therefore, an important capability is to separate the deformation due to the manner of contact fromthat due to the stimulus geometry.

Original: For example, when continuously exploring an object surface, thesliding-induced shear would need to be removed from tactile image to extract the local contact shape.
Revised: For example, when continuously exploring an object surface, thesliding-induced shear would need to be removed from tactile image to extract the local contact shape.

Original: Previous work on extracting pose information from contact to continuously slide a tactile sensor overcomplex objects used a deep convolutional neural network trained to map between shear-distortedinputs and the desired (pose) outputs, which was necessary for sliding exploration over a range ofcomplex novel objects [5, 6, 7].
Revised: Previous work on extracting pose information from contact to continuously slide a tactile sensor overcomplex objects used a deep convolutional neural network trained to map between shear-distortedinputs and the desired (pose) outputs, which was necessary for sliding exploration over a range ofcomplex novel objects [5, 6, 7].

Original: Here, we suggest, an alternate disentanglement based approach thatlearns to remove motion-induced shear from raw sheared sensor response that has several advantagesover previous task-speciﬁc approach of training insensitivity to shear, in that: (a) it is computationallyand data efﬁcient by only requiring the ‘unshearing’ of the tactile data to be learnt once instead ofrelearning for each tactile task, such as when applied to full object reconstruction; (b) the modelscan then be learnt more easily on unsheared data, which we demonstrate by predicting local objectpose; moreover, in some cases no model need be learnt at all, which we demonstrate with contactsurface reconstruction using a Voronoi tessellation from the tactile marker data; and (c) the methodsare extensible to including other downstream variables of interest appropriate to the task, rather thanhaving to completely retrain new models from scratch.
Revised: Here, we suggest, an alternate disentanglement based approach thatlearns to remove motion-induced shear from raw sheared sensor response that has several advantagesover previous task-speciﬁc approach of training insensitivity to shear, in that: (a) it is computationallyand data efﬁcient by only requiring the ‘unshearing’ of the tactile data to be learnt once instead ofrelearning for each tactile task, such as when applied to full object reconstruction; (b) the modelscan then be learnt more easily on unsheared data, which we demonstrate by predicting local objectpose; moreover, in some cases no model need be learnt at all, which we demonstrate with contactsurface reconstruction using a Voronoi tessellation from the tactile marker data; and (c) the methodsare extensible to including other downstream variables of interest appropriate to the task, rather thanhaving to completely retrain new models from scratch.

Original: Our approach to disentangling environmental factors of variation and object attributes in tactilesensing is based on work in computer vision where disentanglement is considered important toimprove the generalizability of machine learning models, for example to aid in learning of robustobject representations for object classiﬁcation under novel scenarios [8].
Revised: Our approach to disentangling environmental factors of variation and object attributes in tactilesensing is based on work in computer vision where disentanglement is considered important toimprove the generalizability of machine learning models, for example to aid in learning of robustobject representations for object classiﬁcation under novel scenarios [8].

Original: Inspired from this, here wepropose a supervised convolutional neural network model that learns to disentangle the components ofsensor deformation caused by the contacted object geometry from those due to motion-induced shear.
Revised: Inspired from this, here wepropose a supervised convolutional neural network model that learns to disentangle the components ofsensor deformation caused by the contacted object geometry from those due to motion-induced shear.

Original: The models were trained using paired data from discrete (tapping) and continuous (sliding) touch,with an image-to-image ‘Disentangled UnshearNet’ model architecture that outputs unsheared tactileimages from sheared images (Fig. 1).
Revised: The models were trained using paired data from discrete (tapping) and continuous (sliding) touch,with an image-to-image ‘Disentangled UnshearNet’ model architecture that outputs unsheared tactileimages from sheared images (Fig. 1).

Original: Our validation: (1) quantiﬁes that the unsheared images fromcontinuous motion match their paired tactile images from discrete taps (Table 1); (2) demonstratesthat local contact geometry reconstructed from unsheared images using Voronoi match their pairedtapping counterparts over several stimuli shapes (Fig. 4, top row); (3) infers pose from the shearedtactile images by re-using a convolutional neural network model trained on discrete (tapping) data,which was then applied to controlling robust sliding exploration around several planar objects (Fig.
Revised: Our validation: (1) quantiﬁes that the unsheared images fromcontinuous motion match their paired tactile images from discrete taps (Table 1); (2) demonstratesthat local contact geometry reconstructed from unsheared images using Voronoi match their pairedtapping counterparts over several stimuli shapes (Fig. 4, top row); (3) infers pose from the shearedtactile images by re-using a convolutional neural network model trained on discrete (tapping) data,which was then applied to controlling robust sliding exploration around several planar objects (Fig.

Original: 4,middle row)
Revised: 4,middle row)

Original: ; and (4) combines the two previous steps to attain faithful full-object reconstruction ofseveral planar shapes (Fig. 4, bottom row).
Revised: ; and (4) combines the two previous steps to attain faithful full-object reconstruction ofseveral planar shapes (Fig. 4, bottom row).

Original:  Background
Revised:  Background

Original: Measuring strain is a fundamental component of tactile sensing, as emphasised by Platkiewicz, etal [9]: ‘the pressure distribution at the surface of a tactile sensor cannot be acquired directly and mustbe inferred from the [strain] ﬁeld induced by the touched object in the sensor medium.’
Revised: Measuring strain is a fundamental component of tactile sensing, as emphasised by Platkiewicz, etal [9]: ‘the pressure distribution at the surface of a tactile sensor cannot be acquired directly and mustbe inferred from the [strain] ﬁeld induced by the touched object in the sensor medium.’

Original: There are many mechanisms whereby normal strain and shear strain are measured by tactile sensors.
Revised: There are many mechanisms whereby normal strain and shear strain are measured by tactile sensors.

Original: Taxel-based tactile sensors such as the iCub ﬁngertip [10] and BioTac [11] directly measure normalstrain.
Revised: Taxel-based tactile sensors such as the iCub ﬁngertip [10] and BioTac [11] directly measure normalstrain.

Original: Nevertheless, this measurement of normal strain will be affected by shearing of the soft sensormedium, although the effects on sensor output will be much smaller than those from normal pressure.
Revised: Nevertheless, this measurement of normal strain will be affected by shearing of the soft sensormedium, although the effects on sensor output will be much smaller than those from normal pressure.

Original: Camera-based optical tactile sensors can be very sensitive to shear because they transduce deformationof the soft sensor surface into a lateral image.
Revised: Camera-based optical tactile sensors can be very sensitive to shear because they transduce deformationof the soft sensor surface into a lateral image.

Original: For optical tactile sensors such as the TacTip biomimeticoptical sensor used here [13, 1] and other marker-based sensors [2, 4], the surface shear will alsodistort the apparent geometry of the contact.
Revised: For optical tactile sensors such as the TacTip biomimeticoptical sensor used here [13, 1] and other marker-based sensors [2, 4], the surface shear will alsodistort the apparent geometry of the contact.

Original: This distortion is because the normal indentation isrepresented in the local shear strain of the markers, upon which the global shear strain is superimposed.
Revised: This distortion is because the normal indentation isrepresented in the local shear strain of the markers, upon which the global shear strain is superimposed.

Original: This contrasts with optical tactile sensors that reconstruct the indentation ﬁeld from reﬂection, suchas the GelSight [14], where shear will shift the imprint on the tactile image but otherwise the imprintshape should be relatively unaffected.
Revised: This contrasts with optical tactile sensors that reconstruct the indentation ﬁeld from reﬂection, suchas the GelSight [14], where shear will shift the imprint on the tactile image but otherwise the imprintshape should be relatively unaffected.

Original: This effect of shear complicates tactile perception under general motions against an object, andrestricts the application of touch on challenging tasks such as tactile exploration and surface shapereconstruction.
Revised: This effect of shear complicates tactile perception under general motions against an object, andrestricts the application of touch on challenging tasks such as tactile exploration and surface shapereconstruction.

Original: Studies have been limited to discrete contacts where the sensor taps discretely overthe object to minimize shear [16, 17, 19], or to rigid planar tactile sensors [20, 21] that do not shear.
Revised: Studies have been limited to discrete contacts where the sensor taps discretely overthe object to minimize shear [16, 17, 19], or to rigid planar tactile sensors [20, 21] that do not shear.

Original: A recent body of work on the TacTip sensor used here focused on mitigating the effects of motiondependent shear during tactile servo control to slide around planar shapes [5, 18] and complex3D objects [6, 7].
Revised: A recent body of work on the TacTip sensor used here focused on mitigating the effects of motiondependent shear during tactile servo control to slide around planar shapes [5, 18] and complex3D objects [6, 7].

Original: The shear-dependence problem was addressed in three studies that used a deep[Figure] [Figure] convolutional neural network to predict pose by training insensitivity to shear into the predictionnetwork [5, 6, 7].
Revised: The shear-dependence problem was addressed in three studies that used a deep convolutional neural network to predict pose by training insensitivity to shear into the predictionnetwork [5, 6, 7].

Original: While effective, this has a drawback that procedures for learning shear-invarianceneeds to be introduced into the training for each individual task, and thus would not generalize easilyto new tasks.
Revised: While effective, this has a drawback that procedures for learning shear-invarianceneeds to be introduced into the training for each individual task, and thus would not generalize easilyto new tasks.

Original: Another study did not use deep learning, but instead dimensionally reduced the tactiledata [18], where the the pose components naturally separated from the shear components.
Revised: Another study did not use deep learning, but instead dimensionally reduced the tactiledata [18], where the the pose components naturally separated from the shear components.

Original: This work pursues an alternate approach using deep learning to remove the effect of motion-dependentshear from the sheared tactile image, offering several advantages that were summarized in theintroduction.
Revised: This work pursues an alternate approach using deep learning to remove the effect of motion-dependentshear from the sheared tactile image, offering several advantages that were summarized in theintroduction.

Original: Our approach to disentangling environmental factors of variation and object attributesin tactile sensing is based on work in computer vision where disentanglement is considered importantto improve the generalization of machine learning models, for example to aid in learning of robustobject representations for object classiﬁcation under novel scenarios [8].
Revised: Our approach to disentangling environmental factors of variation and object attributesin tactile sensing is based on work in computer vision where disentanglement is considered importantto improve the generalization of machine learning models, for example to aid in learning of robustobject representations for object classiﬁcation under novel scenarios [8].

Original: Once disentangled, differentenvironmental factors and object attributes can be recombined to generate coherent novel conceptsthus extending knowledge to previously unobserved scenarios [8].
Revised: Once disentangled, differentenvironmental factors and object attributes can be recombined to generate coherent novel conceptsthus extending knowledge to previously unobserved scenarios [8].

Original: For example, synthesizing sensorresponses to novel stimuli by combining the stimuli attributes and factor of variation without actuallyobserving them to plan action under previously unobserved conditions.
Revised: For example, synthesizing sensorresponses to novel stimuli by combining the stimuli attributes and factor of variation without actuallyobserving them to plan action under previously unobserved conditions.

Original: In the case of tactile sensingconsidered here, the stimuli attributes could be novel contact geometry and the factors of variationthe motion-induced shear.
Revised: In the case of tactile sensingconsidered here, the stimuli attributes could be novel contact geometry and the factors of variationthe motion-induced shear.

Original:  Methods
Revised:  Methods

Original: [Figure]
Revised: [Figure]

Original: 3.1 Experimental Setup
Revised: 3.1 Experimental Setup

Original: [Figure]
Revised: [Figure]

Original: The setup in this study followed recent experiments with the TacTip biomimetic optical tactile sensor[5, 18, 6, 7] where it is mounted as an end effector on a robot arm (Fig. 1).
Revised: The setup in this study followed recent experiments with the TacTip biomimetic optical tactile sensor[5, 18, 6, 7] where it is mounted as an end effector on a robot arm (Fig. 1).

Original: The TacTip is biomimeticin that its skin morphology is based on the layered dermal and epidermal structure of the humanﬁngertip [13, 1]; the tactile sensor measures the shear displacement of an array of papillae pins on theinner side of the sensing surface caused by the skin deformation.
Revised: The TacTip is biomimeticin that its skin morphology is based on the layered dermal and epidermal structure of the humanﬁngertip [13, 1]; the tactile sensor measures the shear displacement of an array of papillae pins on theinner side of the sensing surface caused by the skin deformation.

Original: The TacTip can be designed with varied morphologies dictated by the application [1].
Revised: The TacTip can be designed with varied morphologies dictated by the application [1].

Original: The TacTipsensor (Fig. 1 (a) & (b)) used in this work had a soft rubber-like hemispherical dome (40 mm radius,
Revised: The TacTipsensor (Fig. 1 (a) & (b)) used in this work had a soft rubber-like hemispherical dome (40 mm radius,

Original: TangoBlack+) manufactured using multi-material 3D printing.
Revised: TangoBlack+) manufactured using multi-material 3D printing.

Original: Its inner side was covered with anarray of 331 pins with hard white tips arranged in a concentric circular grid.
Revised: Its inner side was covered with anarray of 331 pins with hard white tips arranged in a concentric circular grid.

Original: The dome was ﬁlledwith an optically-clear silicone gel to give it a compliance similar to a human ﬁngertip.
Revised: The dome was ﬁlledwith an optically-clear silicone gel to give it a compliance similar to a human ﬁngertip.

Original: The ﬁnalassembly of sensors is done by hand that introduces variability in the response in other otherwisesame sensors.
Revised: The ﬁnalassembly of sensors is done by hand that introduces variability in the response in other otherwisesame sensors.

Original: A camera was used to collect tactile images of the internal surface of the dome.
Revised: A camera was used to collect tactile images of the internal surface of the dome.

Original: Formore details, we refer to the references above.
Revised: Formore details, we refer to the references above.

Original: The tactile stimuli used here were three 3D-printed (ABS plastic) planar shapes: a circular disk, cloverand teardrop (Fig.
Revised: The tactile stimuli used here were three 3D-printed (ABS plastic) planar shapes: a circular disk, cloverand teardrop (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: These objects were chosen to give different edge shapes for applicationof our methods, and were fastened securely to the workspace to prevent accidental motion.
Revised: These objects were chosen to give different edge shapes for applicationof our methods, and were fastened securely to the workspace to prevent accidental motion.

Original: Anadditional four complex planar shapes, including two acrylic shapes with distinct frictional propertiesto the remaining 3D-printed shapes (spiral 1 & 2, Fig. 4) were used to test for generalizability.
Revised: Anadditional four complex planar shapes, including two acrylic shapes with distinct frictional propertiesto the remaining 3D-printed shapes (spiral 1 & 2, Fig. 4) were used to test for generalizability.

Original: 3.2 Data Collection
Revised: 3.2 Data Collection

Original: 3.2.1 Paired Canonical and Sheared Data
Revised: 3.2.1 Paired Canonical and Sheared Data

Original: This work proposes a supervised deep learning model Disentangled UnshearNet to remove the effectof motion-induced global shear in the tactile images, caused by the friction between the sensor skinand its contacted surface during contact.
Revised: This work proposes a supervised deep learning model Disentangled UnshearNet to remove the effectof motion-induced global shear in the tactile images, caused by the friction between the sensor skinand its contacted surface during contact.

Original: We emphasise this motion-induced shear is different fromthe skin deformation due to local shear from the geometry of the stimulus imprint.
Revised: We emphasise this motion-induced shear is different fromthe skin deformation due to local shear from the geometry of the stimulus imprint.

Original: Our method requires paired tactile data with minimal global shear taken from vertically tapping ontothe stimulus (referred to as canonical data) and sliding data with random global shear (referred to assheared data).
Revised: Our method requires paired tactile data with minimal global shear taken from vertically tapping ontothe stimulus (referred to as canonical data) and sliding data with random global shear (referred to assheared data).

Original: The sheared data is collected after sliding randomly and is paired with a tactile imagefrom the canonical data by using the same relative poses between the sensor and the stimuli (Fig. 1).
Revised: The sheared data is collected after sliding randomly and is paired with a tactile imagefrom the canonical data by using the same relative poses between the sensor and the stimuli (Fig. 1).

Original: The raw tactile data was cropped, sub-sampled to a 256 × 256-pixel region and ﬁnally converted tobinary images using an adaptive threshold to minimise sensitivity to internal lighting conditions.
Revised: The raw tactile data was cropped, sub-sampled to a 256 × 256-pixel region and ﬁnally converted tobinary images using an adaptive threshold to minimise sensitivity to internal lighting conditions.

Original: In total, data for 200 distinct poses were collected for each of the three stimuli (Fig.
Revised: In total, data for 200 distinct poses were collected for each of the three stimuli (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: To generateposes, the sensor location relative to the target contact location (red circle, Fig.
Revised: To generateposes, the sensor location relative to the target contact location (red circle, Fig.

Original: 1 (d)) was sampledrandomly from a uniform distribution spanning the range [-5, 5] mm in the two lateral directions(along the x - and y -axes), [-45, -45]
Revised: 1 (d)) was sampledrandomly from a uniform distribution spanning the range [-5, 5] mm in the two lateral directions(along the x - and y -axes), [-45, -45]

Original: deg in yaw ( θ n Z ) and
Revised: deg in yaw ( θ n Z ) and

Original: mm in depth ( z -axis).
Revised: mm in depth ( z -axis).

Revised: Ranges of

Revised: [Figure]

Revised: [Figure]

Revised: L2 Loss

Revised: Encoder (E)

Original: Ranges ofthe target poses and perturbations to generate canonical and sheared data were chosen to ensure safecontact of the sensor with the stimulus (Fig.
Revised: Decoder (D) Patch Loss L [Figure]

Revised: C

Revised: PC

Revised: PC

Original: {+C 
+}
Revised: {+C 
+}

Revised: [Figure]

Revised: the target poses and perturbations to generate canonical and sheared data were chosen to ensure safecontact of the sensor with the stimulus (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: Canonical Data: The canonical data was taken by vertically tapping onto the stimulus, to give arepresentative sample of tactile data with minimal global shear just due to the stimulus geometry.
Revised: Canonical Data: The canonical data was taken by vertically tapping onto the stimulus, to give arepresentative sample of tactile data with minimal global shear just due to the stimulus geometry.

Original: This gives target (reference) data that the sheared data from sliding contacts should be restored to.
Revised: This gives target (reference) data that the sheared data from sliding contacts should be restored to.

Original: The dataset, in total, had 30,000 samples: 50 instances of each of the 200 poses recorded previouslyfor each of the three stimuli shapes (Fig.
Revised: The dataset, in total, had 30,000 samples: 50 instances of each of the 200 poses recorded previouslyfor each of the three stimuli shapes (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: To generate instances, the indentation depth ( z -axes)was varied randomly between [-1, 1] mm from the original indentation depth.
Revised: To generate instances, the indentation depth ( z -axes)was varied randomly between [-1, 1] mm from the original indentation depth.

Original: Sheared Data: The sheared data accords to a canonical tactile image distorted by global shear fromlateral sliding motion of the sensor on the stimulus.
Revised: Sheared Data: The sheared data accords to a canonical tactile image distorted by global shear fromlateral sliding motion of the sensor on the stimulus.

Original: Thus, the sheared data does not give contact-onlyinformation akin to canonical data that represented the geometry of the contacted stimulus, but has asigniﬁcant component due to the contact motion.
Revised: Thus, the sheared data does not give contact-onlyinformation akin to canonical data that represented the geometry of the contacted stimulus, but has asigniﬁcant component due to the contact motion.

Original: The dataset, in total, had 90,000 samples: 150 instances of each of the 200 random poses recordedpreviously for each of the three stimulus shapes (Fig.
Revised: The dataset, in total, had 90,000 samples: 150 instances of each of the 200 random poses recordedpreviously for each of the three stimulus shapes (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: To generate instances for each pose, thelateral position offset for sliding to the target pose was varied randomly between [-2.5, 2.5] mm inlateral direction(s) (along the x - or y -axes or both), using the procedure shown in Fig.
Revised: To generate instances for each pose, thelateral position offset for sliding to the target pose was varied randomly between [-2.5, 2.5] mm inlateral direction(s) (along the x - or y -axes or both), using the procedure shown in Fig.

Original: 1 (d).
Revised: 1 (d).

Original: Training and Test Data: The canonical and sheared datasets were partitioned randomly into training,validation and test sets (ratio 60:20:20); i.e. from the 200 unique poses for each stimulus shape (Fig. 1(c)), 120 poses were assigned to a training set and 40 poses each to validation and testing sets.
Revised: Training and Test Data: The canonical and sheared datasets were partitioned randomly into training,validation and test sets (ratio 60:20:20); i.e. from the 200 unique poses for each stimulus shape (Fig. 1(c)), 120 poses were assigned to a training set and 40 poses each to validation and testing sets.

Original: Overallthe canonical and sheared training set contained 18,000 and 54,000 samples respectively, while thevalidation and testing sets contained 6,000 and 18,000 samples respectively.
Revised: Overallthe canonical and sheared training set contained 18,000 and 54,000 samples respectively, while thevalidation and testing sets contained 6,000 and 18,000 samples respectively.

Original: To generate pairedcanonical and transformed data for training, each transformed tactile image was paired randomlywith one of the canonical instances having the same pose.
Revised: To generate pairedcanonical and transformed data for training, each transformed tactile image was paired randomlywith one of the canonical instances having the same pose.

Original: 3.2.2 Pose-labelled Data
Revised: 3.2.2 Pose-labelled Data

Original: To train pose prediction network PoseNet , a separate set of pose data was collected instead of reusingthe previously collected pose data to train Entangled & Disentagled UnshearNet .
Revised: To train pose prediction network PoseNet , a separate set of pose data was collected instead of reusingthe previously collected pose data to train Entangled & Disentagled UnshearNet .

Original: This was doneto not overﬁt the training set and to ensure performance of the trained UnshearNet models despitepotential dataset shifts due to a host of factors, including drift in sensor characteristics over time orwith continued use due to wear and tear of the tactile sensing surface.
Revised: This was doneto not overﬁt the training set and to ensure performance of the trained UnshearNet models despitepotential dataset shifts due to a host of factors, including drift in sensor characteristics over time orwith continued use due to wear and tear of the tactile sensing surface.

Original: The dataset had, in total, 30,000 samples: 10,000 samples from each of the three object shapes(Fig.
Revised: The dataset had, in total, 30,000 samples: 10,000 samples from each of the three object shapes(Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: To generate contact poses, the sensor location relative to the initial contact location wassampled randomly from a uniform distribution spanning the range [-5, 5]
Revised: To generate contact poses, the sensor location relative to the initial contact location wassampled randomly from a uniform distribution spanning the range [-5, 5]

Original: mm laterally ( x - and y -axisdirections), [-45, -45]
Revised: mm laterally ( x - and y -axisdirections), [-45, -45]

Original: deg in yaw ( θ n Z ) and
Revised: deg in yaw ( θ n Z ) and

Original: mm in depth ( z -axis).
Revised: mm in depth ( z -axis).

Original: Ranges of the target poses,as before, were chosen to ensure safe contact of the sensor without causing damage.
Revised: Ranges of the target poses,as before, were chosen to ensure safe contact of the sensor without causing damage.

Original: As discussedabove, all tactile images were cropped, sub-sampled and thresholded to 256 × 256-pixel to binary(black/white) tactile images.
Revised: As discussedabove, all tactile images were cropped, sub-sampled and thresholded to 256 × 256-pixel to binary(black/white) tactile images.

Original: The dataset was partitioned randomly into training , validation and testsets in the ratio 60:20:20 as above, containing 18,000, 6,000 and 6,000 samples respectively.
Revised: The dataset was partitioned randomly into training , validation and testsets in the ratio 60:20:20 as above, containing 18,000, 6,000 and 6,000 samples respectively.

Original: 3.3 Image-to-Image Models for Shear Removal
Revised: 3.3 Image-to-Image Models for Shear Removal

Original: To achieve our goal of removing the effects of shear induced by the sliding motion, we trained twomodels based on the encoder-decoder framework – Entangled UnshearNet (Fig. 2) and Disentangled
Revised: To achieve our goal of removing the effects of shear induced by the sliding motion, we trained twomodels based on the encoder-decoder framework – Entangled UnshearNet (Fig. 2) and Disentangled

Original: UnshearNet (Fig. 3).
Revised: UnshearNet (Fig. 3).

Original: The former model (baseline) learns the mapping between the input sheared dataand its corresponding canonical (tap) data directly.
Revised: The former model (baseline) learns the mapping between the input sheared dataand its corresponding canonical (tap) data directly.

Original: The later model instead, ﬁrst disentangles thecontact-only (pose) sensor response component from the sliding-induced global shear component inthe latent space of the encoder E ; followed by accurate reconstruction of the input sheared image andits corresponding canonical (tap) image using decoders DS (sheared or sliding) and DC (canonical ortapping).
Revised: The later model instead, ﬁrst disentangles thecontact-only (pose) sensor response component from the sliding-induced global shear component inthe latent space of the encoder E ; followed by accurate reconstruction of the input sheared image andits corresponding canonical (tap) image using decoders DS (sheared or sliding) and DC (canonical ortapping).

Original: We hypothesized that a more robust mapping between sheared and canonical (tap) datacan be learned by disentangling the contact-only and sliding-induced components of sensor response.
Revised: We hypothesized that a more robust mapping between sheared and canonical (tap) datacan be learned by disentangling the contact-only and sliding-induced components of sensor response.

Original: The mathematical formulation of the Disentangled UnshearNet is listed below:
Revised: The mathematical formulation of the Disentangled UnshearNet is listed below:

Original: The encoder E splits the latent space into two components z
Revised: The encoder E splits the latent space into two components z

Original: =
Revised: =

Original: ( z p , z s ) ∈
Revised: ( z p , z s ) ∈

Original: Z p × Z s , given pairedcanonical { x c } Ni = 1 and shear-transformed training samples { x s } Ni = 1 , where x c ∈ X c and x s∈
Revised: Z p × Z s , given paired canonical { x c } Ni = 1 and shear-transformed training samples { x s } Ni = 1 , where x c ∈ X c and x s

Original:  X 
Revised: ∈ X s are in

Original: s are inthe sets of canonical and sheared data, respectively.
Revised: the sets of canonical and sheared data, respectively.

Original: The decoder DC takes as input only the pose part of the latent code z p and reconstructs the canonicalsample corresponding to the input sheared sample.
Revised: The decoder DC takes as input only the pose part of the latent code z p and reconstructs the canonical sample corresponding to the input sheared sample.

Original: In combination with the encoder E , it thus learnsa mapping M : X t → X c from the sheared to canonical data.
Revised: In combination with the encoder E , it thus learns a mapping M : X t → X c from the sheared to canonical data.

Original: The other decoder DS , takes as input the full latent code z and reconstructs the input sheared sample.
Revised: The other decoder DS , takes as input the full latent code z and reconstructs the input sheared sample.

Original: In combination with the encoder E , it thus acts as an auto-encoder and learns a mapping N : X s → X sfrom the sheared data to itself.
Revised: In combination with the encoder E , it thus acts as an auto-encoder and learns a mapping N : X s → X sfrom the sheared data to itself.

Original: Our objective function has three components:
Revised: Our objective function has three components:

Original: (1) L2 loss for matching the generated canonical samples M ( x t ) to their targets x c ; (2) L1 patch loss for matching random patches extracted from generated canonical images M ( x t ) andcorresponding targets x c for enforcing image similarity locally as changes in sensor response due tochange in contact conditions are predominately local in nature; (3) Reconstruction loss for matching the generated sheared samples N ( x t ) to their target input shearedsamples x s .
Revised: (1) L2 loss for matching the generated canonical samples M ( x t ) to their targets x c ;(2) L1 patch loss for matching random patches extracted from generated canonical images M ( x t ) andcorresponding targets x c for enforcing image similarity locally as changes in sensor response due tochange in contact conditions are predominately local in nature;(3) Reconstruction loss for matching the generated sheared samples N ( x t ) to their target input shearedsamples x s .

Original: The objective functions are:
Revised: The objective functions are:

Original: [Equation]
Revised: [Equation]

Original: ) where ( z p , z s ) =
Revised: ) where ( z p , z s ) =

Original: E ( x s ) for all x s ∈ X s .
Revised: E ( x s ) for all x s ∈ X s .

Original: The function crops () selects a patch of size 20 × 20 pix from
Revised: The function crops () selects a patch of size 20 × 20 pix from

Original: M ( x t ) and x c .
Revised: M ( x t ) and x c .

Original: The objective functions L rec , L sup and L patch represent sheared reconstruction loss,canonical supervised loss and canonical supervised patch loss respectively.
Revised: The objective functions L rec , L sup and L patch represent sheared reconstruction loss,canonical supervised loss and canonical supervised patch loss respectively.

Original: The ﬁnal objectives for Encoder E and Decoders DC , DS are:
Revised: The ﬁnal objectives for Encoder E and Decoders DC , DS are:

Original: [Equation]
Revised: [Equation]

Original: 6) where λ is the relative loss scale factor.
Revised: 6) where λ is the relative loss scale factor.

Original: Details of the speciﬁc model architectures and their training are given in Appendices A & B.
Revised: Details of the speciﬁc model architectures and their training are given in Appendices A & B.

Original:  Experimental Results4.1 Shear Removal from Tactile Images
Revised:  Experimental Results4.1 Shear Removal from Tactile Images

Original: We ﬁrst performed an ablation study to demonstrate the separation of latent representations by the
Revised: We ﬁrst performed an ablation study to demonstrate the separation of latent representations by the

Original: Disentangled UnshearNet into pose and shear codes respectively.
Revised: Disentangled UnshearNet into pose and shear codes respectively.

Original: For details see Appendix D.
Revised: For details see Appendix D.

Original: The ﬁrst test of the performance of our approach for removing motion-induced global shear fromtactile images used the multi-scale structural similarity index (SSIM) [25] to compare tactile imagesfrom the shear-transformed test data set with the paired canonical and the Disentangled (Entangled)
Revised: The ﬁrst test of the performance of our approach for removing motion-induced global shear fromtactile images used the multi-scale structural similarity index (SSIM) [25] to compare tactile imagesfrom the shear-transformed test data set with the paired canonical and the Disentangled (Entangled)

Original: UnshearNet -generated unsheared tactile images.
Revised: UnshearNet -generated unsheared tactile images.

Original: All three images represent the same pose.
Revised: All three images represent the same pose.

Original: To compare, we computed the average SSIM between the canonical (tap)-sheared and canonical(tap)-unsheared images in several distinct scenarios (stimuli: disk edge, clover corner, teardropcorner).
Revised: To compare, we computed the average SSIM between the canonical (tap)-sheared and canonical(tap)-unsheared images in several distinct scenarios (stimuli: disk edge, clover corner, teardropcorner).

Original: The unsheared images were structurally closer to the target canonical (tap) samples than theirsheared counterparts.
Revised: The unsheared images were structurally closer to the target canonical (tap) samples than theirsheared counterparts.

Original: The overall SSIM value for unsheared samples was 0.93 (0.90) in comparisonto 0.32 for sheared samples, with closeness to unity indicating a good match (Table 1).
Revised: The overall SSIM value for unsheared samples was 0.93 (0.90) in comparisonto 0.32 for sheared samples, with closeness to unity indicating a good match (Table 1).

Original: Performancewas similar for all stimuli considered with Disentangled UnshearNet achieving closer match tocanonical samples than its Entangled counterpart.
Revised: Performancewas similar for all stimuli considered with Disentangled UnshearNet achieving closer match tocanonical samples than its Entangled counterpart.

Original: [Table]
Revised: [Table]

Original: 4.2 Reconstructed Contact Geometry
Revised: 4.2 Reconstructed Contact Geometry

Original: As stated in the introduction, an application of shear removal is to aid reconstruction of the stimulusgeometry from the tactile images.
Revised: As stated in the introduction, an application of shear removal is to aid reconstruction of the stimulusgeometry from the tactile images.

Original: This is validated by using a Voronoi-based method to infer anapproximate indentation ﬁeld, using the method introduced in Cramphorn et al [12] on tactile datafrom the TacTip.
Revised: This is validated by using a Voronoi-based method to infer anapproximate indentation ﬁeld, using the method introduced in Cramphorn et al [12] on tactile datafrom the TacTip.

Original: The Voronoi method transforms the displacements of the tactile markers into areasof hexagonal cells tessellating the grid of the markers [12, Fig. 4].
Revised: The Voronoi method transforms the displacements of the tactile markers into areasof hexagonal cells tessellating the grid of the markers [12, Fig. 4].

Original: The change in Voronoi areas ofthe cells represents the skin distortion, which correlates with the indentation due to contact geometry.
Revised: The change in Voronoi areas ofthe cells represents the skin distortion, which correlates with the indentation due to contact geometry.

Original: For display purposes, we ﬁt a 3D surface to the ( x , y ) centroid coordinates using the Voronoi cell areasas the corresponding height values (Fig. 4, top row).
Revised: For display purposes, we ﬁt a 3D surface to the ( x , y ) centroid coordinates using the Voronoi cell areasas the corresponding height values (Fig. 4, top row).

Original: The ﬁtted surface from the original canonicaltactile images taken under tapping visibly represents the shape of the indentation on the tactile sensor,as expected (Fig. 4, top row).
Revised: The ﬁtted surface from the original canonicaltactile images taken under tapping visibly represents the shape of the indentation on the tactile sensor,as expected (Fig. 4, top row).

Original: The same stimulus under sliding motion is visibly distorted by shear,resulting in an indentation ﬁeld that does not resemble the stimulus shape (Fig. 4, top row).
Revised: The same stimulus under sliding motion is visibly distorted by shear,resulting in an indentation ﬁeld that does not resemble the stimulus shape (Fig. 4, top row).

Original: Evidently, the indentation ﬁelds reconstructed from the model-generated unsheared tactile imagesunder sliding motion resemble those from the canonical data (Fig. 4, top row; last
Revised: Evidently, the indentation ﬁelds reconstructed from the model-generated unsheared tactile imagesunder sliding motion resemble those from the canonical data (Fig. 4, top row; last

Original: , all stimuli)
Revised: , all stimuli)

Original: Disk
Revised: Disk

Original: Spiral Spiral
Revised: Spiral Spiral

Original: Star
Revised: Star

Original: Clover
Revised: Clover

Original: Volute
Revised: Volute

Original: Teardrop
Revised: Teardrop

Original: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d
Revised: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d

Original: ( D i s e n t a n g l e d ) T a p
Revised: ( D i s e n t a n g l e d ) T a p

Original: S h e a r e d
Revised: S h e a r e d

Original: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d
Revised: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d

Original: ( D i s e n t a n g l e d ) T a p
Revised: ( D i s e n t a n g l e d ) T a p

Original: S h e a r e d
Revised: S h e a r e d

Original: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d
Revised: U n s h e a r e d ( E n t a n g l e d ) U n s h e a r e d

Original: ( D i s e n t a n g l e d ) T a p
Revised: ( D i s e n t a n g l e d ) T a p

Original: S h e a r e d
Revised: S h e a r e d

Original: 4.3 Validation on Shape Exploration
Revised: 4.3 Validation on Shape Exploration

Original: To further conﬁrm that our model can remove shear from transformed images we trained a predictionnetwork (PoseNet) on stimulus pose parameters: lateral position ( x -horizontal, τ X ) and in-planeorientation (yaw, θ n Z ), using canonical tactile images from vertical tapping contacts.
Revised: To further conﬁrm that our model can remove shear from transformed images we trained a predictionnetwork (PoseNet) on stimulus pose parameters: lateral position ( x -horizontal, τ X ) and in-planeorientation (yaw, θ n Z ), using canonical tactile images from vertical tapping contacts.

Original: PoseNetperformance was quantiﬁed with the mean-absolute error (MAE) between predicted pose and targetwhen input data were canonical, sheared and model-generated unsheared tactile images respectivelyusing both the entangled and disentangled networks (results in Table 2).
Revised: PoseNetperformance was quantiﬁed with the mean-absolute error (MAE) between predicted pose and targetwhen input data were canonical, sheared and model-generated unsheared tactile images respectivelyusing both the entangled and disentangled networks (results in Table 2).

Original: On the sheared data, thelowest prediction error was obtained on unsheared samples with the disentangled network, withthe entangled network about 50% higher, and on the original sheared samples about 5 × poorer.
Revised: On the sheared data, thelowest prediction error was obtained on unsheared samples with the disentangled network, withthe entangled network about 50% higher, and on the original sheared samples about 5 × poorer.

Original: Again, the UnshearNet managed to successfully correct the distortion in sensor response cause byslide-induced global shear, with the disentangled representation giving the best performance.
Revised: Again, the UnshearNet managed to successfully correct the distortion in sensor response cause byslide-induced global shear, with the disentangled representation giving the best performance.

Original: Another application of shear removal that also tests the real-time performance of UnshearNet is to usethe above pose prediction network to servo control smoothly over varied object shapes during slidingmotion.
Revised: Another application of shear removal that also tests the real-time performance of UnshearNet is to usethe above pose prediction network to servo control smoothly over varied object shapes during slidingmotion.

Original: The robot successfully traced contours across all three shapes by a combination of the use of
Revised: The robot successfully traced contours across all three shapes by a combination of the use of

Original: UnshearNet for global shear removal, PoseNet for prediction of object pose from unsheared images(Fig. 4, second row) and a simple control policy (see Appendix C).
Revised: UnshearNet for global shear removal, PoseNet for prediction of object pose from unsheared images(Fig. 4, second row) and a simple control policy (see Appendix C).

Original: This clearly indicates that theproposed Disentangled UnshearNet is well suited for correcting the sliding-induced shear-distortedsensor response for real-time servo control applications.
Revised: This clearly indicates that theproposed Disentangled UnshearNet is well suited for correcting the sliding-induced shear-distortedsensor response for real-time servo control applications.

Original: 4.4 Full Object Reconstruction
Revised: 4.4 Full Object Reconstruction

Original: As a ﬁnal demonstration of our model’s utility to remove shear, we reconstructed full object shapesby combined use of contact geometry estimation (section 4.3) and sliding shape exploration (section4.4), both of which are adversely affected by motion-induced shear (Fig. 4, bottom row).
Revised: As a ﬁnal demonstration of our model’s utility to remove shear, we reconstructed full object shapesby combined use of contact geometry estimation (section 4.3) and sliding shape exploration (section4.4), both of which are adversely affected by motion-induced shear (Fig. 4, bottom row).

Original: [Table]
Revised: [Table]

Original: To attain full object reconstructions, we fused together the contact information extracted fromunsheared images corresponding to sliding contacts (acquired during shape exploration) and linearlyinterpolated them over a rectangular grid (Fig. 4, bottom row).
Revised: To attain full object reconstructions, we fused together the contact information extracted fromunsheared images corresponding to sliding contacts (acquired during shape exploration) and linearlyinterpolated them over a rectangular grid (Fig. 4, bottom row).

Original: These results clearly demonstratesthe effectiveness of our approach in learning the sheared-unsheared mapping that can then later bereused for multiple downstream tasks.
Revised: These results clearly demonstratesthe effectiveness of our approach in learning the sheared-unsheared mapping that can then later bereused for multiple downstream tasks.

Original: In addition, the faithful shape reconstructions along with theshape exploration across multiple planar objects show that the model generalizes to novel situations.
Revised: In addition, the faithful shape reconstructions along with theshape exploration across multiple planar objects show that the model generalizes to novel situations.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we proposed a tactile image-to-image supervised convolutional neural network model toremove the effects of motion-induced global shear from the tactile sensor response while preservingthe sensor deformations due to the spatial geometry of the contacted stimuli.
Revised: In this work, we proposed a tactile image-to-image supervised convolutional neural network model toremove the effects of motion-induced global shear from the tactile sensor response while preservingthe sensor deformations due to the spatial geometry of the contacted stimuli.

Original: Two UnshearNet modelswere considered: one based on a basic encoder-decoder structure with an entangled pose and shearlatent space, and a second that disentangled these components in the model latent space (Figs 2, 3).
Revised: Two UnshearNet modelswere considered: one based on a basic encoder-decoder structure with an entangled pose and shearlatent space, and a second that disentangled these components in the model latent space (Figs 2, 3).

Original: This approach was validated in three ways: (1) Comparing similarity of tactile images collected byvertically tapping without a shearing motion to those collected with an unknown shearing perturbation,for which the unsheared data passed through the Disentangled UnshearNet gave a good matchaccording to the structural similarity index (Table 1).
Revised: This approach was validated in three ways: (1) Comparing similarity of tactile images collected byvertically tapping without a shearing motion to those collected with an unknown shearing perturbation,for which the unsheared data passed through the Disentangled UnshearNet gave a good matchaccording to the structural similarity index (Table 1).

Original: (2) Reconstructing the indentation ﬁeld fromthe contact geometry using sheared tactile images passed through the Disentangled UnshearNet(Fig.
Revised: (2) Reconstructing the indentation ﬁeld fromthe contact geometry using sheared tactile images passed through the Disentangled UnshearNet(Fig.

Original: 4, top row), via a (Voronoi) method used previously to reconstruct the sensor deformation [12].
Revised: 4, top row), via a (Voronoi) method used previously to reconstruct the sensor deformation [12].

Original: The removal of motion-induced shear was apparent in the model-generated unsheared tactile images,with the contact geometry only visible after use of the UnshearNet with the original sheared imageobscuring the contact shape.
Revised: The removal of motion-induced shear was apparent in the model-generated unsheared tactile images,with the contact geometry only visible after use of the UnshearNet with the original sheared imageobscuring the contact shape.

Original: (3) Predicting the local pose of part of an object in a shear-insensitivemanner with a prediction network trained only on canonical (tapping) data (Table 2).
Revised: (3) Predicting the local pose of part of an object in a shear-insensitivemanner with a prediction network trained only on canonical (tapping) data (Table 2).

Original: This approachwas then applied to smooth servo control around several planar objects (Fig. 4, middle row).
Revised: This approachwas then applied to smooth servo control around several planar objects (Fig. 4, middle row).

Original: The combination of smooth servo control and local shape reconstruction was later used for full objectreconstruction (Fig. 4, middle row), which remains an problem under active investigation in theﬁeld [23, 24].
Revised: The combination of smooth servo control and local shape reconstruction was later used for full objectreconstruction (Fig. 4, middle row), which remains an problem under active investigation in theﬁeld [23, 24].

Original: Our approach improves upon related work [5, 6] that instead trained insensitivity toshear into the prediction network itself.
Revised: Our approach improves upon related work [5, 6] that instead trained insensitivity toshear into the prediction network itself.

Original: An obvious drawback of this approach, even though it workswell for smooth servo control, is that it does not extend readily to predict other tactile dimensions suchas parameters of the local contact geometry.
Revised: An obvious drawback of this approach, even though it workswell for smooth servo control, is that it does not extend readily to predict other tactile dimensions suchas parameters of the local contact geometry.

Original: In that case, new labelled data would need collectingto train a new model for each intended task.
Revised: In that case, new labelled data would need collectingto train a new model for each intended task.

Original: Our approach, of shear removal from tactile images,instead means that a single model based on the canonical (tapping) data can be re-used.
Revised: Our approach, of shear removal from tactile images,instead means that a single model based on the canonical (tapping) data can be re-used.

Original: The present study considered only planar objects in 2D and two components of pose.
Revised: The present study considered only planar objects in 2D and two components of pose.

Original: As recentresults have shown how to extend the PoseNet to 3D surfaces (3 pose components) or 3D edges (5pose components) for tactile servo control [5, 6], we expect the present results will extend to thosesituations, although questions remain on the best way to do this with the large number of degreesof freedom.
Revised: As recentresults have shown how to extend the PoseNet to 3D surfaces (3 pose components) or 3D edges (5pose components) for tactile servo control [5, 6], we expect the present results will extend to thosesituations, although questions remain on the best way to do this with the large number of degreesof freedom.

Original: Although the proposed disentanglement-based approach to remove shear from sensorresponse has been tested on TacTip optical tactile sensor, it should hold well for other soft opticalsensors.
Revised: Although the proposed disentanglement-based approach to remove shear from sensorresponse has been tested on TacTip optical tactile sensor, it should hold well for other soft opticalsensors.

Original: The models, however, would have to be retrained as the sensor response depends on thefrictional and elastic properties of the tactile surface, which varies across sensor types.
Revised: The models, however, would have to be retrained as the sensor response depends on thefrictional and elastic properties of the tactile surface, which varies across sensor types.

Original: One limitation of our current approach is that we limited our training to just translation shear.
Revised: One limitation of our current approach is that we limited our training to just translation shear.

Original: However,smooth servoing over the objects necessarily introduces rotational shear, as the servo control rotatesthe sensor while continuously sliding over the object surface.
Revised: However,smooth servoing over the objects necessarily introduces rotational shear, as the servo control rotatesthe sensor while continuously sliding over the object surface.

Original: However, the fact that the methodsperformed well both in controlling the sensor and reconstructing the object geometry in the presenceof this rotational shear shows that some unshearing carries over from translation to rotational shear atleast for planar objects.
Revised: However, the fact that the methodsperformed well both in controlling the sensor and reconstructing the object geometry in the presenceof this rotational shear shows that some unshearing carries over from translation to rotational shear atleast for planar objects.

Original: Our expectation is that bringing rotational shear into the training will becomeimportant when extending the methods to 3D objects.
Revised: Our expectation is that bringing rotational shear into the training will becomeimportant when extending the methods to 3D objects.

Original: Another limitation of the supervised approachinvestigated here is the requirement of paired canonical and sheared data, which complicates thedata collection.
Revised: Another limitation of the supervised approachinvestigated here is the requirement of paired canonical and sheared data, which complicates thedata collection.

Original: Thus, a semi-supervised approach would be a worthwhile extension to this work, byobviating the need for paired data collection.
Revised: Thus, a semi-supervised approach would be a worthwhile extension to this work, byobviating the need for paired data collection.

Original: Overall, we expect the methods developed here cancan be applied more widely to various other exploration and manipulation tasks involving soft tactilesensors that are adversely affected by motion-induced shear.
Revised: Overall, we expect the methods developed here cancan be applied more widely to various other exploration and manipulation tasks involving soft tactilesensors that are adversely affected by motion-induced shear.

Original: APPENDIXA: Network Architectures
Revised: APPENDIXA: Network Architectures

Original: Entangled UnshearNet: The encoder E compresses the input 256 × 256-pixel tactile image usingﬁve convolutional (Conv) layers, each followed by batch normalization (BN) and rectiﬁed linearunit (ReLU) activation layers respectively (architecture in Fig. 2).
Revised: Entangled UnshearNet: The encoder E compresses the input 256 × 256-pixel tactile image usingﬁve convolutional (Conv) layers, each followed by batch normalization (BN) and rectiﬁed linearunit (ReLU) activation layers respectively (architecture in Fig. 2).

Original: The output of the last Conv layer(8 × 8 × 64) is passed to the decoder D which upsamples it to an output 256 × 256-pixel canonicalimage PC .
Revised: The output of the last Conv layer(8 × 8 × 64) is passed to the decoder D which upsamples it to an output 256 × 256-pixel canonicalimage PC .

Original: Similarly to the Conv layers, all transposed convolutional layers (T-Conv) were followedby BN and ReLU activation layers except the output layer which used sigmoid activation functioninstead (see also Fig. 2).
Revised: Similarly to the Conv layers, all transposed convolutional layers (T-Conv) were followedby BN and ReLU activation layers except the output layer which used sigmoid activation functioninstead (see also Fig. 2).

Original: This model was used as a baseline to evaluate the performance of proposed Disentangled UnshearNetmodel on its effectiveness in removing distortion in sensor response caused by sliding-induced globalshear.
Revised: This model was used as a baseline to evaluate the performance of proposed Disentangled UnshearNetmodel on its effectiveness in removing distortion in sensor response caused by sliding-induced globalshear.

Original: For details of the training, please see Appendix B.
Revised: For details of the training, please see Appendix B.

Original: Disentangled UnshearNet: The encoder has the same inputs and architecture as the Entangled
Revised: Disentangled UnshearNet: The encoder has the same inputs and architecture as the Entangled

Original: UnshearNet , except the the output of the 5 th Conv layer is followed by two additional Conv layers,one each for the two latent codes: Pose (8 × 8 × 64) and Shear (8 × 8 × 64).
Revised: UnshearNet , except the the output of the 5 th Conv layer is followed by two additional Conv layers,one each for the two latent codes: Pose (8 × 8 × 64) and Shear (8 × 8 × 64).

Original: One decoder DC takes as input pose latent code and upsamples it to 256 × 256-pixel canonical outputtactile image PC .
Revised: One decoder DC takes as input pose latent code and upsamples it to 256 × 256-pixel canonical outputtactile image PC .

Original: All T-Conv layers were followed by BN and ReLU activation layers except theoutput layer which used sigmoid activation layer instead (architecture shown in Fig. 3).
Revised: All T-Conv layers were followed by BN and ReLU activation layers except theoutput layer which used sigmoid activation layer instead (architecture shown in Fig. 3).

Original: The other decoder DS takes as input both pose and shear codes, merges them and upsamples to anoutput 256 × 256-pixel sheared image PS (intended to match the encoder input).
Revised: The other decoder DS takes as input both pose and shear codes, merges them and upsamples to anoutput 256 × 256-pixel sheared image PS (intended to match the encoder input).

Original: Apart from the abovedifferences, DS has the same architectures as DC .
Revised: Apart from the abovedifferences, DS has the same architectures as DC .

Original: For details of the training, please see Appendix B.
Revised: For details of the training, please see Appendix B.

Original: PoseNet:
Revised: PoseNet:

Original: This model takes as input 256 × 256 image, compresses it to extract features using the
Revised: This model takes as input 256 × 256 image, compresses it to extract features using the

Original: Conv part which are then combined using fully connected (FC) part to predict continuous-value posecomponents at the output.
Revised: Conv part which are then combined using fully connected (FC) part to predict continuous-value posecomponents at the output.

Original: In total, the network consists of ﬁve convolution layers, two max pooling(MP) layers following 2 nd and 4 th Conv layers; the CONV layers were followed by BN and ReLUactivation layers respectively.
Revised: In total, the network consists of ﬁve convolution layers, two max pooling(MP) layers following 2 nd and 4 th Conv layers; the CONV layers were followed by BN and ReLUactivation layers respectively.

Original: The 1 st FC layer used ReLU activation while the output FC layer usedsigmoid activation.
Revised: The 1 st FC layer used ReLU activation while the output FC layer usedsigmoid activation.

Original: For details of the architecture and training see Fig.
Revised: For details of the architecture and training see Fig.

Original: 5 and Appendix B.
Revised: 5 and Appendix B.

Original: [Figure]
Revised: [Figure]

Original: B: Training Details
Revised: B: Training Details

Original: All the inputs (binary 256 × 256-pixel images) and outputs (binary 256 × 256-pixel images forimage-to-image models, pose parameters for image-to-pose model) were scaled to the range [ 0 , 1 ] .
Revised: All the inputs (binary 256 × 256-pixel images) and outputs (binary 256 × 256-pixel images forimage-to-image models, pose parameters for image-to-pose model) were scaled to the range [ 0 , 1 ] .

Original: All networks weights were initialized from a zero-centered normal distribution with 0.02 standarddeviation.
Revised: All networks weights were initialized from a zero-centered normal distribution with 0.02 standarddeviation.

Original: Both image-to-image models were trained on data collected from all three stimuli shapes(Fig.
Revised: Both image-to-image models were trained on data collected from all three stimuli shapes(Fig.

Original: 1 (c)) as discussed in section 3.2.
Revised: 1 (c)) as discussed in section 3.2.

Original: For image-to-image models, we used a batch size of 32.
Revised: For image-to-image models, we used a batch size of 32.

Original: All convolutional/transposed convolutionallayers used L1/L2 regularization (10e-4) along with random image shifts, 1-2% of image size toprevent overﬁtting.
Revised: All convolutional/transposed convolutionallayers used L1/L2 regularization (10e-4) along with random image shifts, 1-2% of image size toprevent overﬁtting.

Original: The L2 loss computed across the entire image and L1 patch loss – computedbetween 100 random crops of size 20 × 20 of generated unsheared images and correspondingcanonical images – were used to train the networks in the ratio 10:1.
Revised: The L2 loss computed across the entire image and L1 patch loss – computedbetween 100 random crops of size 20 × 20 of generated unsheared images and correspondingcanonical images – were used to train the networks in the ratio 10:1.

Original: In case of Disentangled
Revised: In case of Disentangled

Original: UnshearNet , the encoder (E) and shear decoder (DS) were trained using reconstruction loss.
Revised: UnshearNet , the encoder (E) and shear decoder (DS) were trained using reconstruction loss.

Original: For image-to-pose model, we used a batch size of 256.
Revised: For image-to-pose model, we used a batch size of 256.

Original: Like image-to-image models, all layersused L1/L2 regularization (10e-4) along with random image shifts, 1-2% of image size to preventoverﬁtting.
Revised: Like image-to-image models, all layersused L1/L2 regularization (10e-4) along with random image shifts, 1-2% of image size to preventoverﬁtting.

Original: The network was trained via the L2 loss computed between predicted and target poseparameters.
Revised: The network was trained via the L2 loss computed between predicted and target poseparameters.

Original: We used ADAM optimizer [22] with learning rate of 0.0001, β 1 = 0.5 and β 2 = 0.999.
Revised: We used ADAM optimizer [22] with learning rate of 0.0001, β 1 = 0.5 and β 2 = 0.999.

Original: We usedlearning rate scheduling for training.
Revised: We usedlearning rate scheduling for training.

Original: For image-to-image models, the learning rate was reduced toone-ﬁfth and one-tenth after 40 and 80 epochs.
Revised: For image-to-image models, the learning rate was reduced toone-ﬁfth and one-tenth after 40 and 80 epochs.

Original: The models were trained, in total, for 100 epochs.
Revised: The models were trained, in total, for 100 epochs.

Original: Forimage-to-pose model, the learning rate was reduced to half and one-tenth after 100 and 200 epochsrespectively.
Revised: Forimage-to-pose model, the learning rate was reduced to half and one-tenth after 100 and 200 epochsrespectively.

Original: The model was trained for 250 epochs.
Revised: The model was trained for 250 epochs.

Original: For all models, the model with best validationaccuracy was used for testing.
Revised: For all models, the model with best validationaccuracy was used for testing.

Original: Finally, training and optimization of the networks was implemented in the Tensorﬂow 2.0 library ona NVIDIA GTX 1660 (6 GB memory) hosted on a Ubuntu machine.
Revised: Finally, training and optimization of the networks was implemented in the Tensorﬂow 2.0 library ona NVIDIA GTX 1660 (6 GB memory) hosted on a Ubuntu machine.

Original: C: Control Policy for Continuous Contour Following
Revised: C: Control Policy for Continuous Contour Following

Original: Local pose estimation allows a robot to maintain contact while safely moving over the object, thusenabling complex robot-object interactions.
Revised: Local pose estimation allows a robot to maintain contact while safely moving over the object, thusenabling complex robot-object interactions.

Original: To demonstrate continuous 2D contour following, weused a simple control policy with following two aims: 1) keep the sensor normal to object surfacewhile in motion and 2) at every time step t , move the sensor tangentially along the surface by apredeﬁned step (0.5 mm in this case).
Revised: To demonstrate continuous 2D contour following, weused a simple control policy with following two aims: 1) keep the sensor normal to object surfacewhile in motion and 2) at every time step t , move the sensor tangentially along the surface by apredeﬁned step (0.5 mm in this case).

Original: To achieve these aims, a discrete-time proportional-integral(PI) controller was implemented to output a change in the pose of the sensor ( ∆ p ( t ) ) in its referenceframe
Revised: To achieve these aims, a discrete-time proportional-integral(PI) controller was implemented to output a change in the pose of the sensor ( ∆ p ( t ) ) in its referenceframe

Original: [Equation]
Revised: [Equation]

Original: where K p and K i are diagonal gain matrices with proportional gain of 0.5 and integral gains of 0.3 and0.1 for translations and rotations respectively.
Revised: where K p and K i are diagonal gain matrices with proportional gain of 0.5 and integral gains of 0.3 and0.1 for translations and rotations respectively.

Original: e was error between the predicted pose and a referencenormal to the edge.
Revised: e was error between the predicted pose and a referencenormal to the edge.

Original: D: Disentanglement of Latent Representations
Revised: D: Disentanglement of Latent Representations

Original: An ablation study was used to verify the separation of latent representations in the Disentangled
Revised: An ablation study was used to verify the separation of latent representations in the Disentangled

Original: UnshearNet into pose and shear codes respectively.
Revised: UnshearNet into pose and shear codes respectively.

Original: To do this, we passed the ‘Shear Code’ to theunsheared reconstruction decoder (DC) instead of the ‘Pose Code’.
Revised: To do this, we passed the ‘Shear Code’ to theunsheared reconstruction decoder (DC) instead of the ‘Pose Code’.

Original: As expected, this led to a severedegradation in performance with mean-square error (MSE) between the ground truth images C (tap)and unsheared images PC increasing to 0.22, an order of magnitude higher than original of 0.023.
Revised: As expected, this led to a severedegradation in performance with mean-square error (MSE) between the ground truth images C (tap)and unsheared images PC increasing to 0.22, an order of magnitude higher than original of 0.023.

Original: In similar fashion, the SSIM index dropped to 2% when using the Shear Code to reconstruct theunsheared images PC instead of the original 93% when the Pose Code was used.
Revised: In similar fashion, the SSIM index dropped to 2% when using the Shear Code to reconstruct theunsheared images PC instead of the original 93% when the Pose Code was used.

Original: Likewise, asimilardegradation was observed when the ‘Shear Code’ was replaced by the Pose Code to reconstruct thesheared input S .
Revised: Likewise, asimilardegradation was observed when the ‘Shear Code’ was replaced by the Pose Code to reconstruct thesheared input S .

Original: The mean-squared error between S and the sheared output PS increased to 0.1, whichwas 50-times the original of 0.002 when both ‘Pose and Shear Codes were used to reconstruct PS .
Revised: The mean-squared error between S and the sheared output PS increased to 0.1, whichwas 50-times the original of 0.002 when both ‘Pose and Shear Codes were used to reconstruct PS .

Original: The
Revised: The

Original: SSIM index showed a similar trend with the similarity dropping from 99 .
Revised: SSIM index showed a similar trend with the similarity dropping from 99 .

Original: 5% to only 11%.
Revised: 5% to only 11%.

Original: This showsthat the Shear Code is indeed encoding relevant information required for successful reconstruction ofsheared input S .
Revised: This showsthat the Shear Code is indeed encoding relevant information required for successful reconstruction ofsheared input S .

Original: These results clearly demonstrate that the Disentangled UnshearNet successfullydisentangles the latent representations as desired.
Revised: These results clearly demonstrate that the Disentangled UnshearNet successfullydisentangles the latent representations as desired.

Original: [Figure]
Revised: [Figure]