<figure><img src="image_1.png" /></figure>

HAL
open science

# Haptic Edge Detection Through Shear

Jonathan Platkiewicz, Hod Lipson, Vincent Hayward

► To cite this version:

Jonathan Platkiewicz, Hod Lipson, Vincent Hayward. Haptic Edge Detection Through Shear. Scientific Reports, 2016, 6, pp.23551. 10.1038/srep23551. hal-01296656

HAL Id: hal-01296656

https://hal.sorbonne-universite.fr/hal-01296656

Submitted on 1 Apr 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
SCIENTIFIC REPORTS

<figure><img src="image_3.png" /></figure>

OPEN

# Haptic Edge Detection Through Shear

Jonathan Platkiewicz<fnref n="1" />, Hod Lipson² & Vincent Hayward³

Most tactile sensors are based on the assumption that touch depends on measuring pressure. However, the pressure distribution at the surface of a tactile sensor cannot be acquired directly and must be inferred from the deformation field induced by the touched object in the sensor medium. Currently, there is no consensus as to which components of strain are most informative for tactile sensing. Here, we propose that shape-related tactile information is more suitably recovered from shear strain than normal strain. Based on a contact mechanics analysis, we demonstrate that the elastic behavior of a haptic probe provides a robust edge detection mechanism when shear strain is sensed. We used a jamming-based robot gripper as a tactile sensor to empirically validate that shear strain processing gives accurate edge information that is invariant to changes in pressure, as predicted by the contact mechanics study. This result has implications for the design of effective tactile sensors as well as for the understanding of the early somatosensory processing in mammals.

Received: 13 November 2015

Accepted: 02 March 2016

Published: 24 March 2016

A reliable mapping between the physical world and the acquired data is a basic issue faced by any artificial or natural sensory system. For instance, in vision, a fundamental challenge is to access robustly the geometry of a body from the structure of the captured light intensity, despite variations in the viewing conditions. This task is difficult because the intrinsic geometry of a body is not mapped one-to-one to the geometry of images<fnref n="1" />⁻³. Solutions to this problem have many applications, chief among them is computer vision⁴. Insights into this process also have important consequences for our understanding of the neurophysiology of visual perception⁵,⁶. In haptics, an analogous problem arises when attempting to access shape information from the deformation of a sensing probe in contact with an unknown rigid object⁷. In vision, analysis of the variations in distribution of light intensity enables the projected image to be decomposed into a set of primitives—edges, bars, blobs, and terminations—giving rise to the so-called 'raw primal sketch'⁸. This decomposition leads to the notion of scale-space, where an image is analyzed at different resolutions since similar structures may occur at different scales⁹⁻¹², thus enabling scale-invariant feature detection and analysis¹³. Given appropriate constraints, the primal sketch makes the identification of key aspects of the three-dimensional structure of a scene not only possible, but robust and efficient.

Here, it is proposed that a similar concept is applicable to touch, subject to the differences in the type of available sensory data in the two modalities. The regions of high strain caused by the protruding, low curvature features of a touched solid are undoubtedly of key relevance to haptic perception at different stages of somatosensory processing¹⁴,¹⁵. The robust detection of regions of high strain is thus of fundamental importance in the elaboration of any intermediate shape representation. In this study, we highlight an intriguing similarity between the inverse problems that must be solved in haptic shape processing and in visual shape processing when considering the mechanical behavior of an elastic medium deformed by an unknown object.

Artificial tactile sensors are typically comprised of an elastic sheet set against a rigid foundation in which mechanically sensitive elements are embedded¹⁶⁻²³. Several mechanical parameters have been proposed for being tactually relevant: surface deformation, normal strain or stress, shear strain or stress, principal strains or axial strains, or a combination of these parameters. Currently, there is no consensus as to which of these parameters the sensors should detect (see a recent review²⁴).

In primate fingers, which are very soft, the assumption that a high firing rate of the afferents innervating a given region of skin is a sufficient representation of an edge is rarely questioned²⁵. However, the match with *in vivo* data has generally required the fit of models associated to numerous assumptions and parameters; and has been achieved for only certain contact conditions²⁶⁻²⁸. These discrepancies, often attributed to the complexities of skin mechanics, may also be a reflection of the fact that at the length-scale of the mechanoreceptors, the

<footnote n="1">¹The City College of New York, The City University of New York, Department of Mathematics, New York, NY 10031, USA. ²Columbia University, Department of Mechanical Engineering, New York, NY 10027, USA. ³Sorbonne Universités, UPMC Univ Paris 06, UMR 7222, ISIR, F-75005, Paris, France. Correspondence and requests for materials should be addressed to J.P. (email: jonathan.platkiewicz@gmail.com)</footnote>
<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

**Figure 1.** Shear sensing hypothesis. (a) First consider an elastic half-space indented by a line load (knife-edge). The black line shows surface displacement according to the classical theory³⁸. Surface displacement is undefined at *x*=0 in the case of an infinitely narrow edge. A deformation field develops inside the elastic body. The gray and black blocks respectively represent undeformed and deformed cubic elements. Shear strain corresponds to the angular deformation of these elements. (b) The graph shows the evolution of shear along the *x* direction. (c) Generic contact patch with the axes aligned with the principal curvatures, *k*<sub>*x*</sub> and *k*<sub>*y*</sub>, the inverses of the radii of the osculating circles. Surface displacement caused (d) by an edge of finite extent, such as the edge of a coin where *k*<sub>*x*</sub> is large and *k*<sub>*y*</sub> small, and (e) by a small axi-symmetric punch where *k*<sub>*x*</sub> and *k*<sub>*y*</sub> are both high. (f) Regions of shear develop on each side of the impinging edge. (g) For a small punch, the region of shear deformation forms an annulus centered where it is zero. If the curvature of the indenting object becomes small, the magnitude plot of the shear morphs into the case of the punch.

macroscopic laws of continuum mechanics do not apply, and the receptors exhibit sensitivity to privileged modes
of deformation<sup>7</sup>.

In the present study, we propose that shear strain is a highly informative quantity to be extracted from a sensing sheet because the information that it contains is largely invariant to contact conditions. This hypothesis is justified by a mechanical analysis of the contact between an object and an elastic medium. The theory was tested using the universal jamming gripper<sup>29,30</sup>. Taking advantage of its shape-memory property, this device was employed as a tactile sensor by imaging its surface after contact with an object.

**Results**

**Why shear?** The shear sensing hypothesis has been advanced by several authors³¹⁻³³. Using the finite-element method, Ricker and Ellis pointed out that shear strain enables one to distinguish contacts causing similar normal strain profiles. Using a contact mechanics analysis, Wang and Hayward suggested that the shear strain distribution resulting from a line load resembles the derivative of a Gaussian function. In effect, line loads induce shear strain distributions that exhibit local maxima on each side of a narrow contact while going through zero where the edge impinges on the surface (Fig. 1a,b), irrespective of the magnitude of the pressure.

When a rigid object with protruding edges comes in contact with an elastic half-space, the contacts can be modeled in first approximation as a line load or punch indentation according to the magnitude of the principal curvatures in the local region of contact. For instance, in human fingers, ‘quasi infinite edges’ arise if one curvature of the object is greater than that of the finger and if the other curvature is much smaller. Curved or short edges with high and low principal curvatures, Fig. 1c, like those of a small coin (or of the tip of a screwdriver), will create deformation fields that resemble those shown in Fig. 1d. These edges create contact conditions where the magnitude of the shear has local maxima in two regions, indicating the presence of an edge where shear crosses zero, see Fig. 1f. Another frequent case is that of the punch indentation, such as a protruding corner or a Braille dot (high local principal curvatures), see Fig. 1e. These cases correspond to the formation of an annulus of shear around the point of contact. Shear strain crosses zero along all radial directions, see Fig. 1g.

These local extrema and zero crossings can serve as descriptive primitives of the underlying signal in a very
general way³⁴. In haptic processing we propose that the local extrema of the distribution of shear can be robust
descriptors of the local topography of a touched object. In a given domain, pairs of extrema of shear strain would
reveal the presence of an edge and its orientation under the condition that local shear crosses zero. A detection
method based on such generic features would be invariant to surface pressure, making the detection robust to
contact conditions.

**Edges in vision and touch.** In vision, edges correspond to discontinuities in the light intensity. These dis-
continuities can arise from the occlusion of an object by another, discontinuities in surface orientation or reflec-
tance, or cast shadows. These circumstances create edges in a great variety of cases. In touch, edges primarily
indicate discontinuities in surface orientation, and only if these discontinuities are protruding. They are rarely
<figure><img src="image_11.png" /><figcaption>**Figure 2.** Visual edges compared to tactile edges. The detection of a step cannot be performed by direct differentiation because it is an ill-posed problem⁵⁶. Sensor signals being invariably corrupted by noise, including sampling noise, differentiation is inherently unstable. (a) In machines and animals it is well accepted that edge detection must begin by a smoothing operation (convolution, ‘*’, by a Gaussian) to set the scale, σ, at which the signal ought to be processed according to contrast and width. The ability to handle multiple scales is a fundamental requirement to achieve invariance with respect to distance and illumination. This step is followed by double differentiation to locate the edge by detection of local extrema and zero crossings. In animals, this process begins in the retinal cells and is distributed in the visual pathway and the cortical primary visual areas. (b) In touch, since individual sensing units are set at a distance, z, from the surface, the detection of a roof begins by blurring owing to the elastic mechanics of the sensor. If, according to our hypothesis, shear deformation is sensed then edges can be located robustly, like in vision, by the detection of local extrema and zero crossings, without the need to differentiate twice. The mechanical processing scale is determined by the depth of the receptors and the depth of indentation. Scale invariance should apply to indentation depth since the same edge must be detected regardless of how much pressure is applied. In mammals, this process could begin in tactile afferents⁵⁴, continue in the dorsal column nuclei⁵⁵, and terminate in the cortical somatosensory areas⁶⁰.</figcaption></figure>

caused by other contact instances. Notably, sharp concave edges cannot be touched and must be inferred³⁵. As a result, the principal curvatures must always be of the same sign. Flush surfaces with discontinuous frictional properties, such as different materials assembled edge-to-edge, are not typically classified as tactile edges (see later a discussion about their illusory occurrence), while their putative visual equivalents—illumination discontinuities on a uniform surface—are certainly classified as genuine edges.

Beyond the greater variety of situations corresponding to visual edges compared to tactile edges, the notion of edge differs more profoundly in the two modalities. Discontinuities in a light field are primarily described by step discontinuities, that is, when the light intensity differs in magnitude on each side of an edge. While roof discontinuities can occasionally arise in vision, in touch, edges are always roof discontinuities in sensor surface displacement (Fig. 2b). Even in the limit case of a knife-edge or sharp punch indentation, the surface displacement must remain continuous, lest the sensor be damaged; it is the surface orientation that is discontinuous and differs on each side of an edge. Because mechanical sensing cannot be performed at the probe’s surface, contact mechanics dictates that what can be sensed must be a blurred transformation of surface displacement (an exception must be made for optically-based tactile sensors that convert mechanical roof discontinuities into optical step discontinuities³⁶).

In vision, it is well accepted that edges can be extracted from an image by filtering it with a Laplacian of Gaussian function (Fig. 2a)¹⁰,³⁷. We propose that tactile edges can be extracted by a similar process, but the implementation of this process must differ fundamentally in the two modalities. Mechanically, it is possible to express shear strain, γ, in terms of the Laplacian of Gaussian filters (Methods). Thus, whenever a narrow contact impinges on the surface of a tactile probe, the condition, γ = 0, must be satisfied just below the surface. This result can be seen by locating a narrow contact at the origin, x = 0, and expressing the shear field, γ(x, z), as the convolution of the surface pressure, p(x), by a Gaussian of width ε, φ(x), differentiated along the spatial variable, x, scaled by depth, z, and by the inverse of the elastic modulus of the medium, E, (Methods),

$$ \gamma(x, z) \approx \frac{3}{E} z \frac{d}{dx} (p * \phi_{\epsilon})(x). \qquad (1) $$

It implies that in terms of displacements, if w(x) is the surface displacement normal to the contact plane, the condition,

$$ \frac{d^2}{dx^2} (w * \phi_{\epsilon})(x) = 0, \qquad (2) $$

must hold for an incompressible elastic medium. The function, φε(x), acts like ‘spread function’ from the view point of sensing units embedded in the surrounding elastic medium. From the limit case of a knife edge, and the Taylor expansion of the solution of the Boussinesq-Flamant’s problem³⁸ (Methods), we can deduce that the blurring width is ε = zg/π/8, which thus scales with depth in the medium. Finite angle roof edges, blunt edges, and
<figure><img src="image_12.png" /><figcaption>**Figure 3.** Comparison of the information provided by normal vs. shear strain sensing. (a) The common roof edge does correspond to a extremum in normal strain. The detection of this extremum, however, is mathematically equivalent to the computation of the point where its derivative is zero, the computation of which is an ill-posed problem⁵⁶. The value of its maximum increases with indentation and thus lacks an important invariant property. In contrast, shear strain crosses zero for any indentation depth at the edge location. (b) A smooth edge corresponds to a weak, undetectable variation of normal strain. The zero-crossing of shear strain is conserved regardless of the local geometry of the edge. (c) A raised edge has a sufficiently small curvature such that, once blurred, it is detected like a sharp edge despite lack of conformability. (d) A bar, or double edge, corresponds to two maxima in an appropriate range of scales. Shear strain preserves the zero crossings for a wide range of bar geometries and sensor depths. Cases (a,b,d) were considered in³²,⁴⁰,⁶¹. The first term of (3) suggests that if pressure is an even function then shear strain is an odd function as in cases (a–d). For pure differential traction, case (e) could be confused with (a) owing to the second term yielding the same profile of shear as the first term in case (a) for appropriate values of the spacing and up to a 90° rotation³³. The two cases are felt similarly⁴⁷. (f) Strips imposing alternating tractions produce below the surface normal strains and shear strains that vary similarly since the first term is zero in this case⁵⁰. The central strip is felt as protruding (or recessing) from the surface as in case (d)⁴⁹,⁵⁰.</figcaption></figure>

round edges will generally give rise to the same blurring property as sharp edges provided that the local details of high curvature are smoothed by the mechanics. For example, in the case of roof edges of support 2a in contact with the surface, Fig. 2b, the equivalent blur width is $\epsilon = z(1 + a^2/3z^2)^{\frac{2}{\pi/8}}$. Thus, the dullness of the edge operates as a correction term (Methods). An edge with an obtuse angle will appear more blurred than a sharp edge. In all cases, the deeper the mechanoreceptors are located, the blurrier is their vision of the surface.

Expression (1) considers the ideal case of a frictionless surface, but realistic interactions generally also include a tangential component, the traction, $q(x)$. Considering the contributions of both pressure and traction gives (Methods),

$$ \gamma(x, z) \approx \frac{3}{E} \left[ z \frac{d}{dx} (p * \phi_{\epsilon})(x) - (q * \phi_{\epsilon})(x) \right], \quad (3) $$

which shows that certain types of interactions other than edges can create shear zero-crossings, further suggesting that tactile sensing, as noted earlier, is inherently ambiguous.

Despite these ambiguities, in typical cases, shear analysis enables the robust detection of the presence and the localization of an edge, whereas normal strain measurement leads to solving an ill-posed problem (Fig. 3a–d). The complexity of the mechanical equations enables the existence of ambiguous contact conditions that cannot be resolved by normal strain or shear strain sensing, see Fig. 3e,f.

**Predictions relative to haptic sensing.** The application of contact mechanics theory to the interaction of a soft probe with a rigid object, (3), allows us to draw important conclusions regarding the haptic sensing of shape:

* The elasticity of the probe creates a Gaussian filter of scale, $z^{\frac{2}{\pi/8}}$, with this quantity receiving a correction term dependent on the edge sharpness. This implies that the deeper the sensors, the more blurred the effects of surface deflection; the zero-crossings are preserved at all depths just below the surface.

* Stiffer medium decreases tactile sensitivity; resolution is independent of the stiffness of the medium. Protecting the sensing units from damage limits how close they can be to the surface, which can be compensated by a softer material.
<figure><img src="image_13.png" /><figcaption>**Figure 4.** The universal jamming gripper as a tactile sensor. (a) Illustrated. (b) The gripper membrane was initially in its rest configuration and faced the stimulus straight edge. (c) The edge indented the gripper in its elastic state at a given depth. (d) Negative pressure rigidified the gripper, the stimulus was removed and its surface scanned in 3D. (e) Incremental indentations made it possible to estimate the shear component of the surface deformation by finite differences (black circles of one frame and white circles of the next frame projected on the x-y plane). (f) Shear strain map in gray scale—white for the most negative value and black for the most positive one—with the locus of zero crossing marked by a black line for a single indentation value. Arrows indicate the position of the true edge. (g) Shear map averaged over all tested indentations. The average zero crossing locus is indicated by a solid line and the errors (average ± standard-deviation) by dashed lines.</figcaption></figure>

* In the absence of friction, the elastic behavior of a probe can give rise to a Laplacian of Gaussian filter of the surface displacement normal to the contact plane. It follows that, according to the fingerprint theorem³⁹, this displacement can be recovered if the shear strain is known at different depths below the surface of the probe.

* Tangential displacements and normal indentation can cause similar sensory inputs. For example, if $q(x) = -zp'(x)$ (Methods), certain pressure distributions and tangential tractions distributions give rise to the same shear magnitudes. The same is true of normal strain.

**Experimental detection of an edge.** We took advantage of the high conformability capability of the ‘universal jamming gripper’ to make a sensing probe²⁹. This device is comprised of an elastic bag filled with a granular material, Fig. 4a. The bag was attached to a rigid base communicating to an air conduit connected to a pressure source, see Fig. 4b. When the pressure inside the bag was lowered, the particles jammed and the system underwent a phase transition from an elastic state to a solid state. The device in its elastic state was pressed against a target object, a close approximation to a line load. When the device conformed tightly with the target, Fig. 4c, pressure was lowered and an imprint of the object was memorized, Fig. 4d. The surface of the probe had 80 optical markers that could be imaged in three dimensions. The shear strain map and the locus of zero-crossing could then be computed by finite differences from successive incremental indentations, as exemplified in Fig. 4e, (Methods). The shear strain map would typically have the form shown in Fig. 4f, where the locus of the zero-crossing is shown by a black line (Methods).

To test invariance with respect to pressure, the process was repeated for different depths of indentation, Fig. 4g. Over all indentations, the quality of a straight-line regression (Deming method) applied to the zero-crossing points was 0.2 ± 0.1 mm (average ± standard-deviation). The distance of each zero-crossing point to the actual edge was 1.1 ± 1.1 mm and the angular difference between the regressed line and the actual edge was 1.6 ± 1.0 degrees. The locus and the orientation of the edge were recovered regardless of indentation.

**Discussion**

Access to the topography of a touched object should be immune to sensing conditions and sensor noise. It should be robust but also efficient. In quasi-static touch, the sources of variability include the contact conditions, the pressure applied, and the sharpness of protruding features. The commonly adopted method of detection of regions of maximal strain by differentiation is inherently unstable because it is sensitive to contact conditions,
pressure applied, and sensor noise. The measurement of shear strain rather than normal strain naturally creates a Laplacian of Gaussian-type filter ("Mexican hat" filter) on the spatial distribution of the surface tangential and normal displacements. In other words, shear sensing in an elastic probe provides a scale-space analysis where the detection of zero-crossing makes it possible to reconstruct the topography of a touched object robustly and at different scales.

**Implications for conventional artificial tactile sensing.** A tactile sensing principle based on shear strain detection could be applied to artificial tactile sensing with the benefit of invariance to contact conditions, which previous techniques did not afford. Following the approach of previous designs, arrays of shear-sensitive micro-sensors could be embedded within an elastic membrane to detect mechanical stimuli by any known sensing principle (e.g. electrical, optical, or fluidic). Our theory suggests that these individual sensing units ought to be sensitive to shear deformation and relatively insensitive to normal strain.

Tactile sensors often leverage the piezoresistive effect by using arrays of thin conductive elastomer wires or beads that exhibit a variation of conductance under compression¹⁶⁻²³. Such reliance on normal strain renders these devices overly dependent on how an object is pressed upon their surface which introduces noise and ambiguity (see Fig. 3a–d). By contrast, the detection of the zero-crossing of the shear strain would remain invariant under a wide range of sensing conditions. One such important and common case arises when the sensor surface does not conform with an object. Under this scenario, the zero-crossing of the shearing strain would not be different from the case of perfect conformation (see Fig. 3c,d).

**Beyond normal strain.** Tactile sensing based on the measurement of parameters beyond just normal strain were considered by Fearing and Hollerbach: "Sensors that combine surface shear sensors, depth strain sensors, and surface deflection sensing would simplify the problem considerably"⁴⁰; see also the discussion by Ricker and Ellis³². Tactile sensors that directly detect the displacements of the sensing membrane by optical projection benefit from having direct access to shear strain⁴¹,⁴². Optics also provides access to shear strain in the bulk, by imaging the relative displacement of two layers⁴³. Photogrammic analysis of the light reflected by a surface of constant albedo, can provide detailed topography of a contacted object, and thus shear near the surface⁴⁴. However, to our knowledge, no study has considered the option of detecting the zero-crossing of shear strain, nor explicitly formulated the natural processing performed by mechanical blurring.

**Tactile illusions.** In several reported haptic illusions, a sensation of shape can be elicited from distributed cutaneous lateral deformation⁴⁵. For instance, in the “comb illusion”, the sensation of a raised dot traveling on the finger is experienced when a progressive wave of shearing deformation is produced on the fingerpad³³,⁴⁶,⁴⁷. In the “fishbone illusion” and its variants, a sensation of a raised shape is induced when rubbing one’s finger on surfaces divided into strips made of different materials and/or textures⁴⁸⁻⁵⁰. The undulations of a surface can be magnified when a brush-like structure, the “tactile contact lens”, is interposed between the surface and the skin⁵¹. The recently described “chop-stick illusion” could be similarly explained⁵². All these illusions are based on the principle that certain surface tractions and pressure distributions can produce similar strains. If the surface pressure and traction are such that $q_{\text{illusion}} \approx -z \partial_x p_{\text{real}}$ and $p_{\text{illusion}} = \text{cst}$, where $p_{\text{real}}$ is the pressure applied by a fingertip pressed against a raised surface, the shearing strain beneath the surface, $\gamma_{\text{illusion}}(x, z)$, will be indistinguishable from the shear strain observed in normal conditions, $\gamma_{\text{real}}(x, z)$.

**Neurophysiology of edge detection.** During static touch, the geometric features of haptic shapes are thought to be encoded by a population of slowly adapting mechanoreceptors⁵³. Yet, the computational mechanism accounting for the detection of such features remains obscure. Recently, Pruszynski and Johansson proposed that these computations could be mediated by the combined activity of tactile afferents in the glabrous skin⁵⁴. The authors propose a model based on the convolution of an afferent receptive field map with a spatially-filtered version of the presented stimuli. Under the assumption that slowly adapting mechanoreceptors encode shear strain, our framework suggests that second-order somatosensory neurons could detect the zero-crossing of the sensory input⁵⁵. Thus, geometric haptic feature detection could be performed by a combination of pre-neural and early neural processing. A similar computational mechanism was proposed in vision⁹⁻¹¹,³⁷,⁵⁶, with Laplacian of Gaussian filtering assumed to begin in the neural interaction of ganglion cells of the retina. By contrast, if mechanoreceptors indeed encode shear strain, this filtering process in touch may be directly mediated by the skin deformation. Here, we propose that a simple way for biologic and artificial tactile devices to realize robust edge detection is through shear strain sensing.

## Methods

**Experimental details.** It was assumed that the curvature of the probe was much smaller than that of the object, thus approximating the case of a semi-infinite half-space indented by a line load. It was further assumed that the contribution of the tangential displacement to shear was negligible and that the contribution from normal displacement varied little with depth (for validity of these assumptions, see Supplementary Information). Under these approximations, the measurement of the normal displacement gradient at the surface provides an estimate of the shear in the medium close to the surface. We ensured that the edge passed by the center of the probe. We tested nine levels of indentation, the relative distance by which the edge moved after the initial contact with the probe. Indentation was varied by increments of +2.5 mm within a range of 20 mm. The measurements were repeated three times for each level of indentation. From the 3D scans, we computed the shear strain map and its zero-crossing by tracking the markers from two frames incrementally distant from each other. The incremental
components, (u, v, w), were then computed over the whole scan area using thin-plate spline interpolation. The shear strain was then estimated by differentiating the interpolated profile.

**Mechanical blurring.** The shear stresses caused by concentrated pressure P = 1 and concentrated traction Q = 1 constant along y and applied at x = 0 on a linear elastic half space, with z pointing inside it, are³⁸,

$$ \tau_0^p = -\frac{2}{\pi} \frac{xz^2}{(x^2 + z^2)^2}, \quad \tau_0^q = -\frac{2}{\pi} \frac{x^2z}{(x^2 + z^2)^2}. \qquad (4) $$

A contact with distributed pressure p(x) and traction q(x) can be viewed as an infinite sum of concentrated contacts³⁸,

$$ \tau = \int_{-\infty}^{+\infty} p(s) \tau_0^p(x - s, z) ds + \int_{-\infty}^{+\infty} q(s) \tau_0^q(x - s, z) ds, $$

which can be rewritten using the convolution symbol,

$$ \tau = [p * \tau_0^p + q * \tau_0^q](x, z). \qquad (5) $$

Since we are interested in the character of shearing stress near the contact surface, we can expand the components of shear stress as ‘moment asymptotic expansion’ in the limit $1/z \to \infty$ up to the first order⁵⁷⁻⁵⁹. Posing $f(s) = -\frac{2}{\pi} \frac{s}{(1+s^2)^2}$,

$$ \tau_0^p = \frac{1}{z} f\left(\frac{x}{z}\right) - \frac{1}{z} [M_0 z \delta(x) - M_1 z^2 \delta'(x)], $$

$$ \tau_0^q = \frac{1}{z} \frac{x}{z} f\left(\frac{x}{z}\right) - \frac{1}{z} [M_1 z \delta(x) - M_2 z^2 \delta'(x)], $$

where

$$ M_n = \int_{-\infty}^{+\infty} s^n f(s) ds. $$

Since the integrands of $M_0$ and $M_2$ are odd functions, these moments are equal to zero. Thus, in the limit when $z \to 0$,

$$ \tau_0^p \approx z \delta'(x), $$

$$ \tau_0^q \approx -\delta(x), $$

since $M_1 = -1$. Note that $\tau_0^p$ is the function $zL'_z(x)$ and $\tau_0^q$ the function $-L_z(x) + (xL_z(x))'$, where $L_z$ is the Lorentzian function of scale parameter z that converges to the delta function as $z \to 0$. To make the blurring nature of contact mechanics explicit, take the case of Gaussian pressure and traction of the form $\phi_\epsilon(x) = \frac{1}{\epsilon\sqrt{2\pi}} \exp(-\frac{x^2}{2\epsilon^2})$, where $\epsilon$ is the width of the Gaussians,

$$ \tau_\epsilon = [\phi_\epsilon * \tau_0^p + \phi_\epsilon * \tau_0^q](x, z), $$

thus,

$$ \tau_\epsilon \approx z \left( \phi_\epsilon * \frac{d}{dx} \delta \right) (x) - (\phi_\epsilon * \delta)(x), $$

and by the convolution sifting property,

$$ \tau_\epsilon^p \approx z \frac{d}{dx} \phi_\epsilon(x), $$

$$ \tau_\epsilon^q \approx -\phi_\epsilon(x), $$

since $\tau_0^p \approx (\phi_\epsilon * \tau_0^p) \approx \tau_\epsilon^p$ and $\tau_0^q \approx (\phi_\epsilon * \tau_0^q) \approx \tau_\epsilon^q$ when $\epsilon \to 0$, then from (5), we can express shear stress in terms of Gaussian functions,

$$ \tau \approx p * z \frac{d}{dx} \phi_\epsilon + q * (-\phi_\epsilon) $$

$$ \approx z \frac{d}{dx} (p * \phi_\epsilon)(x) - (q * \phi_\epsilon)(x). $$

From Hooke’s law applied to an incompressible medium, when z is small the shear strain is,
$$ \gamma \approx \frac{3}{E} \left[ z \frac{d}{dx} (p * \phi_{\epsilon})(x) - (q * \phi_{\epsilon})(x) \right]. \qquad (6) $$

**Blur Size.** To evaluate the size of the spread function in the case of a concentrated load as a function of the depth, z, we compare the Taylor series expansions of the exact expression of shear stress in the limit when $x \ll z$. From (4),

$$ \tau_0^p = -\frac{2}{\pi} \frac{x}{z^2} + \dots, $$

with its blurring counterpart (6),

$$ \tau_{\epsilon}^{p} = - \frac{xz}{\epsilon^{3} \sqrt{2\pi}} + \dots, $$

which gives by identification, $\epsilon = z \sqrt{\pi/8}$. Similarly, the scale of the Gaussian blurring due to a sharp corner can be found by the Taylor expansion of the shear stress expression of a triangularly distributed pressure, $p(x) = 1/a^2(a - |x|)$, for $-a \le x \le a$ and $p(x) = 0$ elsewhere,

$$ \tau_a^p = -\frac{2}{\pi} \frac{x}{z^2} \left( 1 + \frac{a^2}{z^2} \right)^{-1} + \dots, $$

which indicates that blur is both a function of z and $a: \epsilon \approx z(1 + a^2/3z^2) \sqrt{\pi/8}$, when $a \ll z$.

**Pressure-traction equivalence.** From (6), there is a relationship between pressure and traction,

$$ \gamma^q = -z p'(x) \approx \gamma^p. $$

**Displacements.** We can express the contact mechanics in terms of displacements since, during interaction with solids, tactile stimuli are specified this way. From elasticity theory³⁸, with incompressible media of elastic modulus, E, surface pressure and tractions are related to tangential, u, and normal, w, surface displacements by,

$$ \int_{-\infty}^{\infty} \frac{p(s)}{x-s} ds = -kw'(x), \quad \text{and} \quad \int_{-\infty}^{\infty} \frac{q(s)}{x-s} ds = -ku'(x), $$

with $k = 2\pi E/3$. Introducing the reciprocal function, $r(s) = 1/s$, these equations can be rewritten as,

$$ p * r = -k \frac{dw}{dx}, \quad \text{and} \quad q * r = -k \frac{du}{dx}. $$

The convolution inverse of the reciprocal function being $-r(x)/\pi^2$, we can write,

$$ p = \frac{k}{\pi^2} \left( \frac{dw}{dx} * r \right), \quad \text{and} \quad q = \frac{k}{\pi^2} \left( \frac{du}{dx} * r \right), $$

From (6), we have as $z \to 0$,

$$ \gamma \approx \frac{2}{\pi} \left\{ \left[ z \frac{d^2}{dx^2} (w * \phi_{\epsilon}) - \frac{d}{dx} (u * \phi_{\epsilon}) \right] * r \right\}, \qquad (7) $$

from the differentiation property of convolution. We note that for a uniform tangential displacement, $u'(x) = 0$, the zero-crossing of the shear stress, $\{(x, z): \gamma(x, z) = 0, \partial_x \gamma(x, z) \neq 0\}$, is given at a given depth, z, by the solution of

$$ \frac{d^2}{dx^2} (w * \phi_{\epsilon}) = 0. $$

This condition resembles the computation performs in classic visual edge detection techniques, where an edge is detected by computing the zero-crossing of the Laplacian of Gaussian of an image intensity³⁷.

## References

1. Koenderink, J. J. & van Doorn, A. J. Representation of local geometry in the visual system. *Biol. Cybern.* **55**, 367–375 (1987).

2. Adelson, E. H. & Bergen, J. R. The plenoptic function and the elements of early vision in *Computational Models of Visual Processing* (eds. Landy, M. & Movshon, J. A.) 3–20 (MIT Press, Cambridge, MA, 1991).

3. Howe, C. Q. & Purves, D. In *Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics* 1–14 (Springer, New York, 2005).

4. Horn, B. In *Robot Vision* 1–17 (MIT press, Cambridge, MA, 1986).

5. Connor, C. E., Brincat, S. L. & Pasupathy, A. Transformation of shape information in the ventral pathway. *Curr. Opin. Neurobiol.* **17**, 140–147 (2007).

6. Orban, G. A. The extraction of 3d shape in the visual system of human and nonhuman primates. *Annu. Rev. Neurosci.* **34**, 361–388 (2011).
7. Hayward, V. Is there a plenhaptic function? *Phil. Trans. R. Soc. B* **366**, 3115–3122 (2011).

8. Marr, D. In *Vision: A Computational Investigation into the Human Representation and Processing of Visual Information* 41–98 (Freeman, San Francisco, 1982).

9. Koenderink, J. J. The structure of images. *Biol. Cybern.* **50**, 363–370 (1984).

10. Lindeberg, T. Scale-space in *Encyclopedia of Computer Science and Engineering* (ed. Wah, B.) 2495–2504 (Wiley, Hoboken, 2009).

11. Palmer, S. In *Vision Science: Photons to Phenomenology* 180–181 (MIT press, Cambridge, MA, 1999).

12. Rolls, E. T. & Deco, G. In *Computational Neuroscience of Vision*. 21–28 (Oxford university press, Oxford, 2002).

13. Lowe, D. G. Object recognition from local scale-invariant features. *Proc. IEEE ICCV* **2**, 1150–1157 (1999).

14. Bensmaia, S. J., Denchev, P. V., Dammann, J. F., Craig, J. C. & Hsiao, S. S. The representation of stimulus orientation in the early stages of somatosensory processing. *J. Neurosci.* **28**, 776–786 (2008).

15. Plaisier, M. A., Bergmann Tiest, W. M. & Kappers, A. M. Salient features in 3-d haptic shape perception. *Atten. Percept. Psychophys.* **71**, 421–430 (2009).

16. Briot, M., Renaud, M. & Stojilkovic, Z. Approach to spatial pattern-recognition of solid objects. *IEEE Trans. Syst., Man, Cybern.* **8**, 690–694 (1978).

17. Hillis, W. D. A high-resolution imaging touch sensor. *Int. J. Robot. Res.* **1**, 33–44 (1982).

18. Harmon, L. R. Automated tactile sensing. *Int. J. Robot. Res.* **1**, 3–32 (1982).

19. Nicholls, H. R. & Lee, M. H. A survey of robot tactile sensing technology. *Int. J. Robot. Res.* **8**, 3–30 (1989).

20. Lee, M. H. & Nicholls, H. R. Tactile sensing for mechatronics-a state of the art survey. *Mechatronics* **9**, 1–31 (1999).

21. Cutkosky, M. R., Howe, R. D. & Provancher, W. R. Force and tactile sensors in *Springer Handbook of Robotics* Part C.19 (eds. Siciliano, B. & Kathib, O.) 455–476 (Springer, Heidelberg, 2008).

22. Dahiya, R. S., Metta, G., Valle, M. & Sandini, G. Tactile sensing-from humans to humanoids. *IEEE Trans. Robot.* **26**, 1–20 (2010).

23. Lucarotti, C., Oddo, C. M., Vitiello, N. & Carrozza, M. C. Synthetic and bio-artificial tactile sensing: a review. *Sensors* **13**, 1435–1466 (2013).

24. Hammock, M. L., Chortos, A., Tee, B. C. K., Tok, J. B. H. & Bao, Z. 25th Anniversary Article: The Evolution of Electronic Skin (E-Skin): A Brief History, Design Considerations, and Recent Progress. *Adv. Mater.* **25**, 5997–6038 (2013).

25. Phillips, J. R. & Johnson, K. O. Tactile spatial resolution. II A continuum mechanics model of skin predicting mechanoreceptor responses to bars, edges, and gratings. *J. Neurophysiol.* **46**, 1204–1225 (1981).

26. Phillips, J. R. & Johnson, K. O. Tactile spatial resolution. III A continuum mechanics model of skin predicting mechanoreceptor responses to bars, edges, and gratings. *J. Neurophysiol.* **46**, 1204–1225 (1981).

27. Sripati, A. P., Bensmaia, S. J. & Johnson, K. O. A continuum mechanical model of mechanoreceptive afferent responses to indented spatial patterns. *J. Neurophysiol.* **95**, 3852–3864 (2006).

28. Gerling, G. J., Rivest, I. I., Lesniak, D. R., Scanlon, J. R. & Wan, L. Validating a population model of tactile mechanotransduction of slowly adapting type I afferents at levels of skin mechanics, single-unit response and psychophysics. *IEEE Trans. Haptics* **7**, 216–228 (2014).

29. Brown, E. et al. Universal robotic gripper based on the jamming of granular material. *Proc. Natl. Acad. Sci. USA* **107**, 18809–18814 (2010).

30. Amend, J., Brown, E., Rodenberg, N., Jaeger, H. & Lipson, H. A positive pressure universal gripper based on the jamming of granular material. *IEEE Trans. Robot.* **28**, 341–350 (2012).

31. Howe, R. Tactile sensing and control of robotic manipulation. *Adv. Robot.* **8**, 245–261 (1993).

32. Ricker, S. L. & Ellis, R. E. 2-d finite-element models of tactile sensors. *Proc. IEEE ICRA* 941–947 (1993).

33. Wang, Q. & Hayward, V. Tactile synthesis and perceptual inverse problems seen from the viewpoint of contact mechanics. *ACM Trans. Appl. Percept.* **5**, 1–19 (2008).

34. Witkin, A. P. Scale-space filtering: A new approach to multi-scale description. *Proc. IEEE ICASSP* **9**, 150–153 (1984).

35. Hayward, V. Haptic shape cues, invariants, priors and interface design in *Human Haptic Perception: Basics and Applications* (ed. Grunwald, M.) 381–392 (Birkhäuser, Basel, 2008).

36. Li, R. & Adelson, E. H. Sensing and recognizing surface textures using a gelsight sensor. *Proc. IEEE CVPR* 1241–1247 (2013).

37. Marr, D. & Hildreth, E. Theory of edge detection. *Proc. R. Soc. Lond. B* **207**, 187–217 (1980).

38. Johnson, K. L. In *Contact Mechanics* 11–29 (Cambridge University Press, Cambridge, 1985).

39. Yuille, A. L. & Poggio, T. A. Fingerprints theorems for zero crossings. *J. Opt. Soc. Am.* **2**, 683–692 (1985).

40. Fearing, R. S. & Hollerbach, J. M. Basic solid mechanics for tactile sensing. *Int. J. Robot. Res.* **4**, 40–54 (1985).

41. Ferrier, N. J. & Brockett, R. W. Reconstructing the shape of a deformable membrane from image data. *Int. J. Robot. Res.* **19**, 795–816 (2000).

42. Chorley, C., Melhuish, C., Pipe, T. & Rossiter, J. Tactile edge detection. *IEEE Sensors* 2593–2598 (2010).

43. Sato, K., Kamiyama, K., Kawakami, N. & Tachi, S. Finger-shaped gelforce: sensor for measuring surface traction fields for robotic hand. *IEEE Trans. Haptics* **3**, 37–47 (2010).

44. Johnson, M. K. & Adelson, E. H. Retrographic sensing for the measurement of surface texture and shape. *Proc. IEEE CVPR* 1070–1077 (2009).

45. Hayward, V. A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. *Brain Res. Bull.* **75**, 742–752 (2008).

46. Levesque, V., Pasquero, J., Hayward, V. & Legault, M. Display of virtual Braille dots by lateral skin deformation: Feasibility study. *ACM Trans. Appl. Percept.* **2**, 132–149 (2005).

47. Hayward, V. & Cruz-Hernandez, M. Tactile display device using distributed lateral skin stretch. *Proc. HAPTICS* **69**, 1309–1314 (2000).

48. Nakatani, M., Howe, R. D. & Tachi, S. Surface texture can bias tactile form perception. *Exp. Brain Res.* **208**, 151–156 (2011).

49. Nakatani, M., Howe, R. D. & Tachi, S. The fishbone tactile illusion. *Eurohaptics.* 69–73 (2006).

50. Nakatani, M., Sato, A., Tachi, S. & Hayward, V. Tactile illusion caused by tangential skin strain and analysis in terms of skin deformation in *Haptics: Perception, Devices and Scenarios* Vol. 5024 (ed. Ferre, M.) 229–237 (Springer, Heidelberg, 2008).

51. Kikkuwe, R., Sano, A., Mochiyama, H., Takesue, N. & Fujimoto, H. Enhancing haptic detection of surface undulation. *ACM Trans. Appl. Percept.* **2**, 46–67 (2005).

52. Handzic, I. & Reed, K. B. The 'chopstick' illusion: A simply demonstrated tactile illusion. *Proc. HAPTICS* 551–555 (2014).

53. Hsiao, S., Johnson, K. O. & Yoshioka, T. Processing of tactile information in the primate brain in *Handbook of Psychology* (ed. Weiner, I. B.) Ch. 8, 211–236 (Wiley, 2003).

54. Pruszynski, J. A. & Johansson, R. S. Edge-orientation processing in first-order tactile neurons. *Nat. Neurosci.* **17**, 1404–1409 (2014).

55. Jörntell, H. et al. Segregation of tactile input features in neurons of the cuneate nucleus. *Neuron* **83**, 1–9 (2014).

56. Torre, V. & Poggio, T. A. On edge detection. *IEEE Trans. Pattern Anal. Mach. Intell.* **3**, 147–163 (1986).

57. Argatov, I. I. The pressure of a punch in the form of an elliptic paraboloid on a thin elastic layer. *Acta Mech.* **180**, 221–232 (2005).

58. Estrada, R. & Kanwal, R. P. A distributional theory for asymptotic expansions. *Proc. R. Soc. Lond. A* **428**, 399–430 (1990).

59. Lindell, I. V. Delta function expansions, complex delta functions and the steepest descent method. *Am. J. Phys.* **61**, 438–442 (1992).

60. Hsiao, S. Central mechanisms of tactile shape perception. *Curr. Opin. Neurobiol.* **18**, 418–424 (2008).

61. Speeter, T. H. Three-dimensional finite element analysis of elastic continua for tactile sensing. *Int. J. Robot. Res.* **11**, 1–19 (1992).
Acknowledgements

We thank Masashi Nakatani, Abhishek Srivastava, Arthur Szlam, Tiberiu Tesileanu, and the members of the
Cornell Creative Machines Lab for insightful discussions. We thank Sam McKenzie for helpful comments on
earlier versions of the manuscript. This work was supported by the European Research Council (FP7 Program)
ERC Advanced Grant (PATCH) to V.H. (No. 247300) and a Core Fulbright Visiting Scholar Program fellowship
to J.P.

Author Contributions

J.P. and V.H. developed the theory. J.P., H.L. and V.H. conceived the experiment. J.P. conducted the experiment and analysed the results. J.P. and V.H. prepared the manuscript. All authors discussed the results and reviewed the manuscript.

**Additional Information**

**Supplementary information** accompanies this paper at http://www.nature.com/srep

**Competing financial interests:** The authors declare no competing financial interests.

**How to cite this article:** Platkiewicz, J. et al. Haptic Edge Detection Through Shear. *Sci. Rep.* **6**, 23551 (doi: 10.1038/srep23551 (2016)).

<figure><img src="image_14.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License. The images
or other third party material in this article are included in the article's Creative Commons license,

unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license,
users will need to obtain permission from the license holder to reproduce the material. To view a copy of this
license, visit http://creativecommons.org/licenses/by/4.0/