From Swarm Robotics to Smart
Materials
Nikolaus Correll
University of Colorado at Boulder

The ultimate robotic swarm:
a liquid that thinks

Subsets of Liquids that thinkConstruction

Distributed Computing

Appearance ChangeIntelligent function

Collective Motion

Viscoelasiticy

Making a liquid that thinks“Droplets”“Liquid that thinks”

Liquid that thinks

Useful novel
materials?

“Droplets”“Liquid that thinks”

Background: Materials that think• Embedded…
– Actuation– Sensing– Computation– Communication

Distributed MEMS (Berlin, 1997)• Periodic, amorphous

Smart Dust (Pister, 1999)

Amorphous Computing (Abelson, 2000)
Programmable Matter (Goldstein, 2005)
6

Enablers• Cheap sensing,computation, actuation• Cheap manufacturing• Available polymers

Materials that Sense• Prosthetic / augmented limbs• Improved situationalawareness for robots• More subtle Human / RobotInteraction• Structural monitoringMaterials that change Appearance• Smart facades• Camouflage• Smart clothes

Materials that change Shape• Reconfigurable airplanewings• Reconfigurable
aerodynamic structures• Reconfigurable architecture• Reconfigurable furniture

New forms of artistic expression

K. Sugawara, K. Hata, N. Correll, M. Theodore (2013): 群れ行動の数理モデルとその応用～ 群
れお絵かきツールとSwarm Wall ～. 1st International Conference on Human Agent Interaction(HAI), Sapporo, Japan, 2013.

Materials that Sense

Soft Sensing Skin

• Sensor network woven inrubber mesh
• Embedded in EcoFlex silicone
rubber• Surface textured using 60-grit
sandpaper during curing
D. Hughes, N. Correll (2014): A Soft, Amorphous Skin that can Sense and Localize Texture . IEEE
International Conference on Robotics and Automation (ICRA), Hong Kong, 2014.

Soft Sensing Skin
RubbedTextureSignal

Measured SignalMicrophoneSignal FFT
FFTAmbient
Update1000 Hz

Ambient Signal

Transient Signal

1. Texture localization

2. Texture Classification• Logistic regression• Classifier trained on 15
predefined textures• 128 inputs• 15 outputs• 1935 weights stored onboard

𝑦𝑦 = 𝑔𝑔 𝜃𝜃0 + 𝜃𝜃1 𝑓𝑓1 + 𝜃𝜃2 𝑓𝑓2 + ⋯ + 𝜃𝜃𝑛𝑛 𝑓𝑓𝑛𝑛

Brillo: 2.4%
Skin: 93.5%…

Classifier Training
abc

de

fg

hi

jkl

mnoa)
b)
c)
d)
e)
f)
g)
h)
i)
j)
k)
l)
m)
n)
o)

brillobrushcardboardcoarse wire meshcottondense foamfine wire meshplasticsandpaper
silicone foamskinsponge
terry cloth
textured siliconewood• Classifiers trained using 15 textures• 100 samples per texture• 10-fold cross-validation used to determine
accuracy

Classification Results Summary• Logistic Regression– Overall Classification
Accuracy:
71.2%– Total Number of Weights:
1,935• Possible improvements

Confusion Matrix– Multi-modal sensing– Joint classifiers– Better classifiers

Non-trivial, material-centric computation

Materials that Change AppearanceAppearance changing materials

Interacting with a Distributed SystemN. Farrow, N. Sivagnanadasan, N. Correll (2014): Gesture Based Distributed User Interaction System for a
Reconfigurable Self-Organizing Smart Wall. In: Proceedings of the 8th International Conference on Tangible,Embedded and Embodied Interaction (TEI), pp. 245-246 , ACM 2014.

Distributed Gesture Recognition

1. Sample

2. Dilate

3. Interpolate

4. Encodek-NN

Distribute Memory

5. Consensus“S”

with Rick Han

Distribute Computation

Accuracy vs. Packet loss
Distributed MemoryHybrid

Distributed Computation

Tight relationship between material and algorithm design trade-offs

Materials that Shape Change

Variable Stiffness Control
Stiffness control = shape control
Melting of PCL
2-200MPa change in Youngs’ M.
Local feedback temperaturecontrol• Global distributed shape control••••

M. A. McEvoy, N. Correll (2014): "Thermoplastic variable stiffness composites with embedded, networked
sensing, actuation, and control. In: Journal of Composite Materials, 2014.

25

Principle of Operation

Resulting Shape Change

M. A. McEvoy, N. Correll (2014): Shape Change Through Programmable Stiffness. International
Symposium on Experimental Robotics (ISER), Springer Verlag, Marrakech, Morocco, 2014.

Pneumatic Shape Change
Modeling• Geometry• Material properties• Pressure• Curvature• Resulting force

OutsideModular soft robotic actuator withembedded sensing and control

Contraction force

Curvature

InsideExpansion force

GeometryPneumatic Shape Change
Feedback control

Model prediction

Geometry 2

Curvature

Pressure

Geometry 1

TimePressure

Tight interaction between distributed control and material physics

Materials that Self-Assemble

Vision

1. Self-assembly

2. Cell differentiation

3. Collective behavior

DNA Tile Assembly Model• Rules to encode
neighborhood
relationships• Maximal N rules forstructures with N tiles• Many interesting
structures can be madewith fewer rules!
Yuriy Brun. Solving np-complete problems in the tile assembly model.
Theoretical Computer Science, 395(1):31–46, 2008.

Rules

ExampleRobotic Materials
Amorphous Computing

Material Physics

Robotic Materials: New challenges
for Education

A soft skin that cansense distance and force

Variable Stiffness by
Sheet Jamming
Computation X Materials

An interactive Sushi tray

Wireless data transmission
powered by Seebeck effect

Shortest path routing
in a distributed system

Shape-changing materials thatcan self-restore

Conclusion• Robotic Materials pose newopportunities and challenges in
distributed algorithms• Understanding the link between“crowd dynamics” and “materialphysics”• Timely problems:– Novel capabilities for robots– Novel materials with revolutionary
functionality for every-day applications

Acknowledgements