NISTIR 7993

A Roadmap to Progress Measurement Science 

in Robot Dexterity and Manipulation 

Joe Falco

Jeremy Marvel

Elena Messina

http://dx.doi.org/10.6028/NIST.IR.7993
NISTIR 7993

A Roadmap to Progress Measurement Science 

in Robot Dexterity and Manipulation

Joe Falco

Jeremy Marvel

Elena Messina

Intelligent Systems Division

Engineering Laboratory 

http://dx.doi.org/10.6028/NIST.IR.7993 

March 2014 

U.S. Department of Commerce 

Penny Pritzker, Secretary

National Institute of Standards and Technology 

Patrick D. Gallagher, Under Secretary of Commerce for Standards and Technology and Director 
1

1. Introduction

The shifting global economy has driven considerable change in the way industry produces and delivers goods. 

As manufacturing philosophies adopt increasingly lean and agile solutions to adapt to changes in the 

marketplace, technologies must also evolve to meet the demands on production. Traditional automation 

solutions are limited in capability, and changes in products or processes generally equate to expensive retrofits 

and downtime. Robotic manipulation promises to help overcome these barriers, but the current technologies are 

limited in utility. Human-like perception and dexterity have been identified as critical capabilities for the future 

of robotics [1]. Dexterous manipulation has been defined as “an area of robotics in which multiple 

manipulators, or fingers, cooperate to grasp and manipulate objects” [2]. This definition is limited, however, in 

that it does not enable us to qualify or quantify dexterity as it applies to robotics. Nor does this definition 

capture the benefits or disadvantages of increasing the dexterity of robot systems. 

This roadmap highlights potential long- and short-term work to progress measurement science in the area of 

dexterous manipulation. The roadmap was developed based on a National Institute of Standards and 

Technology (NIST) review of industrial manipulation technologies [3], information gathered during a NIST 

organized workshop on dexterous manipulation for manufacturing applications [4], and the Roadmap for U.S. 

Robotics [1]. The NIST review of industrial manipulation technologies found significant research in the area of 

human-like dexterous robots for assembly applications and robotic hands that is forming the foundation for the 

next generation of robot gripper technologies. Discussions during the NIST workshop identified the need for 

performance metrics and test methods for emerging robotic hand and arm technologies, as well as for integrated 

systems consisting of both arms and hands. The U.S. Robotics Roadmap highlights robotic hands and tactile 

sensing as important enablers for autonomous manipulation and specifies a roadmap for human-like dexterous 

manipulation as: 

• 5 years: Low-complexity hands with small numbers of independent joints will be capable of robust 

whole-hand grasp acquisition. 

• 10 years: Medium-complexity hands with ten or more independent joints and novel mechanisms 

and actuators will be capable of whole-hand grasp acquisition and limited dexterous manipulation. 

• 15 years: High-complexity hands with tactile array densities, approaching that of humans and 

with superior dynamic performance, will be capable of robust whole-hand grasp acquisition and 

dexterous manipulation of objects found in manufacturing environments used by human workers. 

Preliminary sets of performance metrics, artifacts, and test methods were identified as useful tools to advancing 

the development of robotics components—such as robotic hands—in the short-term. Performance measures for 

dexterity are dependent on the integrated system, and in the long-term tests will be needed to assess the 

dexterity of robotic systems, such as arm/hand combinations in application scenarios defined with the help of 

industry. Participants in the NIST workshop agreed that dexterous robotics will continue to be a topic of active 

research. This is partly motivated by manufacturer’s requirements for greater safety of humans on the factory 

floor, improved quality control, reduction of costs, and increased throughput. Additional concepts discussed in 

the NIST workshop that could be employed for advancing the state of dexterous manipulations include: 
2

• The use of competitions, along the lines of those used by the Defense Advance Research Project Agency 

(DARPA), such as for autonomous driving and disaster response. A significant hurdle is the broad range 

of tasks and parts that require dexterous manipulation – anything from handling highly-sculpted 

consumer products to aircraft assembly. This breadth makes it impossible to create a single abstract 

challenge problem. 

• The use of crowd-sourcing to leverage creativity. This would require the hardware platforms to be more 

readily accessible (economically, physically) than they currently are. Hardware and software standards 

for interconnecting the components would be key requirements. 

• Having mechanisms available to facilitate transferring results from research institutions to applied 

manufacturing applications. These could include testbeds, standard test methods, or other means of 

validating the innovative solutions to make sure they meet manufacturer’s requirements.

 

2. Robotic Hand Grasping and In-Hand Manipulation

Commercially available robotic hands are beginning to make their way into manufacturing environments. With 

the onset of affordable rapid prototyping technologies, many novel designs are emerging from academia. The 

hands used by industry have low numbers of joints and fingers (typically 3 fingers), while academic prototypes 

are more anthropomorphic, and may feature as many as 5 fingers. In addition, tactile sensing technologies are 

progressing beyond the traditional matrix-based tactile sensor pad which typically has limited resolution and 

can only sense forces and pressures normal to the sensor’s surface. Many of the workshop participants noted 

that, currently, research approaches to grasp planning for robotic hands are complex and time consuming, and 

that the problem of grasping is not independent of object perception. 

Challenges of implementing robotic hands include weight reduction to improve the robot’s payload limitations 

without impacting the hand’s ability to securely hold a work piece, and the use of modularity to increase 

flexibility toward handling different classes of part geometry. Performance requirements are described as 

position and force control as well as actuation speed. In addition, robot hands need to be energy efficient, 

adaptable, compact, commercially accepted for the factory environment, and safe. Sensors must be integrated 

to offset limitations in mechanical repeatability.

The NIST workshop identified the need for benchmarking dexterity, similar to the Southampton Hand 

Assessment Procedure (SHAP) that was developed as a means to measure the performance of upper limb 

prostheses [5]. SHAP defines 26 tasks using 8 abstract objects and 14 activities to be performed by a subject 

fitted with a prosthetic device. Measurements of success and speed are used to calculate overall performance of 

prostheses. The procedures discriminate between functional and force limitations. 

The sense of touch is the primary sense for human dexterity. Experimentation demonstrated that the absence of 

tactile feedback was severely detrimental to manipulation tasks, while visual feedback was able to compensate 

only marginally [6]. One goal of robotic grasping systems is to mimic the human capability of reactive 
3

grasping, the ability to detect object size and shape without the use of vision. This will require sensor systems 

with higher resolution and more sensing capabilities than the tactile sensors that are currently available. In 

particular, it is believed that future sensors should model all three cutaneous touch sensory modalities of the 

human: response to normal and shear forces, vibrations, and temperatures. 

Tactile sensing technologies could greatly improve grasp flexibility. The costs associated with such 

improvements will most likely be small when comparing those of sensing electronics to flexibility 

improvements using mechanical components. As the quest towards grasp flexibility progresses, new 

technologies in tactile sensing will become available. NIST must ensure that performance measures are 

available to support these new sensing technologies as well as the robotic hands that are equipped with them. 

3. Robot Arm Manipulation

A limiting factor in the implementation of robot assembly cells is the dexterity of the manipulator and end 

effector. Though the utility of a standard 6 degrees of freedom (DOF) robot arm with a traditional parallel jaw 

gripper or suction device has been demonstrated through decades of implementation, the actual number of tasks 

that can be performed by such configurations is limited. Assembly configurations must be designed with the 

limitations of the robot in mind. Such restricting factors include the robot’s reach, lift capacity, sensing 

capabilities, and dexterity.

A new class of robots is now commercially available that are lightweight and are highly dexterous with 7 or 

more DOF. These new robots often contain joint level force sensing capabilities or naturally compliant joint 

motors, and are sometimes paired in two-arm torso mounted configurations. Arms with 7 DOF provide a 

significant advantage over traditional arm designs having fewer degrees of freedom, and represent a 

breakthrough in dexterity. Multiple arms can be controlled separately or jointly, depending on the task 

requirements. The coordination of multiple arms allows for greater dexterous manipulation capabilities even 

with the use of conventional grippers. 

Mechanical assembly was expected by many to be the dominant application domain of industrial robots; yet 

assembly tasks are still primarily addressed by manual labor. New advances in force control, machine vision, 

and robot dexterity are enabling industrial robots to catch up with the original visions of robotic assembly [7]. 

As these technologies progress, performance metrics and associated test methodologies are needed to unify 

research efforts, characterize the state of technology, and provide a means for end-users to evaluate the 

capabilities of robotic assembly systems.
4

4. Proposed NIST Roadmap for Dexterous Manipulation

As part of the development of a dexterous manipulation roadmap, NIST has identified a series of near- and 

long-term topics that need to be addressed. The term “dexterity” itself must be fully and formally defined, 

along with its constituent functional capabilities. Examples of functional capabilities range from simple tactile 

sensing (contact achieved or not) to force-guided insertion of parts for snap-fit assemblies. Performance profiles 

and requirements within the constituent capabilities will vary depending on the manufacturing operations that 

the robots will be performing, as well as the environmental conditions. Hence, part of the research effort will 

include developing an understanding of the operational scenarios and how they influence the performance 

requirements.

The impact areas focus on several aspects of dexterous arm and hand performance, including sensing, motion, 

control, and applications. 

4.1. Short-term (1-3 year) impact areas

• Informal industry working groups (IIWG) to address initial measurement needs and develop a 

classification of robotic hand capabilities to match associated performance measures.

• With IIWG and others, define common vocabulary pertaining to dexterity and manipulation and 

taxonomy of manufacturing operations that require dexterity and/or advanced manipulation 

capabilities.

• Metrics, artifacts, and test methods for evaluating the performance of commercially available three 

fingered robotic hands based on known part location and grasp points. These metrics, methods, and 

artifacts will be used to measure basic grasp and in-hand manipulation characteristics, and will 

include metrics for assessing tactile sensing capabilities. These test methods will address at least 

some level of the following capabilities:

o Hand Mechanics

 Position control 

 Torque control of fingers/digits 

 Grasp capacity (e.g., graspable object size and mass) 

 Grasp types supported 

 Accuracy

 Repeatability

o Tactile Sensing: 

 Normal forces and pressure

 Force and impact sensitivity

 Location of touch 

o Functional Tasks 
5

 Quasi-static and dynamic effects on grasp stability

 In-hand manipulation of objects 

 Touch sensitivity (e.g., using touch to control finger position/force) 

• Metrics, artifacts and test methods for evaluating the force control characteristics as well as the 

performance of robotic assembly operations utilizing [7] as a basis for this work. Work may also 

include functionality testing to accomplish integrated tasks such as the use tools for fastening or 

drilling.

• Explore the use of metrics, artifacts and test methods for evaluating the dexterity of robotic arms. 

These include:

o Reachable volume (i.e., the positions and orientations that an arm can achieve within the 

workspace)

o Operational space (i.e., the positions and orientations in which the arm and/or hand can 

effectively perform the required operation. 

o Confined space access

o Grasping objects while in motion 

o Multi-arm cooperative grasping 

• The test methods within this time frame will focus on fairly structured environments and lowcomplexity manufacturing operations and geometries. Therefore, tests will assume nearly 

standalone parts, minimal clutter, controlled lighting and other environmental conditions, and static 

conditions (e.g., no picking from a conveyor belt). 

4.2. Long-term (4-5 year) impact areas 

• Expand measurement science to address grasping capabilities and closed-loop control as robot hand 

technology advances from low complexity to medium and high complexity as specified in the U.S. 

Robotics Technology Roadmap [1]. 

o Reactive gripping 

o Agility/flexibility/versatility: Devise means of characterizing the range of parts that can be 

grasped and manipulated and the range of manufacturing operations that can be performed 

with the same hand. 

• Metrics, artifacts, and test methods for evaluating the performance of robotic hands and dexterous 

arms.
6

• Work with tactile sensor developers to help them understand the measurement, reliability, and 

robustness requirements associated with industrial robot applications, and to explore new 

measurements related to advancing tactile sensing capabilities such as sensing shear forces, 

temperature, and vibrations. 

• Work with industry to develop application-based tests to assess the dexterous capabilities of robotic 

systems such as arm/hand combinations. For example, tests may involve complex assembly 

artifacts, such as a kit of parts to be assembled, where performance is assessed over a sequenced set 

of assembly operations. Considerations on operations could include, speed to completion, ease of 

programming, robustness to variability. Candidate operations may also be used to assess the 

performance of external perception systems, such as vision to identify and locate parts. 

• In this longer-range timeframe, test methods will be extended to cover more complex and 

unstructured environments and tasks, including cluttered, mixed parts that may be moving. 

5. Conclusion

In order for robots to expand their applicability within manufacturing operations, their dexterity needs to 

advance considerably. There is active research in many areas that is contributing to creating a new generation 

of much more dexterous arms and hands. A complementary measurement science research effort, which 

defines the metrics, artifacts, and test methods, is needed to characterize and understand performance of new 

capabilities. In this document, a high-level roadmap is proposed for the development of the measurement 

science to help advance dexterous manipulation for robots in manufacturing applications. 

6. References

[1] A Roadmap for U.S. Robotics: from Internet to Robotics, 2013 Edition, http://roboticsvo.us/sites/default/files/2013%20Robotics%20Roadmap-rs.pdf 

[2] Okamura, Smaby, Cutkosky, An overview of dexterous manipulation, Proceedings of the IEEE 

International Conference on Robotics and Automation, 2000, pp 255-262 

[3] Bostelman, Falco, Survey of Industrial Manipulation Technologies for Autonomous Assembly 

Applications, NISTIR 7844, 

[4] Falco, Marvel, Messina, Dexterous Manipulation for Manufacturing Applications Workshop, NISTIR 

7940, 2013. 

[5] http://www.shap.ecs.soton.ac.uk/ 

[6] Venkadesan, Guckenheimer, Valero-Cuevas, Manipulating the Edge of Instability, Journal of 

BioMechanics, 2007, pp. 1653-1661 

[7] Marvel, Falco, Best Practices and Performance Metrics Using Force Control for Robotic Assembly, 

NISTIR 7901, 2012.