Abstract— Exoskeletons are now being marketed by several 

manufacturers and yet there are currently no standard test 

methods to help match exoskeletons1to desired tasks. The 

National Institute of Standards and Technology (NIST) has been 

a key contributor to the formation of a new ASTM F48 

standards committee on exoskeletons and has a project to 

research measurement science and test methods in support of 

exoskeleton standards. This paper describes the NIST 

exoskeleton project efforts for one of several ongoing research 

areas that target typical industrial tasks - i.e., load handling. The 

paper will describe the design of the NIST Position and Load 

Test Apparatus for Exoskeletons (PoLoTAE), a reconfigurable 

testbed and its use within a NIST human subject’s study. 

Experimental results from the first load handling test are 

described and future tests are outlined.

I. INTRODUCTION

Exoskeletons can be passive, with only springs and/or 

counterweight augmentation of human motion, or active, with 

motor augmentation - sometimes called wearable robots, or a 

combination of passive and active. In the early 2000’s, the 

U.S. Defense Advanced Research Project Agency (DARPA) 

began development and demonstration of “... critical 

technologies such as power, control, and actuation that will 

lead to a self-powered external structure to enable a Soldier to 

effortlessly carry over 45 kg (100 lbs) of additional weight 

[1].” Exoskeletons were recognized as potentially beneficial 

to many other domains: as of December 2016, [2] identified 

58 commercial and/or non-profit organizations that have 

developed or were developing exoskeletons or wearable 

robotics. Today, there are roughly 80 commercial systems on 

the market. 

Several events laying the foundation for standards have 

occurred since the DARPA developments, including a Round 

Table and several Technical Interchange Meetings, fostered by 

U.S. government organizations and with invitees from 

international government, industry, and academia [3]. One 

common theme that was discussed was the need for standard 

test methods for exoskeletons. To this end, an exploratory 

project within the NIST Engineering Laboratory (EL), began 

in September 2017 and cumulated into the ASTM 

International Committee F48 Exoskeletons and Exosuits 

(ASTM F48) [4]. Six F48 subcommittees were established: 

F48.01 Design and Manufacturing, F48.02 Human Factors and 

R. Bostelman is with the National Institute of Standards and Technology, 

Gaithersburg, MD 20899, USA (phone: 301-975-3426; e-mail: 

roger.bostelman@nist.gov). 

Ya-Shian Li-Baboud is with the National Institute of Standards and 

Technology, Gaithersburg, MD 20899, USA (e-mail: yashian.libaboud

@nist.gov). 

Ann Virts is with the National Institute of Standards and Technology, 

Gaithersburg, MD 20899, USA (e-mail: ann.virts@nist.gov). 

Soocheol Yoon is with the National Institute of Standards and 

Technology, Gaithersburg, MD 20899, USA (e-mail: soocheol.yoon

@nist.gov). 

Ergonomics, F48.03 Task Performance and Environmental 

Considerations, F48.04 Maintenance and Disposal, F48.05 

Security and Information Technology, and F48.91 

Terminology. An exoskeleton is currently defined by ASTM 

F48.91 as a “wearable device that augments, enables, assists, 

and/or enhances physical activity.” Two notes support the 

definition with 1) “an exoskeleton may include rigid and/or 

soft components (see exosuit)” and 2) “physical activity may 

be static or dynamic”. 

The NIST EL project also includes developments in two 

areas that have continued into a longer-term project beginning 

in fiscal year 2019: 1) measurement science towards 

development of new methods to measure the exoskeleton fit 

and movement of the exoskeleton to the user, and 2) the impact 

that wearing an exoskeleton has on the performance of users 

executing tasks that are representative of activities in industrial 

settings. Based on the United States Bureau of Labor statistics, 

in 2014 there were about 12,188,300 manufacturing jobs in the 

United States [5]. Approximately 5,086,905 of manufacturing 

employees belong to small and medium enterprises according 

to the US Census Bureau for 2012 [6]. With so many 

manufacturing jobs performed, including nearly half in the 

small and medium size organizations, even a small percentage 

of these jobs could benefit from using exoskeletons if they are 

safe and effective. As a result, exoskeleton standards are

necessary.

Initial NIST EL efforts to support exoskeleton standards 

include developing test methods from previous research areas 

such as response robots and autonomous industrial vehicles 

[7]. A NIST study, which began in June 2018 [8], is 

researching the two EL project areas described above. The 

results of the study will inform future test method development 

at NIST and at other organizations, all under the purview of 

ASTM F48. Area 1 has been completed for exoskeleton fit to 

the leg and analysis is underway. Within area 2, the focus of 

this paper, a reconfigurable Position and Load Test Apparatus 

for Exoskeletons (PoLoTAE) [9] was designed. Six test 

methods were designed to target industrial tasks where the first 

load positioning task will be the focus experiment and findings 

described in this paper. The outcome of the load handling tasks 

is also expected to provide support for development of ASTM 

Mili Shah is with The Cooper Union for the Advancement of Science and 

Art, Department of Mathematics, New York, NY 10003 (e-mail: 

mili@cooper.edu). Research for Shah is performed under grant

70NANB17H251 from U.S. Department of Commerce, National Institute of 

Standards and Technology. 

1 Disclaimer: Commercial equipment, software, and materials are 

identified in order to adequately specify certain procedures. In no case does 

such identification imply recommendation or endorsement by the National 

Institute of Standards and Technology, nor does it imply that the materials, 

equipment, or software are necessarily the best available for the purpose.

Towards Standard Exoskeleton Test Methods for Load Handling

Roger Bostelman, Ya-Shian Li-Baboud, Ann Virts, Soocheol Yoon, Mili Shah
subcommittee F48.03 work item WK65295 - Load Handling 

When Using an Exoskeleton.

II. TEST METHOD DESIGN

Repeatable test methods can help exoskeleton 

manufacturers and users highlight capabilities of their systems, 

compare exoskeletons to their motion tasks, show design flaws 

or enhancements, and help with procurement requirements. 

Ideally, these test methods are not only repeatable, but also 

standardized, such that both manufacturers and users can 

simply select a document that describes how to perform the 

test method no matter which exoskeleton they make or use. 

Although NIST has been focusing on industrial test methods, 

there is overlap with military, medical, response, and perhaps 

commercial exoskeletons. For example, military logistics 

personnel carry relatively heavy loads and, for example, mount 

wheels on military equipment. Example crossindustry/medical use of exoskeletons may be for nurses and 

orderly staff to pick-up and maneuver patients. Movement of 

an elderly person at home with the help of an exoskeleton

device would not be considered a medical application if the 

device is not prescribed by a medical doctor. 

In addition to supporting the development of ASTM F48, 

which includes all of the above applications, NIST began 

designing test methods that are representative of industrial 

tasks including: applying forces (e.g., grinding, sawing, 

drilling, pushing, etc.), inserting pegs (e.g., screw, drill bit, 

etc.) into holes, non-contact alignment (e.g., laser pointing, 

siting, etc.), and load handling (e.g., holding a load and 

aligning to a fixture, hanging a load on hooks, and 

placing/positioning loads). All of these tests are planned for 

the experiments within the NIST study. In this paper, we 

describe the study of the first of the six planned tasks – i.e., 

load positioning. 

Towards a standardized approach to a replicable apparatus 

that can be used by the exoskeleton manufacturers, users, and 

researchers, NIST designed the PoLoTAE and will provide the 

PoLoTAE design in a future NIST internal report [8]. The 

PoLoTAE, shown in Figure 1, was designed to be 

reconfigurable, minimal cost, and expandable to larger 

dimensions or to focus more specifically on the required 

exoskeleton tasks. Variability in generic loads/tools, 

positioning heights, defined motion spaces, etc. can also be 

added to the PoLoTAE to help exoskeleton designers fit the 

tasks to the potentially wide variety of exoskeleton wearers.

Loads can vary dramatically depending on the application. 

However, load handling includes not only picking up but also 

placing loads, sometimes with relative precision (e.g., 

installing a wheel on a vehicle axle bolt- circle). Ideally, a 

single, replicable artifact (e.g., the load artifact shown in 

Figure 1) is used for picking, placing, aligning, and hanging 

loads. 

Figure 2 a, b, and c show the series of load handling tasks 

that are planned for experimentation on at least 30 subjects

each. Figure 2 a and b show load hanging and load alignment 

tests planned for future experiments within the study and will 

not be further explained in this paper. Figure 2 c shows the 

load positioning test design setup which is the focus of the 

remainder of this paper. 

Figure 2 d shows the 6.8 kg (15 lbs) load artifact designed 

for use in all three load handling tests. For load positioning, 

the artifact base was approximately 3 mm smaller on all four 

sides than the tray-surround on each shelf. The surround 

ensures that the subject being tested must not only place the 

artifact in the correct location but must also position and seat 

the artifact correctly within the surround. Handles or other 

grasp methods can also be incorporated into the artifact.

 

a b

Figure 1. PoLoTAE showing inserts of the force plate 

with tool (top) and drill with bit in a pipe for peg-in-hole 

tests.

Hooks for load 

hanging test

Bolt circle for 

load alignment 

test

Shelves for load 

positioning test

Load handling 

artifact
c

d

Figure 2. PoLoTAE a) load hanging, b) load alignment, and 

c) load positioning test designs setup and numbered order of 

subject movements planned within the NIST study. d) 

Artifact used for all three load handling tasks.

III. LOAD POSITIONING EXPERIMENT

NIST currently owns a full-body, passively-mechanized

exoskeleton which uses springs internal to the frame. This 

exoskeleton will allow for the initial development of study 

procedures with the expectation that additional models will be 

procured or borrowed to validate and expand the tests on a 

variety of exoskeletons. The test methods described below are 

designed to exercise all aspects of a full-body exoskeleton

through timed video recording, so that the sub-tasks (e.g., 

lifting, placing, bending, etc.) can be separated out for 

individual review.

Subject recruitment included posting brochures requesting 

NIST federal employee volunteers that fit the profile required 

of the test and the exoskeleton to be used. In a similar test 

method development, EL researchers worked with the NIST 

Statistical Engineering Division to determine the number of 

tests to perform for statistical significance. The statisticians

determined that for 85% confidence, 30 repetitions were 

required. At least 30 subjects, each performing one of six tests, 

requires nearly 200 subjects. Also, the same subject can return 

for another future test. From a pilot study on a subject having 

average physicality, 30 repetitions of a task appeared to be a 

good limit to test fatigue without strain caused by the task 

motions.

The recruited subjects were:

• At least 18 years of age

• Physically fit to complete the test which relied on the 

subjects perspective of their own capability. 

• Willing to participate for up to 1 1⁄2 hours. 

• Able to wear an exoskeleton that weighs approx. 30.3 

lbs (13.7 kg). 

• Able to perform knee bends, position tools, and apply 

forces 30 times twice (60 total) using tools (up to the 

approx. weight of 30.3 lbs (13.7 kg)). 

• Able to fit within the exoskeleton manufacturers 

specification for height (5’ 0” to 6’ 1”) and weight 

(108 lbs to 225 lbs). 

The research study procedure included the following:

• the subject reviewed a training video [11] and signed

a consent form, 

• the subject wore (one or two) wrist heart rate (HR) 

monitor(s) and the research team recorded the 

subject’s HR throughout the test, 

• the subject first performed the baseline and then 

exoskeleton-use tasks, 30 times each, with help from 

the research team as needed to assist the subject with 

putting on/taking off the exoskeleton where the 

subject could stop the test at any time,

• the subject responded to a brief set of survey 

questions about the test upon its completion.

Repetition 1: The subject stepped one step forward from a 

start line to the load apparatus (refer to Figure 2c); the load 

apparatus was picked up from tray 1 (14 cm above the floor) 

and placed in tray 2 (138 cm above the floor); hands were

released; The load apparatus was picked up from tray 2 and 

placed in tray 3; hands were released and the subject stepped

back to the line. Repetition 2: The subject stepped from the 

line to the load apparatus; the load apparatus was picked up 

from tray 3 and placed in tray 4; hands were released; The 

load apparatus was picked up from tray 4 and placed in tray 

1; hands were released, and the subject stepped back to the 

line. Repetition 1 and 2 are repeated 14 more times for a total 

of 30 repetitions.

IV. SUBJECT SAFETY PROTECTION

Subject safety - including physical safety as well as social 

safety - is a priority during the NIST exoskeleton research. All 

procedures during the study were approved by the NIST 

Institutional Review Board. The research team strictly 

followed the Human Subject Research protocol [8] to keep the 

subject socially safe. In addition, the subject is prevented from 

physical injury due to instability and raised heart rate through 

continuous monitoring. 

To protect the subject’s personal information, the entire

test area is surrounded with curtains. A pre-test survey is 

conducted to check whether the subject has potential risks

(e.g., aches, pains, past broken bones, etc. that may affect their 

task completion). Many exoskeleton novices were unstable at 

first and were asked by the team to perform knee bends on soft 

safety mats to establish their stability. All wearable devices 

and parts were cleaned with sanitary wipes before and after the 

test, particularly for the components in contact with the body.

A. Heart Rate Monitoring

HR was monitored throughout the test. HR has been 

widely used as an indicator for response of heart to exercise 

Holes for load 

hanging – 2 on 

each end

Holes for load 

alignment
since it is easy to measure and directly related to the autonomic 

nervous system [12]. The tests require activity and as a result 

HR was expected to increase during the tests. The American 

Heart Association [13] method of subtracting the subject age 

from 220 beats per minute (bpm) or a particular HR that the 

subject desired was set as the HR limit. 

Wrist HR monitors, now widely used [14], were employed

to measure and record HR. The monitors use optical sensors to 

measure HR which are verified to have 80% to 91% accuracy 

depending on make, application, test conductor, and subject 

response [15][16][19]. The actual accuracy of the HR monitor 

varies according to the subject’s skin type, arm thickness, and 

shape. In addition, since the tasks require a wide range of 

motion, errors in HR measurement were primarily due to loss 

of steady attachment for durations up to 10 s.

To overcome this limitation, the procedure was revised 

during the experiment to include two steps to provide further 

corroboration of HR data. First, two HR monitors from 

different manufacturers were used. Both use optical sensors, 

but they have different sensor arrangements and straps. When 

they showed different HR values, the higher one was always 

accepted. Second, a hand grip monitoring bar, based on 

electrical conductance, was used to verify that the wrist HRs 

were working correctly. The bar uses electrodes to measure 

HR and was used as another technology method to help verify 

the optical HR monitoring method. Before the test, resting HR 

was measured, and the highest wrist HR was recorded after the 

test was verified with the HR bar. 

Wrist HR was measured and recorded at 1 s time intervals

using BlueTooth to offboard heart rate monitoring devices. 

Several HRs were marked at specific task repetitions such as

prior to beginning, after 15, and after 30 repetitions. The 

research team frequently asked subjects their condition when

their HR was elevated, e.g., above 150 bpm, and determined

whether to continue the test, rest, or stop the test.

V. DATA COLLECTION SYSTEM

The data collection system was comprised of four high 

definition (HD) camcorders mounted on tripods, a high 

definition multi-viewer, and an HD video capture device with 

external hard drive. The video was recorded at full HD 1080

pixel resolution. Figure 3 shows the output of the data 

collection system. Two videos of the output were recorded: 

the baseline test and the test when the subject wears the 

exoskeleton.

Figure 3. Snapshot of the data collection system display.

Frame A denotes (top) repetition number, (center) global 

positioning system time, temperature, date, and humidity, and

(left) a timer. For the first load positioning task test and if time 

were a critical metric, video review could have been used to 

determine repetition-completion count within a set time. 

Figure 3 shows the more recent data collection system with 

repetition counter and timer added for the second task (e.g., 

peg-in-hole task currently underway). Frame B shows the 

output of two HR monitors (left and right) as described in 

section IV. Frames C and D are left and right views of the task 

being performed by the subject. 

Digital photographs of the subject performing various 

portions of the task were also captured and added to the stored 

data. For example, the photos show snapshots of: 

• the technique that the subject uses to pick-up and place 

the load artifact (e.g., bend at the hips or bend at the 

knees), 

• the shelf height relative to the subject shoulder height

(i.e., potentially critical for short subjects and less 

critical for tall subjects), 

• and the change in the subject form from when first 

using the exoskeleton to final repetitions after the 

subject appears to be familiar with the exoskeleton.

VI. RESULTS

A. Use of heart rate data

HR was monitored to protect the subject and also used to 

evaluate the physical demands of the task test on the subject 

with and without the exoskeleton. It is difficult to conclude 

whether a subject’s HR response can be an indicator of an 

exoskeleton’s effectiveness in augmenting or supporting 

humans with the physical demands of the task. However, 

combining HR with other test data obtained including video 

and a survey, additional information may be derived. Usually 

HR becomes high at the beginning when wearing an 

exoskeleton, and later it differs for each subject. For example, 

if a subject feels uncomfortable, HR gradually increases until 

the test is finished. Meanwhile, if a subject seems comfortable 

with using the exoskeleton, HR decreases or remains fairly 

constant. Even if a subject feels good, his/her HR can be higher 

than the reference. Thus, rather than the HR value itself, the 

HR value trend tells us much more about the test and 

performance.

B. Data for Load Positioning

The data set for load positioning included fit, HR, and 

survey data sets for 33 subjects who completed the Load 

Positioning task. Each subject completed the tasks in the 

following order, baseline 30 repetitions (without the 

exoskeleton), and 30 repetitions with the exoskeleton after a 

period of returning to a normal, resting heart rate. The 

uncertainty of the data sets are as follows: 

1. The exoskeleton fit based on the anthropometry of the 

subject was estimated by asking the subject for rough 

height and waist measurements, which were typically 

given in inches, while the analysis was done in 

centimeters. A rough estimate of the size uncertainty is 

Load Positioning

A

B

C

D
around 3 cm, based on the 2.54 cm per inch conversion. 

Fit uncertainty is based on comfort of the user and visual 

verification of exoskeleton alignment to the subject’s 

joints. In particular, the shin and femur adjustments are 

done based on alignment to the subject’s heels, hips and 

knees, ensuring the exoskeleton is aligned to the center 

of the sagittal plane, where the inseam of the pants 

provide a sufficient visual indicator. The arms are based 

on a few inches above the elbow. The team received 

training as well as guidance from the exoskeleton 

manufacturer in fitting the suit to subjects. 

2. The HR is estimated using two, and up to three devices, 

including both the use of optical wrist watches, and an 

electrical handheld device for corroboration. The HR 

recorded at the beginning of the task, while the subject is 

seated, and at the completion time of the task is based on 

the highest bpm of either of the optical devices. Of the 

optical devices, one was used predominantly for most of 

the Load Positioning subjects, and the other was only 

acquired later in the experiment. The electrical device is 

used only to verify the optical devices. If the steady heart 

rate is greater than 10 bpm compared to the optical 

wristwatch device(s), the electrical device reading was 

used. Issues that degrade the accuracy of the optical 

devices include the fit of the wristwatch, sensitivity to 

perspiration, and other factors that can impact the ability 

to maintain adequate skin contact for accurate readings. 

Studies have shown the optical wrist-worn monitors have

a concordance correlation of 0.83 to 0.91 at a 95% 

confidence interval [16]. 

3. The survey is based on a series of quantitative categories 

and free-form answers regarding pre-existing 

discomforts of the subject and the respective severity, 

comfort, range of motions, fit, discomfort to the subject 

caused by the exoskeleton or the task, and the subject’s 

opinion on where the exoskeleton provided support 

during the task. While the full survey allowed users to 

enter 33 different regions of the body at 4 severity levels, 

slight, moderate, severe, and extreme, we based the initial 

analysis on just 10 areas that were indicated by the pool 

of initial subjects. The areas included head, neck, upper 

spine, mid-spine, back, lower spine, shoulders, waist, and 

hips. The free-form answers were also reviewed and 

categorized to ease analysis. For example, the responses 

to what the subject liked most about the exoskeleton 

included: (1) arm support, (2) back support, and (3) leg 

support. Similarly, the subjects least favorite attributes 

of the exoskeleton include: (1) heat dissipation, (2) knee 

motion, (3) range of motion, (4) fit, (5) stability, (6) arm 

motion/resistance, (7) impact on fine motor movements, 

and (8) exoskeleton weight. The survey also provided the 

ratings of perceived discomfort (RPDs) [10] based on 

Likert scales of 0 (uncomfortable) to 5 (very 

comfortable) as well as the actual regions of discomfort 

before, during, and after the test.

4. A quad video of the time and environmental conditions, 

HR monitors, and the left and right side of subject 

performing the task was also taken. Our current use of the 

video is limited to visual assessment of subject’s skeletal 

joint positions. The video has also been used to 

corroborate with other data sources including HR, RPD, 

etc. We anticipate expanding the computational analysis 

by using the videos for quantitative assessment on 

skeletal joint angle variability. 

C. Analysis method

An effective test methodology assesses the performance or 

exertion of the subject while wearing the exoskeleton and 

without the exoskeleton to complete the load positioning task. 

The intra-subject independent variable is the exoskeleton. The 

initial exploratory analysis is based on the Spearman’s rank 

correlation coefficient, which assesses whether there is a 

monotonic relationship between two variables. Spearman’s 

rank correlation can be used on both discrete ordinal (gender, 

exoskeleton fit sizes, survey agreement) and continuous 

(height, waist, heart rate differential) variables. 

Various methods have been documented in applying heart 

rate to capture exercise intensity including the Karvonen 

method [17] and variants [18], both of which require age 

information. Because age was not available, the heart rate 

differential with and without the exoskeleton was used as a 

metric to capture the physical intensity of the task to the 

subject:

∆HRLP = (Exoend − Exobegin) − (Baselineend − Baselinebegin)

Equation 1

The final differential with and without the exoskeleton is 

recorded using the following equation:

∆HRFinal = (Exoend) − (Baselineend)

Equation 2

D. Table of Results 

Tables 1 and 2 show a few trends above 95% level of 

significance with respect to gender, anthropometry, and fit. 

Based on [19], the rcrit value of the R coefficient, with 30 

subjects is 0.306 at the 95% confidence level. Females and 

those who indicated prior pains generally had lower heart rates 

in wearing the exoskeleton relative to the baseline. Shoulder 

width also indicated a correlation of broader shoulder width to 

an increase in the subject’s physical exertion. 
 

The findings do not necessarily indicate a causality, but 

potential avenues to explore. For example, if broader shoulders 

led to greater physical exertion, should the shoulders be sized 

for a more snug fit to reduce exertion or is the correlation due 

to the stature of the subject, where broader shoulders would 

lead to greater exertion? 

We only had six females from the 33 subjects, which also 

leads to the need to further explore whether one of the 

performance criteria of the test method can be gender 

neutrality. 

In the survey responses, subjects indicating shoulder 

chafing RPDs had an R coefficient of 0.33, with p-value of 

0.06, relative to the HR differential, ∆HRFinal. Another 

interesting trend was the subjects who indicated what they 

least liked about the exoskeleton had an R coefficient of -0.33, 

with p-value of 0.07. The result is based on converting the freeform response into categories, where the lower values include 

heat dissipation, leg resistance, and range of motion. The 

trends indicate more in-depth investigation in how the 

characteristics of the exoskeleton may impact the physical 

demands on the subject. 

In order to assess the subjects’ opinion towards the 

exoskeleton and the subjects’ HR response, a multiway 

analysis of variance (ANOVA) was computed based on survey 

responses if the subject agreed that the exoskeleton eased and 

helped the task or hindered the task. The factors on whether 

the exoskeleton helped or hindered did not indicate a 

significant response. The result may indicate the need to 

reduce the error tolerance in our current response metric. 

We intend to further expand our analysis to utilize the 

video capture and segment the subject to analyze the subject’s 

range of motion based on skeletal joint angles [20]. In cursory 

visual assessment of the videos, it was noted that subject’s 

often have differing angles in the knee and back when 

comparing postures between the subject wearing the 

exoskeleton (squat lifting technique) and the subject without 

the exoskeleton (stoop lifting technique). The squat lifting 

technique has been widely regarded as a means to avoid lower 

back strain, while the stoop technique has cited benefits of 

ease, reduced energy consumption, and increased stability

[21]. We intend to further explore analysis methodologies

regarding the support of the exoskeleton for completion of 

repetitive, physically strenuous tasks, as well as the ability to 

enable the subject to perform tasks with more ergonomically 

sound postures.

CONCLUSIONS

The NIST Human Subject’s study was approved and is 

underway beginning with the load positioning task test. A 

Position and Load Test Apparatus for Exoskeletons 

(PoLoTAE) was designed to include six generic tasks typical 

of industrial environments, including load handling. Load 

handling tasks are a current ASTM F48 work item towards a 

standard test method. The study used a data collection system 

of cameras, monitors, and HR monitors and the subject 

completed a survey after the completing the task. Results 

showed that the combination of HR and other test data from 

video and the survey provided additional information beyond 

just HR. For example, subject comfort level may affect HR. 

Uncertainties were from: data units conversion, subjective 

exoskeleton fit to the subject by the briefly-trained research 

team, subject responses to survey questions about exoskeleton 

fit, HR monitor devices, and subjective review of video and of 

survey responses. Initial analysis is based on the Spearman 

rank correlation and results are shown in Tables 1 and 2 with 

respect to final and task heart rate differential. Trends showed 

high significance with respect to gender, anthropometry, and 

fit. As in section VI C, females and those who indicated prior 

pains generally had lower heart rates in wearing the 

exoskeleton relative to the baseline. Shoulder width also 

appeared to have an impact on the subject’s physical exertion..

Survey responses of whether the exoskeleton helped or not did 

not indicate a significant response. However, when asked 

outside of the survey if the subjects felt the test method 

exercised the potential capabilities of the exoskeleton, a 

unanimous affirmative was received by all subjects. 

REFERENCES

[1] DARPA Director’s Testimony to the Subcommittee on Terrorism, 

Unconventional Threats and Capabilities, House Armed Services 

Committee, United States House of Representatives, March 29, 2006.

[2] Research Exoskeleton Report, 

http://exoskeletonreport.com/category/research-and-academia/, 

accessed October 28, 2016.

[3] Brian Lowe, William Billotte, ASTM F48 Formation and Standards 

for Industrial Exoskeletons and Exosuits, Submission to IISE 

Transactions on Occupational Ergonomics and Human Factors, 

Special Issue on Occupational Exoskeletons and Exosuits, Oct 2018.

[4] ASTM F48 Exoskeletons and Exosuits,

https://www.astm.org/COMMIT/SUBCOMMIT/F48.htm, accessed 

Jan 2018.

[5] United States Bureau of Labor, “Employment Projections by Major 

Industry Sector”, Web. 26 Jan. 2017.

[6] A. Caruso, “Statistics of U.S. Businesses Employment and Payroll 

Summary: 2012”, Economy Wide Statistics Briefs, US. Census 

Bureau, February 2015. 

[7] Roger Bostelman and Tsai Hong, Test methods for exoskeletons—

lessons learned from industrial and response robotics, Wearable 

exoskeleton systems: design, control and applications, Chapter 13, pp 

335-361, Jan 2, 2018.

[8] NIST Institutional Review Board Study, “Analysis of ExoskeletonUse for Enhancing Human Performance to Complete Industrial 

Tasks”, June 4, 2018 approval.

[9] Roger Bostelman, Position and Load Test Apparatus for Exoskeletons 

(PoLoTAE) Design, draft NIST Internal Report, November 2018.

[10] Saad Alabdulkarima, Maury A. Nussbaumb, Influences of different 

exoskeleton designs and tool mass on physical demands and 

Table 1. Spearman rank correlation to final heart rate 

differential, ∆HRFinal.

R-Coefficient P-Value

Gender -0.36 0.04

Rigid Waist (exo fit) 0.35 0.05

Spine (exo fit) -0.35 0.05

Shoulder (exo fit) 0.41 0.02

Table 2. Spearman rank correlation to task heart rate 

differential, ∆HRLP.

R-Coefficient P-Value

Gender -0.43 0.01

Prior Pain (any) 0.33 0.03
performance in a simulated overhead drilling task, Applied 

Ergonomics 74 (2019) 55–66.

[11] NIST Institutional Review Board training video, 

https://tube.nist.gov/media/t/0_eko39a81, accessed November 11, 

2018.

[12] Robert A. Rogers and Roberto Landwehr, The surprising history of 

the “HRmax = 220 – age” equation, Journal of Exercise Physiology, 

Vol 5, No. 2, May 2002.

[13] American Heart Association, www.AMA.org, accessed Nov 11, 2018.

[14] Wang, R., Blackburn, G., Desai, M., Phelan, D., Gillinov, L., 

Houghtaling, P. and Gillinov, M., 2017. Accuracy of wrist-worn heart 

rate monitors. Jama cardiology, 2(1), pp.104-106.

[15] Parak, Jakub, et al. "Evaluation of the beat-to-beat detection accuracy 

of PulseOn wearable optical heart rate monitor." Engineering in 

Medicine and Biology Society (EMBC), 2015 37th Annual 

International Conference of the IEEE. IEEE, 2015.

[16] Stahl, Sarah E., et al. "How accurate are the wrist-based heart rate 

monitors during walking and running activities? Are they accurate 

enough?" BMJ open sport & exercise medicine 2.1 (2016): e000106.

[17] Karvonen, M.J., 1957. The effects of training on heart rate: a 

longitudinal study. Ann Med Exp Biol Fenn, 35, pp.307-315.

[18] Asselin, P., Knezevic, S., Kornfeld, S., Cirnigliaro, C., AgranovaBreyter, I., Bauman, W.A. and Spungen, A.M., 2015. Heart rate and 

oxygen demand of powered exoskeleton-assisted walking in persons 

with paraplegia. Journal of Rehab. Res. & Dev., 52(2), pp.147-160. 

[19] Gautheir, T.D., 2001. Detecting trends using spearman's rank 

correlation coefficient. Environmental forensics, 2(4), pp.359-362.

[20] Omran, M., Lassner, C., Pons-Moll, G., Gehler, P. V., & Schiele, B. 

(2018). Neural Body Fitting: Unifying Deep Learning and ModelBased Human Pose and Shape Estimation. 

https://doi.org/10.1109/3DV.2018.00062

[21] Bazrgari B, Shirazi-Adl A, Arjmand N. Analysis of squat and stoop 

dynamic liftings: muscle forces and internal spinal loads. Eur Spine J. 

2006;16(5):687-99.