U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/el/intelligent-systems-division-73500/standard-test-methods-response-robots/aerial-systems-1
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Engineering Laboratory
      Intelligent Systems Division
    
      
  Standard Test Methods for Response Robots
                            Expand or Collapse
          NIST's Robotics Test Facility
                      Collaborating Facilities
                      Ground Systems
                      Aerial Systems
                                              Expand or Collapse
                Aerial Program Overview
                                  Open Test Lanes and Scenarios
                                  Obstructed Test Lane and Scenarios
                                  Confined Test Lane and Scenarios
                                  Sensor Test Lane
                                  Simulation
                                  Forms
                                  Frequently Asked Questions
                      Aquatic Systems
                      Robot Competitions
                                              Expand or Collapse
                RoboCup Rescue
                                  Aerial Competitions
                      Events and Meetings
                                              Expand or Collapse
                2023.01 ASTM E54.09 Meeting San Antonio, Texas
                                  2022.06 ASTM E54.09 Virtual Meeting
                                  2022.01 ASTM E54.09 Virtual Meeting
                                  2021.10 UTAC Conference, Guardian Centers, Perry, GA
                                  2021.06 ASTM E54.09 Virtual Meeting
                                  2021.02 ASTM E54.09 Virtual Meeting
                                  2018.06 ASTM E54.09 San Diego, CA
                                  2018.02 ASTM E54.09, Gaithersburg, MD
                                  2017.06 ASTM E54.09, Toronto, Canada
                                  2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA
                                  Prior Events and MeetingsObstructed Test Lane and Scenarios

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

**Obstructed Test Lane**
	**Obstructed Scenario**

	**Hostage Situation**
		**Vehicle Takedown**
		**Hazardous Tanker Truck Accident and Fire**
		**Structure Exterior Inspection**
		**Flooded House and Collapsed Bridge**
		**Night Operations House Surveillance and Search**

**Sticker Files (Obstructed Test Lane and Scenarios)(8/29/22)**
	NIST sUAS Obstructed Test Lane - Scoresheets
	**NIST sUAS Obstructed Test Lane - Pocket Guides**Obstructed Test Lane

The Obstructed Test Lanes enable remote pilots to fly safe and repeatable flight paths to inspect objects within close proximity to obstructions. They include a comprehensive set of 5 different tests that guide remote pilots through various standoff positions, orientations, and perches at 2-3 m (6-10 ft) from objects. They can be performed outdoors or indoors to control lighting, weather, and access to the Global Positioning System (GPS).



 
      Obstructed Test Lanes set up
LEFT: The Obstructed Test Lanes are typically set up in order of increasing difficulty left to right as  Perch, Wall, Ground, Alley, and Post. RIGHT: The Post test shown requires circumnavigating all the wayaround to align with high and low buckets while passing between the post and the wall as if inspecting a  vehicle parked along a 3 m (12 ft) curb near a building.Perch, Wall, Alley and Post TestsThe Obstructed test apparatuses use “dual bucket alignments” to enable remote pilots to triangulate a safe standoff position by simultaneously aligning with a perpendicular bucket and its associated angled bucket. The dual bucket alignments form a right triangle with equal dimensions for the bucket separation and the aircraft standoff. They use pairs of 7.5 liter (2 gallon) buckets with 20 cm (8 in) diameter recessed targets inside at a 2 m (6 ft) spacing.

Alignment rings inside the perpendicular (90 degree) white buckets visually guide safe flight paths toward and away from the objects. Alignment rings inside the associated angled (45 degree) buckets indicate when the aircraft is in the designated safe and repeatable standoff position. That’s where a remote pilot can maintain position and operate their interface to identify increasingly small features on acuity targets inside the buckets. This modest operational workload includes zoom, focus, and exposure to capture images for scoring after the trial. White and black buckets require maximum control of exposure levels or thermal palettes to discern more details and score more points.

There are 5 different tests with increasing difficulty called **Perch, Wall, Ground, Alley**, and **Post**. The dual bucket alignments guide remote pilots through a series of 10 positions, orientations, and perches within both the standard test lanes and the operational scenarios embedded with scoring tasks. All tests and scenarios result in quantitative scores up to 100 points maximum to facilitate measurement, tracking, and comparison across different aircraft and/or remote pilots.

**Procedure**: The procedure is the same for all Obstructed tests. Each test has dual bucket alignments designated 1 – 4 that at are performed in a sequence. The sequence includes some backtracking to ensure the tasks are performed in various directions relative to the obstacles involved. The sequence of positions is 1 2 3 4 – 3 2 1 – 2 3 4 with the red underlined numbers indicating the backtracking part of the sequence. That results in 10 dual bucket alignments or 20 bucket alignments total.

**Maneuvering Trials**: A complete trial totals up to 100 points maximum for 20 bucket alignments. Points are scored using a single no zoom image of each bucket showing either a full alignment ring (5 points), a partial alignment ring (1 point), or no alignment ring (0 point).

**Acuity Targets**: All trials result in maneuvering scores. But Payload Functionality trials add an operational workload to identify acuity targets while aligned with buckets. The level of detail the system can discern should be known and set prior to conducting a Payload Functionality trial. Each acuity target has 5 increasingly small gap orientations to identify correctly. The smallest features are 1 mm (0.04 in) needed to read small text on shipping labels, for example. Each identifiably gap orientation is verbally conveyed to a Proctor during the trial, or when operating alone a single full zoom image of the acuity target can be captured to score after the trial.

**Time Limits**: Test trials are not intended to be races. But trial time limits can be imposed to minimize fatigue across multiple tests or to maintain a schedule. Trial time limits should be long enough for an “expert” to complete a perfect trial. Scores of incomplete trials due to expired time limits can only be compared to trials with the same elapsed time limits. Typical time limits are typically set to 5 minutes for Maneuvering trials and 10 minutes for Payload Functionality trials. Although any time limits may be used depending on the drone, the pilot, and the environment.

**Faults**: A fault is any contact with a test apparatus or any safety issue such as crossing a designated boundary between test lanes or the remote pilot flight line. Any fault results in an end of trial.

**Metrics**: A complete trial requires performing all the designated bucket alignments in order with no faults (contact with an apparatus) or safety issues (exceeding a boundary). If the trial is not complete, the metrics below do not apply. Keep practicing until complete trials are routinely achieved before applying these metrics:

**Score (total points)** – primary: Presuming a complete Maneuvering trial or Payload Functionality trial, the total points are a measure of the combined effectiveness of the aircraft system and the remote pilot to maneuver through all the positions, orientations, and perches necessary to align with all the buckets. Trial scores add up to 100 points maximum. These scores can be used to compare remote pilot proficiencies when using the same drone and interface in the same test method. They also can be compared to the score of an “expert” pilot, typically provided by the manufacturer, which is considered the 100th percentile of proficiency on a particular aircraft system.
	**Efficiency (elapsed time)** – optional: Presuming a perfect score for a Maneuvering Trial or a Payload Functionality trial, the elapsed time is a measure of the combined efficiency of the aircraft system and the remote pilot. Elapsed trial times can help distinguish between perfect scores to identify more efficient techniques or approaches.**Note**: The verge points of the dual bucket alignments designate the most efficient locations to score all buckets. However, stable hovers at the verge points between buckets can be difficult to enforce similarly for various drone sizes and pilot proficiencies. So each bucket can be scored individually from any desired proximity, understanding that the resulting scores and trial times may be negatively affected.

[TOP]

Obstructed Scenario: Hostage Situation

The bus hostage scenario included role players as perpetrators and victims with actual law enforcement organizations from the region demonstrating simulated tactics including drones and a ground robot.

 
      Bus Hostage Situation
Figure 14) LEFT: One drone dropped a mobile phone to establish communications while another surveilled  the situation. RIGHT: The ground robot also supported interactions prior to tactical intervention by law  enforcement officers.We added quantitative measures of performance to this scenario using dual bucket alignments to guide remote pilots safely through a designated series of inspection tasks. This enables comparison of scores up to 100 points for pilots and aircraft that can reliably perform the various bucket alignments and identify the smallest visual/thermal acuity features across all available targets. The trial time limit was set to 20 minutes to remain within one battery charge and to maintain a schedule for multiple pilots. Time limited trials also enable direct comparisons of scores for completeness and efficiency. Only scores using similar aircraft and time times are directly comparable to evaluate pilot proficiency, but a variety of different aircraft can be used to compare overall scores and ease of use.

The flight paths included safe and repeatable standoff positions, orientations, and perches within 2 m (6 ft) of objects for aircraft flying the most efficient flight path. But bucket alignments can be achieved from any standoff for larger aircraft using more capable zoom lenses. The embedded scoring tasks are 7 liter (2 gallon) buckets with 20 cm (8 in) diameter recessed targets inside. The perpendicular (90 degree) white buckets provide visual alignments for the remote pilot to trust as safe vectors to approach and leave the object being inspected. The associated angled (45 degree) buckets enable triangulation to maintain a safe proximity from the object while operating their zoom, focus, and exposure to capture images for scoring after the trial.

Each set of tasks included 10 positions and orientations worth 10 points each distributed throughout the scenario. The pilot scores 5 points for getting aligned with each perpendicular bucket plus another 1-5 points for correctly identifying increasingly small features on the acuity target located inside the associated angled bucket. The smallest features to identify are 1 mm (0.04 in) representing small text on shipping labels, for example. The white and black pairs of buckets require maximum exposure control to capture images for scoring after the trial.

 
      Obstructed Scenario: Bus Hostage Situation 2
There were two orbits to perform sequentially. The large dual bucket alignments defined positions #1-10 to assess the situation quickly during the initial assault. Then the smaller dual bucket  alignments defined positions #11-20 to identify key details and gather evidence. Scenarios include an  accurate perch position and orientation to demonstrate a persistent point of view that’s considered  operationally important. In this case the large bucket #10 required landing with a view through the door up  the stairs to the driver. Another exterior small bucket perch was set up for a suspicious package placed  under the vehicle. There was also an interior perch location to identify a package found inside on the floor  near a seat.Obstructed Scenario: Bus Hostage Situation 2
LEFT TO RIGHT: Exterior alignment with bucket 9 and 9A (white) look inside the door directly above the perch position. The aircraft perched while aligned with buckets 10 and 10A (black). The view from  the aircraft while perched is a persistent view of the door and driver. Exterior perch positions are also  designated for underbody inspection tasks using the smaller buckets during a second phase.Quad screen video of the bus hostage
The quad screen video of the bus hostage scenario showing the drone point of view in the perch position (upper left).Large vehicles like buses can require interior searches as well. These relate to the Confined Test Lanes described later, but are typically part of these types of scenarios. They are intended only for scoring by aircraft that can safely fly inside the bus.

 
      Obstructed Scenario: Bus Hostage Situation
LEFT: An interior dual bucket alignment (buckets 1 and 1A) just inside the entry door at the driver’s seat. The perpendicular (90 degree) bucket provides a safe vector to approach the apparatus until  the angled bucket (45 degree) is simultaneously aligned. That’s the safe 1m (3ft) standoff position at which  the pilot should maintain position knowing the drone is safe from obstacles and work the interface controls  for zoom, focus, and exposure to capture an image that can be scored after the trial. RIGHT: Other dual  bucket alignments were related to views of objects on the seats, under the seats, and near the back door on  the floor.Obstructed Scenario: Bus Hostage Situation
LEFT: Other interior tasks using the small buckets in confined search spaces. CENTER: Two apparatuses require vertical alignments. RIGHT: Three apparatuses require horizontal alignments. This is the  final perch position to maintain a view of an object of interest.[TOP]

Obstructed Scenario: Vehicle Takedown

 
      Obstructed Scenario: Vehicle Takedown
LEFT: Vehicles of all sizes and conditions can be bucketized similarly for quantitative scoring, including panel vans and trucks. RIGHT: Dual bucket alignments guide the drone into position to look inside  the front windshield from two different angles while using triangulation between the buckets to stay safe.Obstructed Scenario: Vehicle Takedown
The bucket numbering leads the pilot around an orbit starting in front of the vehicle, then around to the driver side, and so on. The large buckets targets are identified in the first orbit to assess the situation for up to 100 points. The small buckets targets are identified in the second orbit as evidence gathering for  another 100 points, including where the gun landed during the demonstration.Obstructed Scenario: Vehicle Takedown
The quad screen video of the tactical vehicle takedown with evidence gathering.[TOP]

Obstructed Scenario: Hazardous Tanker Truck Accident and Fire

The tanker truck accident and fire scenario used 5 large bucket apparatuses to guide remote pilots to points of view around the vehicles for a fast tempo initial assessment. We also embedded 5 small bucket apparatuses near features needing detailed inspections such as the inside of the cab, dripping valves, and gauges to read. This scenario included drop accuracy tasks to emplace remote sensors trying to determine the nature of the hazard (see the sensors on the ground).

 
      Obstructed Scenario: Hazardous Tanker Truck Accident and Fire
Figure 23) LEFT: The tanker accident scenario included a simulated fire and potential hazmat spill requiring firefighters to react as appropriate based on reconnaissance from the drones and dropped sensors. RIGHT:  Thermal overview image from the drone and the dropped sensor packages.Obstructed Scenario: Hazardous Tanker Truck Accident and Fire
Figure 24) Our quad screen video captures simultaneous views of what the remote pilot sees through the interface, all interactions with the interface, and off-board views of the situation around the aircraft.[TOP]

Obstructed Scenario: Structure Exterior Inspection

 
      Obstructed Scenario: Structure Exterior Inspection
The collapsed structure had several different scenarios to fly. The Obstructed scenarios (some  shown with ovals) were perimeter windows, doors, and surrounding ground objects. The Confined scenarios  (some shown with rectangles) were vehicles in the semi-collapsed parking garage and room-to-room  searches inside the semi-collapsed building.Structure exterior inspection tasks include looking through windows and doors along with surrounding ground objects of interest. In this case, the tasks were embedded around a partially collapsed structure. The objective was to safely fly in close proximity of about 2 m (6 ft) from the windows and doors to perform a window/door clearing maneuver with high/low and left/right views inside the structure.

 
      Obstructed Scenario: Structure Exterior Inspection
Obstructed apparatuses are set up in sets of vertical and horizontal tasks throughout the scenario.Obstructed Scenario: Structure Exterior Inspection
The scenario facade was embedded with window inspection tasks on two sides with partially collapsed walls. Each Dual Bucket Alignment guides the remote pilot to positions and orientations with  high/low and left/right views inside the windows.Obstructed Scenario: Structure Exterior Inspection
Our quad screen video captures simultaneous views of a trial, including what the remote pilot sees through the interface, all interactions with the interface, and off-board views showing the situation  around the aircraft.Obstructed Scenario: Structure Exterior Inspection
This final vertical test apparatus leads the remote pilot to perch with a view through the doorway. This was the 10th and final position and orientation in the scenario that totaled up to 100 points for a  complete search through the windows of the façade on two sides of the structure.[TOP]

Obstructed Scenario: Flooded House and Collapsed Bridge

Several houses were submerged with victims on the roof and others in the water nearby. The tasks to perform included delivery of personal floatation devices to those stranded on the roof, searching in windows for survivors, and identifying/tracking moving swimmers. The omni bucket apparatuses replaced the actors as objects to identify and the landing served as target for the dropped payload accuracy.

 
      Obstructed Scenario: Flooded House and Collapsed Bridge
Victims on the rooftop of a flooded house were replaced with omni bucket stands to perform identification tasks and a landing was used as an accuracy target for dropping a personal flotation device.Obstructed Scenario: Flooded House and Collapsed Bridge
LEFT: Other inspection tasks required the drone to hover low above the water to look in windows and doors. The two positions and orientations designated by the white and black pairs of buckets guided  the pilot to completely inspect the floating mattress looking for survivors (visual or thermal). RIGHT: Similar  window inspection tasks were located under a more difficult covered porch near a door. That’s where  underwater versions of these same tests were embedded for underwater system reconaissance.Obstructed Scenario: Flooded House and Collapsed Bridge
A floating omni bucket task was pulled through the water by a remotely controlled rescue device to represent floating or swimming victims. It provided a moving object for the drone to identify on all sides.Obstructed Scenario: Flooded House and Collapsed Bridge
The flashflood bridge collapse included an overturned school bus and vehicles requiring a comprehensive search for survivors.[TOP]

Obstructed Scenario:Night Operations House Surveillance and Search

This scenario was actually conducted the week prior at a different event, but nicely augments all of the scenarios above. This house surveillance at night used two sets of horizontal and vertical Obstructed test apparatuses on all four sides of the house, guiding the remote drone pilot to safe locations among a variety of very difficult obstacles such as the overhanging roof, trees, shrubs, power lines, shrubs, etc. A set of 5 vertical test apparatuses with 10 positions and orientations were attached to all sides of the house totaling 100 points. A set of and 5 horizontal test apparatuses with 10 positions and orientations were also placed on or near objects of interest around the house for another 100 points.

 
      Obstructed Scenario: Night Operations House Surveillance and Search
Obstructed test apparatuses embedded into an exterior house search at night show the designated hover positions and orientations including a perch position in view of the back door. The aircraft  was outfitted with lights to illuminate the scene but also used its thermal vision capabilities to negotiate the  myriad of obstacles such as trees, power lines, shrubs, etc.[TOP]



Sticker Files - 10 cm (4 in) diameter  - Obstructed Test Lanes and Scenarios

**Level 4  Obstructed Lane - Payload Functionality:**

**Note: Obstructed and Confined use the same stickers.**

**NIST sUAS 4 Inch Obstructed Lane - Stickers for Linear Rails - WHITE and BLACK - 3.5in Diam (v2020B Answer Key)**(4/19/23)**Level 4 Obstructed Scenario - Payload Functionality:**

**NIST sUAS 4 Inch Obstructed Scenario - Stickers for Linear Rails - WHITE and BLACK - 3.5in Diam (v2020B Answer Key)**(4/19/23)Sticker Files - 20 cm (8 in) diameter  - Obstructed Test Lanes and Scenarios

**Read First**

**Directions for printing stickers (8/29/22)****Level 4  Obstructed Lane - Payload Functionality:**

NIST sUAS Obstructed Lane - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key)
	NIST sUAS Obstructed Lane - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key) (ROUND STICKERS ONLY)
	NIST sUAS Obstructed Lane - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key) (RECTANGULAR STICKERS ONLY)**Level 4 Obstructed Scenario - Payload Functionality:**

NIST sUAS Obstructed Scenario - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key)
	NIST sUAS Obstructed Scenario - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key) (ROUND STICKERS ONLY)
	NIST sUAS Obstructed Scenario - Stickers for Linear Rails - WHITE and BLACK - 8in Diam (v2020B Answer Key) (RECTANGULAR STICKERS ONLY)

[TOP]

    
    Created August 3, 2022, Updated April 25, 2023 ‹ Open Test Lanes and Scenarios
              Confined Test Lane and Scenarios ›
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov