54th Lunar and Planetary Science Conference 2023 (LPI Contrib. No. 2806)

2511.pdf

REAL-TIME MONITORING OF ONBOARD IMAGE PROCESSING PERFORMANCE DURING DARTTERMINAL APPROACH. D. L. Bekker1, R. T. Daly1, C. M. Ernst1, Z. J. Fletcher1, L. M. Rodriguez1, C. A.
Sawyer1, 1 Johns Hopkins University Applied Physics Laboratory, Laurel, MD (dmitriy.bekker@jhuapl.edu)
Introduction: On 26 September 2022, the world
watched a live image feed from the Double Asteroid
Redirection Test (DART) spacecraft as it headed toimpact with its target – the asteroid moonlet
Dimorphos of the (65803) Didymos binary asteroid
system [1]. This successful demonstration of kinetic
impactor technology enabled the collection of crucial
observational data [2, 3] needed for the design offuture planetary defense missions to protect Earth fromhazardous asteroids – if such a threat is ever identified.
In the final four hours of the DART mission,termed Terminal Approach, the spacecraft navigatedautonomously to impact using the onboard Small-body
Maneuvering Autonomous Real Time Navigation(SMART Nav) system [4]. SMART Nav processed alist of candidate objects in the field of view that wassupplied by the onboard processing component [5] ofthe Didymos Reconnaissance and Asteroid Camera forOpNav (DRACO) – the sole instrument onboard [6].
DRACO images were captured at 1.04 Hz andstreamed in real-time to Earth, with all data passing
through the DRACO Image Processing Pipeline(DRIP) onboard [5]. This live image feed was provided
to the public to follow DART’s progress duringterminal approach. At the Johns Hopkins Applied
Physics Lab Mission Operations Center (MOC),
operators used enhanced telemetry and image displays
to monitor image quality and the status of onboard
processing. Visualizing these data was important toensure DRIP and SMART Nav were operating
properly and did not require any parameter adjustmentsthat had been prepared as part of pre-planned mission
contingencies and practiced in rehearsals. We present
an overview of the displays used to monitor DART
onboard image processing performance, shown at theT- 20min mark prior to impact.

Fig. 1aFig. 1bFig. 1dFig. 1cFig. 1eFigure 1. Blob and Centroid Display (T- 20min), updated at a 1 sec cadence. a) Geometric view of the location of centroids fromDRIP, annotated with SMART Nav-reported identification as Didymos (obj. A) and Dimorphos (obj. B). Red dotted squaredenotes active 512x512 window used for live image streaming. b) Listing of up to 16 blobs reported by DRIP along with theircoordinates, characteristics, and SMART Nav assigned track IDs – all computed onboard. c) Sizes of largest two blobs, ascompared to expected results from models. d) DRIP image processing state telemetry. e) Number of all blobs found vs. blobs
reported to SMART Nav for processing (after reduction by filtering), track IDs of largest two blobs, and IDs assigned asDidymos (obj. A) and Dimorphos (obj. B).

54th Lunar and Planetary Science Conference 2023 (LPI Contrib. No. 2806)

Blob and Centroid Display: During terminal
approach, DRIP was configured to capture images for
streaming and provide connected-component analysis(CCA) results to SMART Nav. The CCA algorithm
was used to generate a list of contiguous objects abovea threshold (blobs), compute their center coordinates(centroids), and report statistics such as blob number ofpixels and blob intensity. These data were then used by
SMART Nav to build tracks and ultimately determine
which object was Didymos and which was Dimorphos.
In the MOC, operators monitored blob statistics tounderstand where they were located in view (Figure1a), how many, how big, and how bright these were
calculated to be by DRIP (Figure 1b), and how thelargest two blobs tracked expected model performance(Figure 1c). DRIP included onboard filtering functions
to discard small blobs that otherwise would put
unnecessary pressure on downstream SMART Nav
processing. This functionality was monitored as well
(Figure 1e), and used to monitor how Dimorphos camein and out of view when first being resolved byDRACO.
Engineering Image Display: While the blob andcentroid display was useful for monitoring onboard
processing performance, a big picture view was needed
to put this information in context with actual image
data as received from the spacecraft. The engineering
image display (Figure 2) showed this concisely, withkey summary data displayed alongside and overlaid
with the image. Focusing more on the image
processing performance and telemetry in-context, thisview differed from the web-based display broadcast
live during the impact event, intended for a broader
audience and with a different set of overlays. Theengineering image display let the MOC team confirm
that DRIP and SMART Nav were tracking the correct
objects, and that the blob ID’d as Dimorphos wasindeed the smaller, dimmer object. Automatic zoomins of both Didymos and Dimorphos provided firstlook basic shape and size characteristics, and an ability
to visually track the DRACO image detector maximum
pixel value to monitor and inform if there is saturation.
Immediately after impact, the scripts used to generate
this display were reused for quickly generating animpact sequence movie and impact image mosaic forbroadcast to the public.
Conclusion: The DART mission terminal approach
was a four-hour dynamic event that required real-timemonitoring of onboard image processing performance.
Unlike many flyby missions, relying solely on static
telemetry view data and analyzing post-event imagery
was not an option. A carefully overlaid live-view ofboth telemetry and image data was necessary toquickly visualize spacecraft performance, health,2511.pdf

image quality, and targeting state. Future missions withdynamic events can benefit from a similar approach,joining telemetry data and instrument observational
data in real-time for mission operations use.
Fig. 2aFig. 2bFig. 2cFigure 2. Engineering Image Display (T- 20min, at sametime instance as display in Figure 1), updated at a ~30 sec
cadence as live image data became available for analysis in
the MOC. Contrast stretching applied on the ground toenhance visual. a) Overlay view, showing blobs identified as
Didymos and Dimorphos together with the real image as
computed onboard the spacecraft. Time to impact (T-),
distance to-go, number of blobs detected, and otherengineering state data are also shown. b) Zoom-in of blob
identified as Didymos. c) Zoom-in of blob identified as
Dimorphos. In both b) and c), number of pixels in blob, total
blob intensity, value of maximum pixel, and scale bars arealso shown.

References: [1] Daly, R. T. et al. (2023) Nature, inrevision. [2] Thomas, C. et al. (2023) Nature, inrevision. [3] Cheng, A. F. et al. (2023) Nature,submitted. [4] Chen, M. H. et al. (2018) AAS GNC, 18063. [5] Bekker, D. et al. (2021) IEEE SCC, 122-133.
[6] Fletcher, Z. J. et al. (2022) SPIE, 121800E.