Real-Time Identification of LEO Objects
in the Tomo-e Gozen Camera Images
with a Multi-GPU System

Manuel Cegarra Polo, Toshifumi Yanagisawa, Hirohisa Kurosaki
日本学術振興会 海外特別研究員
研究開発部門 第二研究ユニット
JAXA Chofu

年度プラネタリーディフェンス・シンポジウム
14-15 February 2022

Introduction
BACKGROUND--LEO objects will appear as streaks in images when tracking at sidereal rate, with lengths depending on object
elevation and camera exposure time.
In the literature, there are many research works aimed to detect streaks, mainly focused in the detection sensitivity,
where only a few of them focus on the processing speed:- (1) 4096 x 4096 (16.7 Mpixels) -> 3.19 sec GPU- (2) 1024 x 1024 (1.05 Mpixels) -> 3 sec
CPU- (3) 2049 x 2047 (4.2 Mpixels) -> 2 – 7 min CPU- (4) 2048 x 2048 (4.2 Mpixels) -> 13 sec CPU
Reduction of processing speed can be beneficial when:- There is a massive amount of images to process.
- Orbital parameters of detected objects are sent to follow-up observation stations in next LEO object pass.

OUR SOLUTION--We developed a real-time processing system to detect and identify objects as streaks, based in heterogenous
computing (mixed multi CPU-GPU), in two variants (standard and high sensitivity):- STANDARD:
18 x 2000 x 1128 (40.6 Mpixels) -> 0.3 sec (detection); 3.3 sec(including identification)- HIGH:18 x 2000 x 1128 (40.6 Mpixels) -> 3.7 sec (detection); 6.7 sec(including identification)
We achieved Real-Time performance (file processed before next file arrives):
18 frames of 2000 x 1128 pixels with 0.5 sec exposure time = 9 sec per file (without considering nudging time)(1) Diprima, F et al., T. 2017, 7th European Conference on Space Debris, Darmstadt, Germany(2) Hickson, P. 2018, Adv. Space Res., 62, 3078-3085(3) Cvrcek V. et al. 2019, 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 5, 498-509, Prague(4) Virtanen, J. et al. 2016, Adv. Space Res., 57, 1607-1623

GPU Technology
IMPORTANT CONCEPTS-GPU technology is well suited for image processing of massive number of pixels (memory) in parallel (sameoperations).
Heterogeneous computing: systems that use several types of processors or cores. Current workstations and highperformance computers include many CPU cores and a high number of GPU cores.
Nowadays there exist mature NVIDIA CUDA GPU libraries for image processing and algebra to help in development:
opencv, cublas, magma, etc.

GPU0014 x NVIDIA Quadro RTX8000
4608 CUDA CORES each

Hardware architecture CPU vs GPU(source: NVIDIA CUDA C++ Programming Guide)

Our Top Level Parallel Processing heterogeneous
CPU-GPU computing system

GPU0022 x NVIDIA Ampere A6000
10752 CUDA CORES eachProcess steps: Preprocessing-Includes 2 substages:- Removal of stars: (variation of star removal method*) to avoid detection of false positives of streaks created bydiffraction spikes of bright stars or other astronomical objects.
- Binarization: global or local (adaptive thresholding).

Diffraction spikes of bright star (Tomo-e camera)

Globular cluster (Tomo-e camera)
Substages of preprocessing step(*) Yanagisawa, T. Et al. Automatic Detection Algorithm for Small Moving Objects, PASJ, Vol. 57, Issue 2, 2005, Pages 399–408

Process steps: Detection
STANDARD SENSITIVITY (HOUGH TRANSFORM*)- CUDA OpenCV implementation of the PPHT
(Progressive Probabilistic Hough Transform) -> Fast,detection of segments instead of lines- Longest of each group of segments of len. > 200 pixels.

HIGH SENSITIVITY (STACKING METHOD**)- Pixel binary values (0/1) are accumulated oversegments of length 200 pixels.
- Casefile includes 1128 possible directions.
- Cases over threshold (~130/200) are candidates.

Theoretical principle of Hough Transform
source:https://commons.wikimedia.org/w/index.php?curid=74745305

Generation of casefile

Segments detected for a narrow (left) and a wide (right) streak* Hough, P.V.C. 1962, U.S. Patent 3069654

Faint streak after binarization

raw output: case, pixel and value over threshold** Detection of Small GEO Debris by Use of the Stacking Method", Yanagisawa
T. et al, T. Japan Soc. Aero. Space Sci., V.44, No. 146, pag. 190-199, 2005

Process steps: Identification-Includes 3 substages:- Astrometry: Obtention of astronomical coordinates (RA, Dec) of each streak end.
- Identification: coordinates ends are compared with those of known objects of a space-track.org database of29,082 entries, based in the SGP4 orbital propagation or their initial TLE coordinates.
- Differential Photometry: Only to characterize system sensitivity, not intrinsic to the object, and not included in themain loop process.

1 Elliptical area surrounding the detected streak to compute the total brightness.
Circular areas surrounding both ends of the detected streak, that defines the error

2 with the TLE database of objects comparison.

Trajectory of real object in the database obtained through the propagation of its

3 TLE data with SGP4 algorithm.

4 photometric stars for differential photometry.
(RA,Dec) coordinates of object from database corresponding to the timestamps of5 start and end of camera integration time. if these points are encompassed by thecircular areas (2), the streak is associated with the candidate object.

Results: Experimental Set-up
TOMO-E CAMERA AT 1M SCHMIDT TELESCOPE AT KISO OBSERVATORY
FITS files:
18 consecutive frames of 0.5 s exposure time2000 x 1128 x 18 ~ 40.61 Mpixels ~ 162.4 Mbytes
Data rate:~ 1.7 Tbytes / hourSensor model:
Canon 35MMFHDXMSensor type:
CMOS Front illuminatedPixel size:
19 μm/pixel
Pixel scale:
1.198 arcsec/pixel
No. sensors:
84
OBSERVATION SET
STANDARD SENSITIVITY: 21,084 FITS files (2 hours)
HIGH SENSITIVITY:
2.373 FITS files

Layout of CMOS sensors in Tomo-e Gozen camera. 1.198 arcsec/pix
source: www.ioa.s.u-tokyo.ac.jp/tomoe~3.4 TB~386 GB

35 mm Full HD 2000 x 1128 pixels
source: www.ioa.s.u-tokyo.ac.jp/tomoe

Results: Sample Images

Detected by Standard and High Sensitivity Methods

Detected ONLY by High Sensitivity Method

Results: Performance
STANDARD SENSITIVITY (HOUGH METHOD)
detection time: ~100 ms per FITS file (18 frames)
12-130 times faster than CPU
REAL-TIME (4xGPU): 3.3 s < 9 s
Detection rate: ~ 0.5%
Identification rate: 83%Min. apparent magnitude: +11.3
HIGH SENSITIVITY (STACKING METHOD)
detection time: ~14 s per FITS file (18 frames)
60 times faster than CPU
REAL-TIME (4XGPU): 7 s < 9 s
Detection rate: ~ 5%
Identification rate: ~66%.
Min. apparent magnitude: >+11.3

Comparison of CPU vs. GPU processing time inthe detection stage for a range of thresholds(Hough Method)

Comparative speed performance at CPU vs. GPU vs. 4xGPUof Hough and Stacking Detection algorithms

Photometry of 35 objects detected. NC are Non-Catalogued Objects. 8 objects where notmeasured as one of their ends was out of bounds of the image sensor (Hough Method)

Results: Output table

Fragment of output table with the High Sensitivity method

Details for one FITS fileConclusions--We developed a real-time processing system to detect and identify objects as streaks, based in heterogenous
computing (mixed multi CPU-GPU), in two variants (standard and high sensitivity), with Real-Time performance.
CPU-GPU heterogeneous computing is a convenient solution for the need of big data processing in SSA andAstronomy.
Plan to shorten Postprocessing stage time through GPU implementation.
Next stage: Integration and installation in existing SSA observatories under consideration.

ACKNOWLEDGEMENTS-This research is funded by the fellowship program of JSPS.
Astronomical images were kindly provided by Tomo-e Gozen Team of University of Tokyo.

--- どうもありがとうございました ---