SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 10(4); March (2024), pp. 299-303
ISSN (Online): 2352-6386; ISSN (Print): 2405-9021
https://alife-robotics.org/jrnal.html

Research Article

Coastal Litter Detection through Image Analysis-Employing
Deep Learning to Identify MicroplasticsYuto Okawachi1, Shintaro Ogawa1, Takamasa Hayashi1, Tan Chi Jie1, Janthori Titan1, Eiji Hayashi1, Ayumu Tominaga2,
Satoko Seino3

Department of Mechanical Information Science and Technology, Kyushu Institute of Technology, 680-4, Kawazu, Iizuka-City, Fukuoka, 820-8502, Japan
Department of Creative Engineering Robotics and Mechatronics Course, National Institute of Technology Kitakyushu Colllege, 5-20-1 Shii, Kokuraminamiku,Kitakyushu, Fukuoka, 802-0985, Japan
3
Graduate School of Engineering, Kyushu University 744 Motooka, Nishiku, Fukuoka city, Fukuoka, 819-0395, Japan
1
2

ARTICLE INFO
Article History
Received 30 November 2023
Accepted 08 April 2024
Keywords
Deep learning
Object detection
YOLOv7

ABSTRACTThe challenge of coastal litter accumulation led to the creation of a detection system powered bydeep learning, aimed at identifying microplastics. The system harnessed the yolov7 [1] deeplearning architecture, known for its proficiency in real-time object detection, and integrated theSAHI (Slicing Aided Hyper Inference) [2] vision library to augment its capabilities. Within thescope of our study, we conducted four separate evaluations using two versions of yolov7—thebase model and the advanced yolov7-e6e—alongside SAHI. The performance of each setup wasmeasured against a set of metrics, including Intersection over Union (IoU), Precision, Recall, Fmeasure, and Detection Time, recorded in seconds. The dataset for the study was composed ofimages sourced from real-world beach clean-up sites, including Hokuto Mizukumi Park. Thedetection algorithm was subjected to 700 rounds of training, with an initial learning rate of 0.001.
Our findings indicated that the system was adept at identifying relatively small microplastics.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

1. Introductionsubstantial logistical challenges, hindering survey efforts.
Addressing the accumulation of coastal debris
necessitates comprehensive data collection, a task thatproves to be formidable using conventional survey
techniques.
This research proposes a novel approach to streamline
the survey process of coastal debris through theapplication of deep learning in the analysis of imagery fordebris identification. A focal point of our investigation
was the adept detection of microplastics—particles sosmall they have eluded detection in prior examinations.
By harnessing deep learning to generate a coastalimagery dataset, we facilitated the development andsubsequent validation of a model dedicated to detecting
these particles. An exemplar from our dataset, depicted inFig. 1, illustrates the type of imagery analyzed. The study
prioritizes the development of a dedicated model for thedetection of microplastics, showcasing its potential forSpecialized application.

Litter drift has become a pervasive issue across Japan,
with a significant portion of this litter, in terms of volume,
weight, and quantity, being synthetic rather than organic.
Plastics, which do not break down in nature, fragment
into microplastics under the action of UV radiation andwave motion, subsequently adrift in marine environments
before accumulating on beaches. Microplastics can carryhazardous chemicals incorporated during manufacturing
or may adsorb dangerous substances like PCBs
(polychlorinated biphenyls) while adrift, posing a risk toaquatic life if consumed. Due to their diminutive size,
under 5mm, spotting these microplastics on beaches
becomes a challenge.
Japan's shores are inundated with debris, including
minuscule particles like microplastics that easily blend
into the sand, rendering them nearly invisible to current
detection methodologies. Moreover, the exploration of
remote or technologically reinforced coastlines,characterized by the presence of wave-breakers, presents

Corresponding author E-mail: ookawachi.yuuto693@mail.kyutech.jp, tominaga@kct.ac.jp, haya@mse.kyutech.ac.jp URL: http://www.kyutech.ac.jp/299

Fig. 3 ELAN, E-ELAN Structure
2.2. SAHI(Slicing Aided Hyper Inference)
SAHI is a streamlined vision toolkit designed forexecuting large-scale object detection and instancesegmentation tasks with efficiency. Demonstrated in Fig.
4, the technique processes an input image by partitioning
it into a series of smaller, manageable rectangles. The
model assesses each of these individual sections insequence and then compiles the results, producing a
consolidated output of the detection findings. Although
SAHI is compatible with a variety of deep learningmodels, it does face a constraint in that the time required
for estimation proportionally increases with the numberof image segments being analyzed.

Fig. 1 Examples Dataset
2. Methodology
2.1. Composition of YOLOv7
In this investigation, the YOLOv7 deep learning model
was utilized. This model represents an evolution of theYOLO (You Only Look Once) family, providing asignificant increase in processing speed for object
detection tasks. As demonstrated in Fig. 2, when applied
to the MS COCO dataset, YOLOv7 outperforms previous
iterations of the YOLO series in terms of Average
Precision (AP) values. Our study made use of two
specific variants within the YOLOv7 suite: the standard
YOLOv7, referred to as YOLOv7 (plain), and anenhanced version, YOLOv7-E6E. Both models employ atri-level pyramid architecture that generates a variety offeature maps by alternating between upsampling anddownsampling processes. For object prediction tasks,
three distinct feature maps from this architecture areutilized. The YOLOv7 (plain) is constructed using aseries of convolutional layers named ELAN, while
YOLOv7-E6E incorporates a more advanced network
structure, designated as E-ELAN. The configurations ofthe ELAN and E-ELAN networks are depicted in Fig. 3.

Fig. 4 Image segmentation image using SAHI
2.3. Dataset creation
This research utilized a dataset generated fromphotographs captured at actual beach cleaning sites,
including Hokuto Miukumi Park, with a focus on
microplastics interspersed in the sand. To enhance themodel's ability to distinguish microplastics from othercommon beach detritus such as shells and stones, thedataset was supplemented with images exclusively
featuring these items. The singular category labeled andidentified within this dataset was microplastics. From aninitial collection of 534 photographs, the dataset wasexpanded to encompass 1,376 images by applying abrightness filter to augment the dataset for training andtesting purposes. This expanded set was allocated asfollows: 977 images for the training set, 291 for the final
test set, and 108 for a general test set.

Fig. 2 Comparison of YOLOv7 with other real-time
object detectors
300

3. Results and Discussion

2.4. Identifier creation and evaluation methods
In this study, the detection system was refined through700 iterations of training with the assembled dataset. The
models were fine-tuned with an emphasis on maximizing
the combined weighted values of Average Precision (AP)and mean Average Precision (mAP), using a weighting
ratio of 1:9 to guide the optimization process during
training. The investigative procedures incorporated twovariants of YOLOv7—YOLOv7 (plain) and YOLOv7E6E—alongside the application of SAHI. The efficacy ofthe detection system was appraised by setting a thresholdfor the Intersection over Union (IoU) metric at above 0.65.
Comparative analysis was carried out, focusing on
metrics such as precision, recall, F1-score, and theduration of estimation. The F-score, which balances
precision and recall through their harmonic mean, alongwith the estimation time—the interval required for themodel to process an input image and generate an output—
were central to the performance evaluation. Thecalculations for precision, recall, and the F-score aredelineated in equations (1) through (4), and thecorresponding confusion matrix is delineated in Table 1
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 =𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 =𝐼𝐼𝐼𝐼𝐼𝐼 =(2)2 ∗ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 ∗ 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 + 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅(3)𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝑜𝑜𝑜𝑜 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝑜𝑜𝑜𝑜 𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈Actual Result

Table2. Performance metrics by model pairing

𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇(1)
𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 + 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹

𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇
𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 + 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹

𝐹𝐹 − 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 =Fig.5 Example of detection results

Table1. Confusion Matrix
Positive
Negative

Fig. 5 showcases the detector's capability to pinpoint
even diminutive microplastics and effectively discernthem from similarly shaped stones, avoiding false
positives. The integration of SAHI with both YOLOv7
models enhanced precision across most evaluated metrics,
except for the time taken for estimation. Notably, when
paired with the YOLOv7-E6E model, SAHI substantiallyboosted the recall metric by about 0.2, signifying anenhancement in the model's ability to identify
microplastics when supplemented with SAHI. This
improvement, however, came with the trade-off of atenfold increase in the time required for estimation.

(4)

Despite the promising ability of the detection system toaccurately identify microplastics, as evidenced by itsperformance, the precision and recall rates reported inTable 2 were suboptimal. One potential factor
contributing to this could be the inherent quality of thedataset. Since the annotations within the dataset weremanually generated, and items not clearly identifiable asdebris were omitted from labeling, this practice might
have led to the identification of previously unlabeled
microplastics, thus affecting the precision rate. The issue
with recall might stem from the model's inability to detectmicroplastics that share a visual similarity with stones orshells. The photographs in the dataset were taken using
an iPhone at high resolutions, such as 3024 × 4032. To
lessen the computational demand during the experiment,
the images were downsized to 1280 × 1280 before being
fed into the model, which could hinder the recognition ofsmaller items. Remedying these limitations could entail
labeling even the smallest detectable objects andPredictions
Positive
Negative
True Positive
False Negative
False Positive
True Negative301

enriching the training dataset with more varied
representations of materials like stones and shells. Finetuning model hyperparameters, for example, batch size,may also lead to improved accuracy. The longer
estimation times associated with the use of SAHI might
be alleviated through the adoption of software solutions
like TensorR [3], designed to expedite deep learning
inference processes.

Mr. Takamasa Hayashi
He received his Bachelor’s degree inEngineering in 2023 from intelligent
and Control Systems, Kyushu Instituteof technology in Japan. He is currently
a Master student in Kyushu Institute ofTechnology and conducts research at
Hayashi Laboratory.

4. Conclusion

Mr. Tan Chi Jie

In our validation trial, we engineered a detection systemutilizing the YOLOv7 model specifically to recognize
microplastics. We measured the detector's precision
through a quartet of metrics: precision, recall, F1-score,
and the time required for estimation. The findings
indicate that the detector operates with an accuracy levelthat is deemed adequate for practical applications. It is
posited that refining the approach to annotation andtweaking the model's hyperparameters might pave theway to enhancing the model's detection accuracy.

Mr. Janthori Titan

He received his Bachelor ofEngineering in Robotic Engineeringand Automation System from theDepartmentofProduction
Engineering, Faculty of Engineering,
King Mongkut’s University ofTechnology North Bangkok in 2022.
He is currently a Master student atKyushu Institute of Technology andconducts research at Hayashi Laboratory.

References
1.

2.

3.

Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan
Mark Liao, YOLOv7: Trainable bag-of-freebies sets newstate-of-the-art
forreal-time
objectdetectors
arXiv:2206.14651,2022.3,9,10
Akyon, Fatih Cagatay and Altinuc, Sinan Onur andTemizel, and Alptekin， Slicing Aided Hyper Inference
and Fine-tuning for Small Object Detection 2022 IEEE
International Conference on Image Processing(ICIP),2022.966- 970
NVIDIA Tensort.

Dr. Ayumu Tominaga
He is a professor in Department ofCreative Engineering Robotics andMechatronics Course at National
Institute of Technology Kitakyushu
College. He received the Ph.D. (Dr.
Eng.) degree from Kyushu Institute ofTechnology in 2021. His researchinterests
includeIntelligent
mechanics, Mechanical systems and Perceptual
information processing.

Authors Introduction
Mr. Yuto Okawachi

He received his Bachelor’s degree inEngineering in 2023 from intelligent
and Control Systems, Kyushu Instituteof technology in Japan. He is currently
a Master student in Kyushu Institute ofTechnology and conducts research atHayashi Laboratory.

Mr. Shintaro Ogawa

He received his Master of Creative
Informatics from the Department ofComputer Science and Systems
Engineering, Kyushu Institute ofTechnology, Japan in 2023. He iscurrently a Doctoral student at
Kyushu Institute of Technology andconducts research at HayashiLaboratory.

Dr. Eiji Hayashi

He is a professor in the Department ofIntelligent and Control Systems at
Kyushu Institute of Technology. He
received the Ph.D. (Dr. Eng.) degreefrom Waseda University in 1996. His
research interests include Intelligent
mechanics, Mechanical systems andPercentual information processing.
He is a member of The Institute of Electrical andElectronics Engineers (IEEE) and The Japan Society ofMechanical Engineers (JSME).

He received his Bachelor’s degree inEngineering in 2022 from intelligent
and Control Systems, Kyushu
Institute of technology in Japan. He iscurrently a Master student in Kyushu
Institute of Technology and conducts
research at Hayashi Laboratory.

302

Dr. Satoko Seino

She is an associate professor in
Graduate School of Engineering,
Kyushu university. She received thePh.D.
(Eng.)
fromKyushu
University. Her research interests
includecoastalandriverine
environmental conservation.

303