www.nature.com/scientificreports

OPEN

Quantification of floating riverine
macro-debris transport using animage processing approach
Tomoya Kataoka* & Yasuo Nihei
A new algorithm has been developed to quantify floating macro-debris transport on river surfaces thatconsists of three fundamental techniques: (1) generating a difference image of the colour difference
between the debris and surrounding water in the CIELuv colour space, (2) detecting the debris pixels
from the difference image, and (3) calculating the debris area flux via the template matching method.
Debris pixels were accurately detected from the images taken of the laboratory channel and riverwater surfaces and were consistent with those detected by visual observation. The area fluxes were
statistically significantly correlated with the mass fluxes measured through debris collection. The
mass fluxes calculated by multiplying the area fluxes with the debris mass per unit area (M/A) weresignificantly related to the flood rising stage flow rates and agreed with the mass fluxes measured
through debris collection. In our algorithm, plastic mass fluxes can be estimated via calibration using
the mass percentage of plastics to the total debris in target rivers. Quantifying riverine macro-plastictransport is essential to formulating countermeasures, mitigating adverse plastic pollution impacts andunderstanding global-scale riverine macro-plastic transport.
Quantifying the transport of macro-debris floating on the world’s rivers, which are major sources of ocean debris,
is essential in formulating countermeasures to mitigate the adverse impacts of land-based loads. In particular, theadverse impacts on aquatic ecosystems of plastics containing toxic chemicals (e.g., persistent organic pollutants(POPs))1–4 are recognized as a serious concern in the global aquatic environment5–8. Many of the plastics in theoceans originate from land9, and thereafter, macro-plastics (>25 mm in diameter)10 are evenly broken down into
smaller plastic fragments known as meso- (5−25 mm in diameter) and micro-plastics (<5 mm in diameter)10 due
to photo- and thermo-oxidative degradation6. Micro-plastics are rarely removed from the aquatic environment
when released and are thus gradually transported far away due to ocean currents. Hence, to formulate countermeasures against oceanic plastic pollution, the macro-plastics in rivers must be efficiently captured before being
fragmented into smaller pieces and/or being released to the oceans. In particular, because most macro-plastics
float on the surface, it is important to understand how floating macro-debris is transported via rivers and thenreleased into the oceans.
Recently, a few studies have attempted to estimate the plastic waste emissions from land9,11. Jambeck et al.9,
for instance, estimated that 4.8 to 12.7 million tonnes of mismanaged plastic waste could be entering the oceansfrom 192 coastal countries in 2010 by considering the waste management level in each country and the coastal
population. In their estimate, 0.02 to 0.06 million MT of plastic waste can enter the Pacific Ocean from Japan.
Moreover, Lebreton et al.11 estimated that between 1.15 and 2.41 million tonnes of plastic waste could be entering
the oceans from rivers every year by applying a global model of plastic inputs from rivers based on waste management, population density and hydrological information. These two estimates of land-based plastic waste werebased on data on the mean waste generation in kilograms per inhabitant and per day in each country, althoughwaste generation differs spatially among the various regions of each country because of different land uses. Theseestimates, therefore, need to be verified in each country using more detailed data.
As a suitable method to verify these estimates, we focus on quantifying floating macro-debris based on fieldsurveys. Recently, several studies have investigated micro-plastic contamination in rivers around the world12–19,
while there have only been a few studies on collecting, monitoring and quantifying floating macro-debris basedon field surveys20–23. Nihei et al.20 collected floating macro-debris using a net (2.5 cm mesh) at the Noda Bridge
across the Edo River, Japan, and reported that the proportion of anthropogenic debris to the total debris was
Department of Civil Engineering, Faculty of Science and Technology, Tokyo University of Science, Chiba, 278-8510,
Japan. *email: tkata@rs.tus.ac.jp

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

1

www.nature.com/scientificreports/

www.nature.com/scientificreports

Figure 1. Images of the image processing flow steps of our algorithm. Panels (a–h) show the laboratory
experiment and river monitoring, respectively. Panels (a) and (e) are the original images, (b) and (f) are thesmoothed images, (c) and (g) are the difference images, and (d) and (h) are the binary images. The colour scaleof the difference images is shown at the upper-right of panels (c) and (g). The red and blue boxes in panels(c) and (g) are the analysis and template planes shown in Fig. 6, respectively. The grey box in panel (h) is thecalculated covered area.
approximately 6% by weight. In addition, Gasperi et al.21 reported that plastic debris represented between 0.8%
and 5.1% of the total macro-debris by weight and annually intercepted between 22 and 36 tons of floating plastic
debris through the use of floating debris-retention booms. Recently, a tablet computer application has been developed that harmonizes visual observations from debris monitoring to systematically gather comparable floating
macro-debris data involving many activities and institutes in various countries23. The application is useful to efficiently record characteristics of the debris (e.g., size, item, and number) on river surfaces. These previous methods
can gather reliable data on the quantity and characteristics of floating macro-debris, although they are labour
intensive and costly. In addition, these methods experience difficulties in sequentially observing the temporalfluctuations in the debris quantity.
Video monitoring of river surfaces could be an effective approach to safely and efficiently quantify floating
macro-debris transport. However, there have been very few studies on remote monitoring of riverine debris
except for our research24,25, although the macro-debris in marine environments has already been quantified using
webcams26–28, aerial photography29,30–35, satellite imagery36, and light detection and ranging (LIDAR) technology37. It is difficult to apply aerial photography, satellite imagery, and LIDAR technology because monitoring witha high temporal resolution is needed in rivers. In particular, because a large quantity of floating macro-debris regularly flows downstream during floods due to heavy rainfall38, quantifying the floating macro-debris that is transported under flood conditions is crucial to assess the land-based loads of riverine macro-debris in the oceans.
Traditionally, the fluxes of pollutants in rivers have been related to flow rates (i.e., the water volume per unit oftime)11,39,40. If a relationship between the floating macro-debris fluxes and flow rates can be established throughvideo monitoring, we can not only verify the estimates of plastic waste input to the oceans but also manage theland-based plastic waste emissions. Thus far, we have been attempting to develop a technique for quantifying thedebris transport on river surfaces using video data24,25. Our previous studies exhibit a weakness in the detection ofdebris pixels. We experienced difficulty in extracting the debris pixels of large items and conglomerations because
an edge detection algorithm was implemented. Thus, the detection accuracy is low because the pixels around theedge of the debris are detected, while the pixels around its centre are unable to be detected using this algorithm.
Here, we develop a new algorithm for quantifying floating riverine macro-debris transport, which consistsof three fundamental techniques. First, an image of the colour difference between the debris and water surface(difference image) is obtained. Next, debris pixels are detected by binarizing the difference image using a constantthreshold value. Finally, the area of the floating macro-debris that is passively transported per unit time (areaflux) is calculated by applying the template matching method. To verify the performance of our algorithm, bothlaboratory and river experiments were conducted. In the laboratory experiment, natural and anthropogenic items
floating along an open channel were filmed perpendicularly using a digital video camera installed at the top of thechannel to verify the detection performance for the items floating on the water surface. In the river experiment,
floating macro-debris in the Edo River was collected using a net, and the river surface on the upstream side of
the net was recorded using a camera. In both experiments, the area covered by debris (covered area) was evaluated by counting the pixels identified as debris to validate the detection accuracy. Furthermore, we examine theapplicability of the mass flux estimation method based on the area flux. This paper presents a basis for remotely
assessing the mass flux using digital video cameras in rivers and can be useful in formulating waste management
guidelines in different countries.

Results

Verification of the performance of our technique in the laboratory experiment. To investigate
the detection performance of floating macro-debris on the water surface, the covered area and transport velocityof the twenty items (Supplementary Fig. S1 and Table S1) were verified in the laboratory experiment. The twenty
items were successfully detected (Supplementary Fig. S2) by calculating the colour difference in the CIELuv colour space41 (see Supplementary Notes) between each original frame extracted from the video (Fig. 1a) and its

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

2

www.nature.com/scientificreports

www.nature.com/scientificreports/

Figure 2. Scatterplots of the covered area (a,c) and transport velocity (b) calculated by our algorithm versus
those measured by the observers. Panels (a) and (c) show the comparison of the covered areas obtained in thelaboratory and river experiments, respectively. Panel (b) shows the comparison between the transport velocities
measured by a stopwatch and calculated via image processing. The line y = x is added to each panel.

smoothed frame (Fig. 1b). The colour difference of the debris pixels was significantly high (Fig. 1c), and the debris
pixels were extracted by selecting a colour difference of 10 as the threshold value (Fig. 1d).
The covered areas of the twenty items were calculated by analysing the videos, and consequently, the calculated covered areas agreed well with their true projected areas measured by observers (n = 20, r = 0.996, andp < 0.001; Fig. 2a). In addition, the mean transport velocity calculated from all frames of each video was also consistent with that measured using a stopwatch (n = 20, r = 0.879, and p < 0.001; Fig. 2b). These results demonstratethat our algorithm enables us to measure the area flux using video data.

Verification of the applicability of our technique in the river experiment. The applicability of ouralgorithm was verified by processing 29 videos that were perpendicularly recorded using a digital video camera
at the Noda Bridge across the Edo River during the flooding events caused by the two typhoons that occurredin 2010 (see Supplementary Notes). The verification was conducted via two approaches. First, the covered areas
were compared with those calculated by extracting the debris pixels from each frame by visual observation. For
comparison, both covered areas in the grey boxed area shown in Fig. 1h were calculated from multiple frames
derived from four videos (videos 1, 2, 3 and 6; Supplementary Table S2) at one-second intervals. The covered area
calculated via image processing was very consistent with that calculated via visual observation (n = 372, r = 0.983,
and p < 0.001; Fig. 2c), indicating that our algorithm can be successfully applied to detect debris pixels.
Next, the relationship between the area and mass fluxes was investigated by comparing both fluxes based on
the 29 observations (Supplementary Table S2). The area flux was calculated by dividing the summation of thesurface area of the debris that is transported at the analysis frame rate (transport areas) across the width of the collection equipment (1 m; see Fig. 1e) by the filming time, which is referred to as LAo. In addition, the mass flux wasmeasured by dividing the total mass of the debris captured by the collection equipment by the collection duration,
which is referred to as LMo. Thus, both fluxes were defined as fluxes per unit width (m2/s/m and g/s/m, respectively). Comparing LAo and LMo based on the 29 observations (Fig. 3a), LAo was significantly correlated withLMo (n = 29, r = 0.447, and p < 0.05; Fig. 3a). The significant relationship makes it possible to estimate the massflux from the area flux. Nevertheless, as expected, LMo was much more variable than LAo. The large variance wascaused by the mass per unit area of the debris (i.e., the ratio of LMo to LAo; hereinafter referred to as M/A). M/A
depends on the debris composition and different volumes, shapes and weight densities of each debris item. The
mean M/A was 0.14 ± 0.05 kg/m2, and its relative error was 35% (Supplementary Table S2). Notably, the uncertainty was defined as the standard error of the mean (SEM), and thus the 95% confidence interval was calculated
by multiplying the SEM by the t value for a 5% two-tailed probability with 28 degrees of freedom (t0.05 = 2.048).

Estimation of the mass flux from the area flux. The significant relationship between LAo and LMo canbe used to estimate mass fluxes without collecting debris. The simplest estimation of the mass flux is to multiply
the area flux by the mean M/A (0.14 kg/m2), which is hereafter referred to as LMa. The root mean square error(RMSE) between LMo and LMa was 4 g/s/m. Here, to demonstrate the significance of the area flux, LMo was compared with LMa relative to the flow rate.
As with the traditional modelling of suspended matter40, LMo and LMa can be modelled by the following equation, which is referred to as the L-Q equation.
L = aQ b(1)where L and Q are the mass flux and flow rate, respectively. Coefficients a and b are determined by the leastsquares method using the mass fluxes (LMo and LMa) and flow rate. Coefficient b is the most important parameterfor determining the transport characteristics of the floating macro-debris in a river. Additionally, if the

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

3

www.nature.com/scientificreports/

www.nature.com/scientificreports

Figure 3. Scatterplots of the area flux (LAo) calculated with our algorithm versus the mass flux (LMo) measured
using the collection equipment (a), the relationship between LMo and mass flux (LMa) computed from LAo
with the flow rate (b), and the correlation between the mass fluxes (LMe-o, LMe-a) estimated by substituting theflow rate for the L-Q equations (c). In panel (a), the symbols are based on the collection date of floating macrodebris. The black and grey solid lines in panel (b) are the L-Q equations for LMo and LMa, respectively. The
symbols are shown in the box of each panel. The line y = x is added to panel (c).

coefficients can be correctly determined, the mass fluxes can be estimated from the flow rate without river video
monitoring.
First, both LMo and LMa at the rising stage, in which the difference in the water level per unit time (hereafter referred to as the water level rate) is greater than 0, were regressed using Eq. (1) (Fig. 3b), as summarised inTable 1 (p < 0.05). Moreover, neither LMo nor LMa were statistically significantly related to the flow rate at thefalling stage (i.e., water level rate < 0). The mass fluxes at the falling stage are independent of the flow rate because
the macro-debris can become trapped on the river bank when the water level drops42. In fact, the mass fluxes at
the falling stage were non-significantly regressed with the flow rate (p > 0.05; see Table 1).
The mass fluxes were estimated by applying the L-Q equations for LMo and LMa to the flow rates at the rising
stage, which are referred to as LMe-o and LMe-a, respectively. Both LMe-o and LMe-a were significantly correlatedwith LMo (Fig. 3c; n = 16, r = 0.757, and p < 0.001 for LMe-o; n = 16, r = 0.831, and p < 0.001 for LMe-a). In addition, the RMSE of LMe-a (1.6 g/s/m) was slightly smaller than the RMSE of LMe-o (4 g/s/m). The uncertainty inthe mass flux estimation is discussed below. Nevertheless, the estimation error of LMe-o was equivalent to that of
LMe-a; hence, this result demonstrates that the area fluxes calculated by our algorithm are useful for understanding the relationship between the mass fluxes and flow rate.

Discussion

Our algorithm enables us to safely and efficiently measure the area flux and transport velocity of the floating
macro-debris on a river surface without debris sampling. To date, riverine macro-debris has been less monitored,
although this necessity has rapidly increased according to the progress of plastic pollution in the world’s oceans
because rivers are major sources of plastic debris. Monitoring riverine debris is essential to reduce the emissions
from land to the ocean and to obtain scientific data related to the transport of floating riverine macro-debris.
Nevertheless, there are three main issues in evaluating the mass fluxes by river video monitoring: (1) evaluation of
the mass fluxes, (2) conversion of the plastics mass fluxes from the total mass fluxes, and (3) vertical distributionof the mass fluxes.
To evaluate the mass flux determined via river video monitoring, the M/A value of the macro-debris floatingon the surface of a target river was needed. In the Edo River, the mean M/A value of the debris was 0.14 kg/m2(dark grey line in Fig. 4) and had a 95% confidence interval ranging from 0.040 to 0.24 kg/m2 (light grey areain Fig. 4). To examine the M/A value of the debris in the Edo River, we investigated the M/A values of various
potential macro-debris used in the laboratory experiment. In addition, the M/A values of twenty dead plantpieces from dead plant conglomerations (items 18–20) were also measured. The M/A values of polyethylene terephthalate (PET) bottles, plastic flotation devices, aluminium cans, and glass vessels were on the order of 1, 1, 1,
and 10 kg/m2, respectively, while the M/A value of vinyl bags had a wider range compared with the M/A values ofthe other anthropogenic debris because the surface area of the vinyl bags varied greatly according to the flowing
state, such as a crumpled or bent state (Fig. 4). The M/A of natural debris had a wide range compared with that of
the anthropogenic debris. In particular, both M/A values of the dead plant pieces and dried wood greatly variedon the order of 0.1 and 10 kg/m2. Hence, M/A depends strongly on the composition of the floating macro-debris
in target rivers. In the Edo River, natural debris dominates the total debris (Supplementary Fig. S3), and its masspercentage was 69–100%. Thus, the mean M/A value ranged between 0.01 and 0.1 kg/m2. The mass percentage
and M/A values of the floating macro-debris collected in target rivers, therefore, should be measured; then, theM/A value of the total debris can be estimated by summarizing the M/A values weighted by the mass percentages.
Quantifying the floating riverine macro-plastics is essential to take countermeasures against plastic emissions into the oceans. A simple solution for quantifying floating macro-plastics is to calculate its mass flux bymultiplying the overall mass flux by the mass percentage of plastics. In the present study, the mass percentage
Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

4

www.nature.com/scientificreports

www.nature.com/scientificreports/δln (L)*b

ln(a)*c

b*d

n

r2

p

Rising

1.6

−101
± 25

18 ± 416

0.54

1.1 × 10−3

Falling2

9 ± 18−2 ± 313

0.043

5.0 × 10−1

Rising1.3−40 ± 20

8±4

16

0.25

4.7 × 10−2

Falling2−20 ± 20

3±3

13

0.079

3.5 × 10−1

Stage
LMo
LMa*a

Table 1. Model parameters of the mass fluxes. *aThe mass flux was estimated by multiplying LAo by the mean
M/A. *bThe error of the mass flux (δln(L)) is defined by Eq. (6). *cThe error of ln (a) (δln(a)) is defined by Eq. (3).
*d
The error of b (δb) is defined by Eq. (4).

Figure 4. M/A of the various types of potential floating macro-debris. The black bar is the range between the
maximum and minimum M/A values, and the black circle is its mean. The light grey area is the 95% confidence
interval of the M/A value of the macro-debris collected from the Edo River, and the dark grey line is its mean
(0.14 kg/m2).

of anthropogenic debris (not only plastics) ranged from 0% to 31% by weight. Its mean was 4.2 ± 1.6% (SEM).
Additionally, Gasperi et al.21 demonstrated that the mass percentage varied between 0.8% and 5.1%, and the highest plastics mass percentage was obtained by the floating debris-retention booms installed near the outlet of thelargest combined sewer outflows within the Paris metropolitan area43. This observation indicates that the masspercentage reflects the land use in the river basin. Hence, at present, to calculate the mass plastics fluxes from thetotal mass fluxes, we need to measure the mass percentage in target rivers, e.g., using booms.
Our algorithm might also be useful in measuring the mass flux of floating macro-plastics. The amount ofmacro-plastics on beaches has been sequentially monitored by applying image analysis focused on the colour ofthe plastics in photographs taken by webcams26,28,44. Because the colour of plastics differs greatly from the colourof natural debris, the characteristic plastic colour might resolve this issue of determining the mass flux of plastics.
In fact, as shown in Fig. 1c, g, the colour difference of plastics from the river surface was larger than that of thenatural debris. In the experiment classifying floating macro-debris into natural and anthropogenic debris by thecolour difference (e.g., >80: anthropogenic debris) using the 29 videos of the Edo River, the area percentage ofanthropogenic debris ranged from 0% to 44%, and its mean was 4.5 ± 1.5% (SEM), consistent with the mean
mass percentage (i.e., 4.2 ± 1.6%). However, it is difficult to statistically compare the mass and area percentages
because the mass percentages were very low in the 29 observations (Supplementary Table S2). Meanwhile, toquantify the transport of floating macro-plastics, we need to develop a detection technique of plastics in the
future. The detection of plastics can be improved using cameras extended beyond the visible band37,45–48. Recently,
several researchers have studied the application of near-infrared hyperspectral cameras to identify plastics in theenvironment45–48. If hyperspectral imaging is available in rivers, macro-plastics can be distinguished from themacro-debris flowing on the river surface. Because hyperspectral imaging can identify polymer materials, themass fluxes of macro-plastics can be calculated from the area fluxes using the M/A value of each polymer material
and/or item. Thus, the use of a hyperspectral camera might also resolve the issues of the difference between thearea and mass percentages. A combination of our algorithm and hyperspectral imaging, therefore, could be auseful tool to measure the mass fluxes of macro-plastics in the future.
Meanwhile, when quantifying the mass flux of macro-plastics using our algorithm, the uncertainty of the massflux would propagate due to several parameters, such as the M/A, coefficients of Eq. (1) (i.e., a and b) and masspercentage of macro-plastics. In the quantification of the mass flux, the error of M/A initially propagates to theerror of LMa, subsequently to the error of the coefficients of Eq. (1) (a and b), and finally to the mass flux estimation of macro-plastics. From the mass flux calculation (LMa = (M /A) × LAo ), the error of LMa (δLMa) is estimated using the following equation:
δLMa = δM /A × LAo

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1(2)

5

www.nature.com/scientificreports

www.nature.com/scientificreports/

where δM /A is SEM of M/A, namely, 0.05 kg/m2. By substituting the mean of LAo (0.022 m2/s/m; see Supplementary
Table S2) into Eq. (2), δLMa was 1.1 g/s/m smaller than the RMSE between LMo and LMa (4 g/s/m). Meanwhile, theerror of LMa would propagate to the coefficients (a and b) of Eq. (1). From the linearization of Eq. (1), the uncertainties of a and b are estimated as follows:δ ln(a) = δ ln(L) ∑ (ln(Q))2 /∆(3)δb = δ ln(L) N /∆(4)

∆ = N ∑ (ln(Q))2 − (∑ln(Q))2(5)

where N is the number of data points used to determine a and b. The lower and upper bounds of the summation
are 1 and N, respectively; δ ln(a) and δb represent the errors of a and b, respectively. The uncertainty of LMa (δ ln(L))
is estimated using the following equation:
δ ln(L) =1∑ (ln(LMo) − ln(a) − b ln(Q))2
N−2

(6)

where ln(LMo) and ln(Q) are the natural logarithms of LMo and Q, respectively. Consequently, the uncertainties ofδ ln a and δb are shown in Table 1. The uncertainties of a and b for LMa were equivalent to those for LMo. On theother hand, the SEM of the mass percentage was 1.6%, and hence 38% (=1.6%/4.2%) of its relative error would
propagate to the mass flux estimation of macro-plastics. Nevertheless, we postulate that our algorithm based on
the river video monitoring data plays a role as a substitute tool for debris collection from the river surface because
the RMSE of LMe-a (1.6 g/s/m) was slightly smaller than LMe-a (4 g/s/m).
Furthermore, macro-debris could be transported via river water with a vertical distribution due to turbulent
flow. The macro-debris flowing below the river surface cannot be measured by our algorithm. This condition is alimitation of river surface monitoring. One approach to resolving this issue is to consider a vertical distribution
model according to the type of macro-debris. To date, vertical distribution models of various materials, such as
sediments49,50, nutrients51, soil organic carbon52 and plastic debris42, have been investigated in rivers. These consider the influences of water flow, wind, vertical mixing, and buoyance/settling depending on the specific gravityof the materials. The goal of this study was to evaluate the mass fluxes of macro-plastics, which are secondary
sources of micro-plastics. The total mass fluxes of macro-plastics in river cross-sections will be evaluated by considering a vertical distribution model in the future42.
In the present study, an algorithm for quantifying the transport of floating macro-debris by river video monitoring has been described. Applying our algorithm would be helpful to more easily and safely establish the relationship of the mass fluxes and flow rates compared with the collection of floating macro-debris, although several
field surveys will be needed to accurately estimate the mass fluxes. The establishment of this relationship permits
us to estimate the mass flux from the flow rate observed at each station. In particular, quantifying the mass flux atthe rising stage is essential to evaluate the emission of floating macro-debris from rivers because the mass fluxes
at the rising stages were found to be one to two orders of magnitude greater than those at the falling stages/undernormal flow conditions (Supplementary Table S2). In the future, we will quantify the floating macro-debris inrivers by applying our algorithm to videos recorded under various flow conditions.

Conclusions

Floating macro-debris transport can be successfully quantified by monitoring the river surface using a digital video camera and applying an image processing technique based on the colour difference of the floating
macro-debris. Our algorithm can capture the area flux of the debris, which is its covered area per unit time. The
area flux (LAo) can be converted into the mass flux (LMa) using the mass per unit area (M/A) of the floating
macro-debris. LMa was significantly regressed with the flow rate in the river at the rising stage and was consistent
with the mass flux (LMo) measured by collecting floating macro-debris. If the mass percentage of macro-plastics
to the total debris in a river is obtained, the mass flux of floating macro-plastics can be quantified. Quantifying
the floating macro-plastics in rivers is essential to formulate countermeasures and mitigate the adverse impactsof land-based loads, such as plastic pollution. Our algorithm could be the basis for understanding the transport of floating riverine macro-plastics around the world and consequently can contribute to more effective
countermeasures.

Methods

Fundamental technique for generating difference images from video data. The fundamental
technique for generating difference images from video data consists of three steps (Fig. 5): (1) dividing a video file(e.g., m2ts, mp4, and mov) into multiple frame images (e.g., jpeg, and png; Fig. 1a,e, respectively), (2) generating asmoothed image from each frame image (Fig. 1b,f), and (3) computing the colour difference between the originaland smoothed images in the CIELuv colour space converted the RGB colour space (Fig. 1c,g).
In the first step, a video file is divided into multiple frame images based on the analysis frame rate. First, thedivided frame images are smoothed using a uniform box filter (5 px × 5 px) to remove noise. The divided frame
image after smoothing is used as the original frame image. In the present study, the analysis frame rate was arbitrarily selected as 5 frames per second (fps) considering the flow of floating macro-debris in the video. Video data
are often obtained at a high filming frame rate (e.g., 29.97 fps). The computational load could be excessive if the
Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

6

www.nature.com/scientificreports

www.nature.com/scientificreports/

Figure 5. Flowchart for quantifying the transport of floating riverine macro-debris.

analysis frame rate is too high when the video data are divided. Conversely, if the analysis frame rate is too low, wemight not correctly detect floating macro-debris.
To effectively extract the pixels of floating macro-debris, the smoothed image is generated using the originalframe image (Fig. 1b,f). For smoothing, the median filter is applied to the original frame image, and the window
size of the filter is 200 px × 200 px in the present study. The window size is arbitrary, and we determined thiswindow size through trial and error in the present study. If the window size is too small, it is difficult to extract
floating macro-debris because the colours in the original frame image are similar to those in the smoothed image.
To quantify the difference in colour between the original and smoothed images, the RGB colour space of both
images is converted into the CIELuv colour space (see Supplementary Notes). The CIELuv colour space attempts
to obtain a perceptual uniformity of the colour difference in the three-dimensional space (i.e., L*, u*, and v*)41.
The colour difference ΔE is expressed by the Euclidean distance between two points in the CIELuv colour spaceas follows:
ΔE =(L1⁎ − L 2⁎)2 + (u1⁎ − u 2⁎)2 + (v1⁎ − v2⁎)2(7)

where the subscripts of 1 and 2 denote the original and smoothed images, respectively. The colour difference
between the original and smoothed images is computed using Eq. (7) (Fig. 1c,g).

Fundamental technique for detecting the pixels of floating macro-debris from the difference
images. The pixels of macro-debris can be extracted by determining the appropriate constant threshold value

in the difference image (hereafter referred to as the binary image) (Fig. 1d,h). In the present study, the thresholdvalue is 10 for the laboratory experiments and 20 for the river video monitoring data. Notably, the application ofa higher threshold value in river video monitoring is a better approach because the waves generated on the water
surface are potentially mis-detected. The area covered by floating macro-debris is computed by multiplying thenumber of debris pixels in the binary image by the area of a single pixel. For each video, the optimum thresholdvalue must be selected by trial and error compared with the covered area calculated through visual observation(Fig. 2a,c).

Fundamental technique for measuring the area flux and transport velocity of floating macro-debris
using the difference images. The flux and velocity of floating macro-debris are measured using mul-

tiple difference images. In the present study, the flux was defined as the area of floating macro-debris transported per unit time and unit width (e.g., m2/s/m). In general, the flux is often evaluated as the mass of floating
macro-debris (e.g., g/s/m). However, it is difficult to directly evaluate the mass flux from two-dimensional video
data because the vertical size and specific gravity of the debris are unknown. First, the area flux is evaluated using
our algorithm.
The area flux and velocity can be computed by applying the template matching method to two difference
images at steps tk and tk+1. Note that the subscript k is a time step index. First, as shown in Fig. 6, the template
plane is defined from the difference image ΔE at step tk in the analysis plane, where x and y are the coordinates inScientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

7

www.nature.com/scientificreports

www.nature.com/scientificreports/

Figure 6. Schematic image of the area flux calculation by applying the template matching method. The template
plane (bold outline area) is defined in the analysis plane (light grey area) at t = tk, and its position is arbitrary. I
and J are the numbers of pixels in the x and y directions, respectively. The scanning plane (bold dashed outlinedarea) is searched in the analysis plane at t = tk+1 using the template matching method (Eq. (8)). The differential
area between the template and scanning planes is the plane used to calculate the area flux (grey area) and iscalled as calculation area. The number of debris pixels in the calculation plane, namely, the transport area
(black area), is counted. The area flux is calculated with Eq. (11) using the number of debris pixels. Note that thefloating macro-debris is shown by the grey and black dashed outline areas at t = tk and t = tk+1, respectively.

the lateral and flow directions, respectively, and (x 0, y0 ) are the origin coordinates of the template plane in thex − y plane. Notably, the size of the template plane should be as large as possible because the precision of templatematching method is increased by tracking many objects in the template plane during ∆t (=tk+1 − tk) corresponding to 1/(analysis frame rate). The template plane is then searched in the difference image at step tk+1 by calculating the zero-mean normalized cross-correlation (ZNCC)53 as follows:
RZNCC (xs , ys , tk) ={()(T (x , y , t ) − T )}∑ ∑ (S(x + x , y + y , t ) − S ) ∑ ∑ (T (x , y , t ) − T )1 I−1∑ Jj −= 0∑ i = 0 S(xi + xs , yj + ys , tk + 1) − S

J−1
j =0

I−1
i=0

S =

i

2

s

j

s

k+1

J−1
j =0

i

I−1
i=0

j

k

2

i

j

k

(8)

1 J − 1I − 1∑∑S(xi + xs , yj + ys , tk+1)
IJ j =0i =0(9)

1 J − 1I − 1∑∑T (xi, yj , tk)
IJ j =0i =0

(10)

T =

where xi and yj are the lateral and flow positions at pixel (i, j), and T is ΔE at position (xi , yj ) and step tk in thetemplate plane. In addition, S is ΔE at position (xi + xs , yj + ys ) and step tk+1 in the scanning plane, and (xs , ys ) are
the origin coordinates of the scanning plane. Note that the size of the scanning plane is the same as that of thetemplate plane. As shown in Eqs. (9) and (10), S and T are the spatial averages of the pixel values in the scanningand template planes, respectively. I and J are the maxima of the pixels in the x and y directions of the template
plane, respectively. For the laboratory channel and river, the analysis and template planes are shown in Fig. 1c,g,
respectively.
The area flux LAo is calculated by the following equations:J − 1I − 1

LAo = ∑∑ rC(xi + xsmax , yj + ysmax , tk+1)/∆t
j = 0i = 0

u = (xsmax − x 0)/∆tv = (ysmax − y0 )/∆t




(11)

(12)

where C is the pixel value in the calculation plane of the binary images, which is the portion of the scanning plane
except the range that overlaps the template plane at step tk (Fig. 6), and r is the area of a pixel. Note that the calculation plane is defined in the binary image after detecting the debris pixels (see Fig. 5). When RZNCC is maximized,(xs , ys ) becomes (xsmax , ysmax ). In addition, u and v are the transport velocities in the lateral and flow directions,
Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

8

www.nature.com/scientificreports/

www.nature.com/scientificreports

respectively. In the present study, the lateral transport in the x-axis was disregarded because the y-axis in theframes is almost parallel to the flow direction. Finally, the area fluxes in all frames are averaged over the filming
time.

Laboratory experiment in the open channel. To evaluate the performance of our algorithm, we conducted a laboratory experiment by filming twenty natural and anthropogenic items (see Supplementary Notes
and Table S1) floating on the water surface of the open channel in our laboratory on 6 May and 13 September
2019. The open channel is 1.0 m wide, 1.8 m high, and 20 m long (Supplementary Fig. S4). A pump and flow meter
were installed upstream of the open channel; thus, the flow rate could be adjusted in the range of 0.0001 m3/s to0.2535 m3/s. In this experiment, the flow rate was 0.055 m3/s, and thus, the uniform water depth was set to 0.58 m.
Videos were recorded perpendicularly using a video camera (HDR-XR550V; Sony, Japan) fixed at the lateralcentre on the upper edge of the channel. The filming speed or frame rate was 29.97 fps, and the camera resolution
was 1920 × 1080.
Collection and video monitoring of the floating macro-debris in the Edo River. The collection and video monitoring of floating macro-debris were performed at the Noda Bridge across the Edo River(Supplementary Fig. S5) during two typhoon events in 2010 (Supplementary Notes) to verify the in situ applicability of our algorithm.
Floating macro-debris was collected using the collection equipment for 1–5 min (see Supplementary Table S2).
Simultaneously, a video of the river surface was perpendicularly recorded from the bridge using a video camera(HDR-XR550V; Sony, Japan). The video frame rate was 29.97 fps, and the camera resolution was 1920 × 1080.
The collection equipment was included in the filmed video footage. The spatial range of the video camera wasapproximately 3.0 m in the flow direction and 4.0 m in the lateral direction. Because the location of the collection equipment in the river surface changed horizontally and vertically according to the flow conditions during
recording, the horizontal location was captured (see Supplementary Notes). Additionally, the spatial resolutionchanged depending on the vertical location and water level, and the mean spatial resolution recognizable asdebris was 8.0 cm2/px within a range of 5.9–14 cm2/px (see Supplementary Notes), corresponding to the size ofmacro-debris (>2.5 cm10). By analysing the area upstream of the collection equipment in the video (see Fig. 1e),
we can compare the area flux calculated via video monitoring with the mass flux measured by debris collection.
Statistical analysis. Correlation and regression analyses were conducted using R version 3.3.2 (2016–10–31).

In the correlation analysis, the Pearson correlation coefficient was calculated and evaluated to identify any statistically significant relationships between the mass and area fluxes and between the observed and estimated mass
fluxes. Moreover, regression analysis was conducted to test the statistical significance of the relationship between
the mass flux and flow rate. Both statistical analyses were evaluated at the 95% confidence level.

Received: 15 October 2019; Accepted: 10 January 2020;
Published: xx xx xxxx

References

1. Mato, Y. et al. Plastic resin pellets as a transport medium for toxic chemicals in the marine environment. Environ. Sci. Technol. 35,
318–324 (2001).
2. Teuten, E. L. et al. Transport and release of chemicals from plastics to the environment and to wildlife. Philos. Trans. R. Soc. Lond. B.
Biol. Sci. 364 (2009).
3. Ogata, Y. et al. International Pellet Watch: Global monitoring of persistent organic pollutants (POPs) in coastal waters. 1. Initial
phase data on PCBs, DDTs, and HCHs. Mar. Pollut. Bull. 58, 1437–1446 (2009).
4. Browne, M. A., Niven, S. J., Galloway, T. S., Rowland, S. J. & Thompson, R. C. Microplastic moves pollutants and additives to worms,
reducing functions linked to health and biodiversity. Curr. Biol. 23, 2388–2392 (2013).
5. Thompson, R. C. et al. Lost at sea: where is all the plastic? Science 304 (2004).
6. Andrady, A. L. Microplastics in the marine environment. Mar. Pollut. Bull. 62, 1596–1605 (2011).
7. Cole, M., Lindeque, P., Halsband, C. & Galloway, T. S. Microplastics as contaminants in the marine environment: a review. Mar.
Pollut. Bull. 62, 2588–2597 (2011).
8. Gall, S. & Thompson, R. The impact of debris on marine life. Mar. Pollut. Bull. 92, 170–179 (2015).
9. Jambeck, J. R. et al. Plastic waste inputs from land into the ocean. Science 347, 768–771 (2015).
10. Kershaw, P. J. & Rochman, C. M. Sources, Fate and Effects of Microplastics in the Marine Environment: A Global. Assessment. Rep.
Stud. 90, 97 (2016).
11. Lebreton, L. C. M. et al. River plastic emissions to the world’s oceans. Nat. Commun. 8, 15611 (2017).
12. Mani, T., Hauk, A., Walter, U. & Burkhardt-Holm, P. Microplastics profile along the Rhine River. Sci. Rep. 5, 17988 (2014).
13. Eerkes-Medrano, D., Thompson, R. C. & Aldridge, D. C. Microplastics in freshwater systems: a review of the emerging threats,
identification of knowledge gaps and prioritisation of research needs. Water Res. 75, 63–82 (2015).
14. McCormick, A. R. et al. Microplastic in surface waters of urban rivers: concentration, sources, and associated bacterial assemblages.
Ecosphere 7, e01556 (2016).
15. Horton, A. A., Walton, A., Spurgeon, D. J., Lahive, E. & Svendsen, C. Microplastics in freshwater and terrestrial environments:
Evaluating the current understanding to identify the knowledge gaps and future research priorities. Sci. Total Environ. 586, 127–141(2017).
16. Brennholt, N., Heß, M. & Reifferscheid, G. In Freshwater Microplastics: Emerging Environmental Contaminants? (eds.Martin Wagner& Scott Lambert) 239–272 (Springer International Publishing, 2018).
17. Dris, R., Gasperi, J. & Tassin, B. in Freshwater Microplastics: Emerging Environmental Contaminants? (eds. Martin Wagner & Scott
Lambert) 69–83 (Springer International Publishing, 2018).
18. Wu, C., Zhang, K. & Xiong, X. in Freshwater Microplastics: Emerging Environmental Contaminants? (eds. Martin Wagner & Scott
Lambert) 85–99 (Springer International Publishing, 2018).
19. Kataoka, T., Nihei, Y., Kudou, K. & Hinata, H. Assessment of the sources and inflow processes of microplastics in the riverenvironments of Japan. Environ. Pollut. 244, 958–965 (2019).

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

9www.nature.com/scientificreports/

www.nature.com/scientificreports

20. Nihei, Y., Shirakawa, A., Suzuki, T. & Akamatsu, Y. Field measurements of floating-litter transport in a large river under flooding
conditions and its relation to DO environments in an inner bay. Journal of Japan Society of Civil Engineers, Ser. B2 (Coastal
Engineering) 66, 1171–1175 (2010).
21. Gasperi, J., Dris, R., Bonin, T., Rocher, V. & Tassin, B. Assessment of floating plastic debris in surface water along the Seine River.
Environ. Pollut. 195, 163–166 (2014).
22. Castro-Jiménez, J., González-Fernández, D., Fornier, M., Schmidt, N. & Sempéré, R. Macro-litter in surface waters from the Rhone
River: Plastic pollution and loading to the NW Mediterranean Sea. Mar. Pollut. Bull. 146, 60–66 (2019).
23. González-Fernández, D. & Hanke, G. Toward a Harmonized Approach for Monitoring of Riverine Floating Macro Litter Inputs to
the Marine Environment. Front. Mar. Sci. 4 (2017).
24. Onoi, T. & Nihei, Y. A new technique for evaluating floating-litter transport using temporal variation rate of water elevation. Proc. of
18th IAHR-APD Congress, 6p. (2012).
25. Minami, M., Onoi, T., Nihei, Y., Kataoka, T. & Hinata, H. An automatic and continuous monitoring system for floating-littertransport in river and its application to field survey in Mogami River. Proc. of the 36th IAHR World Congress, 8p. (2015).
26. Kako, Si, Isobe, A. & Magome, S. Sequential monitoring of beach litter using webcams. Mar. Pollut. Bull. 60, 775–779 (2010).
27. Kako, S. i. et al. Sequential webcam monitoring and modeling of marine debris abundance. Mar. Pollut. Bull. (2018).
28. Kataoka, T., Hinata, H. & Kako, Si A new technique for detecting colored macro plastic debris on beaches using webcam images and
CIELUV. Mar. Pollut. Bull. 64, 1829–1836 (2012).
29. Deidun, A., Gauci, A., Lagorio, S. & Galgani, F. Optimising beached litter monitoring protocols through aerial imagery. Mar. Pollut.
Bull. 131, 212–217 (2018).
30. Bao, Z., Sha, J., Li, X., Hanchiso, T. & Shifaw, E. Monitoring of beach litter by automatic interpretation of unmanned aerial vehicle
images using the segmentation threshold method. Mar. Pollut. Bull. 137, 388–398 (2018).
31. Kako, Si, Isobe, A. & Magome, S. Low altitude remote-sensing method to monitor marine and beach litter of various colors using aballoon equipped with a digital camera. Mar. Pollut. Bull. 64, 1156–1162 (2012).
32. Kataoka, T., Murray, C. C. & Isobe, A. Quantification of marine macro-debris abundance around Vancouver Island, Canada, basedon archived aerial photographs processed by projective transformation. Mar. Pollut. Bull. 132, 44–51 (2018).
33. Moy, K. et al. Mapping coastal marine debris using aerial imagery and spatial analysis. Mar. Pollut. Bull. 132, 52–59 (2018).
34. Veenstra, T. S. & Churnside, J. H. Airborne sensors for detecting large marine debris at sea. Mar. Pollut. Bull. 65, 63–68 (2012).
35. Mace, T. H. At-sea detection of marine debris: overview of technologies, processes, issues, and options. Mar. Pollut. Bull. 65, 23–27(2012).
36. Topouzelis, K., Papakonstantinou, A. & Garaba, S. P. Detection of floating plastics from satellite and unmanned aerial systems(Plastic Litter Project 2018). Int. J. Appl. Earth. Obs. 79, 175–183 (2019).
37. Ge, Z., Shi, H., Mei, X., Dai, Z. & Li, D. Semi-automatic recognition of marine debris on beaches. Sci. Rep. 6, 25759 (2016).
38. Kataoka, T., Hinata, H. & Nihei, Y. Numerical estimation of inflow flux of floating natural macro-debris into Tokyo Bay. Estuar.
Coast. Shelf Sci. 134, 69–79 (2013).
39. Borah, D. K. & Bera, M. Watershed-scale hydrologic and nonpoint-source pollution models: review of applications. Trans. ASAE 47,
789 (2004).
40. Nizzetto, L., Bussi, G., Futter, M. N., Butterfield, D. & Whitehead, P. G. A theoretical assessment of microplastic transport in rivercatchments and their retention by soils and river sediments. Environ. Sci.-Proc. Imp. 18, 1050–1059 (2016).
41. Fairchild, M. D. Color Appearance Models. Second edition edn, (John Wiley & Sons Ltd, 2013).
42. Kooi, M., Besseling, E., Kroeze, C., van Wezel, A. P. & Koelmans, A. A. In Freshwater Microplastics: Emerging Environmental
Contaminants? (eds Martin Wagner & Scott Lambert) 125–152 (Springer International Publishing, 2018).
43. Gasperi, J. et al. Priority pollutants in urban stormwater: Part 2 – Case of combined sewers. Water Res. 46, 6693–6703 (2012).
44. Kako, Si, Isobe, A., Kataoka, T. & Hinata, H. A decadal prediction of the quantity of Plastic marine debris littered on beaches of theEast Asian marginal seas. Mar. Pollut. Bull. 81, 174–184 (2014).
45. Acuña-Ruz, T. et al. Anthropogenic marine debris over beaches: Spectral characterization for remote sensing applications. Remote
Sens. Environ. 217, 309–322 (2018).
46. Goddijn-Murphy, L., Peters, S., van Sebille, E., James, N. A. & Gibb, S. Concept for a hyperspectral remote sensing algorithm forfloating marine macro plastics. Mar. Pollut. Bull. 126, 255–262 (2018).
47. Serranti, S., Palmieri, R., Bonifazi, G. & Cózar, A. Characterization of microplastic litter from oceans by an innovative approachbased on hyperspectral imaging. Waste Manage. 76, 117–125 (2018).
48. Shan, J. et al. Simple and rapid detection of microplastics in seawater using hyperspectral imaging technology. Anal. Chim. Acta
1050, 161–168 (2019).
49. Merritt, W. S., Letcher, R. A. & Jakeman, A. J. A review of erosion and sediment transport models. Environ. Modell. Softw. 18,
761–799 (2003).
50. Aksoy, H. & Kavvas, M. L. A review of hillslope and watershed scale erosion and sediment transport models. CATENA 64, 247–271(2005).
51. Drewry, J. J., Newham, L. T. H., Greene, R. S. B., Jakeman, A. J. & Croke, B. F. W. A review of nitrogen and phosphorus export towaterways: context for catchment modelling. Mar. Freshwater Res. 57, 757–774 (2006).
52. Jobbágy, E. G. & Jackson, R. B. The vertical distribution of soil organic carbon and its relation to climate and vegetation. Ecol. Appl.
10, 423–436 (2000).
53. Di Stefano, L., Mattoccia, S. & Tombari, F. ZNCC-based template matching using bounded partial correlation. Pattern Recogn. Lett.
26, 2129–2134 (2005).

Acknowledgements

We are grateful to many students at the hydraulics laboratory of the Tokyo University of Science for theirassistance in collecting floating macro-debris. This study was supported by Dow Packaging and Specialty Plastics,
USA, JSPS KAKENHI (Grant Number: 17H04937), the River Fund of the River Foundation (Grant Number:
2019-5211-050), Japan and Tokyo University of Science Grant for President’s Research Promotion.

Author contributions

T.K. and Y.N. created the concept for quantifying floating riverine macro-debris transport. T.K. developed thealgorithm for quantifying floating macro debris using videos, and Y.N. designed the surveys for collecting floating
macro-debris and recorded the videos at the Edo River. T.K. and Y.N. wrote the manuscript, and all authors
reviewed and evaluated the manuscript.

Competing interests

The authors declare no competing interests.

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

10www.nature.com/scientificreports/

www.nature.com/scientificreports

Additional information

Supplementary information is available for this paper at https://doi.org/10.1038/s41598-020-59201-1.
Correspondence and requests for materials should be addressed to T.K.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps andinstitutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International
License, which permits use, sharing, adaptation, distribution and reproduction in any medium or
format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in thisarticle are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to thematerial. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the
copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
© The Author(s) 2020

Scientific Reports |(2020) 10:2198 | https://doi.org/10.1038/s41598-020-59201-1

11