Weed Detection Using Drone For Smart Agriculture• significance of research⎯ Increasing crop yields

Deep Learning Neural Networks Based Weed Detection
Using Aerial RGB And NIR Image Processing Methods For
Smart Agriculture⎯ Reducing the indiscriminate use of pesticides⎯ Saving labor to manage farms
spray
herbicides

JIAQI ZOU

• Weeding process

Advanced Network Architecture Research Laboratory
Graduate School of Information Science and Technology
Osaka University

1. Drone captures farm images at high altitude
2. Input the image into the neural network and find the positions

2023/8/21

3. Spray this area with the corresponding herbicide

Proposal Approach —— Near-Infrared （NIR） Images

Challenges in Implementing

NIR Image

1. Distinguish between weeds and crops
2. Aerial images of weeds are blurred
3. Weed labeling is hard

Proposal Approach⎯

Wavelength range of 700 to 1400 nanometers.

⎯Determine the health of plants⎯

The reflectance of NIR for different plants is different⎯

Normalized Difference Vegetation Index (NDVI)⎯

1. Use near-infrared images
2. Super-resolution
3. Self-supervised learning (contrastive learning)It's hard to see.

!"# $ #%&"#$% = !"' (#%&⎯

This research using Red, Green, NIR 3-channels (RGN) images⎯Problems:Not enough RGN images

Proposal Approach —— Super Resolution

Proposal Approach —— Contrastive Learning

Photo-Realistic Single Image Super-Resolution Using a
Generative Adversarial Network (SRGAN)

Emerging Properties in Self-Supervised
Vision Transformers

1.

Low-resolution images (LR) are obtained by down-sampling high-resolution images (HR)⎯2.

Input LR into the generator network to get super resolution images (SR)3.

Training the discriminator to distinguish between SR and HR

4.

Training generator generates SR to confuses the discriminator⎯

Same category should be in the same positionin the feature space
The student networks ("!! ) and teacher network("!" ) are two Vits, turn input image into a vector
by self attention⎯x1 input into "!! is the local picture, x2 input
into "!" is the global picture

LRSR

⎯

Preventing Collapse
1.
2.

⎯

Momentum update
Centering

Deeper MLP

Proposed Architecture
1.
2.
3.
4.

Input RGN images taken by drones
Improvement of image resolution by generators network in SRGAN
Split the super-resolution image into pieces of 480x480
Weed Segmentation network is student network obtained by DINO
training
5. The segmentation of weeds is obtained based on the self-attention ofVision Transformer

Dataset For Weeds Segmentation
4 types of RGN images taken by drones
403 images⎯ 33 Weeds and plants in white sheet⎯ 80 Weeds and plants in blue sheet⎯ 33 Weeds and plants in black sheet

Dataset For Super Resolution⎯

Shooting with Survey3W⎯

Obtained by cropping 242 2k
images⎯1500 480p RGN images⎯main subject is a plant

Experimental Results and Conclusions 1
1. Pre-training on ImageNet
2. Fine-tuning on local datasets⎯⎯⎯ 227 Weeds and plants in realenvironment⎯

Pre-training is very effective
Red and green channels converge fasterthan NIR channels
Peak signal-to-noise ratio（PSNR）

Cropped out the weeds and paste themto different backgrounds⎯ 400 480p weeds in the center images

Experimental Results and Conclusions 2

LRHRSR

Experimental Results and Conclusions 3

1. Pre-training on ImageNet
2. Fine-tuning on local datasets⎯ Two scales of ViT were trained⎯ Testing on three different types

Performance of ViT-S/8 after fine-tuning on
simple background

Performance of ViT-S/8 after fine-tuning on
complex context

of test sets
⎯ Comparison of before and afterfine-tuning using local datasets
Performance of ViT-S/8 before fine-tuning
on real environment

Performance of ViT-S/8 before fine-tuning
on real environment

Look Forward To The Future• Larger scale neural networksØThe larger the model of the transformer, the higher the vector
dimension and the more accurate the features.

• Creating larger scale datasetØtransformer requires more training data to train self-attention due tolack of inductive bias

• Exploring better self-supervised learning models

Appendix 2

Appendix 1