Collaborative Multi-Task Learning
across Internet Edges
with Device-to-Device Communications
Available Supervise Labels (y, z, w) for xInput(x)PersonObject Class(y)Color(z)Size(w)

Person A

Apple

N/A

N/A

Person B

N/A

PurpleN/A

Person C

N/A

N/A

Big

Device-to-Device Contacts for Communication

Collaborative Model Trainingx

TrainedModel

Brainstormer(Apple, Purple, Big)

yzw

Ryusei Higuchi, Hiroshi Esaki, Hideya Ochiai
The University of Tokyo, Japan1

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion

2

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion3

Toward On-Device Training• Machine Learning has evolved with cloud computing.
• Nowadays :• AI Chips are available for 10 USD.
• Machine Learning shifts onto IoT Edges.

• Issues:• Learning on an Edge may lead to model overfit.

AI Chip (10USD)
Mode Switch
0: Neutral
1: Train
2: PredictLabel for Supervising (Binary)
01234…FPrediction (Binary)
01234…F• Approach of this issue:• Collaborative Training with Device-to-Device
Communication.

• Focus of this research:• Training a Multi-Task Model• Using a Standard Benchmark Dataset

Edge AI Testbed4

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion

5

Previous Studies (1/5):

Wireless Ad Hoc Federated Learning (WAFL) [1]with Device-to-Device Communication
1. Each node individually trains
its ML model using its local data.

Data

Data

2. Each node encounters the other.
3. They can communicate with local
wireless communication media
such as Wi-Fi Ad Hoc modeor Bluetooth

6[1] Ochiai, Hideya, et al. "Wireless ad hoc federated learning: A fully distributed cooperative machine learning." arXiv preprint arXiv:2205.11779 (2022).

Previous Studies (2/5):

Wireless Ad Hoc Federated Learning (WAFL) [1]with Device-to-Device Communication
1. Each node individually trains
its ML model using its local data.

2. Each node encounters the other.
3. They can communicate with local
wireless communication media
such as Wi-Fi Ad Hoc modeor Bluetooth

4. They exchange and aggregate
the models to develop a new model.
5. This enables collaborative training.
7[1] Ochiai, Hideya, et al. "Wireless ad hoc federated learning: A fully distributed cooperative machine learning." arXiv preprint arXiv:2205.11779 (2022).

Previous Studies (3/5):

Wireless Ad Hoc Federated Learning (WAFL) [1]with Device-to-Device Communication
WAFL has achieved almost thesame performanceas conventional federated learning.
10.0-11.5% Improvement

Accuracy of Self-Trainstatic_line

static_tree

static_ringstar static_dense
24

Random Waypoint Mobility

Ochiai, Hideya, et al. "Wireless ad hoc federated learning: A fully distributed cooperative machine learning." arXiv preprint arXiv:2205.11779 (2022).
8Previous Studies (4/5):
Extensions and Variations of WAFL (1/2)
WAFL-Autoencoder[2]WAFL-Vision Transformer [1]Device A
Class
Auditorium
Eng Bldg. 1
Eng Bldg. 2…

Device C

Device B
Parameter Exchange

MLP
HeadClass

Transformer
Encoder
Linear Projection

Auditorium
Eng Bldg. 1
Eng Bldg. 2…

WAFL-GAN[3]MLP
HeadMLPHeadTransformer
Encoder

Transformer
Encoder
Data Collection

Linear Projection

Device DMLP
HeadData Collection

Data Collection

Transformer
Encoder
Data Collection

Mission-Oriented
TaskE.g., UTokyo Campus Building Recognition[1] Hideya Ochiai, Atsuya Muramatsu, Yudai Ueda, Ryuhei Yamaguchi, Kazuhiro Katoh, and Hiroshi Esaki, "Tuning
Vision Transformer with Device-to-Device Communication for Targeted Image Recognition", IEEE World Forumon Internet of Things, 2023 (Best Paper Award).
[2] Hideya Ochiai, Riku Nishihata, Eisuke Tomiyama, Yuwei Sun, and Hiroshi Esaki, "Detection of Global Anomalies onDistributed IoT Edges with Device-to-Device Communication", ACM MobiHoc AIoT workshop, 2023.
[3] Eisuke Tomiyama, Hiroshi Esaki, Hideya Ochiai, "WAFL-GAN: Wireless Ad Hoc Federated Learning for Distributed9
Generative Adversarial Networks", IEEE International Conference on Knowledge and Smart Technology, 2023.

Previous Studies (5/5):
Extensions and Variations of WAFL (2/2)
WAFL-Personalization[4]WAFL-Localization[5]WAFL-Security[6]Poisoned model (constant)persistently injected by Attacker m

Flow of poison

Flow of resilience
Force of driving

in response to the input flowLegitimate Area

[4] Ryusei Higuchi, Hiroshi Esaki, and Hideya Ochiai, "Personalized Wireless Ad Hoc Federated Learning for Label PreferenceSkew", IEEE World Forum on Internet of Things, 2023.
[5] Yusuke Sugizaki, Hideya Ochiai, Muhammad Asad, Manabu Tsukada, and Hiroshi Esaki, "Wireless Ad-Hoc Federated
Learning for Cooperative Map Creation and Localization Models", IEEE World Forum on Internet of Things, 2023.
[6] Naoya Tezuka, Hideya Ochiai, Yuwei Sun, Hiroshi Esaki, "Resilience of Wireless Ad Hoc Federated Learning against
Model Poisoning Attacks", IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, andApplications (TPS-ISA), 2022.
10We propose MT-WAFL and demonstrate the effectiveness in training a Brainstormer.

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion

11Multiple Cognitive Axes
PurpleApplezyBigwx
12

Training Brainstormer from
Weekly-Labeled Distributed Data on Edge Devices
Available Supervise Labels (y, z, w) for xInput(x)PersonObject Class(y)Color(z)Size(w)

Person A

Apple

N/A

N/A

Person B

N/APurple

N/A

Person C

N/A

N/A

Person B

Person A

BigPurple

ApplezyBigw

Person Cx

Collaborative Model Trainingx

TrainedModel

Brainstormer(Apple, Purple, Big)

yzw

13

Multi-Task Learning Model Integration
with Wireless Ad Hoc Federated Learning
Strongly

WeaklyWeaklyWeaklyStrongly

MT-WAFL

Device A

WeaklyWeaklyWeaklyStrongly

MT-WAFL

Device B

Device C

14

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion

15Extension of Fashion-MNIST for Multi-Task Learning
Original Fashion-MNIST
5: SandalyRotated and Resized Fashion-MNIST

5: Sandalyz 1: +90°Rotate
0: No ResizexSingle Axis Datasetx

Rotate and Resizew(and add corresponding labels)Multiple Axes Dataset

16

Examples of the Expanded Fashion-MNIST
After the rotation and resizing,
1. We split theminto train and test data.
2. We distributed the train data
to 10 mobile devices.
3. We removed some labelsin the training dataon the device to reveal
their major cognitive axis.
17

Major Cognitive Axis Skew in our study
Available Supervise Labels (y, z, w) for x
Input(x)PersonObject Class(y)Rotation(z)Size(w)

Device 0-4

5

N/A

N/A

Device 5-7

N/A

1

N/A

Device 8-9

N/A

N/A

1

Distributionof Availability

Availability of Supervised LabelsYZ

W

Device 0-4

99%

1%

1%

Device 5-7

1%99%

1%

Device 8-91%

1%

99%
18

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Demonstration• Conclusion

19

Demonstration: Experiment Settings• ML model (MLP-based)• 𝜃ℎ : FC (𝑖 = 784, 𝑜 = 256) - ReLU - FC (𝑖 = 256, 𝑜 = 128)• 𝜃𝑦 : FC (𝑖 = 128, 𝑜 = 64) - ReLU - FC (𝑖 = 64, 𝑜 = 10)• 𝜃𝑧 : FC (𝑖 = 128, 𝑜 = 32) - ReLU - FC (𝑖 = 32, 𝑜 = 4)
• 𝜃𝑤 : FC (𝑖 = 128, 𝑜 = 32) - ReLU - FC (𝑖 = 32, 𝑜 = 2)

• Dataset• Expanded Fashion-MNIST• Mobility Pattern• Random Waypoint Mobility (RWP)• Simulation• We carried out the experiment bysimulation on a single computer.

Random Waypoint Mobility

20

Integration of Multiple Perspectives by WAFL

Device 8’s Predictions for Object Class, Rotation, and Size

Device 8 originally has many labels in Size(w) perspective.
As the training proceeds, misclassifications in Object Class and Rotation predictions have improved.
21

Integration of Multiple Perspectives by WAFL

Accuracy of Class, Rotation, and Size @Device (0, 5, 8)
MT-WAFL (our proposal) achieved identical accuracies

Device 0 has Class (Y) labels.

Device 5 has rotation (Z) labels.

Device 8 has size (W) labels.

Accuracy has been improved in all the perspectives.
22

Table of Contents• Introduction• Previous Studies• Multi-Task Wireless Ad Hoc Federated Learning• Rotated and Resized Fashion-MNIST Dataset for MTL• Evaluation• Conclusion

23

Conclusion• Multi-Task WAFL (MT-WAFL) has successfully integrated multitask models with other devices through D2D communication.
• We demonstrated a brainstormer trained by MT-WAFL in the caseof our expanded Fashion-MNIST dataset.
• Further applications could be considered using the realistic photos.
The trained model will help the people of the community.

24

Thank you.
Strongly

WeaklyWeaklyWeaklyStrongly

MT-WAFL

Device A

WeaklyWeaklyWeaklyStrongly

MT-WAFL

Device B

Device C25

26

Epoch 100

Epoch 1000

Epoch 2500

Size (W)

Rotation (Z)Class (Y)Epoch 027