Meta Learning in Decentralized Neural Networks
Towards More General AI
Yuwei Sun
The University of Tokyo
RIKEN

The state of deep learning
Large language modelGo

Text to video

Self-driving car

Protein folding2

Out-of-distribution generalization• Training to test
• Distribution difference• Undesired performance with unseen data

ACM

Beyond the training distribution
3

Why meta learning?
• Learning different perspectives with multiple agents (models)• Meta learning selects and combines learning algorithms to tackle a newtask [Pratt, 1991]• Extract information from past experiences and tasks• Reusable features and models
Task CTask A

Reuse knowledge

Task B

4

Cooperating and competing neural networks• Global workspace theory [Baars, 1988]• Cooperating and competing neuralnetwork models

Workingmemory• Specialized processors• Selection and reuse of processors

Unconscious
resources

5

Reusable knowledge representation learning• Improve neural networks generalization through knowledge transfer• Discussion on replica neural networks (a), hierarchy of neuralnetworks (b), and multi-modal models (c) as key approaches

Task A
Task B

Task A Task BabVision

Textc6

Replica neural networks in multi-agent settings
Society of AI

Social learning

Large foundation models

Yuwei Sun, Hideya Ochiai, Hiroshi Esaki. Decentralized Deep Learning for Multi-Access Edge Computing: A Survey. IEEE Transactions on Artificial Intelligence. 2022.

7

Distributional shift between train and test dataset• Good in-distribution performance• Struggle in out-of-distribution (OOD) settings• Given !! = #! , %! and !" = #" , find & %" #"Train!!

Test
ColorfulbackgroundTrain !!

Test !"!"
Sketch to real8

Reusable representation sharing

SourcedomainTargetTask B
domainKnowledge !!
distillation!!→#Reusable
representation

Irrelevant info.
9

Feature distribution matching for federated domain
generalization
Local ModelDS(1)Target/Source

Discrepancy

Feature Representation

Features
Disentangler

Embedding
Matching

Fine-tuning

Classifier
FeatureExtractors
L a be lDS(2)Federated
Voting

...

DS(K)

DT

Global Workspace

AgentsNovel task

Yuwei Sun, Ng Chong, Hideya Ochiai, Feature Distribution Matching for Federated Domain Generalization, ACML 2022.

10

Datasets
Office-Caltech10 [Gong, 2012]
Digit-Five [Ganin, 2015]
TargetSource

Amazon review [Blitzer, 2007]
Book: This book turns the entire concept of intelligence inside out
DVD: This is a great DVD for all collections
Electronics: This is perfect for my iPod and keeps it totally secure while driving
Kitchen: Simple, straight forward to use, very easy to clean, and durable

11

Late convergence
Group Effect:(loss)

Initialized, high loss
Baseline
FedKA W/O Voting
FedKA

0.0

Convergence
Late convergence12Performance evaluation
Digit Five

+4.0%

Office-Caltech10

+2.3%

13

T-SNE visualization of representation distributions

Without knowledge transfer

With knowledge transfer14

Hierarchical learning as a Markov decision process

r=-2
Module1s1

r=2Module3s3
r=10Starts0

Module4
s2
r=-1

s4

GoalMaximize theaccumulative reward
15

Neural states of modules

★

•••

★

Problem: Non-independent and identically distributed data (non-IID)
Cluster: Compressed weights of modules with similar training data classes
Reliable incremental measures of progressYuwei Sun, Hideya Ochiai. Homogeneous Learning: Self-Attention Decentralized Deep Learning. IEEE Access. 2022.

16

Neural states of modules

★

★

17

Neural states of modules★

★

18

Outer loop reinforcement learning
Previous action
States'# = ($% ('#&' , +* #&' )Module A
Module B
Module C
Module D
Module E
Module F

PolicyRewardreward.-# = arg max(4# ('# ) + 6 ⋅ 4#)'̂ ('#)' ))(!

Next action+9# = arg max((+% ('# , .-# ))*!

Update using theselected module
Yuwei Sun, Hideya Ochiai. Homogeneous Learning: Self-Attention Decentralized Deep Learning. IEEE Access. 2022.

19

Reward learningndRound

MNIST with100with 100 Clients
MNISTwithMNIST100Clients
Modules

Fashion-MNIST
Fashion-MNISTwith 100 Clientswith 100
100 Modules
Clients
Fashion-MNIST
with-6-6-10-10-20-14-14-30-18-18-40

Round loop)
Roundroundround(outer-10

Episodic reward

Episodic reward

withon-MNIST
10 Clientswith 10 Clients-10-20-30-40Round loop)
Round loop)
round
round round(outerround(outer20

Reduced convergence time
Reduce the training
time by 50%!

Our method

Goal

0.8

Goal

Validation accuracy

0.7
0.6
0.5
0.4
0.3

H: non-IIDness
H = 0.2
H = 0.6
H = 0.90.2
0.10102030

40

Round
Convergencetime (round)21

Cross-modal knowledge transfer• World model with different
modalities• Cross-modal knowledge transfer• Visual Question Answering tasks

Question:
What shape are the pizzas?
Answer: square

22

Multi-agent Visual Question Answering
Transferredknowledgevs.

• Training on the entire distribution• Subsets representing different
perspectives
23

!NHA!NHA

Modality alignment with self-supervised learning!LTA!LTA

Before training

After training!NHA

Contrastive learning!NHA

InfoNCE Loss!LTA

Faster-RCNN
Object Detection

MaskedBefore images
trainingwith the questions!LTA

Attention mapsAfter training

NHAAnchor

Nonlinear
Head Adapter

Positive

LTA

Nega0ve

Linear Tail
Adapter

Shared Projection Space

Cross-Modal Representation
Learning
Image

Forward propagation
Backpropagation“Where isthe bag?”

Answer Projection NetworkQuestion

Yuwei Sun, Hideya Ochiai. UniCon: Unidirectional Split Learning with Contrastive Loss for Visual Question Answering. NeurIPS 2022 workshop“on table”

Answer24

Evaluation

Q: Which room is this?
A: bedroom
Ground Truth: bedroom
Q: How many pictures on the wall?
A: 6
Ground Truth: 725

Conclusions• Limitation: Out-of-distribution generalization ability of NNs• Benefits of knowledge sharing and social learning among NN models inunseen tasks• Network of interconnected NN models with similar architecture• Hierarchical NNs with a meta model to optimize policy• Self-supervised learning for cross-modal knowledge transfer withoutlabels

26

Reusable modular knowledge for systematic generalization• Decompose high-level knowledge into reusable components• Attention mechanism• Switch from System 1 to System 2 processing• Routing of reusable components to tackle the OOD problem• Graph-structured elements of causality• Interventions, effects, and interpretability

28

29

Meta Learning in Decentralized Neural Networks
Towards More General AI
Yuwei Sun
The University of Tokyo
RIKEN