How to run (Model-Agnostic Meta-Learning) MAML algorithm | by Rahul Bhadani | Towards Data Science

MAML is a class of meta-learning algorithms created by Stanford Research and UC Berkeley Alum Dr. Chelsea Finn. MAML was inspired by the idea behind the question that how much data is really needed to learn about something. Can we teach algorithms to learn how to learn?
In such a context, there are a few challenges with conventional machine learning algorithms:
Intensive training is requiredLabeled data for some problems may be limitedThe performance of the network may be sensitive to the choice of hyperparameters
Intensive training is required
Labeled data for some problems may be limited
The performance of the network may be sensitive to the choice of hyperparameters
In such respect, meta-learning algorithms can be designed to address the following tasks:
Dynamic selection of inductive biasBuilding meta-rules for multi-task learningLearning to learn with hyperparameter optimization
Dynamic selection of inductive bias
Building meta-rules for multi-task learning
Learning to learn with hyperparameter optimization
Taken from Chelsea Finn’s original research:
MAML is a meta-learning algorithm that is compatible with any model trained with gradient descent algorithm and covers problems from classification, reinforcement learning (RL)and regression
MAML is designed such that it trains a model on a variety of tasks such that it can learn a new learning task with only a small number of training samples.
A few important points of MAML are:
MAML doesn’t expand the number of learned parameters.No constraint on the architecture or network of the model.Can be combined with other deep-learning frameworks such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and Multi-Layer Perceptron (MLP).
MAML doesn’t expand the number of learned parameters.
No constraint on the architecture or network of the model.
Can be combined with other deep-learning frameworks such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and Multi-Layer Perceptron (MLP).
Problem setup for MAML is reproduced from the original paper:
MAML introduces an outer loop called meta-training.
Chelsea Finn’s Github repo provides the code to reproduce the MAML results. You can use the following steps to reproduce its results:
I am going to create a python virtual environment and install dependencies:sudo apt install virtualenvvirtualenv — python=python3.6 maml source maml/bin/activateNext, we install dependencies:pip install tensorflow==1.11.0pip install imageI didn’t use the latest version of Tensorflow as MAML code was written a couple of years ago when TF2 was still not released for public use.Clone the MAML repo: git clone https://github.com/cbfinn/mamlDownload omniglot data, for this article, I am only going to run omniglot example besides sinusoidal example:wget https://github.com/brendenlake/omniglot/raw/master/python/images_background.zipwget https://github.com/brendenlake/omniglot/raw/master/python/images_evaluation.zipUnzip images_background and images_evaluation zip files to maml/data/omniglot folder where mamal folder is the Github repo folder. The directory structure looks as follows:
I am going to create a python virtual environment and install dependencies:sudo apt install virtualenvvirtualenv — python=python3.6 maml source maml/bin/activate
Next, we install dependencies:
pip install tensorflow==1.11.0pip install imageI didn’t use the latest version of Tensorflow as MAML code was written a couple of years ago when TF2 was still not released for public use.
Clone the MAML repo: git clone https://github.com/cbfinn/maml
Download omniglot data, for this article, I am only going to run omniglot example besides sinusoidal example:wget https://github.com/brendenlake/omniglot/raw/master/python/images_background.zipwget https://github.com/brendenlake/omniglot/raw/master/python/images_evaluation.zip
Unzip images_background and images_evaluation zip files to maml/data/omniglot folder where mamal folder is the Github repo folder. The directory structure looks as follows:
7. Navigate to data subfolder of maml folder and copy the content of omniglot to omniglot_resized . The run resize-image script
cd maml/datacp -r omniglot/* omniglot_resized/cd omniglot_resizedpython resize_images.py
8. Now, we go back to the root directory of maml repo and run two examples:
a. Sinusoid example:
python main.py --datasource=sinusoid --logdir=logs/sine/ --metatrain_iterations=70000 --norm=None --update_batch_size=10
b. omniglot example:
python main.py --datasource=omniglot --metatrain_iterations=60000 --meta_batch_size=32 --update_batch_size=1 --update_lr=0.4 --num_updates=1 --logdir=logs/omniglot5way/
The checkpoint of each example will be saved in log/ directory.
Update: I discovered that there is a bug in the codebase. I believe the bug might have been introduced inadvertently. In main.py , line 160 saver.save(sess, FLAGS.logdir + ‘/’ + exp_string + ‘/model’ + str(itr)) , itr variable is the for loop iterator and I believe that saver.save statement should be executed inside the for loop at the end of the loop and not outside the loop.
From here onwards, I believe an intermediate-level machine learning practitioner should be able to modify the pipeline to meet his/her requirements.