Using Codex to translate Keras to fastai | by Mark Ryan | Towards Data Science

OpenAI’s Codex provides a set of computer language-specific operations that can be used to generate code from English language descriptions, automatically document code, and translate between coding languages. Over the last few months I have been exporing Codex’s capabilities, including its ability to generate web applications from English-language prompts and its ability to generate modern code from COBOL. I was curious to see how Codex would handle translating between two different deep learning frameworks.
I have had the privilege of writing two books, one on deep learning with structured data that featured deep learning examples written in Keras (the high-level deep learning framework for TensorFlow), and another on deep learning with the fastai, a high-level framework built on top of PyTorch. In the course of writing the second book I needed to create a fastai implementation of an existing Keras application. When I wrote the book I created the fastai application “by hand” through trial and error until I got working code was roughly equivalent to the Keras code. What if I had had access to Codex when I was writing the book— would I have been able to take advantage of it to generate fastai automatically from Keras? In this article I’ll describe an experiment to answer this question.
The simplest deep learning problem you can tackle is using the MNIST dataset of hand-written digits to train a model that predict the number depicted in an image of hand-written digits.
You could say that creating a model for MNIST is the “hello world” of deep learning.
My book on fastai includes a side-by-side comparison of Keras and fastai implementations of MNIST models. This comparison illustrates the differences between these two frameworks and sets the stage for developers who are familiar with Keras to explore fastai.
To exercise Codex’s ability to translate from Keras to fastai I decided to take the same Keras MNIST implementation that I had hand-converted into fastai for my book and see if Codex could accomplish the translation correctly.
Here is the starting point Keras MNIST implementation, adapted from https://github.com/tensorflow/docs/blob/master/site/en/tutorials/quickstart/beginner.ipynb:
To get Codex to translate this to fastai, I used the following prompt pattern:
// Rewrite this as a fastai model[ Keras code goes here ]// fastai version:
The result that Codex generated looked reasonably plausible as fastai:
I copied this code into a Colab notebook, but it wouldn’t work as is. I needed to make two changes to get it to run correctly in Colab:
Replace the include statements with include statements for a notebook:
#import fastai#from fastai.vision.all import * !pip install -Uqq fastbookfrom fastbook import *from fastai.vision.all import *
Comment out the compile statement:
#hello_world_model.compile(optimizer=optim.Adam,#              loss_func=nn.CrossEntropyLoss(),#              metrics=[accuracy])
With these changes I was able to successfully run the fastai code in Colab and get a working MNIST deep learning model:
While the fastai code that Codex generated worked with a couple of minor fixes, it wasn’t 100% equivalent to the Keras input code. In particular, the input Keras code included statements to show the model’s loss and accuracy for the test dataset:
test_scores = hello_world_model.evaluate(x_test,  y_test, verbose=2) print('Loss for test dataset:', test_scores[0])print('Accuracy for test dataset:', test_scores[1])
The fastai code that Codex generated includes the following statement to exercise the trained model on the test set, but it doesn’t show the aggregated results the way that the original Keras code does:
test_dl = dls.test_dl(get_image_files(path/'testing'))preds,_ = learn.get_preds(dl=test_dl)
Given Keras code for an MNIST deep learning model as input, Codex was able to produce fastai code as output that (after a couple of minor changes) successfully trained an MNIST deep learning model. Here are my key observations:
On one hand, this is impressive. Code that uses the Keras framework is quite different from code that uses fastai, and as a human being it took me a few tries to get working fastai code for MNIST with the Keras code as as starting point. Codex produced (almost) working fastai code on its first try. I didn’t have to fiddle with the prompts or make any changes to the in put Keras code.
On the other hand, I have to wonder whether Codex had simply “memorized” Keras and fastai MNIST implementations that were part of the github corpus on which it had been trained. For example, the code for my Deep Learning with fastai book (including this hand-written fastai MNIST application and its companion Keras MNIST application) has been in github since mid 2021, so it’s possible that the training set for Codex included the examples from my book, and all Codex was doing was “looking up” the fastai statements from the hand-written MNIST application that it had seen in its training set.
On balance, I have to say that I am impressed that Codex was able to generate (just about) working fastai code from Keras code. Even translating “hello world” from one framework to another automatically is an accomplishment. I look forward to testing Codex with more challenging Keras to fastai translations.
Video covering the same topic as this article: https://youtu.be/xUMBiMP4xNw