Multi-Output Model with TensorFlow Keras Functional API
By: Andrej Baranovskij
Tags: Python, TensorFlow, API, Keras

When implementing a slightly more complex use case with machine learning, very likely you may face the situation, when you would need multiple models for the same dataset. Take for example Boston housing dataset. This dataset comes with various features and there is one target attribute — Price. We would use a single output model to predict Price. However, if in addition, we would decide to predict Pupil-Teacher Ratio by Town, we would have two outputs. The simplest option would be to create two separate models based on the same dataset to predict two variables. This would be manageable, but what if in a real scenario we would have 100 outputs. It could be a pain to manage all these separate models. Instead, it is more productive to create a single model with multiple outputs.

My goal was to create a simple and reproducible example, which would act as a blueprint for your own implementation. An example is based on Boston housing dataset and the model implements two outputs — Price and Ptratio (Pupil-Teacher Ratio by Town).

Data preparation

I’m using Boston housing dataset from sklearn.datasets, this way it is easy to run notebook in Google Colab, no need to upload data separately:

# Importing the Boston Housing dataset

from sklearn.datasets import load_boston

You can load the dataset and assign it to the local variable:

# Loading the Boston Housing dataset

boston = load_boston()

For convenience, we can wrap it with Pandas dataframe:

# Initializing the dataframe

data = pd.DataFrame(boston.data)

We should assign feature names for the dataframe columns:

#Adding the feature names to the dataframe

data.columns = boston.feature_names

Price feature will be added to dataframe as well:

#Adding target variable to dataframe

data['PRICE'] = boston.target

data.head()

There are 506 rows and 14 columns/features in this dataset. Price feature is described by 13 other features, in addition with choose Ptratio to be output target.

We split data into training, test, and validation sets. Random state = 1 is used, to make sure the data is split always in the same way, to generate reproducible training results:

# Split the data into train and test with 80 train / 20 test

train,test = train_test_split(data, test_size=0.2, random_state = 1)

train,val = train_test_split(train, test_size=0.2, random_state = 1)

There are two helper functions defined, to normalize data and process output targets. We select Price and Ptration features as targets, remove them from the dataframe:

# Helper functions

def norm(x):

return (x - train_stats['mean']) / train_stats['std'] def format_output(data):

y1 = data.pop('PRICE')

y1 = np.array(y1)

y2 = data.pop('PTRATIO')

y2 = np.array(y2)

return y1, y2

Next, we call the helper function to extract output targets and build stats to be used for data normalization:

# Get PRICE and PTRATIO as the 2 outputs and format them as np

# arrays

# PTRATIO - pupil-teacher ratio by town

train_stats = train.describe()

train_stats.pop('PRICE')

train_stats.pop('PTRATIO')

train_stats = train_stats.transpose()

train_Y = format_output(train)

test_Y = format_output(test)

val_Y = format_output(val)

Train, test, and validation datasets are normalized using helper function:

# Normalize the training and test data

norm_train_X = np.array(norm(train))

norm_test_X = np.array(norm(test))

norm_val_X = np.array(norm(val))

Here we come to the core part of this example — the function where the model is built. Look at the last line of the function, where the Model is defined. We are using an array for the outputs variable, which is assigned with two output layers. The first output layer structure is based on a single Dense layer, while the second output layer is constructed with two Dense layers. You are free to adjust and create any configuration, intermediate layers can be merged and split, this is the beauty of Keras functional API:

def build_model():

# Define model layers.

input_layer = Input(shape=(len(train .columns),))

first_dense = Dense(units='128', activation='relu')(input_layer)

# Y1 output will be fed from the first dense

y1_output = Dense(units='1', name='price_output')(first_dense) second_dense = Dense(units='128',activation='relu')(first_dense)

# Y2 output will be fed from the second dense

y2_output = Dense(units='1',name='ptratio_output')(second_dense) # Define the model with the input layer

# and a list of output layers

model = Model(inputs=input_layer,outputs=[y1_output, y2_output]) return model

The model should be compiled with loss and metrics for each of the outputs:

model = build_model() # Specify the optimizer, and compile the model with loss functions for both outputs

optimizer = tf.keras.optimizers.SGD(lr=0.001)

model.compile(optimizer=optimizer,

loss={'price_output': 'mse', 'ptratio_output': 'mse'},

metrics={'price_output':tf.keras.metrics.RootMeanSquaredError(),

'ptratio_output':tf.keras.metrics.RootMeanSquaredError()})

Then we train the model with training and test datasets:

# Train the model for 100 epochs

history = model.fit(norm_train_X, train_Y,

epochs=100, batch_size=10, validation_data=(norm_test_X, test_Y))

When evaluating the model, we can print the loss and rmse for both outputs. Variables are returned in sequential order, in the same order as training output targets were specified:

# Test the model and print loss and rmse for both outputs

loss,Y1_loss,Y2_loss,Y1_rmse,Y2_rmse=model.evaluate(x=norm_val_X, y=val_Y) print()

print(f'loss: {loss}')

print(f'price_loss: {Y1_loss}')

print(f'ptratio_loss: {Y2_loss}')

print(f'price_rmse: {Y1_rmse}')

print(f'ptratio_rmse: {Y2_rmse}')

I don’t have a goal in this example to achieve the best possible accuracy for the model, my focus is on the model structure. You may tune hyperparameters to improve accuracy.

Predict function call returns results for both outputs, in the same sequential order:

# Run predict

Y_pred = model.predict(norm_test_X)

price_pred = Y_pred[0]

ptratio_pred = Y_pred[1] plot_diff(test_Y[0], Y_pred[0], title='PRICE')

plot_diff(test_Y[1], Y_pred[1], title='PTRATIO')

We can plot predicted results vs actual values for both outputs:

Predicted results vs Actual values

We can save the model for reuse:

# Save model

model.save('./model_boston/', save_format='tf')

To make sure the saved model works, we can restore it:

# Restore model

loaded_model = tf.keras.models.load_model('./model_boston/')

Run predict function from the restored model. If there are no errors and results are returned, it means we are good:

# Run predict with restored model

predictions = loaded_model.predict(norm_test_X)

price_pred = predictions[0]

ptratio_pred = predictions[1]

Conclusion

The goal of this post is to provide a simple and clean ML model with multiple outputs, running on Keras functional API. I believe handling multiple outputs in a single model can improve code quality and simplify model maintenance. Hopefully, you will find this example useful in your own implementations.

Source code

GitHub repo

Run it yourself in Colab notebook

Youtube