Repository: keras-team/keras

Issue #5396: Passing variable to Lambda function and loading model JSON
Created: 2017-02-14T17:45:27Z
I'm not sure if I'm just ignorant of how Lambda functions work, but if I do the following:
x = Lambda(lambda x: x * 0.1)(x)
I have no problem saving and loading a model.  However, if I try to pass a variable:
x = Lambda(lambda x: x * factor)(x)
I get the following error on the function closure when loading a model:

TypeError: arg 5 (closure) must be None or tuple.

When I printed out the closure when the model failed, I get whatever that value passed to the variable was - [0.1], for example.  I can train and run models, no problem, but I just can't load from the JSON.
Relevant JSON of what produces the error - getting a [0.1] where it expects a null:
{"class_name": "Lambda", "config": {"function": ["c\u0001\u0000\u0000\u0000\u0001\u0000\u0000\u0000\u0002\u0000\u0000\u0000\u0013\u0000\u0000\u0000s\b\u0000\u0000\u0000|\u0000\u0000\u0088\u0000\u0000\u0014S(\u0001\u0000\u0000\u0000N(\u0000\u0000\u0000\u0000(\u0001\u0000\u0000\u0000t\u0001\u0000\u0000\u0000x(\u0001\u0000\u0000\u0000t\u000e\u0000\u0000\u0000scale_residual(\u0000\u0000\u0000\u0000s-\u0000\u0000\u0000/home/jesse/Source/dl_code/mdl/model_keras.pyt\b\u0000\u0000\u0000<lambda>5\u0004\u0000\u0000s\u0000\u0000\u0000\u0000", null, [0.1]], "name": "lambda_1", "trainable": true, "function_type": "lambda", "arguments": {}, "output_shape": null, "output_shape_type": "raw"}, "inbound_nodes": [[["batchnormalization_18", 0, 0]]], "name": "lambda_1"}
JSON of working model:
{"class_name": "Lambda", "config": {"function": ["c\u0001\u0000\u0000\u0000\u0001\u0000\u0000\u0000\u0002\u0000\u0000\u0000S\u0000\u0000\u0000s\b\u0000\u0000\u0000|\u0000\u0000d\u0001\u0000\u0014S(\u0002\u0000\u0000\u0000Ng\u009a\u0099\u0099\u0099\u0099\u0099\u00b9?(\u0000\u0000\u0000\u0000(\u0001\u0000\u0000\u0000t\u0001\u0000\u0000\u0000x(\u0000\u0000\u0000\u0000(\u0000\u0000\u0000\u0000s-\u0000\u0000\u0000/home/jesse/Source/dl_code/mdl/model_keras.pyt\b\u0000\u0000\u0000<lambda>5\u0004\u0000\u0000s\u0000\u0000\u0000\u0000", null, null], "name": "lambda_1", "trainable": true, "function_type": "lambda", "arguments": {}, "output_shape": null, "output_shape_type": "raw"}, "inbound_nodes": [[["batchnormalization_18", 0, 0]]], "name": "lambda_1"}

Duplicate issue #5401: Problem in loading custom layer
Created: 2017-02-15T07:19:44Z
I have used the ZSumLayer from here to average all of the word embeddings from the embedding layer. I am able to train the model and save it successfully. But when I am loading it back for scoring, it is throwing following error:
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/models.py", line 197, in model_from_json
    return layer_from_config(config, custom_objects=custom_objects)
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 36, in layer_from_config
    return layer_class.from_config(config['config'])
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2375, in from_config
    process_layer(layer_data)
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 2353, in process_layer
    custom_objects=custom_objects)
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 36, in layer_from_config
    return layer_class.from_config(config['config'])
  File "/home/shwegarg/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 929, in from_config
    return cls(**config)
TypeError: __init__() takes exactly 2 arguments (1 given)

The full code is as follows:
ZSumLayer code:
from keras.layers import Layer
from keras import backend as K
class ZSumLayer(Layer):
    def __init__(self, output_dim, **kwargs):
        self.supports_masking = True
        self.output_dim = output_dim
        super(ZSumLayer, self).__init__(**kwargs)

    def call(self, x, mask=None):
        count = K.sum(mask, axis=1)[:, np.newaxis]
        avg = K.sum(K.switch(mask[:, :, np.newaxis], x, 0), axis=1) / count
        return K.cast(avg, x.dtype)

    def get_output_shape_for(self, input_shape):
        return (input_shape[0], self.output_dim)

    def compute_mask(self, input, mask):

Training code
from keras.layers.embeddings import Embedding
from keras.layers import Masking, Activation, Input, LSTM, merge
from keras.models import Model
import numpy as np

wt = np.array([[0,0,0,0,0],
              [1,2,0,3,6],
              [4,6,0,5,7],
              [4,5,7,0,4]])
            
def s(x):
    return K.mean(x, axis=1)

sen = Input(shape=(4,), dtype='int32')

embedding = Embedding(
    input_dim = 4, 
    output_dim = 5,
    mask_zero = True, 
    trainable=False, 
    weights=[wt]
)(sen)

out = ZSumLayer(5)(embedding)

model = Model(input=sen, output=out)

model.compile(loss='mse', optimizer='rmsprop')

print model.summary()

with open('/home/shwegarg/data_files/model_arch.json', 'w') as mfp:
    mfp.write(model.to_json())
model.save_weights('/home/shwegarg/data_files/model_wts.hdf5')

Scoring code:
from keras.models import model_from_json

mfp = open('/home/shwegarg/data_files/model_arch.json','r')
mj = mfp.read()
model2 = model_from_json(mj, {'ZSumLayer': ZSumLayer})

Closing comment (2017-02-15T12:29:10Z): Hello,
Is this by any chance related to #5396 ?