Repository: magenta/magenta

Issue:
polyphony_rnn_train: Incpomatible shapes (InvalidArgumentError)
Hi all,

I was wondering what output would a polyphony_rnn give when trained on some jazzy tunes, so I gathered some midi files from [The Jazz Page](http://www.thejazzpage.de/) and generated dataset, but when attempting to train, I get `Incompatible shapes`:

> InvalidArgumentError (see above for traceback): Incompatible shapes: [27776] vs. [28416]

I am attaching full error traceback [Here](https://gist.github.com/Nimeas/2e8c3cc641c82dc575f39bfe54da6dfc#file-terminal_executions-log), including output from dataset preparation.

I use tensorflow-gpu r0.12 (in order to workaround #538 as suggested by @brannondorsey)

Any hints would be highly appreciated.

Thanks!


Discussion:
I'm running into this as well.  I think this is related to https://github.com/tensorflow/magenta/issues/525

By default the polyphony model has `skip_first_n_losses` set to 10.  

I tried passing the hyperparms as a flag, (e.g.: `--hparams="{'batch_size': 100, 'skip_first_n_losses':0}"` but I still get different parameters being printed when I call the train method, e.g.:
```
INFO:tensorflow:hparams = {'rnn_layer_sizes': [256, 256, 256], 'decay_rate': 0.95, 'dropout_keep_prob': 0.5, 'batch_size': 64, 'decay_steps': 1000, 'clip_norm': 5, 'initial_learning_rate': 0.001, 'skip_first_n_losses': 10...
```

So it seems there might be two potential issues, 1 as described in https://github.com/tensorflow/magenta/issues/525 w/ nonzero `skip_first_n_losses`, and another w/ setting `hparams` on the polyphony CLI interface for training.

It looks like `melody_rnn` uses the tf.app config FLAGs to read the hparams, and default to the magenta common params, and then parse any given params (see here: `magenta/magenta/models/melody_rnn/melody_rnn_config_flags.py`), but there is no code yet to do that in the polyphony rnn model.  A quick hack to at least get started with the model is to just change the default value for `skip_first_n_losses` to 0 in the file `magenta/models/polyphony_rnn/polyphony_model.py`.
From my testing, the pull request https://github.com/tensorflow/magenta/pull/551 eliminates the skip_first_n_losses error. However, the hparams no effect problem is still yet to be solved.
I am also having the problem of hparams taking no effect. Mac OSx. I used the provided script to install everything. No matter what my --hparams is defined to be, I am getting a log that prints:
INFO:tensorflow:hparams = {'rnn_layer_sizes': [256, 256, 256], 'decay_rate': 0.95, 'dropout_keep_prob': 0.5, 'batch_size': 64, 'decay_steps': 1000, 'clip_norm': 5, 'initial_learning_rate': 0.001, 'skip_first_n_losses': 10}

And training runs forever... even if I tell it to run 1 training step


Resolving patch:
diff --git a/magenta/models/polyphony_rnn/polyphony_rnn_train.py b/magenta/models/polyphony_rnn/polyphony_rnn_train.py
--- a/magenta/models/polyphony_rnn/polyphony_rnn_train.py
+++ b/magenta/models/polyphony_rnn/polyphony_rnn_train.py
@@ -55,6 +55,11 @@
 tf.app.flags.DEFINE_string('log', 'INFO',
                            'The threshold for what messages will be logged '
                            'DEBUG, INFO, WARN, ERROR, or FATAL.')
+tf.app.flags.DEFINE_string(
+    'hparams', '{}',
+    'String representation of a Python dictionary containing hyperparameter '
+    'to value mapping. This mapping is merged with the default '
+    'hyperparameters.')
 
 
 def main(unused_argv):
@@ -72,6 +77,7 @@ def main(unused_argv):
   run_dir = os.path.expanduser(FLAGS.run_dir)
 
   config = polyphony_model.default_configs[FLAGS.config]
+  config.hparams.parse(FLAGS.hparams)
 
   mode = 'eval' if FLAGS.eval else 'train'
   graph = events_rnn_graph.build_graph(


Test patch: