Repository: tensorflow/tensor2tensor

Issue #1692: trax examples don't work due to `data_dir` being required
Created: 2019-09-08T03:19:06Z
I was trying to run the trax examples, but they all fail due to data_dir being required.
$ python3 -m tensor2tensor.trax.trainer   --dataset=mnist   --model=MLP   --config="train.train_steps=1000"
WARNING: Logging before flag parsing goes to stderr.
W0907 20:15:53.617917 139662293460800 lazy_loader.py:50]
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W0907 20:15:55.026820 139662293460800 deprecation_wrapper.py:119] From /usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/utils/expert_utils.py:68: The name tf.variable_scope is deprecated. 
Please use tf.compat.v1.variable_scope instead.

W0907 20:15:56.419855 139662293460800 deprecation_wrapper.py:119] From /usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/jaxboard.py:38: The name tf.HistogramProto is deprecated. Pleas
e use tf.compat.v1.HistogramProto instead.

W0907 20:15:56.420063 139662293460800 deprecation_wrapper.py:119] From /usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/jaxboard.py:39: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.

W0907 20:15:56.420177 139662293460800 deprecation_wrapper.py:119] From /usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/jaxboard.py:40: The name tf.SummaryMetadata is deprecated. Please use tf.compat.v1.SummaryMetadata instead.

W0907 20:15:56.430300 139662293460800 deprecation_wrapper.py:119] From /usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/trainer.py:99: The name tf.enable_eager_execution is deprecated. Please use tf.compat.v1.enable_eager_execution instead.

2019-09-07 20:15:56.499428: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...

I0907 20:15:56.499788 139662293460800 trax.py:139] No --output_dir specified
No --output_dir specified
I0907 20:15:56.499950 139662293460800 trax.py:139] Using --output_dir ~/trax/MLP_mnist_20190907_2015
Using --output_dir ~/trax/MLP_mnist_20190907_2015
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/trainer.py", line 133, in <module>
    app.run(main)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/trainer.py", line 129, in main
    trax.train(output_dir=output_dir)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1073, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/utils.py", line 49, in augment_exception_message_and_reraise
    six.raise_from(proxy.with_traceback(exception.__traceback__), None)
  File "<string>", line 3, in raise_from
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1050, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/trax.py", line 878, in train
    save_steps=save_steps, has_weights=has_weights)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1073, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/utils.py", line 49, in augment_exception_message_and_reraise
    six.raise_from(proxy.with_traceback(exception.__traceback__), None)
  File "<string>", line 3, in raise_from
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1050, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/trax.py", line 540, in __init__
    inputs = inputs(n_devices)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1073, in gin_wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/utils.py", line 49, in augment_exception_message_and_reraise
    six.raise_from(proxy.with_traceback(exception.__traceback__), None)
  File "<string>", line 3, in raise_from
  File "/usr/local/google/home/jlebar/.local/lib/python3.6/site-packages/gin/config.py", line 1050, in gin_wrapper
    return fn(*new_args, **new_kwargs)
  File "/usr/local/google/home/jlebar/code/tensor2tensor/tensor2tensor/trax/inputs.py", line 80, in inputs
    assert data_dir, 'Must provide a data directory'
AssertionError: Must provide a data directory
  In call to configurable 'inputs' (<function inputs at 0x7f0564007598>)
  In call to configurable 'Trainer' (<class 'tensor2tensor.trax.trax.Trainer'>)
  In call to configurable 'train' (<function train at 0x7f0563f890d0>)

I am happy to send a patch to fix this, but I am not sure what is the desired fix.  Do any of the following sound good to you?

Update the examples to pass an explicit data_dir (e.g. $TMPDIR/trax-data) on the command line.
Update the configs to pass an explicit data_dir.  If so, are you OK running os.path.expandvars on the config value, so you can include $TMPDIR in the config?
Set a default datadir in the code, rather than requiring an explicit one?

Duplicate issue #1694: trax resnet50 example doesn't download imagnet dataset, so doesn't work
Created: 2019-09-08T13:37:09Z
Description
I'm trying to follow the trax examples in the trax README.md.  I ran
python -m tensor2tensor.trax.trainer \
  --config_file=$PWD/trax/configs/resnet50_imagenet_8gb.gin

and this fails with
ValueError: No data files found in /usr/local/google/tmp/trax/image_imagenet-train*

apparently it didn't download the imagenet dataset.  The mnist example from the README works (and downloads the dataset).
Full log: https://gist.github.com/9d4268c04465e5ca8bbf0fbd372c6370
Environment information
OS: macos

$ pip3 freeze | grep 'tensor|jax'
tpip3 freeze | grep 'tensor\|jax'
jax==0.1.44
jaxlib==0.1.28
tensorboard==1.14.0
tensorflow-datasets==1.2.0
tensorflow-estimator==1.14.0
tensorflow-gpu==1.14.0
tensorflow-metadata==0.14.0
tensorflow-probability==0.7.0

$ python3 -V
Python 3.6.8rc1

$ git rev-parse HEAD
648184c46abb86239d8fdb317d7921d4bae8800c

Closing comment (2019-09-08T20:23:53Z): Yes, it won't, we'll have to explicitly ask it to f generate_data
…
On Sun, Sep 8, 2019, 10:38 AM Justin Lebar ***@***.***> wrote:
 This seems related to #1692
 <#1692>

 Could be, but I don't think so. I worked around #1692
 <#1692> by adding an
 explicit data dir and it still doesn't download the training data anywhere.

 —
 You are receiving this because you commented.
 Reply to this email directly, view it on GitHub
 <#1694?email_source=notifications&email_token=AAIUEFRSJAWHQHXGFJT2G3TQIU2DBA5CNFSM4IUTPC72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FVKMI#issuecomment-529225009>,
 or mute the thread
 <https://github.com/notifications/unsubscribe-auth/AAIUEFQDXC4UO5YDG5F7LZ3QIU2DBANCNFSM4IUTPC7Q>
 .