Repository: tensorflow/tensorflow (language: cpp)

Issue:
 Text generation tutorial issue
**My issues relate to the following tutorial "Text generation using a RNN with eager execution"**: 
- https://www.tensorflow.org/tutorials/sequences/text_generation

**System information**
- TensorFlow version: 1.12

(disclaimer: I am fairly new to using tensorflow, so please let me know if I am misunderstanding something!)

**1) Questions about the 'Try the Model' section**
`sampled_indices = tf.random.categorical(example_batch_predictions[0], num_samples=1)
sampled_indices = tf.squeeze(sampled_indices,axis=-1).numpy()`

When I run this code above, my program would produce an error message:
`AttributeError: module 'tensorflow._api.v1.random' has no attribute 'categorical'`

I checked the Tensorflow API for 1.12, and indeed random does not have a 'categorical' attribute. So as per suggestion by a Stack Overflow comment, I am using the 'multinomial' attribute (namely  tf.random.multinomial instead of  tf.random.categorical) : https://www.tensorflow.org/api_docs/python/tf/random/multinomial

**My questions:** Is this an adequate solution? Am I missing something, or does random indeed not have the categorical attribute? If so, would it be possible to update the tensorflow tutorial to prevent future users from running into this issue?

**2) Questions about the 'Train the Model' section**
`def loss(labels, logits):`
                   `return tf.keras.losses.sparse_categorical_crossentropy(labels, logits, from_logits=True)`

`example_batch_loss  = loss(target_example_batch, example_batch_predictions)`
`print("Prediction shape: ", example_batch_predictions.shape, " # (batch_size, sequence_length, vocab_size)") `
`print("scalar_loss:      ", example_batch_loss.numpy().mean())`

When I run this above code, my program would produce an error message:
`sparse_categorical_crossentropy() got an unexpected keyword argument 'from_logits'`

I checked the Tensorflow API for 1.12, and indeed tf.keras.LOSSES.sparse_categorical_crossentropy() (alias for metrics which I linked to) only has two parameters (neither of which are 'from_logits'):
https://www.tensorflow.org/api_docs/python/tf/keras/metrics/sparse_categorical_crossentropy

Whereas tf.keras.BACKEND.sparse_categorical_crossentropy() has four paramaters, one of which is
'from_logits`: https://www.tensorflow.org/api_docs/python/tf/keras/backend/sparse_categorical_crossentropy

So I changed my program to use tf.keras.backend.sparse_categorical_crossentropy() instead of  tf.keras.losses.sparse_categorical_crossentropy().

**My questions:** Is this an adequate solution? If the tutorial is indeed mistaken for using losses instead of backend, would it be possible to update it with this as well?

Thank you!







Discussion / hints:
The tutorial installs tf-nightly build which contains TF 1.13rc0
1) [tf.random.multinomial](https://github.com/tensorflow/tensorflow/blob/9f30ec59fda6e9134dff2c7d9cc42b3fb455f8b7/tensorflow/python/ops/random_ops.py#L330) will be deprecated starting from TF 1.13rc0 thus can use tf.random.categorical instead.
The work around will be to use tf.random.multinomial if using TF 1.12 or lower. 
2) Your workaround is correct.
Thank you for your confirming my workarounds! 

This all makes sense now, since I think I am using TF 1.12. Just one more question:

When I run:
`!pip install -q tf-nightly`
and then :
`!pip show tensorflow`

I get the following output:
`Name: tensorflow`
`Version: 1.12.0`
`Summary: TensorFlow is an open source machine learning framework for everyone.`
`Home-page: https://www.tensorflow.org/`
`Author: Google Inc.`
`Author-email: opensource@google.com`
`License: Apache 2.0`
`Location: /usr/local/lib/python3.6/dist-packages`
`Requires: gast, six, termcolor, keras-preprocessing, tensorboard, grpcio, absl-py, numpy, protobuf,` `keras-applications, astor, wheel`
`Required-by: stable-baselines, magenta, fancyimpute`

For the version, shouldn't I be getting :
`Version: 1.13rc0`

since I am using tf-nightly, as you described?
I suspect you forgot to restart the runtime after tf-nightly installation. Can you please confirm?
Alternatively, you can also try,
>pip install tensorflow==1.13rc0
Restarting the runtime didn't fix it, but running `pip install tensorflow==1.13rc0` did! Thank you for the help! 
You are welcome! Closing this issue since its solved.
Are you satisfied with the resolution of your issue?
[Yes](https://goo.gl/forms/Oe0tEvODFRoI2gJF3)
[No](https://goo.gl/forms/fUjzOfrtkFbrOT8d2)

Fix patch:
diff --git a/third_party/xla/xla/tools/multihost_hlo_runner/BUILD b/third_party/xla/xla/tools/multihost_hlo_runner/BUILD
index 685f72b8fd2d01..f6df8eafb70ea3 100644
--- a/third_party/xla/xla/tools/multihost_hlo_runner/BUILD
+++ b/third_party/xla/xla/tools/multihost_hlo_runner/BUILD
@@ -168,6 +168,7 @@ cc_library(
         "//xla/service:hlo_module_config",
         "//xla/service:hlo_module_util",
         "//xla/service:hlo_proto_cc",
+        "//xla/service:slow_operation_alarm",
         "//xla/tests:test_utils",
         "//xla/tools:hlo_control_flow_flattening",
         "//xla/tsl/platform:env",
diff --git a/third_party/xla/xla/tools/multihost_hlo_runner/functional_hlo_runner.cc b/third_party/xla/xla/tools/multihost_hlo_runner/functional_hlo_runner.cc
index 70713335cbda74..d072e24db5264a 100644
--- a/third_party/xla/xla/tools/multihost_hlo_runner/functional_hlo_runner.cc
+++ b/third_party/xla/xla/tools/multihost_hlo_runner/functional_hlo_runner.cc
@@ -65,6 +65,7 @@ limitations under the License.
 #include "xla/service/hlo.pb.h"
 #include "xla/service/hlo_module_config.h"
 #include "xla/service/hlo_module_util.h"
+#include "xla/service/slow_operation_alarm.h"
 #include "xla/shape_layout.h"
 #include "xla/shape_util.h"
 #include "xla/status_macros.h"
@@ -1161,6 +1162,11 @@ FunctionalHloRunner::CreateArgumentsOnDevice(
         client, executable, running_options, flatten_arguments);
   }
 
+  SlowOperationAlarm alarm(
+      absl::Seconds(5),
+      absl::StrFormat("Argument initialization is slow. Consider changing "
+                      "--hlo_argument_mode."));
+
   absl::Span<PjRtDevice* const> addressable_devices =
       executable->addressable_devices();
   size_t num_addressable_devices = addressable_devices.size();
@@ -1185,7 +1191,7 @@ FunctionalHloRunner::CreateArgumentsOnDevice(
           ModuleArgumentMode::kUseZerosAsInput;
 
   for (int i = 0; i < num_addressable_devices; ++i) {
-    VLOG(3) << "Creating fake argument for device " << i;
+    VLOG(3) << "Creating fake arguments for device " << i;
     LiteralVec& argument_literals =
         per_device_argument_literals[addressable_devices[i]->id()];
     int executable_idx = hlo_modules.size() == 1


Test patch: