Issue 1:
Summary/tldr:

It would be nice with more String support in the Java API and a tf.reduce_sum which supports string concatenation.

Body:

I have successfully hacked in the custom ops of syntaxnet into the Tensorflow master as I wish to run syntaxnet from my Java NLP pipeline.

I do this by saving a SavedModelBundle and load it again from Java.

My project is based on a saved model of parsey_mcparseface from this branch:

https://github.com/dmansfield/models/tree/documents-from-tensor

By user @dmansfield.

And a tensorflow build based on the custom ops from:

https://github.com/tensorflow/models/tree/master/syntaxnet

Hacked into master of tensorflow.

Hacking in the ops, as build in ops, was necessary as the Java api through the JNI does not support user loaded ops yet (AFAIK).

The code with the syntaxnet ops hacked in can be found here:

https://github.com/marhlder/syntaxnet-ops-hack

I have everything running except that the strings in the output tensor from my saved model includes junk (non-printable chars) which I think is due to the offset table described in tensor_jni.cc file.

I can run the same model from Python without any issues.

The Java API does currently not seem to support anything but scalar string tensors. It looks like using a scalar string tensor will solve my problem as that part of the codes seems to handle the offset table.

I would therefore like to reduce the string tensor produced by my parsey mcparseface model.

I then found that the tf.reduce_sum does not yet support string concatenation.

There should already be an op for it:

#701

User @mrry commented that he was looking for a use case for this here:

https://stackoverflow.com/questions/34247374/merge-string-tensors-in-tensorflow

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Arch Linux

Tensorflow installed from (source or binary): pip

Tensorflow version (use command below): tf-nightly-gpu-2.0-preview | 2.0.0.dev20190204 |

Python version: 3.6.8

CUDA/cuDNN version: 10.0, V10.0.130

GPU model and memory: GeForce RTX 2018 Ti 11GB

Describe the current behavior

During the training of a simple "toy-example" classificator using TensorFlow 2.0 and tfds, I'm trying to use the Keras callback in order to log the results and to be able to use TensorBoard. When the first epoch of training ends I get the following error: tensorflow.python.framework.errors_impl.NotFoundError: Resource localhost/logdir:./log/N10tensorflow22SummaryWriterInterfaceE does not exist. [Op:WriteScalarSummary] name: epoch_loss/

Describe the expected behavior

I expect to be able to use the TensorBoard callback without issues.

Code to reproduce the issue

Classificator:

This is a toy example based on the inception architecture and it is written inside a model.py file using Keras Functional API.

def inception(input_shape: Tuple[int, int, int]) -> k.Model:
# set the input
input_img = k.layers.Input(shape=input_shape)

tower_1 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(input_img)
tower_1 = k.layers.Conv2D(64, (3, 3), padding="same", activation="relu")(tower_1)

tower_2 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(input_img)
tower_2 = k.layers.Conv2D(64, (5, 5), padding="same", activation="relu")(tower_2)

tower_3 = k.layers.MaxPooling2D((3, 3), strides=(1, 1), padding="same")(input_img)
tower_3 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(tower_3)

output = k.layers.concatenate([tower_1, tower_2, tower_3], axis=3)

output = k.layers.Flatten()(output)
out = k.layers.Dense(10, activation="softmax")(output)

model = k.models.Model(inputs=[input_img], outputs=[out])

print(model.summary())

return model

Training code:

The training code it is inside a test.py file and use the tfds to import the datasets and the Keras fit function to train.

import tensorflow_datasets as tfds
import tensorflow as tf
from tensorflow import keras as k

import multiprocessing
import model

DATADIR = "/run/media/federico/XData/tensorflow_datasets"
CLASSES = 10
EPOCHS = 25
BATCH = 128
LRATE = 0.01
DECAY = LRATE / EPOCHS

# this is applied to every single data passed
def process_features(feature):
image, label = feature["image"], feature["label"]

# image conversion into [0, 1]
image = image / 255
image = tf.cast(image, tf.float32)
feature["image"] = image

# label conversion into one-hot
print(label)
label = tf.one_hot(label, CLASSES)
feature["label"] = label

return feature

# print(tfds.list_builders())

dataset, dataset_info = tfds.load(name="cifar10", data_dir=DATADIR, with_info=True)
train_dataset, test_dataset = dataset["train"], dataset["test"]

# process features
train_dataset = (
train_dataset.map(process_features, num_parallel_calls=multiprocessing.cpu_count())
.shuffle(1000)
.batch(BATCH)
.repeat(EPOCHS)
# .prefetch(10)
)

test_dataset = test_dataset.map(
process_features, num_parallel_calls=multiprocessing.cpu_count()
).batch(dataset_info.splits["test"].get_proto().statistics.num_examples)

for f in test_dataset:
images_t, labels_t = f["image"], f["label"]

model = model.inception(dataset_info.features["image"].shape)

# compile the model
model.compile(
optimizer=k.optimizers.RMSprop(learning_rate=LRATE, decay=DECAY),
loss=k.losses.categorical_crossentropy,
metrics=[k.metrics.categorical_accuracy],
)

tbCallback = [
k.callbacks.TensorBoard(
log_dir="./log", histogram_freq=0, write_graph=False, write_images=False
)
]
# tbCallback = [k.callbacks.TensorBoard(log_dir='./log')]

step = 0
for f in train_dataset:
images, labels = f["image"], f["label"]
step += 1
model.fit(
images,
labels,
validation_data=(images_t, labels_t),
steps_per_epoch=100,
epochs=1,
batch_size=BATCH,
callbacks=tbCallback,
)

Other info / logs

Traceback (most recent call last):
File "/run/media/federico/XData/PycharmProjectsXData/classificator/test.py", line 105, in <module>
callbacks=tbCallback,
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 963, in fit
steps_name='steps_per_epoch')
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 386, in model_iteration
callbacks.on_epoch_end(epoch, epoch_logs)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 261, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 1389, in on_epoch_end
self._write_custom_summaries(step, logs)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 1335, in _write_custom_summaries
summary_ops_v2.scalar(name, value, step=step)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 564, in scalar
return summary_writer_function(name, tensor, function, family=family)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 508, in summary_writer_function
should_record_summaries(), record, _nothing, name="")
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
return true_fn()
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 501, in record
with ops.control_dependencies([function(tag, scope)]):
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 562, in function
name=scope)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py", line 727, in write_scalar_summary
writer, step, tag, value, name=name, ctx=_ctx)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py", line 763, in write_scalar_summary_eager_fallback
attrs=_attrs, ctx=_ctx, name=name)
File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 66, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: Resource localhost/logdir:./log/N10tensorflow22SummaryWriterInterfaceE does not exist. [Op:WriteScalarSummary] name: epoch_loss/