Issue 1:
The current Java API's Tensor.create(Object) is really slow - for a batch of 128 images of size 224x224x3 it's taking around 1.5seconds. To put this into perspective runner.run() with that data and an InceptionV3 graph took below 1second so data prep is x1.5 of the runtime here (for a batch of 32 images it's around 0.35-0.45sec).

Is this working as intended? When running the Python code (using simple sess.run(fetches, feed_dict=feed_dict)) with which the graph meta file was generated (TF 1.0.1) and feeding a Python array I don't see such hiccups, the speed is the same as the Java runner.run().

Might it be because of build flags used, maybe I'm missing some optimizations?

For now this small part is killing the whole performance, bringing it down from 130obs/sec (runner.run() time) to about ~45obs/sec (Tensor.create+run()).

A bit of a sidenote, the performance page states:

This will result in poor performance.

sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

But currently there's no other way to feed data from the Java API, right? A queue (able to read from a file and from memory, i.e. from a Java structure) would be amazing.

Jar build command

export CC="/usr/bin/gcc"
export CXX="/usr/bin/g++"
export TF_NEED_CUDA=1
export GCC_HOST_COMPILER_PATH=$CC
export BUILDFLAGS="--config=cuda --copt=-m64 --linkopt=-m64 --copt=-march=native"

bazel build -c opt \
//tensorflow/java:tensorflow \
//tensorflow/java:libtensorflow_jni \
$BUILDFLAGS --spawn_strategy=standalone --genrule_strategy=standalone

Environment info

OS: Ubuntu 16.04

GPU: GPU TITAN X (Pascal) 12GB

CPU: Intel® Xeon® Processor E5-2630 v4 10core

GPU Drivers:

NVidia CUDA Driver Version: 375.39

CUDNN 5.1.5

CUDA 8

Tensorflow version: JAR file built from current master (c25ecb5)

Example

public void test() {
Random r = new Random();
int imageSize = 224 * 224 * 3;
int batch = 128;
float[][] input = new float[batch][imageSize];
for(int i = 0; i < batch; i++) {
for(int j = 0; j < imageSize; j++) {
input[i][j] = r.nextFloat();
}
}

long start = System.nanoTime();
Tensor.create(input);
long end = System.nanoTime();
// Around 1.5sec
System.out.println("Took: " + (end - start));
}

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): OS X El Capitan 10.11.6 (15G22010)

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A

TensorFlow installed from (source or binary): binary (anaconda)

output from conda list tensorflow:

tensorflow 1.12.0 mkl_py36h2b2bbaf_0
tensorflow-base 1.12.0 mkl_py36h70e0e9a_0

TensorFlow version (use command below): b'unknown' 1.12.0

Python version:

Python 3.6.6 |Anaconda custom (64-bit)| (default, Jun 28 2018, 11:07:29) **

[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin

Bazel version (if compiling from source): N/A

GCC/Compiler version (if compiling from source): N/A

CUDA/cuDNN version: N/A

GPU model and memory: N/A

Describe the current behavior

I get the following error message only under specific circumstances (see code below):

OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.

OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it candegrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Abort trap: 6

Describe the expected behavior

No error message

Code to reproduce the issue

import tensorflow as tf
import numpy as np

def get_dense(input, widths, activations):
assert len(widths) == len(activations)
output = input
for i, (w, a) in enumerate(zip(widths, activations)):
output = tf.layers.dense(output, units=w, activation=a)
return output

nn = lambda input, depth, width: get_dense(
input,
[width for _ in range(depth - 1)] + [3 ],
[tf.nn.relu for _ in range(depth - 1)] + [None]
)

tf.reset_default_graph()

# Works fine when N = 8192,
# but breaks with N = 8193:
N = 8193
n = 100

X = np.random.normal(loc=0.0, scale=1., size=(N, 1)).astype(np.float32)
nx = X.shape[1]
Y = X**2

XY = np.concatenate([X, Y], axis=1)

ds = tf.data.Dataset.from_tensor_slices(XY).batch(n).make_one_shot_iterator().get_next()

pred = nn(ds[:,:nx], 2, 10)
loss = tf.reduce_mean((pred - ds[:,nx:])**2)
op = tf.train.RMSPropOptimizer(0.0005).minimize(loss)

with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
try:
for i in range(20):
print(i)
_, l1 = sess.run([op, loss])
print(l1)
except tf.errors.OutOfRangeError:
pass

print("Done")

Other info / logs

I would consider this an anaconda problem if only it happened at all times. However, what I find weird, and why I suspect it might be a tensorflow issue, is that it only occurs when input dataset size is large enough. In this example the error pops up when N is 8193 or higher, no error when N is 8192 or lower. This "threshold" value is different in my original project where I first faced the problem - there it happened when dataset array length was 100001 (100k+1) or higher, while working fine with 100000 (100k) long dataset. Apologies if this is irrelevant to tensorflow and indeed an anaconda issue.

P.S.

Adding KMP_DUPLICATE_LIB_OK=TRUE does silence the error.