Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 17.10

TensorFlow installed from (source or binary): Source

TensorFlow version (use command below): master

Python version: 3.6.3

Bazel version (if compiling from source): bazel release 0.16.0

GCC/Compiler version (if compiling from source): gcc (Ubuntu 5.5.0-1ubuntu2) 5.4.1 20171010

CUDA/cuDNN version:9.0/7.0

GPU model and memory: Nvidia GTX 970

Exact command to reproduce: bazel build --verbose_failures -c opt //tensorflow/tools/pip_package:build_pip_package

Describe the problem

I tried building TF from source, however my build always fails. During the build, some standard library is not found. So far I have seen cassert, fstream, cstdint and stddef.h. When I compile a simple test program which just loads these and returns 0 with gcc everything goes well. I have tried adding paths to CROSSTOOL, but that didn't help (though I am not sure if the place in the file was correct).

Bazel output

ERROR: /home/kocur/.cache/bazel/_bazel_kocur/fe42bf71aeb545557cf2194b2de024c0/external/protobuf_archive/BUILD:260:1: C++ compilation of rule '@protobuf_archive//:js_embed' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
(cd /home/kocur/.cache/bazel/_bazel_kocur/fe42bf71aeb545557cf2194b2de024c0/execroot/org_tensorflow && \
exec env - \
LD_LIBRARY_PATH=:/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64:/usr/local/cuda-9.0/targets/x86_64-linux/lib/:/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64:/usr/local/cuda-9.0/targets/x86_64-linux/lib/ \
PATH=/home/kocur/.local/share/virtualenvs/code-B3GmqseA/bin:/home/kocur/bin:/home/kocur/.local/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/cuda/bin:/usr/local/Matlab/R2012a/bin:/usr/lib/jvm/java-10-oracle/bin:/usr/lib/jvm/java-10-oracle/db/bin \
PWD=/proc/self/cwd \
external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/host/bin/external/protobuf_archive/_objs/js_embed/embed.d '-frandom-seed=bazel-out/host/bin/external/protobuf_archive/_objs/js_embed/embed.o' -iquote external/protobuf_archive -iquote bazel-out/host/genfiles/external/protobuf_archive -iquote bazel-out/host/bin/external/protobuf_archive -iquote external/bazel_tools -iquote bazel-out/host/genfiles/external/bazel_tools -iquote bazel-out/host/bin/external/bazel_tools '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIE -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -g0 -g0 -c external/protobuf_archive/src/google/protobuf/compiler/js/embed.cc -o bazel-out/host/bin/external/protobuf_archive/_objs/js_embed/embed.o)
external/protobuf_archive/src/google/protobuf/compiler/js/embed.cc:31:19: fatal error: cassert: No such file or directory
compilation terminated.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 4.996s, Critical Path: 0.19s
INFO: 10 processes: 10 local.
FAILED: Build did NOT complete successfully

Output of echo | gcc -E -xc++ - -v

Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.5.0-1ubuntu2' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.1 20171010 (Ubuntu 5.5.0-1ubuntu2)
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-linux-gnu/5/cc1plus -E -quiet -v -imultiarch x86_64-linux-gnu -D_GNU_SOURCE - -mtune=generic -march=x86-64 -fstack-protector-strong -Wformat -Wformat-security
ignoring duplicate directory "/usr/include/x86_64-linux-gnu/c++/5"
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/5
/usr/include/x86_64-linux-gnu/c++/5
/usr/include/c++/5/backward
/usr/lib/gcc/x86_64-linux-gnu/5/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/5/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "<stdin>"
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):

Yes

OS Platform and Distribution

Windows 10 x64

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:

N/A

TensorFlow installed from (source or binary):

Source

TensorFlow version (use command below):

1.13.1

Python version:

3.7.1

Bazel version (if compiling from source):

N/A

GCC/Compiler version (if compiling from source):

N/A

CUDA/cuDNN version:

N/A

GPU model and memory:

N/A

Describe the current behavior

I try to fit a Sequential model with both a training dataset and a validation dataset with fit_generator function. After of running it shows - 'Sequential' object has no attribute 'total_loss'-

Describe the expected behavior

Training should work fine.

Code to reproduce the issue

import os
import random
import numpy as np
from PIL import Image
import multiprocessing
import tensorflow as tf
from resizeimage import resizeimage

def images_list(path):
images_list = []
XY = []
with open(path,"r") as file:
images_list = file.read().split('\n')
XY = [row.split(" ") for row in images_list if len(row.split(" ")) > 1]
return np.asarray(XY)

def load_images(X,Y,i):
root = 'E:\\images\\rvl-cdip\\rvl-cdip\\images'
img_matrixes = []
labels = []
length = len(X)
for index in range(len(X)):
matrix = Image.open(os.path.join(root, X[index].replace('/',"\\")))
img_matrixes.append(resize_image_500(matrix))
labels.append(Y[index])

img_matrixes = np.asarray(img_matrixes)
labels = np.asarray(labels)

assert len(img_matrixes) == len(labels)

#print("{}: Loaded {} images".format(i,length))

return np.reshape(img_matrixes,(img_matrixes.shape[0],500,500,1)),labels

def resize_image(img):
np_img = np.asarray(img)

if(np_img.shape[1] < 3235):
missing_width = 3235 - np_img.shape[1]
white_matrix = np.empty((1000,missing_width),dtype=float)
white_matrix.fill(255)
np_img = np.hstack((np_img, white_matrix))

assert np_img.shape[0] == 1000
assert np_img.shape[1] == 3235

return np_img

def resize_image_500(img):
resized = resizeimage.resize_cover(img, [500, 500])
np_img = np.asarray(resized)

assert np_img.shape[0] == 500
assert np_img.shape[1] == 500

return np_img

def iterate_minibatches(inputs, targets, batchsize):
assert len(inputs) == len(targets)
indices = np.arange(len(inputs))
np.random.shuffle(indices)
i = 0
for start_idx in np.arange(0, len(inputs) - batchsize + 1, batchsize):
excerpt = indices[start_idx:start_idx + batchsize]
i+=1
yield load_images(inputs[excerpt], targets[excerpt],i)

from tensorflow.keras import layers

model = tf.keras.models.Sequential()
#H1
model.add(layers.Conv2D(8, kernel_size=(5, 5), strides=(1, 1),
activation='relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
#H2
model.add(layers.Conv2D(16, kernel_size=(5, 5), strides=(1, 1),
activation='relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
#H3
model.add(layers.Conv2D(32, kernel_size=(5, 5), strides=(1, 1),
activation='relu'))
model.add(layers.MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
#H4
model.add(layers.Conv2D(64, kernel_size=(5, 5), strides=(1, 1),
activation='relu'))
model.add(layers.MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
#H5
model.add(layers.Flatten())
#Dense
model.add(layers.Dense(1024, activation='relu'))
model.add(layers.Dense(1024, activation='relu'))

model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),
optimizer='adam',
metrics=[tf.keras.metrics.Accuracy()])
#Training data
XY_train = images_list(train_path)
X_train = XY_train[:,0]
Y_train = XY_train[:,1].astype(int)

#Testing data
XY_test = images_list(test_path)
X_test = XY_test[:,0]
Y_test = XY_test[:,1].astype(int)

#Validation data
XY_val = images_list(valid_path)
X_val = XY_val[:,0]
Y_val = XY_val[:,1].astype(int)

batch_size = 750
history = model.fit_generator(generator=iterate_minibatches(X_train, Y_train,batch_size),
validation_data=iterate_minibatches(X_test, Y_test, batch_size),
# validation_data=None,
steps_per_epoch=len(X_train)//batch_size,
validation_steps=len(X_test)//batch_size,
verbose=1,
epochs=100,
use_multiprocessing=True,
workers=multiprocessing.cpu_count()
)

batch_size = 750
history = model.fit_generator(generator=iterate_minibatches(X_train, Y_train,batch_size),
validation_data=iterate_minibatches(X_test, Y_test, batch_size),
# validation_data=None,
steps_per_epoch=len(X_train)//batch_size,
validation_steps=len(X_test)//batch_size,
verbose=1,
epochs=100,
use_multiprocessing=True,
workers=multiprocessing.cpu_count()
)

WARNING:tensorflow:Using a generator with `use_multiprocessing=True` and multiple workers may duplicate your data. Please consider using the `keras.utils.Sequence` class.
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-10-367767e34155> in <module>
8 epochs=100,
9 use_multiprocessing=True,
---> 10 workers=multiprocessing.cpu_count()
11 )

~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
1424 use_multiprocessing=use_multiprocessing,
1425 shuffle=shuffle,
-> 1426 initial_epoch=initial_epoch)
1427
1428 def evaluate_generator(self,

~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training_generator.py in model_iteration(model, data, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch, mode, batch_size, **kwargs)
123
124 batch_function = _make_execution_function(
--> 125 model, mode, class_weight=class_weight)
126
127 # Create the queue for the generator.

~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training_generator.py in _make_execution_function(model, mode, class_weight)
425 if mode == 'train':
426 if not context.executing_eagerly():
--> 427 model._make_fit_function()
428 f = functools.partial(model.train_on_batch, class_weight=class_weight)
429 elif mode == 'test':

~\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py in _make_fit_function(self)
1924 ]
1925 self._make_train_function_helper(
-> 1926 '_fit_function', [self.total_loss] + metrics_tensors)
1927
1928 def _make_test_function_helper(self, fn_name, outputs, metric_updates=None):

AttributeError: 'Sequential' object has no attribute 'total_loss'