Can I use `tf.nn.ctc_beam_search_decoder` on Android?

Question:
I’m working with tflite on Android, and I have a model architecture that’s a MobilenetV2 backbone, and a GRU RNN, trained with CTC loss. I want to use ctc beam search decoding at inference time.
I’ve successfully made a .tflite model and tested that I can load it up and get the expected behaviour on my local machine.
BUT, it doesn’t work on my Android project as I get this error:
Could not build model from the provided pre-loaded flatbuffer: Unsupported custom op: FlexCTCBeamSearchDecoder, version: 1
The error is raised when I try to instantiate this class:
package com.stampfree.validation.tflite;
import android.app.Activity;
import java.io.IOException;
/** This TensorFlowLite classifier works with the float MobileNet model. */
public class DigicodeMobileNet extends Classifier {
    /**
     * Initializes a {@code ClassifierFloatMobileNet}.
     *
     * @param device a {@link Device} object to configure the hardware accelerator
     * @param numThreads the number of threads during the inference
     * @throws IOException if the model is not loaded correctly
     */
    public DigicodeMobileNet(Activity activity, Device device, int numThreads)
            throws IOException {
        super(activity, device, numThreads);
    }
    @Override
    protected String getModelPath() {
        return "tf_mobilenetv2_128x768_ctc_post_epoch08.tflite";
    }
}

I have tried this using the default dependencies:
dependencies {
    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
    implementation 'org.tensorflow:tensorflow-lite-select-tf-ops:0.0.0-nightly-SNAPSHOT'
}

and by building the AAR in a docker container.
Both approaches gave the same result.
Any tips? Happy to share more context on request.

Context:
Hi @Alexander_Soare
I have used a FlexCTCGreedyDecoder operator before at this 8 project. It seems that adding the
implementation 'org.tensorflow:tensorflow-lite-select-tf-ops:0.0.0-nightly dependency worked fine and to drop 70MB of the final .apk I have used also the building the AAR in a docker container. It is strange that it does not work for you.
First try with the above dependency and not the SNAPSHOT one. If you need explicitly the SNAPSHOT dependency check again the documentation here 2.
Ping me again if neither worked.
Thanks


The link to tflite’s HelloWorld example is broken and cannot be opened

Question:
微信截图_202107122034342560×1238 304 KB

微信截图_202107122034482560×1238 293 KB

      TensorFlow

Get started with microcontrollers  |  TensorFlow Lite 2

I can’t open the links to these examples

Context: Thank you @xudong_tang for the links. I will forward it properly.


FFT used in the Tensorflow Time Series tutorial

Question:
I am following the tutorial about Time Series from the official documentation of Tensorflow (Time series forecasting  |  TensorFlow Core 6)
Very early in the tutorial they transform the datetime to Sine and Cosine signals to help train the model. This signals are created using the number of seconds in a day and year. They then explain that it can be checked using a FFT function. In the graph that is generated there are clear peaks on “1/year” and “1/day”. I guess this is the reason they used this timelapses to create the Sine and Cosine columns.
In a classic FFT, the Y axis usually represents the amplitude of a sound, for instance. My question relies on what this stands for in terms of time and why the peaks are the values chosen for the sine and cosine columns.

Context: @markdaoust do you know about this?


Model is not predicting correct results keras tensorflow python

Question:
I am using tensorflow==1.15.5.
And I load my model like this
modelBlstm = tf.keras.models.load_model(modelPath,
                                            custom_objects={
                                                'elmo': hub.Module("https://tfhub.dev/google/elmo/2", trainable=False),
                                                'tf': tf
                                            })

My model needs elmo variable to be declared, so I passed the elmo  variable in custom objects, but with this it does not predict correctly, now I declare the elmo as different variable
and just use
modelBlstm = tf.keras.models.load_model(modelPath)
it works well with the same user query.
But this works only if i am running the same code in single file.
But now if integrate the same code in my project it shows elmo variable is undefined.
Tried declaring elmo outside the class, creating elmo variable inside the class but it does not work, even with in the project, i removed the class and just cerated the functions like this…
dirName = os.path.dirname(__file__)

modelPath = os.path.join(dirName, "models/elmo-strat-bilstm-dense-2.h5")

modelBlstm = tf.keras.models.load_model(modelPath)

labelModelPath = os.path.join(dirName, "..\config\labels_mapping.pkl")

modelLabel = joblib.load(labelModelPath)

stopwords = ['i',  'am', 'a',

             'to', 'is', 'do', 'in', 'of']

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=False)

print("************elmo",elmo)

with tf.Session() as session:

    K.set_session(session)

    session.run(tf.global_variables_initializer())

    session.run(tf.tables_initializer())

    modelBlstm = tf.keras.models.load_model(modelPath, )

def process(query: str) -> Intent:

    Logger.log("Processing query : " + query)

    query = __textPreprocess(query)

    print("query", query)

    intent = __predict(query)

    print(query)

    return intent

but even this does not work,
all other details i have mentioned in this SO question.
Any help?

      stackoverflow.com

  model is not predicting correct results keras tensorflow python

  python, tensorflow, keras, data-science

  asked by

    Sunil Garg

  on 10:09AM - 09 Jul 21 UTC

Context:
Sorry, I’m replying but I don’t have an answer to you.
What I’d suggest to maybe help you is, if you want to use TF 2.x and with another text embedding, you can try this tutorial: Classify text with BERT  |  Text  |  TensorFlow


Question: How would I respond to a user using the narrator’s voice?

Context:
@dmelt, can you give more information about the problem?
That would make it easier for the community to help you.


Question:
How to train and save the model in google colab or GCP & AWS  then import into pycharm.
I tried the pickle h5 model save method. is the h5 file enough for grading or do I need to buy Computing VM at GCP. I really don’t have GPU to train my model and can’t afford cloud GPU’s it’s expensive. please explain any other method.
Thank you for your time.

Context:
This answer can help you: I have a question to Exam(TF certification) - #5 by Laurence_Moroney 13
and to pass model around, don’t use pickle, try saving to a saved model (maybe compress it if necessary):  Using the SavedModel format  |  TensorFlow Core 2


Question:
Hi all ,
Is there are any method for clustering techniques using TensorFlow like KMeans , Meanshift , BRICH algorithms ?

Context:
We don’t have this implemented in the library but If are looking for an unsupervised solution you can take a look at:

      TensorFlow

TensorFlow Decision Forests 4

  A collection of state-of-the-art Decision Forest algorithms for regression, classification, and ranking applications.


How to re-use training results when migrate from 1.x to 2.3

Question:
Hi
I have results trained with tensorflow.contrib.slim.fully_connected layer and the training results are stored as “fully_connected/weights”.  When I migrate to TF2.3, the contrib package was not supported anymore. Instead, I am using tf.compat.v1.layers.dense(…, name=‘fully_connected’). But dense layer’s kernal is named as “kernel” while in tensorflow.contrib.slim.fully_connected, it was named as “weights”.
When load my trained results, it complain it cannot find “fully_connected/kernel”  in my checkpoints. (it was saved as “fully_connected/weights”.
How can I resolve this kernel name difference?
Thanks for your help

Context: Any comment is appreciated. Thanks.


Question:
Can anyone help me with this:
I am following the Tensorflow notebook for Few shot learning ( Google Colaboratory 3 )
In it, I saw that they were annotating the images using colab_utils.annotate(). I can’t understand the annotation format they are using (like YOLO or COCO format). Another problem is that we can’t specify the classes at the time when we are drawing the bounding boxes and I have to remember the order in which I annotate the different images and classes so I can add them by code later on.
If someone can tell me what’s that format so I can annotate the images on my PC locally rather than on COLAB which will save a lot of time.
Any help would be appreciated.
Regards
Original Question Credits: tensorflow - colab_utils.annotate(), annotation format - Stack Overflow 5

Context:
The annotation format is in a docstring  in the source code:

      github.com

    tensorflow/models/blob/master/research/object_detection/utils/colab_utils.py#L436-L459 6

          The input of the call back function is a list of list of objects
          corresponding to the annotations. The format of annotations is shown
          below

[
            // stuff for image 1
            [
              // stuff for rect 1
              {x, y, w, h},
              // stuff for rect 2
              {x, y, w, h},
              ...
            ],
            // stuff for image 2
            [
              // stuff for rect 1
              {x, y, w, h},
              // stuff for rect 2
              {x, y, w, h},
              ...

  This file has been truncated. show original


GradientBoostedTreeClassifier Model in Tensorflow Lite

Question: How to convert tfdf.keras.GradientBoostedTreesModel into tensorflow lite format?

Context:
Hi,
TF-DF is not yet compatible with TF Lite. Here is 6 the feature request. You can ping on the issue to help prioritize it .
Cheers,
M.


Teaching neural network to predict HCF of 2 numbers

Question: Hello all. Hope you would be doing good. I am a newbie, I just want to ask that finding an HCF of 2 numbers, by teaching the ANN giving the data of 2 numbers with their HCF. Is this problem achievable?

Context:
You can take a look at:

      github.com

deepmind/mathematics_dataset 3

  This dataset code generates mathematical question and answer pairs, from a range of question types at roughly school-level difficulty.


Question:
Hi, I need to implement a custom convolution layer which is not supported by Tensorflow and TF Lite, so I tried to define it by using the tutorial to have a TF operator for a custom op and the guide to have a custom op supported by TF Lite. However, when I try to convert the operator with TF Lite converter, I get this error:
 Traceback (most recent call last):
 File "es.py", line 39, in <module>
 converter =
 tf.lite.TFLiteConverter.from_concrete_functions([tf.function(convol1d).get_concrete_function(inp)])
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py",
 line 1299, in get_concrete_function
 concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py",
 line 1205, in _get_concrete_function_garbage_collected
 self._initialize(args, kwargs, add_initializers_to=initializers)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py",
 line 725, in _initialize
 self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-
 access
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line
 2969, in _get_concrete_function_internal_garbage_collected
 graph_function, _ = self._maybe_define_function(args, kwargs)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line
 3314, in _maybe_define_function
 self._function_spec.canonicalize_function_inputs(*args, **kwargs)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line
 2697, in canonicalize_function_inputs
 inputs, flat_inputs, filtered_flat_inputs = _convert_numpy_inputs(inputs)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line
 2753, in _convert_numpy_inputs
 a = _as_ndarray(value)
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line
 2711, in _as_ndarray
 return value.__array__()
 File "/home/em/venv/lib/python3.8/site-packages/tensorflow/python/keras/engine
 /keras_tensor.py", line 273, in __array__
 raise TypeError(
 TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may
 indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or,
 you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register
 dispatching, preventing Keras from automatically converting the API call to a lambda layer in the
 Functional Model.

The code is like this:
import tensorflow as tf
tf.config.run_functions_eagerly(True)
from keras.datasets import mnist
from keras.models import Model
from keras.layers import add,Input,Activation,Flatten,Dense

def convol(inp):
    conv_module = tf.load_op_library('./conv.so')
    x = conv_module.conv(inp, name="Conv")
    return x

def read_mnist(path):
    (train_x,train_y), (test_x,test_y)=mnist.load_data()
    return train_x,train_y,test_x,test_y

def tcn(train_x,train_y,test_x,test_y):
      inp=Input(shape=(28,28))
      x = convol(inp)
      x=Flatten()(x)
      x=Dense(10,activation='softmax')(x)
      model=Model(inputs=inp,outputs=x)
      model.summary()
      model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=
      ['accuracy'])
       model.fit(train_x,train_y,batch_size=100,epochs=10,validation_data=(test_x,test_y))
       pred=model.evaluate(test_x,test_y,batch_size=100)
       print('test_loss:',pred[0],'- test_acc:',pred[1])

 train_x,train_y,test_x,test_y=read_mnist('MNIST_data')
 tcn(train_x,train_y,test_x,test_y)
 tflite_model_name = 'net'
 inp=Input(shape=(28,28))
 converter =
 tf.lite.TFLiteConverter.from_concrete_functions([tf.function(convol).get_concrete_function(inp)])
 converter.allow_custom_ops = True
 tflite_model = converter.convert()
 open(tflite_model_name + '.tflite', 'wb').write(tflite_model)

Context:
Hi @Gianni_Rossi
So does the model train properly but not converted? Is there a colab notebook to reproduce?


Expanding tflite object detector functionality

Question:
Hi,
I am trying to expand the model from tflite object detector with OCR, I have exported the underlying serving model and I have some trouble understanding the output I’m getting from the keras model. The output I’m getting is 2 arrays with pyramid like outputs.

EfficientDet architecture2264×820 207 KB

I’m trying to understand how I can get bboxes and predictions (scores and class) out of it. I’ve been looking for clues here but I can’t wrap my head around it.
I’d like to stay with tflite implementation of the model as it provides me with very convenient training and the results with just some augmentation added on top of what tflite already provides is getting me satisfying results and just use the underlying model to combine it with OCR and later export it back to tflite for mobile use.

Context:
Can you try something like this?

interpreter = tf.lite.Interpreter(model_content=tflite_model_path)
interpreter.allocate_tensors()
fn = interpreter.get_signature_runner()
output = fn(images=image)
print(output)


importError: cannot import name ‘L2’ from ‘tensorflow.keras.regularizers’ (/usr/local/lib/python3.7/dist-packages/tensorflow/keras/regularizers/__init__.py)

Question:
I have just installed Tensorflow 2.2.0 from a source on my Raspberry Pi 4B running Raspbian 32-bit (armv7l) OS, but when I try to run python code, it gives me the following error, I don’t know if it is something about the version as it doesn’t say anything specifically about the version of TF.
importError: cannot import name ‘L2’ from ‘tensorflow.keras.regularizers’ (/usr/local/lib/python3.7/dist-packages/tensorflow/keras/regularizers/init.py)

Context: Check 2.2 api docs. Changing L2 to l2 should work. tf.keras.regularizers.l2  |  TensorFlow Core v2.2.0 11


Problem running Tensorflow BERT tutorial on GPU

Question:
Hello everybody.
I was trying to run the Tensorflow BERT tutorial for text classification ( Classify text with BERT  |  Text  |  TensorFlow 5 ), but it does non want to run on my GPU. Does anyone faced the same issue?
Thank you.

Context:
Hi @Stefano_Di_Pietro , I am training the model with GPU enabled in Colab (in the cloud) without issues (notebook: Google Colaboratory 2 → Edit > Notebook settings > GPU).
So, this may be specific to your setup. Have you had issues with training other TensorFlow 2.x models on your GPU? Can you share your OS, GPU model, cuDNN/CUDA versions?
Hardware requirements: GPU support  |  TensorFlow 2
Software requirements: GPU support  |  TensorFlow 2
GPU support: GPU support  |  TensorFlow


Cannot read property ‘children’ of undefined with using Tensorflow Object Detection API model in tfjs

Question:
Please make sure that this is a bug. As per our
GitHub Policy 1,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow.js installed from (npm or script link): npm install @tensorflow/tfjs
TensorFlow.js version (use command below): 3.6.0
Browser version: Chrome 90 on Windows 10
Tensorflow.js Converter Version: 3.6.0

Hyperlinks are capitalized to prevent confusion(because the color settings(and blue filter) on my monitor make it hard to see the difference between them)
Describe the current behavior
I used transfer learning using a pretrained model from the Tensorflow Object Detection API, which i converted to tensorflow-js using the tensorflow-converter API in python. View the ipynb notebook HERE 3. I then followed @hugozanini’s REPO 5 which had a template for using your tensorflow object detection models in javascript;  I copied the index.json file and put it into a sandbox, replacing his model.json file with mine. The repo which contains it can be found here 2
Describe the expected behavior
I was expecting the model to work and the program to run normally, like it did in this DEMO. Instead, I got
index.js:1437 TypeError: Cannot read property 'children' of undefined
    at operation_mapper.js:409
    at Array.forEach (<anonymous>)
    at operation_mapper.js:403
    at Array.forEach (<anonymous>)
    at OperationMapper.mapFunction (operation_mapper.js:401)
    at operation_mapper.js:163
    at Array.reduce (<anonymous>)
    at OperationMapper.transformGraph (operation_mapper.js:162)
    at GraphModel.loadSync (graph_model.js:159)
    at GraphModel._callee$ (graph_model.js:119)
    at tryCatch (runtime.js:62)
    at Generator.invoke [as _invoke] (runtime.js:288)
    at Generator.prototype.<computed> [as next] (runtime.js:114)
    at asyncGeneratorStep (asyncToGenerator.js:3)
    at _next (asyncToGenerator.js:25)

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/CodePen/any notebook.
HERE 3 is the colab notebook I used to train my model
HERE 4 is the sandbox where I tried to use the tensorflow.js model
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
Error Message:
index.js:1437 TypeError: Cannot read property 'children' of undefined
    at operation_mapper.js:409
    at Array.forEach (<anonymous>)
    at operation_mapper.js:403
    at Array.forEach (<anonymous>)
    at OperationMapper.mapFunction (operation_mapper.js:401)
    at operation_mapper.js:163
    at Array.reduce (<anonymous>)
    at OperationMapper.transformGraph (operation_mapper.js:162)
    at GraphModel.loadSync (graph_model.js:159)
    at GraphModel._callee$ (graph_model.js:119)
    at tryCatch (runtime.js:62)
    at Generator.invoke [as _invoke] (runtime.js:288)
    at Generator.prototype.<computed> [as next] (runtime.js:114)
    at asyncGeneratorStep (asyncToGenerator.js:3)
    at _next (asyncToGenerator.js:25)

Context: Generally I suggest you to select tags from the menu on a new thread cause specialized technical team members could be subscribed only to a tag subset (e.g. in this case I suggest tfjs and help_request)


Installation problem before applying reinforce agent

Question:
Hello everyone,
I am new to TensorFlow and I want to use reinforce agent codes. Before that, I need to install the followings:
sudo apt-get update
sudo apt-get install -y xvfb ffmpeg
pip install ‘imageio==2.4.0’
I tried to install them in a Jupyter cell, Anaconda prompt and Command prompt but it was unsuccessful.  I get syntax error as “Invalid syntax”. Could you please help me and let me know how to fix this problem?
Thank you.

Context:
Hi F_J,
I’ve just tried on colab and it works like this (I tried one line per cell):
!apt-get update
!apt-get install -y xvfb ffmpeg
!pip install imageio==2.4.0


Question:
I’m wondering why all Keras layers use Glorot initialization as default. Since Relu is the most popular activation function, shouldn’t He be the default initialization?
The prebuilt application models such as ResNet50, also use Glorot initialization as default and there is no parameter to pass and modify it.

Context:
Are you talking about this:

      Medium – 6 Jul 19

Why default CNN are broken in Keras and how to fix them 15

  A deep dive into CNN initialization…

    Reading time: 6 min read


I have a question to Exam(TF certification)

Question:
Hey masters,
maybe the question is impossible to get an answer.
Does the exam grader judge the code or just the last uploaded models?
Because I am pro-user of colab, I think I can use it to complete the exam fully, however, I really not sure, must I copy back the code to IDE(exam environment)?
Thank you

Context: Maybe @Jocelyn_Becker or @Laurence_Moroney might be able to shine some light here


‘function_optimizer.py’ returns empty graph

Question:
Hello,
I am trying to run a SavedModel that I have exported from a checkpoint using ‘exporter_main_v2.py’ and during conversion, the ‘function_optimizer.py’ optimizer seems to return an empty graph (0 nodes). Is there any reason that it does that?
Thanks

Context:
More context: I am trying to convert a MobilenetV2 SSD re-trained SavedModel into another framework format. I have successfully re-trained the model and exported the checkpoint to a SavedModel using exporter_main_v2.py. However, when I try to run the conversion script, it calls the 'meta_optimizer.cc file, which eventually results in an empty graph that cannot be converted.
The following is the output that I get:

empty_nodes_grappler1337×207 51 KB


Help wanted: best way to get TF, Python, GCC and CUDA versions matched for installation

Question:
Hello Community
1st) Pls tell me if I am completely at the wrong place with my request/question
2nd) My Question is:
Which is the best/most easiest way to setup a TF/CUDA env on Debian Buster?
May I describe my difficulties at first:
There is (a very, very special) dependency in setting both successfully up on a (Debian) Linux System: (on which’s investigations not always lead to a successful one)

GCC used/requested in TF - CUDA working only w/ spec. GCC which is in contrast to TF - std. inst. of GCC on Linux system which differs very often to both

 → How to find out which CUDA incl. GCC works with which version of TF AND the GCC version on Linux system? And, which may be the traps/pit falls I may stumble in?

There are also (nearly the same very special) dependencies concerning the used/necessary python/pip versions to be installed/already installed.

 → So, the same question here belonging to Python

There is also a specialty regarding the way/prereq’s on how to run the ‘.run’ file of CUDA

Once done wrong, it is very hard to find all the places where there had been tracks/spots been left of a former setup to be get the system cleaned up very clearly and than to get a second try successfully run of the CUDA ‘.run’ file to get it installed.
Since I am not a developer it is very hard for me to setup CUDA together with TF successfully on Debian Buster to have it right at hand for teaching purposes.
So, which way of setting up those incl. correct dependencies would you suggest to be successful in this matter?
Thank you very much for any helpful response which will be highly appreciated.
Kind regards from Switzerland,
Roger

Context:
Personally I prefer to directly use or build derived images from the official Dockerhub repo:
https://hub.docker.com/r/tensorflow/tensorflow/ 4


Model.evaluate() not returning expected loss in certain contexts

Question:
In the following snippet, I construct a very simple neural network and evaluate it on some synthetic data. I don’t train the network, just evaluate it with the initial weights. I’m using binary cross entropy, and I compute the loss in two ways:

By calling model.evaluate().
By calling model.predict() and manually computing the loss.

I expect to get the same result in each case, but I consistently get very different values.
import keras
import numpy as np

input_dimension = 100

rng = np.random.default_rng(0)
train_xs = rng.uniform(0, 255, (1, input_dimension))
train_ys = rng.uniform(0, 1, (1, 1))

keras_model = keras.Sequential([
    keras.layers.Dense(1000, activation="relu"),
    keras.layers.Dense(1, activation="sigmoid")
])

keras_model.compile(
    optimizer=keras.optimizers.SGD(learning_rate=0.01, momentum=0.0, nesterov=False),
    loss=keras.losses.binary_crossentropy,
    metrics=["accuracy"]
)

keras_model.build(input_shape=(1, input_dimension))

y = keras_model.predict(train_xs)

loss, _ = keras_model.evaluate(train_xs, train_ys, verbose=0)

print(loss)
print(keras.losses.binary_crossentropy(train_ys, y))

If the activation in the hidden layer is changed from ReLU to sigmoid, the behavior goes away. What’s going on?

Context:
I believe I’m closer to figuring it out. Here’s a simpler case:
import keras
import numpy as np
import tensorflow as tf

def binary_cross_entropy(y_hat, y):
    epsilon = 1e-7
    y_hat = np.clip(y_hat, epsilon, 1 - epsilon)
    bce = y * np.log(y_hat + epsilon)
    bce += (1 - y) * np.log(1 - y_hat + epsilon)
    return -np.mean(bce)

train_xs = np.ones((1, 1000)) * 10000
train_ys = np.zeros((1, 1))

tf.random.set_seed(2)
keras_model = keras.Sequential([
    keras.layers.Dense(1000, activation="linear"),
    keras.layers.Dense(1, activation="sigmoid")
])

keras_model.compile(
    loss=keras.losses.binary_crossentropy,
    metrics=[]
)

y = keras_model.predict(train_xs)
assert y.shape == train_ys.shape

metrics = keras_model.evaluate(train_xs, train_ys, verbose=0, return_dict=True)

print("Outputs:", y)
print(metrics["loss"])
print(keras.losses.binary_crossentropy(train_ys, y))
print(binary_cross_entropy(y, train_ys))

So now I’m using non-random, constant inputs, and my target is zero. I also changed the activation in the hidden layer to linear (it also works with ReLU - just not with sigmoid) I’m computing the BCE in three ways:

model.evaluate()

model.predict() and Keras’s implementation of BCE

model.predict() and my own implementation of BCE

(2) and (3) match, but (1) gives a very different value. What surprises me is that the loss should depend only on the outputs of the model, but if I change the constant 1000 at line 12 to 10000 or 100000, the value of (1) continues to increase, even though the output of the model is constant the whole time (equal to 1). For example, when the constant is 1000:
Outputs: [[1.]]
909.5416870117188
tf.Tensor([15.333239], shape=(1,), dtype=float32)
15.33323860168457

when it’s 10000:
Outputs: [[1.]]
9095.416015625
tf.Tensor([15.333239], shape=(1,), dtype=float32)
15.33323860168457

Notice the output of model.evaluate() was scaled by a factor of 10, just like the inputs to the model. This leads me to the following hypothesis. Presumably, inside model.evaluate(), the model’s predictions are computed using a higher precision than what I’m getting by running model.predict() myself. Because the sigmoid exponentially decays towards 1 at infinity, when I scale the inputs by 10, this scales the inputs to the sigmoid by 10 (because I’m using linear or ReLU activations in the hidden layer), which causes the sigmoid’s output to be, basically, 1 - exp(-10t) instead of 1 - exp(-t). Because BCE is logarithmic, this undoes that and we get a loss which is ten times higher. However, when I do this same calculation in (2) or (3), the output of the model is saturated at 1 and so nothing changes.
This difference might seem trivial, but I think it actually has dramatic consequences. When I train certain networks using model.fit(), which apparently uses the same higher-precision calculations that model.evaluate() is using, I get 90% test accuracy. At the lower precision of the manual model.predict() method (implemented using another library, namely JAX), in the same scenario I get 50% accuracy! I believe this discrepancy is what’s ultimately behind issue #7171 I raised on the JAX issue tracker (sorry, I’m not allowed to post a link).
What I would like to know is: how does model.evaluate() compute the model’s predictions and the BCE loss? I tried debugging it, but the source code is completely opaque to me, jumping through layer after layer of wrappers and compilation steps. How can I, for example, compute my model’s predictions in the same way that model.evaluate() does it, and then compute the loss in the same way that model.evaluate() does it?


Extreme model accuracy loss due to TFLITE conversion w/ quantization

Question:
Hello together,
i currently work on training a object detection model using a ssd mobilenet v2 configuration in tensorflow 2.5.
This in general works ok with the training finishing around ~0.1 loss. Loading the model results in good detections with which i can work so far.
Problem is:
My current test cases all run on single images. In the final application this model is supposed to do the object detection live on incoming camera images. So the plan was to use a tpu and convert the trained model to tflite for the coral edgetpu compiler to be able to handle it.
For this to work the tflite model needs to be 8 bit integer quantized but the model becomes unusable after beeing converted to tflite with 8 bit quantization. If my understanding of my model configuration is correct i even used quantization aware training to reduce the model accuracy loss but it still just results in garbage detections.
I use the following configuration for an ssd mobilenet v2:
model {
  ssd {
    inplace_batchnorm_update: true
    freeze_batchnorm: false
    num_classes: 3
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
        use_matmul_gather: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    encode_background_as_zeros: true
    anchor_generator {
      ssd_anchor_generator {
        num_layers: 6
        min_scale: 0.2
        max_scale: 0.95
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        aspect_ratios: 3.0
        aspect_ratios: 0.3333
      }
    }
    image_resizer {
      fixed_shape_resizer {
        height: 514
        width: 614
      }
    }
    box_predictor {
      convolutional_box_predictor {
        min_depth: 0
        max_depth: 0
        num_layers_before_predictor: 0
        use_dropout: false
        dropout_keep_probability: 0.8
        kernel_size: 1
        box_code_size: 4
        apply_sigmoid_to_scores: false
        class_prediction_bias_init: -4.6
        conv_hyperparams {
          activation: RELU_6,
          regularizer {
            l2_regularizer {
              weight: 0.00004
            }
          }
          initializer {
            random_normal_initializer {
              stddev: 0.01
              mean: 0.0
            }
          }
          batch_norm {
            train: true,
            scale: true,
            center: true,
            decay: 0.97,
            epsilon: 0.001,
          }
        }
      }
    }
    feature_extractor {
      type: 'ssd_mobilenet_v2_keras'
      min_depth: 16
      depth_multiplier: 1.0
      conv_hyperparams {
        activation: RELU_6,
        regularizer {
          l2_regularizer {
            weight: 0.00004
          }
        }
        initializer {
          truncated_normal_initializer {
            stddev: 0.03
            mean: 0.0
          }
        }
        batch_norm {
          train: true,
          scale: true,
          center: true,
          decay: 0.97,
          epsilon: 0.001,
        }
      }
      override_base_feature_extractor_hyperparams: true
    }
    loss {
      classification_loss {
        weighted_sigmoid_focal {
          alpha: 0.75,
          gamma: 2.0
        }
      }
      localization_loss {
        weighted_smooth_l1 {
          delta: 1.0
        }
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
    normalize_loss_by_num_matches: true
    normalize_loc_loss_by_codesize: true
    post_processing {
      batch_non_max_suppression {
        score_threshold: 1e-8
        iou_threshold: 0.6
        max_detections_per_class: 5
        max_total_detections: 15
      }
      score_converter: SIGMOID
    }
  }
}

train_config: {
  batch_size: 8
  sync_replicas: true
  startup_delay_steps: 0
  replicas_to_aggregate: 8
  num_steps: 40000
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    random_crop_image {
      min_object_covered: 0.0
      min_aspect_ratio: 0.75
      max_aspect_ratio: 3.0
      min_area: 0.75
      max_area: 1.0
      overlap_thresh: 0.0
    }
  }
  optimizer {
    momentum_optimizer: {
      learning_rate: {
        cosine_decay_learning_rate {
          learning_rate_base: .13
          total_steps: 40000
          warmup_learning_rate: .026666
          warmup_steps: 1000
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  max_number_of_boxes: 100
  unpad_groundtruth_tensors: false
}

train_input_reader: {
  label_map_path: "...\\label_map.pbtxt"
  tf_record_input_reader {
    input_path: "...\\training.tfrecord"
  }
}

eval_config: {
  metrics_set: "coco_detection_metrics"
  use_moving_averages: false
}

eval_input_reader: {
  label_map_path: "...\\label_map.pbtxt"
  shuffle: false
  num_epochs: 1
  tf_record_input_reader {
    input_path: "...\\eval.tfrecord"
  }
}
graph_rewriter {
  quantization {
    delay: 1000
    weight_bits: 8
    activation_bits: 8
  }
}

I’d like to show some images in here but the forum wont let me. Will try to figure out a way to post some example images in here…
As a description of what you’d see: The tflite model creates like 5 detections with 50 cofidence and horrible bounding boxes (2-3x times too large and the center of the object is somewhere on the border line) sometimes even with the wrong label. Meanwhile the original model is always 100% spot on with at least 99% confidence and perfectly fitting bounding boxes (max 2-3 pixels off)
I am fully aware that a quantized model will never bring the same accuracy as a float model but the tensorflow documentation led me to believe that the accuracy loss should be somewhat around <3%.
While i do not know how the term “accuracy loss” is defined here i would not think that my results are something that can be described as a <3% accuracy loss.
For completion of my workflow: I convert the model from checkpoints to saved model format using the script export_tflite_graph_tf2.py from the object detection API. After this i use the following python script to load the model, convert it and save it:
def representative_dataset_gen():
    data = tf_training_helper.load_images_in_folder_to_numpy_array(IMAGE_BASE_PATH)
    (count, x,y,c) = data.shape
    for i in range(count):
        yield [data[i,:,:,:].reshape(1,x,y,c).astype(np.float32)]

input_data = tf_training_helper.load_image_into_numpy_array(IMAGE_PATHS)

converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_PATH)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.target_spec.supported_types = [tf.int8]

tflite_model = converter.convert()
print("model conversion finished. Starting validation... ")
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

interpreter.set_tensor(input_details[0]['index'], input_data.astype(np.uint8))

interpreter.invoke()
out_boxes = interpreter.get_tensor(output_details[0]['index'])
out_classes = interpreter.get_tensor(output_details[1]['index'])
out_scores = interpreter.get_tensor(output_details[2]['index'])
with tf.io.gfile.GFile(MODEL_SAVE_PATH, 'wb') as f:
  f.write(tflite_model)

(i redacted some unecessary parts from it to make it a bit shorter. (e.g. path definitions))
If you have any idea on how this loss in accuracy can be prevented or even just why it is there please let me know. Any ideas or suggestions are welcome. (e.g. is ssd mobilenet the correct thing to go for? are other models more robust to conversion?)
Please also let me know if i missed to add any information, i’d be happy to provide it.
Have a great weekend,
Cheers
Georg

Context:
hello,

how exactly you performed QAT? , did you fine tune the model or trained it from scratch? fine tune is better , are you quantizing all layers in the model? , there might be operations in the first layers that once quantized make the model generalization performance decrease.

I think once you have done QAT, you don’t need a representative dataset to calibrate the range of the activations, just run the converter with converter.optimizations = [tf.lite.Optimize.DEFAULT] like it is done here in the docs: Quantization aware training comprehensive guide 5

you can check the model layers details in the Netron app, I use it to debug and see if the layer inputs/outputs and weights make sense after quantization.


TypeError: Input ‘y’ of ‘Sub’ Op has type float16 that does not match type float32 of argument ‘x’

Question:
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS
TensorFlow installed from (source or binary): Colab
TensorFlow version (use command below): 2.5.0
Python version: python 3.7
GPU model and memory: Tesla T4

Error
TypeError: Input 'y' of 'Sub' Op has type float16 that does not match type float32 of argument 'x'
Current behaviour
While using Mixed Precision and building a Keras Functional API model (EfficientNet B0), it shows the below error

image1870×762 160 KB

image2496×684 128 KB

Describe the expected behaviour
The Global Policy I set in the previous cell was mixed_float16. The problem works fine when running on tensorflow 2.4.1 so the bug is with tensorflow 2.5.0
You can reproduce the same error using the below notebook :

      colab.research.google.com

Google Colaboratory 24

Context: Can you post the code in the screenshot? It seems that the colab Is a more complex and slow example to reproduce.


Question:
Hi all,
Which is the best way to learn pure tensorflow without using any libraries. Also if there are researches or projects used that, please mentioned it.
Thank you.

Context:
Hi there,
Before I provide a good source to learn TensorFlow code-first, I want to ask why exactly the constraint of not using any libraries?
Other than that, there is a beautiful TensorFlow course that teaches in a code-first approach, meaning you learn by doing. I think this is a perfect way to be introduced to this topic.
https://dev.mrdbourke.com/tensorflow-deep-learning/ 11
It’s written by Daniel Bourke, who is sometimes also roaming these forums
Hope this helps!


Question:
Hi all!
I’ve to implement cnn on a multi-dimensional signal data whose shape is somehow similar to that of a colorful image, as in each of my data points has a dimension of [65 * 1 * 1100] (analogy with a colored image is that instead of 3 layers of colors i have 65 layers, and for (length * width) of an image i have (1*1100). These data are currently in python list form because from raw csv file i first extract & convert them to python list.
I just wanted to know, how exactly can they be converted from list to a form which will be suitable for tensorflow, and all the train/test split type tasks may be done easily?
I know this seems to be a silly problem but I’m relatively new to tf and i really don’t know what to do here. Any help is really appreciated!

Context:
hello, you can check the tf.data.Dataset documentation here:  tf.data.Dataset  |  TensorFlow Core v2.5.0 2
if you want to build a tensorflow dataset from a list you can use data = tf.data.Dataset.from_tensor_slices(your_list) once you have this dataset, you can compute any other stuff wtih tf.data.Dataset methods and pass the dataset to your model, either with model.fit or looping through the elements of the dataset like in the documentation example (in that case you will need to create your custom training loop and training/test step)


Question:
I need to deploy a RandomForestModel regression model, then consume it.

I have made a zip file of the entire folder containing the trained RandomForestModel regression model for deployment.
In a separate notebook, I unzipped and loaded the model: loadedModel = tf.saved_model.load(’/RF_model’)
With the loaded model, I need to predict with a new dataset. However, I can’t find any information about what methods I have to use.
Any help is greatly appreciated.

Context:
Hi Frank,
you can try something like this:
reloaded_model = tf.saved_model.load('./my_model')
result = reloaded_model(your_data)


Problem about Copying the weight from pytorch model to tf2 model

Question:
I try to copy the weights from a pytorch model to tf2 model. I can successfully do the influence. I output the feature map to have a look. However, the feature map after a conv2d has black bars on the top and left hand side of the map.
i-stack-imgur-com/E45K6.png
The pytorch feature map does not have these black bars. The pytorch code of conv2d is nn.Conv2d(1, 56, kernel_size=5, padding=5//2) The tensorflow build is Conv2D(56, 5, padding=“same”)(inp) The copying weight from pytorch model to tf2 is :
    onnx_1_w_num = onnx_l.weight.data.permute(2,3,1,0).numpy()
    onnx_1_b_num = onnx_l.bias.data.numpy()
    tf_l.set_weights([onnx_1_w_num,onnx_1_b_num])

What is wrong? Thanks.

Context: anyone can help? I have no clue for that. Many thanks!


Delivering a URL in the Tensorflow Chatbot “tag” attribute output

Question:
We’re having difficulty being able to identify which object or parameter we could use to deliver a URL (e.g., the chatbot transcript) to our user at the end of a chatbot session. Does anyone have any suggestions? What we’ve tried so far just over-layers the chatbot window we’ve created in Python.
Example from our json structure:
{“intents”: [
{“tag”: “greeting”,
“patterns”: [“Hi”, “How are you”, “Is anyone there?”, “Hello”, “Good day”, “Whats up”],
“responses”: [“Hello!”, “Good to see you again!”, “Hi there, how can I help?”],
“context_set”: “”
},
{“tag”: “goodbye”,
“patterns”: [“cya”, “See you later”, “Goodbye”, “I am Leaving”, “Have a Good day”],
“responses”: [“Sad to see you go :(”, “Talk to you later”, “Goodbye!”],
“context_set”: “”
},

Context:
Hi Dave,
I didn’t understand your question, can you clarify a little bit?
that would make easier for the community to help.


How do I build a neural network with variable length outputs?

Question:
Hi! I’m a beginner when it comes to TensorFlow and machine learning in general, and I want to try and create a model that can syllabify words, for example:
Input data: 'syllable', 'tensorflow'
Output data: 'syl-la-ble', 'ten-sor-flow'
I think that a Recurrent Neural Network with LSTM would be most appropriate for this, but I am unsure as to what the model’s structure would be because the output (syllabified words) would be variable-length based on the length of the word that is inputted, and I have not encountered anything like this in the course of my self-study.

Context:
Just to take an initial idea you can find an old reference of Language-Agnostic Syllabification with Neural Sequence Labeling in:

      github.com

jacobkrantz/lstm-syllabify 15

  Breaks a word into syllables using an LSTM-based neural network.


Question:
I’m working on a Transformer based model and I followed the great example of the positional encoding from:
Since the original implementation relies heavily on Numpy, I’ve created a pure TF variation that runs 100 times faster.
Hopes it would help others too:
import tensorflow as tf
def get_angles(pos, i, d_model):
return pos * (1 / tf.math.pow(10000.0, (2 * (i // 2)) / d_model))
@tf.function
def positional_encoding(pos, d_model):
angle_rads = get_angles(tf.range(pos, dtype=tf.float32)[:, tf.newaxis],
tf.range(d_model, dtype=tf.float32)[tf.newaxis, :], d_model)
return tf.reshape(tf.concat([tf.expand_dims(tf.sin(angle_rads[:, ::2]), axis=-1),
tf.expand_dims(tf.cos(angle_rads[:, 1::2]), axis=-1)], axis=-1), [1, pos, -1])
n, d = 2048, 512
timeit pos_encoding = positional_encoding(n, d)
137 µs ± 2.2 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

Context:
Sorry, I omitted the from link path (text/tutorials/transformer)
And the original timing of the tutorial implementation is:
timeit pos_encoding = positional_encoding(n, d)
17.8 ms ± 582 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)


Question: AttributeError: module 'tensorflow.core.framework.types_pb2   this error giving when i was executing code please help me

Context: We need a little bit of context here. What is your use case?


Question:
I’m using TensorFlow 1.15 on anaconda.
However “The TensorFlow contrib module will not be included in TensorFlow 2.0.” error occurs.
Why? I’m not using TensorFlow 2.0.
There any reason that it does that?
Thanks

Context:
It’s just a warning message that informs you that the tf.contrib module won’t be included in the next version of TensorFlow (the first 2.0 release) and you should upgrade your code for making it work with the new version.
If you don’t plan to upgrade it, you can safely ignore that warning


Question:
Hi folks,
I am delighted to share my latest example on keras.io 5 - Self-supervised contrastive learning with SimSiam:

      keras.io

Keras documentation: Self-supervised contrastive learning with SimSiam 50

This one introduces you to the world of self-supervised learning for computer vision and at the same time walks you through a decent self-supervised learning method (SimSiam) for the field.
While self-supervision has been pretty predominant in the NLP world since time immemorial but it was only in the past year these methods showed progress for the vision systems. We have evidence that these methods can often beat their supervised counterparts but only that their pre-training does not require any labeled data (but far more inductive priors).
Happy to address any feedback on the post.

Context: Thank you for posting this tutorial! I’ve been trying to use it as a baseline for something I’m working on and I am having trouble reproducing the original paper’s results with the example. In the supplemental section D of the Simsiam paper, they report their results on Cifar10 after training for what I presume is 100 epochs. By carefully looking at your code and the paper they seem to be the same besides a few hyperparameter changes and the data augmentation. I added a true random cropping function to the code you provided and have tried training for up to 800 epochs. Despite all this, I never get a linear evaluation higher than 35%. Any idea what else might be missing in the tutorial?


Problem with Conv2dtranspose for super resolution

Question:
Hi
When I doing the deconvolution for super-resolution, the Conv2dtranspose generates a black bar on the top (2xh in pixels) and left (2xw in pixels).
The photo: drive-google-com/file/d/12Ob2gVrd52j2mDbQ8_wMSUDkISmA37ZA/view?usp=sharing
The input before Conv2dtranspose layer is (600,720,1), and it should be 4x times bigger to (2400, 2880,1) after the Conv2dtranspose.
The layer is : out = Conv2DTranspose(num_channels, 9, strides=scale_factor, padding=‘same’)(x)
Does anyone familiar with this?
Thanks.

Context: Update, The layer is : out = Conv2DTranspose(1, 9, strides=4, padding=‘same’)(x)


Question:
The Movenet model is not (yet) available in ml5.js however Movenet can still be used directly.
When using a video stream on mobile the speed is very slow unless the input size is reduced. This works great in the demo but when reducing the size of the input video in p5.js only the reduced input region seems to be used, almost like the input video is cropped.
Looking at detector.js there is a initCropRegion to initially set the region, I wonder if this is using the wrong input size and therefore using a cropped region as an input?
Example here: p5.js Web Editor 43
(Try moving your nose past half way…)

Context:
It seems you are working with 2 diff sizes in your code. 1 is 1280x720 and the other is 640x480. In order to render the correct position to the larger canvas you will need to scale all resulting points.
Assuming input video was 640x480 from getUserMedia then you can take the co-ordinates that come back and convert them to percentage offsets instead. For example, if nose point  was at 320,240 then it can be represented as 0.5,0.5 which you then simply multiply by the new width/height eg 1280x720 to get a new enlarged co-ordinate of 640,360 - and then it should overlap correctly on your new video size.


How to efficiently write down and reuse a model (without doubly defining it, without calculating the same tensors multiple times)

Question:
I am porting some of my tf1 code to tf2 and am wondering how can I define a model for training and then reuse that same model at intermediary steps for predictions? Example:
x = tf.keras.Input(shape=(5,))
s = tf.keras.Sequential([...])(x)

# now do several things with s independently
a = tf.keras.Dense(...)(s)
b = tf.keras.Sequential(...)(s)

# do some more things with a and b and calculate the output for the model
output = tf.keras.Sequential([tf.keras.Dense(tf.concat([a,b])), ...])

model = tf.keras.Model(inputs=x, outputs=output)

Now, I want to be able to just calculate a and b (without calculating output) but also including the output.
I could write models for a, a and b and another for b where all the models get input x. But what if I want to calculate a,b and output. Now if I would do that using all the models I have previously defined, I would forward pass the graph multiple times, for each output that I need. I feel like I am missing something here.
Alternatively, I could consecutively define models that don’t have the original input as input, but instead I layer multiple input nodes within my code everytime I am working the next part of the graph like so
...
a = tf.keras.Dense(...)(s)
b = tf.keras.Sequential(...)(s)

model_a = tf.keras.Model(inputs=input, output=a)
model_b = tf.keras.Model(inputs=input, output=b)

input_a = tf.keras.Input(shape_of_a)
input_b = tf.keras.Input(shape_of_b)

output = tf.keras.Sequential([tf.keras.Dense(tf.concat([a,b])), ...])

model = tf.keras.Model(inputs=[input_a, input_b], outputs=output)

This is all so complicated and ugly… Now during training I have to first pass forward through each model on the way and so on.
It seems that I am in a bind. Either define multiple models using the same input, each time rewriting at least some of my code (which is in itself ugly) and accept the inefficiency during non-training when calculating a lot of nodes;
Or: Have tons of input nodes and complicated feed forward structures
Coming from tf1 where all of this was trivial, I feel like I am missunderstanding something very fundamental about how tf2 is supposed to work. Please feel free to answer generally what I am missing, my code was just for illustration.

Context:
You can modify the last line of your first code snippet to:
model = tf.keras.Model(inputs=x, outputs=[output, a, b])
This will allow you to access the intermediate tensors when calling the forward pass.
Internally this will not call your layers multiple times.
Please let me know if this was your intended point.


Question:
Hello everyone! I am trying to implement an inference function with tensorflow. I find that tf.keras.applications is particularly slow in my case.
This is my main file.
import argparse

import tensorflow as tf
import tensorflow.keras as keras

from util import load_data, Timer

def main(use_cuda):
    model_name = 'resnet152'

    with Timer("step 0: dummy input", use_cuda):
        dummy_input = tf.zeros([1])

    with Timer('step 1: load model', use_cuda):
        model = keras.applications.ResNet152(weights=None)
        model.load_weights(f'models/{model_name}.h5')

    with Timer('step 2: preprocess data', use_cuda):
        img_files, img_data = load_data(use_tf=True)
        data = img_data

    with Timer('step 3: predict', use_cuda):
        pred = model(data)
        result = tf.argmax(pred, axis=1).numpy()

    return dict(zip(img_files, result))

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument("--use_cuda", default="0", type=int)
    args = parser.parse_args()

    main(use_cuda=args.use_cuda)

This is the util file.
import os
import time
from contextlib import ContextDecorator

import numpy as np
from PIL import Image

def load_data(root='./data', use_tf=False):
    pass

class Timer(ContextDecorator):

    def __init__(self, description, cuda):
        self.description = description
        self.cuda = cuda

    def __enter__(self):
        self.start_time = time.perf_counter()

    def __exit__(self, *args):
        self.end_time = time.perf_counter()
        print(f'>>>>>>>>>> {self.description:25s}---{(self.end_time - self.start_time) * 1000:8.2f} ms <<<<<<<<<<')

This is the output. I cannot below it takes around 2 seconds in the load model step.
2021-07-07 18:29:31.453675: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
>>>>>>>>>> step 0: dummy input      ---    1.34 ms <<<<<<<<<<
>>>>>>>>>> step 1: load model       --- 1994.82 ms <<<<<<<<<<
>>>>>>>>>> step 2: preprocess data  ---   29.40 ms <<<<<<<<<<
>>>>>>>>>> step 3: predict          --- 2787.95 ms <<<<<<<<<<

Thanks!

Context:
Can you check if It Is running on GPU with:

      TensorFlow

Use a GPU  |  TensorFlow Core 3


Question:
Hi, I encountered an accuracy issue when computing the backprop of some layers through TensorFlow OPs.  The gradients was computed through two different ways:

tf.gradients
compute the gradients directly through TF APIs, take softmax as an example, we can compute the gradients as follows:

  sum_channels = math_ops.reduce_sum(grad_softmax * softmax, -1, keepdims=True)
  grad =  (grad_softmax - sum_channels) * softmax

But I found that the results from the two implementation are not exactly the same. Anyone knows what is the problem?
And another question is that when training in Tensorflow, whether the gradients are the same as that computed through tf.gradients? Thanks.
The entire testing code is as follows (tested with TF 1.15):
import numpy as np
import tensorflow as tf

batch_size = 20
num_heads = 8
from_seq_len = 50
to_seq_len = 50

class testSoftmaxBackprop:
    def __init__(self,
                 batch_size,
                 num_heads,
                 from_seq_len,
                 to_seq_len):
        self.batch_size = batch_size
        self.num_heads = num_heads
        self.from_seq_len = from_seq_len
        self.to_seq_len = to_seq_len

        self.input_data = tf.placeholder(tf.float32, shape=[
            self.num_heads * self.batch_size,
            self.from_seq_len,
            self.to_seq_len
        ])

        self.out  = tf.nn.softmax(self.input_data)

        # self.out = tf.identity(softmax)

    def forward(self, np_data):
        with tf.Session() as sess:
            sess.run(tf.global_variables_initializer())
            out = sess.run([self.out],
                           feed_dict={
                               self.input_data:np_data
                           })
        return out[0]

    def back_auto(self, np_grads, np_data, np_softmax):
        grads = tf.placeholder(tf.float32,
                               shape=[
                                   self.num_heads * self.batch_size,
                                   self.from_seq_len,
                                   self.to_seq_len
                               ])

        g = tf.gradients(self.out, [self.input_data], grad_ys=grads)

        with tf.Session() as sess:
            sess.run(tf.global_variables_initializer())
            g_out = sess.run(g,
                             feed_dict={
                                 self.input_data:np_data,
                                 grads:np_grads,
                             })

        return g_out

    def back_api(self, np_grads, np_data, np_softmax):
        tf_grads = tf.constant(np_grads, dtype=tf.float32)
        # tf_data = tf.constant(np_data, dtype=tf.float32)
        tf_softmax = tf.constant(np_softmax, dtype=tf.float32)

        sum_channels = tf.reduce_sum(tf_grads * tf_softmax, axis=-1, keepdims=True)
        d_out = (tf_grads - sum_channels) * tf_softmax # [h*N, T_q, T_k]

        with tf.Session() as sess:
            sess.run(tf.global_variables_initializer())
            grad = sess.run([d_out])
        return grad

def main():
    np.random.seed(0)
    np_data = np.random.rand(num_heads * batch_size, from_seq_len, to_seq_len)
    np_data = np_data.astype(np.float32)
    np_grad = np.random.rand(num_heads * batch_size, from_seq_len, to_seq_len)
    np_grad = np_grad.astype(np.float32)

    test_back = testSoftmaxBackprop(batch_size, num_heads, from_seq_len, to_seq_len)

    np_softmax = test_back.forward(np_data)

    grad_auto = test_back.back_auto(np_grad, np_data, np_softmax)
    grad_api = test_back.back_api(np_grad, np_data, np_softmax)

    api_data = grad_api[0]
    auto_data = grad_auto[0]

    api_save = api_data.reshape(-1)
    auto_save = auto_data.reshape(-1)
    np.savetxt("api_data.txt", api_save)
    np.savetxt("auto_data.txt", auto_save)

    print("Results:")
    print("Comparison :" + str(np.allclose(api_data, auto_data, atol = 5e-6)))
    print("max diff " + str(np.fabs(api_data - auto_data).max()))

if __name__ == "__main__":
    main()

Context: Anyone has an idea on this issue?


No model size reduction in Tflite model size with integer Quantisation

Question:
I have trained a PyTorch model, which I converted to Keras using the pytorch2keras lib.
using this Keras model to convert to tflite. I want to run the tflite on coral devices.
things which i noticed :

Keras model size (57.6MB)
Using dynamic range quantization the generated tflite is of size(15MB)
Using integer only quantization the generated tflite is also of size(15MB)

Ideally, we should be able to reduce the model size even further as we convert fp32 to int8
Can Anyone help me to understand why this happening?
sharing my conversion notebook and Keras modelkeras_model : data.zip - Google Drive 7

Context: @letdive_deep The link shared by you requires access.


Question:
I’m very new to coding so please explain in layman’s terms what to do about this error. Thank you!!
Traceback (most recent call last):
File “C:\Users\Maggie\Desktop\Manatees\Tensorflow Object Detection\tfod\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 64, in
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “Tensorflow\models\research\object_detection\builders\model_builder_tf2_test.py”, line 22, in
import tensorflow.compat.v1 as tf
File “C:\Users\Maggie\Desktop\Manatees\Tensorflow Object Detection\tfod\lib\site-packages\tensorflow_init_.py”, line 41, in
from tensorflow.python.tools import module_util as module_util
File "C:\Users\Maggie\Desktop\Manatees\Tensorflow Object Detection\tfod\lib\site-packages\tensorflow\python_init.py", line 39, in
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
File “C:\Users\Maggie\Desktop\Manatees\Tensorflow Object Detection\tfod\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 83, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File “C:\Users\Maggie\Desktop\Manatees\Tensorflow Object Detection\tfod\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 64, in
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.
Failed to load the native TensorFlow runtime.

Context: TensorFlow release binaries are prebuilt with AVX instruction sets. Therefore on any CPU that does not have these instruction sets, either CPU or GPU version of TF will fail to load. For other possible reasons you can check here 54.


Question:
GradientTape cannot compute the gredients for the model. How can I debug this code?
class Training(keras.Model):

  def __init__(self, model):

    super(Training, self).__init__()

    self.model = model

  def compute_loss(self, texts, labels):

    texts = tf.math.l2_normalize(texts, axis=0)
    losses = tf.Variable(tf.zeros_like(labels, dtype=tf.float32), trainable=True, dtype=tf.float32)

    for index, label in enumerate(labels):

      pos_pairs = texts[labels == label]

      neg_pairs = texts[labels != label]

      if len(pos_pairs) > 1:

        p_list = tf.Variable( tf.zeros(pos_pairs.shape[0], dtype=tf.float32), trainable=True, dtype=tf.float32 )

        i = 0

        for pos_pair in pos_pairs:

          p_list[i].assign( keras.losses.cosine_similarity(texts[index], pos_pair) )

          i += 1

        p_list = tf.exp( p_list )

        p_list = p_list / tf.reduce_sum(p_list)

        p_loss = tf.reduce_sum( - tf.math.log( p_list ) )

      else:

        p_loss = 0.0

      if len(neg_pairs) > 1:

        n_list = tf.Variable( tf.zeros(neg_pairs.shape[0], dtype=tf.float32), trainable=True, dtype=tf.float32 )

        i = 0

        for neg_pair in neg_pairs:

          n_list[i].assign( keras.losses.cosine_similarity(texts[index], neg_pair) )

          i += 1

        n_list = tf.exp( n_list )

        n_list = n_list / tf.reduce_sum(n_list)

        n_loss = tf.reduce_sum( tf.math.log( n_list ) )

      else:

        n_loss = 0.0

      loss_on_sentence = p_loss + n_loss

      losses[index].assign( loss_on_sentence )

    loss = tf.reduce_mean(losses)

    return loss

  def train_step(self, data):

    texts = data[0]

    labels = data[1]

    #print(labels, texts)

    with tf.GradientTape() as tape:

      texts = self.model(texts)

      loss = self.compute_loss(texts, labels)

      print(loss)

    trainable_vars = self.trainable_variables

    #print(trainable_vars)

    gradients = tape.gradient(loss, trainable_vars)

    print(gradients)

    self.optimizer.apply_gradients(zip(gradients, trainable_vars))

    loss_tracker.update_state(loss)

    return {"loss": loss_tracker.result()}

  @property

  def metrics(self):

    return [loss_tracker]

trainer = Training(model)

trainer.compile(optimizer='adam',  run_eagerly=True)

trainer.fit(train_dataset, callbacks=[tensorboard_callback])

Error:
ValueError: No gradients provided for any variable: ['dense1/kernel:0', 'dense1/bias:0', 'dense2/kernel:0', 'dense2/bias:0', 'bn1/gamma:0', 'bn1/beta:0', 'dense3/kernel:0', 'dense3/bias:0', 'dense4/kernel:0', 'dense4/bias:0', 'bn2/gamma:0', 'bn2/beta:0', 'dense5/kernel:0', 'dense5/bias:0', 'dense6/kernel:0', 'dense6/bias:0', 'bn3/gamma:0', 'bn3/beta:0', 'dense7/kernel:0', 'dense7/bias:0', 'dense8/kernel:0', 'dense8/bias:0', 'bn4/gamma:0', 'bn4/beta:0', 'dense9/kernel:0', 'dense9/bias:0', 'bn5/gamma:0', 'bn5/beta:0'].
Can anyone help to debug this error!!
Thanking you in advance.

Context: Which part breaks the gradient flow? This type of code works in pytorch but here it breaks flow somewhere and I cannot figure out which part cause the problem.


Question: For the tfdf RandomForestModel regression model, is there any way to generate R2 value, and SHAP value plot?

Context:
Hi Frank,
Keras does not have a native R2 score implementation (see the list of Keras’ regression metrics 2). However, the R2 score can be computed using a custom metric. See an example 3.
TF-DF does not have a native TreeSHAP implementation (yet? ). If this is of interest, please create a feature request here 6 so we can prioritize new features. In the mean time, I would use a model agnostic implementation of SHAP.
Cheers,
M.


Trying to understand Tensorflow Internals, would appreciate any insight on Tensor data

Question:
I have been spending a lot of time trying to understand the TensorFlow internals for a personal project by looking at source code and messing around in GDB. I know that Tensors are stored in the tensorflow::Tensor class which contains a shape, element count and a pointer to a tensorflow::TensorBuffer. TensorBuffer has a reference count for the actual data in the tensor and a pointer to the data. I have been able to find tensorflow::Tensors c++ data structures when messing around in GDB and am able to access their underlying buffers. However, I am struggling to find the data structures that manage the tensorflow::Tensors and how they are kept track of internally in a model.
For example, how do I associate a trainable variable in a model to its underlying in memory buffer?
When looking at how model.predict() works I know that the inputs to the execute function (TFE_Py_Execute) in quick_execute in tensorflow/python/eager/execute.py are passed in as tf.Tensor. When looking at the C++ code for TFE_Py_Execute in GDB the inputs are passed into that function as TFE_InputTensorHandles and an individual TFE_TensorHandle can be accessed with inputs->at(n) where n is any number below input size. Casting an individual TFE_TensorHandle to a tensorflow::TensorHandle in gdb lets me access the methods of the TensorHandle class for each TensorHandle.
However, there is no method that lets me pull out the data buffer for the TensorHandle and I seem to be missing some connection as when I go to access the data_ field of the TensorHandle which should contain a LocalTensorHandleData it is filled with info that I was not expecting. Where as a typical tensorflow::Tensor has a shape, element count, and pointer to a TensorBuffer which subsequently contains a pointer to a in memory buffer filled with tensor elements, the data_ for these TensorHandles is filled with a bunch of strings such as device string, localhost, a couple byte sections with 2c in them etc. I am not sure of what type this is and why this isnt just an in memory buffer.
So I would really like a clarification on what I am missing to more easily access the underlying tensorflow::Tensor data structure and its buffer as well as how I can maybe get from a model and its trainable variables to its underlying buffer.
Thanks!

Context:
I recently started implementing custom ops, and I agree TF C++ is a lot harder to grapple with that the Python side, and the current state of the docs does not help at all.
That said, grep or Search in VS Code is your ally: searching class TensorHandle in the TF repo, you can pull up tensorflow/cc/experimental/base/public/tensorhandle.h and see that the Tensor Resolve(Status* status) from which you could then get the buffer?


Question:
Hi, first question here, sorry if wrong format/category:
I’m writing a custom op (using the custom-op Docker images mentioned in the tutorial/repo) which is trying to allocate a temporary during the op but hits a segfault, which I have a hard time to understand.  The snippet where this happens is inside the ::Compute method of an OpKernel, where I’ve got
Tensor *q_t;
const TensorShape &sh = TensorShape({sdr->sht->nlm});
OP_REQUIRES_OK(context, context->allocate_temp(DT_COMPLEX128, sh, q_t));

And that allocation results in a segfault:
Thread 1 "spharde_ops_tes" received signal SIGSEGV, Segmentation fault.
0x00007fe3c216b973 in tensorflow::OpKernelContext::allocate_tensor(tensorflow::DataType, tensorflow::TensorShape const&, tensorflow::Tensor*, tensorflow::AllocatorAttributes, tensorflow::AllocationAttributes const&) ()
   from /usr/local/lib/python3.6/dist-packages/tensorflow/python/../libtensorflow_framework.so.2

with a backtrace like so
#0  0x00007fe3c216b973 in tensorflow::OpKernelContext::allocate_tensor(tensorflow::DataType, tensorflow::TensorShape const&, tensorflow::Tensor*, tensorflow::AllocatorAttributes, tensorflow::AllocationAttributes const&) ()
   from /usr/local/lib/python3.6/dist-packages/tensorflow/python/../libtensorflow_framework.so.2
#1  0x00007fe3c216e696 in tensorflow::OpKernelContext::allocate_temp(tensorflow::DataType, tensorflow::TensorShape const&, tensorflow::Tensor*, tensorflow::AllocatorAttributes, tensorflow::AllocationAttributes const&) ()
   from /usr/local/lib/python3.6/dist-packages/tensorflow/python/../libtensorflow_framework.so.2
#2  0x00007fe3a5895190 in tensorflow::OpKernelContext::allocate_temp (this=0x7ffe37e7d710, type=tensorflow::DT_COMPLEX128, shape=..., out_temp=0x5727690, allocator_attr=...)
    at /usr/local/lib/python3.6/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:1025
#3  0x00007fe3a5895217 in tensorflow::OpKernelContext::allocate_temp (this=0x7ffe37e7d710, type=tensorflow::DT_COMPLEX128, shape=..., out_temp=0x5727690) at /usr/local/lib/python3.6/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:1029
#4  0x00007fe3a5899ae2 in ApplyShtDiffOp<Eigen::ThreadPoolDevice>::Compute (this=0x574e380, context=0x7ffe37e7d710) at tf_spharde/cc/kernels/shtdiff_kernels.cc:101

I could understand an out-of-memory or whatever but why the segfault?  Any tips would be greatly appreciated.

Context:
I seem to have fixed it by using
Tensor q_t;
const TensorShape &sh = TensorShape({sdr->sht->nlm});
OP_REQUIRES_OK(context, context->allocate_temp(DT_COMPLEX128, sh, &q_t));

but it’s not clear to me why that works.  When running the op, I get an abort, double-free or corruption.
From reading use of allocate_temp in the TF codebase, this seems like it should work.


TypeError: only integer scalar arrays can be converted to a scalar index

Question:
Hi, I want to visualize the image in my data but I’m getting an error

image596×706 58.2 KB

image753×60 6.33 KB

image791×591 59 KB

Context:
It might have to do with the fact that labels is a list, not an array. Therefore you cannot use array slicing on it. Try to convert labels to an array first:
labels_array = np.array(labels)
Get back to me if it does or doesn’t work.


Massive performance impact with “run_eagerly=True”

Question:
Hi,
I work on a model based on this tutorial:

      keras.io

Keras documentation: Variational AutoEncoder 2

I want to make this vae an annealed beta-vae, so I have defined a “tf.keras.backend.variable” which is updated every epoch in a custom callback. This variable is then applied as a factor to the latent loss in the train_step function.

My first concern is that if I’m not forcing eager mode, the value of the variable is never updated in the train_step() function.

My second concern is that if I force eager mode in .compile() with run_eagerly=True, the value is now correctly updated in the train_step() function but the impact on runtime is HUGE : it’s twice the time for each epoch.

Do you have any idea of what is going on here ?
Thanks

Context: I’ve tested a lot of things, still no luck :-/ Nobody to give me an insight ?


Question:
OSError: SavedModel file does not exist at: C:\Users\Xiang\AppData\Local\Temp\tfhub_modules\a96501569a79510824b3ed96d5158fae078cd465{saved_model.pbtxt|saved_model.pb}
请教下这个怎么解决

bug1847×806 149 KB

Context:
Hi,
Do you have a small code snippet to reproduce the error?
It looks like the model you tried to use isn’t present in the disk.
if you could try hub.resolve(“URL_OF_THE_MODEL”) that would show where the model is cached and you can try to verify if it’s indeed there.


TensorFlow Decision Forest Regression Model Log

Question:
I have a question about the regression model in tutorials (model_7), Build, train and evaluate models with TensorFlow Decision Forests 4
I want to log the accuracy, but the output is blank. I have tried:
model_7.compile(metrics=[“accuracy”])
or
model_7.compile(metrics=[“mae”, “acc”])
After evaluation, the accuracy is vary low:
evaluation = model_7.evaluate(test_ds, return_dict=True)
print()
loss: 0.0000e+00 - mae: 0.9490 - acc: 3.4258e-04
The log has no accuracy data:
logs = model_7.make_inspector().training_logs()
Do I miss something?
Thank you!

Context:
The accuracy metric is used for classification (binary and multi-class) tasks. For regression, use one of the regression  metrics 6.
model_7.make_inspector().training_logs() should contains the RMSE metric.
model_7.compile(metrics=[“acc”]) will evaluation a regression output (the scale depends on the label) as a probability. In other words, you will get garbage :).


Question:
Hi again
Thanks to all of your help, I can build Faster-RCNN model.
But it goes well except training step, and I hit the wall.
I debugged functions, so found a suspicious part, however I can’t catch what is root cause.
First, the version is:

tensorflow-gpu==2.5.0
CUDA==11.2.0
cuDNN==8.1.0.77

The whole tack trace is:

2021-06-17 16:46:58.163220: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-06-17 16:47:04.565562: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll
2021-06-17 16:47:04.610494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1660 Ti computeCapability: 7.5
coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s
2021-06-17 16:47:04.618787: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-06-17 16:47:04.634109: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2021-06-17 16:47:04.638019: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2021-06-17 16:47:04.647980: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll
2021-06-17 16:47:04.655237: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll
2021-06-17 16:47:04.670023: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll
2021-06-17 16:47:04.679152: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll
2021-06-17 16:47:04.685911: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2021-06-17 16:47:04.690109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-06-17 16:47:04.693839: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-06-17 16:47:04.703798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1660 Ti computeCapability: 7.5
coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s
2021-06-17 16:47:04.711944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-06-17 16:47:05.263458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-17 16:47:05.268143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0
2021-06-17 16:47:05.270931: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N
2021-06-17 16:47:05.273788: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3983 MB memory) → physical GPU (device: 0, name: GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
D:\dev\anaconda3\envs\dl_env\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:3703: UserWarning: Even though the tf.config.experimental_run_functions_eagerly option is set, this option does not apply to tf.data functions. To force eager execution of tf.data functions, please use tf.data.experimental.enable.debug_mode().
warnings.warn(
WARNING:tensorflow:input_shape is undefined or non-square, or rows is not in [96, 128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
2021-06-17 16:47:06.528692: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2021-06-17 16:47:07.040698: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8100
2021-06-17 16:47:07.690211: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2021-06-17 16:47:08.195097: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
WARNING:tensorflow:From D:\dev\anaconda3\envs\dl_env\lib\site-packages\tensorflow\python\ops\array_ops.py:5043: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The validate_indices argument has no effect. Indices are always validated on CPU and never validated on GPU.
2021-06-17 16:47:09.868833: I tensorflow/core/profiler/lib/profiler_session.cc:126] Profiler session initializing.
2021-06-17 16:47:09.872503: I tensorflow/core/profiler/lib/profiler_session.cc:141] Profiler session started.
2021-06-17 16:47:09.875622: I tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1611] Profiler found 1 GPUs
2021-06-17 16:47:09.886018: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘cupti64_112.dll’; dlerror: cupti64_112.dll not found
2021-06-17 16:47:09.898400: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘cupti.dll’; dlerror: cupti.dll not found
2021-06-17 16:47:09.902934: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1661] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
2021-06-17 16:47:09.910278: I tensorflow/core/profiler/lib/profiler_session.cc:159] Profiler session tear down.
2021-06-17 16:47:09.914039: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1752] function cupti_interface_->Finalize()failed with error CUPTI could not be loaded or symbol could not be found.
2021-06-17 16:47:09.953551: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/100
214/214 [==============================] - 60s 275ms/step - loss: 7.0047 - rpn_reg_loss: 0.0160 - rpn_cls_loss: 0.1079 - frcnn_reg_loss: 6.5493 - frcnn_cls_loss: 0.3315 - val_loss: 5.5020 - val_rpn_reg_loss: 0.0161 - val_rpn_cls_loss: 0.1088 - val_frcnn_reg_loss: 5.2288 - val_frcnn_cls_loss: 0.1484

And the error message is:

2021-06-17 16:48:09.849872: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]

Suspicious code is:

@tf.function
def rpn_generator(dataset, anchors):
    while True:
        for data in dataset:
            image, gt_boxes, gt_labels = data
            bbox_deltas, bbox_labels = calculate_rpn_actual_outputs(anchors, gt_boxes, gt_labels)
            yield image, (bbox_deltas, bbox_labels)

Last, I referenced https://github.com/FurkanOM/tf-faster-rcnn 29
It must cause the same problem, because when I run that code, still get it.

Context:
I tried downgrading tensorflow to 2.4.0 and the error still occurred.
The strange thing is that when I run the code, the trained epoch has not consistency.
For example, at first run, train stopped at 2 epoch, and next run, train stopped at 13 epoch.
I thought the problem is my gpu(GTX 1660 Ti) memory, but running the code has taken about 55% of gpu memory.


Problem when save model with exporter_main_v2.py

Question:
I recive the warning
WARNING: skip full serialization of Keras
It happens to me with several downloaded and retrained models of the Tensorflow 2 model zoo

Context: Hello. Have you found a solution to this yet? I have been stuck on the same problem for a while now.


TensorFlow Object Detection API Installation with Specific TensorFlow Version

Question:
Hello,
Is it possible to install the TensorFlow Object Detection API with a specific version (TF2.3)? I have TensorFlow 2.3 installed and I need to be using the API with that specific version, but when I try to install it it automatically uninstalls TF2.3 and installs TF2.5.
Any advice?

Context:
You can clone repo with 2.3 tag.

      github.com

tensorflow/models 12

  v2.3.0/research/object_detection

  Models and examples built with TensorFlow. Contribute to tensorflow/models development by creating an account on GitHub.


Two questions about TF Certification Exam?

Question:
Hey there,
Preparing for the exam.

In the handbook it points out that the TF 2.4.1 is the condition, however, I have installed 2.5 on my machine.
GPU support is impossible on Mac according to the official doc. Has anyone who had the experience of the exam tried during the exam, was it heavily to train the exam models?

thank you 🤷🏼

Context: @Jocelyn_Becker @Laurence_Moroney might have some insights here


Does re-training an Object-Detection SSD Network give a different ‘format’ of model?

Question:
Hello,
I am trying to re-train an object detection model for a project. The end goal is for me to convert the ‘saved_model.pb’ model into another format through an SDK provided.
I have no problem at all converting the pre-trained object-detection SavedModels provided by TensorFlow on TensorFlow Hub. However, when I retrain from a checkpoint of one of the models, following every step in the TensorFlow Object Detection API training tutorial, eventually I start to get errors when I try to convert my re-trained result.
Does re-training cause anything significant in the format or components of the saved_model.pb to change? What can I do to simply re-train but not change anything important?
I would really appreciate any help with this, please.
Thanks,
Ahmad

Context:
Hi,
When you follow those tutorials, as far as I know, they don’t change the model structure, they will only change weights to be customized to your data.
what’s the error?


Saving the predicted images of bounding boxes - Tensorflow Object Detection API

Question:
How to save the images inside the bounding boxes !?, So that I can use the image for OCR.
I came over this tutorial 18, in the end I found a code sample for this,
output_directory = 'some dir'

# get label and coordinates of detected objects
output = []
for index, score in enumerate(output_dict['detection_scores']):
    label = category_index[output_dict['detection_classes'][index]]['name']
    ymin, xmin, ymax, xmax = output_dict['detection_boxes'][index]
    output.append((label, int(xmin * image_width), int(ymin * image_height), int(xmax * image_width), int(ymax * image_height)))

# Save images and labels
for l, x_min, y_min, x_max, y_max in output:
    array = cv2.cvtColor(np.array(image_show), cv2.COLOR_RGB2BGR)
    image = Image.fromarray(array)
    cropped_img = image.crop((x_min, y_min, x_max, y_max))
    file_path = output_directory+'/images/'+str(len(df))+'.jpg'
    cropped_img.save(file_path, "JPEG", icc_profile=cropped_img.info.get('icc_profile'))
    df.loc[len(df)] = [datetime.datetime.now(), file_path]
    df.to_csv(output_directory+'/results.csv', index=None

But the above code is unclear to me, I am new to computer vision and Tensorflow Object Detection API, here is my query,
How can I leverage the above code to save multiple images(In my case, I am doing invoice extraction, where I have to save the invoice number, date, table, etc…)

Context: The code seems to be saving multiple images based on the detections. The output should be on the images folder.


Question:
Hi i will be honest i dont have a clue in coding(yet) nor ML subject
And im wondering if someone heard or get his/her knowledge by the next sites which offer some data(which im not sure how much good it is)
Datacamp(with their paid version, for now im kind of doing their free trial course)
Pluralsight
Codeacademy
Udacity(which is more expensive then those 3 above,but the knowledge is important enough to me to think of it as an option)
I want to hear some “PRO” thoughts on what to do next i mean i can start with something but i really prefer not do waste my time on something bad so…
THANKS

Context:
Hi Yovel,
There’s a lot of great free content for you to start before you need/want to pay for a course.
I’d start with those.
Something like Machine Learning Crash Course  |  Google Developers 7
There’s also this thread:

How to audit courses on Coursera for Free 6
Any suggestions to start with TensorFlow? - #2 by 8bitmp3 6

I hope this helps!


Do you have any models in production, or do you plan to have any?

Question: Do you currently have any models in production, being used for a product, service, or something else important?  Do you ever plan to have any?  If so, how do you create and manage your production deployment?  How do you train your models?  How do you serve your models?

Context: Sir I am planning to build a smart bot that talks and guides people towards a more healthy mental physique, I being an engineer start with the very first tools, pen and paper plan out the model, its requirements (software and hardware) and then do the coding part, I plan to soon make it on production. TF helps to a great deal in doing complex coding in just few lines.


UpdatePruningStep() throwing an error ” ValueError: Error processing property ‘_dropout_mask_cache’ of <ContextValueCache at 0x1c2604d02e0>”

Question:
I am trying to prune my pre-trained model and for that It is mandatory to use UpdatePruningStep() in the callbacks while fitting the model. When I do so, I am getting the error as follows -
ValueError: Error processing property ‘_dropout_mask_cache’ of <ContextValueCache at 0x1c2604d02e0>
code for pruning is as follows -
from tensorflow_model_optimization.sparsity import keras as sparsity
num_train_samples=X_train.shape[0]
batch_size=128
epochs = 4
end_step = np.ceil(1*num_train_samples/batch_size).astype(np.int32)*epochs
print(end_step)
new_pruning_params = {
‘pruning_schedule’ : sparsity.PolynomialDecay(initial_sparsity=0.50,
final_sparsity=0.90,
begin_step = 0,
end_step = end_step,
frequency=100)
}
new_pruned_model = sparsity.prune_low_magnitude(loaded_model, ** new_pruning_params)
new_pruned_model.summary()
new_pruned_model.compile(loss = tf.keras.losses.categorical_crossentropy,
optimizer=‘adam’,
metrics=[‘accuracy’])
logdir = tempfile.mkdtemp()
callbacks = [sparsity.UpdatePruningStep(), sparsity.PruningSummaries(log_dir=logdir)]

keras.callbacks.EarlyStopping(monitor=‘val_loss’, patience=10)]
new_pruned_model.fit(X_train, Y_train.values, batch_size=batch_size, epochs=epochs,verbose=1,
validation_data = (X_test,Y_test.values), callbacks=callbacks)
score= new_pruned_model.evaluate(X_test,Y_test.values, verbose=0)
print('Test Loss : ’ , score[0])
print('Test Accuraccy: ', score[1])

Context:
This seems similar to Error processing property '_dropout_mask_cache' when using PrunableLayer with DropoutRNNCellMixin · Issue #753 · tensorflow/model-optimization · GitHub 22
Does the fix there work for you?


Question:
I have one question about markdown in git, and how it rendered in website.
for example,

      github.com

    tensorflow/tensorflow/blob/master/tensorflow/compiler/xla/g3doc/operation_semantics.md#allgather 1

    # Operation Semantics

The following describes the semantics of operations defined in the
[`XlaBuilder`](https://www.tensorflow.org/code/tensorflow/compiler/xla/client/xla_builder.h)
interface. Typically, these operations map one-to-one to operations defined in
the RPC interface in
[`xla_data.proto`](https://www.tensorflow.org/code/tensorflow/compiler/xla/xla_data.proto).

A note on nomenclature: the generalized data type XLA deals with is an
N-dimensional array holding elements of some uniform type (such as 32-bit
float). Throughout the documentation, *array* is used to denote an
arbitrary-dimensional array. For convenience, special cases have more specific
and familiar names; for example a *vector* is a 1-dimensional array and a
*matrix* is a 2-dimensional array.

## AfterAll

See also
[`XlaBuilder::AfterAll`](https://www.tensorflow.org/code/tensorflow/compiler/xla/client/xla_builder.h).

  This file has been truncated. show original

source code is

Arguments
Type
Semantics

operand
XlaOp
Array to concatenate across

:                  :                      : replicas.                   :
| all_gather_dim | int64              | Concatenation dimension.    |
| replica_groups | vector of vectors of | Groups between which the    |
:                  : int64              : concatenation is performed. :
| channel_id     | optional int64     | Optional channel ID for     |
:                  :                      : cross-module communication. :
rendered in github is wrong

image772×480 23.7 KB

but rendered in website is right.
I read Contribute to the TensorFlow documentation 1,
but find nothing about the format.
Can anyone help to solve my confusion?
how to follow markdown format in table?
is there any format tool?
any suggestions is welcome,
thanks in advance.

Context:
Hello. Tables aren’t part of the original Markdown spec so different systems implemented their own syntax. For the TensorFlow docs, we optimize viewing for the webpage and use GitHub as a preview for convenience—but there are some Markdown discrepancies. Some pages can also be viewed in Colab.
Since the webpage 3 renders correctly, please use that Markdown syntax for the table on that page. And, depending on the data and format, might just prefer a regular ole’ HTML table that renders everywhere. We use those, too.


Question:
I have a sequential keras model using dense and lstm layers. After training the model, I saved in .h5 format. I am trying to convert this model to a tensorflow lite model with 8-bit integer quantization to run it on the Coral Dev board. I can perform the conversion to a lite model just fine, but when i try to quantize i get the “ValueError: Failed to parse the model: Only models with a single subgraph are supported, model had 3 subgraphs.”.
System Information:
Ryzen 5 3600
AMD 5700xt
Tensorflow version: TF nightly
Model design:
self.model = tf.keras.Sequential([
InputLayer(input_shape=(WINDOW_SIZE // WINDOW_STEP, 1), name=‘input’),
Dense(DENSE_LAYERS, activation=‘relu’),
LSTM(LSTM_LAYERS),
Dense(len(CLASSIFICATION.keys()), activation=‘softmax’, name=‘output’)
])
self.model.compile(optimizer=‘adam’,
loss=‘categorical_crossentropy’,
metrics=METRICS)
To reproduce error:
Clone GitHub - jboothby/LSTM_Error_Report 2 and run convert_to_lite.py
I used the example code from: Post-training integer quantization  |  TensorFlow Lite 1 for integer-only quantization . My representative data is include in the .csv file in the repository.
The error seems to be coming from the representative dataset line. If i change the current code
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = tf.lite.RepresentativeDataset(representative_data_gen)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()

to
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
Then it executes fine, but doesn’t do full integer quantization.
This is my first time posting a help question on this forum, so please let me know what else I can add to clarify.

Context:
This really seems:

      github.com/tensorflow/tensorflow

      Failed to convert weights to 8 bit precision: "Quantize weights tool only supports tflite models with one subgraph"  21

        opened

        Dec 18, 2019

          closed

        May 19, 2021

          rutrilla

        TF 2.1
        comp:lite
        stalled
        stat:awaiting response
        type:feature

    ### System information
- **Have I written custom code (as opposed to using a st…ock example script provided in TensorFlow)**: Yes
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Google Colab (GPU)
- **TensorFlow installed from (source or binary)**: Binary
- **TensorFlow version (use command below)**: 2.1.0-dev20191217
- **Python version**: 3
- **Exact command to reproduce**:

```bash
!pip install tf-nightly
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import GRU, Dense, Dropout

model = Sequential()
model.add(GRU(100, activation='relu', return_sequences=False, input_shape=(128,2)))
model.add(Dropout(0.2))
model.add(Dense(11, activation='softmax'))

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.experimental_new_converter = True
tflite_model = converter.convert()
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_model_quant = converter.convert()
```

### Error message
```bash
E tensorflow/lite/tools/optimize/quantize_weights.cc:351] Quantize weights tool only supports tflite models with one subgraph.
```

### Describe the problem
First, I used the new converter (with the experimental flag converter.experimental_new_converter = True) to convert an RNN model from TensorFlow to TensorFlow Lite's flat buffer format, as suggested in the issue #32608.

This works correctly, but then when I try to perform a post-training weight quantization, I got an error saying that the quantize weights tool only supports tflite models with one subgraph.

Is there a problem with my procedure? Or is that feature not yet supported? In that case, I would like to request this feature.

Thanks in advance for your help.

### Source code / logs
The attached file can be used to reproduce the error with a trained model (.h5).
[GRU_1L.zip](https://github.com/tensorflow/tensorflow/files/3974453/GRU_1L.zip)


Image transformations require Scipy but Scipy already installed

Question:
Hello, I’m running Tensorflow 2.4.1 on a Windows 10 computer (Python 3.8.5 with Anaconda). I’m using a NVIDIA GeForce RTX 2060 GPU with TF. I’m having problems trying to launch a CNN model using the ImageDataGenerator feeding from a Pandas dataframe. After defining the data generator and the model, I get the following error when running model.fit: “Image transformations require SciPy. Install SciPy.” However, when I can confirm that Scipy is installed by running “import scipy” without errors.
The definition of the image generator is as follows:
train_datagen = ImageDataGenerator(horizontal_flip=False,
                               vertical_flip=False,
                               rescale=1/255.0).flow_from_dataframe(dataframe=X,
                                                                    x_col='image_name',
                                                                    y_col='response',
                                                                    shuffle=False,
                                                                    directory=src_path,
                                                                    target_size=(128, 128),
                                                                    class_mode=None
                                                                   )

And error comes when running:
model.fit(train_datagen,
epochs=5)
Please help!

Context:
PS. I believe the error results from my GPU configuration. I am running a CPU version of the code and training is running (although extremely slow). I had to change my imagegenerator declaration to:
class_mode='raw',
as I am running a regression model.