Question:
In the beginning, I made a CNN model train to predict cats vs dogs.
it was unstable during training however I get 2000 photos for training and another 1000 validation data set so that thought the problem with my model and I load a pre-trained model resnet50 and made a fully connected layer with a softmax of 2 outputs.
and it is still unstable.
I changed the batch_size, epochs, and steps_per_epochs much time and it is still unstable.
what should I do and what is my fault
#Data base
from os.path import join as p
from os import getcwd as g
train = p(g(), 'train')
validation = p(g(), 'validation')
train_cat = p(train, 'cat')
train_dog = p(train, 'dog')
validation_cat = p(validation, 'cat')
validation_dog = p(validation, 'dog')

from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.models import load_model, Sequential, Model
from tensorflow.keras.layers import Flatten, Dense, Dropout, Conv2D, MaxPooling2D, Input, GlobalMaxPooling2D
from tensorflow.keras.preprocessing.image import ImageDataGenerator as IDG
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint

i = Input(shape=(224, 224, 3))
x = ResNet50(weights='imagenet', include_top=False)(i)
x = GlobalMaxPooling2D()(x)
x = Flatten()(x)
x = Dense(1024, activation='relu')(x)
x = Dense(512, activation='relu')(x)
x = Dense(2, activation='softmax')(x)
model = Model(inputs=i, outputs=x)

model.get_layer('resnet50').trainable = False
model.summary()

model.compile(optimizer='adam',loss='categorical_crossentropy', metrics=['accuracy'])

stop = EarlyStopping(monitor='val_loss', patience=100)
saving = ModelCheckpoint('resnet50.h5', save_weights_only=False, save_best_only=True)

#adjust dataset before training
train_datagen = IDG( rescale = 1.0/255.,
                     rotation_range=40,
                     width_shift_range=0.2,
                     height_shift_range=0.2,
                     shear_range=0.2,
                     zoom_range=0.2,
                     horizontal_flip=True,
                     fill_mode='nearest')
test_datagen  = IDG( rescale = 1.0/255.)
train_generator = train_datagen.flow_from_directory(train,
                                                    batch_size=1,
                                                    class_mode='categorical',
                                                    target_size=(224, 224))
validation_ganerator = test_datagen.flow_from_directory(validation,
                                                        batch_size=1,
                                                        class_mode='categorical',
                                                        target_size=(224, 224))
#Model training
history = model.fit(train_generator,
                    validation_data=validation_ganerator,
                    steps_per_epoch=1,
                    epochs=100,
                    validation_steps=50,
                    verbose=1,
                    callbacks=[stop, saving])

#show results
import numpy as np
import matplotlib.pyplot as plt
acc      = history.history['accuracy']
val_acc  = history.history['val_accuracy']
loss     = history.history['loss']
val_loss = history.history['val_loss']

epochs   = range(len(acc)) # Get number of epochs
plt.plot  ( epochs,     acc )
plt.plot  ( epochs, val_acc )
plt.title ('Training and validation accuracy')
plt.figure()
plt.plot  ( epochs,     loss )
plt.plot  ( epochs, val_loss )
plt.title ('Training and validation loss')
plt.show()

Context: Why are you using a batch size of 1 and steps_per_epoch=1? It means that every epoch you train your model on exactly one train image and evaluate it on 50 validation images (since validation_steps=50).


Question:
I wrote this code to use expand dms
val test123: Constant[TFloat64] = tf.constant(array_data)
val test124: Constant[TFloat64]= tf.constant(0)
val final_test = tf.expandDims(test123, test124)

It gives me the following errror :
 thread "main" org.tensorflow.exceptions.TFInvalidArgumentException: Value for attr 'Tdim' of double is not in the list of allowed values: int32, int64
	; NodeDef: {{node ExpandDims}}; Op<name=ExpandDims; signature=input:T, dim:Tdim -> output:T; attr=T:type; attr=Tdim:type,default=DT_INT32,allowed=[DT_INT32, DT_INT64]>
	at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:87)
	at org.tensorflow.EagerOperationBuilder.execute(EagerOperationBuilder.java:273)
	at org.tensorflow.EagerOperationBuilder.build(EagerOperationBuilder.java:67)
	at org.tensorflow.EagerOperationBuilder.build(EagerOperationBuilder.java:56)
	at org.tensorflow.op.core.ExpandDims.create(ExpandDims.java:84)
	at org.tensorflow.op.Ops.expandDims(Ops.java:2452)
	at HelloTensorFlow$.main(sdasdasdas.scala:211)
	at HelloTensorFlow.main(sdasdasdas.scala)

Context: The second argument to expandDims should be a 32-bit or 64-bit integer tensor, not a 64-bit floating point tensor. Looks like we should tighten the type requirements to TIntegral from TNumber for the second argument.


Question:
Hi,
Would like to use GradientTape in c++ but cannot find any code sample that demonstrates how to do so.

In python:
with GradientTape as tape:
tape.watch(x)
y = x * x

In C++: ?

Does anyone have some sample to share?
Best,
Dom

Context:
You can find something at:

      github.com

    tensorflow/tensorflow/blob/master/tensorflow/c/eager/gradients_test.cc 16

    /* Copyright 2020 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include "tensorflow/c/eager/gradients.h"

#include <memory>

#include "absl/container/flat_hash_set.h"
#include "absl/types/span.h"

  This file has been truncated. show original


Question: Greetings, I’m new to the community. I’m new to artificial intelligence. Do you have any projects you can recommend? There are also datasets. Thanks…

Context: hi, You may want to look for projects in Kaggle https://www.kaggle.com/ 4,  They have projects for all levels


Bert/Bert Based Pre-processing with TFLite issues

Question:
Hi there everyone, Pre-processing with BERT based models has been the biggest cause of pain for me while working on my project (Android), currently I am using sentencepiece’s C++ library (I was already using NDK for other things) and it works as I expected, kinda ! But I would like to figure out if there is a way to integrate it with the tflite model itself because it’s a lot of code removed if that could be done.
This is one of the Notebooks 2 I was experimenting on before doing it the C++ way, it takes less than 5 minutes to run the entire notebook (pretty small) and make the tflite embedding model and to reach the error when you run the tflite model.

interpreter.invoke()
RuntimeError                              Traceback (most recent call last)
 in ()
----> 1 interpreter.invoke()
/usr/local/lib/python3.7/dist-packages/tensorflow/lite/python/interpreter.py in invoke(self)
873     “”"
874     self._ensure_safe()
 → 875     self._interpreter.Invoke()
876
877   def reset_all_variables(self):
RuntimeError: Table not initialized.
(while executing ‘WordpieceTokenizeWithOffsets’ via Eager)Node number 2167 (TfLiteFlexDelegate) failed to invoke.

I can assume that there is some kind of table being used in pre-processing model which isn’t being initialized in the tflite version for the model when loaded into interpreter or during allocation. But I have no idea how to go about debugging it.

Context:
I have encountered similar issues with converting bert to tflite in the past running on tf v2.5.
Firstly, the vocab.txt was not able to initialise the hashtable in tflite. That causes your runtime error of table not initialized. I’m not sure with the latest tf v2.6, using converter.experimental_enable_resource_variables = True during tflite conversion will improve the situation now?
Secondly,  if you are using preprocessor = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3") or similar tokenizer helper layers that depends on tensorflow-text, you will have difficulties compiling mobile tflite binaries that support tensorflow-text ops as flex delegate ops. See Using tensorflow_text with tflite. · Issue #287 · tensorflow/text · GitHub 2
I ended up having to rewrite the vocab.txt to load into tflite friendly tensors to by-pass the initialization issue. Also, rewriting parts of bert tokenizers 3  on mobile or rewriting tensorflow-text ops to tflite friendly ops. It can work, but you will likely end up with spaghetti codes, with some parts either lives on tflite models, or android. If you are already using c++ ndk sentence piece, it might be much neater to remain that way until the above issues are fixed.


Facing issue while initializing parameters in tensorflow

Question: 41051×551 53.6 KB

Context:
hi, you can try something like this (low level):
def weights(name, shape, mean=0.0, stddev=0.02):
    var = tf.Variable(tf.random_normal_initializer(mean=mean, stddev=stddev)(shape), name=name)
    return var

values = weights(name='w1', shape=[25, 1])

or following the tensorflow docs (keras):
def weights(name, shape):
    initializer = tf.keras.initializers.GlorotNormal()
    val = initializer(shape=shape)
    return tf.Variable(val, name=name)

values = weights(name='w1', shape=[25, 1])


Strassen’s Matrix Multiplication on TensorFlow

Question:
I am trying to do Strassen’s matrix multiplication on TensorFlow. I am trying to perform a graph execution using the following code
import tensorflow as tf

@tf.function  # The decorator converts `split_tf` into a `Function`.
def split_tf(matrix):
  shape = tf.shape(matrix)
  shape2 = tf.cast(tf.divide(shape, 2), dtype=tf.int32)
  return tf.slice(A, [0, 0], shape2), tf.slice(A, [0, shape2[1]], shape2),
         tf.slice(A, [shape2[0], 0], shape2),
         tf.slice(A, [shape2[0], shape2[1]], shape2)

@tf.function  # The decorator converts `strassen_tf` into a `Function`.
def strassen_tf(x, y):
  # Base case when size of matrices is 1x1
  if tf.rank(x) == 1:
      return tf.math.multiply(x, y)

  # Splitting the matrices into quadrants. This will be done recursively
  # until the base case is reached.

  a, b, c, d = split_tf(x)
  e, f, g, h = split_tf(y)

  # Computing the 7 products, recursively (p1, p2...p7)
  p1 = strassen_tf(a, tf.math.subtract(f, h))
  p2 = strassen_tf(tf.math.add(a, b), h)
  p3 = strassen_tf(tf.math.add(c, d), e)
  p4 = strassen_tf(d, tf.math.subtract(g, e))
  p5 = strassen_tf(tf.math.add(a, d), tf.math.add(e, h))
  p6 = strassen_tf(tf.math.subtract(b, d), tf.math.add(g, h))
  p7 = strassen_tf(tf.math.subtract(a, c), tf.math.add(e, f))

  # Computing the values of the 4 quadrants of the final matrix c
  c11 = tf.math.add(tf.math.subtract(tf.math.add(p5, p4), p2), p6)
  c12 = tf.math.add(p1, p2)
  c21 = tf.math.add(p3, p4)
  c22 = tf.math.subtract(tf.math.subtract(tf.math.add(p1, p5), p3), p7)

  # Combining the 4 quadrants into a single matrix by stacking horizontally and vertically.
  c = tf.concat([tf.concat([C11, C12], axis=1), tf.concat([C21, C22], axis=1)], axis=0)

  return c

Now when I use the above code to multiply two tensors as below:
n = 2;
A = tf.random.normal([n, n], 0, 1, dtype=tf.float64)
B = tf.random.normal([n, n], 0, 1, dtype=tf.float64)
D = strassen_tf(A, B)

the code stuck at the last statement.
Thanks in advance!

Context:
First, I would 100% advise against using recursion in a tf.function. I don’t think tf.function will play well with recursion. tf.function needs to know the entire graph at compile-time and so it will (I think) trace through the recursion until it has a complete graph. With large matrices this will result in REALLY big graphs that are difficult to compile. You should opt for tf control-flow instead e.g. tf.scan, tf.while_loop, etc.
Second, if you add some debug prints and inspect the rank, you’ll see the rank always remains 2. Notice your comment for your base case says the base case is a 1 x 1 matrix, which tf.rank will always evaluate to 2. I think instead what you’re looking for is tf.size(x) == 1 OR tf.linalg.matrix_rank(x) == 1. tf.rank returns the number of dimensions in the Tensor,  whereas tf.linalg.matrix_rank returns the rank in the sense you are trying to use it. Changing that and commenting out the tf.function decorators evaluates the function correctly. If you want to use tf.function, try to rework the recursion to use native TensorFlow control-flow.


Tensorflow BERT pooled_output vs sequence_output

Question:
I would like to understand difference between pooled_output, sequence_output and encoder_output in

encoder = hub.KerasLayer(“TensorFlow Hub 1”)
text_embeddings = encoder(text_preprocessed)
text_embeddings.keys() # this has pooled_output, sequence_output etc as keys

My understanding is that pooled_output is an embedding for entire sentence where sequence_output is contenxtualized embdeding of individual tokens in a sentence Going by that shouldn’t the pooled_output embdeding vector and sequence_output embdeding vector for first token be the same? I looked at them in my notebook and they are different. I couldn’t find much documentation online so it will be great if someone can provide detailed understanding of these three keys in the output object above.

Context: Do you the answer for this @markdaoust ?


How to continue training a pre-trained model?

Question:
I’m currently using VGG16 pretrained model on my set of images
model = VGG16()
model = Model(inputs = model.inputs, outputs = model.layers[-2].output)
and then for each image I get its features model.predict(img, use_multiprocessing=True)
but I also would like to take some subset of my images to make a little further training of VGG16 model on that subset (so the final model would be trained on VGG16’s images + on my images)
how can I implement that?

Context:
It is very similar to the fine-tuning described  in:

      keras.io

Keras documentation: Keras Applications 8


Detection results are worse for smaller images than those specified in pipeline.config

Question:
Hi i have a question,
I use an SSD Mobilenet V2 network for detection. I have found that the result is worse if my input image has a smaller image size than what is specified in the pipeline.config. For bigger pictures, that would make sense to me. Is there a logical explanation why I get a poorer quality with smaller pictures? A resolution of 768x320 is specified in the pipeline.config and I tested it with images with 738x119.

Context:
Have you verified that the performance worse for small objects?
check:

      stackoverflow.com

  How to modify ssd mobilenet config to detect small objects using tensorflow object detection API? 10

  tensorflow, object, detection, mobilenet

  asked by

    stujo

  on 03:25AM - 05 Mar 20 UTC


TextVectorization significantly slower than sklearn’s CountVectorizer

Question:
Hello,
We are trying to replace sklearn CountVectorizer 8 with TextVectorization 8. We experimented with a text dataset of 400K sentences with unigrams and bigrams. We used adapt to build the vocabulary.
We wrote a small Keras model as shown in the documentation with a vectorization layer and compiled the model.
The predict is significantly slower (by 500 times on a desktop, no GPUs) than sklearn’s transfrom function. Any suggestions?
Thank you!

Context:
I would guess this is an issue with output representation. By default, TextVectorization will use a dense Tensor output–this is simpler for small examples, and guarantees the output of the layer can be used with any other keras layer. But when using output_mode='count' (or 'multi_hot' or 'tf_idf') and a large vocabulary (vocab is probably quite large when using bigrams), this dense output gets very inefficient very quickly.
We recently added an option for sparse output 6 from the TextVectorization layer. This is currently on only in tf-nightly, and will be available in a stable release in tensorflow 2.7.
tf.keras.layers.TextVectorization(output_mode='count', sparse=True)
The sparse output from the layer constructed like this could then be fed into a tf.keras.layers.Dense, and scale up much more effectively.


Question:
I am playing with a coral board (TF 1.X).
I would like to do a transfer learning able to distinguish small metal parts by size and orientation with great reliability.
Which network do you recommend starting from?

Context:
Take a look at:

      Coral

Models - Semantic segmentation | Coral 2

  Models that identify specific pixels belonging to different objects.


Why tflite api is different from regular tensorflow api

Question:
To perform inference on tflite model one has to write this code,

import numpy as np
import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path=“converted_model.tflite”)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0][‘shape’]
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0][‘index’], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0][‘index’])
print(output_data)

To run inference on regular tensorflow model, you will just do,

tfmodel.predict(test_dataset)

My question is why tflite api is not made simple and consistent with regular tensorflow model api? If they were same it would be very easy to remember and work with.
Also I was not able to find a function to evaluate tflite performance on test dataset, i.e.

tflite_model.evaluate(test_ds)

Am I missing something or there exist this method may be with a different name?

Context: @markdaoust Do you have any historical background to share on why we don’t have a unified high level API?


What’s the recommended way to load Tensorflow 2 saved models

Question:
I have a general question regarding the Tensorflow 2 API when it comes to use saved models.
The guideline on this post explains how to save and load TF models

      TensorFlow

Save and load models  |  TensorFlow Core 2

I am wondering why in the post the saved model gets loaded like this:
model = tf.keras.models.load_model('saved_model_folder')

Is this way of loading the model equal to
model = tf.saved_model.load('saved_model_folder')

I am asking because I read that the  Keras H5 format is the old format and it is recommended to use the saved model format.
See here: Save and load Keras models  |  TensorFlow Core
When I save my model like this
model.save(saved_model_folder)

I get the follwoing files

It looks like I get both the Keras H5  and the saved model file
I would perfer to use  “new_model = tf.keras.models.load_model(‘saved_model_folder’)” because it gives me a better handling of the model. I can directly call “model.predict()” which is not possible when I load the model via “model = tf.saved_model.load(‘saved_model_folder’)”.

Context:
You can see:

      TensorFlow

tf.saved_model.load  |  TensorFlow Core v2.6.0 1

  Load a SavedModel from export_dir.

Loading Keras models
Keras models are trackable, so they can be saved to SavedModel. The object returned by tf.saved_model.load 1 is not a Keras object (i.e. doesn’t have .fit, .predict, etc. methods). A few attributes and functions are still available: .variables, .trainable_variables and .__call__.
Use tf.keras.models.load_model to restore the Keras model.


Question:
I’m in the process of fine-tuning EfficientDetLite0 with information from this tutorial:

      TensorFlow

Object Detection with TensorFlow Lite Model Maker 7

I’m using object_detector.DataLoader.from_csv() to load my data which uses PNG image files.
The from_csv api points to this documentation for the csv format which mention that PNG files are allowed:

      Google Cloud

Formatting a training data CSV  |  AutoML Vision Object Detection 2

References to non-image files (such as PDF or PSD files). Likewise, files that are not image files (JPEG, PNG, GIF, BMP, or ICO) but that have been renamed with an image extension will cause an error.

Even the example is using PNG files.
However on my side I get an error:
Traceback (most recent call last):
  File "rgb_finetune.py", line 125, in <module>
    csv_file_path, images_dir=args.images)
  File "/azureml-envs/tensorflow/lib/python3.7/site-packages/tensorflow_examples/lite/model_maker/core/data_util/object_detector_dataloader.py", line 292, in from_csv
    cache_writer.write_files(cache_files, csv_lines=csv_lines)
  File "/azureml-envs/tensorflow/lib/python3.7/site-packages/tensorflow_examples/lite/model_maker/core/data_util/object_detector_dataloader_util.py", line 258, in write_files
    ann_json_dict=ann_json_dict)
  File "/azureml-envs/tensorflow/lib/python3.7/site-packages/tensorflow_examples/lite/model_maker/third_party/efficientdet/dataset/create_pascal_tfrecord.py", line 134, in dict_to_tf_example
    raise ValueError('Image format not JPEG')
ValueError: Image format not JPEG

Any idea why I get this error?

Context: @Yuqi_Li might be able to help here


Question:
Hi,
Does someone  know what type of algorithm, model maker use for training object detection model?

Context: Do you mean what optimizer or what model arch?


How to create constants from list of doubles jvm

Question: I have a list of doubles in scala and I want to convert it to constant for further processing , how would I go about that

Context:
By list, you mean a java.util.List? If so, you need to convert your list to an array of primitive doubles then pass it to the Constant op (or TFloat64 if you want to create a tensor that feeds a model). There are many ways of doing this. e.g. in Java,
Ops tf = Ops.create(...);
double[] array = list.stream().mapToDouble(x -> x).toArray();
Constant<TFloat64> constant = tf.constant(array);
try (TFloat64 tensor = TFloat64.vectorOf(array)) { ... }

Whenever possible, you should try to work directly with primitive types rather than their boxed type though (i.e. double vs Double), performance gains can be substantial.


NotFoundError for import tensorflow_decision_forests as tfdf

Question:
My code was working on Colab, but today it got following error.
My tfdf installation is successfully:
!pip install tensorflow_decision_forests
It is the import statement got error:
import tensorflow_decision_forests as tfdf
here is the error message:
NotFoundError                             Traceback (most recent call last)
 in ()
6 import matplotlib.pyplot as plt
7 import tensorflow as tf
----> 8 import tensorflow_decision_forests as tfdf
9 from keras import backend as K
10 from sklearn.model_selection import train_test_split

5 frames

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)
56     RuntimeError: when unable to load the library or get the python wrappers.
57   “”"
—> 58   lib_handle = py_tf.TF_LoadLibrary(library_filename)
59   try:
60     wrappers = _pywrap_python_op_gen.GetPythonWrappers(
NotFoundError: /usr/local/lib/python3.7/dist-packages/tensorflow_decision_forests/tensorflow/ops/training/training.so: undefined symbol: _ZN10tensorflow11GetNodeAttrERKNS_9AttrSliceEN4absl14lts_2020_09_2311string_viewEPSs
Any idea?

Context:
We had an issue with TF 2.6 and there’s a fix in the works.
If you use with TF 2.5 it will work fine until the fix lands


Access to variable importances other than NUM_AS_ROOT

Question:
I am evaluating TensorFlow Decision Forests code as a replacement for existing code that usesTensorFlow’s BoostedTreesClassifier.
I am running TFDF v0.1.7 on Ubuntu Linux 20.04.2 LTS.
Using the example code in this tutorial as a starting point:

      TensorFlow

Build, train and evaluate models with TensorFlow Decision Forests 1

I have been able to train, evaluate, obtain a summary of the model, and save the model successfully using my own data.  I have been using tfdf.keras.RandomForestModel() as a starting point.
My problem is getting programmatic access to the different types of variable importances.  My understanding is that I should be able to do this by obtaining an inspector from the trained model.
   inspector = tfdf_model.make_inspector()
   variable_importances = inspector.variable_importances()

The  return value of variable_importances() is a dict as expected, but when I invoke keys(), the only key returned is NUM_AS_ROOT.  Judging from what I have seen in the output from summary(), I would have expected NUM_NODES, SUM_SCORE, and MEAN_MIN_DEPTH` to be present as well.
Is there something I need to specify to make those statistics accessible from variable_importances()?
Thanks!
–
P.J. Hinton

Context:
hi P.J.
You are doing exactly right.
Some metrics are more expensive to be calculated, need to be enabled by passing compute_oob_variable_importances=True. See list of flags here 3.
Please let me know if this helps.
But odd indeed, when I turn off compute_oob_variable_importances (I usually have it enabled), I do get the following keys: ['NUM_AS_ROOT', 'MEAN_MIN_DEPTH', 'NUM_NODES', 'SUM_SCORE'].
Are you doing regression or classification btw ?


Question:
So long story short, makesense.ai csv export is not compatible with AutoML, so I am trying to load my data using VOC XML (which I assume is equivalent to PASCAL VOC).
When I try to load the label map, I get this issue:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-b16cb19650e1> in <module>()
----> 1 train_data, validation_data, test_data = object_detector.DataLoader.from_pascal_voc('/content/drive/MyDrive/Colab\ Notebooks/example_data\ frames\ 16fps','/content/drive/MyDrive/Colab\ Notebooks/labels_deep-learning_2021-08-26-03-32-39',{1: 'person', 2: 'notperson'})

TypeError: cannot unpack non-iterable DataLoader object

Pretty sure my arguments are correct according to this documentation: tflite_model_maker.object_detector.DataLoader  |  TensorFlow Lite 3
I have passed the label_map as a copy/paste from the documentation, my images_dir argument is my folder with all of my images, and my annotations_dir argument is my folder with all of my xml files.
Note, I have more images than annotations - I don’t know if this causes problems but I assume that all that matters is that the images referenced in annotations exist in the images_dir - extra images not referenced by the xml annotation files will just be ignored.
Thoughts?

Context:
Erik, do you have a short snippet on a Colab with what you are trying to do? that would make it easier to help you.
@Yuqi_Li might be able to help


Using LSTM layer causes accuracy to stay at 50% in text classification

Question:
I’m doing text classification on the built-in IMBD dataset. A network with a dense layer and averaging over the temporal dimension with layers.GlobalAveragePooling1D does fairly well, but if I switch out the averaging with an LSTM layer (which has the same input and output dimensions, so the rest of the network can remain unchanged) the accuracy remains very close to 50%! I’m really confused as to why this could be. Below is code that anyone should be able to run. If you have any idea as to why this is happening please share!
import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, losses
from tensorflow.keras.layers.experimental.preprocessing import TextVectorization
import tensorflow_datasets as tfds

train_data, test_data = tfds.load(name="imdb_reviews", split=('train', 'test'), batch_size=-1, as_supervised=True)

xTrain, yTrain = tfds.as_numpy(train_data)
xTest, yTest = tfds.as_numpy(test_data)

max_features = 5000
max_len = 500
vectorize_layer = TextVectorization(max_tokens=max_features, output_mode='int', output_sequence_length=max_len)
vectorize_layer.adapt(xTrain)
xTrainV = vectorize_layer(xTrain)
xTestV = vectorize_layer(xTest)

model = tf.keras.Sequential([
  layers.Embedding(max_features+1, 16),
  layers.GlobalAveragePooling1D(),
  #layers.LSTM(8),
  layers.Dense(1)])

model.compile(loss=losses.BinaryCrossentropy(from_logits=True),
              optimizer='adam',
              metrics=tf.metrics.BinaryAccuracy(threshold=0))

model.fit(xTrainV,yTrain,validation_data=(xTestV, yTest), epochs=10)

The LSTM and GlobalAveragePooling1D layers can be switched between freely. An example progression of validation accuracies with  GlobalAveragePooling1D over 10 epochs is:
0.65, 0.71, 0.67, 0.76, 0.77, 0.79, 0.8, 0.8, 0.81, 0.83.
If the LSTM layer is used instead, both the validation accuracies as well as the training accuracies are all either 0.49, 0.50, or 0.51, after every epoch. There is no improvement and the network appears to predicting randomly. Why?

Context: Check out this tutorial on text classification with RNN: Text classification with an RNN  |  TensorFlow 3


Understanding Keras Dense Layer for rank >2

Question:
Hey all,
the official API doc states on the page regarding tf.keras.layers.Dense that

Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 0 of the kernel (using tf.tensordot 1). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).

My understanding is that only one kernel is created for ranks larger than 2. That would mean that (in case of rank 3) the “middle” dimensions are acted upon by the same kernel and thus the outputs for different indeces of the “middle” dimension are not independent.
Is that understanding correct? And if it is, is there a simple way to get the Dense layer to use a stack of kernels? (The network currently has the tensor multiplication implemented manually.)
Thank you all in advance.
Fabian

Context:
Your understanding is correct. If you want to contract away all non-batch dimensions, you can do something like this with the custom layer API:
class ExtendedDense(tf.keras.layers.Layer):
  def __init__(self, units):
    super(ExtendedDense, self).__init__()
    self.units = units

  def build(self, input_shape):
    rank = len(input_shape)
    self.axes = [list(range(1, rank)), list(range(0, rank - 1))]
    self.kernel = self.add_weight("kernel", shape=input_shape[1::] + (self.units,))
    self.bias = self.add_weight("bias", shape=(self.units,))

  def call(self, x):
    return tf.tensordot(x, self.kernel, self.axes) + self.bias

inp = tf.keras.layers.Input(shape=(64, 128,))
out = ExtendedDense(256)(inp)

model = tf.keras.models.Model(inp, out)

And usage:
x = tf.random.normal((32, 64, 128))
model(x)

Output:

<tf.Tensor: shape=(32, 256), dtype=float32, numpy=
array([[ 5.87857895e+01,  7.20561981e+01,  4.76380234e+01, ...,
         1.35572968e+02, -7.85310516e+01,  8.38421555e+01],
       [-7.71801834e+01,  1.77247452e+02, -1.48629440e+02, ...,
         3.40917473e+01,  1.80250931e+00, -8.22327881e+01],
       [ 2.06383377e+02,  5.67772293e+01, -1.80862427e-01, ...,
         6.32748642e+01,  3.95997772e+01, -4.52198296e+01],
       ...,
       [ 5.45089569e+01,  1.02864510e+02,  1.28402420e+02, ...,
        -1.03204765e+02, -1.85087776e+01,  3.91985130e+01],
       [-2.94154816e+01, -8.31197052e+01, -9.01190338e+01, ...,
         1.73478546e+02, -1.15511642e+02, -8.12764282e+01],
       [-1.32574982e+02,  5.21116562e+01,  1.01657272e+02, ...,
        -2.43008301e+02,  5.90627060e+01,  1.45800140e+02]], dtype=float32)>

<tf.Tensor: shape=(32, 256), dtype=float32, numpy=
array([[-2.30272830e-01, -1.33582354e+00, -2.89380431e-01, ...,
         1.00469515e-01, -1.17660590e-01,  2.11346745e-01],
       [-5.74482441e-01, -1.97411704e+00, -1.08441144e-01, ...,
        -1.35504484e+00,  3.19112301e-01,  1.21400483e-01],
       [ 2.24482596e-01, -3.09928954e-01,  1.68636054e-01, ...,
        -1.44872054e-01, -8.07262778e-01,  2.01987267e-01],
       ...,
       [ 1.22694087e+00, -1.46654761e+00, -5.89887261e-01, ...,
        -1.97368228e+00, -1.25852346e+00,  1.33283436e-04],
       [-1.33344865e+00, -5.02050817e-01,  8.51780176e-01, ...,
        -7.39977777e-01, -3.36842000e-01, -2.30333179e-01],
       [ 2.70515770e-01, -9.70930576e-01,  1.15700647e-01, ...,
         2.89812565e-01, -1.02232158e+00, -6.77653134e-01]], dtype=float32)>


Question:
Hi I am Sreejan Chattopadhyay. I have recently started with open sorce contribution. So I had to add test cases for some functions. After adding the test cases I tried to test it. But I got some errors.

image1699×913 301 KB

This is the error. So for crosschecking if it is my my code which caused the error I cloned the tensorflow repo without my changes. On testing the original code I got the same error.

image1885×943 314 KB

On careful analysing I saw the problem might be with my laptop beacuse the error was raised from tfjs backend file. My laptop does not contain any gpu or some highend configurations. So can anyone guide me on what to do next?
Regards,
Sreejan Chattopadhyay
@tfjs

Context: Thanks for posting and welcome to the forum. Looping in @Matthew_Soulanille who is our build expert on the TensorFlow.js team - do you know what may be going on here? Seems to be a Bazel thing.


How to load models from hub in tensorflow java

Question: I imported tensorflow java into scala and i cant find any good tutorials about how to load models from the hub

Context:
Hi Ahmed,
TensorFlow Hub doesn’t have a java version that’s why there’s no tutorial.
Are you planning on creating models in scala or just use them?
in either case, you can download the model (from the browser or with a python script) and load it using regular TensorFlow, the assets have nothing specific to Hub.


Is it normal to take 20-70 ms to predict on a sentence for Transformer?

Question:
I have a TF1.14 text classification problem, where each text is short since they are from social media. It uses a standard Transformer architecture from the Bert github:

      github.com

GitHub - google-research/bert: TensorFlow code and pre-trained models for BERT 3

  TensorFlow code and pre-trained models for BERT. Contribute to google-research/bert development by creating an account on GitHub.

In predictions, some text takes up to 70 ms, and most takes around 20 ms. I am running the predictions for each single example.
Is this normal for Bert’s prediction for a typical text classificsation?

Context: Also, on the same machine, I run the same program on the same data, the time (average) spent on predictions differs quite a lot.


Question:
Hi, all. I’m trying to run my model through tfjs with webgl backend. But I do not know how to get the time cost of model prediction. I did call new Date.now() before and after model.predict() and got a result around 100ms, it’s good enough. But between predict and my actural output with tensor.array(), I got A UI block around 700ms. Dose that mean it did not finish prediction in 100ms and need the following 700ms?
As tf doc say:
When an operation is called, like tf.matMul(a, b), the resulting tf.Tensor is synchronously returned, however the computation of the matrix multiplication may not actually be ready yet. This means the tf.Tensor returned is just a handle to the computation. When you call x.data() or x.array() , the values will resolve when the computation has actually completed.
It sounds like general operations return synchronously before they actually done, so do model.predict()?
Another thing border me is that if model.predict acts like the other general operations, then why do that take 100ms to return a not done handle? 10ms predict with a handle return, and 700ms to run model before output with tensor.array() would be more reasonable.

Context: Welcome to the forum and thanks for the question. Could you possibly share a live working version of your code on Glitch.com 2 or Codepen.io 3 for us to inspect? Thank you.


Question:
Hello everybody,
I am new to TensorFlow and to this community. I am trying to learn it from a very basic level. So, I have few questions about TensorFlow, which I could not find answers online. Any online tutorial on the following questions are welcome:

How does TensorFlow work under the hood? For example: DAG (Logical Model) and Stages (physical Execution Model) in the case of Apache Spark.
How does Tensorflow optimize the tensor graph for lazy (non-eager) evaluation? How does it gain speed-up while pipelining operations? (Stage pipelining in terms of Spark which can run in parallel)
What algorithm is followed by matmul function to multiply two dense matrices? I could not understand the algorithm from the code? Any reference would be appreciated.

Thanks in advance!

Context:
Hi Chandan,
I don’t have the answers but these videos might give you some insights:

TensorFlow from the ground up (ML Tech Talks) - YouTube 4
tf.function and Autograph (TF Dev Summit ‘19) - YouTube 1

more in-depth info you can find in these videos on this list: Inside TensorFlow 2


My libnnappi_delegate.so is debug or release

Question:
System information

OS Platform: ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: MI 9(mobile phone）
TensorFlow installed from: Source
TensorFlow version: 2.5.0
Bazel version (if compiling from source): 3.7.2
GCC/Compiler version (if compiling from source): 7.3.0
CUDA/cuDNN version: 10.0
GPU model and memory: Tesla K40m

Describe the problem
I use bazel to build libnnappi_delegate.so.
like this:
bazel build -c opt  --config=android_arm64  --cxxopt="-std=c++11"  --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain     //tensorflow/lite/delegates/nnapi:nnapi_delegate
get libnnappi_delegate.so .

How can I know the shared library is “Debug” or “Release” version?
How to “Debug” builds of libnnappi_delegate?
TF issue refuse to answer my question and push me here  .

Context: Since you compile with the “-c opt” option, this can be considered as an optimized binary size.


TensorFlow.js Op support matrix - if you are having Python model conversion issues, check this first!

Question:
So I have seen a few forum posts around conversion of TF Python models to TensorFlow.js format and sometimes running into an issue where the Python model uses some unsupported op.
Well, I am pleased to share a nice easy to view spreadsheet that shows the current state of op support in TensorFlow.js across the many backends we support such as CPU, WASM, WebGL, Node, WebGPU (each is like its own version of TensorFlow) for which each op needs to be written in that backend for it to be supported on that backend. As such, not all backends are equal in op parity right now as there are a lot to implement!
If you find one that is not supported and are willing to contribute its implementation, please do!
Here is the spreadsheet (created by our wonderful SWE @Jing_Jin) which we aim to update over time:
TFJS Ops Matrix (last updated: 2021-8-26) - Google Sheets 23.
One other thing to note is that some ops are partially implemented. What this means is that some parts of the function definition work, but maybe extra non essential parameters are not. In this case if function definition is something of the form:
function (a, b, c, d, e) {
   // code...
}

But a given backend’s implementation of that op is partial eg:
function (a, b, c) {
   // code...
}

Then clearly here if you attempt to use parameters d and e, it may not behave as expected which can cause issue. Of course if you are not using these parameters then it will work as intended. Something to consider if you are finding odd issues with a converted model where all ops are “supported” you may find one of those ops is partial leading to the issue you see.
We hope to update the spreadsheet above in the future showing which are partial but this is a work in progress right now.

Context: Thank you @Jason! This info really helpful


Remove any Background image - TensorFlowJs

Question:
Friends, I have a big problem.
Today with tensorflowjs/body-pix I can remove the background from an image and leave just one person.
Turns out I have a motorcycle parts system and the background photos have ‘noises’ and I wanted to remove that background and leave just the part so my model can tell me which part it is.
How would I get TensorFlowJs to remove the background and leave just the part once I have the pre-trained model of the parts?
Is there anything like Body-pix that does this?

Context:
Welcome to the forum Cristian! Thanks for posting your question and for trying out TensorFlow.js!
So our bodypix model is currently trained on human bodies as you have guessed. If you want to make your own segmentation model for a custom thing you will need to start collecting annotated training data (1000s of images)  with examples of the thing you are trying to recognize and ensure you have a pixel level segmentation available for each image from which the model can learn from.
Do you have the training data? That would be your first task.
You can then choose a segmentation model you want to retrain for it to learn from your data instead.
To do that you have a number of options:

Find an unfrozen model that has already been defined and retrain it with your data (easier but may have to start in Python first and then convert to TensorFlow.js with our converter).

A good tutorial for (1) is by our community expert Hugo Zanini:
#MadeWithTFJS Interview:
Real-time semantic segmentation in the browser  - Made with TensorFlow.js

Write up:

      Medium – 19 Jun 21

Real-Time semantic segmentation in the browser using TensorFlow.js 45

  Perform semantic segmentation directly from your browser. There is no need for complex libraries or powerful computers.

    Reading time: 5 min read

Code here:

      github.com

GitHub - hugozanini/realtime-semantic-segmentation: Implementation of RefineNet... 34

  Implementation of RefineNet to perform real time instance segmentation in the browser using TensorFlow.js - GitHub - hugozanini/realtime-semantic-segmentation: Implementation of RefineNet to perfor...

Write the code yourself to define the model using the TensorFlow.js APIs and train that (harder)

Maybe @Shanqing_Cai has some thoughts here though as he actually worked on BodyPix!


Question:
I’m trying to make a model that can predict that y = sqr(x)^2 or multiplying two numbers of input like y = x1 * x2 with a lot of data of multiplying two numbers but the model can’t learn or predict.
I read in some pages that I need to use something like log input and exp output of numbers before the model and after prediction, I think it may work but I want to make it inside the model so that I can predict more complex functions with multi-input with different operations like summation, multiplication powers and roots in the same function for multi-input
how to build that model :\

Context:
I suggest to take a look at:

      arXiv.org

A Primer for Neural Arithmetic Logic Modules 4

  Neural Arithmetic Logic Modules have become a growing area of interest,
though remain a niche field. These units are small neural networks which aim to
achieve systematic generalisation in learning arithmetic operations such as {+,
-, *, \} while...


Need help training with ModelMaker & Cloud TPU in Colab

Question:
I’m trying to train an object detection model in colab using tflite model maker and I’d like to use a cloud TPU, but so far all I’m getting are errors… I have searched for any tutorial that shows how to do it, but have come up empty. I’d appreciate any tips on what to try next!
The essence of what I have so far is derived from the Model Maker Object Detection Tutorial 1:
tpu = tf.distribute.cluster_resolver.TPUClusterResolver()
print(tpu.master()) # --> grpc://10.5.111.234:8470
train_data, validation_data, test_data = object_detector.DataLoader.from_csv(
    drive_dir + 'cub.csv', images_dir = "images", num_shards = 10)
spec = object_detector.EfficientDetLite0Spec(tflite_max_detections=10,
                                             strategy='tpu', tpu=tpu.master(), debug=True)
model = object_detector.create(train_data=train_data,
                               model_spec=spec,
                               validation_data=validation_data,
                               epochs=15,
                               batch_size=30,
                               train_whole_model=True)

The error I get in the last step:
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in shape(self)
   1196         # `_tensor_shape` is declared and defined in the definition of
   1197         # `EagerTensor`, in C.
-> 1198         self._tensor_shape = tensor_shape.TensorShape(self._shape_tuple())
   1199       except core._NotOkStatusException as e:
   1200         six.raise_from(core._status_to_exception(e.code, e.message), None)

InvalidArgumentError: Unsuccessful TensorSliceReader constructor: Failed to get matching files on /tmp/tfhub_modules/0771ebbebbfa831cd20c20680920ec4ad9deb2bd/variables/variables: Unimplemented: File system scheme '[local]' not implemented (file: '/tmp/tfhub_modules/0771ebbebbfa831cd20c20680920ec4ad9deb2bd/variables/variables')

Is the issue that I have to put the training images into GCS? Or is there more to getting this to work?

Context: I moved all the images to a GCS bucket and adjusted the references in the CSV, but I still get the same error…


Tensorflow Task Library - Max 4 clases detected

Question:
Hello, we are working on Android application.
For model creation we are using Google Colab.
We created our model which contains 8 classes. Unfortunatelly we have issue with detection of more then 4 classes. Our first model was containing only 4 clases - all single word named. After our dataset update we added 4 more classes into the model, but in application we are still able to detect only 4 previous (old) classes. The new clases contains name in this way: “car-main” etc. could that cause the problem?
In the Google colab after data validatations we are getting all 8 classes with validation score . In our app we are working with Task Library for connection with .tflite model.
Edit: Things what I forget mention:
First 4 classes have validation score in this range:  from 70 % to 30 %.
Last 4 classes have validation score in this range:  from 28 % to 11 %.
-both after quantization.
We cannot detect new classes even on the test image.
Tensorflow version: 2.5.0.
Thank you for any suggestions!

Context:
Hi Civo,
let me try to understand, you:

fine tuned a model with 4 classes. That worked fine
you’ve added another 4 classes, fine tuned again, the model works but with very low accuracy
on-device the model is not recognizing any of the new classes.

Did I get it right?
some suggestions:

by your results, it seems that the quality of the data used for training the additional 4 classes is worse than the original 4. Maybe you might need to look into that with more data for example
since the accuracy is low, Task Library might have a default scoreThreshold that is too high for the new classes and so it’s filtering out them during inference.


Warnings on model conversion from PyTorch (ONNX) to TFLite

Question:
I was following this guide 15 to convert my simple model from PyTorch to ONNX to TensorFlow to TensorFlow Lite for deployment. But I received the following warnings on TensorFlow 2.3.0:
2021-08-19 12:01:09.231710: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2021-08-19 12:01:09.347085: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:313] Ignored output_format.
2021-08-19 12:01:09.347119: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored drop_control_dependency.

On TensorFlow 2.5.0:
2021-08-19 14:50:23.764805: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as gen_tensor_dict while saving (showing 1 of 1). These functions will not be directly callable after loading.
2021-08-19 14:50:23.853142: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:345] Ignored output_format.
2021-08-19 14:50:23.853172: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:348] Ignored drop_control_dependency.
2021-08-19 14:50:23.853177: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:354] Ignored change_concat_input_ranges.

Other versions (The onnx-tf was from PyPI instead from their latest GitHub):
numpy == 1.19.5
onnx == 1.10.1
onnx_tf == 1.8.0
tensorflow_gpu == 2.5.0
torch == 1.9.0

Although the model gets converted without any errors and I’ve checked the ONNX graph and it looks correct and I’ve also tried inference with dummy values and that also works but I want to be sure that the model doesn’t break after deployment because of something I missed in these warnings.

Context: You can safely ignore the above warning messages.


How to apply k-means clustering to 4D tensor?

Question:
I received tensor of 4 dimensions (805, 8, 8, 128)  from latent space of autoencoder (as input it was tensor of (1108, 32, 32, 3) - 1108 arrays of 32x32 pictures of 3 colors)
now I want to apply k-means  to my feature  tensor of 4 dimensions (805, 8, 8, 128), but every kmeans function I try (from sklearn, scipy etc) it through an error that number of dimensions different than 1 or 2 are not supported
how could i find clusters above all my arrays of feature tensor?

Context:
As e.g. scikit-learn max 2d arrays inputs. But you can reshape the input.
See python 3.x - Applying Kmeans on 3D data with Scikit-learn - Stack Overflow 8


Question:
I am using a pretrained model for extracting features(tf.keras) for images during the training phase and  running this in a GPU environment.
After the execution gets completed, i would like to release the GPU memory automatically without any manual intervention.

Context: Have you tried with tf.keras.clear_session()?


Question: I want to contribute to the Tensorflow project but because this project is huge  and complex I need a mentor to guide me and make me part of some team ?

Context:
Welcome to the TF forum. What do you want to contribute? Do you have already a dev environment?
/cc @thea @Joana


Question: I am not sure how to compile the yggdrasil c++ library on a raspberry pi 3b+ (armv7). I have tried just about everything I can think of and still it errors out. Is there any tips or documentation I can use for building the libraries required to run a tensorflow decision forest model on a raspberry pi armv7?

Context: @Jan can you share any insights here?


Question: Is there any way to rename an existing tf.Variable without creating a new object?

Context:
Could you be more specific about the context?
What name are you trying to change? why?


How to change seq length in BERT preprocessor from TF Hub

Question:
I am following this example 2 to use BERT for sentiment classification.
text_input = tf.keras.layers.Input(shape=(), dtype=tf.string)
preprocessor = hub.KerasLayer(
    "https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3")
encoder_inputs = preprocessor(text_input)
encoder = hub.KerasLayer(
    "https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4",
    trainable=True)
outputs = encoder(encoder_inputs)
pooled_output = outputs["pooled_output"]      # [batch_size, 768].
sequence_output = outputs["sequence_output"]  # [batch_size, seq_length, 768].
embedding_model = tf.keras.Model(text_input, pooled_output)sentences = tf.constant(["(your text here)"])print(embedding_model(sentences))

The sequence length by default seems to 128 from looking at the output shape from encoder_inputs. However, I’m not sure how to change this? Ideally I’d like to use to a larger sequence length.
There’s an example of modifying sequence length from the preprocessor page 4, but I’m not sure how to incorporate this into the functional model definition I have above? I would greatly appreciate any help with this.

Context:
Hi Kay,
If you look into this tutorial: Solve GLUE tasks using BERT on TPU  |  Text  |  TensorFlow 47
You’ll get this exactly information to help you. To be more exact, in the make_bert_preprocess_model  function


Question:
Write checkpoint:
  class Mymodule (tf.Module):
    def __init__ (self):
      super().__init__()
      self.m = tf.Variable (5.0)
      self.b = tf.Variable (5.0)
    def __call__ (self, x):
      return self.m * x + self.b

  m = Mymodule()
  cp = tf.train.Checkpoint (model=m)
  print ("cp:", cp)
  cp.write ("mycheckpoint")
  print ("list_variables:", tf.train.list_variables ("mycheckpoint"))

Output:
cp: <tensorflow.python.training.tracking.util.Checkpoint object at 0x7f95fc917940>
list_variables: [('_CHECKPOINTABLE_OBJECT_GRAPH', []), ('model/b/.ATTRIBUTES/VARIABLE_VALUE', []), ('model/m/.ATTRIBUTES/VARIABLE_VALUE', [])]

Context:
Your tf.Module has variables. You can verify with  m.variables before saving the checkpoint.
But I don’t think It is available in that train checkpoint call without you have trained the model.


NN implementation using Low Level TF API

Question: I have previously used Keras or high-level API for creating NN. However, currently, I wish to create a multi-layer NN using the low-level APIs offered by TF. If you guys know helpful blogs, articles, videos on this topic, that’d be great. Any suggestions are welcome!

Context:
You can start with:

      TensorFlow

Introduction to modules, layers, and models  |  TensorFlow Core 3


Error while converting saved model to mlir

Question:
I am trying to convert tensorflow saved model to mlir. I am using tensorflow/bazel-bin/tensorflow/compiler/mlir/tf-mlir-translate --savedmodel-objectgraph-to-mlir  /path/to/saved_model  -o out.mlir.
iam getting error as  SavedModel import failed: Invalid argument: Exported function with exported name(s) ‘call’, ‘keras_api.call’ with multiple concrete functions. Add @tf.function(input_signature=[…]) on this function, or use a narrower list of exported names that excludes this function.
Thanks

Context: /cc @Jacques_Pienaar


Why I am getting n predictions instead of one for single sample?

Question:
I wonder why my neural network give me n predictions for single sample. I was expecting to get only one prediction if I pass only one sample with size of input of neural network input layer, but I am always getting as many predictions as my input vector length. For example, if I have a neural-network which input layer size is 3, and I pass a vector with 3 values, I will get 3 predictions (each with size of my output layer size). Why is it like that, and what should I change to get only one prediction?
function createNetwork(inputSize, outputSize) {
  const model = tf.sequential();
  model.add(
    tf.layers.dense({
      units: inputSize,
      inputShape: [1],
      activation: 'relu'
    })
  );
  model.add(
    tf.layers.dense({
      units: 16,
      activation: 'relu'
    })
  );
  model.add(
    tf.layers.dense({
      units: outputSize
    })
  );
  model.compile({
    loss: 'meanSquaredError',
    optimizer: 'adam'
  });
  return model;
}

const network = createNetwork(3, 2);
const sample = tf.tensor1d([1, 2, 3]);
const prediction = network.model.predict(sample).arraySync();
console.log(prediction)

In this example, I have created a neural network with input layer of size 3 and output layer of size 2 then I have passed a vector with 3 variables and get 3 predictions (3×2 matrix insted 1x2 matrix)

Live demo 1

Context:
Hi,
So, as far as I understood your code,
you defined an input of 1 variable into a dense layer with 3 neurons. The output is 2 values as your last dense layer has 2 neurons.
If you give an input with 3 values, for each one you get 2 values, hence the 3x2 result
to change that, you need to change input_shape to 3, this will make your model understand the 3 values as one input and the output will be 1x2
model = keras.Sequential(
    [
        layers.InputLayer(input_shape=(3,)),
        layers.Dense(3, activation="relu", name="layer1"),
        layers.Dense(16, activation="relu", name="layer2"),
        layers.Dense(2, name="layer3"),
    ]
)
# Call model on a test input
model.summary()
x = tf.ones((1, 3))
y = model(x)
print(y)

sorry for the non-JS code


Question:
Recently i do training with tensorflow and then convert it to tensorflow lite model
This is how i convert the model

Convert model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

save model.
with open(‘C:/Users/MobileNetV2_2/model52/model.tflite’, ‘wb’) as f:
f.write(tflite_model)

model.save(‘C:/Users/PAKPAHAN/MobileNetV2_2/2’)
How can i know the accuracy from tflite model, i already try
model.evaluate_tflite(“model.tflite”, test_dataset)
But it got error, the error is
AttributeError: ‘Functional’ object has no attribute ‘evaluate_tflite’

Context:
Apparently, by the code snippet, the model variable was not created by Model Maker, right? It seems you converted a model using the TFLite converter
the method evaluate_tflite is specific to Model Makers model class and it’s not a regular TFLite method.
you can see the code here 37
As you can see, it’s specific per problem domain. You can follow the code to implement your own version if needed.


Question:
Hi,
i have some trained Models on TF2 and i want to measure the performance while executing the inference. I have seen that there is something like that for TensorFlow Lite, where you can measure
Initialization time
Inference time of warmup state
Inference time of steady state
Memory usage during initialization time
Overall memory usage

Is there something for normal TensorFlow?
I use the Python API on Jetson Platform (Linux aarch64)
Thank you very much

Context: Have you seen TensorFlow Profiler: Profile model performance  |  TensorBoard 23?


TensorFlow Developer Stipend didn’t send me my Coursera Courses

Question:
Hi,
I got approved for the TensorFlow Developer Stipend 2 weeks ago and am still waiting for the Coursera course codes that come with it. I have emailed Coursera Support and they referred me to the forum.
Thanks,
Andrew

Context:
Hi Andrew,
Welcome to the TensorFlow Forum! Thank you for contributing to the community.
For TensorFlow education program related questions, please reach out to the TensorFlow Certificate Team at tensorflow-certificate-support@google.com. Thanks!


Question: Does plugable devices also support FPGAs?

Context:
Hi Chris,
Sorry for the late reply! We have talked through another channel but I’ll post here too for other’s info:
If the FPGA code can connect to TensorFlow through C API, it should work. Here are the overall steps:

Create a PluggableDevice 2.
Write your custom TensorFlow kernels/ops and register them to TensorFlow through the kernel and op registration C API 1. (We also extended it to support ResourceVariable ops recently.)
Use the StreamExecutor C API for device execution and memory management.
If you’d like to do graph optimization, your plug-in can register a custom graph optimization pass through the graph optimization C API 2.
We are also looking into a TF Profiler C API 2 for PluggableDevices.

Here’s a tutorial and example code 6 under construction.
I should also add that PluggableDevice is focused on TensorFlow’s current runtime stack. It may require some migration efforts to work with the new runtime stack 4.
Best,
Penporn


Applying Grad-Cam to a model that was converted to Tensorflow Lite

Question:
I have a TensorFlow model converted to TensorFlow Lite, where my objective is to run it on Raspberry Pi. My model is a classification model that uses quantization aware training before conversion to the TFLite model. According to this example: Grad-CAM class activation visualization 6. I successfully applied grad-cam to an actual Tensorflow model. However, I did not find the methods to compute the gradients for Tensorflow Lite.
How can I replicate the TensorFlow grad-cam but for TensorFlow lite?

Context:
Please export the generated TensorFlow model as a saved model and convert it through the TFLite converter API.

      TensorFlow

TensorFlow Lite converter 9


Custom Object Detection With TensorFlow.js Possible?

Question:
Hi, is there any good tutorial out there that shows how to train an object detection model using a custom dataset? is it even possible with tfjs?
I’ve seen the coco-ssd tfjs repository 11 and it got it working fine, but if the object you are trying to detect is not in the very limited number of pre-trained classes, the example is not very useful.
Thanks.

Context:
I haven’t tried it myself but found some promising articles which you might be interested in:

Custom object detection in the browser using TensorFlow.js 54
TFJS-object-detection repository 17
 Edge TensorFlow.js tutorial 15

I hope these help. Kind regards,
Gilbert Tanner


When using keras to train the model, how to use the output of the current round as the input of the next round

Question: Hello. I want to use keras for time series forecasting. When forecasting, I hope that the forecast result at time t will be used as the input at time t+1. So when I want to train, I hope to use the output at time t as the input at time t+1. May I ask how it can be implemented in keras. Or how can it be achieved with tensorflow. The desired effect is shown in the figure.https://user-images.githubusercontent.com/52521165/127980768-1e6ae109-5761-46ca-8c43-50ccd103d8c8.png 5

Context:
If you train LSTM model to predict future values one step ahead, you can use a deque object to obtain a forecast for several steps ahead. At first you put your latest available data to the deque and pass this same data to the model to get a predicted value for the next time step. Then you use popleft() to remove the earliest value from the deque and append the predicted next value to the right. Repeat this procedure as many times as necessary.
Other option is to create LSTM model which will output predicted values for several steps ahead. If your model contains one or several LSTM layers put a dense layer with several units at the end and train this model with targets representing equal number of future data points.


AutoML TensorflowLite export is broken for Object Detection

Question:
Hey all!
Not sure if this is the right place to post.
I think in the current version of VertexAI/AutoML Vision for Object Detection, there is a bug with tensorflow lite exporting. This is something new, it worked perfectly fine a month ago.
Specifically, the TensorFlow Lite object detector’s confidence scores cap at 0.5. It’s not the model itself, since the TFJS export works well without the 0.5 maximum on the bounding boxes.
I think it’s just a bug in AutoML/Vertex AI that renders the TFLite Object Detection model completely unusable.
I reported the issue here 7, but just wanted to give it more visibility since it’s a complete roadblock for anyone using Object Detection with Vertex AI/AutoML.
Let me know if there’s anything I can do to help! Happy to test any fixes and verify they work

Context:
Whats odd is one environment works, and the other one I’m running is getting the RunTimeError:
Failed to prepare for TPU. Failed precondition: Package requires runtime version (14) which is newer than this runtime version (13). Node number 2 (EdgeTpuDelegateforCustomOp) failed to prepare


Question:
I’m sorry, I made mistake.
I trained a neural network and it works fine in the code below.
I’m trying to modify, to detect multiple objects.
I appreciate any help.
Thank you very much.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import argparse
import time

import numpy as np
from PIL import Image
import tensorflow as tf # TF2

def load_labels(filename):
  with open(filename, 'r') as f:
    return [line.strip() for line in f.readlines()]

if __name__ == '__main__':
  parser = argparse.ArgumentParser()
  parser.add_argument(
      '-i',
      '--image',
      default='/tmp/grace_hopper.bmp',
      help='image to be classified')
  parser.add_argument(
      '-m',
      '--model_file',
      default='/tmp/mobilenet_v1_1.0_224_quant.tflite',
      help='.tflite model to be executed')
  parser.add_argument(
      '-l',
      '--label_file',
      default='/tmp/labels.txt',
      help='name of file containing labels')
  parser.add_argument(
      '--input_mean',
      default=127.5, type=float,
      help='input_mean')
  parser.add_argument(
      '--input_std',
      default=127.5, type=float,
      help='input standard deviation')
  parser.add_argument(
      '--num_threads', default=None, type=int, help='number of threads')
  args = parser.parse_args()

  interpreter = tf.lite.Interpreter(
      model_path=args.model_file, num_threads=args.num_threads)
  interpreter.allocate_tensors()

  input_details = interpreter.get_input_details()
  output_details = interpreter.get_output_details()

  # check the type of the input tensor
  floating_model = input_details[0]['dtype'] == np.float32

  # NxHxWxC, H:1, W:2
  height = input_details[0]['shape'][1]
  width = input_details[0]['shape'][2]
  img = Image.open(args.image).resize((width, height))

  # add N dim
  input_data = np.expand_dims(img, axis=0)

  if floating_model:
    input_data = (np.float32(input_data) - args.input_mean) / args.input_std

  interpreter.set_tensor(input_details[0]['index'], input_data)

  start_time = time.time()
  interpreter.invoke()
  stop_time = time.time()

  output_data = interpreter.get_tensor(output_details[0]['index'])
  results = np.squeeze(output_data)

  top_k = results.argsort()[-5:][::-1]
  labels = load_labels(args.label_file)
  for i in top_k:
    if floating_model:
      print('{:08.6f}: {}'.format(float(results[i]), labels[i]))
    else:
      print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))

  print('time: {:.3f}ms'.format((stop_time - start_time) * 1000))
    indent preformatted text by 4 spaces
[/quote]

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import argparse
import time

import numpy as np
from PIL import Image
import tensorflow as tf # TF2

def load_labels(filename):
  with open(filename, 'r') as f:
    return [line.strip() for line in f.readlines()]

if __name__ == '__main__':
  parser = argparse.ArgumentParser()
  parser.add_argument(
      '-i',
      '--image',
      default='/tmp/grace_hopper.bmp',
      help='image to be classified')
  parser.add_argument(
      '-m',
      '--model_file',
      default='/tmp/mobilenet_v1_1.0_224_quant.tflite',
      help='.tflite model to be executed')
  parser.add_argument(
      '-l',
      '--label_file',
      default='/tmp/labels.txt',
      help='name of file containing labels')
  parser.add_argument(
      '--input_mean',
      default=127.5, type=float,
      help='input_mean')
  parser.add_argument(
      '--input_std',
      default=127.5, type=float,
      help='input standard deviation')
  parser.add_argument(
      '--num_threads', default=None, type=int, help='number of threads')
  args = parser.parse_args()

  interpreter = tf.lite.Interpreter(
      model_path=args.model_file, num_threads=args.num_threads)
  interpreter.allocate_tensors()

  input_details = interpreter.get_input_details()
  output_details = interpreter.get_output_details()

  # check the type of the input tensor
  floating_model = input_details[0]['dtype'] == np.float32

  # NxHxWxC, H:1, W:2
  height = input_details[0]['shape'][1]
  width = input_details[0]['shape'][2]
  img = Image.open(args.image).resize((width, height))

  # add N dim
  input_data = np.expand_dims(img, axis=0)

  if floating_model:
    input_data = (np.float32(input_data) - args.input_mean) / args.input_std

  interpreter.set_tensor(input_details[0]['index'], input_data)

  start_time = time.time()
  interpreter.invoke()
  stop_time = time.time()

  output_data = interpreter.get_tensor(output_details[0]['index'])
  results = np.squeeze(output_data)

  top_k = results.argsort()[-5:][::-1]
  labels = load_labels(args.label_file)
  for i in top_k:
    if floating_model:
      print('{:08.6f}: {}'.format(float(results[i]), labels[i]))
    else:
      print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))

  print('time: {:.3f}ms'.format((stop_time - start_time) * 1000))```

Context: It’s hard to get what the problem is here if you only paste in your code. So please add some explanation of why you created this discussion. Also, I’d recommend using codeblocks 3  as it makes the code a lot more readable and also please try to keep the correct indentation of the code as Python code is hard to read otherwise.


Help needed: GPU version slow at (likely) device setup

Question:
I just got a new GPU and am trying out the GPU version of tensorflow. I installed it through conda
conda create -n tf-gpu tensorflow-gpu
conda activate tf-gpu

My little test script worked okay except that
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28,28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])

takes quite a while. It prints out things like
2021-08-22 19:28:51.617228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-08-22 19:28:51.617263: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-08-22 19:34:08.926351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-08-22 19:34:08.926382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2021-08-22 19:34:08.926392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N

where as you can see it takes minutes before the “Device interconnect” line is printed.
The above Sequential line is done within a second if I ran the CPU version of tensorflow.  Also the actually training of the network with the GPU version is only twice as fast as the CPU version.
I see that the conda tensorflow package comes with CUDA 10.1.  I have CUDA 11.2 installed elsewhere.  Not sure if that’s a problem or not.
Am I missing something in the setup of the GPU or elsewhere?
Thanks!

Context:
Can you try to run this in our official GPU image:

      TensorFlow

Docker  |  TensorFlow 1


Question:
I’m having a problem deploying my model on the browser.

image959×238 36.5 KB

image938×389 53.9 KB

Context: Are you already make sure to call initialize function? It’s look like your model not loaded yet.


TensorflowLite error: Didn’t find op for builtin opcode ‘SOFTMAX’ version ‘1’. An older version of this builtin might be supported. Are you using an old TFLite binary with a newer model?

Question:
My envs:
tensorflow version==2.0.0 with cpu
python version==3.7
os: windows10
When I trained a keras model, I used tensorflow lite converter to transfer the code into .tflite, and then use xxd tool to transfer .tflite file into C++ code,  in order to deploy the model on the embedded system. But when I compiled the code, it showed the error:

Here is the part of code during the training procedure:

图片793×319 33.7 KB

Thanks so much!

Context: We see that you are using tf version 2.0.0 is not actively supported, please update to stable version tf v2.5, v2.6 and let us know if you are facing same issue.Thanks!


TypeError fitting Beta-binomial mixture density network

Question:
I am trying to fit a Mixture Density Network model using tensorflow, following the example posted in Taboola’s blog - Predicting Probability Distributions Using Neural Networks 2 (acompaniying notebook here - https://github.com/taboola/mdn-tensorflow-notebook-example/blob/master/mdn-tf2.ipynb 1).
The data I am fitting are positive integers so I initially modified the blog example to use a Poisson distribution instead of a Normal from the blog example:
def poisson_mdn_cost(lambda_, y_):
    dist = tfp.distributions.Poisson(rate = lambda_)
    return tf.reduce_mean(-dist.log_prob(y_))

epochs = 500
batch_size = 50
learning_rate = 0.0003
InputLayer = Input(shape=(1,))
Layer_1 = Dense(12,activation="tanh")(InputLayer)
Layer_2 = Dense(12,activation="tanh")(Layer_1)
lambda_ = Dense(1, activation=lambda x: tf.nn.elu(x) + 1)(Layer_2)

Y = Input(shape=(1,))
lossF =  poisson_mdn_cost(lambda_,Y)
model2 = Model(inputs=[InputLayer, Y], outputs=[lambda_])
model2.add_loss(lossF)
model2.compile(optimizer = 'adam',
               metrics = ['mse'],
             )
history_cache2 = model2.fit([x_data, y_data],
                            verbose=0,
                            epochs=epochs,
                            batch_size=batch_size)
print('Final cost: {0:.4f}'.format(history_cache2.history['loss'][-1]))

The Poisson MDN fitted fine and I am now running an experiment with a Beta-binomial MDN using the following code:
def betabin_mdn_cost( a_,b_, y):
   dist = tfp.distributions.BetaBinomial(total_count =10,
                                         concentration0=a_,
                                         concentration1=b_)
   return tf.reduce_mean(-dist.log_prob(y))

epochs = 500
batch_size = 50
learning_rate = 0.0003
InputLayer = Input(shape=(1,))
Layer_1 = Dense(12,activation="tanh")(InputLayer)
Layer_2 = Dense(12,activation="tanh")(Layer_1)
a = Dense(1, activation="relu")(Layer_2)
b = Dense(1, activation="relu")(Layer_2)

Y = Input(shape=(1,))
lossF =  betabin_mdn_cost(a,b,Y)
model3 = Model(inputs=[InputLayer, Y], outputs=[a,b])
model3.add_loss(lossF)
model3.compile(optimizer = 'adam',
               metrics = ['mse'],
             )
history_cache3 = model3.fit([x_data, y_data],
                            verbose=0,
                            epochs=epochs,
                            batch_size=batch_size)
print('Final cost: {0:.4f}'.format(history_cache3.history['loss'][-1]))

The data is truncated between 0 and 10, so fixed the Beta-binomial total_count parameter to 10. As far as I can see, the latter code has minimal changes compared with the Poisson above, essentially just swapping the Poisson cost function by the Beta-binomial and adding the additional parameter. However this time I can’t get it to run and get the error:
  TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

I an new to tf and have exhausted my ability to try to get this to work. I Was wondering if there is anybody out there with experience in fitting these types of models using tf, that might have some insights?
[tf.version.VERSION = 2.6.0]

Context: /cc @Christopher_Suter


Question:
Hello, I’m workink on the text similarity between two variables which have str and number as a type(police number and text about explaination).
So, I found a tutorial on TensorFlow Blog on semantic with tf_hub and I want to know if is good for my project.

Context:
HI Armel,
if you the problem you are trying to solve is: given sentence A and Sentence B, are they a paraphrase of each other?
is this it?
if so, you can use this: Solve GLUE tasks using BERT on TPU  |  Text  |  TensorFlow 7
more specifically the QQP benchmark.


Very low FPS in tensorflow lite example on Windows 10

Question:
Hi, i succesfully build and run the code example from here: GitHub - Qengineering/TensorFlow_Lite_Face_Mask_RPi_64-bits: Face mask detection on a bare Raspberry P 12
I am using Windows 10 PC (32 gb RAM, core i7, nvidia GEForce 1080TI), but FPS is only 5-6.
Could anybody please help me to find out why is so low fps? What should i change in code to increase fps to 20-30?
Thank you!!!

Context:
I suppose that you want to run this on a Raspberry.
Have you explored the option to use an accelerator like this one:

      Coral

Get started with the USB Accelerator | Coral 9

  Learn how to set up the Coral USB Accelerator and run some demo code.

Here you can find the performance of some reference models for the object detection on Google Coral TPU:

      Coral

Models - Object Detection | Coral 9

  Models that identify multiple objects and provide their location.


Question:
I am trying to figure out tf.function converts a python function to a graph and calls it. And I have found that in tensorflow/python/ops/functional_ops.py:partitioned_call,
op = graph.create_op(op_name, args, tout, name=op_name, attrs=op_attrs)
outputs = op.outputs

It seems that the graph is converted to an operation, but when I read the documentation of Operation , it says

An Operation is a node in a tf.Graph that takes zero or more Tensor
objects as input, and produces zero or more Tensor objects as output.

So I assume that Operation should be a node in the graph. That makes me wonder what graph.create_op really does. Is graph converted into a big Operation here?

Context:
Is graph converted into a big Operation here?

That is my understanding. That functions in the tensorflow runtime are implemented as a separate (partitioned) graph, and called from the main graph as if they were a regular Op.


Question:
I wanted to give the prediction output from esp32 to AWS cloud.  Through coreMQTT.
The problem I am facing is about the definition of the library that is TensorFlow library and .h file of the model.

Context: Have you checked First steps with ESP32 and TensorFlow Lite for Microcontrollers | by Dmytro Korablyov | Medium 19 ?


Question:
Can I use the json format instead of xml to train the models. xml only gives me teh bounding box and while I JSON was used creating teh ploygon method.
So is it possible to traing models using the json format . so please let me know

Context: On what kind of Model?


Question:
Hi community, I am doing inference of faster rcnn-resnet, having 24 GB titan rtx GPU, GPU memory utilisation is 100% but GPU utilisation is around 50%, and I am getting around 10 fps and the benchmark of that same model on rtx is 115 FPS can anyone Guide for this? Thank

Screenshot from 2021-08-19 16-21-35726×441 48.4 KB

Context:
Have you tried to profile your run with this?

      TensorFlow

TensorFlow Profiler: Profile model performance  |  TensorBoard 5


Tensorflow c API lib is missing “cc” folder with lot of header files

Question:
I downloaded an official tensorflow c api lib for windows with gpu from Install TensorFlow for C 1.
The file name is:
libtensorflow-gpu-windows-x86_64-2.5.0.zip
But the lib doesn’t contains folder “cc” that contains lot of header files used in examples in the github page. (For example: tensorflow/main.cc at master · tensorflow/tensorflow · GitHub 6)
So I guess that the lib files do not contains compiled code for some functions.
I confused, where can I downloaded a tensorflow c api compiled lib that allows me to write code like in the github examples?

Context:
If I remeber correctly I think we are still in this status

      github.com/tensorflow/tensorflow

      Bazel target to create C++ library headers 2

        opened

        Jul 28, 2016

          closed

        Jul 29, 2016

          hillegass

        stat:awaiting response

    Thanks to the `//tensorflow:libtensorflow.so` target, I can now build a shared l…ibrary that I can use in my C++ programs.  I want to install that library in `/usr/local/lib` and put the headers for it in `/usr/local/include`, but there is no target to build and install those headers.  In particular, building the library doesn't build the headers from the proto files, so I have `error_codes.proto`, but not `error_codes.pb.h`, so I can't even use the source directory for the headers.


Convert Source Code and Model Weights from Tensorflow to Pytorch

Question:
Hi Community,
I have the following code of a two-layer BiLSTM as follows . It is a fairly large implementation.
from utils import *
import tensorflow as tf
from functools import reduce
from operator import mul
import numpy as np

def rnn_encoder(rnn_type, x, mask, d_dim, n_layer, initializer, pooling=None):
    seq_len = tf.reduce_sum(mask, 1)
    with tf.variable_scope('rnn_{}'.format(0)):
        h_outs, h_final = rnn_encoder_single_layer(rnn_type, x, seq_len, d_dim, initializer)
    if n_layer > 1:
        for i in range(1, n_layer):
            with tf.variable_scope('rnn_{}'.format(i)):
                h_outs, h_final = rnn_encoder_single_layer( rnn_type, h_outs, seq_len, d_dim, initializer)

    if pooling is None:
        h = h_outs
    else:
        if pooling == 'max':
            h = dynamic_max_pooling(h_outs, mask)
        elif pooling == 'last':
            h = h_final

    h_size = d_dim * (int(rnn_type.find('bi') > -1) + 1)
    return h, h_size, h_final

def dynamic_max_pooling(x, mask):
    mask = tf.expand_dims(tf.cast(mask, tf.float32) - 1., 2)
    return tf.reduce_max(x + mask * 999999, axis=1)

def rnn_encoder_single_layer(rnn_type, input, seq_len, d_dim, initializer):
    bi_directional = rnn_type.find('bi') == 0
    with tf.variable_scope('rnn_encoder'):
        batch_size = tf.shape(seq_len)[0]

        cell = {}
        initial_state = {}
        for d in ['forward', 'backward'] if bi_directional else ['forward']:
            with tf.variable_scope(d):
                cell[d] = tf.contrib.rnn.CoupledInputForgetGateLSTMCell(d_dim, forget_bias=1.0, initializer=initializer, state_is_tuple=True)

                i_cell = tf.get_variable(d + 'i_cell', shape=[1, d_dim], dtype=tf.float32, initializer=initializer)
                i_output = tf.get_variable(d + 'i_output', shape=[1, d_dim], dtype=tf.float32, initializer=initializer)

                c_states = tf.tile(i_cell, tf.stack([batch_size, 1]))
                h_states = tf.tile(i_output, tf.stack([batch_size, 1]))
                initial_state[d] = tf.contrib.rnn.LSTMStateTuple(c_states, h_states)

        if bi_directional:
            raw_outputs, (fw, bw) = \
                tf.nn.bidirectional_dynamic_rnn(cell['forward'], cell['backward'], input, dtype=tf.float32, sequence_length=seq_len, initial_state_fw=initial_state['forward'], initial_state_bw=initial_state['backward'])
            raw_outputs = tf.concat(raw_outputs, axis=2)
            final_states = tf.concat([fw.h, bw.h], axis=1)

        else:
            raw_outputs, final_states = \
                tf.nn.dynamic_rnn(cell['forward'], input, dtype=tf.float32, sequence_length=seq_len, initial_state=initial_state['forward'])
            final_states = final_states.h
    return raw_outputs, final_states

def shape_list(x):
    ps = x.get_shape().as_list()
    ts = tf.shape(x)
    return [ts[i] if ps[i] is None else ps[i] for i in range(len(ps))]

def ffnn(inputs, h_dim, out_dim, n_layer, initializer,
         act_mid=tf.nn.tanh, act_last=tf.nn.tanh, use_bias=True,
         scope_name='ffnn'):
    h = inputs
    for i in range(n_layer-1):
        with tf.variable_scope('{}_{}'.format(scope_name, i)):
            w = tf.get_variable('W',
                                shape=[h_dim, h_dim],
                                initializer=initializer)
            if len(inputs.get_shape().as_list()) == 3:
                w = tf.tile(tf.expand_dims(w, axis=0),
                            [tf.shape(inputs)[0], 1, 1])
            h = tf.matmul(h, w)
            if use_bias:
                b = tf.get_variable('b',
                                    shape=[h_dim],
                                    initializer=tf.zeros_initializer())
                h = h + b
            if act_mid is not None:
                h = act_mid(h)

    with tf.variable_scope('{}_{}'.format(scope_name, n_layer-1)):
        W = tf.get_variable('W',
                            shape=[h_dim, out_dim],
                            initializer=initializer)
        if len(inputs.get_shape().as_list()) == 3:
            W = tf.tile(tf.expand_dims(W, axis=0), [tf.shape(inputs)[0], 1, 1])
        y_raw = tf.matmul(h, W)
        if use_bias:
            b = tf.get_variable('b',
                                shape=[out_dim],
                                initializer=tf.zeros_initializer())
            y_raw = y_raw + b
        if act_last is not None:
            y_raw = act_last(y_raw)
    return y_raw

def create_mixed_trainable_emb(dim, n_ws, n_special_ws, initializer, is_trainable, scope_name):
    """ Reserve index 0 for non-trainable padding, following by
    n_ws pretrained embeddings and n_special_ws trainable embeddings.
    """
    with tf.variable_scope(scope_name):
        pad_e = tf.get_variable(
            "pad_e",
            dtype=tf.float32,
            shape=[1, dim],
            initializer=tf.zeros_initializer(),
            trainable=False)

        e = tf.get_variable(
            "e",
            dtype=tf.float32,
            shape=[n_ws, dim],
            initializer=initializer,
            trainable=is_trainable)

        special_e = tf.get_variable(
            "special_e",
            dtype=tf.float32,
            shape=[n_special_ws, dim],
            initializer=initializer,
            trainable=True)

        mixed_e = tf.concat([pad_e, e, special_e], axis=0)
    return mixed_e, e

def dropout(x, keep_prob, is_train):
    if is_train and keep_prob > 0:
        x = tf.nn.dropout(x, keep_prob)
    return x

class BaseModel:
    def create_holder(self):
        self.x = tf.placeholder(tf.int32, [None, None, None])
        self.x_mask = tf.placeholder(tf.int32, [None, None, None])
        self.xw = tf.placeholder(tf.int32, [None, None])
        self.xw_mask = tf.placeholder(tf.int32, [None, None])

        self.y = tf.placeholder(tf.int32, [None, None])
        self.drp_keep = tf.placeholder(tf.float32)
        self.lr = tf.placeholder(tf.float32)

    def __init__(self, params, session):
        self.initializer = tf.contrib.layers.xavier_initializer()
        self.params = params
        self.session = session
        self.ignored_vars = []

    def build_graph(self):
        pass

    def get_hybrid_emb(self, x, x_mask, xw, ce, we, initializer):
        with tf.variable_scope('hybrid_emb'):
            dims = shape_list(x)
            n_w, cpw = dims[-2], dims[-1]
            dims_1 = reduce(mul, dims[:-1], 1)
            dims_2 = reduce(mul, dims[:-2], 1)

            x_flat = tf.reshape(x, (dims_1, cpw))
            x_mask_flat = tf.reshape(x_mask, (dims_1, cpw))
            x_rep = tf.nn.embedding_lookup(ce, x_flat)

            with tf.variable_scope('c_encoder'):
                xc_rep, xc_size, _ = rnn_encoder( self.params['c_rnn_type'], x_rep, x_mask_flat, self.params['c_h_dim'], self.params['c_rnn_layers'], initializer, self.params['c_pooling'])
                xc_rep = tf.reshape(xc_rep, [dims_2, n_w, xc_size])

            xw_fat = tf.reshape(xw, (dims_2, n_w))
            xw_rep = tf.nn.embedding_lookup(we, xw_fat)
            w_rep = tf.concat([xc_rep, xw_rep], axis=-1)
            hw_size = self.params['we_dim'] + xc_size
        return w_rep, hw_size

    def encode(self, x, x_mask, xw, xw_mask,
               drp_keep, initializer, reuse):
        with tf.variable_scope('encoder', reuse=reuse):
            n_ce = max(self.params['c2id'].values()) + 1
            ce, _ = create_mixed_trainable_emb( self.params['ce_dim'], n_ce - len(RESERVE_TKS), len(RESERVE_TKS) - 1, initializer, True, 'ce')

            n_we = max(self.params['w2id'].values()) + 1
            assert n_we == len(self.params['w2id'])
            we, we_core = create_mixed_trainable_emb(
                        self.params['we_dim'],
                        n_we - len(RESERVE_TKS),
                        len(RESERVE_TKS) - 1,
                        initializer,
                        self.params['we_trainable'],
                        'we'
                        )

            self.we = we
            self.we_core = we_core
            self.ignored_vars.append(we_core)

            with tf.variable_scope('main_text'):
                w_rep, _ = self.get_hybrid_emb(x, x_mask, xw, ce, we, initializer)
                w_rep = tf.nn.dropout(w_rep, drp_keep)

            with tf.variable_scope('w_encoder'):
                if self.params['w_encoder'] == 'rnn':
                    hw, hw_size, h_last = rnn_encoder(self.params['w_rnn_type'], w_rep, xw_mask, self.params['w_h_dim'], self.params['w_rnn_layers'], initializer, None)
        return hw, hw_size, h_last

    def hw_pooling(self, hw, mask, hw_size, out_size, use_l2, initializer,
                   scope_name, reuse):
        with tf.variable_scope(scope_name, reuse=reuse):
            h = dynamic_max_pooling(hw, mask)

            with tf.variable_scope('ff0', reuse=reuse):
                h = ffnn(
                        h, hw_size, out_size,
                        1, initializer)
            if use_l2:
                h_ = tf.nn.l2_normalize(h, -1)
            else:
                h_ = h
        return h_, h

    def fill_token_idx(self, words_batch, b_size, mlen, cpw):
        x = np.ones((b_size, mlen, cpw)) * self.params['c2id'][PAD]
        x_mask = np.zeros((b_size, mlen, cpw))
        xw = np.ones((b_size, mlen)) * self.params['w2id'][PAD]
        xw_mask = np.zeros((b_size, mlen))

        for i, words in enumerate(words_batch):
            c_ids = np.ones((mlen, cpw)) * self.params['c2id'][PAD]
            c_mask = np.zeros((mlen, cpw))
            for j, w in enumerate([[START_S]] + words[:mlen-2] + [[END_S]]):
                tmp = [self.params['c2id'][c]if c in self.params['c2id'] else self.params['c2id'][UNK]
                       for c in [START_W] + list(w)[:self.params['max_c_per_w']]+[END_W]]
                c_ids[j, :len(tmp)] = tmp
                c_mask[j, :len(tmp)] = 1
                if w[0] in RESERVE_TKS:
                    w_ = w[0]
                elif self.params['we_is_lw']:
                    w_ = w.lower()
                else:
                    w_ = w
                if w_ in self.params['w2id']:
                    xw[i, j] = self.params['w2id'][w_]
                else:
                    if self.params['try_lw_emb'] and w_.lower() in self.params['w2id']:
                        xw[i, j] = self.params['w2id'][w_.lower()]
                    else:
                        xw[i, j] = self.params['w2id'][UNK]
            x[i] = c_ids
            x_mask[i] = c_mask
            xw_mask[i, :len(words) + 2] = 1
        return x, x_mask, xw, xw_mask

class NameEncoder(BaseModel):
    def __init__(self, params, session):
        super().__init__(params, session)
        self.cf = params['tasks']['diff_name']
        self.params = params
        self.build_graph()

    def build_graph(self):
        self.create_holder()
        hw, hw_size, h_last = self.encode(self.x,
                                    self.x_mask,
                                    self.xw,
                                    self.xw_mask,
                                    self.drp_keep,
                                    self.initializer,
                                    reuse=tf.AUTO_REUSE)
        h, h_raw = self.hw_pooling(hw,
                                   self.xw_mask,
                                   hw_size,
                                   self.params['w_h_dim'],
                                   self.cf['use_l2_norm'],
                                   self.initializer,
                                   scope_name='diff_name_classifier',
                                   reuse=tf.AUTO_REUSE)
        self.h = h

    def get_fd_data(self, data_batch):
        max_len = max([len(row) for row in data_batch]) + 2
        b_size = len(data_batch)
        chars_per_word = self.params['max_c_per_w'] + 2
        x, x_mask, xw, xw_mask = self.fill_token_idx(data_batch, b_size, max_len, chars_per_word)
        data_dict = {
            self.x: x,
            self.x_mask: x_mask,
            self.xw: xw,
            self.xw_mask: xw_mask,
            self.drp_keep: 1.,
            }
        return data_dict

I want to convert this implementation to pytorch source code. Is there any automated way to do this? Or is there any mapping from tensorflow methods to pytorch methods?
Additionally, there are also model checkpoints which are loaded into tensorflow of this sort - here is the link - BNE/models/BNE_SGsc at master · minhcp/BNE · GitHub 1
How do I convert these tensorflow model weights to pytorch weights? Is there any software to do that?
Thank you in advance for your help,
Megh

Context: Hi, I suggest to post this in the pytorch community forum.


Question:
i’m doing image captioning using this model
def define_model(vocab_size, max_length):

    inputs1 = Input(shape=1120)
    fe1 = Dropout(0.3)(inputs1)
    fe2 = Dense(512, kernel_regularizer=regularizers.l2(1e-4), activation='relu')(fe1)
    inputs2 = Input(shape=(max_length,))
    se1 = Embedding(vocab_size, 256, mask_zero=True)(inputs2)
    se2 = Dropout(0.3)(se1)
    se2=BatchNormalization()(se2)
    se3 = LSTM(512)(se2)
    decoder1 = Concatenate()([fe2, se3])
    decoder2 = Dense(512, activation='relu')(decoder1)
    outputs = Dense(vocab_size, activation='softmax')(decoder2)
    model = Model(inputs=[inputs1, inputs2], outputs=outputs)
    opt =tf.keras.optimizers.Adam(learning_rate=1e-4)
    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
    model.summary()
    return model

i got results
epoch 1     loss: 5.3386 - accuracy: 0.1759 - val_loss: 4.9454 - val_accuracy: 0.2214
epoch 2     loss: 4.2776 - accuracy: 0.2832 - val_loss: 3.9343 - val_accuracy: 0.3105
epoch 3     loss: 3.5354 - accuracy: 0.3599 - val_loss: 3.8278 - val_accuracy: 0.3210
epoch 4     loss: 3.2257 - accuracy: 0.4039 - val_loss: 3.9480 - val_accuracy: 0.3101
epoch 5     loss: 3.0297 - accuracy: 0.4326 - val_loss: 4.1156 - val_accuracy: 0.3072
epoch 6     loss: 2.9005 - accuracy: 0.4505 - val_loss: 4.2219 - val_accuracy: 0.3053
epoch 7     loss: 2.8103 - accuracy: 0.4622 - val_loss: 4.2751 - val_accuracy: 0.3020
epoch 8     loss: nan - accuracy: 0.1060 - val_loss: nan - val_accuracy: 0.0000e+00

callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3)
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
history = model.fit(train_generator, epochs=50, steps_per_epoch=train_steps, verbose=1, callbacks=checkpoint, validation_data=val_generator, validation_steps=val_steps,batch_size=64)

Context:
Sorry, I don’t have an answer for that, but…
It looks like a exploding gradients problem
I have two suggestions for you:

take a look on this Image captioning with visual attention  |  TensorFlow Core 2 for some inspiration
There’s this SO answer that might also help you: python - NaN loss when training regression network - Stack Overflow 6

hope it helps


Which models would you like to see on TensorFlow Hub?

Question:
TFHub have +1000 models but given the fast development of ML, there’s always a cool new one that you might want to try.
Which one are you missing?

Context:
Can we add more audio tasks? More specifically in music projects (pitch detection, chord detection, source separation).
Example of a super recent model (published today) that I’m excited about: https://tfhub.dev/google/spice/2 18


About the method of finding the farthest distance of the color block

Question: I have a small problem in my project about how to find the distance between the longest two points and the shortest two points in an irregular block of color on a picture, rest assured that a straight line can pass through the area without a block of color, and I would like to ask if you have any ideas

Context: Do you have an example image?


Tensorflow build and train model with C++ API

Question:
Hi dear developers! This is my first day of using tensorflow and i am really new to this library. Today i successfully compile the library with vcpkg (with the following command: vcpkg install tensorflow-cc:x64-windows-static)., and now i want to learn how to make an object detection model with C++ tensorflow, train that model and use it in my project? My goal is to take a bunch of pictures with faces (with face mask) and (without face mask) and make a trained model and after i want to make a face mask detection with openCV and tensoflow.
Could anybody give me a hand with that task and let to know from wich point to start? Where to find C++ code examples and is there any GUI for constructing building and training tensorflow models?

Context:
Just to give you an example of a pipeline take a look at:

      PyImageSearch – 4 May 20

COVID-19: Face Mask Detector with OpenCV, Keras/TensorFlow, and Deep Learning... 32

  In this tutorial, you will learn how to train a COVID-19 face mask detector with OpenCV, Keras/TensorFlow, and Deep Learning.

    Est. reading time: 33 minutes

I don’t know why you need to use C++ but generally you will train in python then you could use your model with TF directly or on different targets  exporting on TFLite (mobile, embedded) , TFMicro (microcontrollers), TFJS (browser).


Question:
My question is much the same as : Can I get the gradient of a tensor with respect to the input without applying the input? 1
Is there a way to get the gradient wrt. to an input and then inserting a new input? Something that would allow us to find phi_z(x,z,t)|z=a.
I have tried forcing a layer to be equal to phi_z(x,z,t), and the evaluating z. But this did not yield great results.
Any help would be appreciated. Thanks.

Context:
stackoverflow.com

  Is there a way to calculate the gradient wrt the input without applying it? 2

  tensorflow, tensorflow2.0, tensorflow-gradient

  asked by

    CarlArblue

  on 09:57PM - 17 Aug 21 UTC

/cc @thea @Joana Just to monitor stackoverflow->forum duplicates


Tensorflow gpu serving without docker on “windows”

Question:
stackoverflow.com

  tensorflow gpu serving without docker on "windows" 10

  python, tensorflow, windows-10, tensorflow-serving

  asked by

    Jerry7

  on 08:23AM - 20 Aug 21 UTC

Context: WSL2 Is stable now so I think the stackoverflow topic is outdated.