Repository: tensorflow
Issue: Multithreading is not working with teansorflow
Labels: stat:awaiting tensorflower,type:bug,TF 2.15

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

tensorflow==2.15.0.post1

### Custom code

Yes

### OS platform and distribution

_No response_

### Mobile device

_No response_

### Python version

python:3.10.12

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

I am using bert model for classification and serving the model with gunicorn worker_class=gthreads, tf.config.threading.set_intra_op_parallelism_threads(1)
tf.config.threading.set_inter_op_parallelism_threads(1)

when I using above two line of code the code is working fine as expected and if increase the number to more than 1, the code getting blocked at the below line of code

# Make predictions
outputs = model_obj(inputs)

and also inorder to reduce the docker image size I am using 
RUN pip3 install torch==2.0.0+cpu -f https://download.pytorch.org/whl/torch_stable.html


before installing all the dependencies
Flask==2.2.5
g2p-en==2.1.0
gunicorn==21.2.0
jellyfish==1.0.3
kenlm==0.2.0
nltk==3.8.1
numpy==1.26.3
pandas==2.2.0
python-dotenv==1.0.1
requests==2.31.0
scikit-learn==1.4.0
semantic-router==0.0.17
semantic-router[fastembed]
sentence-transformers==2.3.1
tensorflow==2.15.0.post1
tensorflow-hub==0.16.0
theano==1.0.5
transformers==4.37.2
Werkzeug==2.2.2


please tell me why is my code is getting blocked if I use more than 1 thread.

### Standalone code to reproduce the issue

```shell
def intent_prediction(self, sentence, thresold_score):
        logger.info(f"Threshold Score: {thresold_score}")
        try:
            model_obj = intent_object_dict[self.model]
        except KeyError:
            logger.error("Model not found in intent_object_dict")
            model_obj = self.load_model()

        if INTENT_MODEL == "cohere":
            score, intent = self.cohere_intent_prediction(sentence, model_obj)
        else:
            score, intent = self.Bert_intent_prediction(
                sentence, model_obj, thresold_score
            )
        return score, intent

def Bert_intent_prediction(self, sentence, model_obj, thresold_score):
        inputs = self.load_BERT_tokenizer(sentence)
        # Make predictions
        outputs = model_obj(inputs)
        # Get predicted class
        probabilities = tf.nn.softmax(outputs.logits, axis=1)
        predicted_class = tf.argmax(probabilities, axis=1).numpy()[0]
        matching_score = probabilities[0][predicted_class].numpy()
        try:
            intent_data = intent_label_dict[self.model]
        except Exception as e:
            logger.error(f"error while getting label: {e}")
            intent_data = self.get_intent_labels()

        if matching_score >= thresold_score:
            logger.info(
                f"Matched Main Intent:\
    {intent_data[predicted_class]},\
    SCORE :{matching_score}"
            )
            logger.info(f"Matched Sentence: {intent_data[predicted_class]}")
        else:
            logger.info(f"Intent not matched, score is {matching_score}")

        intent = intent_data[predicted_class]

        return str(matching_score), intent
```


### Relevant log output

```shell
30-Sep-2024 15:50:42.761|INFO    |__init__|I want my sofa get cleaned|
    __init__.py:171|Enter into PUNC for intent...
30-Sep-2024 15:50:42.761|INFO    |phrase_sim|I want my sofa get cleaned|
    phrase_sim.py:72|Threshold Score: 0.7


after this the code is blocked
```