clips-ocr 0.1.0
An OCR package to read text in clips
Requires Python: >=3.5

# CRNN_Tensorflow

This is a TensorFlow implementation of a Deep Neural Network for scene 
text recognition.

Cloned from original repository [MaybeShewill-CV/CRNN_Tensorflow](https://github.com/MaybeShewill-CV/CRNN_Tensorflow).

The model consists of a CNN stage extracting features which are fed to an RNN stage (Bi-LSTM) and a CTC loss.

## Updates

**[UPDATE 08/03/2021]:**  
**A python package was created to expose prediction functionality of the OCR model.** Refer [OCR Package Section](#ocrpackage) of this documentation or [full documentation](ocr_package/README.md) available at the [package folder](ocr_pacakage).

**[Update 16/03/2021]:**
**Added functionality to write train and prediction details to unified db.**  
Refer [Unified DB Update](#unidbup) and [Naming Convention](#nameconv) below for further details.  

## Directory Structure <a name="dirs"></a>

There are 9 directories in the repository.

**Directories**

 Directory Name | Description
 ------------|------------
 `config`    | Contains a custom python module to save global configurations
 `crnn_model` | Contains custom python modules used to create the CRNN model structure
 `data` | Contains `char_dict.json` and `ord_map.json` inside the subfolder `char_dict`
 `data_provider` | Contains custom python modules used to read & write tfrecords
 `docs` | Contains additional documentation files like `ocr_model_spec` and `ocr_workflow`.
 `local_utils` | Contains custom python modules with utilities functions like creating char dict files and providing evaluation tools
 `ocr_package` | Contains the `clips_ocr` package
 `tfserve` | Contains custom python modules and shell scripts that can be used to serve the CRNN model using Tensorflow model server
 `tools` | Contains tools that are used in analysis etc.

## Files <a name="files"></a>

This section will be focused on files located in the root directory of the repository.

**Files:**

 File Name | Description
 ----------|------------
 `build_model_crnn.py` | Python script to build .pb models from checkpoint files. For further info, refer [Building Model (checkpoint to .pb)](#modelbuild) section.
 `conda_config.yml` | Configuration file that should be used to create a conda environment with all necessary dependencies installed in a **Linux** system. For further info, refer [Installing Dependencies - Linux](#depslin) section.
 `conda_config_win.yml` | Configuration file that should be used to create a conda environment with all necessary dependencies installed in a **Windows** system. For further info, refer [Installing Dependencies - Windows](#depswin) section.
 `evaluate_shadownet.py` | Python script used to evaluate the CRNN model. For further info, refer [Evaluating the Model](#evaluate) section.
 `test_shadownet.py` | Python script used to test the CRNN model with a single image.
 `train_shadownet.py` | Python script used to train the CRNN model with created tfrecords. For further info, refer [Training the Model](#train) section.
 `write_tfrecords.py` | Python script used to write tfrecords from given image dataset. For further info, refer [Data Preparation](#tfrecords) section.
 `README.md` | Documentation file
 `LICENSE` | License file

## Table of Contents  

1. [Installing Dependencies](#installdeps)
    - [1.1 Linux](#depslin)
    - [1.2 Windows](#depswin)
2. [Data Preparation](#tfrecords)
    - [2.1 Arguments](#tfrecordargs)
    - [2.2 Dataset Architecture](#tfrecordarch)
    - [2.3 Create Image Dataset from Clips Predictions](#createimdset)
3. [Training the Model](#train)
    - [3.1 Arguments](#trainargs)
4. [Evaluating the Model](#evaluate)
    - [4.1 Arguments](#evaluateargs)
5. [Building Model (checkpoint to .pb)](#modelbuild)
    - [5.1 Arguments](#modelbuildargs)
6. [Analysis Tools](#antools)
    - [Class-wise Confusion Matrices](#confmat)
    - [Resize Images](#imgrez)
7. [Serving Model[WIP]](#serve)
8. [Configurations](#config)
    - [8.1 Architecture](#configarch)
    - [8.2 Train](#configtrain)
    - [8.3 Test](#configtest)
9. [OCR Package](#ocrpackage)
    - [9.1 Installation](#pkginstall)
    - [9.2 Import](#pkgimport)
    - [9.3 Initialize Model](#pkginit)
    - [9.4 Make Predictions](#pkgpreds)
    - [9.5 Close Session](#pkgclosesess)
    - [9.6 Generate Package](#pkgen)
10. [Unified DB Update](#unidbup)
    - [10.1 Installing `unified_db_update`](#unidbupins)
    - [10.2 Update Training Details](#unidbupt)
    - [10.3 Prediction Details](#unidbupp)
11. [Naming Convention](#nameconv)
    - [11.1 Train](#nameconvt)
    - [11.2 Predictions](#nameconvp)

## 1. Installing Dependencies <a name="installdeps"></a>

This software has been developed on Ubuntu 16.04(x64) using python 3.5 and TensorFlow 1.12. Since it uses some recent features of TensorFlow it is incompatible with older versions.

### 1.1 Linux <a name="depslin"></a>

You can create a conda environment with the required dependencies in a Linux system by running the following command:

```bash
conda env create -f conda_config.yml
```

Then activate the created Conda environment by running the following command:

```bash
conda activate crnntf
```

### 1.2 Windows <a name="depswin"></a>

You can create a conda environment with the required dependencies in a Windows system by running the following command:

```bash
conda env create -f conda_config_win.yml
```

Then activate the created Conda environment by running the following command:

```bash
conda activate crnntf
```

If you're not familiar with Conda, kindly check their official documentation [here](https://docs.conda.io/en/latest/).

## 2. Data Preparation <a name="tfrecords"></a>

In data preparation, Tfrecords are created using the dataset images.  

Syntax:

```bash
python write_tfrecords.py 
    --dataset_dir PATH/TO/DATASET/ROOT/DIR
    --save_dir PATH/TO/TFRECORDS/DIR
```

### 2.1 Arguments <a name="tfrecordargs"></a>

| Argument | Description | Notes |
|----------|-------------|-------|
| `--dataset_dir` / `-d` | Path to dataset root directory | The directory must contain two folders for `Train` and `Test` sets. Refer to [dataset architecture](#tfrecordarch) below for further info. |
| `--save_dir` / `-s` | Path to the directory where you want tfrecords written into | Make sure you save tfrecords to a directory named `tfrecords`. |
| `--char_dict_path` / `-c` | File path to save char dictionary | **Not mandatory**. If no path is provided, the dictionary will be saved to the directory `../tf_records/data/char_dict`. |
| `--ord_map_dict_path` / `-o` | File path to save ord map dictionary | **Not mandatory**. If no path is provided, the dictionary will be saved to the directory `..tf_records/data/char_dict`. |
| `--multiprocessing` / `-m` | Whether to use multiprocessing to write tfrecords. Might not work in Windows. | Takes in boolean values: (yes/no), (true, false), (y/n), (t/f), (1/0). Defaulted to false. |  

- `char_dict.json` and `ord_map_dict.json` are dictionary files which are used for encoding and decoding label values.

### 2.2 Dataset Architecture <a name="tfrecordarch"></a>

The dataset root directory should have two folders for Test and Train sets named `Train_Data` and `Test_Data` accordingly. Inside those folders should be dataset images and the annotation file named `data.txt`.  

The annotation file should contain records in the format of: *image_name* < tab > *label*  

For an example, *1234_56789.jpg*    *crnn*

### 2.3 Create Image Dataset from Clips Predictions <a name="createimdset"></a>

Images dataset from Clips model predictions can be created by running the script @ `nameplate_mak_rcnn/nameplate_mask_rcnn/unified_model/code_files/crnn_train_data/crnn_ocr_train_data.py`.

## 3. Training the Model <a name="train"></a>

For details about all available training parameters like number of steps and batch sizes, refer [Configurations](#config).  

The model can be trained with single GPU or multiple GPUs. Training can be done either from scratch or can be continued from pre-trained weights.

Syntax:

```bash
python train_shadownet.py
    --dataset_dir PATH/TO/TFRECORDS/FOLDER
    --char_dict_path PATH/TO/CHARACTER/DICTIONARY
    --ord_map_dict_path PATH/TO/ORD/MAP/DICTIONARY
    --database_path PATH/TO/LOCAL/DATABASE
    --model_output_path PATH/WHERE/CHECKPOINT/FILES/SHOULD/BE/WRITTEN/INTO
```

### 3.1 Arguments <a name="trainargs"></a>

| Argument | Description | Notes |
|----------|-------------|-------|
| `--dataset_dir` / `-d` | Path to the directory containing the `tfrecords` folder which has tfrecords for Train and Validation sets | The directory should contain a folder named `tfrecords`. |
| `--weights_path` / `-w` | Path to pre-trained weights to continue training | **Not Mandatory**. If no path is provided, training will start from scratch. |
| `--char_dict_path` / `-c` | Path to the character dictionary file for the dataset | |
| `--ord_map_dict_path` / `-o` | Path to the ord map dictionary file for the dataset | |
| `--decode_outputs` / `-e` | Activate decoding of predictions during training | Takes in boolean values: (yes/no), (true, false), (y/n), (t/f), (1/0). Defaulted to false. Setting this to True will **slow down** the training process. |
| `--multi_gpus` / `-m` | Whether to use multi GPUs to train the model | Takes in boolean values: (yes/no), (true, false), (y/n), (t/f), (1/0). Defaulted to false. This can be set to True if there are multiple GPUs installed to the computer the script is being run |
|`--database_path` / `-db` | Path to the database where training details will be written into | Make sure you enter a path to a file with the extension of `.db`. |
| `--model_output_path` / `-op` | Path to the directory where model checkpoint files will be written | A folder will be created inside the location provided for each training instance and checkpoint files will be written into that folder. This folder and checkpoint files will be named according to the [naming convention](#nameconvt).

Weights will be saved once every 5000 steps. Saved weights can be found in the directory you specified for `--model_output_path` in arguments.

## 4. Evaluating the Model <a name="evaluate"></a>

Trained model weights can be evaluated using the test set.

Syntax:

```bash
python evaluate_shadownet.py
    --dataset_dir PATH/TO/TFRECORDS/DIRECTORY
    --weights_path PATH/TO/MODEL/WEIGHTS
    --char_dict_path PATH/TO/CHAR/DICTIONARY
    --ord_map_dict_path PATH/TO/ORD/MAP/DICTIONARY
    --process_all y
    --prediction_data_path PATH/WHERE/PREDICTION/RESULTS/SHOULD/BE/WRITTEN/INTO
    --ckpt_num CHECKPOINT/NUMBER
    --database_path PATH/TO/THE/LOCAL/DATABASE
    --logic_version LOGIC/VERSION/USED
    --info_code INFO/CODE/OF/THE/DATASET
```

### 4.1 Arguments <a name="evaluateargs"></a>

| Argument | Description | Notes |
|----------|-------------|-------|
| `--dataset_dir` / `-d` | Path to the directory containing the `tfrecords` folder which has tfrecords for Test sets | The directory should contain a folder named `tfrecords`. |
| `--char_dict_path` / `-c` | Path to the character dictionary file for the dataset | |
| `--ord_map_dict_path` / `-o` | Path to the ord map dictionary file for the dataset | |
| `--weights_path` / `-w` | Path to pre-trained model weights | |
| `--visualize` / `-v` | Whether to display images used for evaluation | Takes in boolean values: (yes/no), (true, false), (y/n), (t/f), (1/0). Defaulted to false. |
| `--process_all` / `-p` | Whether to process the whole Test set | Takes in boolean values: (yes/no), (true, false), (y/n), (t/f), (1/0). Defaulted to false. Set this value to True in order to evaluate model weights using the whole Test set. Otherwise, the evaluation will be done using only one batch of Test dataset. |
| `--prediction_data_path` / `-f` | Path to a directory where the prediction results csv will be written into | Image names, labels and their predictions will be written to the csv file. It will be located in a folder created inside the path provided here. Both folder and csv  file will be named according to the [naming convention](#nameconvp). |
| `--ckpt_num` / `-cnum` | Number indicating the step of checkpoint file used for taking predictions | For instance, if you use the chekpoint file saved in the 80000th step, you need to set this value to 80000.
| `--database_path` / `-db` | Path to the database to which prediction details will be written into | Make sure you enter a path to a file with the extension of `.db`. |
| `--logic_version` / `-lv` | Logic version of the current script | |
| `--info_code` / `-ic` | Information code of the dataset used for training the model | |

 - Image name, labels and predictions will be displayed in the Terminal / Command Prompt as every image is evaluated. Images will be displayed if `--visualize` argument is set to True.
 - Image names, labels and their predictions will be written to the csv file as every image is evaluated.
 - After the whole process, **Mean test per character accuracy** and **Mean test full sequence accuracy** will be displayed.
 - Then the normalized confusion matrix will be displayed in Terminal / Command Prompt and visualized.
 - **Mean test per character accuracy** and **Mean test full sequence accuracy** will be written into the unified db and local database given.

## 5. Building Model (checkpoint to .pb) <a name="modelbuild"></a>

Tensorflow saved models can be created using checkpoint files.

Syntax:

```bash
python build_model_crnn.py
    --export_dir PATH/TO/THE/FOLDER/WHERE/MODEL/WILL/BE/SAVED/TO
    --ckpt_path PATH/TO/CHECKPOINT/FILE
    --char_dict_path PATH/TO/CHAR/DICTIONARY
    --ord_map_dict_path PATH/TO/ORD/MAP/DICTIONARY
```

#### 5.1 Arguments <a name="modelbuildargs"></a>

| Argument | Description | Notes |
|----------|-------------|-------|
| `--export_dir` / `-ex` | Path to the folder where model will be saved to | Make sure the given folder is empty or not existing(**only last folder on the path**) |
| `--ckpt_path` / `-ck` | Path to the checkpoint file that need to be converted | Make sure to provide a valid checkpoint path |
| `--char_dict_path` / `-c` | Path to the character dictionary file for the dataset | Make sure you enter the path to a valid .json file |
| `--ord_map_dict_path` / `-o` | Path to the ord map dictionary file for the dataset | Make sure you enter the path to a valid .json file |

## 6. Analysis Tools - WORK IN PROGRESS <a name="antools"></a>

### 6.1 Class-wise Confusion Matrices <a name="confmat"></a>

### 6.2 Resize Images <a name="imgrez"></a>

## 7. Serving Model [Work In Progress - Not tested yet] <a name="serve"></a>

Tensorflow model server is a very powerful tool to serve the model in an industry environment. There are scripts in the repository to convert the checkpoint weights into tensorflow saved model which can be used with tensorflow model server to serve the CRNN model. If you can not run the script normally you may need to check if the checkpoint file path is correct in the bash script.

```bash
bash tfserve/export_crnn_saved_model.sh
```

To start the tensorflow model server you may check following script

```bash
bash tfserve/run_tfserve_crnn_gpu.sh
```

There are two different ways to test the python client of crnn model. First you may
test the server via http/rest request by running

```bash
python tfserve/crnn_python_client_via_request.py ./data/test_images/test_01.jpg
```

Second you may test the server via grpc by running

```bash
python tfserve/crnn_python_client_via_grpc.py
```

## 8. Configurations <a name="config"></a>

Only main configurations will be mentioned here. For full list of configs, refer [config/global_config.py](config/global_config.py). 

- All changes should be done to the `global_config.py` file mentioned above in order for them to take effect.

### 8.1 Architecture <a name="configarch"></a>

- Architecture configurations are located in `ARCH` dictionary inside the configuration dictionary. Defines **CRNN model configurations**.

| Configuration | Description | Notes |
|---------------|-------------|-------|
| HIDDEN_UNITS | Number of units in each LSTM cell | Increase or decrease this number to make the model more advanced or simple accordingly |
| HIDDEN_LAYERS| Number of stacked LSTM cells | Increase or decrease this number to make the model more advanced or simple accordingly |
| INPUT_SIZE | (width, height) into which all input images are resized | This is also used in defining input layer of the Neural Network |
| INPUT_CHANNELS | Number of channels in input images | Set this number to 1 if input images are grayscale, 3 if input images are RGB |
| NUM_CLASSES | Number of character classes | Number of distinct characters in labels of Train, Test and Validation sets. If unsure about this number, refer [data/char_dict_char_dict.json](data/char_dict/char_dict.json) and see the count of characters |

### 8.2 Train <a name="configtrain"></a>

- Architecture configurations are located in `TRAIN` dictionary inside the configuration dictionary. Defines **Train function configurations**.

| Configuration | Description | Notes |
|---------------|-------------|-------|
| EARLY_STOPPING | Whether to use early stopping | For more early stopping configurations, refer `global_config.py` |
| EPOCHS | Number of **STEPS** the model should train for | Though it is called 'EPOCHS', this defines the number of steps |
| DISPLAY_STEP | Set the display step | Setting this to 1 would display every step that the cript is going through |
| CHECKPOINT_FREQ | Checkpoint saving frequency during the training process | For instance, if 5000 was provided for this number, a checkpoint file will be saved for every 5000 training steps the model go through |
| LEARNING_RATE | Initial learning rate | To see learning rate decay configs, refer `global_config.py` |
| GPU_MEMORY_FRACTION | Set the fraction of GPU usage during the training process | |
| BATCH_SIZE | Training set batch size | |
| VAL_BATCH_SIZE | Validation set batch size | |
| GPU_NUM | Number of GPU | Set this to the number of GPUs while training with multiple GPUs |
| VAL_DISPLAY_STEP | Set validation display steps | Setting this to 1 would display validation data for every step that the cript is going through |

### 8.3 Test <a name="configtest"></a>

- Architecture configurations are located in `TEST` dictionary inside the configuration dictionary. Defines **Test function configurations**.

| Configuration | Description |
|---------------|-------------|
| GPU_MEMORY_FRACTION | Set the fraction of GPU usage during the testing process |
| TF_ALLOW_GROWTH | Set the GPU allow growth parameter during testing process |
| BATCH_SIZE | Testing set batch size |

## 9. OCR Package <a name="ocrpackage"></a>

A python package was created to expose the predict functionality of the OCR model. The package was created using scripts of this repository. 

- Originally created as a tool to evaluate the label detection model.

- The package is located in the [ocr_package](ocr_package) folder.

This section is a general introduction to the package. For more in-depth details and examples, refer [package documentation](ocr_package/README.md).  

This is a simple guide to install and use the latest version of OCR packages. For details about older versions, refer package documentation.

### 9.1 Installation <a name="pkginstall"></a>

Package can be installed on any python or Conda environment using `pip` and `.whl` file located in [dist](ocr_package/dist) folder. 

Provided that you execute it in root directory of this repository, the following command should install the latest version of the package:

```bash
pip install ocr_package/dist/clips_ocr-0.1.0-py3-none-any.whl
```

### 9.2 Import <a name="pkgimport"></a>

`OcrModel` should be imported from the package as shown below:

```python
from clips_ocr import OcrModel
```

`OcrModel` contains every function needed for getting OCR predictions.

### 9.3 Initialize Model <a name="pkginit"></a>

In order to get predictions, first you need to initialize an `OcrModel` object which was imported above. 3 **mandatory** parameters should be provided in the initialization process.

#### Parameters

| Parameter | Description |
|-----------|-------------|
| model_path | Path to the folder containing `saved_model.pb` file and `variables` folder. For futher info on converting checkpoint files to Tensorflow SavedModel files, refer [Building Model](#modelbuild) section. |
| char_dict_path | Path to the character dictionary file for the dataset. |
| ord_map_dict_path | Path to the ord map dictionary file for the dataset. |

#### Example

```python
ocr_model = OcrModel(weights_path="/path/to/weights/file",
                    char_dict_path="/path/to/char/dict",
                    ord_map_dict_path="/path/to/ord/map/dict")
```

### 9.4 Make Predictions <a name="pkgpreds"></a>

Predictions can be taken by simply calling `predict` function of the created `OcrModel` object. **The image you want to get predictions of should be passed as a parameter to the function.** The image should be loaded with opencv.

- Returns the prediction as a String.

#### Example

```python
prediction = ocr_model.predict(image="image_loaded_with_openCV")
```

### 9.5 Close Session <a name="pkgclosesess"></a>

This function will close the session and free the resources used for predictions. Simply invoking the function is sufficient.

```python
ocr_model.close_session()
```

### 9.6 Generate Package <a name="pkgen"></a>

Run the following command to generate the package. Make sure you're running this package from the base directory of the package.

```bash
python setup.py bdist_wheel
```

## 10. Unified DB Update <a name="unidbup"></a>

Updates to Unified DB and other databases are done using `unified_db_update` package which can be found [here](https://gitlab.com/verico-research/nameplate_mask_rcnn/-/tree/dev/packages/Unified_DB_Update/unified_db_update).

### 10.1 Installing `unified_db_update` <a name="unidbupins"></a>

If you're running training and prediction processes in a conda environment, you need to install the `unified_db_update` package.  

In order to install the package, you have to follow the following procedure.

1. Clone [nameplate_mask_rcnn](https://gitlab.com/verico-research/nameplate_mask_rcnn) repository. You can do this by executing the following command on your terminal/cmd.

```bash
git clone https://gitlab.com/verico-research/nameplate_mask_rcnn
```

2. Navigate into the repository directory and switch to `dev` branch.

```bash
cd nameplate_mask_rcnn
git checkout dev
```

3. Navigate into the directory containing `.whl` file.

```bash
cd packages/Unified_DB_Update/unified_db_update/dist
```

4. Install the package using `pip`. If you have not activated your conda environment which you use to run train and prediction scripts, activate it beforehand.

```bash
pip install uni_db_update-0.0.1-py3-none-any.whl
```

- For further details, refer packages documentation [here](https://gitlab.com/verico-research/nameplate_mask_rcnn/-/blob/dev/packages/python_packages.md).

### 10.2 Update Training Details <a name="unidbupt"></a>

Training details will be written into a table named `training_details` in the Unified DB and other databases you provide in training process.

A record consists of 8 columns. Following is a brief explanation for each.

Column Name | Description
------------|------------
id | id of the record.
model_type | Model identifier. Model type for OCR model would be `CRNN_OCR_model`.
training_datetime | Date and the time when training process was initiated.
user_arguments | List of user arguments and their values given to the training script.
model_config | Configurations used for the training process. For the OCR model, the dictionary from `config/global_config.py` will be written into this column.
logfile_path | Path to the directory where checkpoint files will be written into.
description | Other details about the model. For OCR model, this column will consist of `model checkpoints file path`, `number of train steps`, `tensorboard file path` `whether multi gpu were used or not` and `file paths to char_dict.json and ord_map.json`
model_number | Model number. Ex: 1000, 1001, 1002 etc.

### 10.3 Prediction Details <a name="unidbupp"></a>

Prediction details will be written into a table named `model_pred_desc` in the Unified DB and other databases you provide in training process.

A record consists of 5 columns. Following is a brief explanation for each.

Column Name | Description
------------|------------
TYPE | Prediction type; for OCR it would be `CRNN_OCR_Model`.
DATE_TIME | Date and the time when prediction process was initiated.
USER_ARGUMENTS | List of user arguments and their values given to the prediction script.
DESCRIPTION | Details about the model. For OCR model, this column will consist of `average per character accuracy`, `average full sequence accuracy`, `prediction details csv file path`, `file paths to char_dict.json and ord_map.json`.
MODEL_NAME | Name of the model checkpoint file used for taking predictions.

## 11. Naming Convention <a name="nameconv"></a>

The naming convention for the OCR model will be as follows.

### 11.1 Train <a name="nameconvt"> </a>

In the training process, checkpoint files will be saved to a directory inside the location that was provided in arguments under `model_output_path`. The directory and checkpoint files will follow standard naming conventions. 

For the directory, the format would be **crnn_ocr_`<model_number>`_`<train_start_datetime>`**.

 - For an example, **crnn_ocr_1004_20210315T1253**

For checkpoints, the format would be **crnn_ocr_`<train_start_datetime>`.ckpt-`<step_number>`**

 - For an example, the 80000th step would be saved as **crnn_ocr_20210315T1253.ckpt-80000**

 As per these examples, the location of checkpoint will be `model_output_path/crnn_ocr_1004_20210315T1253/crnn_ocr_20210315T1253.ckpt-80000`

### 11.2 Predictions <a name="nameconvp"> </a>

In prediction process, prediction result csv files will be saved to a directory inside the location that was provided in arguments under `prediction_data_path`. The directory and csv file will follow standard naming conventions.

For the directory, the format would be **`<name_of_checkpoint_file_used_for_prediction>` _ckpt`<checkpoint_number_used_for_prediction>`_`<logic_version>`_`<info_code>`**

- For an example, **crnn_ocr_20210315T1253.ckpt-80000_ckpt80000_lv1_IO01** for predictions taken from 80000th step checkpoint file.

For the csv file, the format would be **ocr_predictions_ckpt_`<checkpoint_number>`_D_`<date_time>`.csv**

- For an example, **ocr_predictions_ckpt_80000_D_20210315T1313.csv**

As per these examples, the location of csv will be `prediction_data_path/crnn_ocr_20210315T1253.ckpt-80000_ckpt80000_lv1_IO01/ocr_predictions_ckpt_80000_D_20210315T1313.csv`