Python Flask: Interacting with Docker Containers
Tags: python, docker, tutorial
Published: 2022-10-30 18:16:45

> Python3 Flask: Interacts with Docker Containers

Last Friday, I was talking with my co-worker at my work. Just about work, you know. He told me he had to implement a server for serving his features. Although I didn't understand what he was going to do exactly, it sounded really interesting and I wanted to try. I got really busy at my company a couple of weeks ago, I've had to write a bunch of HTML.  It wasn't technically interesting(I love my work but the work was almost the same in this release). I kind of got tired of that. I want to do something new. While I was feeling like this, I had a small talk with the coworker.

He was going to do

* Make an API server that gets an image file
* Execute a docker container and pass the image file to the container and the container will create the text file that is generated from the image file 
* Execute another docker container and pass the text
* The API Server knows that the container has done with its work and notify to another server(The backend that servers the data to users).

It sounds interesting, ha?, I decided to implement the system on the weekend. I thought it's going to be fun. But it's been really a long time since I used Python and Docker. So, I first had to learn these. During the weekends, I couldn't learn all the things, so that, I would use the minimum skills for implementing the system. If you're a python developer or working with the docker, it might be not in your mind at some points. Please, comment your opinion comment below. I'd love that.

Anyways, Let's get started!

---

### Prerequisite

- pipenv

> Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.
  When I first use python a few years ago, I used conda or venv for managing packages. When I searched for package management, I saw this. It seems like `npm`, I didn't know there is like this tool. I thought it must be worth it to try.

- docker

> Docker is a platform designed to help developers build, share, and run modern applications. We handle the tedious setup, so you can focus on the code.
  As I introduced at first, this is the main in this post.

- Flask

> Flask is a web application framework written in Python. It was developed by Armin Ronacher, who led a team of international Python enthusiasts called Poocco.
  To implement a simple API web server, I used `Flask`. there were other libraries like Fast API. Since I used to Flask a little, I went with Flask For reducing my hours. 

---

![Process](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/73tct0lfmd05nu29uaq0.png)

This is the process that I'm going to implement.

1. The API server receives a file from the user

2. Container A processes A file then saves it to the B File

3. Container B processes the B File then saves it to the C File

4. Users can see the result through API

---

### Python Apps and Docker Images

I made two images and uploaded to my them repository in docker-hub.

A tokenizer and word-counting app.

[Tokenizer]
```python
import sys, json, os, requests
from textblob import TextBlob


def extract_nouns(text):
	blob = TextBlob(text)
	filtered_tags = list(filter(lambda tag: tag[1] == "NN", blob.tags))
	nouns = list(map(lambda tag: tag[0], filtered_tags))
	return nouns


def read_file(path):
	with open(path) as f:
		contents = f.read()
	return contents


def save_data(path, data):
	with open(path, "w") as f:
		json.dump(data, f)


def get_filename_from_path(path):
	return os.path.splitext(os.path.basename(path))[0]


def notify_done(url, file_name):
	requests.get(f"{url}/docker/tokenizer_done?file_name={file_name}")


if __name__ == "__main__":
	if len(sys.argv) < 4:
		print("You must pass file path as an argument")
		print("python3 main.py [file path to read] [dir to save] [notification api]")
		print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")
		sys.exit()
	
	api_url = sys.argv[3]
	file_path = sys.argv[1]
	file_name = get_filename_from_path(file_path)
	target_path = os.path.join(sys.argv[2], file_name + ".json") 

	text = read_file(file_path)
	nouns = extract_nouns(text)

	save_data(target_path, {"nouns": nouns})
	notify_done(api_url, file_name)
 
	print("Done")
```

[word-counting]
```python
import sys, json, os, requests


def count_word(nouns_list):
	count_dict = dict()

	for noun in nouns_list:
		if noun in count_dict:
			count_dict[noun] += 1
		else:
			count_dict[noun] = 1

	return count_dict


def load_data(path):
	with open(path) as f:
		json_data = json.load(f)
	return json_data


def save_data(path, data):
	with open(path, "w") as f:
		json.dump(data, f)


def get_filename_from_path(path):
	return os.path.splitext(os.path.basename(path))[0]


def notify_done(url, file_name):
	requests.get(f"{url}/docker/word_count_done?file_name={file_name}")


if __name__ == "__main__":
	if len(sys.argv) < 4:
		print("You must pass file path as an argument")
		print("python3 main.py [file path to read] [dir to save] [notification api]")
		print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")
		sys.exit()
	
	api_url = sys.argv[3]
	file_path = sys.argv[1]
	file_name = get_filename_from_path(file_path)
	target_path = os.path.join(sys.argv[2], file_name + ".json") 

	json_data = load_data(file_path)
	count_dict = count_word(json_data["nouns"])

	save_data(target_path, {"result": count_dict})
	notify_done(api_url, file_name)
	print("Done")
```

For running the apps from the API server, I built both python files with below Dockerfiles.

[Tokenizer]
```python
FROM python:3.9

WORKDIR /app
COPY . .

RUN pip install pipenv
RUN pipenv install
RUN pipenv run python3 -m textblob.download_corpora

ENTRYPOINT ["pipenv", "run", "python3", "./main.py"]
```

[word-counting]
```python
FROM python:3.9

WORKDIR /app
COPY . .

RUN pip install pipenv
RUN pipenv install

ENTRYPOINT ["pipenv", "run", "python3", "./main.py"]
```

---

### API Server

This is the main code and it's kind of simple.

```python
from flask import Flask
from dotenv import load_dotenv

load_dotenv()

app = Flask(__name__)

import routes
```

#### Routes

[routes/docker.py]
```python
import os
from flask import jsonify, request
from server import app
from lib import docker, json


result = []


@app.route('/docker/tokenizer_done')
def get_tokenizer_done():
    file_name = request.args.get("file_name")
    docker.run_word_count_container(file_name)
    return "run a word_count container"
    

@app.route('/docker/word_count_done')
def get_word_count_done():
    file_name = request.args.get("file_name")

    json_data = json.load_data(
        os.path.join(os.getenv("SHARED_VOLUME_PATH"),
        "word_count_output",
        f"{file_name}.json"
    ))
    result.append(json_data)

    return "all works done"


@app.route('/docker/result')
def get_result():
    file_name = request.args.get("file_name")
    return jsonify({
        "result": result
    })
```

[routes/upload.py]
```python
import os
from flask import jsonify, request
from werkzeug.utils import secure_filename
from server import app
from lib import docker


@app.route("/upload", methods=["POST"])
def upload_file():
    f = request.files["file"]

    file_name = secure_filename(f.filename)
    f.save(os.path.join(os.getenv("SHARED_VOLUME_PATH"), "input", file_name))
    
    docker.run_tokenizer_container(file_name)
    
    return "succeed to upload"
```

[routes/\_\_init\_\_.py]
```python
from routes import docker, upload
```

[lib/docker.py]
```python
import os

API_URL = os.getenv("API_URL")
VOLUME_ROOT_PATH = os.getenv("SHARED_VOLUME_PATH")
RUN_TOKENIZER_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/tokenizer:0.2 /shared_volume/input/{FILE_NAME_WITH_EXTENSION} /shared_volume/tokenizer_output ' + API_URL
RUN_WORD_COUNT_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/word_count:0.2 /shared_volume/tokenizer_output/{FILE_NAME_WITHOUT_EXTENSION}.json /shared_volume/word_count_output ' + API_URL


def run_tokenizer_container(file_name):
    print(RUN_TOKENIZER_CONTAINER.format(
        FILE_NAME_WITH_EXTENSION = file_name
    ))
    os.popen(RUN_TOKENIZER_CONTAINER.format(
        FILE_NAME_WITH_EXTENSION = file_name
    ))



def run_word_count_container(file_name):
    os.popen(RUN_WORD_COUNT_CONTAINER.format(
        FILE_NAME_WITHOUT_EXTENSION = file_name
    ))
```

[iib/json.py]
```python
import json


def load_data(path):
	with open(path) as f:
		json_data = json.load(f)
	return json_data
```

This app read environment variables from `.env` file, so you need to set up like this.
The below variables just fit my system.

```
API_URL=http://host.docker.internal:20000
ROOT_PATH=C:\Users\hskco\OneDrive\바탕 화면\stuff\docker\api
SHARED_VOLUME_PATH=C:\Users\hskco\OneDrive\바탕 화면\stuff\docker\api\shared_volume
```

---

You can run the server with this script

```
python3 -m pipenv run flask run -h 0.0.0.0 --port 20000
```

Before running this command, you need to set an environment variable `FLASK_APP`.
Since I was developing in Windows, I ran this command in `api` dir.

```
$env:FLASK_APP = './server.py'
```

If you enter `http://127.0.0.1/docker/result` you will see this page.


![Empty Result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xwk837jwcuuli7hjam2f.png)


Let's send a file to the API server and see the result.

[1]
![step one](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tuk4yopko9recwh43v9.png)

[2]
![step two](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4jdyeozl7uqdwkz7rrg5.png)

[3]
![result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7kdpj362db2ez3pfsad.png)

---

### Conclusion

It was really fun. I've learned a lot of things.
Regardless of your position, I think It would be really good to try anything you're interested in.

This example is a really beginner level though, I mean.
It should've considered like

* Authorization and Security (In this example, the API server exposes all routes to the public)
* Communication between containers (I adopted RestfulAPI for interacting between containers, however there must be better ways)
* Managing containers (When containers that have done, they need to be deleted. And, load balancing must be also needed. There are tools for this like Kubernetes)
* Deployment for production

there are many things that I didn't mention and you need to consider. (I respect for backend developers) I was just focusing on implementing the system, If I tried to make it perfect, I couldn't write this article. Haha,, I have to go back to work tomorrow...

Anyways, it was worth it, it's definitely true :) 
I hope it'll be helpful for someone.

---

### References

##### Github Source Code

[Github Source Code](https://github.com/hsk-kr/docker-python-run-test)

##### Python
- https://realpython.com/python-modules-packages/
- https://www.geeksforgeeks.org/reading-and-writing-json-to-a-file-in-python/

##### Pipenv

- https://realpython.com/pipenv-guide/

##### Flask 

- https://auth0.com/blog/developing-restful-apis-with-python-and-flask/

##### Docker
- https://www.atlassian.com/microservices/microservices-architecture/kubernetes-vs-docker
- https://blog.payara.fish/do-you-need-kubernetes
- https://docs.docker.com/build/building/multi-stage/
- https://textblob.readthedocs.io/en/dev/quickstart.html#noun-phrase-extraction
- https://www.baeldung.com/ops/dockerfile-run-cmd-entrypoint
- https://www.howtogeek.com/devops/how-to-connect-to-localhost-within-a-docker-container/