Dockerizing your Python Keras Model

Getting out from this versioning guerrilla with docker

6 min readNov 11, 2019

When dealing with python versioning you can easily get lost with all the possible combinations of your favorite library version, and the versions of the interpreter. Many different projects, specially the ones involving Tensorflow and Keras, require a very specific library version to run. Many other projects are not well designed for retro- or forward-compatibility between python2 and python3. If you are just like me trying to migrate from the good old python2 to the new brilliant python3, you will easily forget that damn 3 at the end of your pip or python commands, and you will quickly end up not knowing anymore which was the correct interpreter version to use for your project.

That’s a nightmare. I’ve also tried to solve this problem with virtual environments, but I quickly started to forget activating them and the problem has risen again. I actually ended up having a model running in my development environment, that was giving a segfault ( a never heard python segfault ) in the production environment. This motivated me a lot to search around how to definitely solve this versioning guerrilla.

The solution

I’ve decided to try the approach of isolating my application inside a container, which will hopefully help me dealing with all this concurrent library and interpreter versions. In fact, while python do not supports having multiple version of the same library installed, it becomes possible to run projects depending on different python versions concurrently thanks to the usage of docker images.

Introduction

I will create a docker image that exclusively uses my model to run predictions and returns back the result. This will permit me to isolate the context of the application running the model and avoid any non-reproducibility issue.

I will not focus on the code used, but instead on the configuration needed to create a container, configure and run it.

Let’s begin

Let’s write down how to use docker to isolate the model dependencies and libraries.

In the following sections you will find:

How to define the container’s content?
Difference RUN and CMD and ENTRYPOINT
How do I update a container image ?
The Flask Development Server is incredibly slow.
Flask and Keras are not best friends.
Create a docker hub and push a new image to it
Docker Containers VS Images VS Repositories
Useful Commands
Conclusions

How to define the container’s content?

I’ve figured out that the Dockerfile defines how the docker container image layout will be set up: you will specify here the main parent image to build on, the commands to run at build time (typically installations of the dependencies specific to your project) and the Volume to mount inside the container ( shared folders between host and container ). An example Dockerfile to run with docker build -t <newcontainer-name>:<chosenversion> <appdir> follows:

FROM python:3.7
WORKDIR /app
ENV MODEL /app/model.hdf5
COPY requirements.txt /app
RUN pip install -r ./requirements.txt
COPY app.py /app
COPY model.hdf5 /app
CMD ["python", "app.py"]~

Dockerfile-s are only used at docker build time, the information of the file that generates a container gets lost once it is built.

The above Dockerfile will be used to create a container called <newcontainer-name> with version <chosenversion> using files taken from <appdir>

The Dockerfile contains the following directives:

FROM: the base image to build on ( this one e.g. contains a simple linux install with python 3.7 already configured ).
WORKDIR: the working directory inside the container filesystem, any added file will be placed in this directory
ENV MODEL /app/model.hdf5: creates an environment variable with the model path
COPY <host-file> <container-dir>: copies the file from the host filesystem <host-file> to the <container-dir> container directory.
CMD [“python”,”app.py”]: the command that will be started when the container has booted.

Difference RUN and CMD and ENTRYPOINT

RUN: adds another layer to the existing image you are building on. Often used to install dependencies and packages
CMD: runs when the container is started, it is ignored if you start the container with an explicity command such as docker run <container> <command> .
ENTRYPOINT: runs when the container is started, it is not ignored if you start the container with an explicit command

How do I update a container image ?

I’m always prone to discover immediately what can go wrong, and I’ve missed a row inside the requirements.txtfile. How do I fix this? Well, as follows:

# Creates a new container, starts it, and also starts 
# an interactive shell inside its fsystem
docker run -t -i <image-name> /bin/bash# Change what you need to change to the filesystem
# for example add remove modify files, install packages etc.# Press the following shortcut to exit the container without
# terminating it (otherwise we would lost filesystem changes)
CTRL-P  CTRL-Q# commit to a new image the changes you made
docker commit <container> <image-name>:<image-version>

We can also change the entry point of our container as follows:

docker commit -c "CMD python app.py" <container> <image-name>:<image-version>

The Flask Development Server is incredibly slow.

The warning that no one considers about not using the Development server is revelatory. The flask development server is incredibly slow in responding requests. Therefore it is better to use the Waitress server:

# instead of app.run() use
from waitress import serve
serve(app,host="0.0.0.0",port=80)

Flask and Keras are not best friends.

Flask will try to run keras on a different thread for each new request. If on your road you will encounter this error: '_thread._local' object has no attribute 'value' , disable Flask threading with: app.run("0.0.0.0",port=80,threaded=False)

This will incredibly disrupt performances. A better approach to solve this issue is to finalize the Tensorflow Graph before starting with multithreading. We will need to :

# first run a dummy prediction.
# it modifies the graph, it is needed
model.predict(0)# now we can finalize the graph
tf.get_default_graph().finalize()

Create a docker hub and push a new image to it

First thing to do is going to docker hub and create an account. Then you need to create a repository, that will be referenced as <username>/<reponame>.

Once you have create your account you need to create a tag to push to the docker hub. To create a new tag to push inside the repository use:

docker tag <image-name>:<image-version> <username>/<reponame>:<tagname>

Now use the docker login command to actually authenticate to docker hub in your shell session and upload your image to docker hub with:

docker push <username>/<reponame>:<tagname>

Fix the error creating overlay mount to <file>: device or resource busy while pushing (remove multithreaded upload that causes race conditions):

$ sudo systemctl stop docker
$ sudo nano /etc/docker/daemon.json
{
  "max-concurrent-uploads": 1
}
$ sudo systemctl start docker
$ docker push [...]

Docker Containers VS Images VS Repositories

With a bottom up approach: A container is an instance of an Image. An Image is the prototype o a container an is pulled from a Repository. A Repository contains different tags ( versions ) of images. A good approach can be to create a different repository for each different project, and to use tags for versioning.

Useful Commands

It follows a list of usefull commands:

# List running containers
docker ps# List all running an stopped containers
docker ps -a# List all docker images locally available
docker images# Remove an image named <image>
docker images rm <image># Create and start a container from <image>
docker run <image># Create and start a container from <image> with name <name>
docker run --name <name> <image> # Create a port mapping from (host)8080 to (container)80
docker run -p 8080:80 <image># Start a new container from <image> with a shell as an entry point (-it flag needed for interactive shell and tty)
docker run -it <image> /bin/bash# Attach stdin and stdout to running container
docker attach <container># Stop a running container
docker stop <container>

Conclusions

Docker containers really help to isolate environments and dependencies. Using them it is possible to create fully independent micro-services that only share REST interfaces, even on a single host machine.