How to Dockerize Python Applications With Miniconda [A Hybrid Approach]

If you are familiar with Docker, you probably know that you can create custom docker image using Dockerfile. I have written about in details.

How to Create Custom Docker Image With Dockerfile [very Easy]

The real power of Docker lies in tweaking the base docker image to your requirements. With that, you can easily and quickly deploy a custom Linux environment by running a container from that custom docker image.

Linux HandbookAvimanyu Bandyopadhyay

In this tutorial, I will show you how to follow the same idea but for Python applications alone. This is going to be useful for both users and developers.

I’ll be using a minimal Python image. Once you've modified and revised that image, you'll no longer need to worry about installing your Python Application on different operating systems. You'll be able to straightaway run your Python application with Docker every single time. So, you can bid goodbye to those host-based installation hiccups!

Creating a docker image for your Python application

I am going to use Miniconda here. Miniconda is a free minimal installer for conda and gives you a small, bootstrap version of Anaconda with just the bare-bones necessities you need to run Python applications.

Why Miniconda?

There is more than one reason:

When you install Miniconda on a host, you are not actually using the Python version provided by the operating system's package manager. Miniconda installs on a separate location having its own Python environment. So, this provides you an added level of isolation when done on a Docker container.

Due to the above point, you also get another advantage: Since you are using conda that Miniconda installed, you can use this tool to change the relevant Python version of your application as and when needed. This is a huge help for developers of applications that are based on say different versions of Python 3: They could be 3.6, 3.7, 3.8, 3.9 or earlier versions too.

For example, if by default you're running Python 3.9 but your Python application requires Python 3.7 because of relevant dependencies, what would you do?

This is where conda can help you. With it, you can run conda install python=3.7 to change the Python version required by installing it with all dependencies necessary.

• Miniconda allows you to install both Python 2 and Python 3 applications. Though Python 2 is officially dead, you can still test older applications on top of this environment without needing to audit your new Python 3 port with 2to3.
• There are also many use cases where a Python application running on Miniconda seeks non-Python host-side dependencies (for example g++). This is when the combined power of Miniconda and Docker becomes a great solution!
• Did I forget to mention that you can also create and activate your own Python Application environments with conda? Isolation again!
• At any time, you always have the option to switch between the Docker container's default Python version and that of Miniconda. This gives you more flexibility as you can always rebuild a new image with the change whenever you want.

So, let us now go ahead with creating the new Python Application image with Miniconda and Docker!

Prerequisite

If you have not already, please install Docker on Ubuntu or whichever Linux distribution you are using. Make sure to add yourself to the docker group so that you can run docker without sudo. You’ll need an active internet connection for downloading the base docker image.

For a sample Python application, I am using a simple "Hello World!" example named python-app.pyto make it easier for you to understand how to run it via Miniconda on Docker.

A full-fledged Python application that uses different Python libraries would greatly benefit from the same procedure, especially because of Miniconda's various dependency provisions.

Step 1: Get Docker image [optional]

I chose Python Slim in this example instead of Alpine Linux. The latter is really small but can greatly affect performance when it comes to running applications. However, Python Slim is around 40 MB in size, based on Debian Buster and Python 3.9.1.

This step is optional. I included it to show that you can compare it with the customized python application image like in the previous Dockerfile tutorial.

Pull the latest docker image of Python Slim using the docker pull command:

docker pull python:slim

Step 2: Create Dockerfile with the needed customization

Now let’s create a new empty file named Dockerfile using the touch command. But first create a directory for your docker app.

mkdir python-docker
cd python-docker
touch Dockerfile

I will enclose the complete Dockerfile below after I've finished explaining the steps within the image building process via a step by step walk-through of the file.

Here's how to begin and progress building the image:

Prepare the base image

Update the latest default packages by using Python Slim as the base image.

FROM python:slim
RUN apt-get update && apt-get -y upgrade \

Install non-Python package dependencies

Install any non-Python dependencies for your Python app(say g++ and any other as per your requirement).

  && apt-get install -y --no-install-recommends \
    git \
    wget \
    g++ \
    gcc \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

Git and Wget can be very handy when fetching Python Applications from different repositories and URLs. Finally, clean up some space with rm -rf /var/lib/apt/lists/* for minimizing the final Docker image size.

Install Miniconda

After it's installed, Miniconda updates .bashrc to switch to its own Python version.

ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
    && mkdir /root/.conda \
    && bash Miniconda3-latest-Linux-x86_64.sh -b \
    && rm -f Miniconda3-latest-Linux-x86_64.sh \

Here an environment variable is set within the container system path. ENV is meant for the containers that you'll be running on the image and ARG is meant for the intermediate containers that are created while it's built for the first time.

So, the difference between the ENV and ARG instructions in the above code block is that the latter is only available while the image is being built. Check out this beautiful explanation here.

With wget, download the latest version of Miniconda from the official Anaconda repository. After creating the essential configuration directory, install it and finally remove the installer.

Configure Miniconda to the Bash Shell

After installing Miniconda, display its version number for confirmation and initialize it to the Bash shell for the container. The second line updates your default .bashrc file:

	&& echo "Running $(conda --version)" && \
    conda init bash && \

Reload Bash with the new changes

Reload Bash for the Docker build system to switch to the Miniconda Python version rather than Debian's(the base OS image).

	. /root/.bashrc && \

Also, update the current Miniconda packages bundled by default.

	conda update conda && \

Prepare a Conda environment for your app

Create and activate a separate Conda environment for your Python Application.

    conda create -n python-app && \
    conda activate python-app && \

Install the relevant Python version you need for your app. Assuming your application is based on Python 3.6, set this version within the new virtual environment alongwith Pip, which also comes very handy when managing Python applications.

    conda install python=3.6 pip && \

Install your Python Application

Depending upon how you use your app, you can either:

i. Install it with pip that conventionally uses the setup.py file available in your repository. It is the same tool discussed earlier but here I'm using it via Conda instead.

	git clone replace-me-with-repo-url
	cd repo-name
	pip install -e .

ii. ..or directly run it with the python command:

	git clone replace-me-with-repo-url
	cd repo-name
	python python-app.py

In the demo Dockerfile, I'm going use the "Hello World!" example to make it easier for you to understand how you can run it directly by launching a container or inside its bash shell with Docker. So, let's say I'm using the 2nd way:

    echo 'print("Hello World!")' > python-app.py

So now that I've included the above line, a file called python-app.py is created that is supposed to generate the Hello World message whenever you execute it with the command python python-app.py.

Update the .bashrc file for your app like Miniconda does:

The Miniconda installer automatically updates the .bashrc file after you run conda init bash as shown earlier. You can do the same for your Python application as well. Whenever you run the container bash, the environment will be activated, and you can also use your Python application name as a command to run it. Here I've used the name as python-app:

RUN echo 'conda activate python-app \n\
alias python-app="python python-app.py"' >> /root/.bashrc

Prepare the app for final execution

Finally, I create an entrypoint and assign the command that will enable you to run it every-time you run a container based on this image:

ENTRYPOINT [ "/bin/bash", "-l", "-c" ]
CMD ["python python-app.py"]

Complete Dockerfile

You can use an editor like Vim or Nano or use the cat command to add the above discussed lines to the Dockerfile.

FROM python:slim
RUN apt-get update && apt-get -y upgrade \
  && apt-get install -y --no-install-recommends \
    git \
    wget \
    g++ \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*
ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
    && mkdir /root/.conda \
    && bash Miniconda3-latest-Linux-x86_64.sh -b \
    && rm -f Miniconda3-latest-Linux-x86_64.sh \
    && echo "Running $(conda --version)" && \
    conda init bash && \
    . /root/.bashrc && \
    conda update conda && \
    conda create -n python-app && \
    conda activate python-app && \
    conda install python=3.6 pip && \
    echo 'print("Hello World!")' > python-app.py
RUN echo 'conda activate python-app \n\
alias python-app="python python-app.py"' >> /root/.bashrc
ENTRYPOINT [ "/bin/bash", "-l", "-c" ]
CMD ["python python-app.py"]

When you try your own app, replace the echo 'print("Hello World!")' > python-app.py line above with any of the two ways described in the Install your Python Application section above.

Step 3: Build the Python Application image with the Dockerfile

As you may already know, the command to build the modified Docker image from the Dockerfile looks like:

docker build -t python-app PATH_to_Dockerfile

With the -t tag, you specify the name of your app's Docker image. I've set it as python-app in the above example command.

Considering that the Dockerfile is in your current directory, you can create the new Docker image of your Python Application like this:

docker build -t python-app .

avimanyu@iborg-desktop:~/python-docker$ docker build -t python-app .
Sending build context to Docker daemon   2.56kB
Step 1/8 : FROM python:slim
 ---> 677f7ac99e48
Step 2/8 : RUN apt-get update && apt-get -y upgrade   && apt-get install -y --no-install-recommends     git     wget     g++     ca-certificates     && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 15ee9c47c83b
Step 3/8 : ENV PATH="/root/miniconda3/bin:${PATH}"
 ---> Using cache
 ---> cfd5ed6b5ec9
Step 4/8 : ARG PATH="/root/miniconda3/bin:${PATH}"
 ---> Using cache
 ---> e70d06b5ff10
Step 5/8 : RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh     && mkdir /root/.conda     && bash Miniconda3-latest-Linux-x86_64.sh -b     && rm -f Miniconda3-latest-Linux-x86_64.sh     && echo "Running $(conda --version)" &&     conda init bash &&     . /root/.bashrc &&     conda update conda &&     conda create -n python-app &&     conda activate python-app &&     conda install python=3.6 pip &&     echo 'print("Hello World!")' > python-app.py
 ---> Using cache
 ---> 8a7957a6abb2
Step 6/8 : RUN echo 'conda activate python-app \nalias python-app="python python-app.py"' >> /root/.bashrc
 ---> Running in e3193e93b631
Removing intermediate container e3193e93b631
 ---> 948f45eb6024
Step 7/8 : ENTRYPOINT [ "/bin/bash", "-l", "-c" ]
 ---> Running in 621624951dcf
Removing intermediate container 621624951dcf
 ---> 6e8824889502
Step 8/8 : CMD ["python python-app.py"]
 ---> Running in dc97f9d0d8fe
Removing intermediate container dc97f9d0d8fe
 ---> 01bae0a9903c
Successfully built 01bae0a9903c
Successfully tagged python-app:latest

The above output is based on cached data but when you run it for the first time, it would take some time and produce a much longer log output.

Now, let’s verify that your modified Docker image has the example app installed by running a container from it:

docker run python-app

avimanyu@iborg-desktop:~/python-docker$ docker run python-app
Hello World!

So, through Docker and Miniconda, you are now able to run the program directly without any prior installation needed! From now onwards, all you need is the image.

Let us now login to the bash shell inside this container:

docker run -ti python-app bash

avimanyu@iborg-desktop:~/python-docker$ docker run -ti python-app bash
(python-app) root@2ceec4c9eaa4:/# 

As you can see, you are now inside the Conda activated environment you created earlier through the Dockerfile. The -ti flag is used to create an interactive terminal for you. You can now alternatively use the command you aliased to run the app:

(python-app) root@2ceec4c9eaa4:/# python-app
Hello World!

Let's also confirm that you are indeed using the Miniconda Python version and not the default Python version:

(python-app) root@2ceec4c9eaa4:/# python --version
Python 3.6.12 :: Anaconda, Inc.

As I had mentioned earlier, Miniconda is a miniaturized version of Anaconda.

Once you have everything set, you can push your final image to Docker Hub if you host an Open Source Python application on GitHub, GitLab, Gitea, Bitbucket or any other repository.

Exit the container by typing exit in the terminal. Stop the container, remove the container and remove the Docker images (if you want to) to free up disk space.

Congrats! You just learned how to create your very own Docker image for your Python application.

Was it helpful to you?

So you see, not only does Miniconda help you make your application more flexible and future proof at the user level, it also makes the developer's role much easier.

Think of how convenient it would be for setting this up with PyCharm! You install Miniconda and your Python Application like you do on a host system, but since you build and save it as a Docker image, it becomes a one-time process only!

If you want, you can experiment with the different examples shown in this previous tutorial instead of the "Hello World!" example that I've used in this one.

Hope this helps! If you have questions or suggestions, please leave a comment below.