How to configure Docker for consistent development environments, even in C++!

The Problem

A scenario we have all been through, a new developer joins the team. She is using her own laptop with her favorite operating system, be it Windows, Linux, MacOS. On it, some favorite tools, fine-tuned and tweaked over the years. Possibly a remote worker. The tasks are ready; business is impatiently awaiting new features. Our poor dev now grabs some rather large, and possibly outdated, document describing the steps necessary to spin up a development environment.

Let's make things more complicated. While our intrepid engineer is working on a C++ project, in reality, it could be any language. Libraries need to be installed, proper compiler versions present, a myriad of additional tools, frameworks and scripts need to be ready and working flawlessly. A couple of days have passed …

There is a slim chance that everything went fine, hey, this is how we’ve doing things for the last decades. The sad reality, however, is that usually something is messed up. Different versions here and there, scripts behaving differently on Windows or Linux. There is also the small matter of our production environment. Let’s assume that our prod is running CentOS 7 (in real world use-case it will be probably RedHat, but its subscription model would complicate our example).

Some devs build and test using Windows, Mac, Ubuntu or Arch. There is very little chance that anyone is using the same OS and setup as production one. Subtle errors may pop up on our staging environment or maybe even on production. These errors can add up, slowing down the development cycle, increasing the time spent in deciphering and reducing the overall benefit.

Docker can help us improve each step of the way.The canonical use case for Docker, as advertised by the official guides and numerous resources, is to create one image at a developer’s build station and pass it on through testing, staging to production. At Appliscale we are not so fond of running Docker on production. We need to create and maintain very large and scalable systems for which short response times are critical. Anyhow this post is not about why we think that Docker is not good for production, so let’s focus on the topic.

Let’s assume that we have Centos 7 on production that needs to run C++ binary. We want to have the same C++ development and runtime versions, compilers, linkers, dynamic libraries, external libraries and tools on developer, QA, CI, staging and production machines. As you know, a small mismatch in any of the above can have dire consequences.

The Solution

Our goal is to mirror production 100%. The assumption here is that production is managed by professional DevOps/administrators, so all machines are consistently provisioned and kept in good shape. So we have Centos 7 with C++ runtime 6.2.1. Let’s start creating Docker image with such configuration. You can find the source here: git@bitbucket.org:plenza/docker_env.git – let's look at Dockerfile which is very simple:

FROM centos:7

COPY .ssh/* /root/.ssh/

COPY etc /etc

RUN chown root:root /root/.ssh/* && chmod 600 /root/.ssh/*

RUN yum -y upgrade

RUN yum install -y centos-release-scl &&

yum install -y make git devtoolset-6 cmake rpm-build

First, we define our base image which is Centos 7. Then copy the contents of .ssh subdirectory inside image at root user home. To provide your identity, you should copy your ssh files into .ssh dir before building the image.

Next, copy the contents of the etc subdirectory which in our case contains only bare git configuration like username and email. Modify gitconfig file according to your needs.

The set the permissions on ssh files, which is a requirement for them to work on Linux.

And finally, we can install required stuff. As we are using RedHat or Centos I have selected devtoolset version 6 (https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/) which brings fixed version of C++ environment. Our assumption here is that the same version is being used in production. Some additional tools like git, cmake and rpm-build are also installed. And that is all.

We can now build the image (be inside a directory with Dockerfile):

docker build --rm -t cpp_dev_image .

This command will process Dockerfile and run all steps, resulting in image creation. You can run:

docker images -a

to check if cpp_dev_image is present.

So now, our developer (or Jenkins for that matter) is ready to start development/build.

Let’s start the container please:

docker run -v $PWD/project:/project -it cpp_dev_image scl enable devtoolset-6 bash

And analyse what is going on:

-v option maps host project subdirectory into /project inside the container

-it makes run interactive because we want to run shell commands

scl enable devtoolset-6 bash enables our fixed c++ environment for this one bash session

After this command, you should be inside the container. Try running gcc –version, you should see version 6.2.1 due to devtoolset-6.

We are ready to start development. Cd into /project and git clone tiny c++ project:

git clone https://plenza@bitbucket.org/plenza/docker_env_cpp.git

It uses CMake to build and package into RPM. Go into docker_env_cpp/build directory and run:

cmake ..

This will prepare all required build files – Makefile in our case:

-- The C compiler identification is GNU 6.2.1

-- The CXX compiler identification is GNU 6.2.1

-- Check for working C compiler: /opt/rh/devtoolset-6/root/usr/bin/cc

-- Check for working C compiler: /opt/rh/devtoolset-6/root/usr/bin/cc -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Check for working CXX compiler: /opt/rh/devtoolset-6/root/usr/bin/c++

-- Check for working CXX compiler: /opt/rh/devtoolset-6/root/usr/bin/c++ -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Configuring done

-- Generating done

-- Build files have been written to: /project/docker_env_cpp/build

Now running:

make

will create binary output. You can try to run it by executing ./prog and:

make package

will create RPM inside _CPack_Packages directory. Such RPM can be easily taken from Jenkins build and used as install source on production.

So we can build and test but what about everyday development?

To exit our container we just execute exit command. Now if we want to run it again we just need to start it. First, execute:

docker ps -a

to find what name docker assigned to it (you can also provide your own name during run if you prefer). You should see your container with Exited state. So execute:

docker start <name>

and

docker attach <name>

and you are back in your container exactly where you left off.

If you want to attach a second shell then simply execute:

docker exec -it <name> /bin/bash

from a second terminal. Make sure to run:

scl enable devtoolset-6 bash inside it to switch to our preferred versions (remember it is per shell session).

Exiting from this second shell will not stop the container, but exiting from the main one (started with start-attach) will stop all connections.

It really shouldn’t take more than 5-10 minutes to spin up this development environment. Starting, attaching and running new shells takes less than a second.This demonstrates the power and lightness of Docker.

At this moment, however, this is just the bare minimum. To develop new features the simplest way would be to enter the container and use your favorite text editors like Vim or Emacs. They are not installed in Centos by default. You can either use yum to install them in you local container or modify Dockerfile if you want them to be globally accessible across the whole team.

Another possible way would be to use some IDE from our host machine and modify sources that are mapped to the project directory (on our host). The only drawback is that Docker will map project files as owned by root, so our IDE needs to run as root as well. It can create some problems and is not recommended, but if you are careful, it can be a valid way of developing.

What's Next – CLion over X11 sockets anyone?

My preferred way, however, is to install IDE inside Docker container. This IDE can be CLion for example. I know it works on Linux because we can easily map X11 socket to our server running on the host. I will add a separate article about that setup. It should be possible on Mac by using some VNC remote desktop and on Windows by running X-Server ...that's next on my TODO list. Having such installation is almost transparent for us, we don’t have to be concerned about root files, and nothing is being messed up on our host.

**What we have learned **

To sum this up – it takes very little to provide all team members with a consistent development environment. The same steps can be invoked on CI, Jenkins for example. It is also very easy to provide a consistent and predictable build that produces well-tested artefacts that are delivered to production. Having consistent environment shortens the feedback loop, can eliminate errors that usually pop on production and keeps all the team in sync. Reasons enough for us to use it, hopefully, it has convinced you!