Creating a local development environment with Docker Compose

Creating a simple and reliable development environment is essential to developer productivity as well as on-boarding new team members. It's far too common for companies to have extremely intricate and fragile development environments. Teams should constantly be improving their local development environments. Small annoyances here and there may not seem like much but remember it impacts every developer using that code base. Let's take a look at one of the most more difficult parts of a developer environment, data stores. We will see how to manage them cross platform utilizing Docker Compose.

What is Docker?

According to their website "Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud." In layman's terms it's basically an image containing a low level OS and all software preconfigured. Essentially a much more lightweight VM image. Multiple docker containers can run in a single OS unlike VMs where each VM runs an OS of its own eating up resources. Containers can be anything along the lines of a database, document store, message broker, search service, an existing application, or anything else you can think of. Our use case is quite simple, we want MySQL and Elasticsearch running in our development environment without the need to manually install them. This also gives us flexibility to have different docker compose files for different applications and have no need to worry about naming conflicts or different versions of a dependency across projects.

Docker Hub

Docker Hub is a repository of shared docker containers you can readily import. We will be using it as the base for our MySQL and Elasticsearch containers.

Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. Each application we have can have it's own docker compose file for a single command boot up of our development environment. Below is our StubbornJava docker compose file. We will take a look at each container later.

version: '2'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.1
    restart: always
    container_name: stubbornjava_elasticsearch
    environment:
      - node.name=stubbornjava
      - cluster.name=stubbornjava-cluster
      - http.port=9200
      - transport.tcp.port=9300
      - discovery.zen.minimum_master_nodes=1
      - discovery.type=single-node
      - http.cors.enabled=true
      - node.data=true
      - node.master=true
      - xpack.security.enabled=false
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - stubbornjava_esdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
  mysql:
    build: mysql
    restart: always
    container_name: stubbornjava_mysql
    environment:
      - MYSQL_ROOT_PASSWORD=
      - MYSQL_ALLOW_EMPTY_PASSWORD=yes
    ports:
      - "3306:3306"
    volumes:
      - stubbornjava_mysql:/var/lib/mysql

volumes:
  stubbornjava_esdata:
    driver: local
  stubbornjava_mysql:
    driver: local

Docker Compose MySQL Container Example

We are starting with a base MySQL docker image from Docker Hub and adding some customizations to the my.cnf file. Mainly we want to support full unicode using utf8mb4 see MySQL 8.0: When to use utf8mb3 over utf8mb4? for more info.

MySQL Dockerfile

Starting with our base docker image we then apply our custom my.cnf file as well as run a script to help configure things such as creating databases or users.

FROM mysql:5.7

COPY mysqld.cnf /etc/mysql/mysql.conf.d/mysqld.cnf
COPY ./setup.sh /docker-entrypoint-initdb.d/setup.sh

EXPOSE 3306

CMD ["mysqld"]

MySQL my.cnf

Our custom mysql config file.

[mysqld]
pid-file	            = /var/run/mysqld/mysqld.pid
socket		            = /var/run/mysqld/mysqld.sock
# Where the database files are stored inside the container
datadir		            = /var/lib/mysql

# My application special configuration
max_allowed_packet          = 32M
sql-mode                    = 'STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION'

# Accept connections from any IP address
bind-address	            = 0.0.0.0

character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci

MySQL Setup script

Create databases and users or anything else needed to bootstrap.

#!/bin/sh

echo 'creating databases'
mysql -u root -e 'CREATE DATABASE IF NOT EXISTS stubbornjava;'

mysql -u root -e 'CREATE DATABASE IF NOT EXISTS sj_access;'

Docker Compose Elasticsearch Contianer Example

Since we are just using the default ES container we don't need a separate Dockerfile here. All of the configuration can be passed in directly from the docker compose file. Take note this is a single server set up which is probably ok for local development or small projects but you will probably want a better setup for non local environments.

Running the dev environment

Simply change directory to where the docker compose file lives and run docker-compose up. You should see logs coming from both containers and you should be up and running.

mysql -u root --host=localhost --protocol=tcp -e 'SHOW DATABASES';
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| stubbornjava       |
| sys                |
+--------------------+

You need to specify protocol=tcp since mysql is running in a container. You can also share a socket file between the volume and local OS.

curl localhost:9200
{
  "name" : "stubbornjava",
  "cluster_name" : "stubbornjava-cluster",
  "cluster_uuid" : "-2ALiVFtTr6iJZkip8KXzA",
  "version" : {
    "number" : "5.6.1",
    "build_hash" : "667b497",
    "build_date" : "2017-09-14T19:22:05.189Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

Both MySQL and Elasticsearch are up and running.