keyboard

Hugo Soltys

Symfony developer

Since 2013

A step into microservices with Docker, Træfik and Git submodules

Posted on by Hugo - 1558 views - 0 Comments


These last years, microservices based applications has really grown in popularity, due to its high scalability. 

In this article, I will show you a way to set up a microservices based local environment with Træfik, Docker Compose and Git submodules.


NB : For this tutorial, we will assume that you are developping an online store.

Every online store has elements that are systematically found. A product catalog, a cart, a checkout ... So instead of developping a single block with all in it, why not divide it to smaller pieces that you may reuse in another project ?

That's the spirit of the microservices architecture. In our case, we will create a Git repository for each pieces of our online store and make them communicate and work together thanks to Docker.

 

CREATING THE REPOSITORIES

First of all, go to your preferred development platform (GitHub, GitLab, Bitbucket ...) and create the following repositories :

- store

- catalog

- checkouts

- users

Once created, open a terminal and clone the store repository. It will be our main repository that will manage the others.

$ git clone git@preferredplatform.org:username/store.git

 

This is where the power of Git submodules comes into play. We will declare each other projects as submodules. Just do the following.

$ cd store
$ git submodule add git@preferredplatform.org:username/catalog.git
$ git submodule add git@preferredplatform.org:username/checkouts.git
$ git submodule add git@preferredplatform.org:username/users.git

 

You have now 3 subdirectories called catalogcheckouts and users in your store directory. Each of these directories are a git repository. 

With a submodules based project, when you do a git pull in the main project (aka store in our case), you will not update your submodules by the same time. But don't panic, there is a magical command that allow you to do it : git submodule foreach. This command requires you to provide the command you want to execute between quotes like the following.

$ git submodule foreach 'git pull'

 

By doing this, a git pull will be done on each of your registered submodules.

 

DOCKERISE YOUR PROJECT

Now we are done with Git, it's time to bring Docker in your project. 

NB : This part requires you to install Docker and Docker Compose

Start by creating a docker-compose.yml file in your store repository. In this file, we will put a Traefik instance. Traefik is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. And lucky for us, it works pretty well with Docker.

So here is our docker-compose.yml file :

version: '2.3'

services:

    traefik:
        image: traefik:latest
        init: true
        networks:
            store:
                aliases:
                    - traefik
        ports:
            - '80:80'
            - '8080:8080'
        volumes:
            - ./src/traefik/rootfs/etc/traefik:/etc/traefik
            - /var/run/docker.sock:/var/run/docker.sock:ro

networks:
    store:
        external:
            name: store

 

Note that we declared a network called store which will allow our containers to communicate with each others.

Let's now assume that your submodules are Symfony projects. So you will need at least a web server and a PHP install. So, create in your 3 submodules a docker-compose.yml file like the following :

 

Catalog docker-compose

version: '2.3'

services:

    catalog-php:
        image: php:7.1
        environment:
            APP_ENV: prod
        networks:
            catalog:
                aliases:
                    - php
            store:
                aliases:
                    - catalog-php

    catalog-nginx:
        build: 
            context: .
            dockerfile: ./docker/nginx/Dockerfile
        networks:
            catalog:
                aliases:
                    - nginx
            store:
                aliases:
                    - catalog
        labels:
            - 'traefik.enable=true'
            - 'traefik.frontend.rule=Host:catalog.local'
            - 'traefik.network=store'

networks:
    store:
        external:
            name: store
    catalog: ~

 

Checkouts docker-compose

version: '2.3'

services:

    checkouts-php:
        image: php:7.1
        environment:
            APP_ENV: prod
        networks:
            checkouts:
                aliases:
                    - php
            store:
                aliases:
                    - checkouts-php

    checkouts-nginx:
        build: 
            context: .
            dockerfile: ./docker/nginx/Dockerfile
        networks:
            checkouts:
                aliases:
                    - nginx
            store:
                aliases:
                    - checkouts
        labels:
            - 'traefik.enable=true'
            - 'traefik.frontend.rule=Host:checkouts.local'
            - 'traefik.network=store'

networks:
    store:
        external:
            name: store
    checkouts: ~

 

Users docker-compose

version: '2.3'

services:

    users-php:
        image: php:7.1
        environment:
            APP_ENV: prod
        networks:
            users:
                aliases:
                    - php
            store:
                aliases:
                    - users-php

    users-nginx:
        build: 
            context: .
            dockerfile: ./docker/nginx/Dockerfile
        networks:
            users:
                aliases:
                    - nginx
            store:
                aliases:
                    - users
        labels:
            - 'traefik.enable=true'
            - 'traefik.frontend.rule=Host:users.local'
            - 'traefik.network=store'

networks:
    store:
        external:
            name: store
    users: ~

 

If you are attentive, you saw that our Nginx have a build part based on a Dockerfile. This will allow us to configure it to work with Symfony.

I chose Nginx, but you are free to use your preferred web server.

So, create a docker/nginx directory in each of your three submodules. In this new directory, create two new files. A Dockerfile, and a default.conf.

 

Here is the Dockerfile

FROM nginx:stable-alpine

ADD default.conf /etc/nginx/conf.d/default.conf

 

And the default.conf

NB : This default.conf file is written to work with Symfony 4

server {
     resolver 127.0.0.11;
     root /srv/app/public;

     location / {
         try_files $uri @rewriteapp;
     }

     location @rewriteapp {
         rewrite ^(.*)$ /index.php/$1 last;
     }

     location ~ ^/index\.php(/|$) {
         fastcgi_pass php:9000;
         fastcgi_split_path_info ^(.+\.php)(/.*)$;
         include fastcgi_params;
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         fastcgi_param HTTPS off;
     }
}

 

And that's it for the Docker part !

You can now launch each submodule separately by launching docker-compose up in the wanted submodule.

But if you want to lauch all your submodules at the same time and communicate between them, you will need to launch Træfik too.

 

A MAKEFILE TO RULE THEM ALL

I propose you to use Makefiles to manage your containers and launch or stop them at will. Make is a build automation tool that builds executable programs by reading files called Makefiles. Start by creating a Makefile in the store project.

In this Makefile, you will find 6 commands : init, install, start, stop, destroy and init-submodules.

 

The store Makefile

init: init-submodules start install

install:
	git submodule foreach 'make install'

start:
	docker network create store || true
	git submodule foreach 'make start'
	docker-compose up -d

stop:
	git submodule foreach 'make stop'
	docker-compose down

destroy:
	make stop
	git submodule foreach 'make destroy'
	docker network rm store || true
	docker-compose rm

init-submodules:
	git submodule init
	git submodule update --recursive --init
	git submodule foreach 'git checkout master'

 

The init function is only to be launched once at the beginning of the project. It will initialize the Git submodules, create the Docker network "store" then start all the submodules containers.

Create also a Makefile in each of your submodules. They will be almost identical that the "store" one, but without the init-submodules command.

 

The catalog, checkouts and users Makefiles

init: start install

install:
	docker-compose exec [catalog|checkouts|users]-php /srv/app/docker/init-dev.sh
	make init-host

start:
	docker-compose up -d

stop:
	docker-compose down

destroy:
	make stop
	docker-compose rm

init-host:
	./init-host.sh

 

Let's take a look at those Makefiles :

init : this command will launch the start then the install command

install : this one will execute the requirements needed to run the project properly. In this case, we will execute a shell script called init-dev.sh then call the init-host command

stop : this command is here to stop the Docker container at will

- destroy :  running this command will execute the stop command then will destroy the Docker container 

init-host : will launch a init-host.sh shell script that will register the wanted local hostname in your /etc/hosts file

 

In our case, we are making Symfony projects, so in our init-dev.sh we will put commands like composer install or cache:clear.

This script must be located in our docker/nginx directory.

 

The init-dev.sh script

#!/usr/bin/env sh
set -e
set +x

START=$(date +%s)
PROGNAME=$0
DIR="$( cd "$( dirname "$PROGNAME" )"/../ && pwd )"

if [ ! -z "${GITHUB_TOKEN}" ]; then
   echo "Define GitHub token"
   composer config --global github-oauth.github.com ${GITHUB_TOKEN}
fi

if [ ! -d "/srv/app/vendor" ]; then
  composer install --no-interaction
fi

if [ "${APP_ENV}" == "dev" ]; then
    php ${DIR}/bin/console c:c
fi

echo "Init done"

 

On the other hand, the init-host.sh script will read your docker-compose.yml file searching for the Træfik label traefik.frontend.rule to add the defined hostname in your /etc/hosts file.

This script must be located in our docker/nginx directory too.

 

The init-host.sh script

#!/usr/bin/env bash

if [ $EUID != 0 ]; then
    sudo "$0" "$@"
    exit $?
fi

TRAEFIK_HOSTLINE=`grep 'traefik.frontend.rule=Host:' ./docker-compose.yml`

IFS=':' read -ra HOST_ARR <<< "$TRAEFIK_HOSTLINE"
HOSTNAME="${HOST_ARR[1]%?}"

EXISTS_HOSTNAME=`grep ${HOSTNAME} /etc/hosts`

if [ -z "$EXISTS_HOSTNAME" ]
then
      echo "127.0.0.1    $HOSTNAME" >> /etc/hosts
else
      echo "$HOSTNAME aleady defined"
fi

 

DEMONSTRATION

Our store project is now fully configured. Open now a terminal, go to your store directory and launch the following command.

$ make init
$ make start

 

Once done, you can access to the Træfik dashboard by calling http://localhost:8080. You will see here all your started containers in one single look. You can access each part of your store individually by accessing http://checkouts.local/, http://users.local, http://catalog.local.

You can now begin your developments with a fully working microservices based application !

Thanks for reading, leave your questions in the comment section.

Hugo, @Decathlon.

 


Hugo Soltys

My name is Hugo, I'm 25 and I'm a Symfony developer since 2013. I love to create websites by myself to learn new technologies or increase my skills.
I love movies, books, music and video games. I also like to drink a few beers with my friends. I'm from Lille (France) and I currently work as Symfony developer at Decathlon since 2016. Before that, I worked as Symfony developer for the IT Room company, in Roubaix, France.


Older articles