If you consider moving your development to use dockers or you are looking to find how to do it in a harmonic way both for development and delivery to production this is for you.
Motivation
Since I started playing with dockers several years ago and tried to develop with it on all levels I’ve faced many things like how to reuse and share the dockers of my local environment to prevent setups of 1–2 days again and again. That was my first question.
Another thing was when I was working with several repositories. How should I handle code from many repositories? As a developer I want all the team around me to look and work on the same code.
Today, In Linnovate, we have many projects of different types and I saw this question (and others) in my mind again and again. So I decided to put a lot of energy to create some method that will handle most of our cases (if not all of them). What I will show is the result of what we use in all of our projects with dockers and came after many project setup and improvement iterations.
I will describe how to manage a microservices-based project with dockers in git and will cover the git repository designing, development, and delivery requirements.
Note: We work with Gitlab but the concept can be transitioned to other repository managers in a similar way
Challenges
- Microservices Integration — how to manage the relations between all microservices?
- Versioning — What will represent the final code version if there are now many repositories?
- Sharing the local environment setup — Working with dockers creates many assets that are not related to the code itself; Dockerfile, configuration files, docker-compose, scripts, and probably more. Where should we store it? How can we share it with others?
- DevOps — Where to store production assets like dockers/deployment scripts which are different from local development dockers
To handle these challenges we decided to base our solution on git submodules and docker-compose to describe and run the local environment
The GS3D Pattern — Git Submodules and Dockers Driven Development
In order to fetch all the repositories, we will use git submodules. Git submodule is a bit tricky to work with but there are ways to work with as can be seen here
Besides fetching the code, each submodule is represented with a specific commit that indicates its version. We’ll use it on the integration repository.
Principles
- One Main repository — one place to work with which gives everything in a clear way.
- Consolidation — Consolidation is first. When having a lot of projects with different technologies — one concept focuses on all developers on the same methodology of work. Easy to jump between projects, easy to set up new projects, and many more advantages when there’s an expected structure. That means keeping the same structure overall projects as much as possible.
- Easy setup — Setting up the local environment should be easy
- Separation — we wanted to have the option to reuse a microservice between several projects
The Integration repository
This is actually the project code repository — The final picture of the code. It represents each code from each microservice in a specific version and also the development environment. This repo will be cloned in order to get the code and set up the local environment in order to develop with its files and dockers.
- Microservices are being referenced using git-submodule for each repo.
- Microservices commit represents the version
- The local repository will be docker-compose based and will use the submodules code.
Structure
- Submodules (ms1, ms2, ms3) — all microservices as submodules. As an example, there can be 3 submodules WordPress, NodeJs, and React.
- Utils — all the files that are required in order to set up the local environment like Dockerfiles and configs. We are storing the Dockerfiles and configurations there.
- docker-compoe.yml — will represent all local services where each service will work with a cloned submodule code. It can also work with a custom Dockerfile from the utils directory.
- .env.example — an example of a working .env file.
- .data (ignored) — for DB files. Can also use virtual docker volume instead.
- .env (ignored)
To set up the local environment