Coffee addicted Software Engineer
Creating review apps per pull requests- 16 mins
In this post I will show a simple example about how to create apps for each pull request (or, creating your own Gitlab/Heroku Review App).
Let’s imagine the following scenario of a development team:
- Developer creates a new feature branch.
- Developer pushes the branch
- Developer opens a pull request so other developers can check his code and test the feature
- CI runs the tests and makes the branch green if it passes
Now the developer should send the feature to someone else to test it. The feature should not be merged into master so it can be tested. They should be tested isolated because one feature can interfere with another and master should only have deployable code. With this in mind, we will create an environment for each pull request using Docker.
I will use Jenkins as CI because I want to use an open source tool to demonstrate this, but you can use any CI tool that you prefer.
This post will assume that you already have Jenkins installed. I will use this opensource Rails application as an example. Fork this repo into your account and clone it in your computer.
You can get the full code repo that I used on https://github.com/duduribeiro/openjobs_jenkins_test
The first step that we need is to configuring our app with docker.
A Dockerfile is a file that has instructions to build an image that contains our application. You can learn more about Docker in their documentation.
Create a file named Dockerfile in the app folder:
This is our instructions to build an image. The first command
FROM specifies to docker that we will use the image ruby:2.4.1 as our base image.
After, the first RUN command installs all dependencies the app needs:
node (to precompile the assets. A fancy solution is to use a different container only with node and sprockets to precompile the assets). The second
RUN command, creates a folder
/var/app that will be responsible to store the application. The
COPY command moves the current folder to the container in the
/var/app folder. The next
RUN command installs all dependencies from Rails and yarn. The
CMD specify the command that the container should execute when it runs the image. You can learn more about Dockerfile here.
With this file we can build our image.
With this command, we are building our Dockerfile and generating an image with the tag
docker image ls we can check our image.
Now we can start our application:
-d option tells to docker that this container will run in background and
-p 3000:3000 will connect the local port 3000 with the exposed 3000 port from the container.
We can navigate now to http://localhost:3000.
And we receive this error. This is because our app requires a database connection. We need to run a new container with the database and link both containers so they can communicate with each other. Each container should have only one purpose, so, you should not run more than 1 service in a single container (ie: a container with the application and the database).
Instead of manually run each container, we will use docker compose, a tool to help us to run multi containers applications.
docker ps and
docker kill to destroy your application container.
In this file, we are creating 2 services. One for the database using the postgres image, and another one with our web application using our Dockerfile image.
environment item on
web service, I’m setting the
DATABASE_URL environment variable. If this environment variable is set, rails will use it replacing the loaded configurations from
config/database.yml. Read more about this here. DATABASE_URL follows this pattern for postgres:
postgres://user:password@host/database_name. Since our database user does not have password, we leave empty after the
:. We don’t fill the
database_name because we want the configured one in
config/database.yml (one database name per environment).
Create the databases and run the migrations:
In this command, we will run a container with the
web service and execute the command
rake db:create db:migrate. The
--rm option is to remove the container after the execution.
Run the tests:
Now we can start the application:
Access http://localhost:3000 again and now our app is working.
Configuring our pipeline
We will use Jenkins pipeline as code to configure our pipeline. Read more about it here. We need to have docker, jq and Jenkins installed on CI server.
We will use the following Jenkins plugins: Blue Ocean, Blue Ocean Pipeline Editor, GitHub Pipeline for Blue Ocean
Create a file named
Jenkinsfile in the project root with the following content:
We have 4 stages on this pipeline:
Build: It will build our Dockerfile and generate an image from this build tagged with openjobs with version named latest . It will use docker-compose to build, install dependencies, create and migrate the database created with compose .
Tests: It will run 2 parallel steps, one to run unit tests, and another to run feature tests.
Deploy to staging: If it is the master branch, it will deploy the app to staging.domain . We will cover this in the next steps.
Create feature environment: If it isn’t the master branch, it will deploy the app to branchname.domain . We will cover this in the next steps.
Let’s create our pipeline on Jenkins. Push your code, go to your Jenkins, and access the blue ocean interface and click in the
New Pipeline button:
In the next screen, select Github, choose you account and find the repository. Click in
And our build is passing 🎉
The red icon on the build is because this step will not run, since the branch built was
Now we need to create our environments after the build and dynamic route a domain to a specific container.
Traefik is a tool that will help us make the dynamic routing and act as a load balancer. Example: If I access
http://mybranch.mydomain.com, I want to access the container containing the app with
mybranch that should be started by Jenkins.
Creating dynamic environments
The following steps should be executed on CI server.
We will use Docker Swarm. It is very helpful to create a docker cluster. I will use only one server to demonstrate, but you are able also to create a cluster.
Initialize swarm cluster:
This will initialize our server as the swarm master. It will generate a command that you can use to join the cluster in other servers.
Create a network that we can use with traefik and our containers:
This will initialize traefik. You should treat this initialization on some startup script on your server.
If we access our server in 8081 port, we can access traefik dashboard.
docker.domain specify that we will access the apps through
appname.apps.carlosribeiro.me. To allow this, I created two alias on my DNS server pointing to the CI server’s IP:
Editing Jenkinsfile to create the environments
Lets add a method on Jenkinsfile that will create a environment when the build is triggered.
This method will accept an argument to inform the name of the environment. This name will be used to prefix the services name. It will stop all docker-compose containers that are still running. After, it will remove if exist, services with the same name (This is to recreate the environment when we push again to the branch). After, we create 3 services. One for Postgres, another for Redis, and the app itself using the imaged that we built on
build step. The last command, runs a temporary docker container to create and migrate the database and precompile the assets (you can fetch your previously dump from production too).
Look that in the app service, we specify some environment variables.
REDIS_URL, DATABASE_URL: Endpoint to connect in both services using the previously created services hosts.
RAILS_ENV: We set it to production . It will enforce that our app should behave like a production environment.
RAILS_SERVE_STATIC_FILES: Since we will not have a nginx in front of the app server, we need to set this to tell rails to service static files for us.
-> label ‘traefik.port’ will inform traefik what is the container’s exposed port. In our case, is the 3000. -> name inform the service name that traefik will identify as the prefix on domain.
Now, lets call this method on our pipeline’s stages.
If it is master branch, we will deploy into an app called
staging. If not, it will call the app with the same name of branch.
Look the final Jenkinsfile:
Push master and let’s wait Jenkins create our
staging for us.
Our build passes…
… and our app is deployed on staging:
Now let’s open a pull request to change this button color.
Push this branch, and open a pull request.
Github will tell us that the build is pending.
Job is passing
and Github is informed:
And now we have our dynamic environment up and running
I can send this URL to a QA or a PO review it before merging. If something is wrong, it can be validated before it is on master.