How to set up Swarm cluster with Ansible?

Edit 2016/07/05: This tutorial was written for Docker Swarm 1.11 or older. Now Docker Swarm is very easy to install and to deploy.

I wanted to learn more about Docker Swarm and I wanted to learn how to use Ansible, so naturally here I show you to quickly set up a Docker Swarm cluster using Ansible for deployment.

Requirements

Hosts

The hosts I use are all Debian 8.4 Jessie servers, dedicated server and VPS from kimsufi (a OVH brand), so if your hosts are of another linux distribution the Ansible scripts could not work.

Be sure that you have root access to your hosts.

Install Ansible

On your OSX system just run the following brew command on a terminal:

For the other operating systems, read more about how to install Ansible at the install Ansible documentation page.

Install Docker Engine

To install Docker Engine on your desktop computer follow the instructions at this page: install docker engine.

What is Swarm?

Briefly, Docker Swarm lets you manage many hosts with Docker Engine as you only have one host with Docker Engine.

Docker Swarm provides native clustering capabilities to turn a group of Docker engines into a single, virtual Docker Engine.

You can read more about Docker Swarm on the official website.

What is Ansible?

Briefly, Ansible is an orchestration tool that helps you deploy your infrastructure.

Ansible seamlessly unites workflow orchestration with configuration management, provisioning, and application deployment in one easy-to-use and deploy platform.

You can read more about Ansible on the official website.

Architecture of the cluster

The architecture is composed of Consul hosts (one or two hosts), Docker Swarm manager hosts (the number of your choice, three or more allows to set up a high availability cluster) and Docker Swarm node hosts (the number of your choice too).

The Ansible scripts here will set up one or two Consul hosts, but if you can set up a complete high availability Consul cluster this will be better.

In this tutorial we will set up a docker cluster with four different hosts: the first host discovery.example.com will host the Consul services, the host manager.example.com will have the role of the swarm manager (we will connect to it to manage the cluster), and finally the two last hosts node1.example.com and node2.example.com will run the containers of the cluster.

Clone the project

You can find the project here: https://github.com/cygy/ansible-docker-swarm-cluster

I will tell more details about it in this post, for now let’s clone the project and go to the project directory:

Prepare your hosts

Update the kernel version

As Docker Swarm documentation says, the kernel version of the Debian hosts must be at least 3.10. Read this post to upgrade the kernel of your Debian servers.

Docker requires a 64-bit installation regardless of your Debian version. Additionally, your kernel must be 3.10 at minimum. The latest 3.10 minor version or a newer maintained version are also acceptable.

Kernels older than 3.10 lack some of the features required to run Docker containers. These older versions are known to have bugs which cause data loss and frequently panic under certain conditions.

Be careful to not update to a kernel >= 4.0, Docker Swarm does not work with these versions.

 Generate the TLS certificates

To secure the communication between your swarm managers and your swarm nodes, it is recommended to use TLS. To use TLS you need certificates. Later I will check to use Letsencrypt, but for now we will generate self-signed certificates.

The certificates must be generated on a Linux system, so if you are on an OSX system, first copy the keys directory of the project on a linux host:

Then log in your host, and go to the keys directory:

Create your private key by running the script create_CA. You must provide a string passphrase when asked. The key and the certificate will be created under the directory keys/CA.

Now, for each of the hosts you must run the command create_keypair with the hostname of the host. The key and the certificate will be created under the directory hosts/[HOSTNAME].

In order to manage our Docker Swarm cluster later, we need to generate certificate for our home IP address too:

You can log out now:

You must get back the generated certificates and keys from your host to your local machine:

Check in the keys/hosts and keys/CA directories if all is here.

Set up the Ansible variables

Declare your hosts

Ansible needs to know what are your hosts. Open the hosts file and define the IP addresses of your hosts at the beginning of the file, the rest of the file defines the roles of the hosts:

Here seven hosts are declared, the format is:

Then under each role (the lines formatted as [role]) add the hosts which have this role:

  • The role [swarm-discovery-networks] will run Docker Engine and Consul: add one host here. The hosts with Docker Engine will register to this Consul host to manage the docker networking.
  • The role [swarm-discovery-nodes] will run Docker Engine and Consul: add one host here. It can be the same host under the role [swarm-discovery-networks]. The hosts with Docker Engine will register to this Consul host in order to be managed by the hosts with the [swarm-managers] role.
  • The role [swarm-managers] will run Docker Engine and Docker Swarm: add all the hosts you want: one host will be elected as the primary manager the others as secondary managers.
  • The role [swarm-nodes] will run Docker Engine and will be managed by the [swarm-managers] hosts: add all the hosts you want.

Don’t edit the other lines.

Set up the main variables

Open the file main.yaml. Here are defined many variables:

  • Under the part users_groups, you can define some user groups to add to your hosts. Set the name and gid of the groups.
  • Under the users group, you can define some users to add to your hosts. Set the name and uid of the users, aliases can also be defined. If a file named [username].keys containing a SSH key is present at the directory keys/users it will be copied to the host.
  • Set the value of the variable paranoid_allow_ping to 0 if you want that your hosts don’t respond to the ping requests.
  • Under the part paranoid_fail2ban you can define the settings of Fail2ban service.
  • You must update the variable node_discovery_host with the hostname (the same of the TLS certificates) of a node listed under the part swarm-discovery-nodes of the hosts file.
  • You must update the variable network_discovery_host with the hostname (the same of the TLS certificates) of a node listed under the part swarm-discovery-networks of the hosts file.
  • Finally, under the manager_ips part, add all the IP addresses of your hosts listed under the part swarm-managers of the hosts file.

You don’t have to edit the other variables.

Set up the hosts variables

Now, into the host_vars directory, create a file for each of your hosts. Name these files with the name of the variables of the host (i.e. here will be node1, node2discovery or manager).

Fill these files with the following content, setting the hostname of your hosts:

You can add here all the variables, defined at another place, that you want to redefine to this specific host.

Run Ansible: let’s the magic do!

Ensure that you are at the root of the project directory and run:

If there is no error, Ansible sets up your hosts as a Docker Swarm cluster.

Manage the Docker Swarm cluster

In order to manager the cluster we need to connect to a manager host defined in the  hosts file. To do the,  a secure connection is needed.

Remember a TLS certificate for our home IP address has been generated sooner. It is located at the directory keys/hosts/[HOME_IP_ADDRESS].

Copy the certificates at the same place:

Export environment variables to point your Docker Engine to a manager host of the cluster:

Now all your Docker Engine commands will point to your Docker Swarm cluster!

Here the result of the command docker info:

Here the result of the command docker ps -a:

Links

Leave a Reply

Your email address will not be published. Required fields are marked *