In the beginning there were containers. Mostly Docker containers, but there were others too. Containers were great, but left to themselves, they were no more than individual musicians, without a conductor to hold them together or even a real orchestra to perform in. It was easy enough to get one container to do one thing well, but it was difficult to get them to play together.
Then came along container orchestration. Tools mostly developed mostly by companies with a need for massively scalable applications, such as Google and Amazon. Eventually, even Docker themselves saw the need, and also jumped in with Compose and Swarm.
Containers provide numerous benefits to developers, including dependency management, increased testability and increased scalability. Applications can now be developed and tested in environments very similar to the production environment they will be running in.
In this series of articles, we will see how to create a development pipeline using Docker containers. We will make use of Kubernetes as our container orchestration platform, as well as other freely-available tools such as Ansible and Vagrant, to help us automate the setup and running of the pipeline as much as possible.
Why are we doing this?
Kubernetes includes a command line tool, Minikube, which is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes ‘cluster’ inside a VM on your local computer. It’s mostly for users looking to try out Kubernetes or develop a simple application with it. While Minikube is easy to use and will help you get going quickly, it is restricted to running just one server node, and so won’t allow you to really test your application in a concurrent, multi-node environment.
Our local cluster
This article will detail the major steps required to setup a Kubernetes cluster running Ubuntu Linux. We will setup the cluster using several virtual machines hosted locally, using the free VirtualBox application. This cluster may be setup on any Mac or PC supported by VirtualBox and Vagrant.
Our cluster will enable us to test application containers in a multi-node environment, so that we can see how well they respond to the challenges of scaling. In particular, it will help us identify any issues related to concurrency even while in the development environment, so they can be resolved as early as possible in the development process.
The cluster will consist of 3 hosts:
- one Kubernetes Master
- two Kubernetes Nodes
Each host will consist of a virtual machine, running an identical copy of Ubuntu Linux. There must always be one master available in our cluster, but we can vary the number of nodes if required. This will allow us to create an environment as similar as possible to production, right on our local machine.
Prerequisites
If you are already familiar with Docker, then by all means, go ahead and dive straight into setting up the cluster by following the steps below. If not, then I recommend you begin by taking some time to learn the basics of Docker, as some of the concepts and terminology may be new to you. Here are some suggested resources to help you get up to speed quickly:
- The Get Started guide for Docker is a good place to start. Go through the first 2 sections of this first. They cover installing Docker, creating containers and running simple containerised applications locally. Once you are clear on this, come back here.
- The Kubernetes page on Wikipedia provides a quick intro to the design and architecture of Kubernetes.
- Kubernetes Tutorials, to help you become familiar with the Kubernetes architecture, features and command line.
Once you have a basic grasp of Docker and Kubernetes, you should be ready to continue.
Download
- VirtualBoxInstall this to run virtual machines on your local Mac or PC.
- VagrantInstall this to allow quick and easy setup of the virtual machines we will be using in this article.
Install this on your local Mac/PC, to allow you to control your cluster and access the Kubernetes dashboard through a web browser.
Configuring and Running the Virtual Machines
By making use of Vagrant, we will spin up 3 virtual machines (VMs) using just one configuration file. One of these VMs will take the role of Master and the other two VMs will act as Nodes in our Kubernetes cluster.
- Download the required Vagrantfile and save it in a new, empty folder on your Mac or PC. You can either clone my GitHub repository or just download the file and save it to an empty folder on your local machine.
- Open a terminal or command line window and change to the folder that you saved the Vagrantfile to.
- Start up the VMs in one go
$ vagrant up
You will then see a number of messages, starting with:
Bringing machine 'master' up with 'virtualbox' provider... Bringing machine 'node1' up with 'virtualbox' provider... Bringing machine 'node2' up with 'virtualbox' provider...
as Vagrant downloads the ‘box’ image (the image of the basic VM we will be using) and sets up each of the 3 VM instances. Once the box image has been downloaded, numerous additional packages will be downloaded and installed automatically, including those required for Docker and Kubernetes.
This process will take approximately 15 minutes to complete.
- Get the configuration for our new Kubernetes cluster so we can access it directly from our local machine
$ export KUBECONFIG="$KUBECONFIG:`pwd`/admin.conf"
- Optionally, proxy the admin console to your local Mac/PC
$ kubectl proxy
Leaving the above command running, access the Kubernetes Admin Console in your web browser.
You should now have a fully working Kubernetes cluster running on your local machine, to which you can deploy containers using either the admin console or the kubectl command line tool.
In the next article in this series, we will see how to make use of Docker and Kubernetes to setup an automated container pipeline, with separate environments for continuous integration, testing and production.
In the mean time, have fun playing with containerised applications on your local machine!