Low-budget self-hosted Kubernetes

Introduction

Kubernetes doesn't need to be something you only use on insanely large projects. It provides countless benefits even for small enterprises.

  • High availability/"no single point of failure" (single machines can completely fail while still having 100% service uptime)
  • Seamless deployment of updates (through kubernetes Deployments)*
  • Easy integrated replicated network storage (copies of data is stored on multiple disks, making data loss almost impossible)
  • Perfect monitoring of all your devices, storage devices and running services
  • Countless easy to setup programs thanks to Docker

Primary obstacle

The reason why there are so few self hosted kubernetes clusters out there is probably because of the lack of hands on tutorial series (part of the reason I created this series). The official kubernetes documentation covers concepts nicely and in detail, but leaves out an easy guide for getting started, leaving home tinkerers having to figure out everything on their own. Cloud providers on the other hand don't share their setup guides because that would directly hurt their business model.

The goal of this series is to give you a hands on tutorial on setting up your own cluster, and everything you need to truly make it functional.

We will setup our cluster for use with IPv6 and IPv4, so that even those that don't have the luxury of multiple IPv4 addresses can properly organize their services. Even the most standard internet tarifs should include a publicly accessible range of IPv6 addresses (providers hand them out like candy since there are so many available addresses).

Prerequisites

It assumes a basic understanding of:

  • linux
    • navigating the system using the command line
    • filesystem and folder layout
  • docker
  • networking fundamentals
    • subnets
    • IPv4 and IPv6 public and private address ranges
    • IP routes

Pro Tip: only start looking into those if you actually find yourself not being able to follow something in this series. I touch on this principle of learning more in Learning complex tools effortlessly

You will need 3 machines, virtual or physical does not matter.

In this series we will setup:

  • highly available kubernetes cluster with 3 master nodes with a shared etcd database (this is where kubernetes stores all its configuration)
  • the official kubernetes dashboard - this gives you an intuitive interface to explore your cluster and really get a feel for everything thats going on
  • ceph cluster for storage

If anything goes off the rails, that my series doesn't cover anymore, checkout Kubernetes Debugging and fix it yourself or post a comment.

Optional deployments used with this setup

  • Backups - automated backups to local files. Put them on an external hard drive and be safe and sound
  • PowerDns - this enables us to deploy wildcard certs for our domains using the DNS01 challenge. It's little effort for tremendous gain
  • Nginx server - a simple example of how to run an ssl encrypted (https) webserver
  • Samba - Network filesharing service that also works easily with windows

The Parts

©
Tobias Hübner