en
English
Deutsch
DrGlitch2
Minimalist Nord
Classic Clean
en
These are the core principles I work by: build for a vendor neutral deployment scenario and attempt to run it as professionally as possible, using open source software.
With cloud service emerging from the 2000s, the web has become a little less open, more complex, and more centralized. At the core of that centralization are the big cloud providers, which offer, at the core, two main benefits to their customers: scalability and the removal of operational burden.
Given their mixed track record of security, availability and privacy, it is still a viable option for companies to run their systems even purely on public cloud infrastructure, as a whole organizational aspect - hardware operations - is off the table when with a cloud hoster. The cloud providers' competition are skilled DevOps engineers and developers who would be responsible for buying, installing, operating and maintaining hardware - at a given service level.
The cloud providers' competition are skilled DevOps engineers and developers who would be responsible for procuring, installing, operating and maintaining hardware - at a given service level.
Most cloud providers offer a wide range of services, from virtual machines to managed databases, to serverless functions. They may offer object storage, SQL and NoSQL databases, backup, managed application hosting, and many more. They offer their brand of golden cage. It is in their economic interest to keep their customers in their ecosystem, and they have no motivation to help them leave.
For this project, I have decided to go the OSS way. Mostly in order to have a reference system in place that is following DevOps best practices (automate everything, infrastructure as code) and is easily compatible with every combination of cloud-hosted VMs and containers, as well as bare metal hardware.
As a whole, it should showcase that modern web technology does not need to rely on a cloud provider's services.
Things evolved over the years, starting 2014, and it began with running
a fairly simple Django application on a single server.
Baremetal, native, no virtualization, no containers. No CI/CD and no automation.
Heck, not even tests.
Initially, for the fun of it.
A few years earlier was when I built my first web applications. That was around 2009.
I was tasked with maintaining a PHP application that would render HTML via XSLT.
Back then, I had a feeling that things should be less cumbersome and unfriendly.
Approximately 2013, I got caught by curiosity and tried to find a technical ecosystem that would appeal to me.
Spring MVC looked intriguing - from a distance. Back then, I thought of it as too heavy for my needs.
At that time, I was looking out for something more pragmatic. Less memory hungry, more expressive.
I found Django, and I was hooked. I started to learn Python along the way.
To me, back then, the biggest competitor was Ruby on Rails. I felt that Python was more likely
to be of use in other areas, as well.
So, that's why this page is made with Django. I like the framework, and I like Python.
They served me well over the years.
I initially ran an apache web server as a reverse proxy in front of the Django application.
Update the page? Local testing, git commit & push, then checkout the latest version on the server.
Django has a thing for automatically restarting the server when wsgi.py detects a change.
Some time around 2017, I learned about docker and containers. I started to containerize the Django application.
In the meantime, I had of course set up DNS and email services (postfix + dovecot). I think GitLab came in
around 2018, and I started to use it for source control and CI/CD.
Things just started to escalate from there. I started to containerize everything. I started to use kubernetes.
Sometimes, I got the opportunity to work for a company that would employ exactly the technology that interested me
the most.
I picked up some great inspiration over the years, I subsequently started to set up an environment that
would usually be found in small to medium-sized companies to serve a lot more developers
than just me - I aim for scalability.
Dear visitor!
If you happen to find any content on this site that you find interesting, or that even helps you in your own
projects and your day job, that would be a great honor to me.
My current approach is to run every web application inside a container, usually orchestrated by kubernetes.
I like to wrap them up in a small, custom helm chart derived directly from the standard provided through
?
Currently, my kubernetes cluster spans 14 nodes of different sizes and capacity.
There are a few tasks that remain directly with certain baremetal machines: email, DNS and TLS termination.
Also, the kubernetes cluster's worker nodes are mostly VMs running on multiple baremetal machines.
From an architecture-centric perspective, there are of course a number of deficits that currently exist with my
setup.
For one, I only have a limited number of physical machines, which puts a restriction on the ability to scale out for
resilience.
Also, a real-life production scenario should not see an email server running on the same machine as the kubernetes
master, for example.
Nevertheless, I want the setup to be suitable for way larger installations and tougher requirements.
Given the appropriate physical resources, it would be straightforward and simple to remove these constraints and
deficits.
As I want to provide a more generic level of abstraction and aim to further isolate operational risk, I had decided
to
set up most worker nodes as virtual machines on my physical hosting.
They are interconnected via a wireguard VPN, which is also used to connect to the kubernetes cluster.
Why yes, here's an architecture diagram:
GitLab is used for source control and CI/CD. The CI/CD pipeline builds docker images and pushes them to the docker registry. I usually bundle my applications as helm charts, which are then pushed to the Nexus repository. Flux CD makes sure to continuously rollout latest versions of the installed helm charts. A public variant of the Flux CD IaC project can be found on my GitLab
The kubernetes cluster was set up using kubeadm. The nodes are managed using fabric, which
I find a more lightweight, but definitely more "hackish" alternative to ansible.
Where ansible is mostly declarative, fabric is strictly imperative and boils down to mostly running shell commands
remotely
on a swarm of machines.
The details can be found on my GitLab.
Nothing too fancy here. This website is still a Django application, running in high availability mode (3 replicas, that is). I try to keep up with the latest dependency versions. I think the largest maintenance tasks were the migration to Python 3 and the overall packaging as a docker image and helm chart. There is some static content with the page, therefore there are NGINX instances serving the static content, as well. The PostgreSQL database of the application is also running in high availability mode (3 replicas) and is managed by Zalando's Postgres Operator. Stylesheets are written in LESS and compiled ahead of time during the application build.