Gitops using Helmsman to apply our Helm Charts to k8s
Marco Franssen /
11 min read • 2119 words
In my last blog series I have shown an example of deploying Hashicorp Vault on Kubernetes using Helm Charts (see references). This time I want to show you how to more easily integrate this into your … wait for it … 😄,
DevSecGitOps flow. Especially Helm charts help a lot in connecting the software part with our infrastructure / deployment (DevOps). Besides that we can embed all kind of security practices in our Helm charts like for example RBAC, Network policies etc. In this blog I want to show how we can make GitOps a bit easier using Helmsman, by defining our environments in the form of a configuration file.
Helmsman is a Helm Charts (k8s applications) as Code tool which allows you to automate the deployment/management of your Helm charts from version controlled code. Using Helmsman we can define which Helm charts we want to deploy as a composed whole. We can do this by defining all of our needs in a configuration file, which can either be in
Folks familiar with Terraform might appreciate this tool as it enables a similar workflow like you would be used to when using Terraform.
To get started we will ofcourse need the
helmsman cli. Next to that Helmsman also requires helm-diff, a plugin for Helm.
brew install -y helmsman helm plugin install https://github.com/databus23/helm-diff
Now the only thing left is that we write some
toml if you prefer. So let's start by defining some local Kubernetes environment as a Helmsman definition.
Please take some time to explore the contents of the yaml, before I will start explaining its contents.
metadata: org: "marcofranssen.nl/research" maintainer: "Marco Franssen" description: "My Awesome Helmsman demo" settings: globalMaxHistory: 5 helmRepos: bitnami: "https://charts.bitnami.com/bitnami" traefik: "https://helm.traefik.io/traefik" hashicorp: "https://helm.releases.hashicorp.com" namespaces: kube-system: protected: true default: protected: true gateway: protected: false identity: protected: false secrets: protected: false apps: traefik: namespace: "gateway" enabled: true chart: "traefik/traefik" valuesFile: "local/traefik.yaml" version: "10.1.1" priority: -4 wait: true keycloak: namespace: "identity" enabled: true chart: "bitnami/keycloak" valuesFile: "local/keycloak.yml" version: "4.1.1" priority: -1 wait: true vault: namespace: "secrets" enabled: true chart: hashicorp/vault version: "0.14.0" valuesFile: local/vault.yaml wait: true
metadata section we can describe our environment.
Next we configured the
globalMaxHistory, which defines how many previous versions of Helm releases are tracked for rollback purposes.
helmRepos section we define the Helm repositories for our environment. Helmsman will take care of executing the
helm repo add commands for the repos that are not yet added.
namespaces section we can define the namespaces where we would like to deploy our Helm charts to. Please note this can also be existing namespaces for deployments you did earlier. By protecting a namespace you can prevent accidental changes to those environments. Doing an update/upgrade of a existing deployment with the same name in such namespace would require you to unprotect the namespace first. For that reason I always protect the
kube-system namespace as there Kubernetes core critical components are usually deployed. I also like to protect the default namespace as that is usually a namespace where I deploy some stuff to play arround with. Helmsman will take care of creating the namespaces that didn't exist yet using
apps section we can define all the Apps we would like to add to our environment. I like to use
Traefik as my ingress controller. Furthermore any modern environment also needs some identity service like an OIDC compliant authentication and authorization component which I can use to integrate my applications. Last but not least I also want to have a way to securily manage my application secrets. As you can see I'm deploying a Helm release, named
traefik, into the
gateway namespace. I'm also deploying Keycloak into the
identity namespace and Vault into the
secrets namespace. Helmsman is also capable of doing all these deployments in parallel if we would specify
wait: false. Using priorities we will be able to control the order of some deployments by assigning them a negative priority they will install before the others. This can be convenient if some of your other apps depend on these services. To configure the specific apps we provide the valuesFile. Helmsman will take these values when invoking the
helm install cli.
Ofcourse we would also add the Helm charts here for the products we are building. E.g. various microservices. Simply add your own helm repository to the list or define a relative path to a local Helm chart in the
chart settings. E.g.
Now we know a bit more on how to define an environment with multiple Helm releases we should also have a look at how we can structure our workspace to manage different environments for our project. Maybe you have multiple development and staging environments. Or your team would like to have specific setups for local development.
Lets say we would like to manage following environments:
- local (commonly shared environment setup for local development by teammembers on a local Kubernetes environemnt, e.g.
- eks-dev (environment on EKS where you will continuously test your latest greatest changes to the product you are working on)
- eks-staging (environment on EKS where you will deploy specific releases to test them before production)
- eks-prod (environment on EKS where you will deploy your production releases)
To structure our workspace we could therefore create a folder layout like this.
$ tree . . ├── README.md ├── eks-dev │ ├── autoscaler.yaml │ ├── consul.yaml │ ├── external-dns.yaml │ ├── keycloak.yaml │ ├── traefik.yaml │ └── vault.yaml ├── eks-staging │ ├── autoscaler.yaml │ ├── consul.yaml │ ├── external-dns.yaml │ ├── keycloak.yaml │ ├── traefik.yaml │ └── vault.yaml ├── eks-prod │ ├── autoscaler.yaml │ ├── consul.yaml │ ├── external-dns.yaml │ ├── keycloak.yaml │ ├── traefik.yaml │ └── vault.yaml ├── eks-dev.yml ├── eks-staging.yml ├── eks-prod.yml ├── local │ ├── keycloak.yml │ ├── traefik.yaml │ └── vault.yaml └── local.yml 4 directories, 24 files
In this layout I consider my
prod to be strictly segregated on different kubernetes clusters. In case you would like to approach it differently by installing on the same cluster but e.g. use namespaces to segregate the environments you probably want to organize your files slightly different. Make sure though to protect the namespaces for the other environments to prevent mistakes.
You can do that by mentioning in your
eks-dev environment the production and staging namespaces as protected and vice-versa in your
eks-prod. This way you can not accidentally overwrite apps in those namespaces.
namespaces: kube-system: protected: true secrets-dev: protected: false secrets-staging: protected: true secrets-prod: protected: true
This would allow us to install Hashicorp Vault in different namespaces for the different environments.
Coming back to my setup where the environments are in completely isolated clusters we will not have to define the namespaces for other environments. To have a quick start you could for example copy paste your
local.yaml and adjust the paths to the value files as well add some more deployments like I did to manage the DNS records, loadbalancing, autoscaling etc. I won't go into detail on how to configure these Charts specifically as that is beyond the scope of this article, however by exploring the charts documentation at Artifact Hub you should be able to manage that on your own.
Next to that I have also written a blog series on the basics of using Helm charts (part-1, part-2). As you might have noticed in the
eks environment above I was also installing Hashicorp Consul. The reason for that is I'm using a High available Setup of Vault on EKS that requires Hashicorp Consul. On local I deploy Hashicorp Vault using a more minimal setup. In part-1 I cover the local setup of Hashicorp Vault and in part-2 I cover the High Available setup on EKS. Consider reading those articles to get some more insight on the basics of deploying a Helm chart.
Now we know how to define a Helmsman configuration and how to organize our workspace and values files in an efficient manageble way we can easily deploy multiple Helm charts in one go using Helmsman.
$ helmsman -f local.yaml -apply _ _ | | | | | |__ ___| |_ __ ___ ___ _ __ ___ __ _ _ __ | '_ \ / _ \ | '_ ` _ \/ __| '_ ` _ \ / _` | '_ \ | | | | __/ | | | | | \__ \ | | | | | (_| | | | | |_| |_|\___|_|_| |_| |_|___/_| |_| |_|\__,_|_| |_| version: v3.7.2 A Helm-Charts-as-Code tool. 2021-08-08 11:45:56 INFO: validating environment variables in /Users/marco/code/my-infrastructure/local/keycloak.yml 2021-08-08 11:45:56 INFO: validating environment variables in /Users/marco/code/my-infrastructure/local/spire.yaml 2021-08-08 11:45:56 INFO: Parsed [[ ./local.yml ]] successfully and found [ 3 ] apps 2021-08-08 11:45:56 INFO: Validating desired state definition 2021-08-08 11:45:56 INFO: Setting up kubectl 2021-08-08 11:45:56 INFO: Setting up helm 2021-08-08 11:46:00 INFO: Setting up namespaces 2021-08-08 11:46:00 INFO: Getting chart information 2021-08-08 11:46:02 INFO: Charts validated. 2021-08-08 11:46:02 INFO: Preparing plan 2021-08-08 11:46:02 INFO: Acquiring current Helm state from cluster 2021-08-08 11:46:11 INFO: Checking if any Helmsman managed releases are no longer tracked by your desired state ... 2021-08-08 11:46:11 INFO: No untracked releases found 2021-08-08 11:46:11 NOTICE: -------- PLAN starts here -------------- 2021-08-08 11:46:11 NOTICE: Release [ traefik ] version [ 10.1.1 ] will be installed in [ gateway ] namespace -- priority: -4 2021-08-08 11:46:11 NOTICE: Release [ keycloak ] version [ 4.1.1 ] will be installed in [ identity ] namespace -- priority: -1 2021-08-08 11:46:11 INFO: Release [ vault ] installed and up-to-date -- priority: 0 2021-08-08 11:46:11 NOTICE: -------- PLAN ends here -------------- 2021-08-08 11:46:11 INFO: Executing plan 2021-08-08 11:46:11 NOTICE: Install release [ traefik ] version [ 10.1.1 ] in namespace [ traefik ] …………………………………………
Ofcourse there are more flags we could use. For example to define the
kubeconfig file to be used. This is conventient for connecting to a specific EKS cluster. When your environment is growing you not necessarly want to apply all the apps. In that case you can use the
-target flag to define one or many specific apps. Simply type
helmsman --help to learn about all the commandline options you can use.
For example to only destroy the
vault app in your
eks-dev environment you can use the following command line.
helmsman -f eks-dev.yaml -kubeconfig ./my-eks-kubeconfig -destroy -target keycloak -target vault
-destroy flags would print you the plan about to be executed. This is very usefull if you only want to check the changes upfront without applying them.
Now the only thing remaining is to commit all you configurations in Git and maybe build some CI/CD jobs arround these to continuously deploy your changes to your
eks-dev environment. Once done, you can start by building your own helm charts for your own apps.
To create helm charts you could simply start with the following commandline to start on a new Chart generated from some scaffold.
helm create charts/service-a helm create charts/service-b -p my-scaffold
service-a in this case uses the default scaffold template, where for
service-b I create a new Helm chart using my custom scaffold. Our local helm charts will now be in the
charts folder. In Helmsman configurations you can now easily refer to these local helm charts to test them in your environment.
apps: my-service-a: namespace: "domain-a" enabled: true chart: ./charts/service-a version: "0.1.0" valuesFile: local/service-a.yaml wait: true the-service-b: namespace: "domain-b" enabled: true chart: ./charts/service-b version: "0.1.3" valuesFile: local/service-b.yaml wait: true
Once you published your charts to a Helm repository you could ofcourse also install them via that repository, so you don't depend on relative filepaths to your charts anymore.
helmRepos: marco: "https://marcofranssen.github.io/helm-charts/" apps: my-service-a: namespace: "domain-a" enabled: true chart: marco/service-a version: "0.1.0" valuesFile: local/service-a.yaml wait: true the-service-b: namespace: "domain-b" enabled: true chart: marco/service-b version: "0.1.3" valuesFile: local/service-b.yaml wait: true
- Helm diff
- Artifact Hub
- Install Hashicorp Vault on Kubernetes using Helm - Part 1
- Install Hashicorp Vault on Kubernetes using Helm - Part 2
Happy Helming 🚀