Blog.

GitOps with Helmsman — Deploy Helm charts to Kubernetes

Marco Franssen

August 9, 2021

10 min read•1803 words

0 likes0 reposts0 replies

Comments

Loading comments...

In my last blog series, I showed how to deploy HashiCorp Vault on Kubernetes using Helm charts (see references). This time, I want to show you how to integrate those deployments more easily into your … wait for it … 😄, DevSecGitOps flow. Helm charts help connect our software with our infrastructure and deployments (DevOps). We can also embed security practices such as RBAC and network policies in our charts. In this post, let's make GitOps a bit easier with Helmsman by defining our environments in configuration files.

What is Helmsman

Helmsman is a Helm-charts-as-code tool for Kubernetes applications. It lets you automate the deployment and management of Helm charts from version-controlled code. We can define which charts to deploy together in a configuration file, using either yaml or toml format.

If you're familiar with Terraform, you might appreciate the similar workflow.

Prerequisites

To get started, we'll need the helmsman CLI. Helmsman also requires helm-diff, a plugin for Helm.

brew install -y helmsman
helm plugin install https://github.com/databus23/helm-diff

Now all that's left is to write some yaml, or toml if you prefer. Let's start by defining a local Kubernetes environment in a Helmsman configuration.

Helmsman configuration file

Take a moment to explore the YAML below before I explain its contents.

metadata:
  org: "marcofranssen.nl/research"
  maintainer: "Marco Franssen"
  description: "My Awesome Helmsman demo"
 
settings:
  globalMaxHistory: 5
 
helmRepos:
  bitnami: "https://charts.bitnami.com/bitnami"
  traefik: "https://helm.traefik.io/traefik"
  hashicorp: "https://helm.releases.hashicorp.com"
 
namespaces:
  kube-system:
    protected: true
  default:
    protected: true
  gateway:
    protected: false
  identity:
    protected: false
  secrets:
    protected: false
 
apps:
  traefik:
    namespace: "gateway"
    enabled: true
    chart: "traefik/traefik"
    valuesFile: "local/traefik.yaml"
    version: "10.1.1"
    priority: -4
    wait: true
  keycloak:
    namespace: "identity"
    enabled: true
    chart: "bitnami/keycloak"
    valuesFile: "local/keycloak.yml"
    version: "4.1.1"
    priority: -1
    wait: true
  vault:
    namespace: "secrets"
    enabled: true
    chart: hashicorp/vault
    version: "0.14.0"
    valuesFile: local/vault.yaml
    wait: true

In the metadata section we can describe our environment.

Next, we configure globalMaxHistory, which defines how many previous versions of Helm releases are tracked for rollback purposes.

In the helmRepos section, we define the Helm repositories for our environment. Helmsman takes care of executing helm repo add for repositories that haven't been added yet.

In the namespaces section, we define the namespaces where we'd like to deploy our Helm charts. These can also be existing namespaces from earlier deployments. Protecting a namespace prevents accidental changes to that environment. Updating an existing deployment with the same name in a protected namespace requires you to unprotect it first. For that reason, I always protect the kube-system namespace, where critical Kubernetes components are usually deployed. I also like to protect the default namespace, where I often deploy things to play around with. Helmsman creates any missing namespaces using kubectl.

In the apps section, we define the applications we'd like to add to our environment. I like to use Traefik as my ingress controller. I also want an OIDC-compliant identity service for authentication and authorization, which I can integrate with my applications. Last but not least, I need a way to securely manage application secrets. Here, I'm deploying a Helm release named traefik into the gateway namespace, Keycloak into identity, and Vault into secrets. Helmsman can also run these deployments in parallel if we specify wait: false. Priorities let us control deployment order: assigning a negative priority installs a release before those with the default priority. That's convenient when other apps depend on these services. To configure each app, we provide a valuesFile, which Helmsman uses when invoking the helm install CLI.

Of course, we'd also add charts for the products we're building, such as our microservices. Simply add your own Helm repository to the list, or provide a relative path to a local chart in the chart setting—for example, chart: ./charts/my-app.

Workspace layout

Now that we know how to define an environment with multiple Helm releases, let's look at organizing our workspace to manage different project environments. You might have several development and staging environments, or your team might want specific setups for local development.

Let's say we'd like to manage the following environments:

• local (a shared setup for team members developing on a local Kubernetes environment, e.g. docker-for-mac, kind, or minikube)
• eks-dev (an EKS environment where you continuously test the latest changes to your product)
• eks-staging (an EKS environment where you deploy specific releases for testing before production)
• eks-prod (an EKS environment where you deploy production releases)

We could organize our workspace with a folder layout like this.

$ tree .
.
├── README.md
├── eks-dev
│   ├── autoscaler.yaml
│   ├── consul.yaml
│   ├── external-dns.yaml
│   ├── keycloak.yaml
│   ├── traefik.yaml
│   └── vault.yaml
├── eks-staging
│   ├── autoscaler.yaml
│   ├── consul.yaml
│   ├── external-dns.yaml
│   ├── keycloak.yaml
│   ├── traefik.yaml
│   └── vault.yaml
├── eks-prod
│   ├── autoscaler.yaml
│   ├── consul.yaml
│   ├── external-dns.yaml
│   ├── keycloak.yaml
│   ├── traefik.yaml
│   └── vault.yaml
├── eks-dev.yml
├── eks-staging.yml
├── eks-prod.yml
├── local
│   ├── keycloak.yml
│   ├── traefik.yaml
│   └── vault.yaml
└── local.yml
 
4 directories, 24 files

In this layout, I keep my dev, staging, and prod environments on separate Kubernetes clusters. If you'd rather use namespaces to separate environments on the same cluster, you'll probably want to organize your files slightly differently. Make sure you protect the other environments' namespaces to prevent mistakes.

You can do that by marking the production and staging namespaces as protected in your eks-dev configuration, and doing the equivalent in eks-staging and eks-prod. That way, you can't accidentally overwrite apps in those protected namespaces.

For example:

namespaces:
  kube-system:
    protected: true
  secrets-dev:
    protected: false
  secrets-staging:
    protected: true
  secrets-prod:
    protected: true

This lets us install HashiCorp Vault in a separate namespace for each environment.

Back in my setup, where the environments run on completely isolated clusters, we don't need to define namespaces for the other environments. For a quick start, you could copy your local.yaml, adjust the paths to the values files, and add deployments for DNS management, load balancing, autoscaling, and so on, as I've done here. Configuring those charts is beyond the scope of this article, but you can explore their documentation on Artifact Hub.

I've also written a series on the basics of Helm charts (part-1, part-2). You might have noticed that I'm installing HashiCorp Consul in the eks environments above. That's because my highly available Vault setup on EKS uses HashiCorp Consul. Locally, I deploy Vault with a more minimal setup. Part-1 covers the local setup, while part-2 covers the highly available setup on EKS. Have a look if you'd like more background on deploying a Helm chart.

Deploying a Helmsman environment

Now that we know how to define a Helmsman configuration and organize our workspace and values files, we can deploy multiple Helm charts in one go.

$ helmsman -f local.yaml -apply
 
 _          _
| |        | |
| |__   ___| |_ __ ___  ___ _ __ ___   __ _ _ __
| '_ \ / _ \ | '_ ` _ \/ __| '_ ` _ \ / _` | '_ \
| | | |  __/ | | | | | \__ \ | | | | | (_| | | | |
|_| |_|\___|_|_| |_| |_|___/_| |_| |_|\__,_|_| |_| version: v3.7.2
A Helm-Charts-as-Code tool.
 
2021-08-08 11:45:56 INFO: validating environment variables in /Users/marco/code/my-infrastructure/local/keycloak.yml
2021-08-08 11:45:56 INFO: validating environment variables in /Users/marco/code/my-infrastructure/local/spire.yaml
2021-08-08 11:45:56 INFO: Parsed [[ ./local.yml ]] successfully and found [ 3 ] apps
2021-08-08 11:45:56 INFO: Validating desired state definition
2021-08-08 11:45:56 INFO: Setting up kubectl
2021-08-08 11:45:56 INFO: Setting up helm
2021-08-08 11:46:00 INFO: Setting up namespaces
2021-08-08 11:46:00 INFO: Getting chart information
2021-08-08 11:46:02 INFO: Charts validated.
2021-08-08 11:46:02 INFO: Preparing plan
2021-08-08 11:46:02 INFO: Acquiring current Helm state from cluster
2021-08-08 11:46:11 INFO: Checking if any Helmsman managed releases are no longer tracked by your desired state ...
2021-08-08 11:46:11 INFO: No untracked releases found
2021-08-08 11:46:11 NOTICE: -------- PLAN starts here --------------
2021-08-08 11:46:11 NOTICE: Release [ traefik ] version [ 10.1.1 ] will be installed in [ gateway ] namespace -- priority: -4
2021-08-08 11:46:11 NOTICE: Release [ keycloak ] version [ 4.1.1 ] will be installed in [ identity ] namespace -- priority: -1
2021-08-08 11:46:11 INFO: Release [ vault ] installed and up-to-date -- priority: 0
2021-08-08 11:46:11 NOTICE: -------- PLAN ends here --------------
2021-08-08 11:46:11 INFO: Executing plan
2021-08-08 11:46:11 NOTICE: Install release [ traefik ] version [ 10.1.1 ] in namespace [ traefik ]
…………………………………………

Of course, there are more flags we can use, such as one to select the kubeconfig file. That's convenient when connecting to a specific EKS cluster. As your environment grows, you won't necessarily want to apply every app. Use the -target flag to select one or more specific apps. Type helmsman --help to explore all the command-line options.

For example, to destroy only the keycloak and vault apps in your eks-dev environment, use the following command.

helmsman -f eks-dev.yaml -kubeconfig ./my-eks-kubeconfig -destroy -target keycloak -target vault

Omitting the -apply and -destroy flags prints the plan without executing it. This is useful when you want to review changes before applying them.

All that's left is to commit your configurations to Git and perhaps build CI/CD jobs around them to continuously deploy changes to eks-dev. Then you can start building Helm charts for your own apps.

Bonus

To create Helm charts, start with the following commands to generate a new chart from a scaffold.

helm create charts/service-a
helm create charts/service-b -p my-scaffold

Here, service-a uses the default scaffold, while service-b uses my custom scaffold. Our local Helm charts are now in the charts folder. We can refer to these charts in our Helmsman configurations to test them in our environment.

apps:
  my-service-a:
    namespace: "domain-a"
    enabled: true
    chart: ./charts/service-a
    version: "0.1.0"
    valuesFile: local/service-a.yaml
    wait: true
  the-service-b:
    namespace: "domain-b"
    enabled: true
    chart: ./charts/service-b
    version: "0.1.3"
    valuesFile: local/service-b.yaml
    wait: true

Once you've published your charts to a Helm repository, you can install them from there instead of relying on relative file paths.

helmRepos:
  marco: "https://marcofranssen.github.io/helm-charts/"
 
apps:
  my-service-a:
    namespace: "domain-a"
    enabled: true
    chart: marco/service-a
    version: "0.1.0"
    valuesFile: local/service-a.yaml
    wait: true
  the-service-b:
    namespace: "domain-b"
    enabled: true
    chart: marco/service-b
    version: "0.1.3"
    valuesFile: local/service-b.yaml
    wait: true

References

• https://helm.sh
• https://github.com/Praqma/helmsman
• Helm diff
• Artifact Hub
• Install HashiCorp Vault on Kubernetes using Helm - Part 1
• Install HashiCorp Vault on Kubernetes using Helm - Part 2

Happy Helming 🚀