Kustomize and GitOps: The Good, the Bad, and the Ugly

Two-part picture. The figure on the left shows a robot startled while looking at a difference between contents on a Git repository folder and the configuration of a Kubernetes cluster. The figure on the right shows the robot replicating the change on the Kubernetes cluster.
A GitOps framework’s primary job is to detect “drift” between the desired configuration in a Git repo and the actual settings in a deployment.

What you see is almost what you get.

A picture of five file folders with four identical attributes and one tiny difference in the fifth attribute. Sticky figure scratching head while staring at folders.
Copying-pasting folder contents may be the fastest way to bootstrap a new environment but will make maintenance more challenging over time.

How does Kustomize work?

Folder with many YAML files and a file named “kustomization.yaml” with a bulleted list of a few of those files.
Kustomize uses a kustomization.yaml file and a base folder as sources. The kustomize CLI uses the kustomization.yaml file to decide which files to process and how to modify them.

Helm charts, the basics

Picture of a folder with a “templates” folder and a “values.yaml” file. The “templates” folder has many files in it. An inset of one of the files shows a couple of variable declarations and the value declared inside “values.yaml”
Helm transformations revolve around a folder named “templates” with resources and a file named values.yaml. Helm looks for templating statements and replaces them with values such as variables and contents from other resources.

The Good #1: Kustomize as a library cart.

The Good #2: What you see (in the files) is what you get.

Side-by-side drawing of a folder with a Kubernetes Job resource and a kustomization.yaml file containing a “resources” element selecting the file for the Job resource and a long “replacements” field targeting a secret deep inside a “container” element inside the Job specification.
Kustomize’s design calls for a deterministic selection of resources based on a precise selection of resource types, names, and field locations.

The Bad: Mixing filesystems with Kubernetes concepts

Sticky figure looks at three files: a.yaml containing a ConfigMap resource, b.yaml containing another ConfigMap resource, and a kustomization.yaml file containing a “resources” element referencing both files. The kustomization.yaml file has a “replacements” section referencing a “ConfigMap”, making the sticky figure wonder which of a.yaml or b.yaml contain matching resources.
A kustomization.yaml file contains a mixture of concepts, with some constructs based on filenames (“resources”) and other constructs based on Go-templating (“replacements”) and a few based on Kubernetes custom resources (some forms of “patches.”)

The Ugly #1: Dealing with variable replacements

Developer sitting at a table and frantically coding a large block of a YAML resource with multiple lines of statements to replace a single value in a Job resource.
Kustomize sometimes makes simple tasks, like replacing a variable inside a resource, labor-intensive and difficult to maintain.

The Ugly #2: File names are (almost) forever

Sticky figure representing the author of a pull request struggling to move two enormous folders (twice his size) to the right.
Refactoring resources across folders and files is a brittle process, where developers must exercise extreme care in matching every name change with the contents of other files.

The Ugly #3: The magic hybrid

Two-part figure. The tree representation of a folder containing a kustomization.yaml file and two Helm charts is on the left. The kustomization.yaml file contains a “resources” section referencing one of the Helm charts. The right side of the picture has two robots carrying two objects, with one of the robots standing on the back of the first.
A custom configuration management plugin can run different commands in a sequence. With great flexibility comes the great need to design, document, and enforce bespoke workflows.

Conclusion

--

--

--

Cloud alchemist, corporate observer, software engineer, inventor. @dnastacio

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Android Bottom Sheet

The Case for a New Online Card Game

Why warning about technical debt only makes it worse

Managing Dependencies

[Unity] Space Colonization

AMD Zen 2 Architecture : The Dethroning of Intel

Journey with Kotlin 002.5 : Ways to create OnClickListener

Continuous Integration in iOS using JENKINS

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Denilson N.

Denilson N.

Cloud alchemist, corporate observer, software engineer, inventor. @dnastacio

More from Medium

Dynamic setups for PRs with help of ArgoCD and ApplicationSet Controller

The road to production with GitOps: Effective CI/CD strategies

Person placing box on long conveyor belt rolling to the right, with pile of processed boxes to the right of the conveyor belt.

Akuity: Next Chapter In My Argo Journey

Are there two Load Balancer Controllers with EKS ?