YAML and JSON. Templating YAML with YAML Processors. Static Checking of Kubernetes YAML Files

1. Introduction. Templating YAML Files
   1. K8s YAML tips
   2. YAML anchors and aliases
   3. YAML Processors
   4. YAML Validators
   5. Other YAML Tools. How to create Kubernetes YAML files
   6. Kubernetes examples
   7. Helm and Kustomize
2. JSON. Templating JSON Files
   1. JSONPath query language
   2. JSON in Ansible
   3. JSON formatting with jq
   4. JSON Tools
   5. Kubernetes JSON Schemas
3. Static Checking of Kubernetes YAML Files. Kubernetes YAML Validation Tools
4. Alternatives
5. Base64
6. Videos
7. Tweets

Introduction. Templating YAML Files

• wikipedia: YAML
• redhat.com: YAML for beginners YAML is an easy, expressive, data-oriented language that distinguishes itself from document markup languages.
• thoughtworks.com: Templating in YAML As infrastructures grow in complexity, so do the configuration files that define them. Tools such as AWS CloudFormation, Kubernetes and Helm expect configuration files in JSON or YAML syntax, presumably in an attempt to make them easy to write and process. However, in most cases, teams quickly reach the point where they have some parts that are similar but not quite the same, for example, when the same service must be deployed in different regions with a slightly different setup. For such cases tools offer templating in YAML (or JSON), which has caused a huge amount of frustration with practitioners. The problem is that the syntax of JSON and YAML requires all sorts of awkward compromises to graft templating features such as conditionals and loops into the files. We recommend using an API from a programming language instead or, when this is not an option, a templating system in a programming language, either a general-purpose language such as Python or something specialized such as Jsonnet.
• Steve Horsfield: DevOps tricks - Templating YAML files Basic text tools fall foul of YAML’s indentation sensitivity. On the other hand, YAML tools like ytt are pretty difficult to interpret. In my case, I opted for a small jq program executed via yq.
• redhat.com: Understanding YAML for Ansible. Validating YAML files with YAMLlint 🌟 Ansible playbooks are written in YAML, YAML Ain’t Markup Language. Understanding YAML syntax is a key to success with Ansible.
• linuxhandbook.com: YAML Basics Every DevOps Engineer Must Know 🌟
• developers.redhat.com: How to configure YAML schema to make editing files easier
• kubernetestutorials.com: Kubernetes : Introduction to YAML 🌟
• betterprogramming.pub: YAML Tutorial: Get Started With YAML in 5 Minutes Syntax, salient features, and advanced features.
• abhisheksaini.hashnode.dev: YAML For Data Representation? YAML is a better option than JSON when comes to representation of Data
• boxunix.com: A Better Way of Organizing Your Kubernetes Manifest Files 🌟
• opensource.com: Make YAML as easy as it looks YAML looks simple so why is it so hard to write? Learn the two secrets to YAML success.
• javascript.plainenglish.io: Everything You Need To Know About YAML Files Learning about YAML gives you an advantage over your peers since it literally makes programming and configuring cloud computing resources easier.
• w3schools.io: YAML - yaml vs yml file
• blog.devgenius.io: YAML with Python This article aims to outline the basics of YAML and write a simple python script that reads configuration details from a YAML file.
• realpython.com: YAML: The Missing Battery in Python Python lacks built-in support for the YAML data format, commonly used for configuration and serialization, despite clear similarities between the two languages. In this tutorial, you’ll learn how to work with YAML in Python using the available third-party libraries, with a focus on PyYAML.
• kubesimplify.com: A Simplified Guide To YAML
• betterprogramming.pub: 10 Things You Might Not Know About YAML Harness the true power of YAML
• spacelift.io/blog/yaml YAML Tutorial : A Complete Language Guide with Examples
• ruudvanasseldonk.com: The yaml document from hell
• itnext.io: Python, YAML, and Kubernetes — The Art of Mastering Configuration A practical guide to simplifying Kubernetes configuration management with Python and YAML

K8s YAML tips

• itnext.io: Kubernetes YAML Tips | Daniele Polencic 🌟

YAML anchors and aliases

• yaml.org: Anchors and Aliases
• support.atlassian.com: YAML anchors and aliases
• medium: Don’t Repeat Yourself with Anchors, Aliases and Extensions in Docker Compose Files
• docs.ansible.com: YAML anchors and aliases: sharing variable values

YAML Processors

• github.com/topics/yaml-processor
• ytt is a templating tool that understands YAML structure allowing you to focus on your data instead of how to properly escape it.
• You should use tools such as yq and kustomize to template YAML resources instead of relying on tools that interpolate strings such as Helm.
• yq 🌟 is a lightweight and portable command-line YAML processor. The aim of the project is to be the jq or sed of yaml files. yq allows to query the yaml tree and highlights it:
  • k get svc a -o yaml | yq r -
  • dev.to: yq : A command line tool that will help you handle your YAML resources better 🌟
  • towardsdatascience.com: yq: Mastering YAML Processing in Command Line 🌟 Learn to parse and manipulate YAML files more efficiently using yq command-line utility and this simple cheat sheet
  • betterprogramming.pub: My Cheatsheet for the “yq” Tool Parsing Kubernetes YAML with yq. yq is a convenient tool for creating, updating and mangling Kubernetes YAML files. In this tutorial, you will learn how to use it as well as some practical tips and tricks.
• Kapitan Generic templated configuration management for Kubernetes, Terraform and other things.
• azohra/yaml.sh A YAML parser completely in bash. Yaml.sh — YAML Sans Helm
• yh - YAML Highlighter is YAML syntax highlighter that works nicely with kubectl output
• Kubectl output options 🌟

YAML Validators

• yamllint.com: YAML Lint - The YAML Validator
• codebeautify.org/yaml-validator
• jsonformatter.org/yaml-validator

Other YAML Tools. How to create Kubernetes YAML files

• onlineyamltools.com 🌟
• avencera/yamine A simple CLI for combining json and yaml files
• k8syaml.com 🌟 Kubernetes YAML Generator - Powered by Octopus
• itnext.io: How to create Kubernetes YAML files 🌟 - ref2 at hackernoon.com
• datree.io Prevent Kubernetes Misconfigurations From Reaching Production
  • dev.to: Automating quality checks for Kubernetes YAMLs
• 23andMe/Yamale A schema and validator for YAML. Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas.
• instrumenta/kubeval Validate your Kubernetes configuration files, supports multiple Kubernetes versions. kubeval is a tool for validating a Kubernetes YAML or JSON configuration file. It does so using schemas generated from the Kubernetes OpenAPI specification, and therefore can validate schemas for multiple versions of Kubernetes.
• medium.com/@slashben81: How to write a YAML file for Kubernetes? | ARMO
• linkedin.com/pulse: How to write YAML file for Kubernetes | Megha S.k
• dotnet-helpers.com: How to Convert YAML to JSON / JSON to YAML using PowerShell

Kubernetes examples

• Kubernetes examples 🌟 A series of YAML references with canonical and as-simple-as-possible demonstrations of kubernetes functionality and features.

Helm and Kustomize

• dex.dev: YAML Templating Solutions: Helm & Kustomize Writing config files by hand is like coding with Notepad instead of an IDE. Let’s find a better way, and take an overview of the popular solutions Helm & Kustomize.
• medium.com/geekculture: Convert Kubernetes YAML Files Into Helm Charts - Helmify

JSON. Templating JSON Files

• wikipedia: JSON
• json.org: Introducing JSON
• Jsonnet A data templating language for app and tool developers
• Building a high performance JSON parser
• json-schema.org: Understanding JSON Schema 🌟
• dzone.com: The Ultimate JSON Library: JSON.simple vs. GSON vs. Jackson vs. JSONP We ran a benchmark test in 2017 and again in 2021 to see how fast four of the most popular JSON libraries for Java parse different sizes of files.
• buger/jsonparser One of the fastest alternative JSON parser for Go that does not require schema
• dev.to: The JSON trick 25% of Python devs don’t know about Reading and writing JSON files in Python.
• dev.to: Convert nested JSON to simple JSON in Javascript
• pythonspeed.com: Processing large JSON files in Python without running out of memory
• thenewstack.io: Why (and How) You Should Manage JSON with SQL
• betterprogramming.pub: How to Make JSON and Python Talk to Each Other Processing and creating JSON data in Python
• betterprogramming.pub: Exploring JSON, JSON5, and Circular References An in-depth guide on JavaScript Object Notation (JSON)
• automationreinvented.blogspot.com: What is Json Schema and how to perform schema validation using Rest Assured?

JSONPath query language

• blog.mariano.cloud: Do you even JSONPath? JSONPath as in a JSON query language that enables you to interact with a JSON structured data set. In this article, you will learn how to use JSONPath expressions to parse and format responses from the Kubernetes API.

JSON in Ansible

• opensource.com: 5 ways to process JSON data in Ansible 🌟 Structured data is friendly for automation, and you can take full advantage of it with Ansible.

JSON formatting with jq

• about.gitlab.com: Tips for productive DevOps workflows: JSON formatting with jq and CI/CD linting automation
• jq -C '.' data.json | less -R Use jq to pretty print some JSON data with ANSI color coded syntax and use -R in less to process the color.
• github.com/ilyash/show-struct Shows possible jq paths in a JSON file
• github.com/01mf02/jaq A jq clone focussed on correctness, speed, and simplicity

JSON Tools

• jsoncrack.com: JSON Crack 🌟🌟 Seamlessly visualize your JSON data instantly into graphs
• github.com/tomnomnom/gron 🌟 Make JSON greppable!

Kubernetes JSON Schemas

• github: Kubernetes JSON Schemas 🌟 Schemas for every version of every object in every version of Kubernetes

Static Checking of Kubernetes YAML Files. Kubernetes YAML Validation Tools

• The ecosystem of static checking of Kubernetes YAML files can be grouped in the following categories:
  • API validators: Tools in this category validate a given YAML manifest against the Kubernetes API server.
  • Built-in checkers: Tools in this category bundle opinionated checks for security, best practices, etc.
  • Custom validators: Tools in this category allow writing custom checks in several languages such as Rego and Javascript.
• Validating Kubernetes YAML for best practice and policies 🌟 In this article, you will learn and compare six different tools:
  • Kubeval
  • Kube-score Kubernetes object analysis with recommendations for improved reliability and security. kube-score is a #opensource tool that performs static code analysis of Kubernetes object definitions. The output is a list of recommendations where one can improve to make application more secure and resilient.
  • Config-lint
  • Copper
  • Conftest
  • Polaris
• kubevious.io: Top Kubernetes YAML Validation Tools
• KubeLinter KubeLinter is a static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices.
  • thomasthornton.cloud: Analyze your Kubernetes YAML files and Helm Charts to ensure best practices using KubeLinter in Azure DevOps Pipeline
• medium.com/@jonathan_37674: How to validate Kubernetes YAML files? | ARMO Explore defining Kubernetes applications using YAML configs and the various steps we can take to effectively validate these config definitions.

Alternatives

• ketch - civo.com: Deployments without YAML using Ketch
  • ketch: Getting Started
  • github.com/shipa-corp/ketch Ketch is an application delivery framework that facilitates the deployment and management of applications on Kubernetes using a simple command line interface.
  • dzone.com: Use Ketch to Deploy Apps on Kubernetes Without YAML An open-source project for deploying and managing applications on Kubernetes with a command-line interface.
• shipa.io: DevOps Challenge – Kubernetes Deployment: Ketch vs YAML
• naml: Not another markup language Framework for replacing Kubernetes YAML with Go.
• jvns.ca: A list of new(ish) command line tools
  • jq (a great JSON-wrangling tool)
  • jc (convert various tools’ output into JSON)
  • jo (create JSON objects)
  • yq (like jq, but for YAML). there’s also another yq
  • fx (interactive json tool)
  • jless (json pager)
  • xsv (a command line tool for csv files, from burntsushi)
  • visidata (“an interactive multitool for tabular data”)
  • miller (“like awk/sed/cut/join/sort for CSV/TSV/JSON/JSON lines”)
• kislyuk/yq Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents. Worth noting that there are two implementations of yq, the other one being https://github.com/kislyuk/yq. It’s a simple® wrapper around jq in Python, so it does support everything jq does, as opposed to the Go implementation, which reimplements jq but probably not fully.

Base64

• base64encode.org Base64 Decode and Encode

Videos

Click to expand!

Tweets

Click to expand!

¿A quién se le ocurrió que sería buena idea el no tener comentarios en ficheros JSON?

— Coding Potions ⚗️ (@CodingPotions) February 4, 2022

Do you know all the YAML tricks and gotchas?

Are there any YAML tips for Kubernetes?

Read along! pic.twitter.com/BXszAVR4sd

— Daniele Polencic — @danielepolencic@hachyderm.io (@danielepolencic) January 30, 2023