Explaining Anti-Affinity in Kubernetes

The aim of this page📝 is to explain the concept of pod anti-affinity in Kubernetes, based on the particular example of a workload (referred to as example_app1) on Google Kubernetes Engine (GKE). We used this concept as part of incident resolution when they example_app1 started to throw high-CPU-related errors and therefore malperforming and thereforing causing lags in data delivery. The example_app1 functions as a real-time data loader. What we did is to ensure that the app is not running on the same node as the other 2 CPU-heavy apps we run in the same k8s cluster.

• Pod anti-affinity is a scheduling concept in Kubernetes.
• It ensures that certain pods are not scheduled on the same node.
• This enhances the reliability and availability of applications.
• In the context of GKE, pod placement can be controlled using Kubernetes mechanisms such as affinity and anti-affinity.
• Anti-affinity rules are defined in the pod’s YAML file.
• The requiredDuringSchedulingIgnoredDuringExecution field indicates that the specified anti-affinity rules must be met when scheduling the pod.
• The labelSelector field is used to select pods based on their labels.
• The matchExpressions field specifies the conditions that must be met for the label selector.
• The key field specifies the label key that the rule applies to.
• The operator field specifies the condition that the label's value must meet.
• The values field specifies the list of values that the label's value must be in.
• The namespaces field specifies the namespaces that the rule applies to.
• The topologyKey field specifies the node attribute that the rule applies to.
• In the provided example, the example_app1 workload should not be scheduled on the same node as the example_app2 and example_app3 workloads.

YAML

Here is the particular part of the pod spec:

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app
          operator: In
          values:
          - example_app2
          - example_app3
      namespaces:
      - example-prod1
      - prod1
      topologyKey: kubernetes.io/hostname

This configuration tells Kubernetes to avoid scheduling any pod (that uses this configuration) on the same node as any other pod that has a label with key “app” and value “example_app2” or “example_app3”, and is in the “example-prod1” or “prod1” namespaces.

LINKS

• https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
• https://cloud.google.com/kubernetes-engine/docs/how-to/pod-topology-spread-constraints
• https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
• https://kubernetes.io/docs/concepts/configuration/overview/
• https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
• https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
• https://kubernetes.io/docs/concepts/architecture/nodes/