Skip to content

6.19 Custom Resource Definitions (CRD) & Custom Controllers

Abstract

Custom Resource Definitions (CRDs) extend the Kubernetes API by allowing you to create your own resource types.

These resources behave like native Kubernetes objects and are stored in etcd.

Custom Controllers implement the logic that reacts to those resources and perform automation to reach the desired state.

Kubernetes Resource Model

Kubernetes resources follow a common architecture:

Component Description
Resource Object definition (Deployment, Pod, etc.)
etcd Persistent cluster datastore
Controller Watches resources and reconciles state

Workflow:

  1. User creates a resource
  2. API server stores it in etcd
  3. Controller watches the resource
  4. Controller performs actions to match desired state

Example: Deployment Resource

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment

spec:
  replicas: 3

  template:
    metadata:
      labels:
        type: front-end

    spec:
      containers:
      - name: nginx
        image: nginx

  selector:
    matchLabels:
      type: front-end

Apply resource:

kubectl create -f deployment.yaml
kubectl get deployments
kubectl delete -f deployment.yaml

Note

Creating a resource only stores the object in etcd. Controllers enforce the actual cluster state.

Controllers in Kubernetes

Controllers continuously monitor resources and ensure desired state.

Resource Controller
ReplicaSet ReplicaSet Controller
Deployment Deployment Controller
Job Job Controller
CronJob CronJob Controller
StatefulSet StatefulSet Controller

Tip

Controllers implement Kubernetes' reconciliation loop.

Extending Kubernetes with Custom Resources

Sometimes applications require custom resource types.

Examples:

  • Database clusters
  • ML jobs
  • Backup resources
  • External service integrations

Example custom resource:

apiVersion: flights.com/v1
kind: FlightTicket

metadata:
  name: my-flight-ticket

spec:
  from: Mumbai
  to: London
  number: 2

If created directly, Kubernetes returns:

no matches for kind "FlightTicket" in version "flights.com/v1"

Warning

Kubernetes only allows resources that are registered in the API.

Custom Resource Definition (CRD)

CRDs register new resource types with Kubernetes.

Once registered, Kubernetes supports:

  • create
  • get
  • list
  • update
  • delete

CRD Example

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition

metadata:
  name: flighttickets.flights.com

spec:
  group: flights.com
  scope: Namespaced

  names:
    kind: FlightTicket
    singular: flightticket
    plural: flighttickets
    shortNames:
      - ft

  versions:
  - name: v1
    served: true
    storage: true

    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              from:
                type: string
              to:
                type: string
              number:
                type: integer
                minimum: 1
                maximum: 10

Create CRD:

kubectl create -f flightticket-crd.yaml

Creating Custom Resources

After CRD creation:

apiVersion: flights.com/v1
kind: FlightTicket

metadata:
  name: my-flight-ticket

spec:
  from: Mumbai
  to: London
  number: 2

Commands:

kubectl create -f flightticket.yaml
kubectl get flighttickets
kubectl delete flightticket my-flight-ticket

Note

The resource is stored in etcd, but no action happens without a controller.

Custom Controllers in Kubernetes

Abstract

Custom Controllers implement the logic that reacts to Custom Resources (CRDs).

They continuously watch the cluster state and perform actions to make the actual state match the desired state.

Why Custom Controllers Are Needed

When a Custom Resource (CR) is created, Kubernetes only stores it in etcd.

Example:

apiVersion: flights.com/v1
kind: FlightTicket

metadata:
  name: my-flight-ticket

spec:
  from: Mumbai
  to: London
  number: 2

Commands:

kubectl create -f flightticket.yaml
kubectl get flightticket

Note

The object is stored in etcd, but Kubernetes does nothing with it unless a controller processes it.

Controller Workflow

Typical flow:

FlightTicket CR → API Server → ETCD → Custom Controller → External API

Example API call: 

https://book-flight.com/api

Controller Responsibilities

Controllers implement the reconciliation loop.

Responsibilities:

  • Watch resources
  • Detect changes
  • Execute automation logic
  • Update status fields

Tip

Controllers must be idempotent — running the controller multiple times should not cause inconsistent results.

Example Controller Logic

package flightticket

var controllerKind = apps.SchemeGroupVersion.WithKind("FlightTicket")

func (dc *FlightTicketController) Run(workers int) {
    // watch resource events
}

func (dc *FlightTicketController) callBookFlightAPI(obj interface{}) {
    // call booking API
}

Deploying Controllers in Production

Controllers should be packaged as Docker images and deployed inside Kubernetes.

Typical deployment:

Custom Controller → Docker Image → Kubernetes Deployment → Controller Pod

Success

Running controllers inside the cluster ensures high availability, scalability, and automation.

Best Practices for Production Controllers

Recommended

  • Use client-go or controller-runtime
  • Implement shared informers
  • Use work queues
  • Ensure controllers are idempotent
  • Implement retry logic
  • Update status fields instead of spec

Security Considerations

Danger

Controllers interact with the Kubernetes API and may have powerful permissions.

Mitigation:

  • minimal RBAC permissions
  • input validation
  • rate limiting
  • resource quotas

Summary

Quote

  • CRDs extend the Kubernetes API
  • Custom Controllers implement automation
  • Controllers follow the reconciliation loop
  • Controllers run as containerized workloads in production
  • CRD + Controller = Kubernetes Operator Pattern