← Back to the library
OperationsHow it works · Core · 14 min

Kubernetes basics, from kubectl apply to a ready Pod

What does Kubernetes actually do with the YAML you apply, and which objects do you need to run a web service on it?

Kubernetes stores your desired state as API objects and runs controllers that keep nudging the real world toward it. A web service is usually a Deployment (Pods via ReplicaSets), a Service for a stable address, a Gateway API route for external HTTP, probes that gate traffic and restarts, and requests/limits that drive scheduling.

THE MENTAL MODEL

Treat the cluster as a database of intentions plus a swarm of reconcile loops. kubectl apply only writes a record through the API server into etcd; nothing runs at that moment. Controllers watch for differences between spec and status and act: the Deployment controller makes a ReplicaSet, the ReplicaSet controller makes Pods, the scheduler picks a node for each Pod, the kubelet on that node starts containers and reports back, and the EndpointSlice controller adds the Pod behind its Service once readiness passes. Every object is level-triggered, so a deleted Pod is not an error to handle but a gap the loop fills. Most debugging is reading which loop is stuck and why: unschedulable (requests), not ready (probes), restarting (liveness, OOM), or not receiving traffic (selectors, routes).

HOW IT FITS TOGETHER

What happens after kubectl applyEach arrow is a different component watching the API server. None of them call each other directly.
  1. kubectl applySends the manifest to the API server (client-side apply by default, or --server-side). Returns as soon as the object is stored.
  2. kube-apiserverAuthenticates, authorizes (RBAC), runs admission (defaults, policy, quotas) and validates the object.
  3. etcdPersists the object. This is the only stateful piece of the control plane; everything else rebuilds its view from watches.
  4. Deployment controllerSees a new pod template hash, creates or scales a ReplicaSet, and walks the rollout within maxSurge / maxUnavailable.
  5. ReplicaSet controllerCreates Pod objects until the observed count matches replicas. The Pods exist but have no node yet.
  6. kube-schedulerFilters nodes that fit the Pod's requests, taints and affinity, scores the rest, and binds the Pod to one.
  7. kubelet + runtimePulls images and starts containers through the CRI (containerd, CRI-O), then runs startup, liveness and readiness probes.
  8. EndpointSlice updateOnce readiness passes, the Pod IP is added to the Service's EndpointSlices and starts receiving traffic.
The objects behind one HTTP serviceTop is closest to the user's request, bottom is the machine. Labels and selectors, not names, connect most layers.
  1. Gateway + HTTPRouteGateway is the listener (hostname, port, TLS) owned by the platform team; HTTPRoute maps paths and headers to Services and is owned by the app team.
  2. Service (ClusterIP)Stable virtual IP and DNS name (app.ns.svc.cluster.local) that selects Pods by label and load-balances over ready endpoints.
  3. DeploymentDesired replicas, pod template and rollout strategy. Keeps revision history for rollout undo.
  4. ReplicaSetOne per pod template revision. Owned by the Deployment; never edit it directly.
  5. PodSmallest schedulable unit: one or more containers sharing a network namespace, IP and volumes. Disposable and replaced, never repaired.
  6. Container + ConfigMap/SecretThe image, its env and mounted config. Requests and limits are set per container.
  7. NodeA VM or machine running the kubelet and a container runtime, whose capacity the scheduler allocates by requests.

KEY TERMS

Reconciliation loop
Controllers repeatedly compare desired state (spec) with observed state (status) and take one step to close the gap. This is why manual fixes to Pods are undone and why deleting a misbehaving Pod is a valid remedy: the ReplicaSet simply creates another.
Pods, ReplicaSets, Deployments
A Pod is a group of containers on one node with one IP. A ReplicaSet keeps N identical Pods alive. A Deployment owns ReplicaSets and performs rolling updates by scaling a new one up and the old one down (defaults maxSurge 25%, maxUnavailable 25%, revisionHistoryLimit 10).
Services, Ingress and Gateway API
A Service gives a label-selected set of Pods a stable name and virtual IP. For external HTTP, the Kubernetes docs now recommend Gateway API: the Ingress API is frozen (still GA, not being removed, but getting no new features), and the community Ingress NGINX controller was retired in March 2026 with no further releases or security patches.
Probes
Startup probes hold off the other probes until a slow app has booted. Liveness failure restarts the container, so it should check only the process itself. Readiness failure removes the Pod from Service endpoints without restarting it, so dependency checks belong there.
Requests and limits
Requests are what the scheduler reserves and what HPA CPU utilization is measured against. Limits are enforced at runtime: exceeding a CPU limit throttles, exceeding a memory limit gets the container OOMKilled. Requests equal to limits for every container gives the Guaranteed QoS class.
ConfigMaps and Secrets
Key/value objects injected as env vars or mounted files. Secrets are only base64-encoded and stored unencrypted in etcd unless encryption at rest is configured; RBAC on get/list secrets is the real boundary. Env vars are read once at container start; mounted volumes refresh eventually, except subPath mounts.
HorizontalPodAutoscaler
Every 15 s by default it computes desired = ceil(current × metric / target), using metrics-server for CPU and memory. Scale-up is immediate; scale-down uses a 300 s stabilization window. Remove spec.replicas from the Deployment manifest, or every apply resets the count.

IN YOUR STACK

YAML · apps/v1 Deployment Server

apiVersion: apps/v1
kind: Deployment
metadata: { name: api, labels: { app: api } }
spec:
  # no replicas: the HPA owns the count
  selector: { matchLabels: { app: api } }
  strategy: { rollingUpdate: { maxSurge: 25%, maxUnavailable: 0 } }
  template:
    metadata: { labels: { app: api } }
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          image: ghcr.io/acme/api:1.4.2 # immutable tag or digest, never :latest
          ports: [{ name: http, containerPort: 8080 }]
          resources:
            requests: { cpu: 250m, memory: 256Mi }
            limits: { memory: 512Mi } # memory limit; CPU limit often omitted to avoid throttling
          startupProbe:
            httpGet: { path: /healthz, port: http }
            failureThreshold: 30
            periodSeconds: 2
          livenessProbe: { httpGet: { path: /healthz, port: http } } # process only
          readinessProbe: { httpGet: { path: /ready, port: http }, periodSeconds: 5 } # may check deps
          securityContext: { runAsNonRoot: true, allowPrivilegeEscalation: false }
  • maxUnavailable: 0 keeps full capacity during a rollout at the cost of one surge Pod's worth of spare room on the nodes.
  • The startup probe gives the app up to 60 s (30 × 2 s) to boot before liveness starts; without it, a slow boot under load turns into a restart loop.
  • Handle SIGTERM: Kubernetes removes the Pod from endpoints and sends SIGTERM concurrently, so finish in-flight requests within the grace period.
apps/v1 Deployment documentation

YAML · Service + Gateway API v1 Server

apiVersion: v1
kind: Service
metadata: { name: api }
spec:
  selector: { app: api } # must match the Pod template labels
  ports: [{ name: http, port: 80, targetPort: http }]
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: { name: api }
spec:
  parentRefs:
    - name: public-gateway # Gateway owned by the platform team
      namespace: infra
  hostnames: ["api.example.com"]
  rules:
    - matches: [{ path: { type: PathPrefix, value: /v1 } }]
      backendRefs: [{ name: api, port: 80 }]
  • HTTPRoute joined the Standard channel in Gateway API v0.5.0 and is GA (gateway.networking.k8s.io/v1) since v1.0. Gateway API CRDs are installed separately from Kubernetes, and you need an implementation (Envoy Gateway, Istio, Cilium, a cloud load balancer controller and others).
  • Attaching a route to a Gateway in another namespace must be allowed by that Gateway's listener allowedRoutes, which is how platform and app teams split ownership.
  • The ingress2gateway tool converts existing Ingress manifests; most nginx.ingress.kubernetes.io annotations have no direct equivalent and need mapping to HTTPRoute filters or policies.
Service + Gateway API v1 documentation

YAML · autoscaling/v2 HPA Server

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 70 } # % of the CPU request
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300 # the default, written out
      policies: [{ type: Percent, value: 50, periodSeconds: 60 }]
  • Utilization is relative to requests. If any container lacks a CPU request, utilization is undefined and the HPA takes no action for that metric.
  • Needs metrics-server (metrics.k8s.io) for CPU and memory; custom or external metrics need an adapter such as one backed by Prometheus.
  • Scaling Pods multiplies database connections and downstream calls. Check pool sizes and dependency limits at maxReplicas, not at today's count.
autoscaling/v2 HPA documentation

YAML · ConfigMap + Secret Server

apiVersion: v1
kind: ConfigMap
metadata: { name: api-config }
data:
  LOG_LEVEL: info
  features.json: '{"newCheckout": false}'
---
# Pod template excerpt
containers:
  - name: api
    envFrom: [{ configMapRef: { name: api-config } }] # read once at start
    env:
      - name: DATABASE_URL
        valueFrom: { secretKeyRef: { name: api-db, key: url } }
    volumeMounts:
      - { name: config, mountPath: /etc/api, readOnly: true } # refreshed by kubelet
volumes:
  - name: config
    configMap: { name: api-config, items: [{ key: features.json, path: features.json }] }
  • Changing a ConfigMap does not restart anything. Env vars stay stale until the Pod restarts; a common pattern is to put a hash of the config in a pod template annotation so a change triggers a rollout.
  • Create the Secret out of band (External Secrets Operator, Sealed Secrets, CSI secrets store) rather than committing base64 values, which are trivially decoded.
  • Mark ConfigMaps and Secrets immutable: true when they never change; it protects against accidental edits and reduces API server watch load.
ConfigMap + Secret documentation

Shell · kubectl Server

# Why is it not running? Events and conditions explain most failures.
kubectl get pods -l app=api -o wide
kubectl describe pod api-7d9c6b-x2k4p        # Pending: Insufficient cpu? ImagePullBackOff?
kubectl events --for pod/api-7d9c6b-x2k4p

# Crash loops: the current container may have no useful logs yet
kubectl logs api-7d9c6b-x2k4p -c api --previous

# Not receiving traffic: does the Service select any ready Pods?
kubectl get endpointslices -l kubernetes.io/service-name=api
kubectl get httproute api -o jsonpath='{.status.parents[*].conditions}'

# Shell into a distroless container via an ephemeral debug container
kubectl debug -it api-7d9c6b-x2k4p --image=busybox:1.36 --target=api -- sh

# Rollouts
kubectl rollout status deployment/api
kubectl rollout undo deployment/api   # back to the previous ReplicaSet

# Are you on the retired Ingress NGINX controller?
kubectl get pods -A --selector app.kubernetes.io/name=ingress-nginx
  • Last State: Terminated, Reason: OOMKilled in describe output means the memory limit was hit; exit code 137 alone only says SIGKILL.
  • --target shares the process namespace of the named container, so ps and /proc/1/root show the app even when its image has no shell.
  • An empty EndpointSlice almost always means a label selector mismatch or every Pod failing readiness.
kubectl documentation

YAML · Kustomize overlay Server

# overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: api-prod
resources:
  - ../../base # deployment.yaml, service.yaml, httproute.yaml, hpa.yaml
images:
  - name: ghcr.io/acme/api
    digest: sha256:4f1c... # pinned by CI on promotion
configMapGenerator:
  - name: api-config
    behavior: merge
    literals: [LOG_LEVEL=warn] # generated name gets a content hash, so changes roll Pods
patches:
  - target: { kind: HorizontalPodAutoscaler, name: api }
    patch: |-
      - op: replace
        path: /spec/minReplicas
        value: 6
  • Built into kubectl: kubectl apply -k overlays/production, or kubectl kustomize to render and review the output first.
  • Kustomize patches plain YAML; Helm templates it and adds release history and values. Many teams use Helm for third-party charts and Kustomize for their own services.
  • Rendering in CI and applying with a GitOps controller (Argo CD, Flux) keeps the cluster reconciled to what is in git, the same loop idea one level up.
Kustomize overlay documentation

WHEN TO REACH FOR WHAT

Managed Kubernetes (EKS, GKE, AKS)

Use when Many services and teams, a platform team to own the cluster, mixed workloads (APIs, workers, cron, stateful add-ons), or a need for portability and a large ecosystem of operators.

Avoid when A handful of stateless services and no one whose job includes upgrades, networking, RBAC and cost tuning. The control plane is managed; everything you install on it is not.

PaaS (Cloud Run, Fly.io, Render, Heroku)

Use when You want deploy-from-image or git, autoscaling and TLS with almost no operations, and your services are HTTP or simple workers.

Avoid when You need custom networking, sidecars, GPUs at scale, long-lived stateful workloads, or fine-grained control the platform does not expose.

Serverless functions (Lambda, Workers)

Use when Event-driven or spiky traffic, scale to zero matters, and handlers are short and stateless.

Avoid when Long-running connections or jobs, heavy startup, or steady high load where per-invocation pricing and cold starts cost more than reserved capacity.

VMs with a process manager

Use when One or two services, predictable load, or software that expects a whole machine. Simple images, systemd and a load balancer are easy to reason about.

Avoid when Many services sharing hardware, frequent deploys needing safe rollouts, or self-healing across machines; you end up rebuilding a scheduler by hand.

WHERE IT BITES

  • Liveness probes that check the database. When the database blips, every Pod fails liveness and restarts at once, adding load and turning a dependency outage into a full outage. Check dependencies in readiness only.
  • No requests, or requests far below real usage. The scheduler overpacks nodes, Pods get evicted or OOMKilled under pressure, and HPA CPU targets are meaningless.
  • Keeping spec.replicas in a manifest managed by an HPA: every kubectl apply or GitOps sync resets the replica count and fights the autoscaler.
  • Treating Secrets as encrypted. They are base64 in etcd by default, and anyone who can create a Pod in the namespace can mount them. Enable encryption at rest and restrict who can read Secrets or create Pods.
  • Staying on the retired Ingress NGINX controller. It still runs, but has received no security patches since March 2026; plan a move to a Gateway API implementation or a maintained Ingress controller.
  • Adopting Kubernetes for two stateless services. The cluster, its add-ons (ingress or gateway, cert-manager, metrics, logging) and their upgrades become the largest system you operate.

CLOSE THE AI. EXPLAIN THIS.

You kubectl apply a new image tag and the rollout never finishes. Walk the flow from API server to EndpointSlice and name what you would check at each stage, and which kubectl command shows it.

SOURCES

Explainer reviewed