Deploying an application to Kubernetes is only the beginning of modern cloud engineering. In day-to-day production environments, cloud architects and site reliability engineers (SREs) must continuously adapt to shifting user traffic, release new software versions without service interruptions, instantly revert faulty updates, and quickly diagnose application issues through live container logs.
In this hands-on guide, you will master the four essential operational pillars of Azure Kubernetes Service (AKS):
- On-Demand Pod Scaling: Dynamically scaling application replicas up and down via Kubectl and the Azure Management Portal.
- Zero-Downtime Rolling Updates: Deploying new container image releases incrementally while maintaining 100% service uptime.
- Instant Rollbacks: Reverting a failed deployment rollout to a previously healthy state in seconds.
- Log Inspection & Pod Health Diagnostics: Streaming live container logs, querying historical pod crashes, and diagnosing failure events with
kubectl describe.
📌 Lab Prerequisites: To follow along, you need an active Azure Subscription and a running AKS cluster with Azure CLI and kubectl connected locally. If you need to provision a cluster and deploy the base web application first, refer to our foundational guide: Deploy Your First Containerized App on Azure Kubernetes Service (AKS).
📦 GitHub Lab Repository & Declarative Manifests
While this guide focuses on imperative CLI commands for fast incident response, you can find the complete declarative YAML manifests (v1, v2, v3) in our official repository:
👉 StackLayerOfficially/azure-aks-hands-on-labs (02-scaling-rolling-updates)
⚠️ Disclaimer & Cloud Cost Notice
This tutorial is built strictly for hands-on educational and lab purposes. The configurations and manifests shown here are simplified demonstrations and should not be used directly in live production or enterprise environments without independent security reviews. Stack Layer and its technical team assume no responsibility or liability for any issues, outages, or data loss resulting from applying these commands. Always follow Step-05 to pause or delete your cluster when finished to protect your cloud balance.
Understanding the Core Concepts
1. Imperative Commands vs. Declarative Manifests
In Kubernetes, there are two distinct ways to manage resources:
- Imperative Commands (
kubectl scale,kubectl set image): Telling Kubernetes what action to execute right now. Imperative commands are ideal for ad-hoc troubleshooting, live inspection, and emergency rollbacks during production incidents. - Declarative Manifests (YAML files): Defining the desired state in version-controlled configuration files. Declarative configurations are standard for long-term project stability and automated workflows.
2. What is Horizontal Pod Autoscaling & Manual Scaling?
In Kubernetes, Scaling refers to increasing or decreasing the number of identical running container instances (replicas) of your application. When website traffic surges, Kubernetes allows you to spin up multiple replicas across cluster worker nodes in seconds. The front-facing Azure Load Balancer automatically registers these new pods as backend endpoints and balances incoming web traffic evenly across them.
3. What is a Zero-Downtime Rolling Update?
A Rolling Update is Kubernetes' default deployment strategy designed to eliminate application downtime during new software releases or security patches. Instead of terminating all existing pods at once (which causes 502/504 errors for active users), Kubernetes adopts a staged transition: it boots up a new container replica with the updated image, runs readiness health probes to verify it is healthy, and only then terminates an older replica. This ensures that your application remains 100% available throughout the release cycle.
4. What is an Instant Rollback?
If a newly released container image introduces a fatal software bug, database connectivity timeout, or crash, Kubernetes maintains a revision history of your deployment controllers. With an Instant Rollback (kubectl rollout undo), you can immediately instruct Kubernetes to reverse the deployment process, restoring the previously healthy pods in seconds without waiting for a manual code rebuild or re-compilation.
5. What is Container Log Inspection & Pod Diagnostics?
Unlike traditional virtual machines where application logs are stored in static syslog files on a local hard drive, container logs in Kubernetes stream directly to standard output (stdout) and standard error (stderr). Using kubectl logs, you can stream real-time HTTP requests, inspect crash stack traces using the --previous flag, and utilize kubectl describe to query the cluster event plane to diagnose issues like ImagePullBackOff, failed health probes, or insufficient node memory.
Connect Your Local CLI to AKS
Before executing cluster operations, ensure your local terminal is authenticated to your running cluster:
[code] # Connect Azure CLI to your running AKS cluster az aks get-credentials --resource-group stacklayer-aks-rg --name stacklayer-aks-cluster --overwrite-existing [/code]Command Breakdown:
az aks get-credentials: Retrieves cluster access certificates and merges the API endpoint into your local~/.kube/configfile.--resource-group stacklayer-aks-rg: Target Azure Resource Group hosting your AKS cluster.--name stacklayer-aks-cluster: The exact name of your AKS cluster.--overwrite-existing: Overwrites any expired or conflicting local kubeconfig contexts.
Step-01: Verify Your Running Workload
Before modifying workload states, inspect your existing web application deployment to ensure it is healthy and accepting traffic:
[code] # Check existing deployment controllers kubectl get deployments # List running pods with node placement details kubectl get pods -o wide [/code]Command Breakdown:
kubectl get deployments: Confirms that your deployment controller reports ready and available replicas.kubectl get pods -o wide: Displays extended details including pod IP, node host assignment, and current container status. Note a pod name for log inspection in later steps.
Step-02: Scale Your Application on Demand
When user traffic surges, Kubernetes allows you to scale up multiple container replicas across your cluster nodes in seconds.
Method A: Scale via Kubectl CLI (Fastest)
Scale your deployment up to 5 replicas:
[code] # Scale your deployment to 5 replicas kubectl scale deployment stacklayer-web-deployment --replicas=5 # Watch the new pods spin up across cluster nodes in real time kubectl get pods -l app=stacklayer-web -w [/code]Command Breakdown:
kubectl scale deployment stacklayer-web-deployment --replicas=5: Updates the desired replica count in the deployment spec. The ReplicaSet controller detects the deficit and instantly schedules 3 new pods.-l app=stacklayer-web: Filters the output to display only pods matching the label selectorapp=stacklayer-web.-w(Watch): Streams real-time pod lifecycle changes fromContainerCreatingtoRunning. PressCtrl + Conce all 5 pods are running.
Your Azure Standard Load Balancer automatically detects the new endpoints and distributes incoming HTTP traffic evenly across all 5 running instances.
Method B: Scale from the Azure Management Portal
If you prefer using the Azure Portal web interface:
- Navigate to your AKS cluster in the Azure Portal.
- In the left sidebar under Kubernetes resources, click Workloads.
- Select your
stacklayer-web-deploymentdeployment. - Click Scale workload in the top toolbar, update the replica count to
2, and click Apply.
Alternatively, scale back down via the terminal when traffic subsides:
[code] # Scale back down to 2 replicas when traffic returns to normal kubectl scale deployment stacklayer-web-deployment --replicas=2 [/code]Command Breakdown:
kubectl scale deployment stacklayer-web-deployment --replicas=2: Gracefully terminates 3 redundant pods, de-allocating node memory and CPU resources while maintaining 2 active pods for baseline traffic.
Step-03: Perform a Zero-Downtime Rolling Update
When releasing a new application version or security patch, service interruption must be avoided. Kubernetes uses a Rolling Update strategy by default: it spins up a new pod with the updated image, verifies its health readiness, and only then terminates an older pod. This process repeats until all pods run the new release.
Let's simulate upgrading our web server image from stacklayer/kubernetes:nginx-v1 to stacklayer/kubernetes:nginx-v2:
Command Breakdown:
kubectl set image deployment/stacklayer-web-deployment stacklayer-web-container=stacklayer/kubernetes:nginx-v2: Directly updates the container image definition inside the running deployment without modifying local YAML files.--record: Saves the applied command inside the deployment's revision history, enabling clear change tracking.kubectl rollout status deployment/stacklayer-web-deployment: Blocks the terminal and outputs real-time progression: "Waiting for deployment rollout to finish: 1 out of 2 new replicas have been updated..." until completion.
Inspect the Deployment Rollout History
View the documented revision history of your deployment:
[code] # View revision history and applied changes kubectl rollout history deployment/stacklayer-web-deployment [/code]Command Breakdown:
kubectl rollout history deployment/stacklayer-web-deployment: Displays the sequential revisions (e.g.,REVISION 1,REVISION 2) and the recorded commands associated with each rollout.
Emergency Rollback: Reverting a Failed Update Instantly
If a new release contains a fatal bug or memory leak, Kubernetes allows you to revert to the previous working release with a single command:
[code] # Undo the previous rollout and revert to the earlier working revision kubectl rollout undo deployment/stacklayer-web-deployment # Verify that the deployment restored the previous healthy pods kubectl rollout status deployment/stacklayer-web-deployment [/code]Command Breakdown:
kubectl rollout undo deployment/stacklayer-web-deployment: Instructs Kubernetes to reverse the rollout strategy, restoring the previous ReplicaSet and its healthy container images in seconds.
Step-04: Troubleshoot Pods & View Live Container Logs
When an application behaves unexpectedly or fails readiness checks, Kubernetes provides deep observability tools directly from the command line.
1. Stream Live Container Logs
Identify one of your running pod names and stream its standard output (stdout/stderr):
[code] # Fetch the latest 50 lines of logs from a specific pod kubectl logsCommand Breakdown:
kubectl logs <pod-name> --tail=50: Reads the container runtime log buffer and outputs the 50 most recent lines.-f(Follow): Keeps the log stream open in your terminal, outputting new application requests as they occur. PressCtrl + Cto exit.
2. Inspect Crashed or Restarted Containers
If a container crashes and restarts, standard kubectl logs only shows the new instance. To view the error logs from the container right before it crashed, use the --previous flag:
3. Deep-Dive Diagnostics with Kubectl Describe
If a pod remains stuck in Pending, CrashLoopBackOff, or ImagePullBackOff, kubectl describe exposes the underlying cluster event log:
Command Breakdown:
kubectl describe pod <pod-name>: Reveals the container state, resource allocations, mounted volumes, and the critical Events table at the bottom (which shows exact reasons for image pull failures, scheduling errors, or failed liveness probes).
Step-05: Clean Up Resources & Manage Cloud Costs
To ensure you do not incur ongoing compute charges after completing your practice, choose one of the two options below:
First: Delete the Application Workload
[code] # Delete deployed application workloads kubectl delete -f kube-manifests/ [/code]Command Breakdown:
kubectl delete -f kube-manifests/: Shuts down the pod containers and de-provisions the Azure Load Balancer public IP address.
Option A: Stop the Cluster (Recommended If Practicing Later Labs)
If you plan to follow additional hands-on labs, you can stop the cluster to pause VM compute billing while keeping all cluster configurations intact:
[code] # Stop the AKS cluster to pause compute billing az aks stop --resource-group stacklayer-aks-rg --name stacklayer-aks-cluster # When you are ready to resume, start it back up: az aks start --resource-group stacklayer-aks-rg --name stacklayer-aks-cluster [/code]Command Breakdown:
az aks stop: Powers down all virtual machine worker nodes and halts hourly compute billing immediately. All cluster configurations and network topologies are preserved.az aks start: Resumes cluster operations and wakes up worker nodes in approximately 2–3 minutes.
Option B: Delete the Cluster Completely (Permanent Teardown)
If you are finished experimenting and want to ensure 100% zero future charges:
[code] # Permanently delete the resource group and all contained resources az group delete --name stacklayer-aks-rg --yes --no-wait [/code]Command Breakdown:
az group delete --name stacklayer-aks-rg: Permanently deletes the resource groupstacklayer-aks-rg, cluster, virtual networks, and managed disks.--yes: Bypasses interactive confirmation prompts.--no-wait: Executes asynchronously in the Azure background so you can close your terminal immediately.
Summary
In this guide, you mastered the day-to-day operational core of running workloads on Azure Kubernetes Service:
- Understood the core distinctions between imperative troubleshooting commands and declarative YAML manifests.
- Scaled application pods dynamically using both
kubectl scaleand the Azure Portal web GUI. - Executed a zero-downtime rolling update to release container version
nginx-v2. - Documented revision history with
--recordand executed an instant emergency rollback withkubectl rollout undo. - Streamed real-time container logs using
kubectl logs -fand diagnosed crashes using--previousandkubectl describe. - Managed resource lifecycles with
az aks stopandaz group deleteto prevent unnecessary cloud costs.
“Did you successfully practice scaling and zero-downtime updates on your AKS cluster?”
If yes, leave a comment below! If you ran into any issues or have questions, feel free to drop them below as well.
Discussions