Managing Vulnerabilities in Container Images and Kubernetes

Managing vulnerabilities in containers comes down to a few habits. Scan images before they ship and keep rescanning them in the registry. Start from small, well-maintained base images. Fix problems by rebuilding and redeploying rather than patching running containers. Then treat the Kubernetes cluster itself, meaning its component versions and configuration, as a separate layer with its own patching and hardening work. Admission control ties it together by stopping non-compliant images from running at all. This post walks through each piece and how they fit into a program a small team can actually run.

Why is container vulnerability management different?

Traditional vulnerability management assumes long-lived servers that you scan and patch in place. Containers break that model. Images are immutable, workloads come and go in minutes, and a single image may run as dozens of copies across several clusters.

Ownership is split, too. Application teams own their code and dependencies, a platform team usually owns base images and the cluster, and a cloud provider may own the control plane. A finding that nobody clearly owns tends to sit open.

It helps to think in three layers:

• Images: the operating system packages and application libraries baked into each image.
• Workloads: what is actually running, where, and with what privileges.
• The cluster: Kubernetes components, node operating systems, the container runtime and add-ons.

Each layer needs its own checks. Most programs start with images, which is sensible, but stopping there leaves real gaps.

How do you scan container images?

Image scanners unpack an image's layers, inventory the OS packages and language dependencies inside, and match them against vulnerability data. The question is where and when to run them.

Scan in CI before images are pushed

Scanning in the build pipeline gives developers feedback while the change is still fresh. Run the scan after the image is built and before it is pushed, and fail the build on clear policy violations, such as a critical vulnerability with a fix available. Generating an SBOM at the same step gives you an inventory you can query later without rescanning.

Keep the CI policy narrow at first. If every build fails on hundreds of unfixable low-severity findings, developers will learn to ignore or bypass the scanner.

Rescan continuously in the registry

A clean scan at build time only covers vulnerabilities known that day. New issues are published constantly, so images already in your registry need to be rescanned on a schedule or whenever vulnerability data updates.

Focus that effort on images that are actually deployed. Registries tend to fill up with old tags nobody runs, and findings in those images mostly create noise. Reference images by digest rather than a mutable tag, so you know exactly which scanned image is running.

What image scanners can miss

Scanners rely on package metadata. Binaries copied in with a download command, vendored code and statically compiled tools often have no package record, so a scanner may not see them. Build images from package managers where possible, and treat anything installed by hand as something to track separately.

How should you choose a base image?

In many images, most of the findings come from the base image rather than the application. Choosing it well is one of the highest-leverage decisions you can make.

Good base images share a few traits:

• They come from an official or trusted source with a clear maintainer.
• They are rebuilt often, so security updates land quickly.
• They contain only what the workload needs.
• They are pinned by digest in your Dockerfiles and updated deliberately.

Standardize on a short list of approved base images per language or runtime, and give each one an owner. That turns "update the base image" from a hundred separate tasks into one.

Minimal and distroless images

Minimal images strip out everything not needed to run the application. Distroless images go further and remove the shell and package manager, leaving only the application and its runtime dependencies. Fewer packages means fewer findings to triage and less for an attacker to work with if a container is compromised.

The trade-off is debugging. Without a shell, you can't exec into a container and poke around. Multi-stage builds help, since you can compile with a full toolchain and copy only the output into the final image. Kubernetes ephemeral containers, stable since version 1.25, let you attach a debugging container to a running pod when you need one.

Why rebuild instead of patching running containers?

Patching a running container, for example by running a package update inside it, fixes nothing for long. The change is lost as soon as the pod restarts, and the image in your registry is still vulnerable. It also creates drift, so what you scanned no longer matches what is running.

The fix for a container vulnerability is always the same pattern:

1. Update the base image or dependency in source.
2. Rebuild the image through the pipeline.
3. Scan the new image and confirm the finding is gone.
4. Redeploy through your normal release process.

Because base images receive updates even when your code doesn't change, rebuild on a regular cadence regardless of application releases. Automating base image update pull requests keeps this from depending on someone remembering to do it.

What is the difference between image findings and runtime findings?

Image scanning tells you what could be vulnerable. Runtime context tells you what is actually exposed. You need both to prioritize well.

A critical finding in an image that runs as root, with host access, behind a public load balancer is a very different problem from the same finding in an unused image. Runtime settings such as privileged mode, added Linux capabilities and host path mounts amplify any vulnerability in the image. Rank findings by combining the two views, and fix exposed, running workloads first.

How do you keep Kubernetes itself patched?

The cluster is software too, and it needs the same attention as the workloads. The components to track include:

• Control plane components: API server, controller manager, scheduler and etcd
• The kubelet and container runtime on every node
• The node operating system
• Add-ons such as the CNI plugin, ingress controllers, CoreDNS and operators installed with Helm charts

The upstream Kubernetes project maintains the three most recent minor releases, and each receives roughly a year of patch support. With about three minor releases a year, a cluster that isn't upgraded regularly falls out of support quickly. Plan minor version upgrades as routine work, not a project you start when something breaks.

Managed Kubernetes services handle much of the control plane for you, but they publish their own support calendars and often leave node upgrades and add-ons to you. Keep an inventory of what is installed in each cluster, including versions, because add-ons are easy to forget and often run with broad permissions.

How do you check cluster configuration against the CIS Kubernetes Benchmark?

Patching fixes known bugs. Configuration determines how much damage any single weakness can do. The CIS Kubernetes Benchmark is a widely used baseline for cluster configuration, covering areas such as:

• Control plane component settings, including API server flags
• etcd configuration
• Authentication, authorization and audit logging
• Worker node and kubelet configuration
• Policies for RBAC, service accounts, pod security, network policies and secrets

CIS also publishes separate benchmarks for major managed Kubernetes services, since you can't inspect or change their control planes. Open-source tools such as kube-bench can run the checks automatically. The NSA and CISA Kubernetes Hardening Guidance, first published in August 2021, is a useful companion with more explanation of the reasoning behind each control.

Common findings include anonymous access to the kubelet, broad cluster-admin role bindings, service account tokens mounted where they aren't needed, missing network policies and secrets not encrypted at rest. Run the benchmark on a schedule, not just once, because configuration drifts as teams add workloads and add-ons.

How does admission control block non-compliant images?

Admission controllers intercept requests to the Kubernetes API before objects are stored, and can reject or modify them. That makes them the natural enforcement point for container security policy.

Kubernetes includes Pod Security Admission, stable since version 1.25, which enforces the Pod Security Standards and replaced the removed PodSecurityPolicy feature. For image-level rules you'll typically add a policy engine, such as the open-source OPA Gatekeeper or Kyverno projects, running as an admission webhook. Kubernetes 1.26 also introduced ValidatingAdmissionPolicy as an alpha feature, which is worth watching but not yet something to depend on.

Useful admission policies include:

• Only allow images from approved registries.
• Require images to be referenced by digest, not a tag like latest.
• Require a valid signature from your build pipeline.
• Reject images without a recent scan, or with critical fixable vulnerabilities.
• Block privileged containers, host namespaces and running as root.

Roll policies out in audit or warn mode first, see what would have been blocked, then switch to enforce. Give exceptions an owner and an expiry date so they don't become permanent.

Frequently asked questions

How often should container images be rebuilt?

At least as often as your base images receive security updates, and on a fixed schedule even when application code hasn't changed. A weekly rebuild, or one triggered by every base image update, is a reasonable starting point.

Should we scan images in CI or in the registry?

Both. CI scanning catches problems before deployment, when they are cheapest to fix. Registry scanning catches vulnerabilities disclosed after an image was built, which CI can't see.

Do distroless images mean no vulnerabilities?

No. They reduce the number of packages, which usually means far fewer findings, but the application's own dependencies and the remaining runtime libraries can still be vulnerable. They still need scanning and rebuilding.

Does a managed Kubernetes service handle patching for us?

Partly. The provider typically patches the control plane, but node upgrades, add-ons, workload configuration and the timing of minor version upgrades often remain your responsibility. Check the shared responsibility model for your specific service.

Key takeaways

• Scan images in CI and keep rescanning deployed images in the registry.
• Standardize on a small set of minimal, well-maintained base images with named owners.
• Fix by rebuilding and redeploying, never by patching running containers.
• Prioritize image findings using runtime context: running, exposed and privileged workloads first.
• Track Kubernetes and add-on versions, and upgrade before releases fall out of support.
• Check cluster configuration against the CIS Kubernetes Benchmark on a schedule.
• Use admission control to enforce the rules, starting in audit mode.