Designing Secure, Minimal, and Multi-Architecture Container Images for Kubernetes
1. Introduction
Modern DevOps practices demand container images that are:
- Secure by default
- Minimal in size
- Reproducible
- Multi-architecture (amd64 + arm64)
- Operationally maintainable
Achieving all of these simultaneously requires a deliberate architecture—not just well-written Dockerfiles, but a layered image strategy, strong build discipline, and a clear separation of concerns.
This article presents a production-grade approach based on:
- Layered image architecture (4 layers)
- Distroless runtime strategy
- Multi-stage and multi-arch builds
- Strong hardening practices
- Tooling separation
- Debugging strategies without compromising security
The ideal structure for our case:
Layer 1: hardened base (distroless preferred)
Layer 2: separate tooling (minimal vs. extended)
Layer 3: runtimes per language (ultra-lean)
Layer 4: apps (zero build tools)
These are the Dockerfiles used in these examples; they are static and simple. The generation of the images and the Dockerfiles should be as automated and dynamic as possible.
https://github.com/faustobranco/devops-db/tree/master/Initial%20Images/structure/images
Example:
#######################################################################################################################
### Base Images
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/base-alpine:1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/base-alpine:1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/base-debian:1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/base-debian:1.0.0
#######################################################################################################################
### Tooling - Debian
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/tooling-minimal-debian:1.0.0 -f Dockerfile-debian .
docker push nexus.devops-db.internal/docker-base-images-public/tooling-minimal-debian:1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/tooling-extended-debian:1.0.0 -f Dockerfile-debian .
docker push nexus.devops-db.internal/docker-base-images-public/tooling-extended-debian:1.0.0
#######################################################################################################################
### Runtime - Go - Debian
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/go-base-debian:1.22-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/go-base-debian:1.22-1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/go-minimal-debian:1.22-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/go-minimal-debian:1.22-1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/go-extended-debian:1.22-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/go-extended-debian:1.22-1.0.0
#######################################################################################################################
### Runtime - Python - Debian
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/python-base-debian:3.12-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/python-base-debian:3.12-1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/python-minimal-debian:3.12-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/python-minimal-debian:3.12-1.0.0
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/python-extended-debian:3.12-1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/python-extended-debian:3.12-1.0.0
#######################################################################################################################
### Services - Python - Debian
docker build --platform linux/amd64 -t nexus.devops-db.internal/docker-base-images-public/app-example:1.0.0 .
docker push nexus.devops-db.internal/docker-base-images-public/app-example:1.0.0
2. Distroless Images
2.1 What Are Distroless Images?
Distroless images are container images that exclude a traditional Linux distribution. They contain only:
- The application
- Its runtime dependencies (e.g., libc, JVM)
- Required system files (e.g., CA certificates)
They deliberately exclude:
- Shells (
bash,sh) - Package managers (
apt,apk) - Debugging tools (
curl,ps,netstat)
The result is a minimal runtime-only container.
2.2 Direct Comparison
| Image Type | Contents | Size | Security | Debug Capability |
|---|---|---|---|---|
| Ubuntu | Full OS | Large | Medium | High |
| Alpine | Minimal OS | Small | Good | Medium |
| Distroless | Runtime only | Very small | High | Low |
2.3 Why Distroless Is More Secure
Distroless reduces attack surface:
- No shell → prevents interactive exploitation
- No package manager → prevents runtime installation of malicious tools
- Fewer binaries → fewer CVEs
- Reduced filesystem footprint → easier auditing
2.4 Practical Example (Go)
Traditional Approach
FROM alpine
RUN apk add --no-cache ca-certificates
COPY app /app
CMD ["/app"]
Distroless Approach
FROM gcr.io/distroless/static
COPY app /app
USER 10001
CMD ["/app"]
Requirements:
- Binary must be statically compiled (
CGO_ENABLED=0)
2.5 Python Example
# builder
FROM python:3.12-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN python -m venv /venv
RUN /venv/bin/pip install --no-cache-dir -r requirements.txt
# runtime
FROM gcr.io/distroless/python3
COPY --from=builder /venv /venv
COPY . /app
ENV PATH="/venv/bin:$PATH"
CMD ["app.py"]
2.6 Limitations
- No shell access (
kubectl execwithshfails) - No runtime package installation
- Harder debugging
- Requires correct dependency handling at build time
2.7 Recommended Pattern
Use multi-stage builds:
- Builder stage → full tooling
- Runtime stage → distroless
3. Layered Image Architecture (4 Layers)
3.1 Overview
A robust structure consists of four distinct layers:
Layer 1 — Base Hardened
Purpose:
- Minimal trusted foundation
Characteristics:
- Distroless preferred
- Non-root user
- CA certificates
- No package manager
Layer 2 — Tooling (Optional)
Split into:
Minimal
- curl
- netcat (busybox)
- tini
Extended
- git
- vault CLI
- bash
- telnet
Goal:
Avoid polluting all images with heavy tooling.
Layer 3 — Language Runtime
Separate per language:
Go
- Static binary
- Distroless/static
Python
- venv-based
- distroless/python3
Java
- jlink minimized runtime
- distroless/java
Layer 4 — Application
- Only application artifacts
- No build tools
- Non-root execution
- Minimal entrypoint
4. Repository Structure
images/
base/
distroless/
alpine-hardened/
tooling/
minimal/
extended/
runtimes/
python/3.12/
go/1.22/
java/21/
apps/
service-a/
service-b/
build/
docker-bake.hcl
5. Multi-Stage Builds
Concept
Separate:
- Build environment
- Runtime environment
Example (Go)
# builder
FROM golang:1.22-alpine AS builder
WORKDIR /build
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=$TARGETARCH \
go build -o app
# runtime
FROM gcr.io/distroless/static
COPY --from=builder /build/app /app
USER 10001
ENTRYPOINT ["/app"]
6. Multi-Architecture Builds
Using buildx
docker buildx build \
--platform linux/amd64,linux/arm64 \
-t registry/app:latest \
--push .
Using docker-bake
group "default" {
targets = ["app"]
}
target "app" {
platforms = ["linux/amd64", "linux/arm64"]
tags = ["registry/app:latest"]
}
7. Hardening (Critical)
Container Runtime
USER nonrootreadOnlyRootFilesystem: true- Drop all capabilities
no-new-privileges
Filesystem
- Remove caches (
/var/cache) - Restrict
/tmp - Minimal permissions
Build Security
- Pin versions (never
latest) - Generate SBOM (e.g., syft)
- Scan vulnerabilities (e.g., trivy, grype)
8. Versioning Strategy
Avoid coupling:
base:1.0
python:3.12-1.0
go:1.22-1.0
app:1.3.2
Separate:
- Runtime version
- Base hardening version
9. Tooling Strategy
Problem
Adding tools like:
- git
- nc
- telnet
- vault
→ increases attack surface
Recommended Approaches
1. Copy binaries
FROM alpine AS builder
RUN wget ...vault.zip && unzip vault.zip
FROM gcr.io/distroless/base
COPY --from=builder /vault /usr/local/bin/vault
2. Use tooling layers
Reusable base images:
tooling-minimal
tooling-extended
3. Avoid embedding tools
Prefer:
- sidecars
- ephemeral containers
10. Package Installation Model (Distroless)
Key Principle
There is no package installation in runtime.
Everything happens in build stage.
Incorrect Approach
apt install git
Correct Approach
FROM alpine AS builder
RUN apk add git
FROM distroless
COPY --from=builder /usr/bin/git /usr/bin/git
Language-Specific Guidance
Go
- Static binary
- No runtime dependencies
Python
- Use virtualenv
- Copy
/venv
Java
- Use
jlink - Minimize runtime
Vault / Tools
- Copy binaries explicitly
- Validate dependencies
11. Debugging Distroless Containers
Problem
No shell:
kubectl exec -it pod -- sh
Fails.
Correct Approaches
Option A — Ephemeral Containers (Recommended)
kubectl debug -it pod-name \
--image=busybox \
--target=app-container
Option B — Sidecar Debug Container
- name: debug
image: alpine
command: ["sleep", "infinity"]
Option C — Debug Image Variant
app:prod → distroless
app:debug → alpine-based
Option D — Logs and Inspection
kubectl logs
kubectl describe
kubectl cp
12. Trade-offs
Distroless vs Alpine
| Factor | Distroless | Alpine |
|---|---|---|
| Security | Higher | Good |
| Size | Smaller | Small |
| Debug | Hard | Easier |
Tooling Inside vs Outside
| Approach | Pros | Cons |
|---|---|---|
| Embedded | Convenience | Security risk |
| External | Secure | More complex |
13. Conclusion
A production-grade container strategy should follow:
- Layered architecture (4 layers)
- Distroless runtime for security
- Multi-stage builds for separation
- Multi-arch support via buildx
- Strict hardening policies
- Tooling decoupled from runtime
- Debugging via external mechanisms
The key shift is conceptual:
Containers are not mutable environments.
They are pre-built, minimal, and purpose-specific execution units.
When implemented correctly, this model yields:
- Smaller images
- Faster deployments
- Reduced CVE exposure
- Strong operational consistency across architectures
