Modern infrastructure relies heavily on TLS certificates to secure communication between services. Kubernetes clusters, CI/CD pipelines, internal APIs, and development platforms all require trusted certificates to ensure encrypted and authenticated connections.
Instead of relying on public certificate authorities for internal systems, many organizations operate their own Internal Certificate Authority (Internal CA). This approach provides full control over certificate issuance and enables automated certificate management across the infrastructure.
In this series we will build a complete internal Certificate Authority (Internal CA) using Step-CA, the certificate authority from Smallstep. The final setup will support:
- Automatic certificate issuance for Kubernetes via ACME
- Certificate issuance for VMs and services using step-cli
- Integration with cert-manager
- A secure offline Root CA
Final architecture:
In Part 1, we will install and configure Step-CA, and prepare the PKI structure.
Configuring DNS for the Certificate Authority
The Step-CA service is exposed through an ingress resource using the hostname:
ca.devops-db.internalFor clients and services to communicate with the certificate authority, this hostname must resolve to the Kubernetes ingress controller.
Create a DNS record pointing the hostname to the ingress IP address.
Example:
ca.devops-db.internal → 127.0.0.1or, our internal network kubernetes:
ca.devops-db.internal → 172.21.5.76The correct address should match the IP assigned to the ingress controller.
You can verify the ingress endpoint using:
nslookup ca.devops-db.internal
Server: 100.64.0.1
Address: 100.64.0.1#53
Name: ca.devops-db.internal
Address: 172.21.5.76Adding the Helm Repository
Step-CA provides an official Helm chart that simplifies installation inside Kubernetes.
First add the Helm repository:
helm repo add smallstep https://smallstep.github.io/helm-charts
helm repo updateThis repository contains the Helm chart we will use to deploy the certificate authority.
Creating an Offline Root Certificate Authority
The Root CA represents the trust anchor of the PKI.
It should remain offline and should only be used to sign intermediate certificate authorities.
The Root CA should only be used to sign intermediate certificate authorities. It should never run inside Kubernetes or any production environment.
First, generate the root private key:
openssl genrsa -out root-ca.key 4096
Then create the root certificate:
openssl req -x509 -new -nodes \
-key root-ca.key \
-sha256 \
-days 3650 \
-out root-ca.crt \
-subj "/C=PT/O=DevOps-DB/CN=DevOps-DB Internal Root CA"This certificate will later be distributed to all systems that must trust the internal certificate authority.
Creating the Intermediate Certificate Authority
The Intermediate CA will be used by Step-CA to issue certificates.
Using an intermediate CA instead of issuing certificates directly from the root is a best practice in PKI design because it allows the root to remain offline.
Create the configuration file used to generate the certificate request.
vi intermediate-ext.cnfFile contents:
[ req ]
distinguished_name = dn
prompt = no
[ dn ]
C = PT
O = DevOps-DB
CN = DevOps-DB Intermediate CA
[ v3_ca ]
basicConstraints = critical,CA:TRUE,pathlen:0
keyUsage = critical,keyCertSign,cRLSign
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
Generate the intermediate private key:
openssl genrsa -out intermediate-ca.key 4096
Generate the certificate signing request:
openssl req -new \
-key intermediate-ca.key \
-out intermediate-ca.csr \
-config intermediate-ext.cnfSigning the Intermediate CA
The Root CA now signs the Intermediate CA certificate.
openssl x509 -req \
-in intermediate-ca.csr \
-CA root-ca.crt \
-CAkey root-ca.key \
-CAcreateserial \
-out intermediate-ca.crt \
-days 1825 \
-sha256 \
-extfile intermediate-ext.cnf \
-extensions v3_ca
At this stage we have the complete CA chain:
root-ca.crt
intermediate-ca.crt
intermediate-ca.key
The Root CA remains offline, while the Intermediate CA will be used by Step-CA.
Preparing Kubernetes for Step-CA
Step-CA needs access to the intermediate certificate and key. These will be stored as a Kubernetes secret.
First create a namespace:
kubectl create namespace step-caThe certificates generated earlier must be stored as a Kubernetes secret so the Helm chart can access them.
kubectl create secret generic step-ca-pki \
--from-file=root-ca.crt \
--from-file=intermediate-ca.crt \
--from-file=intermediate-ca.key \
-n step-caThis secret will later be referenced by the Helm deployment.
Certificate Issuance Methods
This environment will support two certificate issuance methods, which is a common and flexible design.
ACME will be used by Kubernetes workloads via cert-manager, while the JWK provisioner will allow external systems such as virtual machines and infrastructure services to request certificates.
Preparing the Step-CA Helm Values
Before installing Step-CA we need to populate the Helm values file with the certificates, keys, and configuration generated in the previous steps.
The Helm chart allows injecting certificates and secrets directly from the values.yaml file.
This approach makes the deployment reproducible and keeps all configuration in a single place.
Open the configuration file: https://github.com/faustobranco/devops-db/blob/master/SmallStep-CA/values-step1.yaml
vi values-step1.yamlRoot Certificate
Copy the contents of the root certificate.
Content of root-ca.crt:
inject:
certificates:
root-ca: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Intermediate Certificate
Copy the contents of the intermediate certificate.
Content of intermediate-ca.crt:
inject:
certificates:
intermediate_ca: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Intermediate Private Key
Copy the contents of the intermediate private key.
Content of intermediate-ca.key:
inject:
secrets:
x509:
intermediate_ca_key: |
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
CA Password
The CA password must be base64 encoded.
echo -n "yWz&^gGGdEKhBy80EA4K#aQdz" | base64
Result:
eVd6Jl5nR0dkRUtoQnk4MEVBNEsjYVFkeg==
Add the encoded value to the configuration:
inject:
secrets:
ca_password: eVd6Jl5nR0dkRUtoQnk4MEVBNEsjYVFkeg==
Root Certificate Fingerprint
This fingerprint is used by clients to verify the identity of the certificate authority during the bootstrap process.
step certificate fingerprint root-ca.crt
Result:
b8d4d77badb5aeca4d23a67269846aa3dbf3271c94c0cf97e82e482aab6e7732
Update the configuration on yaml file:
fingerprint: b8d4d77badb5aeca4d23a67269846aa3dbf3271c94c0cf97e82e482aab6e7732
Installing Step-CA
Now we can install Step-CA using Helm.
helm upgrade --install step-ca smallstep/step-certificates --values values-step1.yaml -n step-ca --create-namespace
fter installation, retrieve the generated passwords for the PKI and provisioner secrets.
kubectl get -n step-ca -o jsonpath='{.data.password}' secret/step-ca-step-certificates-ca-password | base64 --decode
kubectl get -n step-ca -o jsonpath='{.data.password}' secret/step-ca-step-certificates-provisioner-password | base64 --decodeThe values-step1.yaml file contains the configuration for Step-CA, including the reference to the existing PKI secret.
Once the deployment is complete, verify that the pods are running:
kubectl get pods -n step-ca
NAME READY STATUS RESTARTS AGE
step-ca-step-certificates-0 1/1 Running 0 32s
You should see the Step-CA pod in thCheck the health endpoint of the CA service.
curl -k https://ca.devops-db.internal/healthExpected response:
{"status":"ok"}Inspect the ingress configuration.
kubectl describe ingress -n step-ca
Name: step-ca-step-certificates
Labels: app.kubernetes.io/instance=step-ca
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=step-certificates
app.kubernetes.io/version=0.29.0
helm.sh/chart=step-certificates-1.29.0
Namespace: step-ca
Address: 127.0.0.1
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
ca.devops-db.internal
/ step-ca-step-certificates:443 (10.1.137.157:9000)
Annotations: meta.helm.sh/release-name: step-ca
meta.helm.sh/release-namespace: step-ca
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 43s (x2 over 72s) nginx-ingress-controller Scheduled for syncCertificate Issuance Methods: ACME vs JWK
Step-CA supports multiple methods for issuing certificates. In this setup we use two different approaches depending on the type of system requesting the certificate.
ACME
└ Kubernetes (cert-manager)JWK
└ Virtual machines and infrastructure services
Each method provides a different authentication mechanism for certificate requests.
ACME Provisioner
ACME (Automatic Certificate Management Environment) is a widely adopted protocol designed for automated certificate issuance.
It is the same protocol used by public certificate authorities such as Let’s Encrypt, but it can also be used with private certificate authorities like Step-CA.
In this environment ACME is primarily used by Kubernetes workloads through cert-manager.
The typical workflow is:
This allows certificates to be issued and renewed automatically without manual intervention.
ACME works particularly well in Kubernetes because cert-manager integrates directly with ingress controllers and service resources.
Typical use cases include:
- Ingress TLS certificates
- internal service endpoints
- Kubernetes applications such as Nexus or internal APIs
JWK Provisioner
The JWK provisioner uses JSON Web Keys (JWK) to authenticate certificate requests.
Instead of using a protocol challenge like ACME, clients authenticate by presenting a JWT token signed with a private key.
Step-CA verifies the signature using the public key configured in the provisioner.
The workflow looks like this:
This method is particularly useful for systems that are not running inside Kubernetes.
Typical examples include:
- Jenkins servers
- GitLab runners
- Hashicorp Vault
- artifact registries
- package repositories
Because the authentication is based on cryptographic signatures, this method is well suited for automated infrastructure.
When to Use Each Method
In practice the two methods complement each other.
| Method | Typical Use |
|---|---|
| ACME | Kubernetes workloads via cert-manager |
| JWK | VMs, infrastructure services, CI/CD systems |
ACME provides a fully automated workflow for Kubernetes environments, while JWK provides a flexible authentication mechanism for external systems.
Using both together allows the internal certificate authority to support a wide range of infrastructure components.
Creating a JWK Provisioner
Step-CA uses provisioners to define how clients are authorized to request certificates.
In this setup we create a JWK provisioner, which allows external systems to authenticate using JWT tokens.
Generate the key pair for the provisioner:
step crypto jwk create jwk-provisioner-priv.json jwk-provisioner-pub.json \
--kty EC --curve P-256 --use sig --alg ES256 --no-password --insecureExample output:
Your public key has been saved in jwk-provisioner-priv.json.
Your private key has been saved in jwk-provisioner-pub.json.Note
At the time of writing, step crypto jwk create prints the output messages in reverse order.
The CLI reports that the public key was saved in the private key file and vice-versa.
The files themselves are correct — only the message is inverted.
cat jwk-provisioner-pub.json
{
"use": "sig",
"kty": "EC",
"kid": "PvKMc182_erIthyh2DXdAp39eg1MvB3eMuTrl4cSq4I",
"crv": "P-256",
"alg": "ES256",
"x": "nlQ1XzR0rUOaPXPNZOFvnLPXXhrXFeX9hYgRSHrz70o",
"y": "m1oydTO_j3nTKYAaEvYih-VkrgjyOueFL5kU_Z6-1eQ",
"d": "lAeYP-q420sSYfTQxRZpP9JuSTjwpZGiqF583tbQW2Q"
}The public key will be added to the Step-CA configuration as a provisioner.
Example configuration:
provisioners:
- type: ACME
name: acme
- type: JWK
name: vm-hosts
key:
"use": "sig"
"kty": "EC"
"kid": "PvKMc182_erIthyh2DXdAp39eg1MvB3eMuTrl4cSq4I"
"crv": "P-256"
"alg": "ES256"
"x": "nlQ1XzR0rUOaPXPNZOFvnLPXXhrXFeX9hYgRSHrz70o"
"y": "m1oydTO_j3nTKYAaEvYih-VkrgjyOueFL5kU_Z6-1eQ"After updating the Helm values file, apply the new configuration. https://github.com/faustobranco/devops-db/blob/master/SmallStep-CA/values-step2.yaml
helm upgrade --install step-ca smallstep/step-certificates --values values-step2.yaml -n step-ca --create-namespaceRestart the Step-CA pod so the new configuration is loaded.
kubectl delete pod step-ca-step-certificates-0 -n step-caFinally verify that the provisioner has been created.
step ca provisioner listExpected output:
[
{
"type": "ACME",
"name": "acme"
},
{
"type": "JWK",
"name": "vm-hosts"
}
]At this stage the internal certificate authority is fully operational and ready to issue certificates using both ACME and JWK provisioners.
Verifying the Certificate Authority
Finally, verify that the Step-CA service is running correctly.
kubectl get svc -n step-ca
If an ingress or load balancer is configured, you should be able to access the CA endpoint:
https://ca.devops-db.internal
A health check endpoint is also available:
/health
If the endpoint responds successfully, the certificate authority is ready to start issuing certificates.
curl -k https://ca.devops-db.internal/health
{"status":"ok"}
Trusting the Internal Root Certificate
Because this is an internal certificate authority, its root certificate is not included in the default trust stores of operating systems, containers, or development environments.
Public certificate authorities are automatically trusted because their root certificates are distributed with operating systems and browsers. An internal CA does not have this advantage.
This means that any system communicating with services signed by this CA must explicitly trust the root certificate.
Typical examples include:
- Virtual machines
- Developer laptops
- CI/CD runners
- Kubernetes containers
- Automation tools such as Terraform, Ansible or Jenkins
If the root certificate is not installed in the system trust store, TLS connections will fail with errors such as:
x509: certificate signed by unknown authorityFor this reason, the root certificate must be distributed to every system that needs to trust certificates issued by the internal CA.
In most environments this is handled through infrastructure automation such as:
- base VM images
- Terraform provisioning
- Ansible configuration management
- container base images
- CI/CD pipelines
Installing the Root CA on Linux
On most Linux distributions the system trust store can be extended by copying the root certificate into the local certificate directory.
Copy the root certificate:
sudo cp root-ca.crt /usr/local/share/ca-certificates/devops-db-root-ca.crtUpdate the certificate store:
sudo update-ca-certificatesAfter updating the trust store the certificate should appear in the system CA bundle.
Installing the Root CA on macOS
On macOS the root certificate can be added to the system keychain using the security command.
sudo security add-trusted-cert \
-d -r trustRoot \
-k /Library/Keychains/System.keychain \
root-ca.crtOnce installed, the certificate will be trusted by system applications and development tools.
Automating Root CA Distribution
In real environments the root certificate is rarely installed manually. Instead it is distributed automatically as part of infrastructure provisioning.
Common approaches include:
- embedding the certificate in base VM images
- installing it via Ansible configuration management
- adding it during Terraform provisioning
- including it in container base images
- installing it in CI/CD runners
For example, an Ansible task might look like:
- name: Install internal root CA
copy:
src: root-ca.crt
dest: /usr/local/share/ca-certificates/devops-db-root-ca.crt
mode: '0644'
- name: Update certificate store
command: update-ca-certificatesThis ensures every system in the infrastructure trusts certificates issued by the internal certificate authority.