Skip to main content

Edge: Ingress, TLS, and DNS

This chapter gives each cluster a public face. Because the DO cloud-controller-manager was installed in Part 3, a Service type=LoadBalancer in front of Traefik provisions a real DigitalOcean Load Balancer with a stable public IP, exactly what a production team would do. You'll point a real domain at that LB IP, cert-manager will issue real Let's Encrypt certificates, and Traefik will own cross-cutting HTTP behaviour (HTTPS redirect, security headers) via Middleware resources defined in cluster-bootstrap. Both clusters get the same edge.

A few terms that show up throughout:

  • Service type=LoadBalancer is the third Service type, after ClusterIP and the NodePort you saw in Part 2. It asks the cloud-controller-manager for an external load balancer wired up to the Service's pods.
  • An Ingress is the Kubernetes object that says "route this hostname (and optionally this path) to this Service, and please terminate TLS for me." It's declarative HTTP routing.
  • An Ingress controller is the actual reverse proxy that reads Ingress objects and turns them into routing rules. The cluster doesn't ship one by default; this chapter installs Traefik to play that role.
  • An IngressClass lets several controllers coexist in one cluster. Each Ingress can specify which class handles it; Traefik becomes the default below so you don't have to specify.
  • A CRD (CustomResourceDefinition) is how a controller adds its own object kinds to the Kubernetes API. cert-manager ships Certificate/ClusterIssuer CRDs; Traefik ships Middleware/IngressRoute CRDs. They feel like built-in objects once installed.

Prerequisites​

  • Both clusters from Part 3 are running with Cilium, DO CCM, DO CSI, and metrics-server.
  • A domain name you can edit DNS records for.
  • You can reach the cluster from your laptop with kubectl --context do-staging and kubectl --context do-prod.

URL layout​

Pick subdomains that visually separate environments:

  • app.example.com → prod
  • app.staging.example.com → staging

Substitute your real domain everywhere you see example.com below.

Like Part 3, every command runs once per cluster. Set ENV and the context:

export ENV=staging
export CTX=do-${ENV}
export HOST="app.staging.example.com"      # for prod, set HOST=app.example.com

Step 1 — Install Traefik​

The Traefik Helm chart's default Service type is LoadBalancer. With DO CCM in place that triggers a real DigitalOcean Load Balancer.

helm repo add traefik https://traefik.github.io/charts
helm repo update

helm --kube-context "$CTX" upgrade --install traefik traefik/traefik \
  --version 33.2.1 \
  --namespace traefik --create-namespace \
  --set ingressClass.enabled=true \
  --set ingressClass.isDefaultClass=true \
  --set ingressClass.name=traefik \
  --set service.type=LoadBalancer \
  --set service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-name"="lb-${ENV}-traefik" \
  --set service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-protocol"=tcp \
  --set service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-enable-proxy-protocol"="false" \
  --set service.annotations."service\.beta\.kubernetes\.io/do-loadbalancer-healthcheck-protocol"=tcp \
  --set ports.web.redirectTo.port=websecure \
  --set ports.web.redirectTo.scheme=https \
  --set ports.websecure.tls.enabled=true \
  --set 'providers.kubernetesIngress.enabled=true' \
  --set 'providers.kubernetesCRD.enabled=true' \
  --set metrics.prometheus.enabled=true

The annotations are read by the DO CCM:

  • do-loadbalancer-name: the human label that appears in the DO control panel.
  • do-loadbalancer-protocol=tcp: pass TLS through to Traefik; cert-manager, not the DO LB, owns TLS termination.
  • do-loadbalancer-enable-proxy-protocol=false: leave the PROXY protocol off. You only need it if you require client-IP preservation through a TCP LB.
  • do-loadbalancer-healthcheck-protocol=tcp: the DO Load Balancer probes the data port directly. Traefik exposes an HTTP /ping endpoint but on its own port (9000), which isn't in the public Service. TCP probing the data ports avoids that pitfall. (If you want HTTP-level health, expose traefik port 9000 through a second Service and aim the health check annotation at it explicitly.)

The chart options:

  • ingressClass.isDefaultClass=true: Traefik becomes the default IngressClass. Any Ingress without ingressClassName gets it.
  • ports.web.redirectTo.port=websecure and ports.web.redirectTo.scheme=https: Traefik handles the cluster-wide HTTP→HTTPS redirect at its EntryPoint level. You do not need to attach a redirect-https middleware to every Ingress.
  • providers.kubernetesIngress / kubernetesCRD: Traefik watches both stock Ingress and its own IngressRoute CRDs. The course uses standard Ingress so the app repo stays portable.

After the LB comes up, verify health from the DO control panel side too. doctl compute load-balancer get <id> should show all droplets as healthy once Traefik is accepting connections on port 80.

Watch the LoadBalancer Service. The DO CCM will set EXTERNAL-IP once DigitalOcean has provisioned a real LB (1–3 minutes):

kubectl --context "$CTX" -n traefik get svc traefik -w
# NAME      TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)
# traefik   LoadBalancer   10.x.y.z      <pending>      80:..../TCP,443:..../TCP
# ...
# traefik   LoadBalancer   10.x.y.z      161.35.x.y     80:..../TCP,443:..../TCP

Capture the IP:

LB_IP=$(kubectl --context "$CTX" -n traefik get svc traefik \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo "$LB_IP"

Confirm the LB exists in DigitalOcean:

doctl compute load-balancer list --format Name,IP,Status,Region
# lb-staging-traefik   161.35.x.y   active   fra1

Step 2 — Point DNS at the LB IP​

Add an A record at your DNS provider:

app.staging.example.com    A    161.35.x.y    TTL 300

(And a second one for prod, after you do this chapter on the second cluster.)

Wait and verify:

dig +short A "$HOST"
# 161.35.x.y

If dig returns nothing, give DNS a few minutes and try dig +trace to see which step is stale.

Quick sanity check: Traefik should be answering HTTP 404 (no Ingress rules yet) before any rule is in place. Because the EntryPoint redirect is on, plain HTTP returns a 308 to HTTPS, and HTTPS without a matching route returns 404:

curl -sI "http://$HOST/"
# HTTP/1.1 308 Permanent Redirect
# Location: https://app.staging.example.com/

curl -skI "https://$HOST/"
# HTTP/2 404

A 308 on HTTP and a 404 on HTTPS prove DNS, the DO LB, and Traefik are all on the path.

Step 3 — Install cert-manager​

cert-manager is a controller that automates X.509 certificate issuance and renewal inside Kubernetes. You declare a Certificate (or annotate an Ingress), and cert-manager talks to a CA on your behalf, stores the resulting cert and key in a Secret, and renews it before it expires. The CA isn't built in — you point it at one through an Issuer (namespace-scoped) or ClusterIssuer (cluster-wide). This course uses Let's Encrypt.

helm repo add jetstack https://charts.jetstack.io
helm repo update

helm --kube-context "$CTX" upgrade --install cert-manager jetstack/cert-manager \
  --version v1.15.0 \
  --namespace cert-manager --create-namespace \
  --set installCRDs=true

Verify the three pods come up:

kubectl --context "$CTX" -n cert-manager get pods
# cert-manager-...            Running
# cert-manager-cainjector-... Running
# cert-manager-webhook-...    Running

Step 4 — Two ClusterIssuers (staging first, then production)​

Let's Encrypt is a free, automated public CA. Its protocol, ACME, defines how a client proves it controls a domain before the CA hands over a certificate. The course uses the HTTP-01 challenge: cert-manager temporarily serves a known token at http://<your-domain>/.well-known/acme-challenge/<token>, and Let's Encrypt fetches it to confirm the domain points at you. That's why DNS has to resolve before a certificate can issue.

Always iterate against Let's Encrypt's staging endpoint first. Its certs aren't browser-trusted, but it has generous rate limits while you debug — the production endpoint will lock you out for a week if you flood it with failed attempts.

Save this as cluster-bootstrap/cert-manager/clusterissuers.yaml:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: you@example.com
    privateKeySecretRef: { name: letsencrypt-staging-account-key }
    solvers:
      - http01:
          ingress:
            class: traefik
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: you@example.com
    privateKeySecretRef: { name: letsencrypt-prod-account-key }
    solvers:
      - http01:
          ingress:
            class: traefik

The solvers.http01.ingress.class: traefik tells cert-manager which IngressClass to use for the temporary challenge Ingress it creates while solving the HTTP-01 challenge.

Apply:

kubectl --context "$CTX" apply -f cluster-bootstrap/cert-manager/clusterissuers.yaml
kubectl --context "$CTX" get clusterissuer
# letsencrypt-staging   True   ...
# letsencrypt-prod      True   ...

Step 5 — Shared Middlewares (security headers)​

A Traefik Middleware is a CRD that describes a piece of request/response processing (add a header, rewrite a path, strip a prefix, rate-limit, etc.) you can compose onto any Ingress with a single annotation. Defining it once in cluster-bootstrap and referencing it from every app's Ingress keeps security headers consistent without copy-paste.

The Traefik EntryPoint already handles the cluster-wide HTTP→HTTPS redirect (Step 1). Security headers go in a Middleware you can attach to any Ingress with an annotation. Define the Middleware once per cluster in cluster-bootstrap:

cluster-bootstrap/traefik/middlewares.yaml:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: security-headers
  namespace: traefik
spec:
  headers:
    frameDeny: true
    contentTypeNosniff: true
    referrerPolicy: "strict-origin-when-cross-origin"
    stsSeconds: 63072000
    stsIncludeSubdomains: false
    stsPreload: false
    forceSTSHeader: true

Apply:

kubectl --context "$CTX" apply -f cluster-bootstrap/traefik/middlewares.yaml
kubectl --context "$CTX" -n traefik get middlewares

Any Ingress in any namespace can opt in with the annotation:

metadata:
  annotations:
    traefik.ingress.kubernetes.io/router.middlewares: traefik-security-headers@kubernetescrd

The reference format is <namespace>-<name>@kubernetescrd; Traefik builds that name automatically from the Middleware resource. Note the namespace prefix: the traefik- is literally the namespace name, not a typo.

Step 6 — Add an Ingress for the Express app​

The app from Part 2 isn't deployed here yet (that happens in Part 7), but you can ship a tiny placeholder to prove the TLS path before doing anything fancy.

Deploy a minimal echo workload:

kubectl --context "$CTX" -n "expressapp-${ENV}" create deployment hello \
  --image=nginxdemos/hello:plain-text
kubectl --context "$CTX" -n "expressapp-${ENV}" expose deployment hello --port=80

Create cluster-bootstrap/test/ingress-staging.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello
  namespace: expressapp-${ENV}
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
    traefik.ingress.kubernetes.io/router.middlewares: traefik-security-headers@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
    - host: ${HOST}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: hello
                port:
                  number: 80
  tls:
    - hosts: [ ${HOST} ]
      secretName: hello-tls

envsubst < cluster-bootstrap/test/ingress-staging.yaml | kubectl --context "$CTX" apply -f -

Watch cert-manager request the cert:

kubectl --context "$CTX" -n "expressapp-${ENV}" get certificate
# hello-tls   False   hello-tls   ...    pending
kubectl --context "$CTX" -n "expressapp-${ENV}" describe certificate hello-tls

When the Certificate becomes Ready: True, try the URL:

curl -kI "https://$HOST/"
# HTTP/2 200
# server: ...
# (cert is from the LE staging CA; -k accepts it)

If the certificate stays pending:

  • kubectl describe order and kubectl describe challenge in the same namespace show why. Most common cause: the DNS A record doesn't yet resolve to the LB IP, so the HTTP-01 challenge fails.
  • The challenge response is a temporary Ingress that cert-manager adds at /.well-known/acme-challenge/.... It must be reachable over plain HTTP from the public internet. Traefik's HTTP→HTTPS redirect can interfere with this. Traefik's EntryPoint redirect skips the ACME path by default, but if you customise it, double-check the rule excludes /.well-known/acme-challenge/.

Once letsencrypt-staging works, swap the issuer to letsencrypt-prod and force a re-issue:

kubectl --context "$CTX" -n "expressapp-${ENV}" annotate ingress hello \
  cert-manager.io/cluster-issuer=letsencrypt-prod --overwrite
kubectl --context "$CTX" -n "expressapp-${ENV}" delete secret hello-tls   # forces a re-issue

After a minute the cert is real:

curl -I "https://$HOST/"
# HTTP/2 200       (no -k needed)

Open the URL in a browser and confirm a green padlock. Verify security headers landed:

curl -sI "https://$HOST/" | grep -Ei 'strict-transport|x-frame|x-content-type|referrer'
# strict-transport-security: max-age=63072000
# x-frame-options: DENY
# x-content-type-options: nosniff
# referrer-policy: strict-origin-when-cross-origin

Step 7 — Tear down the placeholder​

The hello workload was a test fixture; the real Ingress will arrive with the Express app in Part 7:

kubectl --context "$CTX" -n "expressapp-${ENV}" delete ingress hello
kubectl --context "$CTX" -n "expressapp-${ENV}" delete service hello
kubectl --context "$CTX" -n "expressapp-${ENV}" delete deployment hello

Leave Traefik, cert-manager, the ClusterIssuers, the security-headers Middleware, and the DO LB in place.

Step 8 — Repeat for the second cluster​

Re-run steps 1–7 with ENV=prod (and the prod hostname). The end state is two DO Load Balancers (one per cluster), each with a public IP, a domain pointing at it, and a real Let's Encrypt cert issued and verified.

Sidebar — cheaper edge options​

The DO Load Balancer costs ~$12/month per cluster. The course uses it because it mirrors what production teams do on managed cloud. Two cheaper alternatives, documented for completeness:

  • MetalLB (L2) on a Reserved IP: install MetalLB, give it a single DO Reserved IP, advertise it via ARP. Free in DO billing terms, but DigitalOcean's network doesn't propagate gratuitous ARP between droplets in arbitrary ways, so the L2 announcement only works inside the cluster's VPC. The Reserved IP must be assigned to one droplet at a time, and you have to script failover. Not recommended for the course's prod path.
  • hostNetwork Traefik pinned to one worker: set hostNetwork=true and kind=DaemonSet in the Helm values, then point DNS at that worker's public IP. Free, but you lose the LB's failover; if that droplet dies, your DNS still resolves to a dead IP until you change it.

The rest of the course assumes the DO Load Balancer path.

Completion signal​

Both clusters serve a real domain over Let's Encrypt production HTTPS (no -k needed), the public IP is owned by a real DigitalOcean Load Balancer (visible in doctl compute load-balancer list and in kubectl -n traefik get svc), and curl -I https://app.staging.example.com/ returns a 200 with strict-transport-security, x-frame-options, and the other headers from Step 5 (provided by the security-headers Middleware). Both ClusterIssuers show Ready: True. Plain HTTP (curl -I http://app.staging.example.com/) returns a 308 redirect to HTTPS, served by Traefik's EntryPoint.

You're ready for 05. Image Pipeline.