Post

Self-Hosted k3s on Azure - Part 7: Single Sign-On with LLDAP & Authelia

One login for every app: a lightweight LDAP directory (LLDAP) plus Authelia doing both forward-auth and OIDC, global default-protect with a bypass list, provisioning 2FA without email, and the three authentication patterns you end up mixing.

Self-Hosted k3s on Azure - Part 7: Single Sign-On with LLDAP & Authelia

This is Part 7 of the Self-Hosted k3s on Azure series. The goal: one identity, one login, protecting every app, whether the app knows how to log in or not.

Placeholders: example.com for the domain, dc=example,dc=com for the LDAP base.

forward-auth versus OIDC (the distinction that drives everything)

Two ways to protect an app:

  • forward-auth (a.k.a. auth proxy): the reverse proxy checks authentication before the request reaches the app. The app is oblivious. In Traefik this is the ForwardAuth middleware. Perfect for bare apps with no login of their own (static sites, small internal tools): pass the gate, you are in, no second login.
  • OIDC: the app itself redirects you to an identity provider and gets back an ID token saying who you are. Right for apps that already have a login (Argo CD, Grafana, Miniflux).

The trap: put forward-auth in front of an app that also does OIDC and you log in twice. So the rule is: OIDC-capable apps use OIDC, bare apps use forward-auth, and both trust the same identity provider. That shared IdP is what actually delivers “log in once,” not forward-auth by itself.

Choosing the tools

I wanted something light, fully declarative, and good at both roles. Authelia fits: a small Go service that is a Traefik ForwardAuth backend and an OIDC provider, configured entirely in YAML. The heavier full IdPs (Authentik, Keycloak) want a gigabyte or two of RAM, which is a lot on 4 GiB nodes. I also do not need SAML. For the user store I use LLDAP, a tiny LDAP server with a friendly web UI, purpose-built for self-hosted auth without the OpenLDAP complexity.

The shape: apps -> Authelia (IdP) -> LLDAP (user store). Apps never touch LDAP; only Authelia does.

LLDAP: the user directory

LLDAP runs as a single stateless replica with its state in the shared Postgres (per Part 6), so it can move freely. LDAP (port 3890) stays cluster-internal; only the web UI is exposed. Users live under ou=people, groups under ou=groups, base dc=example,dc=com.

Its secret has a few keys with very different consequences:

KeyRoleRotatable?
JWT secretsigns login sessionsyes (logs everyone out)
key seedderives the field-encryption key (encrypts stored TOTP seeds, etc.)no, ever (rotating corrupts stored data)
bind passwordthe admin/bind account passwordyes
database URLconnection to shared Postgresfollows the DB

That key seed is why the secret goes into a SealedSecret and is preserved verbatim across rebuilds.

Do not let Authelia bind to LLDAP as admin. Create a dedicated read-only bind account (LLDAP has a lldap_strict_readonly group for exactly this). Then the admin password is yours to change freely without breaking any service, and a leaked bind credential can only read.

Authelia as forward-auth: global default-protect

Rather than opt each app in, I make the default protected and opt specific things out. The Traefik ForwardAuth middleware is attached to the whole websecure entrypoint, so every HTTPS request is checked by Authelia, and the allow/deny decision lives centrally in Authelia’s access_control:

1
2
3
4
5
6
7
8
9
access_control:
  default_policy: deny
  rules:
    - domain: 'auth.example.com'          # the portal itself, or you loop forever
      policy: bypass
    - domain: 'app.example.com'           # a deliberately public app
      policy: bypass
    - domain: '*.example.com'             # everything else needs 2FA
      policy: two_factor

A new app is protected the moment it exists; going public is an explicit bypass entry. Two gotchas earned their keep:

  • Attaching the middleware globally is easy to get subtly wrong. The Traefik values key is ports.websecure.http.middlewares (nested under .http), not ports.websecure.middlewares. Put it at the wrong level and Helm silently ignores it, the pod does not restart, and everything still returns 200 (unprotected). Verify by checking the actual deployment args for --entryPoints.websecure.http.middlewares=..., and reference it as <namespace>-<name>@kubernetescrd.
  • Internal calls use http://. Traefik to Authelia is in-cluster pod traffic; Authelia listens on plain HTTP and TLS is terminated at the edge. Writing https:// there fails the handshake.

Provisioning 2FA without email

Authelia wants to send a confirmation email when a user enrolls a second factor (so a stolen password alone cannot bind a new TOTP and take over the account). With no SMTP configured, that “email” just gets written to a file inside the pod and the enrollment stalls.

There is a real security point hiding here: email as a second channel only works if changing the email is itself protected. If a password alone can change the account’s email (say the directory is public and lets users self-edit it), an attacker changes the email, enrolls TOTP, and confirms to their own inbox. Keeping the directory off the public internet is what makes that anchor real.

For a small deployment, the admin (already the highest trust) just provisions TOTP directly, bypassing the email flow entirely:

1
kubectl exec -it -n authelia deploy/authelia -- authelia storage user totp generate <uid>

Caveats I hit:

  • Run it inside the pod. The TOTP secret is stored encrypted with the storage encryption key, which is injected into the pod via config templating. The CLI needs that environment, which only exists in the pod.
  • The image is distroless. No printenv, cat, tar, or shell, so debugging tricks and kubectl cp do not work. Do not go looking for tools in the container.
  • The CLI does not validate the username. It writes straight to the database, so any <uid> “succeeds,” even a typo, creating an orphan record. It must exactly match the real login uid.

Then turn the resulting otpauth:// URI into a QR code locally (do not paste the secret into a web generator), for example with a local qrencode or a Python one-liner, scan it, and delete the image.

Authelia as an OIDC provider

For apps that speak OIDC, Authelia is also the IdP. Enable the provider and register a client per app:

1
2
3
4
5
6
7
8
identity_providers:
  oidc:
    clients:
      - client_id: argocd
        client_secret: {path: '/secrets/oidc/client_argocd'}   # a hashed secret
        authorization_policy: two_factor
        redirect_uris: ['https://argocd.example.com/auth/callback']
        scopes: [openid, profile, email, groups]

On the app side, point it at Authelia’s discovery endpoint. Argo maps an LDAP group to its admin role; Miniflux auto-creates the user on first OIDC login. A useful trick for adding a client’s hashed secret to an existing sealed secret without re-sealing everything is kubeseal --merge-into.

Crucially, an OIDC app’s domain gets a bypass in access_control, not two_factor. The app runs its own OIDC flow through Authelia; if forward-auth also gated it, you would authenticate twice. The 2FA still happens, just driven by the app’s OIDC redirect instead of the proxy.

The three patterns you end up mixing

By the end I had three coexisting authentication styles, and knowing which is which keeps access_control sane:

App typeHow it authenticatesaccess_control
bare app (no login)forward-authtwo_factor
OIDC app (Argo, Miniflux)its own OIDC flow to Autheliabypass
machine client (calendar feed, API, password-manager clients)none / its own tokenbypass (often path-scoped)

The rule of thumb: if the app cannot do an interactive redirect (a calendar client fetching an .ics, a phone app hitting an API), it must be bypassed at the proxy, because it can never complete a 2FA page.

The cost: a single Authelia is a global SPOF

Being honest about the downside. Once forward-auth is on the entrypoint, Authelia sits on the critical path of every HTTPS request, even bypassed ones (the proxy still asks Authelia for the allow decision). A single replica means its restart window is a site-wide outage of tens of seconds. “Stateless and reschedulable” is durability, not availability; closing the availability gap needs two replicas plus a shared session store. At personal scale I accepted the occasional blip at first, then went back and closed it (below). Worth knowing for disaster recovery: during a rebuild, HTTPS returns 5xx until Authelia is up. The emergency way into Argo is then kubectl port-forward, which does not go through the ingress.

Closing the SPOF: a shared session store, then two replicas

The blip nagged, so later I made Authelia actually highly available. Two problems had to be solved in order.

First, where sessions live. Authelia keeps them in memory by default, so any restart logs everyone out, and two replicas each hold their own set (a request load-balanced to the other pod looks logged out). Both problems disappear once sessions move to an external store shared by all replicas.

I self-host that store rather than reach for a managed cache: it is tiny, cluster-internal, and holds only short-lived session keys, so paying for a managed Redis would dwarf the workload. I run Valkey (the open-source Redis fork) in a small Sentinel setup, three pods with anti-affinity so they spread across nodes, and no PVC. Sessions are ephemeral by nature, so if the whole set is ever lost the worst case is everyone logs in again; that makes persistence unnecessary and keeps the pods free to reschedule anywhere. Sentinel gives Authelia a primary it can always find: if the primary pod dies, Sentinel elects a new one and Authelia follows, no manual failover. Authelia speaks Sentinel natively, so wiring it up is a few lines of session config pointing at the Sentinel endpoints plus the shared password (sealed, of course).

Second, the replicas themselves. With sessions externalized, I bump Authelia to two replicas with anti-affinity and a PodDisruptionBudget of one, so a node drain or a rolling update always leaves one pod serving. Two, not three: it removes the single point of failure (the whole point), and on 4 GiB nodes a third copy of something on every request’s critical path is memory I would rather give to apps. The math from the SPOF section now reads differently, a restart or a lost node is one pod out of two, not the entire site.

One gotcha worth flagging for anyone on the same Helm chart: it validates its values against a schema, and the exact shape of the session and Redis blocks is version-specific (a key nested one level off is silently rejected, or rejected loudly at deploy time). The habit that saves the round-trip is to render the chart locally with the pinned version and your values before committing, so the schema error shows up on your laptop instead of in the cluster.

What’s next

With SSO in place, adding apps becomes a rhythm. Part 8 rolls out Headlamp, Vaultwarden, and Miniflux, and pulls out the reusable patterns (a stateless template, the shared-Postgres pattern, and a technique for keeping a password manager’s sessions alive with no PVC).

This post is licensed under CC BY 4.0 by the author.