Headscale ACL Policy: What It Blocks, Tested

headscalenetworkingsecurityself-hostingvpn
Headscale ACL Policy: What It Blocks, Tested

With a Headscale access policy loaded, a worker with no rule to the database had no status entry, DNS name or ping for it. A removed rule reached the client in 2.4 to 3.6 s, but an open session can keep running. I ran these in containers on Docker networks, so a real VPS and a provider reboot aren't covered.

I tested Headscale 0.29.7 and Tailscale 1.104 clients with the VPS and four machines running as containers on separate Docker networks. app ran Ubuntu, and worker and laptop ran Alpine. Certificates came from Caddy's internal CA and names from /etc/hosts, so real DNS, Let's Encrypt and real NAT traversal were not part of the test. I also didn't run a provider reboot or any macOS, Windows or mobile client, so the restarts below were Headscale container restarts.

The stack is in my Headscale setup guide, which installs everything in /opt/headscale and sets the policy path before any machine joins. This is the full policy, /opt/headscale/config/policy.hujson:

{
  "groups": {
    "group:admins": ["admin@"]
  },
  "tagOwners": {
    "tag:app": ["group:admins"],
    "tag:db": ["group:admins"],
    "tag:worker": ["group:admins"]
  },
  "nodeAttrs": [
    { "target": ["autogroup:member", "autogroup:tagged"], "attr": ["cache-network-maps"] }
  ],
  "grants": [
    { "src": ["group:admins"], "dst": ["tag:app", "tag:db", "tag:worker"], "ip": ["tcp:22"] },
    { "src": ["tag:app"], "dst": ["tag:db"], "ip": ["tcp:5432"] },
    { "src": ["tag:worker"], "dst": ["tag:app"], "ip": ["tcp:4222"] }
  ]
}

Compose mounts ./config at /etc/headscale, so an existing install needs policy.path: /etc/headscale/policy.hujson in its config. The policy gives group:admins SSH on tcp:22 to tag:app, tag:db and tag:worker, lets tag:app reach tag:db on tcp:5432, and lets tag:worker reach tag:app on tcp:4222. laptop joined as the admin user, so it's in group:admins. The nodeAttrs block grants the outage cache that the setup guide covers. The sample download holds this file, and its verify.mjs reruns the allowed and denied paths between app, db and worker.

What Does the Headscale Access Policy Block?

With no policy loaded, every machine can reach every port on every other machine, and the shipped config loads none. Before I loaded a policy, worker -> db:5432, worker -> db:22 and db -> app:22 all connected, and Headscale's policy docs say the same.

With the policy above, I probed the allowed and denied paths:

nc -z -v -w 4 app 22        # from the laptop
nc -z -v -w 4 db 5432       # from the worker: "Name does not resolve", db is hidden from it
tailscale ping 100.64.0.2   # from the worker, with db's address: "no matching peer"

The results (trimmed):

laptop  -> app:22  rc=0  Connection to app (100.64.0.1) 22 port [tcp/ssh] succeeded!
app     -> db:5432  rc=0  Connection to db (100.64.0.2) 5432 port [tcp/postgresql] succeeded!
worker  -> app:4222  rc=0  Connection to app (100.64.0.1) 4222 port [tcp/*] succeeded!
worker  -> db:5432  rc=1  nc: getaddrinfo for host "db" port 5432: Name does not resolve
worker  -> 100.64.0.1:22  rc=1  nc: connect to 100.64.0.1 port 22 (tcp) timed out: Operation in progress
laptop  -> 100.64.0.2:5432  rc=1  nc: connect to 100.64.0.2 port 5432 (tcp) timed out: Operation in progress
worker sees db: 0; ping: no matching peer

The exact error text varies by distro.

The worker has no rule to the database, so it has no status entry, DNS name or ping for it. A capture on the database saw 0 packets.

A timeout to a hidden peer proves nothing, because the packet leaves through the default route. Check the status entry and tailscale ping instead.

When Should You Add Policy Tests?

Wait until every tagged node exists before you add tests, because they fail earlier with resolved to no IP addresses. Add them after grants in /opt/headscale/config/policy.hujson, with a comma after the grants array:

  "tests": [
    { "src": "tag:app", "accept": ["tag:db:5432"] },
    { "src": "tag:worker", "accept": ["tag:app:4222"], "deny": ["tag:db:5432"] },
    { "src": "group:admins", "accept": ["tag:db:22"], "deny": ["tag:db:5432"] }
  ]

Then, as root on the VPS:

test -s /opt/headscale/config/policy.hujson && docker exec headscale headscale policy check --file /etc/headscale/policy.hujson
PID=$(docker inspect --format '{{.State.Pid}}' headscale); [ "$PID" -gt 0 ] && kill -HUP "$PID" || echo "Headscale is not running, nothing reloaded"
docker logs --since 5s headscale 2>&1 | grep -i polic

You should see Policy is valid, then Policy reload completed with changes. The second line reloads with kill -HUP on the PID, because Docker counts any docker kill as a manual stop. Its check is there because a stopped container reports PID 0, and kill -HUP 0 ended my shell.

What Happens When the Policy File Is Bad?

A reload and a restart treat a bad file differently, so run the check before either:

Policy fileOn reloadOn container restart
Syntax errorRejected, old rules stayHeadscale crash-loops, and tunnels keep working
A failing testRejected, old rules stayApplied with a warning, and the port the test denied opened
Empty file (0 bytes)No error, old rules stayed for the 40 s I watchedEverything allowed, with no error

policy check catches the first two rows. It reports an empty file as Policy is valid, so from /opt/headscale as root run test -s config/policy.hujson with it. It also doesn't catch an unknown user. I ran only policy check on that file, so I can't say what a reload or restart does with it.

If you switched to database mode for Headplane, the policy lives in the database and these file steps no longer apply.

How Fast Does a Removed Rule Apply?

A reload that removed a rule reached the client in 2.4 to 3.6 s. In 3 runs, I removed a rule that let worker reach db:5432, reloaded, and probed that port in a loop with a 1 s nc timeout. The first failed probe came 2.4, 3.6 and 3.6 s after the signal.

With another rule still linking two machines, new connections stopped. In one run, I changed the app to db grant from tcp:5432 to tcp:9, so the two stayed peers. An open psql session was still writing 180 s later, until pg_terminate_backend ended it. This ran on db, and it ends every other psql session on that server:

SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE application_name = 'psql' AND pid <> pg_backend_pid();

With the last rule gone, the peer left the map and writes stopped. psql hung without exiting for the 26 s I watched, so end the session at the server as well.

Summary

With no policy, every machine can reach every port on every other machine, so set the policy path before you join machines. With the policy loaded, the worker had no status entry, DNS name or ping for the database, and a capture on the database saw 0 packets.

Run policy check and test -s before a reload or a restart, because a restart crash-looped on a syntax error and applied a policy with a failing test. A removed rule reached the client in 2.4 to 3.6 s, and an open session can outlast it, so end the session at the server as well.

All of this ran in containers on Docker networks, and I didn't run a provider reboot.

The rest of the stack is in my Headscale guide, and my self-hosting course sets up private administration over Tailscale on a real server.

Thanks for reading.

And stay awesome!


Frequently Asked Questions

Does Headscale allow all traffic by default?

Yes. With no policy loaded, every machine can reach every port on every other machine, and the shipped config loads no policy. Set a policy path in the config before you join machines.

How do you check that a Headscale policy hides a machine?

Read the status entry and run tailscale ping with the hidden address. A timeout to a hidden peer proves nothing, because the packet leaves through the default route. In my test the worker had no status entry, DNS name or ping for the database.

What happens if the Headscale policy file is broken?

A reload rejects a syntax error and keeps the old rules. A restart with the same file crash-loops Headscale, and tunnels keep working. A failing test is rejected on reload and applied on restart, so run policy check before either.

Does removing a Headscale policy rule end open connections?

Not always. A reload reached the client in 2.4 to 3.6 s and new connections stopped, but with another rule still linking the two machines an open psql session was still writing 180 s later in one run. End the session at the server as well.

Browse all Self-Hosting guides →
Loading comments...

Whenever you're ready, there are 5 ways I can help you:

  1. Pragmatic Self-Hosting: Learn to deploy and run your apps on a VPS with Dokploy and Tailscale. Keep administration private, automate releases, and practice restoring backups, with source code and a runbook you can reuse.
  2. Pragmatic Clean Architecture: Join 5,000+ students in this comprehensive course that will teach you the system I use to ship production-ready applications using Clean Architecture. Learn how to apply the best practices of modern software architecture.
  3. Modular Monolith Architecture: Join 2,800+ engineers in this in-depth course that will transform the way you build modern systems. You will learn the best practices for applying the Modular Monolith architecture in a real-world scenario.
  4. Pragmatic REST APIs: Join 1,900+ students in this course that will teach you how to build production-ready REST APIs using the latest ASP.NET Core features and best practices. It includes a fully functional UI application that we'll integrate with the REST API.
  5. Patreon Community: Join a community of 5,000+ engineers and software architects. You will also unlock access to the source code I use in my YouTube videos, early access to future videos, and exclusive discounts for my courses.

The .NET Weekly

One practical .NET tip, every Saturday.

Join 66,000+ engineers who read it. Free, and you can unsubscribe any time.