Switch a NetBox token to its own user
Mint a v2 token for the terraform and fleetd NetBox users, move Terraform and fleetd onto them, and revoke the admin tokens.
Use this runbook after Atlantis applies terraform/netbox/folly with the terraform and fleetd users, and again to rotate either token. NetBox says what each user may do.
warning
This runbook changes live state by hand. It is an exception to the GitOps rule because git cannot hold a token and the owner mints it in the UI.
Before you start#
- Atlantis has applied the pull request that declares the users, so
terraformandfleetdexist in NetBox. mise run devshell,opsigned in, andSOPS_AGE_KEY_FILEset as Manage SOPS secrets says.- Until the
terraformtoken is switched, Terraform's writes record asadminand fleetd counts no approved EK binding, so a planned machine waits.
Switch the tokens#
Open
https://capsule.<tailnet>and sign in asadmin.Open Admin, then Users, then
terraform, and add a token. Choose a v2 token, turn on Write enabled, and copy the value when NetBox shows it once. It starts withnbt_.In 1Password, open the item
netbox-folly-terraformin thehomelabvault, replace itscredentialfield with the new token, and save.Make sure that the root reads the token as
terraform. Commentatlantis plan -d terraform/netbox/follyon any pull request that touches the root, or run the plan locally.Result: The plan finishes with no authentication or permission error. A
403names a permission thatterraformlacks; add its object type toterraform/netbox/folly/users.tf.Add a v2 token for
fleetdthe same way, with Write enabled.Put it in spore's SOPS file, under
netbox-tokenin thefleetdmap, without printing it.sops nix/secrets/spore.sops.yamlIn the editor, replace the value and save.
Deploy spore with Deploy a NixOS host from
main, once the pull request that setshomelab.fleetd.bindingAuthoris merged.Make sure that fleetd works with its token.
ssh spore.<tailnet> systemctl is-active fleetdResult:
active.Revoke the old tokens. In Admin, then API Tokens, delete the
admintoken named "terraform: netbox/folly root through Atlantis". Keep the "automation: seed, export and drift" token for the scripts, and thenetbox/api-tokenthatnix/secrets/capsule.sops.yamlholds.
If something goes wrong#
| Symptom | Cause | Action |
|---|---|---|
A plan fails with 403 or Invalid token. | The token is a v1 token, is not write enabled, or belongs to another user. | Mint it again on terraform as a write-enabled v2 token. |
fleetd journals ek-binding-unauthored for a device. | The binding's newest change is by admin, written before the switch. | Change the node's entry in clusters/folly/inventory/nodes.yaml through a pull request so Terraform writes the binding as terraform. |
| fleetd answers every request with a NetBox error. | The fleetd token is wrong or lacks a permission. | Mint it again, or add the object type to terraform/netbox/folly/users.tf. |
Related#
- NetBox: the users, the tokens and the changelog check.
- Manage SOPS secrets: edit spore's SOPS file.
- Deploy a NixOS host: deploy spore.