Operate the Smiirl counter

This runbook checks, calibrates and repairs the Smiirl counter, and returns it to Smiirl's cloud. A cell is the character for one drum: a digit, a for blank, or b for stripes.

warning

This runbook changes counter settings by hand. It is an exception to the GitOps rule because git cannot hold counter settings.

Before you start#

Get kubectl access to folly (Get cluster admin access) and SSH access to capsule and spore.
Reach the iot network from your shell and browser.
Make sure .agents/skills/unifi-network/unifi.sh works (Inspect the UniFi network). If unifi.fml.pulsifer.ca does not resolve, set UNIFI_HOST=https://<udm> and UNIFI_SSH_HOST=<udm>, where <udm> is the first host in FUTURE_CIDR in clusters/folly/config/anchor-topology.json.

Check the counter#

If the counter shows the wrong value, do this procedure.

Read the device state.
```
curl -s --max-time 5 https://smiirl.lolwtf.ca/api/state | jq .device
```
Result: online is true if the counter polled in the last 60 seconds. lastSent is the last cells sent. lastStatus.wlan is <counter-ip>.
Read the app log.
```
kubectl --context folly -n smiirl logs deploy/smiirl --tail=50
```
Result: A cells <old> -> <new> line for each change. github: and daily: lines report problems.

Read the Gateway address.

GW=$(kubectl --context folly -n smiirl get gateway smiirl -o jsonpath='{.status.addresses[0].value}'); echo "$GW"

Make sure capsule and spore give the Gateway address for api.smiirl.com.

for h in capsule spore; do ssh "$h.lolwtf.ca" getent hosts api.smiirl.com; done

Send the counter's internet check, GET /number, through the Gateway.
```
curl -s -D - -H 'Host: api.smiirl.com' "http://$GW/number"
```
Result: content-length: 12 and the body {"number":1}.

Make sure the counter answers.

curl -s --max-time 5 "http://<counter-ip>/cgi-bin/luci/smiirl/api/version"

Result: The firmware version.

Calibrate the drums#

If every value shows a fixed offset, do this procedure.

caution

If a browser opens a counter page, the counter stops its polls until it restarts. If you stop early, power-cycle the counter.

Record the mode and cells.

curl -s --max-time 5 https://smiirl.lolwtf.ca/api/state | jq -c '{mode, cells}'

Show stripes on all drums.

curl -s -X PUT -d '{"cells":"bbbbb"}' https://smiirl.lolwtf.ca/api/number
curl -s -X PUT -d '{"mode":"number"}' https://smiirl.lolwtf.ca/api/mode

Wait until the counter receives the stripes.

until [ "$(curl -s --max-time 5 https://smiirl.lolwtf.ca/api/state | jq -r .device.lastSent)" = bbbbb ]; do sleep 5; done

Open http://<counter-ip>/calibrate/index.html in a browser.
On "WHAT DO YOU SEE?", select each box until it matches its drum, from left to right.
Select NEXT.

Result: After 10 seconds, all drums turn to stripes.
On "CLICK ON THE DIGIT IF YOU SEE STRIPES", select each drum that shows stripes, then select NEXT.
Do step 7 again until the page shows "COUNTER IS REBOOTING".

Wait until the counter polls again.

until curl -s --max-time 5 https://smiirl.lolwtf.ca/api/state | jq -e .device.online >/dev/null; do sleep 5; done

Set the cells from step 1.

curl -s -X PUT -d '{"cells":"<cells>"}' https://smiirl.lolwtf.ca/api/number

If the mode from step 1 is not number, select it at https://smiirl.lolwtf.ca. The page calls countdown Timer and countup Since.
If a drum is one flap off, do this procedure again. A drum still off after two calibrations has a mechanical fault.

Join the counter to Wi-Fi#

If the counter is not a UniFi client, do this procedure.

Find the SSID of the iot Wi-Fi clients.
```
.agents/skills/unifi-network/unifi.sh clients
```
Result: The WIFI/WIRED column of the iot rows gives the SSID.
From a device on SmiirlSetup, the counter's own network, open http://192.168.1.1.
Complete the setup wizard with the SSID from step 1 and its password.
Do step 1 of Check the counter.

Result: Within a few minutes, online is true.

Capture the counter's requests#

If the setup wizard says "Counter does not have access to internet", do this procedure.

Find the MAC address of the counter, <counter-mac>.

.agents/skills/unifi-network/unifi.sh find <counter-ip>

Capture the counter traffic on the UDM Pro for two minutes.
```
.agents/skills/unifi-network/unifi.sh ssh \
  'timeout 120 tcpdump -nn -A -s0 -i br666 "ether host <counter-mac>"'
```
Result: The reply to the internet check matches step 5 of Check the counter.

Return the counter to Smiirl's cloud#

If the counter must use Smiirl's cloud again, do this procedure. The counter then needs a Smiirl account.

Remove api.smiirl.com from the hosts line in nix/services/coredns-sinkhole.nix. Keep the counter and smiirl names.
Merge the change through a pull request.

To apply it before the nightly upgrade, run the upgrade on capsule and spore.

for h in capsule spore; do ssh "$h.lolwtf.ca" sudo systemctl start nixos-upgrade.service; done

Make sure step 4 of Check the counter does not show the Gateway address.
Power-cycle the counter.

After one minute, make sure online is false.

curl -s --max-time 5 https://smiirl.lolwtf.ca/api/state | jq .device.online

If something goes wrong#

Symptom	Cause	Action
The Gateway has no address.	Cilium did not assign the address.	Read `kubectl --context folly -n smiirl describe gateway smiirl`.
capsule or spore gives another address for `api.smiirl.com`.	That host does not run `main`.	Run `sudo systemctl start nixos-upgrade.service` on that host.
`online` is `false`, and all checks pass.	A browser opened a counter page, or the counter cached an old DNS answer.	Power-cycle the counter.
`lastStatus` is `null`.	The counter has not reported since the app started.	Find `<counter-ip>` in `unifi.sh clients`.