You are reading the docs for v0.0.1-beta-6. Latest (main) →

Alerts

View .mdEdit on GitHub

kinhin can push a message to a webhook when something is wrong, so a closed menu bar app or a headless host still reaches you. The macOS notification the app posts is separate and unchanged.

The daemon compares the fleet's active problems (the same ones kinhin status and the app show) every 15 seconds. A problem that appears is sent once; while it stays raised it is sent again only after cooldown_seconds; when it clears, a resolved message follows. A dead endpoint is logged and never becomes a fleet problem itself.

Configure#

alerts:
  enabled: true
  webhook:
    url: https://ntfy.sh/my-kinhin-topic
    format: ntfy
    headers_keychain: ops-alerts
  min_severity: warning
  cooldown_seconds: 3600
  notify_on_resolve: true
  • webhook.url must be https; plain http is accepted only for localhost, 127.0.0.1 and ::1.
  • webhook.format shapes the body: generic (the event as JSON), slack, discord, or ntfy (plain text with a title). A helpdesk or paging service that accepts a webhook works through generic.
  • webhook.headers_keychain names a Keychain item holding a bearer token, sent as Authorization: Bearer <token>. The token is never in config.yaml, and the log redactor masks it.
  • min_severity: critical skips the engine's own retries (spawn backoff, orphan sweeps).

The daemon reads the config when it starts or rebuilds. A bad webhook or a missing Keychain item is logged and leaves alerting off; it never stops the fleet. The daemon tries again every minute, so a token stored with alerts set-token takes effect without a restart. An alert that could not be delivered is sent again a minute later (a problem that cleared, too), and redirects from the endpoint count as failures. When the fleet cannot start at all (no forge's token verifies), the daemon sends one startup alert, since the usual loop only runs for a started fleet.

Store the token and test it#

printf %s "$TOKEN" | kinhin alerts set-token ops-alerts --token-stdin
kinhin alerts test

set-token NAME stores the bearer token under the headers_keychain name (without --token-stdin it asks at a hidden prompt). alerts test sends a test message through the configured webhook, even before enabled is true, and prints only scheme://host of the endpoint. It exits 3 when webhook.url is not set and 4 when the endpoint does not accept the message.

kinhin doctor has an alerts row once alerts are enabled. It is a failure, and ready turns false, when the Keychain item named by headers_keychain is missing, because the daemon would leave alerting off.

Generic payload#

{"host":"mini","message":"disk is full","severity":"critical","source":"disk","state":"firing","timestamp":"2026-11-11T19:33:25Z"}

source is the subsystem that raised the problem and is the key for deduplication.

Not yet#

Email, vendor ticket APIs and per-source routing are not part of this first version; each would be another sink next to the webhook.