You are reading the docs for v0.0.1-beta-7. Latest (main) →

Troubleshooting

View .mdEdit on GitHub

Start with kinhin heal: the healing agent fixes many of the problems below on its own, lists the fixes waiting for your approval, and says what only you can do.

  • kinhin doctor shows "tart: NOT FOUND" — run kinhin setup repair tart (or use the setup window's fix button), then re-run kinhin doctor.
  • "tart: INSTALLED BUT KILLED ON LAUNCH ✗" — the OpenAI-tap bottle is un-notarized; Gatekeeper kills it before it runs. kinhin doctor and the setup app print/offer the exact rollback commands; the manual equivalents are in Installing Tart.
  • Jobs queue but no VMs start — check that the job's runs-on: contains every label in your config; the scaler only counts jobs it could actually run.
  • Scale set: runner group 'default' was not found; create it in GitHub before attaching — with scale_set: on a GitHub account, kinhin looks the runner group up by an exact, case-sensitive name, and its default (default) is not GitHub's built-in group name (Default), so no scale set is ever created and the listener retries forever. Set runner_group: "Default" under scale_set: in config.yaml, spelled exactly as GitHub spells it, then restart the app — account-level config is not live-reloaded. Walkthrough: Troubleshooting scale sets.
  • VM boots but the runner never registers — the base image may predate the injected agent. Re-run kinhin image prepare. Full logs: kinhin logs -n 200 or ~/Library/Application Support/Kinhin/logs/kinhin.log.
  • lifecycle failed: runner registration failed … ssh: connect to host <ip> port 22: Operation timed out — the fresh VM booted and got an IP, but its SSH server did not answer within the bring-up window (at least vm_boot_timeout, default 300s). Common causes: the first boot of a freshly prepared image (macOS first-boot work can delay sshd for minutes — big images like full Xcode are the usual offenders), the host being busy with a bake or other jobs, or a softnet hiccup. kinhin retries the TCP/SSH probes for the whole window, then deletes the VM and scales a fresh one — the job normally lands on the replacement, so an isolated occurrence is harmless. If it repeats: raise vm_boot_timeout (e.g. 600) for slow images, keep the host quiet during boot storms, and re-prepare the image if the guest agent is flaky on every boot.
  • Every cold start takes about 15 seconds longer than the VM needs, with the log line port 22 probe saw nothing …; trying ssh anyway — kinhin watches a new VM's port 22 with a TCP probe before it runs ssh. On macOS, Local Network privacy can block that probe (system log: unsatisfied (Local network prohibited); check with log stream --predicate 'eventMessage CONTAINS "Local network prohibited"') while /usr/bin/ssh still reaches the guest, so the probe never succeeds. kinhin gives the probe 15 seconds, then tries ssh regardless; registration still works, you only lose the grace period. Older releases waited the whole vm_boot_timeout (about five minutes) before the first ssh. The 15-second fallback is the fix that matters. Release builds of the app also declare NSLocalNetworkUsageDescription, which may let macOS ask for Local Network access for the app process; this is unverified, and a daemon started from the CLI or a LaunchAgent has no app bundle for macOS to ask about. If System Settings → Privacy & Security → Local Network lists kinhin, make sure it is on. Do not edit the privacy database or turn off the firewall to work around it.
  • Apple container stops starting after brew upgrade container, with its data on an external volume — macOS needs Full Disk Access or Removable Volumes access before a process reads an external drive. Tart works because kinhin starts tart run itself, so kinhin's grant covers it. The container daemons are started by launchd (container system start registers them), so each needs its own grant, and macOS ties a grant to the code signature. The Homebrew build is ad-hoc signed, so every upgrade drops the grants. kinhin doctor warns about this on its cont from: row. Fix it for good with Apple's signed installer, or re-grant the binaries after each upgrade: Keeping container data on an external volume. Do not edit the privacy database by hand.
  • Tart: Error Domain=VZErrorDomain Code=-9 "The virtual machine encountered a security error." (with Failed to create new HostKey; tart ip then says no IP address found, is your VM running?) — the macOS login keychain's password often differs from your account password. Check it with security unlock-keychain ~/Library/Keychains/login.keychain-db: if that says The user name or passphrase you entered is not correct., reset the keychain. In a reported case, over Screen Sharing, Keychain Access → Settings → Reset My Default Keychain fixed it. This deletes the passwords saved in the login keychain, so re-enter the ones you need, then confirm the kinhin token with kinhin doctor --check-tokens. Do not delete the keychain instead; that removes the same passwords without fixing the mismatch. Full steps: Tart FAQ, Keychain password mismatch. Other errors such as SecKeyCreateRandomKey_ios failed can share this cause. Steps that did not help in the reported cases: reinstalling Tart, rebooting, stopping Multipass, updating the Command Line Tools.
  • image prepare fails while provisioning the new VM with No route to host, Operation timed out or Host is down (the log line is agent provision/update failed … could not reach the guest VM) — the baked VM booted and got an IP, but the host can't open a connection to it. The usual cause is OrbStack or Docker Desktop running on the same host: their VPN/interop networking can re-route the virtual network interface tart uses between host and guest, so the host's ssh to the VM never lands. Quit OrbStack/Docker for the length of the bake (they can be restarted afterwards) and re-run kinhin image prepare; kinhin appends this hint to the failure message automatically. If it still fails with no such app running, re-run with --verbose and check the host's routing table (netstat -rn | grep bridge): a VPN client or firewall rewriting the VM network's interface is the next suspect.
  • The daemon will not start: tart_home (…) is not ready: the volume /Volumes/Storage is not mounted (or the same for container_app_root, registry_cache_path or toolchain_cache_path) — a storage path is on a volume that is not mounted. Mount it (diskutil mount Storage); launchd restarts the daemon and it starts once the volume is there. kinhin never falls back to the internal disk, so nothing is written there in the meantime.
  • failed to bootstrap container … Code=260 … "snapshot" … no such file after moving container_app_root:: an image unpacked before the move still points at the old app root. Put a symlink at ~/Library/Application Support/com.apple.container to the new root, or pull or bake the image again. See Storage on another volume.
  • … is not ready: ownership is disabled on /Volumes/Storage, or Apple container says apiserver is not running and not registered with launchd with an app root on an external volume — the volume ignores file owners, and launchd rejects the container service plists under the app root (bad ownership/permissions). Run sudo diskutil enableOwnership /Volumes/Storage, then copy the container files again: files copied before ownership was enabled can still be rejected even with the right owner and mode. Stop the container system first (container system stop), copy the tree fresh (ditto), then start kinhin again.
  • Token rejected — re-run kinhin auth set; the PAT needs repo scope and access to the repo.
  • First launch blocked — only applies to old releases or locally built copies (official releases are signed and notarized). Right-click → Open, or xattr -cr /Applications/kinhin.app.

Building kinhin and CI failures (for contributors)#

CI builds on the kinhin-xcode runner with Xcode 26.5 (Swift 6.3.x). A newer Xcode on your Mac can accept code that CI's compiler rejects or crashes on, so a green local build is not proof. Run the CI jobs on your branch and read the log from the terminal:

gh workflow run ci.yml --ref my-branch                       # test, tsan and docs on your branch
gh run list --workflow ci.yml --branch my-branch --limit 3   # find the run
gh run view <run id> --json jobs --jq '.jobs[] | "\(.name) \(.conclusion) \(.databaseId)"'
gh api repos/RabinApps/kinhin/actions/jobs/<job id>/logs --allow-escape-sequences | sed 's/\x1b\[[0-9;]*m//g' > job.log
  • error: passing closure as a 'sending' parameter risks causing data races with the note closure captures reference to mutable let 'x' (SendingClosureRisksDataRace) — a local let x = Mutex(...) is captured by a Task or addTask closure that the enclosing task also uses. Swift 6.3 rejects a noncopyable Mutex there; newer compilers accept it. Use Locked(...) (Sources/KinhinKit/Core/Locked.swift) instead: same withLock, but a Sendable class. After a crash or an error the compiler only reports the files it reached, so convert every local Mutex in the file (and its siblings), not only the line it names.
  • compile command failed due to signal 6, SmallVector unable to grow, and While emitting IR SIL function "@$s…_TR" — the compiler crashed building a reabstraction thunk. The one we hit came from handing a main-actor method to an API as a function value (Binding(get: …, set: select)). Pass a closure instead (set: { select($0) }). swift demangle '<the symbol>' names the types in the thunk, which points at the code to look for. Bisect on CI with a throwaway ci-bisect/* branch whose workflow builds one product, since the failing compiler is not on your Mac.
  • Localization tests fail only on CI (the compiled String Catalog is missing from …kinhin_kinhin-app.bundle, or text like 1 instances where 1 instance is expected) — Swift 6.3's default (native) build system copies Localizable.xcstrings without compiling it, so no plural forms exist. Build and test with --build-system swiftbuild (Swift 6.4 already defaults to it). Every kinhin script and workflow passes it; add it when you run swift test by hand with a 6.3 toolchain. A release packaged without it would ship an uncompiled catalog.
  • A test passes alone and locally but fails in swift test --parallel on CI, for example because it finds files another test left behind — parallel runs use several processes that share the machine's temp directory. Give each test its own directory and never assert on, or clean, the shared temp directory. UpdateChecker(downloadRoot:) and the update tests show the pattern.
  • docs job: Video suite … Cannot find package 'yaml' — that job runs the video unit tests with Node only. The video scripts load yaml lazily (only when a YAML config is read) and scripts/video-check.sh runs a one-time npm ci --omit=dev when node_modules is missing. Do not add top-level imports of packages to scripts that the CI planner or the claims check import.
  • Orphaned kinhin daemon processes after the e2e suite — each belongs to a /tmp/ke2e-* sandbox that is gone. A sandbox now stops its daemons when it is disposed, and scripts/e2e-sweep.sh --force clears the rest. Check ps eww -p <pid> | tr ' ' '\n' | grep KINHIN_SUPPORT_DIR shows a /tmp/ke2e-… directory before you kill one by hand.
  • Reproducing CI's compiler locally — the swift.org 6.3.2 toolchain can be unpacked without installing it (pkgutil --expand-full), but it needs an SDK from the same generation: this Mac's macOS 27 SDK needs Swift 6.4, so the build fails with unknown argument: '-target-arch-variant'. Without Xcode 26.5's SDK, use CI as the oracle.