From 3ee7d2147e570fc77a50f8e1d89a09844db2a56d Mon Sep 17 00:00:00 2001 From: gabi-simons Date: Thu, 23 Apr 2026 12:05:34 +0000 Subject: [PATCH 01/23] =?UTF-8?q?feat:=20add=20v1=20=E2=86=92=20v2=20migra?= =?UTF-8?q?tion=20to=20setup=20flow=20(experimental)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `bash nanoclaw.sh` detects a v1 install before channel pairing and does a best-effort automated port of operationally important state. Hands off to a new `/migrate-from-v1` skill for owner seeding and fork customizations. Between the timezone and channel steps, `setup/auto.ts` calls `runMigrateV1()` which orchestrates these registered sub-steps (each a separate entry in the progression log with its own raw log + status block — failures never abort the chain): - **migrate-detect** — scans siblings of the v2 checkout + common $HOME locations; `$NANOCLAW_V1_PATH` overrides authoritatively. Relaxed `package.json` check lets forks + partial installs still match; DB presence is the strongest signal. - **migrate-validate** — asserts v1 DB shape (tables + required columns); writes `schema-mismatch.json` on failure. Subsequent steps short-circuit their DB-dependent parts but still run. - **migrate-db** — seeds `agent_groups` + `messaging_groups` + `messaging_group_agents` from v1's `registered_groups`. JID decomposition (`dc:123` → `channel_type='discord'`, `platform_id='discord:123'`); `trigger_pattern` + `requires_trigger` → `engage_mode` + `engage_pattern` (mirrors migration 010 backfill). Users + user_roles are NOT seeded — the skill does that with an owner interview. Idempotent: existing rows reused, not duplicated. - **migrate-groups** — rsync group folders. v1 `CLAUDE.md` → v2 `CLAUDE.local.md` (v2 composes `CLAUDE.md` at container spawn); v1 `container_config` JSON → `.v1-container-config.json` sidecar for the skill to translate. Tight v1-pattern scan (`/workspace/ipc/tasks`, `store/messages.db`, `[PR_CONTEXT:`, etc.) flags files referencing v1-specific infrastructure — content is NOT modified, just flagged in the handoff. - **migrate-env** — merges v1 `.env` into v2 `.env`, never overwriting existing v2 keys. - **migrate-channel-auth** — per-channel registry tracks v1 env keys, v2 required keys (with source-of-key instructions — e.g. Discord needs `DISCORD_PUBLIC_KEY` which v1 never stored), and candidate on-disk auth state paths (Baileys keystore, matrix sync state, etc.). Missing required v2 keys surface as actionable followups and flip the step to `partial`. - **migrate-channels** — runs `setup/install-.sh` for each detected channel in non-interactive mode. Install-script output is captured to `logs/setup-migration/install-.log` sidecars (silent under the parent spinner). Channels with no v2 adapter get a `not_supported` followup but don't degrade status. - **migrate-tasks** — v1 `scheduled_tasks` → `messages_in` rows with `kind='task'` in each session's `inbound.db`. `schedule_type` mapping (cron / interval / once → v2 cron). Idempotent: skips v1 task ids already present. Inactive rows dumped to `inactive-tasks.json` for reference. Everything writes to `logs/setup-migration/handoff.json` — the source of truth the skill consumes. `.claude/skills/migrate-from-v1/SKILL.md`: - **Phase A** (always): owner seeding + v1 access policy flip (`unknown_sender_policy` public/strict) via `AskUserQuestion`. Pulls sender candidates from v1's `messages` table as hints. - **Phase B** (if followups exist): walks `handoff.followups` — translates `.v1-container-config.json` sidecars, handles `not_supported` channels, fills in missing required keys with instructions on where to get them. - **Phase C** (fork-aware): `git log ..HEAD` in v1. Empty → "no customizations to port." Non-empty → scope choice (mechanical / full interview / reference-only). Portable categories (`container/skills/*`, `.claude/skills/*`, docs) scan+copy with `scanForV1Patterns`. Non-portable (`src/*`, `container/agent-runner/src/*`) stash to `docs/v1-fork-reference/` — explicit "don't translate v1 infra to v2" warning because v1's IPC file queue / single DB don't exist in v2. Clearly marked in README, CLAUDE.md, SKILL.md header, and via a `p.warn` that fires once per run when v1 is detected. Users with no v1 install see a silent skip — no prompts, no noise. Verified end-to-end against a live v1 install (300 discord + 1 discord-supervisor groups, fork with ~15 commits of PR-factory work): - Detect → validate → db (301 rows seeded) → groups (301 CLAUDE.local.md + 178 other files + 1 container_config sidecar) → env (4 keys copied) → channel-auth (flagged missing `DISCORD_APPLICATION_ID` + `DISCORD_PUBLIC_KEY`) → channels (discord installed, discord-supervisor → not_supported) → tasks (0 rows, skipped) - Idempotent re-run: 0 rows created, 903 rows reused; tasks skip if id already present - Fresh-user case: silent skip, no prompts, straight to "You're ready!" - Schema-mismatch case: recorded to `schema-mismatch.json`, chain continues - Unit tests for the pure transforms (`parseJid`, `inferChannelType`, `triggerToEngage`, `scanForV1Patterns`, `looksLikeV1Install`) - Validate `requiredV2Keys` for telegram/slack/matrix/teams/webex/ resend/linear against the actual Chat SDK packages (Discord was verified from real error output) - Widen candidate auth file paths for WhatsApp/Matrix/iMessage based on real non-Discord v1 installs once we have some See docs/v1-to-v2-changes.md for the v1 → v2 architecture diff. --- .claude/skills/migrate-from-v1/SKILL.md | 120 +++++ CHANGELOG.md | 4 + CLAUDE.md | 2 + README.md | 2 + docs/v1-to-v2-changes.md | 172 +++++++ setup/auto.ts | 16 +- setup/index.ts | 8 + setup/migrate-v1.ts | 257 ++++++++++ setup/migrate-v1/channel-auth.ts | 213 ++++++++ setup/migrate-v1/channels.ts | 172 +++++++ setup/migrate-v1/db.ts | 296 +++++++++++ setup/migrate-v1/detect.ts | 107 ++++ setup/migrate-v1/env.ts | 135 +++++ setup/migrate-v1/groups.ts | 230 +++++++++ setup/migrate-v1/shared.ts | 639 ++++++++++++++++++++++++ setup/migrate-v1/tasks.ts | 307 ++++++++++++ setup/migrate-v1/validate.ts | 213 ++++++++ 17 files changed, 2891 insertions(+), 2 deletions(-) create mode 100644 .claude/skills/migrate-from-v1/SKILL.md create mode 100644 docs/v1-to-v2-changes.md create mode 100644 setup/migrate-v1.ts create mode 100644 setup/migrate-v1/channel-auth.ts create mode 100644 setup/migrate-v1/channels.ts create mode 100644 setup/migrate-v1/db.ts create mode 100644 setup/migrate-v1/detect.ts create mode 100644 setup/migrate-v1/env.ts create mode 100644 setup/migrate-v1/groups.ts create mode 100644 setup/migrate-v1/shared.ts create mode 100644 setup/migrate-v1/tasks.ts create mode 100644 setup/migrate-v1/validate.ts diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md new file mode 100644 index 0000000..810ae03 --- /dev/null +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -0,0 +1,120 @@ +--- +name: migrate-from-v1 +description: Finish migrating a NanoClaw v1 install into v2. Run this after `bash nanoclaw.sh` has completed its automated migration step. Seeds the owner user, applies v1 access defaults, fixes any migration sub-step that didn't finish, and interviews the user about custom v1 code to port forward. Triggers on "migrate from v1", "finish migration", "v1 migration", or automatically after setup when `logs/setup-migration/handoff.json` exists. +--- + +# Migrate from v1 to v2 + +> ⚠️ **Experimental.** This skill and the setup migration step are early. Remind the user to back up `data/v2.db` + `groups/` before making destructive changes, and prefer small, reversible edits. Not recommended yet for high-stakes production installs. + +The setup flow's `migration` step (in `setup/migrate-v1.ts`) already ran a best-effort automated pass. Your job is to finish what it couldn't do automatically, then interview the user about any custom code they had in v1 and help port it forward. + +Read [docs/v1-to-v2-changes.md](../../../docs/v1-to-v2-changes.md) before doing anything — it's the vocabulary for where v1 things moved to in v2. + +## What the automation did + +The setup flow ran these sub-steps (each as its own progression-log entry): + +| Sub-step | What it did | +|----------|-------------| +| `migrate-detect` | Found v1 install on disk (scanned `~/nanoclaw`, `~/.nanoclaw`, `~/Code/nanoclaw`, etc., or `$NANOCLAW_V1_PATH`). | +| `migrate-validate` | Checked v1 DB has expected tables + required columns. | +| `migrate-db` | Seeded `agent_groups` + `messaging_groups` + `messaging_group_agents` from `registered_groups`. Mapped `trigger_pattern`/`requires_trigger` → `engage_mode`/`engage_pattern`. Did NOT seed `users`/`user_roles`. | +| `migrate-groups` | Copied v1 `groups//` to v2. v1 `CLAUDE.md` → v2 `CLAUDE.local.md`. v1 `container_config` JSON → `.v1-container-config.json` sidecar (don't silent-map to v2's `container.json`). | +| `migrate-env` | Merged v1 `.env` keys into v2 `.env` (never overwrote existing keys). | +| `migrate-channel-auth` | Copied non-env auth state per channel (Baileys keystore, matrix state, etc.) based on `CHANNEL_AUTH_REGISTRY` in `setup/migrate-v1/shared.ts`. | +| `migrate-channels` | Ran `setup/install-.sh` for each channel detected in `registered_groups`. | +| `migrate-tasks` | Ported active v1 `scheduled_tasks` into each session's `inbound.db` as `kind='task'` rows. Inactive tasks dumped to `inactive-tasks.json` for reference. | + +## Artifacts to read first + +- `logs/setup-migration/handoff.json` — **start here.** Structured summary of every sub-step: `status`, `fields`, `notes`, plus detected channels, group selection, and a top-level `followups` list. The top-level `overall_status` tells you at a glance what kind of session this is. +- `logs/setup.log` — the progression log. Each `migrate-*` sub-step has one entry with status, duration, and a pointer to its raw log. +- `logs/setup-steps/NN-migrate-*.log` — raw per-sub-step stdout+stderr. Read these when a step failed or you need to understand why. +- `logs/setup-migration/schema-mismatch.json` — only exists if `migrate-validate` rejected the v1 DB shape. Describes what was missing. +- `logs/setup-migration/inactive-tasks.json` — v1 scheduled tasks we didn't migrate (completed, stopped, or unmappable schedule types). + +## Flow + +### Phase A — always run: owner seeding + access policy + +The automation deliberately did not seed `users`, `user_roles`, or flip `messaging_groups.unknown_sender_policy`. v1 has no ground truth for who the owner is, and no single source for the "anyone can message / only known users" setting. Ask the user. + +1. Read `handoff.json` → `detected_channels` to know which channel(s) to address the user on. +2. Use `AskUserQuestion` to ask "Which handle on `` is yours?" with options pulled from context if you have any hints (e.g. recent v1 message senders), plus "Let me type it" and "Use a different channel." Build the user id as `:`. +3. Insert into v2 central DB (`data/v2.db`): + - `users(id, kind, display_name, created_at)` — use the channel_type as `kind`. + - `user_roles(user_id, role='owner', agent_group_id=NULL, granted_by=NULL, granted_at=now)`. +4. Ask "In v1, could anyone message your assistant, or only known users?" via `AskUserQuestion`: + - "Anyone could message it" → update every row in `messaging_groups` (for migrated channel_types) to `unknown_sender_policy='public'`. + - "Only known users" → leave `unknown_sender_policy='strict'`; walk the user through seeding `agent_group_members` rows for each trusted handle they name. + +Use the DB helpers in `src/db/agent-groups.ts`, `src/db/messaging-groups.ts`, and `src/db/user-roles.ts` rather than hand-rolling SQL — they keep the companion `agent_destinations` and indexes correct. Always init the central DB first: + +```ts +import { initDb } from '../src/db/connection.js'; +import { runMigrations } from '../src/db/migrations/index.js'; +import { DATA_DIR } from '../src/config.js'; +import path from 'path'; +const db = initDb(path.join(DATA_DIR, 'v2.db')); +runMigrations(db); +``` + +### Phase B — branch on `handoff.json: overall_status` + +**If `overall_status === 'success'`** and `followups` is empty: go straight to Phase C (customization interview). + +**Otherwise (partial, failed, or non-empty followups)**: walk `handoff.steps` and `handoff.followups` top-to-bottom. For each entry: + +- Read the step's `fields` and `notes` and its raw log (`logs/setup-steps/NN-.log`). +- Explain the situation to the user in one sentence, then propose a fix. +- Do the fix yourself when it's mechanical (re-running an install script, seeding a missed `agent_destinations` row, re-copying a channel's auth files, manually translating an unsupported `schedule_type`). Use `AskUserQuestion` when a judgment call is needed (is this orphan channel worth keeping? is this v1 container_config still relevant?). + +Common cases: + +- **`migrate-validate` status=failed**: the v1 DB had an unexpected shape. Read `schema-mismatch.json`. If tables are missing, the user may have run a very old or customized v1 — ask before trying to salvage. If only columns are missing, you can often proceed by hand-writing the SELECT with the columns that exist. +- **`migrate-db` status=partial, SKIPPED>0**: some `registered_groups` rows didn't seed. The `notes` field of the step entry names each failed folder. Most commonly: a JID we couldn't parse. Ask the user whether to manually wire each. +- **`migrate-channels` status=partial, some entries `not_supported`**: v1 had channels v2 doesn't ship a skill for yet. Ask the user whether to keep the `messaging_groups` rows (they'll stay orphaned until v2 grows the adapter) or delete them. +- **`migrate-channel-auth` has `files_missing`**: for WhatsApp specifically, encryption sessions often can't survive the copy — tell the user a fresh pair may be needed via `/add-whatsapp`. +- **Per-folder `.v1-container-config.json` sidecars exist**: read each, discuss with the user, and translate to v2's `groups//container.json` format. + +### Phase C — customizations (fork-aware) + +NanoClaw recommends running on a fork, so most real v1 installs have at least some customizations. + +**Start with divergence detection.** In the v1 repo at `handoff.v1_path`: + +```bash +cd +git remote -v # identify the upstream remote +git log --oneline /main..HEAD # commits ahead of upstream +``` + +If the log is **empty**: stock v1. Tell the user "no customizations to port" and skip the rest of Phase C. + +If the log has commits, show them to the user and offer a scope via `AskUserQuestion`: + +1. **Mechanical** (recommended) — copy the portable categories (skills, docs), stash the rest as reference. +2. **Full interview** — walk each commit with me, decide one-by-one. Use `Explore` sub-agents for diffs > 10 files. +3. **Reference only** — stash everything to `docs/v1-fork-reference/`, copy nothing now. + +**Portability rules of thumb:** +- **Portable**: `container/skills/*`, `.claude/skills/*`, `docs/*`, top-level config. Scan each with `scanForV1Patterns` (in `setup/migrate-v1/shared.ts`) before copying — clean ones land as-is, dirty ones get a followup. +- **Not portable**: `src/*` (host) and `container/agent-runner/src/*` (agent-runner). v2's architecture is fundamentally different (Node host with split session DBs vs v1's single process + IPC file queue). Stash to `docs/v1-fork-reference/` with a README explaining the v1→v2 mapping — **don't translate**. Mechanical translation is a trap; let the user rebuild the feature on v2 primitives. +- **Already handled**: `groups/*` — `migrate-groups` copied these and flagged v1 patterns. Don't redo in Phase C. +- **Case by case**: `package.json` deps — check whether v2 already has each; never add to v2's lockfile without approval (supply-chain `minimumReleaseAge` applies). + +When stashing, write `docs/v1-fork-reference/README.md` with commits list, stashed source files, and the suggested porting plan. + +## Principles + +- **Never silently copy code.** Read, explain, propose, apply. Show diffs before applying when non-trivial. +- **Credentials are masked when displayed** (first 4 + `...` + last 4 characters). The handoff file doesn't contain values; keep it that way. +- **The v1 checkout is read-only.** We never delete or modify `~/nanoclaw`. If the user wants to retire it later, that's a separate conversation. +- **No migration re-runs.** The `migrate-*` sub-steps are idempotent, but re-running them from inside this skill is almost always the wrong move — finish by hand. Re-running is for when the user re-runs `bash nanoclaw.sh`. +- **`handoff.json` is source of truth across context compactions.** If the conversation gets compacted mid-work, re-read it and `git status` to recover state. Do not maintain a separate state file. + +## When you're done + +- Delete `logs/setup-migration/handoff.json` once every followup is cleared and the user confirms. The file is a working document, not a record — if the user wants a record, offer to move it to `docs/migration-.md` before deleting. +- Tell the user: if the service is running (check `launchctl list | grep nanoclaw` on macOS or `systemctl --user status nanoclaw*` on Linux), restart it so the seeded `users` / `user_roles` / any channel installs take effect. diff --git a/CHANGELOG.md b/CHANGELOG.md index ab2fd5a..cf6992b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes to NanoClaw will be documented in this file. For detailed release notes, see the [full changelog on the documentation site](https://docs.nanoclaw.dev/changelog). +## [Unreleased] + +- **v1 → v2 migration (experimental).** `bash nanoclaw.sh` now detects a v1 install (`~/nanoclaw`, `~/.nanoclaw`, siblings of the v2 checkout, or `$NANOCLAW_V1_PATH`) and runs a best-effort port before channel pairing: seeds `agent_groups`/`messaging_groups`/wirings from v1's `registered_groups` (with trigger_pattern → engage_mode/engage_pattern), copies group folders, merges `.env`, installs v2 channel adapters, and ports `scheduled_tasks`. Hands off to `/migrate-from-v1` for owner seeding and fork customizations. Experimental — back up `data/v2.db` and `groups/` first; see [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). + ## [2.0.0] - 2026-04-22 Major version. NanoClaw v2 is a substantial architectural rewrite. Existing forks should run `/migrate-nanoclaw` (clean-base replay of customizations) or `/update-nanoclaw` (selective cherry-pick) before resuming work. diff --git a/CLAUDE.md b/CLAUDE.md index 7115c4c..b584c73 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,6 +77,7 @@ Exactly one writer per file — no cross-mount lock contention. Heartbeat is a f | `container/skills/` | Container skills mounted into every agent session | | `groups//` | Per-agent-group filesystem (CLAUDE.md, skills, per-group `agent-runner-src/` overlay) | | `scripts/init-first-agent.ts` | Bootstrap the first DM-wired agent (used by `/init-first-agent` skill) | +| `setup/migrate-v1.ts` + `setup/migrate-v1/` | v1→v2 migration (**experimental**); runs inside `setup/auto.ts` before channel pairing. Seeds `agent_groups` + `messaging_groups` + wirings from v1's `registered_groups`, copies group folders, merges `.env`, installs channel adapters, ports scheduled tasks. Writes `logs/setup-migration/handoff.json` for the `/migrate-from-v1` skill to pick up (owner seeding + fork customizations). | ## Channels and Providers (skill-installed) @@ -211,6 +212,7 @@ This project uses pnpm with `minimumReleaseAge: 4320` (3 days) in `pnpm-workspac | [docs/setup-wiring.md](docs/setup-wiring.md) | What's wired, what's open in the setup flow | | [docs/architecture-diagram.md](docs/architecture-diagram.md) | Diagram version of the architecture | | [docs/build-and-runtime.md](docs/build-and-runtime.md) | Runtime split (Node host + Bun container), lockfiles, image build surface, CI, key invariants | +| [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) | v1→v2 architecture diff — vocabulary for where v1 things moved. Referenced by the migration step and `/migrate-from-v1` skill | ## Container Build Cache diff --git a/README.md b/README.md index de61f6d..9228d40 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,8 @@ bash nanoclaw.sh `nanoclaw.sh` walks you from a fresh machine to a named agent you can message. It installs Node, pnpm, and Docker if missing, registers your Anthropic credential with OneCLI, builds the agent container, and pairs your first channel (Telegram, Discord, WhatsApp, or a local CLI). If a step fails, Claude Code is invoked automatically to diagnose and resume from where it broke. +**Coming from v1?** `nanoclaw.sh` detects your old install (scans siblings of the v2 checkout + common `$HOME` locations) and migrates automatically; `/migrate-from-v1` in Claude finishes owner setup and helps port custom fork work. If v1 is at a non-standard path, run with `NANOCLAW_V1_PATH=/path/to/nanoclaw bash nanoclaw.sh`. See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). _Experimental — back up `data/v2.db` and `groups/` first; not recommended yet for high-stakes production installs._ + ## Philosophy **Small enough to understand.** One process, a few source files and no microservices. If you want to understand the full NanoClaw codebase, just ask Claude Code to walk you through it. diff --git a/docs/v1-to-v2-changes.md b/docs/v1-to-v2-changes.md new file mode 100644 index 0000000..f81fb58 --- /dev/null +++ b/docs/v1-to-v2-changes.md @@ -0,0 +1,172 @@ +# NanoClaw v1 → v2 — what changed + +Big-picture differences between NanoClaw v1 (the `~/nanoclaw` checkout you've been running) and v2 (this rewrite). Not a migration guide — that's what `bash nanoclaw.sh` and the `/migrate-from-v1` skill are for. This doc is the **vocabulary**: when something has moved or been renamed, find it here. + +Read this before touching the migration code or porting customizations forward. + +--- + +## One-line summary + +v1 was one Node process with one SQLite file and native channel adapters. v2 is a host that spawns per-session Docker containers, splits state across a central DB + per-session DB pair, routes through an explicit entity model, and installs channels as skills from a sibling branch. + +--- + +## Entity model — the biggest shift + +**v1:** one flat table `registered_groups(jid, name, folder, trigger_pattern, requires_trigger, is_main, channel_name)`. A group folder is the unit of agent identity. A chat (JID) is wired to exactly one folder, and `trigger_pattern` is an opaque regex the router applies to every incoming message. + +**v2:** three tables, with a deliberate many-to-many in the middle: + +``` +agent_groups ─┐ + ├─ messaging_group_agents ─┬─ messaging_groups + │ (engage_mode, │ (channel_type, + │ engage_pattern, │ platform_id, + │ sender_scope, │ unknown_sender_policy) + │ ignored_message_policy, + │ session_mode, priority) +``` + +Consequences: + +- **One agent can answer on many chats, and one chat can fan out to many agents.** v1 couldn't do either. +- **No `is_main` flag.** Privilege is now explicit via `user_roles` (owner/admin, global or scoped). See below. +- **No `trigger_pattern` regex.** Replaced with four orthogonal columns. Mapping rule used by the automated migration and by the `/migrate-from-v1` skill: + - v1 `trigger_pattern` non-empty → v2 `engage_mode='pattern'`, `engage_pattern = ` + - v1 `requires_trigger=0` or pattern was `.`/`.*` → v2 `engage_mode='pattern'`, `engage_pattern='.'` (the "always" flavor) + - no pattern and requires a trigger → v2 `engage_mode='mention'` + - `sender_scope` and `ignored_message_policy` are new; defaults `all` / `drop` +- **JID decomposition.** v1's `jid` column stored `dc:12345` / `tg:67890`. v2 splits this into `channel_type` + `platform_id`. Concretely: `dc:12345` becomes `channel_type='discord'`, `platform_id='discord:12345'`. Prefix aliases (`dc` → `discord`, `tg` → `telegram`, `wa` → `whatsapp`) are in `setup/migrate-v1/shared.ts`. +- **`channel_name` was unreliable in v1.** Many rows had it empty; the actual channel had to be guessed from the JID prefix. v2's `channel_type` is always explicit. + +--- + +## Central DB vs session DBs + +**v1:** one SQLite file at `store/messages.db`. Every chat, message, registered group, scheduled task, and session lived there. Host and any agent processes all opened the same file. + +**v2:** three DB shapes. + +1. `data/v2.db` — **central**. Everything that isn't per-session: users, roles, agent groups, messaging groups, wirings, pending approvals, user DMs, schema migrations. +2. `data/v2-sessions//inbound.db` — **host writes, container reads**. `messages_in`, routing, destinations, pending questions, processing_ack. This is where scheduled tasks live (see "Scheduling" below). +3. `data/v2-sessions//outbound.db` — **container writes, host reads**. `messages_out`, session_state. + +Exactly one writer per file. No cross-mount lock contention. Heartbeat is a file touch at `/workspace/.heartbeat`, not a DB update. Host uses even `seq` numbers, container uses odd. + +Message history (v1 `messages` table, v1 `chats` table) is **not migrated**. The migration copies operationally important state forward (agents, channels, wirings, scheduled tasks, group folders) and leaves chat logs behind. + +--- + +## Scheduling + +**v1:** dedicated `scheduled_tasks` table in `store/messages.db` with its own columns (`schedule_type`, `schedule_value`, `next_run`, `last_run`, `context_mode`, `script`, `status`). A separate cron-ish scheduler process read from it. + +**v2:** scheduled tasks are **`messages_in` rows with `kind='task'`** in a session's `inbound.db`. Relevant columns: +- `process_after` (ISO8601) — host sweep wakes the container when `datetime(process_after) <= datetime('now')` +- `recurrence` — cron string; `NULL` = one-shot +- `series_id` — groups recurring occurrences; set to the task id on first insert +- `status` — `pending` | `processing` | `completed` | `failed` | `paused` + +The public API is `insertTask()` in `src/modules/scheduling/db.ts`. Recurrence is computed in the user's TZ via `cron-parser` (see `src/modules/scheduling/recurrence.ts`). The migration maps v1's `schedule_type`+`schedule_value` pair into a single cron string before calling `insertTask()`. + +Tasks can exist before a session is awake — the host sweep creates/wakes the container on the first due tick. + +--- + +## Credentials + +**v1:** `.env` — plain environment variables. `DISCORD_BOT_TOKEN`, `ANTHROPIC_API_KEY`, etc. The host read them directly and passed them in to any code that needed them. + +**v2:** OneCLI Agent Vault. A separate local service at `http://127.0.0.1:10254` holds secrets. Agents are *scoped* to specific secrets and the vault injects them into approved API requests as they leave the container. The container never sees the raw secret value. + +Gotcha: auto-created agents default to `selective` secret mode — no secrets attached, even if matching secrets exist in the vault. See the "auto-created agents start in selective secret mode" section of the root CLAUDE.md for the fix (`onecli agents set-secret-mode --mode all`). + +**What the automated migration does:** copies every v1 `.env` key verbatim into v2 `.env`, never overwriting existing v2 keys. The OneCLI vault migration is a separate step owned by the `/init-onecli` skill, which knows how to pull from `.env`. + +--- + +## Channel adapters + +**v1:** native adapters (e.g. `discord.js` used directly) imported in `src/channels/`. Installing a channel meant editing code, adding a dependency, and setting env vars. + +**v2:** channel adapters live on a sibling `channels` branch. Each `/add-` skill: +1. `git fetch origin channels` +2. `git show channels:src/channels/.ts > src/channels/.ts` +3. Appends `import './.js';` to `src/channels/index.ts` +4. `pnpm install @chat-adapter/@` +5. `pnpm run build` + +Idempotent — re-running is a no-op. Pinned versions keep the supply chain honest. The automated migration detects which channels were wired in v1 (via distinct `channel_name` / JID prefix) and runs the matching `setup/install-.sh` for each. Channels in v1 that don't have a v2 skill (rare now, more common as v2 catches up) are recorded in the handoff file for the `/migrate-from-v1` skill to raise with the user. + +**Channel auth beyond `.env`.** Some channels store session state on disk (Baileys WhatsApp keystore, Matrix sync state, iMessage tokens). The `channel-auth` sub-step has a per-channel registry (`setup/migrate-v1/shared.ts: CHANNEL_AUTH_REGISTRY`) that knows which file globs to copy alongside env keys. + +--- + +## Privilege — from implicit to explicit + +**v1:** `registered_groups.is_main = 1` flagged one group as the privileged one. No `users` table. Permissions were conventions, not enforced. + +**v2:** explicit tables. +- `users(id = ":", kind, display_name)` — one row per messaging-platform identifier +- `user_roles(user_id, role ∈ {owner, admin}, agent_group_id nullable, granted_by, granted_at)` — owner is always global; admin can be global or scoped +- `agent_group_members(user_id, agent_group_id, ...)` — "known" membership for the `sender_scope='known'` gate + +Owner gets seeded during the `/migrate-from-v1` skill's interview phase ("Which handle is you?"). The automated migration doesn't guess — v1 has no source of truth for it. + +**Default access — "anyone can talk to the bot" vs "only known users".** v1 stored this implicitly (via trigger regex + `is_main`). v2 exposes it as `messaging_groups.unknown_sender_policy ∈ {'strict', 'request_approval', 'public'}`. The skill asks the user which mode v1 ran in and flips the migrated messaging groups accordingly. + +--- + +## Group folders on disk + +**v1:** `groups//CLAUDE.md` and optional `logs/`. `CLAUDE.md` was a plain instruction file, group-specific. + +**v2:** each group still lives at `groups//`, but the shape is richer: +- `CLAUDE.md` — **composed at container spawn** from `.claude-shared.md` (symlink to global) + `.claude-fragments/*.md` (module fragments) + `CLAUDE.local.md`. **Don't edit `CLAUDE.md` directly.** +- `CLAUDE.local.md` — per-group content. The migration writes v1's old `CLAUDE.md` here. +- `container.json` — optional per-group container config (apt deps, env, mounts). v1's `registered_groups.container_config` JSON is close but not identical — the migration stores the v1 payload at `groups//.v1-container-config.json` for the skill to reconcile, rather than silently mapping it. +- `.claude-fragments/` and `.claude-shared.md` are installed by `initGroupFilesystem()` the first time the host touches the group, so the migration only has to write `CLAUDE.local.md` and leave the scaffolding to the host. + +--- + +## Host process vs containers + +**v1:** single Node process. The "agent" was the same process as the router. + +**v2:** Node host at top, Bun-runtime Docker container per session. They communicate only via the two session DBs. No shared modules, no IPC, no stdin piping. If you wrote custom code that reached from the agent into host internals (or vice versa), that surface no longer exists — porting it is a `/migrate-from-v1` skill topic, not a mechanical copy. + +Lockfiles: host uses `pnpm-lock.yaml`, agent-runner uses `bun.lock`. `minimumReleaseAge: 4320` on the host side (3-day supply-chain wait); agent-runner has no release-age gate. + +--- + +## Self-modification and MCP tools + +**v1:** if you added MCP servers or self-modification plumbing, it was usually direct edits to the long-running process. + +**v2:** +- MCP servers register through `container/agent-runner/src/mcp-tools/*.ts` and load per-session. There's also `install_packages` and `add_mcp_server` self-mod tools that go through an admin-approval flow (`src/modules/self-mod/apply.ts`) before rebuilding the container image. +- Custom MCP tools you wrote in v1 map cleanly to the v2 tool registry, but the import paths, runtime (Bun vs Node), and SQL helper differences (`bun:sqlite` uses `$name`-prefixed params) may need adjustment. The skill walks through this. + +--- + +## Things that are gone or don't map + +- **`scheduled_tasks` as a separate table** — moved into session `inbound.db` under `kind='task'`. Migration ports active rows; inactive/completed are exported to `logs/setup-migration/inactive-tasks.json` for reference. +- **`messages` / `chats` tables (chat history)** — not migrated. Stay in the v1 checkout if you need them. +- **`router_state` (key/value)** — not migrated. v2 state lives in the explicit tables above. +- **`sessions` (v1 group→session_id)** — v1 sessions don't map; v2 sessions are keyed by `(agent_group_id, messaging_group_id, thread_id)` and are created on demand. +- **Raw access to the old `store/messages.db`** — the v1 DB is left in place and untouched. If migration goes wrong you can re-run it (the migration sub-steps are idempotent for agents/channels/wirings; folders use rsync semantics). + +--- + +## Migration surface — where the code lives + +- `setup/migrate-v1.ts` — orchestrator called from `setup/auto.ts` between the timezone and channel steps. +- `setup/migrate-v1/.ts` — registered in `setup/index.ts` STEPS; each runs under `runQuietStep`. +- `logs/setup.log` — progression log. Each sub-step appends one entry. +- `logs/setup-steps/NN-migrate-.log` — raw per-sub-step stdout/stderr. +- `logs/setup-migration/handoff.json` — summary of what was migrated, what failed, what was deferred. Read by the `/migrate-from-v1` skill. +- `logs/setup-migration/schema-mismatch.json` — written only if `migrate-validate` finds a v1 DB that doesn't match the expected shape. The skill uses this to decide what to hand back to the user. +- `logs/setup-migration/inactive-tasks.json` — completed or stopped v1 `scheduled_tasks`, exported for reference. +- `.claude/skills/migrate-from-v1/SKILL.md` — tells Claude how to finish anything the automation couldn't and then interview the user about custom code changes. diff --git a/setup/auto.ts b/setup/auto.ts index 4c20262..720ea0f 100644 --- a/setup/auto.ts +++ b/setup/auto.ts @@ -14,8 +14,10 @@ * "Terminal Agent". * NANOCLAW_SKIP comma-separated step names to skip * (environment|container|onecli|auth|mounts| - * service|cli-agent|timezone|channel|verify| - * first-chat) + * service|cli-agent|timezone|migration|channel| + * verify|first-chat) + * NANOCLAW_V1_PATH explicit path to a v1 install to migrate + * from (default: scan common locations) * * Timezone is auto-detected after the CLI agent step. UTC resolves are * confirmed with the user, and free-text replies fall through to a @@ -36,6 +38,7 @@ import { pingCliAgent, type PingResult } from './lib/agent-ping.js'; import { brightSelect } from './lib/bright-select.js'; import { offerClaudeAssist } from './lib/claude-assist.js'; import { runWindowedStep } from './lib/windowed-runner.js'; +import { runMigrateV1 } from './migrate-v1.js'; import { getLaunchdLabel, getSystemdUnit } from '../src/install-slug.js'; import { claudeCliAvailable, @@ -306,7 +309,16 @@ async function main(): Promise { await runTimezoneStep(); } + if (!skip.has('migration')) { + // Runs silently when there's no v1 install; otherwise orchestrates the + // detect → validate → db → groups → env → channel-auth → channels → + // tasks sub-steps and writes logs/setup-migration/handoff.json for the + // /migrate-from-v1 skill to pick up. + await runMigrateV1(); + } + let channelChoice: ChannelChoice = 'skip'; + if (!skip.has('channel')) { channelChoice = await askChannelChoice(); if (channelChoice === 'telegram') { diff --git a/setup/index.ts b/setup/index.ts index 25d1934..b327541 100644 --- a/setup/index.ts +++ b/setup/index.ts @@ -22,6 +22,14 @@ const STEPS: Record< onecli: () => import('./onecli.js'), auth: () => import('./auth.js'), 'cli-agent': () => import('./cli-agent.js'), + 'migrate-detect': () => import('./migrate-v1/detect.js'), + 'migrate-validate': () => import('./migrate-v1/validate.js'), + 'migrate-db': () => import('./migrate-v1/db.js'), + 'migrate-groups': () => import('./migrate-v1/groups.js'), + 'migrate-env': () => import('./migrate-v1/env.js'), + 'migrate-channel-auth': () => import('./migrate-v1/channel-auth.js'), + 'migrate-channels': () => import('./migrate-v1/channels.js'), + 'migrate-tasks': () => import('./migrate-v1/tasks.js'), }; async function main(): Promise { diff --git a/setup/migrate-v1.ts b/setup/migrate-v1.ts new file mode 100644 index 0000000..a3c1883 --- /dev/null +++ b/setup/migrate-v1.ts @@ -0,0 +1,257 @@ +/** + * v1 → v2 migration orchestrator. Called from setup/auto.ts after the + * timezone step and before the channel step. + * + * Silent happy path: if no v1 install is found, we emit one "skipped" step + * and return. Users on a fresh v2 install never see anything. + * + * When v1 IS found: detect → [confirm] → group-selection prompt → validate + * → db → groups → env → channel-auth → channels → tasks → handoff. + * Every sub-step is a separate entry in the progression log; failures never + * abort the chain (the handoff file records them for the skill to finish). + * + * After everything runs, a one-line note points the user at the + * `/migrate-from-v1` skill. + */ +import fs from 'fs'; +import path from 'path'; + +import * as p from '@clack/prompts'; +import Database from 'better-sqlite3'; +import k from 'kleur'; + +import { ensureAnswer, runQuietStep } from './lib/runner.js'; +import { wrapForGutter } from './lib/theme.js'; +import * as setupLog from './logs.js'; +import { + HANDOFF_PATH, + MIGRATION_DIR, + inferChannelType, + readHandoff, + v1PathsFor, + writeHandoff, +} from './migrate-v1/shared.js'; + +/** + * Count groups in v1's registered_groups, split by whether the channel_type + * can be inferred. Uses the same `inferChannelType` logic as migrate-db so + * the displayed count matches what will actually get seeded. Open-and-close + * because this runs in the orchestrator before migrate-db's child process. + */ +function countV1Groups(v1Root: string): { total: number; wired: number } { + const dbPath = v1PathsFor(v1Root).db; + try { + const db = new Database(dbPath, { readonly: true, fileMustExist: true }); + const rows = db + .prepare('SELECT jid, channel_name FROM registered_groups') + .all() as Array<{ jid: string; channel_name: string | null }>; + db.close(); + let wired = 0; + for (const r of rows) { + if (inferChannelType(r.jid, r.channel_name)) wired++; + } + return { total: rows.length, wired }; + } catch { + return { total: 0, wired: 0 }; + } +} + +async function askGroupSelection(counts: { total: number; wired: number }): Promise<'all' | 'wired-only' | 'cancel'> { + // Non-interactive escape hatch for CI / re-runs / scripted migrations. + // NANOCLAW_MIGRATE_SELECTION = 'all' | 'wired-only' | 'cancel'. + const envChoice = process.env.NANOCLAW_MIGRATE_SELECTION?.trim(); + if (envChoice === 'all' || envChoice === 'wired-only' || envChoice === 'cancel') { + setupLog.userInput('migrate_selection', `${envChoice} (from NANOCLAW_MIGRATE_SELECTION)`); + return envChoice; + } + // Most v1 installs accumulated many orphan folders. Default the user to + // wired-only (the ones we can actually route) — explicit opt-in for "all". + const choice = ensureAnswer( + await p.select({ + message: `Found ${counts.total} v1 group folders (${counts.wired} wired to a channel). Which to bring over?`, + options: [ + { + value: 'wired-only', + label: `Only the ${counts.wired} wired ones`, + hint: 'recommended — skips orphans', + }, + { + value: 'all', + label: `All ${counts.total} folders`, + hint: 'brings dead/orphan folders over too', + }, + { + value: 'cancel', + label: 'Skip migration', + hint: "I'll migrate later", + }, + ], + }), + ) as 'all' | 'wired-only' | 'cancel'; + setupLog.userInput('migrate_selection', choice); + return choice; +} + +/** + * Finalize the handoff record after every sub-step has run. Computes an + * overall status from per-step statuses: anything `failed` → partial; + * anything `partial` → partial; else success. + */ +function finalizeHandoff(): 'success' | 'partial' | 'failed' { + const h = readHandoff(); + const statuses = Object.values(h.steps).map((s) => s?.status); + const anyFailed = statuses.includes('failed'); + const anyPartial = statuses.includes('partial'); + const overall: 'success' | 'partial' | 'failed' = anyFailed + ? 'partial' // DB or files may have landed; the skill can pick up the rest + : anyPartial + ? 'partial' + : 'success'; + h.overall_status = overall; + writeHandoff(h); + return overall; +} + +function printHandoffNote(overall: 'success' | 'partial' | 'failed'): void { + const relHandoff = path.relative(process.cwd(), HANDOFF_PATH); + const lines: string[] = []; + if (overall === 'success') { + lines.push( + wrapForGutter( + 'Your v1 install has been migrated. Run `/migrate-from-v1` in Claude next — it will seed your owner account and help port any custom code you had.', + 4, + ), + ); + } else { + lines.push( + wrapForGutter( + 'Migration finished with some items for a human. Run `/migrate-from-v1` in Claude — it will read the handoff, finish the unfinished steps, and walk through custom code.', + 4, + ), + ); + } + lines.push(''); + lines.push(k.dim(` Handoff: ${relHandoff}`)); + lines.push(k.dim(` Full log: ${setupLog.progressLogPath}`)); + lines.push(k.dim(` Raw logs: ${setupLog.stepsDir}/`)); + p.note(lines.join('\n'), 'Migration handoff'); +} + +export async function runMigrateV1(): Promise<'proceeded' | 'skipped' | 'cancelled'> { + // 0. Ensure migration log dir exists before any sub-step writes to it. + fs.mkdirSync(MIGRATION_DIR, { recursive: true }); + + // 1. Detect. If nothing obvious, give the user one subtle chance to point + // us at a non-standard path — then accept silently. + const detect = await runQuietStep('migrate-detect', { + running: 'Checking for a previous NanoClaw install…', + done: 'Found a previous install.', + skipped: 'No previous install to migrate.', + }); + + const v1Found = detect.ok && detect.terminal?.fields.STATUS === 'success'; + + if (!v1Found) { + // Silent skip — the 99% case is a fresh install with no v1 anywhere. + // Prompting for a custom path on every fresh run is UX noise. Users + // with a v1 at a non-standard location use `NANOCLAW_V1_PATH= + // bash nanoclaw.sh` (documented in README + setup/auto.ts header). + return 'skipped'; + } + + // 2. Ask the user which groups to bring over. + const h = readHandoff(); + if (!h.v1_path) { + // Shouldn't happen — detect set it if v1Found. Guard anyway. + return 'skipped'; + } + + // Experimental warning — fires only when a v1 install is found, so stock + // v2 users (no v1 to migrate) never see it. Not a blocker; the user can + // still proceed. Skip when NANOCLAW_MIGRATE_SELECTION is set (scripted / + // CI runs have already accepted the risk by defining their selection). + if (!process.env.NANOCLAW_MIGRATE_SELECTION) { + p.log.warn( + wrapForGutter( + 'v1 → v2 migration is experimental. Back up your v2 state (data/v2.db, groups/) before continuing. Not recommended for high-stakes production installs — it does a best-effort port and a human still has to finish via /migrate-from-v1.', + 4, + ), + ); + } + + const counts = countV1Groups(h.v1_path); + const selection = await askGroupSelection(counts); + if (selection === 'cancel') { + // Mark the handoff so the skill can still see what would have happened. + const ho = readHandoff(); + ho.overall_status = 'skipped'; + writeHandoff(ho); + return 'cancelled'; + } + + // 3. Validate — if it fails, subsequent steps will short-circuit the + // DB-dependent parts. Groups + env still run. + await runQuietStep('migrate-validate', { + running: "Checking the v1 database's shape…", + done: 'v1 database looks good.', + failed: "v1 database didn't match what I expected.", + skipped: 'Skipped database validation.', + }); + + // 4. DB seeding — parameterized by the user's selection. + await runQuietStep( + 'migrate-db', + { + running: 'Seeding v2 agents and channels from v1…', + done: 'Seeded v2 database.', + skipped: 'Skipped database seeding.', + failed: "Couldn't seed the v2 database.", + }, + ['--selection', selection], + ); + + // 5. Group folders. + await runQuietStep('migrate-groups', { + running: 'Copying group folders…', + done: 'Group folders copied.', + skipped: 'Skipped group-folder copy.', + failed: "Couldn't copy some group folders.", + }); + + // 6. Env keys. + await runQuietStep('migrate-env', { + running: 'Merging v1 .env into v2 .env…', + done: 'Env keys migrated.', + skipped: 'No env keys to migrate.', + failed: "Couldn't merge .env.", + }); + + // 7. Non-env channel auth (Baileys keystore, matrix state, etc.). + await runQuietStep('migrate-channel-auth', { + running: 'Copying channel auth files…', + done: 'Channel auth copied.', + skipped: 'No channel auth to copy.', + failed: 'Some channel auth files need attention.', + }); + + // 8. Install v2 channel adapters for the detected channels. + await runQuietStep('migrate-channels', { + running: 'Installing v2 channel adapters…', + done: 'Channel adapters installed.', + skipped: 'No channels to install.', + failed: 'Some channel adapters need attention.', + }); + + // 9. Scheduled tasks. + await runQuietStep('migrate-tasks', { + running: 'Porting scheduled tasks…', + done: 'Scheduled tasks ported.', + skipped: 'No scheduled tasks to port.', + failed: 'Some scheduled tasks need attention.', + }); + + // 10. Finalize + hand off. + const overall = finalizeHandoff(); + printHandoffNote(overall); + return 'proceeded'; +} diff --git a/setup/migrate-v1/channel-auth.ts b/setup/migrate-v1/channel-auth.ts new file mode 100644 index 0000000..1ef2bc4 --- /dev/null +++ b/setup/migrate-v1/channel-auth.ts @@ -0,0 +1,213 @@ +/** + * Step: migrate-channel-auth + * + * For each channel detected in migrate-db, copy non-.env auth state from v1 + * to the matching v2 location. Env keys are handled by migrate-env (this + * step reads the registry to confirm they made it over, but doesn't rewrite + * them). Files are copied from the first matching candidate path in the + * registry — missing paths are recorded so the skill can prompt the user. + * + * Destination uses the same relative path on v2 (e.g. v1 has + * `data/sessions/baileys/` → v2 gets `data/sessions/baileys/`). If v2 already + * has a different file/dir at that path, we skip and flag it — never clobber. + */ +import fs from 'fs'; +import path from 'path'; + +import { emitStatus } from '../status.js'; +import { + CHANNEL_AUTH_REGISTRY, + readHandoff, + recordStep, + v1PathsFor, + writeHandoff, +} from './shared.js'; + +/** + * Copy file or directory tree from src to dst. `force: false` means existing + * files on the v2 side are never clobbered — important because we'd otherwise + * overwrite auth state the user may have set up on v2 directly. Returns a + * rough count of files copied (post-hoc walk of the destination). + */ +function copyRecursive(src: string, dst: string): number { + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(path.dirname(dst), { recursive: true }); + fs.cpSync(src, dst, { recursive: true, force: false, errorOnExist: false }); + return countFilesUnder(dst); +} + +function countFilesUnder(p: string): number { + if (!fs.existsSync(p)) return 0; + if (fs.statSync(p).isFile()) return 1; + let n = 0; + for (const entry of fs.readdirSync(p, { withFileTypes: true })) { + n += countFilesUnder(path.join(p, entry.name)); + } + return n; +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-channel-auth', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_CHANNEL_AUTH', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const channels = h.detected_channels; + if (channels.length === 0) { + recordStep('migrate-channel-auth', { + status: 'skipped', + fields: { REASON: 'no-channels-detected' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_CHANNEL_AUTH', { STATUS: 'skipped', REASON: 'no_channels' }); + return; + } + + const v1Paths = v1PathsFor(h.v1_path); + const v1Env = fs.existsSync(v1Paths.env) ? fs.readFileSync(v1Paths.env, 'utf-8') : ''; + const v1EnvKeys = new Set( + v1Env + .split('\n') + .map((line) => line.trim()) + .filter((line) => line && !line.startsWith('#')) + .map((line) => line.split('=')[0].trim()) + .filter(Boolean), + ); + + const results: typeof h.channel_auth = []; + const followups: string[] = []; + let anyMissingRequired = false; + + for (const ch of channels) { + const spec = CHANNEL_AUTH_REGISTRY[ch.channel_type]; + if (!spec) { + // Unknown channel — give the skill enough context to drive a useful + // interview instead of a generic "we don't know." Scan v1's .env for + // keys that look related (substring match on channel name + common + // suffixes) and list v1 state directories the user should check. + const haystack = ch.channel_type.toLowerCase(); + const candidateEnvKeys = [...v1EnvKeys].filter((k) => { + const lk = k.toLowerCase(); + return ( + lk.includes(haystack) || + (haystack.length >= 3 && lk.includes(haystack.slice(0, 3))) + ); + }); + const v1DataDirs = ['data', 'store', 'data/sessions'] + .map((d) => path.join(h.v1_path, d)) + .filter((p) => fs.existsSync(p)); + + results.push({ + channel_type: ch.channel_type, + env_keys_copied: [], + files_copied: [], + files_missing: [], + notes: `Unknown channel (not in CHANNEL_AUTH_REGISTRY). Inferred via ${ch.source}. Candidate v1 env keys: ${candidateEnvKeys.join(', ') || 'none found'}. Check v1 dirs: ${v1DataDirs.join(', ') || '(none)'}.`, + }); + followups.push( + `Channel "${ch.channel_type}" (${ch.group_count} group(s), inferred via ${ch.source}) is not in the auth registry. ` + + `Candidate v1 env keys that may belong to it: ${candidateEnvKeys.length > 0 ? candidateEnvKeys.join(', ') : '(none obvious)'}. ` + + `Check v1 for on-disk auth state under ${v1DataDirs.join(', ') || '(no standard dirs found)'}. ` + + `The skill should interview the user, then add a registry entry to setup/migrate-v1/shared.ts for future migrations.`, + ); + continue; + } + + const envKeysPresentInV1 = spec.v1EnvKeys.filter((key) => v1EnvKeys.has(key)); + + // Check v2's .env for required keys the v2 adapter needs to boot. v1 + // may not have had all of them (e.g. v1's Discord used discord.js + // directly and never stored DISCORD_PUBLIC_KEY which v2's Chat SDK + // requires). Surface missing ones as actionable followups. + const v2EnvPath = path.join(process.cwd(), '.env'); + const v2Env = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; + const v2EnvKeys = new Set( + v2Env + .split('\n') + .map((l) => l.trim()) + .filter((l) => l && !l.startsWith('#')) + .map((l) => l.split('=')[0].trim()) + .filter(Boolean), + ); + const missingRequired = spec.requiredV2Keys.filter((r) => !v2EnvKeys.has(r.key)); + if (missingRequired.length > 0) { + anyMissingRequired = true; + followups.push( + `Channel "${ch.channel_type}" is missing required v2 keys in .env: ${missingRequired + .map((r) => `${r.key} (${r.where})`) + .join('; ')}. The v2 adapter won't boot until these are set.`, + ); + } + + const filesCopied: string[] = []; + const filesMissing: string[] = []; + + for (const relPath of spec.candidatePaths) { + const src = path.join(h.v1_path, relPath); + if (!fs.existsSync(src)) continue; + + const dst = path.join(process.cwd(), relPath); + if (fs.existsSync(dst)) { + followups.push( + `Channel "${ch.channel_type}": v2 already has ${relPath} — left untouched. Reconcile manually if needed.`, + ); + filesMissing.push(`${relPath} (already exists in v2)`); + continue; + } + + try { + const count = copyRecursive(src, dst); + filesCopied.push(`${relPath} (${count} files)`); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + filesMissing.push(`${relPath} (copy failed: ${message})`); + followups.push(`Channel "${ch.channel_type}": failed to copy ${relPath} — ${message}`); + } + } + + if (spec.candidatePaths.length > 0 && filesCopied.length === 0) { + filesMissing.push(`(no candidate paths existed under ${h.v1_path})`); + } + + results.push({ + channel_type: ch.channel_type, + env_keys_copied: envKeysPresentInV1, + files_copied: filesCopied, + files_missing: filesMissing, + notes: spec.note ?? '', + }); + } + + const handoffAfter = readHandoff(); + handoffAfter.channel_auth = results; + handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; + writeHandoff(handoffAfter); + + const anyFileMissing = results.some((r) => r.files_missing.length > 0); + const anyPartial = anyFileMissing || anyMissingRequired; + recordStep('migrate-channel-auth', { + status: anyPartial ? 'partial' : 'success', + fields: { + CHANNELS: channels.map((c) => c.channel_type).join(','), + FILES_COPIED: results.reduce((sum, r) => sum + r.files_copied.length, 0), + FILES_MISSING: results.reduce((sum, r) => sum + r.files_missing.length, 0), + }, + notes: followups, + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_CHANNEL_AUTH', { + STATUS: anyPartial ? 'partial' : 'success', + CHANNELS: channels.map((c) => c.channel_type).join(','), + FILES_COPIED: String(results.reduce((sum, r) => sum + r.files_copied.length, 0)), + FILES_MISSING: String(results.reduce((sum, r) => sum + r.files_missing.length, 0)), + }); +} diff --git a/setup/migrate-v1/channels.ts b/setup/migrate-v1/channels.ts new file mode 100644 index 0000000..89df966 --- /dev/null +++ b/setup/migrate-v1/channels.ts @@ -0,0 +1,172 @@ +/** + * Step: migrate-channels + * + * For each channel detected in migrate-db, run the corresponding v2 + * `setup/install-.sh` script in non-interactive mode. The script + * copies the adapter from the `channels` branch, installs the pinned + * dependency, and rebuilds. Credentials in v2 `.env` (migrate-env already + * copied them) are picked up automatically on the next service restart. + * + * This step does NOT run the pairing flow for each channel (that needs + * interactive prompts). The user is guided through pairing by the normal + * channel-selection step in setup/auto.ts, which happens immediately after + * migration. Installing the adapter first means that step won't have to + * re-install. + * + * Channels not supported in v2 are recorded in the handoff as + * `not_supported` so the skill can raise them with the user. + */ +import { spawn } from 'child_process'; +import fs from 'fs'; +import path from 'path'; + +import { log } from '../../src/log.js'; +import { emitStatus } from '../status.js'; +import { + installScriptForChannel, + readHandoff, + recordStep, + writeHandoff, +} from './shared.js'; + +function runScript(script: string): Promise<{ code: number; stdout: string; stderr: string }> { + return new Promise((resolve) => { + const child = spawn('bash', [script], { + stdio: ['ignore', 'pipe', 'pipe'], + env: { ...process.env, MIGRATION_NONINTERACTIVE: '1' }, + }); + // Capture both streams silently — the parent is under a clack spinner, + // and forwarding to stdout/stderr would break the spinner UI. The full + // transcript still lands in this step's raw log via the parent's tee + // (runner.ts: spawnStep writes this step's stdout/stderr to logs/setup- + // steps/NN-migrate-channels.log already). + let stdout = ''; + let stderr = ''; + child.stdout.on('data', (c: Buffer) => { + stdout += c.toString('utf-8'); + }); + child.stderr.on('data', (c: Buffer) => { + stderr += c.toString('utf-8'); + }); + child.on('close', (code) => + resolve({ code: code ?? 1, stdout, stderr }), + ); + child.on('error', () => + resolve({ code: 1, stdout, stderr: stderr || 'spawn_error' }), + ); + }); +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-channels', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_CHANNELS', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const channels = h.detected_channels; + if (channels.length === 0) { + recordStep('migrate-channels', { + status: 'skipped', + fields: { REASON: 'no-channels-detected' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_CHANNELS', { STATUS: 'skipped', REASON: 'no_channels' }); + return; + } + + const results: typeof h.channels_installed = []; + const followups: string[] = []; + + for (const ch of channels) { + const script = installScriptForChannel(ch.channel_type); + if (!script) { + results.push({ + channel_type: ch.channel_type, + status: 'not_supported', + }); + followups.push( + `Channel "${ch.channel_type}" has no v2 install script. The /migrate-from-v1 skill should ask the user whether to keep it as an orphan messaging_group or drop it.`, + ); + continue; + } + + const absoluteScript = path.join(process.cwd(), script); + if (!fs.existsSync(absoluteScript)) { + results.push({ + channel_type: ch.channel_type, + status: 'failed', + error: `install script missing at ${script}`, + }); + followups.push(`Install script for "${ch.channel_type}" missing at ${script} — this is a v2 repo issue, not a user issue.`); + continue; + } + + log.info('Running channel install script', { channel: ch.channel_type, script: absoluteScript }); + const { code, stdout, stderr } = await runScript(absoluteScript); + // Persist the install-script output to a sidecar so the skill can read it + // if diagnosis is needed. The parent's tee already captures our own + // stdout/stderr but the nested script's output is lost otherwise. + try { + const sidecar = path.join( + process.cwd(), + 'logs', + 'setup-migration', + `install-${ch.channel_type}.log`, + ); + fs.mkdirSync(path.dirname(sidecar), { recursive: true }); + fs.writeFileSync(sidecar, `# ${script}\n# exit ${code}\n\n=== stdout ===\n${stdout}\n=== stderr ===\n${stderr}\n`); + } catch { + // Sidecar is diagnostic-only — don't abort if the log dir is unwritable. + } + if (code === 0) { + results.push({ channel_type: ch.channel_type, status: 'success' }); + } else { + results.push({ + channel_type: ch.channel_type, + status: 'failed', + error: stderr.trim().slice(0, 400) || `exit ${code}`, + }); + followups.push( + `Installing "${ch.channel_type}" failed (exit ${code}). The /migrate-from-v1 skill should retry ${script} or walk the user through /add-${ch.channel_type}.`, + ); + } + } + + const handoffAfter = readHandoff(); + handoffAfter.channels_installed = results; + handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; + writeHandoff(handoffAfter); + + // `not_supported` is an expected/known outcome for channels whose v1 adapter + // has no v2 equivalent yet. It's a followup for the skill to raise — not a + // partial success. Only real install failures degrade status. + const anyFailed = results.some((r) => r.status === 'failed'); + const status: 'success' | 'partial' | 'failed' = anyFailed ? 'partial' : 'success'; + + recordStep('migrate-channels', { + status, + fields: { + INSTALLED: results.filter((r) => r.status === 'success').length, + FAILED: results.filter((r) => r.status === 'failed').length, + NOT_SUPPORTED: results.filter((r) => r.status === 'not_supported').length, + CHANNELS: results.map((r) => `${r.channel_type}=${r.status}`).join(','), + }, + notes: followups, + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_CHANNELS', { + STATUS: status, + INSTALLED: String(results.filter((r) => r.status === 'success').length), + FAILED: String(results.filter((r) => r.status === 'failed').length), + NOT_SUPPORTED: String(results.filter((r) => r.status === 'not_supported').length), + }); +} diff --git a/setup/migrate-v1/db.ts b/setup/migrate-v1/db.ts new file mode 100644 index 0000000..e455012 --- /dev/null +++ b/setup/migrate-v1/db.ts @@ -0,0 +1,296 @@ +/** + * Step: migrate-db + * + * Seed v2.db with the essentials derived from v1's `registered_groups`: + * - agent_groups: one per v1 folder the user selected + * - messaging_groups: one per distinct (channel_type, platform_id) pair + * - messaging_group_agents: the wiring between them, with engage fields + * backfilled from v1's trigger_pattern / requires_trigger + * + * Does NOT seed users, user_roles, or agent_group_members. v1 has no ground + * truth for them — the /migrate-from-v1 skill interviews the user for the + * owner and seeds those tables. + * + * Idempotent: re-running skips any (folder) agent_group, (channel, platform_id) + * messaging_group, and (mg, ag) wiring that already exist. Safe to re-run + * after a partial failure. + * + * Expects `--selection ` where mode is 'all' | 'wired-only'. The + * orchestrator asks the user via clack and passes the result. + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { createAgentGroup, getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { initDb } from '../../src/db/connection.js'; +import { + createMessagingGroup, + createMessagingGroupAgent, + getMessagingGroupAgentByPair, + getMessagingGroupByPlatform, +} from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { log } from '../../src/log.js'; +import { emitStatus } from '../status.js'; +import { + generateId, + inferChannelType, + readHandoff, + recordStep, + triggerToEngage, + v1PathsFor, + v2PlatformId, + writeHandoff, +} from './shared.js'; + +interface V1Group { + jid: string; + name: string; + folder: string; + trigger_pattern: string | null; + requires_trigger: number | null; + is_main: number | null; + channel_name: string | null; +} + +interface DbArgs { + selection: 'all' | 'wired-only'; +} + +function parseArgs(args: string[]): DbArgs { + let selection: 'all' | 'wired-only' = 'wired-only'; + for (let i = 0; i < args.length; i++) { + if (args[i] === '--selection') { + const v = args[++i]; + if (v === 'all' || v === 'wired-only') selection = v; + } + } + return { selection }; +} + +export async function run(args: string[]): Promise { + const parsed = parseArgs(args); + const h = readHandoff(); + + if (!h.v1_path) { + recordStep('migrate-db', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_DB', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const validate = h.steps['migrate-validate']; + if (validate && validate.status === 'failed') { + recordStep('migrate-db', { + status: 'skipped', + fields: { REASON: 'validate-failed' }, + notes: ['DB shape did not validate; skipping DB migration.'], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_DB', { STATUS: 'skipped', REASON: 'validate_failed' }); + return; + } + + const paths = v1PathsFor(h.v1_path); + let v1Db: Database.Database; + try { + v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + recordStep('migrate-db', { + status: 'failed', + fields: { REASON: 'v1-db-open-failed' }, + notes: [message], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_DB', { STATUS: 'failed', REASON: 'v1_db_open_failed', ERROR: message }); + return; + } + + const v1Groups = v1Db + .prepare( + 'SELECT jid, name, folder, trigger_pattern, requires_trigger, is_main, channel_name FROM registered_groups', + ) + .all() as V1Group[]; + v1Db.close(); + + // Filter by selection mode. "wired-only" keeps rows where we can confidently + // say which channel they belong to — either `channel_name` is set, or the + // JID prefix resolves to a known channel type. + const selected: V1Group[] = []; + const detectedChannels = new Map(); + + for (const g of v1Groups) { + const channelType = inferChannelType(g.jid, g.channel_name); + const source: 'channel_name' | 'jid_prefix' = g.channel_name?.trim() ? 'channel_name' : 'jid_prefix'; + if (!channelType) { + // Can't infer — skip in both modes; the skill raises it with the user. + continue; + } + if (parsed.selection === 'wired-only' && source === 'jid_prefix' && !channelType) { + continue; + } + selected.push(g); + const entry = detectedChannels.get(channelType) ?? { source, count: 0 }; + entry.count += 1; + // Prefer explicit channel_name as the source if any row had it. + if (source === 'channel_name') entry.source = 'channel_name'; + detectedChannels.set(channelType, entry); + } + + h.group_selection = { + mode: parsed.selection, + selected_folders: selected.map((g) => g.folder), + total_v1_groups: v1Groups.length, + wired_v1_groups: selected.length, + }; + h.detected_channels = [...detectedChannels.entries()].map(([channel_type, info]) => ({ + channel_type, + source: info.source, + group_count: info.count, + })); + writeHandoff(h); + + // Initialize v2.db (creates schema if not present — runMigrations is no-op + // when the schema is already current, so this is safe on a live v2 install). + fs.mkdirSync(path.join(process.cwd(), 'data'), { recursive: true }); + const v2Path = path.join(DATA_DIR, 'v2.db'); + const v2Db = initDb(v2Path); + runMigrations(v2Db); + + let agentGroupsCreated = 0; + let agentGroupsReused = 0; + let messagingGroupsCreated = 0; + let messagingGroupsReused = 0; + let wiringsCreated = 0; + let wiringsReused = 0; + let skipped = 0; + const followups: string[] = []; + + for (const g of selected) { + const channelType = inferChannelType(g.jid, g.channel_name); + if (!channelType) { + skipped += 1; + continue; + } + + const platformId = v2PlatformId(channelType, g.jid); + const createdAt = new Date().toISOString(); + + try { + // agent_group — one per folder + let ag = getAgentGroupByFolder(g.folder); + if (!ag) { + createAgentGroup({ + id: generateId('ag'), + name: g.name || g.folder, + folder: g.folder, + agent_provider: null, + created_at: createdAt, + }); + ag = getAgentGroupByFolder(g.folder)!; + agentGroupsCreated += 1; + } else { + agentGroupsReused += 1; + } + + // messaging_group — one per (channel_type, platform_id) + let mg = getMessagingGroupByPlatform(channelType, platformId); + if (!mg) { + createMessagingGroup({ + id: generateId('mg'), + channel_type: channelType, + platform_id: platformId, + name: g.name || null, + is_group: 1, // v1 didn't distinguish; default to group (safe for routing) + unknown_sender_policy: 'strict', // skill's interview flips this if v1 was "public" + created_at: createdAt, + }); + mg = getMessagingGroupByPlatform(channelType, platformId)!; + messagingGroupsCreated += 1; + } else { + messagingGroupsReused += 1; + } + + // messaging_group_agents — wire them if not already wired + const existingWiring = getMessagingGroupAgentByPair(mg.id, ag.id); + if (!existingWiring) { + const engage = triggerToEngage({ + trigger_pattern: g.trigger_pattern, + requires_trigger: g.requires_trigger, + }); + createMessagingGroupAgent({ + id: generateId('mga'), + messaging_group_id: mg.id, + agent_group_id: ag.id, + engage_mode: engage.engage_mode, + engage_pattern: engage.engage_pattern, + sender_scope: 'all', + ignored_message_policy: 'drop', + session_mode: 'shared', + priority: 0, + created_at: createdAt, + }); + wiringsCreated += 1; + } else { + wiringsReused += 1; + } + + if (g.is_main === 1) { + followups.push( + `Folder "${g.folder}" was the v1 main group (is_main=1). v2 has no is_main flag — the /migrate-from-v1 skill should grant this folder's channel to the owner user when it runs.`, + ); + } + } catch (err) { + skipped += 1; + const message = err instanceof Error ? err.message : String(err); + log.error('Failed to seed v1 group', { folder: g.folder, err: message }); + followups.push(`Folder "${g.folder}" failed to seed: ${message}`); + } + } + + v2Db.close(); + + const partial = skipped > 0; + const handoffAfter = readHandoff(); + handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; + writeHandoff(handoffAfter); + + recordStep('migrate-db', { + status: partial ? 'partial' : 'success', + fields: { + SELECTION: parsed.selection, + V1_GROUPS_TOTAL: v1Groups.length, + SELECTED: selected.length, + AGENT_GROUPS_CREATED: agentGroupsCreated, + AGENT_GROUPS_REUSED: agentGroupsReused, + MESSAGING_GROUPS_CREATED: messagingGroupsCreated, + MESSAGING_GROUPS_REUSED: messagingGroupsReused, + WIRINGS_CREATED: wiringsCreated, + WIRINGS_REUSED: wiringsReused, + SKIPPED: skipped, + CHANNELS: [...detectedChannels.keys()].join(','), + }, + notes: followups, + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_DB', { + STATUS: partial ? 'partial' : 'success', + SELECTION: parsed.selection, + V1_GROUPS_TOTAL: String(v1Groups.length), + SELECTED: String(selected.length), + AGENT_GROUPS_CREATED: String(agentGroupsCreated), + MESSAGING_GROUPS_CREATED: String(messagingGroupsCreated), + WIRINGS_CREATED: String(wiringsCreated), + SKIPPED: String(skipped), + CHANNELS: [...detectedChannels.keys()].join(',') || 'none', + }); +} diff --git a/setup/migrate-v1/detect.ts b/setup/migrate-v1/detect.ts new file mode 100644 index 0000000..983531d --- /dev/null +++ b/setup/migrate-v1/detect.ts @@ -0,0 +1,107 @@ +/** + * Step: migrate-detect + * + * Find a v1 install on disk. Scans the standard candidate paths; if none + * matches, exits with a NOT_FOUND status (the orchestrator then offers a + * clack prompt so the user can point at a custom path). + * + * Never prompts — this step is pure discovery so it stays safe to run under + * NANOCLAW_SKIP= without blocking on stdin. + */ +import fs from 'fs'; +import path from 'path'; + +import { emitStatus } from '../status.js'; +import { + defaultV1Candidates, + looksLikeV1Install, + readHandoff, + recordStep, + v1PathsFor, + writeHandoff, +} from './shared.js'; + +interface DetectArgs { + /** Explicit path to check, skipping the default candidate list. */ + path?: string; +} + +function parseArgs(args: string[]): DetectArgs { + const out: DetectArgs = {}; + for (let i = 0; i < args.length; i++) { + if (args[i] === '--path') { + out.path = args[++i] || undefined; + } + } + return out; +} + +export async function run(args: string[]): Promise { + const parsed = parseArgs(args); + + // An explicit path — either from --path or $NANOCLAW_V1_PATH — is + // authoritative. If it doesn't validate, we don't fall through to + // the default candidate list. That keeps the user's explicit intent + // from being silently overridden. + const envOverride = process.env.NANOCLAW_V1_PATH?.trim(); + const explicit = parsed.path ?? envOverride ?? null; + const candidates = explicit ? [explicit] : defaultV1Candidates(); + + for (const candidate of candidates) { + const absolute = path.resolve(candidate); + // Don't self-match — if the candidate resolves to the v2 checkout we're + // running inside, skip it. Protects users who cloned v2 into `~/nanoclaw` + // after deleting v1. + if (absolute === path.resolve(process.cwd())) continue; + + const check = looksLikeV1Install(absolute); + if (!check.ok) continue; + + const paths = v1PathsFor(absolute); + let version = 'unknown'; + try { + const pkg = JSON.parse(fs.readFileSync(paths.packageJson, 'utf-8')) as { version?: string }; + version = pkg.version ?? 'unknown'; + } catch { + // Already sanity-checked by looksLikeV1Install — a failure here means + // the file changed under us between calls. Unlikely, not fatal. + } + + const h = readHandoff(); + h.v1_path = absolute; + h.v1_version = version; + writeHandoff(h); + + recordStep('migrate-detect', { + status: 'success', + fields: { V1_PATH: absolute, V1_VERSION: version }, + notes: [], + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_DETECT', { + STATUS: 'success', + V1_PATH: absolute, + V1_VERSION: version, + DB_PATH: paths.db, + ENV_PATH: paths.env, + GROUPS_PATH: paths.groups, + }); + return; + } + + // Nothing matched. Not an error — most v2 installs are fresh, not migrations. + const scanned = candidates.map((c) => path.resolve(c)).join(','); + recordStep('migrate-detect', { + status: 'skipped', + fields: { REASON: 'no-v1-install-found' }, + notes: [`Scanned: ${scanned}`], + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_DETECT', { + STATUS: 'skipped', + REASON: 'not_found', + CANDIDATES_SCANNED: String(candidates.length), + }); +} diff --git a/setup/migrate-v1/env.ts b/setup/migrate-v1/env.ts new file mode 100644 index 0000000..e2530e0 --- /dev/null +++ b/setup/migrate-v1/env.ts @@ -0,0 +1,135 @@ +/** + * Step: migrate-env + * + * Copy every key from v1 `.env` to v2 `.env`. Preserves v2 values that + * already exist (never overwrites). Skips lines that don't look like a + * `KEY=value` pair. + * + * Why copy everything, not a curated list? v1 installs accumulate + * project-specific keys (custom MCP creds, feature flags, webhook tokens) + * that the migration can't enumerate ahead of time. The user explicitly + * asked for everything. We log what we carried so the skill can review. + * + * Security note: we do NOT log values here — only keys. The raw log already + * contains the file contents; we don't echo them to stdout. + */ +import fs from 'fs'; +import path from 'path'; + +import { emitStatus } from '../status.js'; +import { readHandoff, recordStep, v1PathsFor } from './shared.js'; + +interface EnvLine { + key: string; + value: string; + raw: string; +} + +function parseEnv(text: string): EnvLine[] { + const out: EnvLine[] = []; + for (const raw of text.split('\n')) { + const line = raw.trimEnd(); + if (!line) continue; + if (line.startsWith('#')) continue; + const eq = line.indexOf('='); + if (eq <= 0) continue; + const key = line.slice(0, eq).trim(); + if (!/^[A-Z_][A-Z0-9_]*$/i.test(key)) continue; + const value = line.slice(eq + 1); + out.push({ key, value, raw: line }); + } + return out; +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-env', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_ENV', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const paths = v1PathsFor(h.v1_path); + if (!fs.existsSync(paths.env)) { + recordStep('migrate-env', { + status: 'skipped', + fields: { REASON: 'v1-env-missing' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_ENV', { STATUS: 'skipped', REASON: 'v1_env_missing' }); + return; + } + + const v2EnvPath = path.join(process.cwd(), '.env'); + const v1Text = fs.readFileSync(paths.env, 'utf-8'); + const v1Lines = parseEnv(v1Text); + + let v2Text = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; + const v2Lines = parseEnv(v2Text); + const v2Keys = new Set(v2Lines.map((l) => l.key)); + + const copied: string[] = []; + const skipped: string[] = []; + const appended: string[] = []; + + // Tag the appended block so a later re-run can find it and not double-append. + const BLOCK_START = '# ── migrated from v1 ──'; + const alreadyMigrated = v2Text.includes(BLOCK_START); + + for (const line of v1Lines) { + if (v2Keys.has(line.key)) { + skipped.push(line.key); + continue; + } + copied.push(line.key); + appended.push(line.raw); + } + + if (appended.length > 0) { + const suffix = [ + v2Text.endsWith('\n') || v2Text === '' ? '' : '\n', + alreadyMigrated ? '' : `\n${BLOCK_START}\n`, + appended.join('\n'), + '\n', + ].join(''); + v2Text = v2Text + suffix; + fs.writeFileSync(v2EnvPath, v2Text); + } + + // Container reads from data/env/env (host mounts it). Keep it in sync. + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + try { + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { + // Non-fatal; the service restart (later step) will rehydrate if needed. + } + + recordStep('migrate-env', { + status: 'success', + fields: { + KEYS_COPIED: copied.length, + KEYS_SKIPPED_EXISTING: skipped.length, + V1_ENV: paths.env, + V2_ENV: v2EnvPath, + }, + notes: [ + copied.length > 0 ? `Copied: ${copied.join(', ')}` : '', + skipped.length > 0 ? `Skipped (already in v2 .env): ${skipped.join(', ')}` : '', + ].filter(Boolean), + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_ENV', { + STATUS: 'success', + KEYS_COPIED: String(copied.length), + KEYS_SKIPPED_EXISTING: String(skipped.length), + COPIED_KEYS: copied.join(',') || 'none', + }); +} diff --git a/setup/migrate-v1/groups.ts b/setup/migrate-v1/groups.ts new file mode 100644 index 0000000..206f441 --- /dev/null +++ b/setup/migrate-v1/groups.ts @@ -0,0 +1,230 @@ +/** + * Step: migrate-groups + * + * Copy v1 group folders into v2. For each folder selected in migrate-db: + * - Create groups// in v2 if missing + * - Copy v1's CLAUDE.md to v2 as CLAUDE.local.md (v2 composes CLAUDE.md at + * container spawn — don't write directly to CLAUDE.md) + * - If v1 had a container_config JSON, write it to .v1-container-config.json + * for the /migrate-from-v1 skill to reconcile (v2's container.json shape + * has drifted enough that a silent 1:1 copy would be wrong) + * - Preserve any other non-standard files from the v1 folder (e.g. SOUL.md, + * personality.md, custom subdirs) — rsync-style, skipping destination files + * that already exist. + * + * Does not overwrite files already present in v2 — re-running is safe. + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { log } from '../../src/log.js'; +import { emitStatus } from '../status.js'; +import { + readHandoff, + recordStep, + safeJsonStringify, + scanForV1Patterns, + v1PathsFor, + writeHandoff, +} from './shared.js'; + +const SKIP_NAMES = new Set(['CLAUDE.md', 'logs', '.git', '.DS_Store', 'node_modules']); + +/** + * Copy everything in src except SKIP_NAMES. CLAUDE.md is handled separately. + * Returns the count of files actually written (skipped-existing not counted). + */ +function copyTree(src: string, dst: string): number { + let written = 0; + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(dst, { recursive: true }); + + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + if (SKIP_NAMES.has(entry.name)) continue; + const s = path.join(src, entry.name); + const d = path.join(dst, entry.name); + + if (entry.isDirectory()) { + written += copyTree(s, d); + continue; + } + // Don't clobber files v2 already has (e.g. CLAUDE.local.md that the + // operator already wrote). Append-only semantics for this step. + if (fs.existsSync(d)) continue; + fs.copyFileSync(s, d); + written += 1; + } + return written; +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-groups', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_GROUPS', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + if (h.group_selection.selected_folders.length === 0) { + recordStep('migrate-groups', { + status: 'skipped', + fields: { REASON: 'no-folders-selected' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_GROUPS', { STATUS: 'skipped', REASON: 'no_selection' }); + return; + } + + const paths = v1PathsFor(h.v1_path); + const v2GroupsDir = path.join(process.cwd(), 'groups'); + fs.mkdirSync(v2GroupsDir, { recursive: true }); + + // Pull container_config for each selected folder up-front so we can write + // the .v1-container-config.json sidecar without holding the DB open per-folder. + const containerConfigs = new Map(); + try { + const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); + const rows = v1Db + .prepare('SELECT folder, container_config FROM registered_groups WHERE folder IN (SELECT value FROM json_each(?))') + .all(JSON.stringify(h.group_selection.selected_folders)) as Array<{ folder: string; container_config: string | null }>; + for (const r of rows) containerConfigs.set(r.folder, r.container_config); + v1Db.close(); + } catch (err) { + // Older sqlite without json_each would break the query. Fall back to + // per-folder reads — slower but reliable. + log.info('Falling back to per-folder container_config lookup', { err }); + try { + const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); + const stmt = v1Db.prepare('SELECT container_config FROM registered_groups WHERE folder = ?'); + for (const folder of h.group_selection.selected_folders) { + const row = stmt.get(folder) as { container_config: string | null } | undefined; + containerConfigs.set(folder, row?.container_config ?? null); + } + v1Db.close(); + } catch { + // Give up — we still migrate files; the skill handles missing config. + } + } + + let foldersProcessed = 0; + let foldersSkippedMissing = 0; + let claudeMdMigrated = 0; + let claudeLocalPreserved = 0; + let containerConfigsStashed = 0; + let otherFilesCopied = 0; + const followups: string[] = []; + + for (const folder of h.group_selection.selected_folders) { + const v1Folder = path.join(paths.groups, folder); + const v2Folder = path.join(v2GroupsDir, folder); + + if (!fs.existsSync(v1Folder)) { + foldersSkippedMissing += 1; + followups.push( + `Folder "${folder}" was in v1's registered_groups but not on disk at ${v1Folder} — DB entry was seeded, no files to migrate.`, + ); + continue; + } + + fs.mkdirSync(v2Folder, { recursive: true }); + + // CLAUDE.md → CLAUDE.local.md. Don't write CLAUDE.md directly — v2's + // group-init.ts composes that file from shared + fragments + local. + const v1Claude = path.join(v1Folder, 'CLAUDE.md'); + const v2Local = path.join(v2Folder, 'CLAUDE.local.md'); + let claudeContent: string | null = null; + if (fs.existsSync(v1Claude)) { + if (fs.existsSync(v2Local)) { + claudeLocalPreserved += 1; + try { + claudeContent = fs.readFileSync(v2Local, 'utf-8'); + } catch { + claudeContent = null; + } + } else { + try { + claudeContent = fs.readFileSync(v1Claude, 'utf-8'); + fs.writeFileSync(v2Local, claudeContent); + claudeMdMigrated += 1; + } catch (err) { + followups.push(`Failed to copy CLAUDE.md for "${folder}": ${err instanceof Error ? err.message : err}`); + } + } + } + + // Scan the copied content for v1-specific infrastructure patterns. If we + // find any, add a followup so the /migrate-from-v1 skill can triage the + // file with the user. We DON'T edit the file — v1 CLAUDE.md can be + // author-specific and heuristic translation is worse than a flag. + if (claudeContent) { + const matches = scanForV1Patterns(claudeContent); + if (matches.length > 0) { + const summary = matches + .map((m) => `${m.description} (lines ${m.lines.join(',')})`) + .join('; '); + followups.push( + `Folder "${folder}" CLAUDE.local.md references v1-specific infrastructure: ${summary}. The skill should read the file and translate patterns using docs/v1-to-v2-changes.md.`, + ); + } + } + + // Stash container_config JSON so the skill can reconcile it. + const config = containerConfigs.get(folder); + if (config) { + const sidecar = path.join(v2Folder, '.v1-container-config.json'); + try { + // Pretty-print so humans can read it during reconciliation. + const parsed = JSON.parse(config) as unknown; + fs.writeFileSync(sidecar, safeJsonStringify(parsed)); + containerConfigsStashed += 1; + followups.push( + `Folder "${folder}" has a v1 container_config — stashed at ${path.relative(process.cwd(), sidecar)}. The /migrate-from-v1 skill will map it to v2's container.json shape.`, + ); + } catch { + // Non-JSON container_config — write raw so the skill can still read it. + fs.writeFileSync(sidecar, config); + containerConfigsStashed += 1; + } + } + + otherFilesCopied += copyTree(v1Folder, v2Folder); + foldersProcessed += 1; + } + + // Merge followups. + const handoffAfter = readHandoff(); + handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; + writeHandoff(handoffAfter); + + const partial = foldersSkippedMissing > 0; + recordStep('migrate-groups', { + status: partial ? 'partial' : 'success', + fields: { + FOLDERS_PROCESSED: foldersProcessed, + FOLDERS_SKIPPED_MISSING: foldersSkippedMissing, + CLAUDE_MD_MIGRATED: claudeMdMigrated, + CLAUDE_LOCAL_PRESERVED: claudeLocalPreserved, + CONTAINER_CONFIGS_STASHED: containerConfigsStashed, + OTHER_FILES_COPIED: otherFilesCopied, + }, + notes: followups, + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_GROUPS', { + STATUS: partial ? 'partial' : 'success', + FOLDERS_PROCESSED: String(foldersProcessed), + FOLDERS_SKIPPED_MISSING: String(foldersSkippedMissing), + CLAUDE_MD_MIGRATED: String(claudeMdMigrated), + CONTAINER_CONFIGS_STASHED: String(containerConfigsStashed), + OTHER_FILES_COPIED: String(otherFilesCopied), + }); +} diff --git a/setup/migrate-v1/shared.ts b/setup/migrate-v1/shared.ts new file mode 100644 index 0000000..f9f03bc --- /dev/null +++ b/setup/migrate-v1/shared.ts @@ -0,0 +1,639 @@ +/** + * Shared types, constants, and helpers for the v1 → v2 migration. + * + * The migration is a sequence of small steps registered in setup/index.ts + * (migrate-detect, migrate-validate, migrate-db, …). Every step: + * - Reads state it needs from `logs/setup-migration/handoff.json` + * - Writes its own outcome back to that handoff file + * - Emits exactly one `=== NANOCLAW SETUP: MIGRATE_ ===` block on stdout + * + * No step aborts the chain on failure — the orchestrator in setup/migrate-v1.ts + * reads the handoff after each step to decide whether to continue, skip, or + * hand off to the Claude `/migrate-from-v1` skill. + */ +import fs from 'fs'; +import os from 'os'; +import path from 'path'; + +// ── Paths ────────────────────────────────────────────────────────────── + +export const MIGRATION_DIR = path.join('logs', 'setup-migration'); +export const HANDOFF_PATH = path.join(MIGRATION_DIR, 'handoff.json'); +export const SCHEMA_MISMATCH_PATH = path.join(MIGRATION_DIR, 'schema-mismatch.json'); +export const INACTIVE_TASKS_PATH = path.join(MIGRATION_DIR, 'inactive-tasks.json'); + +// ── V1 install discovery ─────────────────────────────────────────────── + +/** + * Default candidate paths to scan for a v1 install. Combines: + * - `$NANOCLAW_V1_PATH` (explicit override, takes priority) + * - Sibling directories of the v2 checkout whose name contains "nanoclaw" + * or "clawdbot" (most common layout — v1 lives next to v2) + * - Common checkout locations under $HOME + * - Common XDG-style state dirs (.nanoclaw, .clawdbot — v1's predecessor) + * + * Kept generic — don't bake specific usernames in. Deduped so a path that + * satisfies multiple rules only appears once. + */ +export function defaultV1Candidates(): string[] { + const home = os.homedir(); + const cwd = process.cwd(); + const cwdParent = path.dirname(cwd); + + const siblings: string[] = []; + try { + if (fs.existsSync(cwdParent)) { + for (const entry of fs.readdirSync(cwdParent, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const lower = entry.name.toLowerCase(); + // Match anything claw-ish next to v2: "nanoclaw", "nanoclaw-v1", + // "clawdbot", user's fork name like "nanoclaw-prod". Excludes the + // v2 checkout we're running from so we don't self-match. + if (!lower.includes('claw')) continue; + const full = path.join(cwdParent, entry.name); + if (path.resolve(full) === path.resolve(cwd)) continue; + siblings.push(full); + } + } + } catch { + // Can't list parent — fall through to the fixed list. + } + + const fixed = [ + path.join(home, 'nanoclaw'), + path.join(home, '.nanoclaw'), + path.join(home, 'clawdbot'), + path.join(home, '.clawdbot'), + path.join(home, 'Code', 'nanoclaw'), + path.join(home, 'code', 'nanoclaw'), + path.join(home, 'projects', 'nanoclaw'), + path.join(home, 'Projects', 'nanoclaw'), + path.join(home, 'src', 'nanoclaw'), + path.join(home, 'dev', 'nanoclaw'), + path.join(home, 'workspace', 'nanoclaw'), + path.join(home, 'Documents', 'nanoclaw'), + path.join(home, 'GitHub', 'nanoclaw'), + path.join(home, 'github', 'nanoclaw'), + path.join(home, 'repos', 'nanoclaw'), + ]; + + // NANOCLAW_V1_PATH is handled authoritatively by detect.ts — if it's set, + // detect doesn't call this function at all. So we only build the + // auto-discovery list here. + const all = [...siblings, ...fixed]; + + // Dedupe by resolved path. A sibling "nanoclaw" and a fixed "$HOME/nanoclaw" + // often resolve to the same thing on single-user machines. + const seen = new Set(); + const out: string[] = []; + for (const p of all) { + const resolved = path.resolve(p); + if (seen.has(resolved)) continue; + seen.add(resolved); + out.push(p); + } + return out; +} + +export interface V1Paths { + root: string; + db: string; + env: string; + groups: string; + packageJson: string; +} + +/** + * Build the expected v1 file layout relative to a root. All paths are returned + * even if they don't exist — callers check existence on the ones they care about. + */ +export function v1PathsFor(root: string): V1Paths { + return { + root, + db: path.join(root, 'store', 'messages.db'), + env: path.join(root, '.env'), + groups: path.join(root, 'groups'), + packageJson: path.join(root, 'package.json'), + }; +} + +/** + * Quick "does this path look like a v1 install?" check — used by detect. + * + * Strategy: the strongest signal is `store/messages.db`, so that's required. + * The package.json check is a weaker corroboration — forks may rename + * `"name"` or strip it, so we allow: + * - `name` missing or non-string + * - `name` containing "nanoclaw" or "clawdbot" (case-insensitive) + * We reject only if `name` looks like a completely unrelated project, OR + * the version is 2.x (the v2 rewrite itself). + * + * This keeps stock + forked v1 installs detectable while filtering out + * unrelated repos that happen to have a `store/messages.db`. + */ +export function looksLikeV1Install(root: string): { ok: boolean; reason?: string } { + if (!fs.existsSync(root)) return { ok: false, reason: 'root_missing' }; + const { db, packageJson } = v1PathsFor(root); + if (!fs.existsSync(db)) return { ok: false, reason: 'db_missing' }; + + // package.json is optional — a user may have stripped it, or be running + // from a state-only dir (.nanoclaw). The DB shape is checked separately + // by migrate-validate, which is authoritative for "is this schema v1?" + if (!fs.existsSync(packageJson)) return { ok: true }; + + try { + const pkg = JSON.parse(fs.readFileSync(packageJson, 'utf-8')) as { name?: string; version?: string }; + const name = (pkg.name ?? '').toLowerCase(); + if (pkg.version && /^2\./.test(pkg.version)) return { ok: false, reason: 'already_v2' }; + if (name && !name.includes('nanoclaw') && !name.includes('clawdbot')) { + return { ok: false, reason: 'unrelated_project' }; + } + } catch { + // Broken package.json doesn't rule out v1 — DB presence is enough. + return { ok: true }; + } + return { ok: true }; +} + +// ── Handoff state (single source of truth across sub-steps) ──────────── + +/** + * Rich state shared between migration sub-steps. Each step reads the whole + * file, merges its section, and writes it back. Never hand-edit — it's + * consumed by the `/migrate-from-v1` skill too. + * + * All paths stored are ABSOLUTE, so subsequent steps don't need to guess + * about cwd. Relative paths would be a footgun once the skill reads this + * file later from a different cwd. + */ +export interface Handoff { + version: 1; + started_at: string; + v1_path: string | null; + v1_version: string | null; + + /** Overall status once migrate-handoff finalizes the run. */ + overall_status: 'pending' | 'success' | 'partial' | 'failed' | 'skipped'; + + steps: Partial>; + + /** Group folders the user chose to bring over (migrate-db populates). */ + group_selection: { + mode: 'all' | 'wired-only' | 'cancelled' | null; + selected_folders: string[]; + total_v1_groups: number; + wired_v1_groups: number; + }; + + /** Distinct channels inferred from v1 registered_groups. */ + detected_channels: Array<{ + channel_type: string; + source: 'channel_name' | 'jid_prefix'; + group_count: number; + }>; + + /** Per-channel auth copy results (migrate-channel-auth populates). */ + channel_auth: Array<{ + channel_type: string; + env_keys_copied: string[]; + files_copied: string[]; + files_missing: string[]; + notes: string; + }>; + + /** Result of each `setup/install-.sh` invocation. */ + channels_installed: Array<{ + channel_type: string; + status: 'success' | 'failed' | 'skipped' | 'not_supported'; + error?: string; + }>; + + /** Scheduled-task migration results (migrate-tasks populates). */ + tasks: { + v1_active: number; + v1_inactive: number; + migrated: number; + failed: number; + skipped: number; + }; + + /** Things the skill must finish manually. Always safe to append to. */ + followups: string[]; +} + +export type MigrateStep = + | 'migrate-detect' + | 'migrate-validate' + | 'migrate-db' + | 'migrate-groups' + | 'migrate-env' + | 'migrate-channel-auth' + | 'migrate-channels' + | 'migrate-tasks' + | 'migrate-handoff'; + +export interface StepOutcome { + status: 'success' | 'partial' | 'failed' | 'skipped'; + fields: Record; + notes: string[]; + at: string; +} + +function emptyHandoff(): Handoff { + return { + version: 1, + started_at: new Date().toISOString(), + v1_path: null, + v1_version: null, + overall_status: 'pending', + steps: {}, + group_selection: { + mode: null, + selected_folders: [], + total_v1_groups: 0, + wired_v1_groups: 0, + }, + detected_channels: [], + channel_auth: [], + channels_installed: [], + tasks: { v1_active: 0, v1_inactive: 0, migrated: 0, failed: 0, skipped: 0 }, + followups: [], + }; +} + +/** Read the handoff, creating an empty one if it doesn't exist yet. */ +export function readHandoff(): Handoff { + fs.mkdirSync(MIGRATION_DIR, { recursive: true }); + if (!fs.existsSync(HANDOFF_PATH)) return emptyHandoff(); + try { + const parsed = JSON.parse(fs.readFileSync(HANDOFF_PATH, 'utf-8')) as Handoff; + if (parsed.version !== 1) throw new Error(`unsupported handoff version ${parsed.version}`); + return parsed; + } catch { + // Broken handoff shouldn't wedge the migration — start fresh and let the + // step that called us re-record its outcome. + return emptyHandoff(); + } +} + +/** Persist a handoff mutation atomically (write-tmp + rename). */ +export function writeHandoff(h: Handoff): void { + fs.mkdirSync(MIGRATION_DIR, { recursive: true }); + const tmp = HANDOFF_PATH + '.tmp'; + fs.writeFileSync(tmp, JSON.stringify(h, null, 2)); + fs.renameSync(tmp, HANDOFF_PATH); +} + +/** Convenience: merge a step outcome into the handoff and persist. */ +export function recordStep(step: MigrateStep, outcome: StepOutcome): void { + const h = readHandoff(); + h.steps[step] = outcome; + writeHandoff(h); +} + +// ── JID parsing + channel inference ──────────────────────────────────── + +/** + * v1 stored chat identifiers as `:` in `registered_groups.jid`. + * The prefix was often a short code (`dc` for Discord, `tg` for Telegram) + * that doesn't match v2's `channel_type` names. This table normalizes them. + * + * Unknown prefixes fall through as-is (`channel_type = prefix`) so a channel + * we didn't anticipate still ends up with a distinct messaging_group per + * chat — the skill can reconcile it interactively. + */ +export const JID_PREFIX_TO_CHANNEL: Record = { + dc: 'discord', + discord: 'discord', + tg: 'telegram', + telegram: 'telegram', + wa: 'whatsapp', + whatsapp: 'whatsapp', + slack: 'slack', + matrix: 'matrix', + mx: 'matrix', + teams: 'teams', + imessage: 'imessage', + im: 'imessage', + email: 'email', + webex: 'webex', + gchat: 'gchat', + linear: 'linear', + github: 'github', +}; + +export interface ParsedJid { + raw: string; + prefix: string; + id: string; + channel_type: string; +} + +export function parseJid(raw: string): ParsedJid | null { + const colon = raw.indexOf(':'); + if (colon === -1) return null; + const prefix = raw.slice(0, colon).toLowerCase(); + const id = raw.slice(colon + 1); + if (!prefix || !id) return null; + return { + raw, + prefix, + id, + channel_type: JID_PREFIX_TO_CHANNEL[prefix] ?? prefix, + }; +} + +/** + * Prefer an explicit v1 `channel_name` when one is set; fall back to the JID + * prefix. v1 left `channel_name` empty on most rows (it was a late addition), + * so the JID prefix is often the only honest source. + */ +export function inferChannelType(jid: string, channelName: string | null): string | null { + if (channelName && channelName.trim()) return channelName.trim(); + const parsed = parseJid(jid); + return parsed?.channel_type ?? null; +} + +/** + * v2's messaging_groups.platform_id is always prefixed with the channel_type + * (see setup/register.ts:118-120). This helper normalizes v1's `jid` into + * that shape so router lookups at runtime find the right row. + */ +export function v2PlatformId(channelType: string, jid: string): string { + const parsed = parseJid(jid); + const id = parsed?.id ?? jid; + return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; +} + +// ── Trigger rules → engage mode (ports migration 010's backfill) ─────── + +/** + * Mirrors the backfill() logic in src/db/migrations/010-engage-modes.ts so + * rows written by the migration land in the same shape as rows written by + * setup/register.ts (which goes through migration 010 at boot). + */ +export function triggerToEngage(input: { + trigger_pattern: string | null; + requires_trigger: number | null; +}): { + engage_mode: 'pattern' | 'mention' | 'mention-sticky'; + engage_pattern: string | null; +} { + const pattern = input.trigger_pattern && input.trigger_pattern.trim().length > 0 ? input.trigger_pattern : null; + const requiresTrigger = input.requires_trigger !== 0; // NULL/1 → true; 0 → false + + if (pattern === '.' || pattern === '.*') { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + if (pattern) { + return { engage_mode: 'pattern', engage_pattern: pattern }; + } + if (!requiresTrigger) { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + return { engage_mode: 'mention', engage_pattern: null }; +} + +// ── Channel auth registry (non-.env state per channel) ───────────────── + +/** + * Describes the auth surface for a channel beyond `.env`. Each entry tells + * the channel-auth step: + * + * - `v1EnvKeys`: env keys we might find on the v1 side and carry over + * - `requiredV2Keys`: env keys v2's adapter REQUIRES to boot — if missing + * from v2's merged .env after migrate-env runs, a followup is emitted so + * the user knows exactly what to add (and where to get it). + * - `candidatePaths`: relative paths under the v1 root that may hold + * on-disk auth state (WhatsApp keystore, matrix sync state, etc.) + * - `note`: short human-readable hint surfaced to the user + * + * Unknown channels fall through as {v1EnvKeys:[], requiredV2Keys:[], + * candidatePaths:[]} — the skill asks the user how to proceed. + * + * Keep `requiredV2Keys` honest: list only what the v2 adapter actually + * refuses to boot without. False positives spam the followups; false + * negatives let the agent silently fail. Verify against the actual + * `@chat-adapter/` package when adding/updating entries. + */ +export interface ChannelAuthSpec { + v1EnvKeys: string[]; + requiredV2Keys: { key: string; where: string }[]; + candidatePaths: string[]; + note?: string; +} + +export const CHANNEL_AUTH_REGISTRY: Record = { + discord: { + v1EnvKeys: ['DISCORD_BOT_TOKEN', 'DISCORD_CLIENT_ID', 'DISCORD_GUILD_ID'], + // v1 used raw discord.js (bot token only). v2 uses Chat SDK which needs + // the interaction-verification public key + application id on top. + requiredV2Keys: [ + { key: 'DISCORD_BOT_TOKEN', where: 'Discord Developer Portal → Application → Bot → Token' }, + { key: 'DISCORD_APPLICATION_ID', where: 'Discord Developer Portal → Application → General → Application ID' }, + { key: 'DISCORD_PUBLIC_KEY', where: 'Discord Developer Portal → Application → General → Public Key' }, + ], + candidatePaths: [], + note: 'v1 used raw discord.js (bot token only). v2 uses Chat SDK and needs APPLICATION_ID + PUBLIC_KEY too.', + }, + 'discord-supervisor': { + v1EnvKeys: ['DISCORD_SUPERVISOR_BOT_TOKEN'], + requiredV2Keys: [], + candidatePaths: [], + note: 'v1-specific secondary bot. v2 does not have a native supervisor channel; the token is preserved in .env for the skill to reconcile.', + }, + telegram: { + v1EnvKeys: ['TELEGRAM_BOT_TOKEN', 'TELEGRAM_API_ID', 'TELEGRAM_API_HASH'], + requiredV2Keys: [ + { key: 'TELEGRAM_BOT_TOKEN', where: 'BotFather on Telegram → /mybots → Bot → API Token' }, + ], + candidatePaths: ['data/sessions/telegram', 'store/telegram-session'], + }, + whatsapp: { + v1EnvKeys: ['WHATSAPP_PHONE', 'WHATSAPP_OWNER'], + requiredV2Keys: [], + candidatePaths: [ + 'data/sessions/baileys', + 'data/baileys_auth', + 'store/auth_info_baileys', + 'store/baileys', + 'auth_info_baileys', + ], + note: 'Baileys keystore — copying is best-effort. Encryption sessions may still need a fresh pair via /add-whatsapp.', + }, + matrix: { + v1EnvKeys: ['MATRIX_HOMESERVER', 'MATRIX_USER_ID', 'MATRIX_ACCESS_TOKEN'], + requiredV2Keys: [ + { key: 'MATRIX_HOMESERVER', where: 'your Matrix homeserver URL (e.g. https://matrix.org)' }, + { key: 'MATRIX_ACCESS_TOKEN', where: 'Element → Settings → Help & About → Access Token (keep secret)' }, + ], + candidatePaths: ['data/matrix-store', 'store/matrix', 'data/sessions/matrix'], + }, + slack: { + v1EnvKeys: ['SLACK_BOT_TOKEN', 'SLACK_APP_TOKEN', 'SLACK_SIGNING_SECRET'], + requiredV2Keys: [ + { key: 'SLACK_BOT_TOKEN', where: 'Slack app → OAuth & Permissions → Bot User OAuth Token (xoxb-…)' }, + { key: 'SLACK_SIGNING_SECRET', where: 'Slack app → Basic Information → Signing Secret' }, + ], + candidatePaths: [], + }, + teams: { + v1EnvKeys: ['TEAMS_APP_ID', 'TEAMS_APP_PASSWORD', 'TEAMS_TENANT_ID'], + requiredV2Keys: [ + { key: 'TEAMS_APP_ID', where: 'Azure portal → App registration → Application (client) ID' }, + { key: 'TEAMS_APP_PASSWORD', where: 'Azure portal → App registration → Certificates & secrets' }, + ], + candidatePaths: [], + }, + imessage: { + v1EnvKeys: ['IMESSAGE_PHOTON_URL', 'IMESSAGE_PHOTON_TOKEN'], + requiredV2Keys: [], + candidatePaths: ['data/imessage', 'store/imessage'], + }, + webex: { + v1EnvKeys: ['WEBEX_BOT_TOKEN'], + requiredV2Keys: [{ key: 'WEBEX_BOT_TOKEN', where: 'Webex developer portal → Bot → Bot Access Token' }], + candidatePaths: [], + }, + gchat: { + v1EnvKeys: ['GCHAT_SERVICE_ACCOUNT', 'GCHAT_WEBHOOK_URL'], + requiredV2Keys: [], + candidatePaths: ['data/gchat-credentials.json', 'store/gchat-sa.json'], + }, + resend: { + v1EnvKeys: ['RESEND_API_KEY', 'RESEND_FROM'], + requiredV2Keys: [{ key: 'RESEND_API_KEY', where: 'resend.com → API Keys' }], + candidatePaths: [], + }, + github: { + v1EnvKeys: ['GITHUB_WEBHOOK_SECRET', 'GITHUB_APP_ID', 'GITHUB_PRIVATE_KEY_PATH'], + requiredV2Keys: [], + candidatePaths: [], + note: 'Webhook channel — secrets carry over, but GitHub webhook URLs are new per v2 install.', + }, + linear: { + v1EnvKeys: ['LINEAR_API_KEY', 'LINEAR_WEBHOOK_SECRET'], + requiredV2Keys: [{ key: 'LINEAR_API_KEY', where: 'Linear → Settings → API → Personal API keys' }], + candidatePaths: [], + }, +}; + +/** + * Map a v2 `channel_type` name to the corresponding `setup/install-.sh` + * script, if one exists. `null` means no v2 skill is available yet — the + * handoff lists the channel as "not supported" and the skill raises it with + * the user. + */ +export function installScriptForChannel(channelType: string): string | null { + const known = new Set([ + 'discord', + 'telegram', + 'whatsapp', + 'whatsapp-cloud', + 'teams', + 'slack', + 'matrix', + 'imessage', + 'webex', + 'gchat', + 'resend', + 'github', + 'linear', + ]); + if (!known.has(channelType)) return null; + return `setup/install-${channelType}.sh`; +} + +// ── Misc helpers ─────────────────────────────────────────────────────── + +export function generateId(prefix: string): string { + return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; +} + +// ── v1-specific pattern scan (for migrate-groups) ────────────────────── + +/** + * Tight set of v1-only infrastructure patterns. When one of these shows up + * in a copied CLAUDE.md, the content referencing v1 plumbing that is genuinely + * gone in v2 (IPC file queue, single-DB paths, v1 pr-factory conventions). + * + * Deliberately excludes portable patterns — `mcp__nanoclaw__*` tool names, + * `agent-browser`, generic `/workspace/` paths — which v2 supports the same + * way. The list is scan-only; the migration does NOT modify file content. It + * just adds a followup so the /migrate-from-v1 skill can triage each file + * with the user. + * + * Keep this list conservative: false positives spam the skill with noise, + * false negatives leave the user with silently-broken agents. When adding, + * include a comment naming the specific v1 thing each pattern points at. + */ +export interface V1PatternMatch { + pattern: string; + description: string; + lines: number[]; +} + +const V1_PATTERNS: Array<{ pattern: RegExp; description: string }> = [ + { + pattern: /\/workspace\/ipc\/tasks/, + description: "v1 IPC file queue (gone in v2 — agents talk to the host via session DBs, not JSON files)", + }, + { + pattern: /\/workspace\/extra\/project\/store\b/, + description: "v1-specific mount + store/ path (v2 mounts differ; state lives under data/)", + }, + { + pattern: /\bstore\/messages\.db\b/, + description: "v1 central DB path (v2 uses data/v2.db + data/v2-sessions//{inbound,outbound}.db)", + }, + { + pattern: /"clear_session"|"retrigger"/, + description: "v1 IPC task types (no v2 equivalent; use session lifecycle + the scheduling MCP tool instead)", + }, + { + pattern: /\[PR_CONTEXT:/, + description: "v1 pr-factory context-tag convention (specific to the supervisor group; needs reworking in v2)", + }, + { + pattern: /\brequires_trigger\b|\btrigger_pattern\b/, + description: "v1 column names on registered_groups (v2 uses engage_mode + engage_pattern on messaging_group_agents)", + }, + { + pattern: /\bchatJid\b(?!\s*[:=]\s*["']dc:)/, + description: "v1 routing key (v2 uses messaging_group_id or channel_type+platform_id)", + }, +]; + +/** Scan a CLAUDE.md-ish text blob for v1-specific infrastructure patterns. */ +export function scanForV1Patterns(text: string): V1PatternMatch[] { + const matches: V1PatternMatch[] = []; + const lines = text.split('\n'); + + for (const entry of V1_PATTERNS) { + const hitLines: number[] = []; + for (let i = 0; i < lines.length; i++) { + if (entry.pattern.test(lines[i])) { + hitLines.push(i + 1); + } + } + if (hitLines.length > 0) { + matches.push({ + pattern: entry.pattern.source, + description: entry.description, + // Cap to first 5 line numbers — we're generating a followup summary, + // not a code index. Full context is in the file itself. + lines: hitLines.slice(0, 5), + }); + } + } + + return matches; +} + +export function safeJsonStringify(value: unknown): string { + try { + return JSON.stringify(value, null, 2); + } catch { + return '{"error":"unserializable"}'; + } +} diff --git a/setup/migrate-v1/tasks.ts b/setup/migrate-v1/tasks.ts new file mode 100644 index 0000000..836be7c --- /dev/null +++ b/setup/migrate-v1/tasks.ts @@ -0,0 +1,307 @@ +/** + * Step: migrate-tasks + * + * Port v1's `scheduled_tasks` into v2's session inbound DBs. v1 had a + * dedicated table with its own scheduling grammar; v2 treats tasks as + * `messages_in` rows with `kind='task'`, `process_after`, and `recurrence` + * (cron string). See docs/v1-to-v2-changes.md "Scheduling". + * + * Flow per v1 row: + * 1. Resolve (agent_group_id, messaging_group_id) from v1 (group_folder, chat_jid) + * 2. resolveSession() — creates the session on demand if absent + * 3. insertTask() into the session's inbound.db + * + * Active v1 rows (status='active') are migrated. Completed/stopped rows get + * exported to logs/setup-migration/inactive-tasks.json for reference. + * + * v1's schedule_type / schedule_value are mapped to cron here. Known types: + * 'cron' → schedule_value is already a cron string + * 'interval' → e.g. '5m'/'1h' → cron equivalent (best effort) + * 'once' → no recurrence, process_after = schedule_value if parseable + * Unknown types go to inactive-tasks.json with a note. + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb, closeDb } from '../../src/db/connection.js'; +import { getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { log } from '../../src/log.js'; +import { insertTask } from '../../src/modules/scheduling/db.js'; +import { openInboundDb, resolveSession } from '../../src/session-manager.js'; +import { emitStatus } from '../status.js'; +import { + INACTIVE_TASKS_PATH, + MIGRATION_DIR, + inferChannelType, + readHandoff, + recordStep, + safeJsonStringify, + v1PathsFor, + v2PlatformId, + writeHandoff, +} from './shared.js'; + +interface V1Task { + id: string; + group_folder: string; + chat_jid: string; + prompt: string; + schedule_type: string; + schedule_value: string; + next_run: string | null; + last_run: string | null; + status: string; + context_mode: string | null; + script: string | null; +} + +/** Convert v1 schedule_type + schedule_value into (processAfter, recurrence). */ +function toProcessAfterAndRecurrence(t: V1Task): { + processAfter: string; + recurrence: string | null; + note?: string; +} | null { + const now = new Date().toISOString(); + + if (t.schedule_type === 'cron') { + // Validate shape — 5 or 6 fields separated by whitespace. cron-parser is + // the runtime source of truth; here we just reject obvious garbage so + // we don't insert tasks that will explode on the first sweep tick. + const fields = t.schedule_value.trim().split(/\s+/).length; + if (fields < 5 || fields > 6) return null; + return { + processAfter: t.next_run || now, + recurrence: t.schedule_value.trim(), + }; + } + + if (t.schedule_type === 'interval') { + // '5m' → '*/5 * * * *'; '1h' → '0 * * * *'; '1d' → '0 0 * * *'. + // Best effort — any unit we don't recognize falls through to null. + const m = /^(\d+)([smhd])$/.exec(t.schedule_value.trim()); + if (!m) return null; + const n = parseInt(m[1], 10); + const unit = m[2]; + if (!n || n < 1) return null; + let cron: string | null = null; + if (unit === 'm' && n < 60) cron = `*/${n} * * * *`; + else if (unit === 'h' && n < 24) cron = `0 */${n} * * *`; + else if (unit === 'd' && n < 28) cron = `0 0 */${n} * *`; + if (!cron) return null; + return { processAfter: t.next_run || now, recurrence: cron }; + } + + if (t.schedule_type === 'once' || t.schedule_type === 'at') { + return { + processAfter: t.next_run || t.schedule_value || now, + recurrence: null, + }; + } + + return null; +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-tasks', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const validate = h.steps['migrate-validate']; + if (validate && validate.status === 'failed') { + recordStep('migrate-tasks', { + status: 'skipped', + fields: { REASON: 'validate-failed' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'validate_failed' }); + return; + } + + const paths = v1PathsFor(h.v1_path); + + // Read v1 tasks into memory so we can close the v1 DB before we open v2's + // central DB via initDb() (which is a module singleton and doesn't love + // having two files open through it). + let activeTasks: V1Task[] = []; + let inactiveTasks: V1Task[] = []; + try { + const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); + const all = v1Db.prepare('SELECT * FROM scheduled_tasks').all() as V1Task[]; + v1Db.close(); + activeTasks = all.filter((t) => t.status === 'active'); + inactiveTasks = all.filter((t) => t.status !== 'active'); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + recordStep('migrate-tasks', { + status: 'failed', + fields: { REASON: 'v1-read-failed' }, + notes: [message], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_TASKS', { STATUS: 'failed', REASON: 'v1_read_failed', ERROR: message }); + return; + } + + if (activeTasks.length === 0 && inactiveTasks.length === 0) { + recordStep('migrate-tasks', { + status: 'skipped', + fields: { REASON: 'no-v1-tasks' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'no_v1_tasks' }); + return; + } + + // Dump inactive tasks for reference — always, even if there are no active ones. + if (inactiveTasks.length > 0) { + fs.mkdirSync(MIGRATION_DIR, { recursive: true }); + fs.writeFileSync(INACTIVE_TASKS_PATH, safeJsonStringify({ tasks: inactiveTasks })); + } + + // Connect to v2 central DB to resolve (folder → ag) and (channel+pid → mg). + const v2Path = path.join(DATA_DIR, 'v2.db'); + fs.mkdirSync(path.dirname(v2Path), { recursive: true }); + const v2Db = initDb(v2Path); + runMigrations(v2Db); + + const followups: string[] = []; + let migrated = 0; + let failed = 0; + let skipped = 0; + + for (const t of activeTasks) { + try { + const ag = getAgentGroupByFolder(t.group_folder); + if (!ag) { + skipped += 1; + followups.push( + `Task "${t.id}" (folder "${t.group_folder}"): agent_group not seeded in v2 — run migrate-db first or deselect the task.`, + ); + continue; + } + + const channelType = inferChannelType(t.chat_jid, null); + if (!channelType) { + skipped += 1; + followups.push(`Task "${t.id}": could not infer channel from chat_jid "${t.chat_jid}".`); + continue; + } + const platformId = v2PlatformId(channelType, t.chat_jid); + const mg = getMessagingGroupByPlatform(channelType, platformId); + if (!mg) { + skipped += 1; + followups.push( + `Task "${t.id}": messaging_group for (${channelType}, ${platformId}) not seeded. Add the channel then re-run this step.`, + ); + continue; + } + + const scheduling = toProcessAfterAndRecurrence(t); + if (!scheduling) { + skipped += 1; + followups.push( + `Task "${t.id}": schedule_type "${t.schedule_type}" / value "${t.schedule_value}" did not map to a v2 cron — exported to inactive-tasks.json for manual review.`, + ); + inactiveTasks.push(t); + continue; + } + + // resolveSession creates (ag, mg) session if not present; 'shared' mode + // matches v1 which had one session per group_folder. + const { session } = resolveSession(ag.id, mg.id, null, 'shared'); + const inboxDb = openInboundDb(ag.id, session.id); + try { + // Idempotence: skip if we've already migrated this task id. We use the + // v1 task id verbatim as the v2 messages_in.id (stable — lets users + // re-run migration without duplicate-key errors or shadow tasks). + const existing = inboxDb + .prepare("SELECT id FROM messages_in WHERE id = ? AND kind = 'task'") + .get(t.id) as { id: string } | undefined; + if (existing) { + skipped += 1; + continue; + } + + insertTask(inboxDb, { + id: t.id, + processAfter: scheduling.processAfter, + recurrence: scheduling.recurrence, + platformId, + channelType, + threadId: null, + content: JSON.stringify({ + prompt: t.prompt, + script: t.script ?? null, + migrated_from_v1: { original_id: t.id, context_mode: t.context_mode ?? null }, + }), + }); + } finally { + inboxDb.close(); + } + + log.info('Migrated v1 scheduled task', { taskId: t.id, session: session.id, mg: mg.id }); + migrated += 1; + } catch (err) { + failed += 1; + const message = err instanceof Error ? err.message : String(err); + followups.push(`Task "${t.id}" failed to migrate: ${message}`); + } + } + + // Re-dump inactive tasks in case scheduling-translation pushed any in. + if (inactiveTasks.length > 0) { + fs.writeFileSync(INACTIVE_TASKS_PATH, safeJsonStringify({ tasks: inactiveTasks })); + } + + closeDb(); + + const handoffAfter = readHandoff(); + handoffAfter.tasks = { + v1_active: activeTasks.length, + v1_inactive: inactiveTasks.length, + migrated, + failed, + skipped, + }; + handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; + writeHandoff(handoffAfter); + + const partial = failed > 0 || skipped > 0; + recordStep('migrate-tasks', { + status: failed > 0 ? 'partial' : partial ? 'partial' : 'success', + fields: { + V1_ACTIVE: activeTasks.length, + V1_INACTIVE: inactiveTasks.length, + MIGRATED: migrated, + FAILED: failed, + SKIPPED: skipped, + INACTIVE_EXPORT: inactiveTasks.length > 0 ? INACTIVE_TASKS_PATH : '', + }, + notes: followups, + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_TASKS', { + STATUS: partial ? 'partial' : 'success', + V1_ACTIVE: String(activeTasks.length), + V1_INACTIVE: String(inactiveTasks.length), + MIGRATED: String(migrated), + FAILED: String(failed), + SKIPPED: String(skipped), + }); +} diff --git a/setup/migrate-v1/validate.ts b/setup/migrate-v1/validate.ts new file mode 100644 index 0000000..73cd377 --- /dev/null +++ b/setup/migrate-v1/validate.ts @@ -0,0 +1,213 @@ +/** + * Step: migrate-validate + * + * Before touching v1 data, assert the DB has the shape we expect. We know + * v1's schema (see docs/v1-to-v2-changes.md "Entity model") — different + * shapes happened over v1's development, but by v1.2.x the `registered_groups` + * columns and `scheduled_tasks` columns stabilized. If we see something else, + * we bail early so later steps don't write garbage to v2.db. + * + * Output: + * - `logs/setup-migration/schema-mismatch.json` on failure (read by the skill) + * - Status block MIGRATE_VALIDATE with OK/FAILED + * - Even on failure, subsequent steps still run — they'll short-circuit + * on their own if validate marked the DB unusable. This keeps env + group + * folder migration working when only the DB is broken. + */ +import fs from 'fs'; + +import Database from 'better-sqlite3'; + +import { emitStatus } from '../status.js'; +import { + SCHEMA_MISMATCH_PATH, + readHandoff, + recordStep, + safeJsonStringify, + v1PathsFor, +} from './shared.js'; + +const EXPECTED_TABLES = [ + 'registered_groups', + 'scheduled_tasks', + 'chats', + 'messages', + 'sessions', + 'router_state', +]; + +const REQUIRED_REGISTERED_GROUPS_COLUMNS = [ + 'jid', + 'name', + 'folder', + 'trigger_pattern', + 'added_at', + 'requires_trigger', +]; + +const REQUIRED_SCHEDULED_TASKS_COLUMNS = [ + 'id', + 'group_folder', + 'chat_jid', + 'prompt', + 'schedule_type', + 'schedule_value', + 'status', +]; + +interface TableInfo { + table: string; + columns: string[]; + missing_columns: string[]; +} + +export async function run(_args: string[]): Promise { + const h = readHandoff(); + if (!h.v1_path) { + recordStep('migrate-validate', { + status: 'skipped', + fields: { REASON: 'detect-not-run' }, + notes: [], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_VALIDATE', { STATUS: 'skipped', REASON: 'no_v1_path' }); + return; + } + + const paths = v1PathsFor(h.v1_path); + if (!fs.existsSync(paths.db)) { + recordStep('migrate-validate', { + status: 'failed', + fields: { REASON: 'db-missing', DB_PATH: paths.db }, + notes: ['v1 DB file does not exist at expected path'], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_VALIDATE', { + STATUS: 'failed', + REASON: 'db_missing', + DB_PATH: paths.db, + }); + return; + } + + let db: Database.Database; + try { + db = new Database(paths.db, { readonly: true, fileMustExist: true }); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + recordStep('migrate-validate', { + status: 'failed', + fields: { REASON: 'db-open-failed' }, + notes: [message], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_VALIDATE', { + STATUS: 'failed', + REASON: 'db_open_failed', + ERROR: message, + }); + return; + } + + try { + const tableRows = db + .prepare("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name") + .all() as Array<{ name: string }>; + const tables = new Set(tableRows.map((r) => r.name)); + + const missingTables = EXPECTED_TABLES.filter((t) => !tables.has(t)); + const tableInfos: TableInfo[] = []; + + for (const t of EXPECTED_TABLES) { + if (!tables.has(t)) continue; + const cols = db.prepare(`PRAGMA table_info(${t})`).all() as Array<{ name: string }>; + const columnNames = cols.map((c) => c.name); + const missing = + t === 'registered_groups' + ? REQUIRED_REGISTERED_GROUPS_COLUMNS.filter((c) => !columnNames.includes(c)) + : t === 'scheduled_tasks' + ? REQUIRED_SCHEDULED_TASKS_COLUMNS.filter((c) => !columnNames.includes(c)) + : []; + tableInfos.push({ table: t, columns: columnNames, missing_columns: missing }); + } + + const columnMismatches = tableInfos.filter((t) => t.missing_columns.length > 0); + const groupCount = + tables.has('registered_groups') + ? ((db.prepare('SELECT COUNT(*) AS c FROM registered_groups').get() as { c: number }).c) + : 0; + const taskCount = + tables.has('scheduled_tasks') + ? ((db.prepare('SELECT COUNT(*) AS c FROM scheduled_tasks').get() as { c: number }).c) + : 0; + + db.close(); + + if (missingTables.length > 0 || columnMismatches.length > 0) { + const mismatch = { + v1_path: h.v1_path, + v1_version: h.v1_version, + present_tables: [...tables].sort(), + missing_tables: missingTables, + column_mismatches: columnMismatches, + scanned_at: new Date().toISOString(), + }; + fs.writeFileSync(SCHEMA_MISMATCH_PATH, safeJsonStringify(mismatch)); + + recordStep('migrate-validate', { + status: 'failed', + fields: { + MISSING_TABLES: missingTables.join(',') || 'none', + COLUMN_MISMATCHES: String(columnMismatches.length), + REPORT: SCHEMA_MISMATCH_PATH, + }, + notes: [ + missingTables.length > 0 ? `Missing tables: ${missingTables.join(', ')}` : '', + columnMismatches.length > 0 + ? `Column mismatches in: ${columnMismatches.map((c) => c.table).join(', ')}` + : '', + ].filter(Boolean), + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_VALIDATE', { + STATUS: 'failed', + REASON: 'schema_mismatch', + MISSING_TABLES: missingTables.join(',') || 'none', + COLUMN_MISMATCHES: String(columnMismatches.length), + REPORT: SCHEMA_MISMATCH_PATH, + }); + return; + } + + recordStep('migrate-validate', { + status: 'success', + fields: { + V1_GROUPS: groupCount, + V1_TASKS: taskCount, + }, + notes: [], + at: new Date().toISOString(), + }); + + emitStatus('MIGRATE_VALIDATE', { + STATUS: 'success', + V1_GROUPS: String(groupCount), + V1_TASKS: String(taskCount), + }); + } catch (err) { + db.close(); + const message = err instanceof Error ? err.message : String(err); + recordStep('migrate-validate', { + status: 'failed', + fields: { REASON: 'validate-error' }, + notes: [message], + at: new Date().toISOString(), + }); + emitStatus('MIGRATE_VALIDATE', { + STATUS: 'failed', + REASON: 'validate_error', + ERROR: message, + }); + } +} From e1c8876a728493a7c940ff63d7c0321340f60a90 Mon Sep 17 00:00:00 2001 From: gabi-simons Date: Thu, 23 Apr 2026 12:30:27 +0000 Subject: [PATCH 02/23] feat(migrate-v1): auto-resolve missing v2 channel keys via adapter APIs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `migrate-channel-auth` now tries to derive v2-required keys that v1 never stored by calling the channel's API with the credential v1 did have. When the gap can be closed automatically, the keys land in v2 `.env` before the missing-required check, and the step reports `success` instead of `partial`. When it can't, the existing followup fires unchanged. ## Discord v1 used raw `discord.js` (bot token only). v2's Chat SDK needs `DISCORD_APPLICATION_ID` + `DISCORD_PUBLIC_KEY`. Both can be fetched with the bot token via: GET /oauth2/applications/@me Authorization: Bot → { id, verify_key, … } For a stock v1 Discord user, this means `bash nanoclaw.sh` now produces a fully working v2 Discord adapter with zero manual key-setting — just stop v1, and v2 takes over. ## Surface - `autoResolveV2Keys(channelType, lookup)` in `setup/migrate-v1/shared.ts` — pluggable per-channel resolver, returns a `{key: value}` map. Never throws; returns `{}` on any failure (network, auth, unexpected shape). Logs keys resolved, never values. - `migrate-channel-auth` wiring: build a lookup over v1 + v2 .env, call the resolver, append resolved keys to v2 .env (never overwriting), sync to `data/env/env`, then re-check `requiredV2Keys` to compute the real gap. Sidecar annotation `(auto-resolved)` on `env_keys_copied` in the handoff so the skill can tell which came from v1 vs derived. ## Extending to other channels Slack has `/auth.test` (bot token → team/app info), Telegram has `/getMe`, Matrix has `/whoami`. Most don't cover the full required-key set v2 needs (e.g. Slack's `SLACK_SIGNING_SECRET` lives only in app config and has no API equivalent). Add resolvers case-by-case when the API supports it; the registry's `requiredV2Keys` + followup fallback covers the rest. ## Testing - Stripped `DISCORD_APPLICATION_ID` + `DISCORD_PUBLIC_KEY` from v2 `.env` - Re-ran migration (wired-only, 301 groups): resolver populated both keys via the API; `migrate-channel-auth: success` (was `partial`); `overall_status: success` - Restarted v2: Discord adapter booted clean, Gateway connected, `GUILD_CREATE` received - v1 stopped, v2 handling Discord traffic --- setup/migrate-v1/channel-auth.ts | 53 ++++++++++++++++++++++++++++++-- setup/migrate-v1/shared.ts | 44 ++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 2 deletions(-) diff --git a/setup/migrate-v1/channel-auth.ts b/setup/migrate-v1/channel-auth.ts index 1ef2bc4..8c8a415 100644 --- a/setup/migrate-v1/channel-auth.ts +++ b/setup/migrate-v1/channel-auth.ts @@ -17,6 +17,7 @@ import path from 'path'; import { emitStatus } from '../status.js'; import { CHANNEL_AUTH_REGISTRY, + autoResolveV2Keys, readHandoff, recordStep, v1PathsFor, @@ -126,8 +127,56 @@ export async function run(_args: string[]): Promise { // Check v2's .env for required keys the v2 adapter needs to boot. v1 // may not have had all of them (e.g. v1's Discord used discord.js // directly and never stored DISCORD_PUBLIC_KEY which v2's Chat SDK - // requires). Surface missing ones as actionable followups. + // requires). Try to auto-resolve the gap by calling the channel's API + // with the v1 credential; fall through to a followup for anything we + // can't resolve. const v2EnvPath = path.join(process.cwd(), '.env'); + const v1EnvMap = new Map(); + for (const line of v1Env.split('\n')) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) continue; + const eq = trimmed.indexOf('='); + if (eq <= 0) continue; + v1EnvMap.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); + } + + // Also let the resolver reach into v2's .env (migrate-env already merged + // v1 keys into v2). Either source is fine for derivation inputs. + const v2EnvPre = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; + const v2EnvPreMap = new Map(); + for (const line of v2EnvPre.split('\n')) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) continue; + const eq = trimmed.indexOf('='); + if (eq <= 0) continue; + v2EnvPreMap.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); + } + + const resolved = await autoResolveV2Keys( + ch.channel_type, + (key) => v1EnvMap.get(key) ?? v2EnvPreMap.get(key), + ); + const resolvedKeys = Object.keys(resolved); + if (resolvedKeys.length > 0) { + // Append to v2 .env (never overwriting existing values) + sync the + // container-side copy. Log keys, never values. + let text = v2EnvPre; + if (text && !text.endsWith('\n')) text += '\n'; + for (const [key, value] of Object.entries(resolved)) { + if (v2EnvPreMap.has(key)) continue; + text += `${key}=${value}\n`; + } + fs.writeFileSync(v2EnvPath, text); + try { + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { + // Best-effort; service restart rehydrates it if needed. + } + } + + // Re-read v2 .env after possible resolution to compute the real gap. const v2Env = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; const v2EnvKeys = new Set( v2Env @@ -179,7 +228,7 @@ export async function run(_args: string[]): Promise { results.push({ channel_type: ch.channel_type, - env_keys_copied: envKeysPresentInV1, + env_keys_copied: [...envKeysPresentInV1, ...resolvedKeys.map((k) => `${k} (auto-resolved)`)], files_copied: filesCopied, files_missing: filesMissing, notes: spec.note ?? '', diff --git a/setup/migrate-v1/shared.ts b/setup/migrate-v1/shared.ts index f9f03bc..bc5d3dd 100644 --- a/setup/migrate-v1/shared.ts +++ b/setup/migrate-v1/shared.ts @@ -518,6 +518,50 @@ export const CHANNEL_AUTH_REGISTRY: Record = { }, }; +/** + * For channels where v2's adapter needs keys v1 never stored (e.g. Discord's + * Chat SDK wants DISCORD_APPLICATION_ID + DISCORD_PUBLIC_KEY, but v1 used + * raw discord.js with just the bot token), try to derive the missing keys + * from the v1 creds we already have by calling the channel's API. + * + * Returns a map of key → value for what we successfully resolved. + * Never throws; returns `{}` on any failure (network, auth, unexpected + * shape). The caller writes the resolved keys to v2 .env, then re-checks + * `requiredV2Keys` so the step reports `success` instead of `partial` when + * auto-resolution covered the gap. + * + * Adding a new channel resolver: pull the needed values from an endpoint + * that accepts only the v1-side credential (bot token, API key). Don't + * prompt, don't log values. If the endpoint has rate limits, keep this + * best-effort and fail silently. + */ +export async function autoResolveV2Keys( + channelType: string, + v1EnvLookup: (key: string) => string | undefined, +): Promise> { + if (channelType === 'discord') { + const token = v1EnvLookup('DISCORD_BOT_TOKEN'); + if (!token) return {}; + try { + const resp = await fetch('https://discord.com/api/v10/oauth2/applications/@me', { + headers: { Authorization: `Bot ${token}` }, + }); + if (!resp.ok) return {}; + const data = (await resp.json()) as { id?: string; verify_key?: string }; + const out: Record = {}; + if (typeof data.id === 'string' && data.id) out.DISCORD_APPLICATION_ID = data.id; + if (typeof data.verify_key === 'string' && data.verify_key) { + out.DISCORD_PUBLIC_KEY = data.verify_key; + } + return out; + } catch { + return {}; + } + } + + return {}; +} + /** * Map a v2 `channel_type` name to the corresponding `setup/install-.sh` * script, if one exists. `null` means no v2 skill is available yet — the From 9faa8a9a2c6824e28c56d2e05a273354c9503b70 Mon Sep 17 00:00:00 2001 From: gabi-simons Date: Thu, 23 Apr 2026 12:41:33 +0000 Subject: [PATCH 03/23] fix(migrate-v1): splice guild_id into Discord platform_id during seed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v2's Chat SDK Discord adapter emits `platform_id` as `discord::` at runtime, but v1 only stored `dc:` (no guild). Before this fix `migrate-db` wrote `discord:` into `messaging_groups.platform_id`, which didn't match what v2 saw on incoming messages — v2 treated every message as a new channel and fired its channel-registration approval flow instead of routing to the migrated agent_group. Now `migrate-db` fetches the bot's guilds once per channel_type via `GET /users/@me/guilds`. When the bot is in exactly one guild (the common case), the guild id is spliced into every Discord platform_id at seed time — matching v2's runtime format. Multi-guild bots fall back to the v1-format id; v2's channel-registration flow repairs on first message. Cost: one extra Discord API call per migration run (not per channel). No new failure modes — network/auth issues return null, fall through to the existing behavior. ## Surface - `v2PlatformId(channelType, jid, { guildId })` — new optional `extra` parameter. Back-compat with existing callers. - `fetchBotGuilds(channelType, lookup)` — new helper in `shared.ts`, same pattern as `autoResolveV2Keys`. Handles Discord today; extending to other channels is a case-by-case API check. - `migrate-db` pre-loop: builds `v1EnvMap`, fetches guilds per channel type, caches single-guild IDs for the row loop. ## Testing Verified on a 300-channel Discord v1 install: - Fresh run produced `discord::` platform_ids from the start - Incoming messages now route to the migrated agent_group instead of firing the unwire approval flow Rate-limit note: `/users/@me/guilds` is a single call. Per-channel `/guilds//channels` lookups for multi-guild bots would need proper rate-limit handling — deferred. --- setup/migrate-v1/db.ts | 27 +++++++++++++++++++- setup/migrate-v1/shared.ts | 50 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 74 insertions(+), 3 deletions(-) diff --git a/setup/migrate-v1/db.ts b/setup/migrate-v1/db.ts index e455012..d338214 100644 --- a/setup/migrate-v1/db.ts +++ b/setup/migrate-v1/db.ts @@ -36,6 +36,7 @@ import { runMigrations } from '../../src/db/migrations/index.js'; import { log } from '../../src/log.js'; import { emitStatus } from '../status.js'; import { + fetchBotGuilds, generateId, inferChannelType, readHandoff, @@ -158,6 +159,29 @@ export async function run(args: string[]): Promise { })); writeHandoff(h); + // For channels where v2's platform_id includes a component v1 didn't record + // (Discord's guild id), fetch the bot's guilds up-front. If the bot is in + // a single guild we can splice that id into every platform_id; otherwise + // fall back to the v1-format id (v2's channel-registration flow will repair + // on first message). Done ONCE per channel_type, not per-row, so this is + // cheap regardless of group count. + const v1EnvText = fs.existsSync(paths.env) ? fs.readFileSync(paths.env, 'utf-8') : ''; + const v1EnvMap = new Map(); + for (const line of v1EnvText.split('\n')) { + const t = line.trim(); + if (!t || t.startsWith('#')) continue; + const eq = t.indexOf('='); + if (eq <= 0) continue; + v1EnvMap.set(t.slice(0, eq).trim(), t.slice(eq + 1)); + } + const singleGuildByChannel = new Map(); + for (const channelType of detectedChannels.keys()) { + const info = await fetchBotGuilds(channelType, (k) => v1EnvMap.get(k)); + if (info && info.guildIds.length === 1) { + singleGuildByChannel.set(channelType, info.guildIds[0]); + } + } + // Initialize v2.db (creates schema if not present — runMigrations is no-op // when the schema is already current, so this is safe on a live v2 install). fs.mkdirSync(path.join(process.cwd(), 'data'), { recursive: true }); @@ -181,7 +205,8 @@ export async function run(args: string[]): Promise { continue; } - const platformId = v2PlatformId(channelType, g.jid); + const guildId = singleGuildByChannel.get(channelType); + const platformId = v2PlatformId(channelType, g.jid, { guildId }); const createdAt = new Date().toISOString(); try { diff --git a/setup/migrate-v1/shared.ts b/setup/migrate-v1/shared.ts index bc5d3dd..4597fcf 100644 --- a/setup/migrate-v1/shared.ts +++ b/setup/migrate-v1/shared.ts @@ -358,11 +358,57 @@ export function inferChannelType(jid: string, channelName: string | null): strin * v2's messaging_groups.platform_id is always prefixed with the channel_type * (see setup/register.ts:118-120). This helper normalizes v1's `jid` into * that shape so router lookups at runtime find the right row. + * + * Some channels need extra structure on the id itself. Discord's Chat SDK + * emits `discord::` at runtime but v1 only stored + * `dc:` (no guild). Callers that know the guild (e.g. bot with + * a single guild) can pass it via `extra`; otherwise the returned id will + * be the v1-format `discord:` and will be repaired on first + * message via v2's channel-registration approval flow. */ -export function v2PlatformId(channelType: string, jid: string): string { +export function v2PlatformId(channelType: string, jid: string, extra?: { guildId?: string }): string { const parsed = parseJid(jid); const id = parsed?.id ?? jid; - return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; + const prefixed = id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; + // For Discord: splice the guild id in between when we know it and the id + // isn't already in `:` form. + if (channelType === 'discord' && extra?.guildId) { + const body = prefixed.slice(`discord:`.length); + if (!body.includes(':')) return `discord:${extra.guildId}:${body}`; + } + return prefixed; +} + +/** + * Fetch the bot's guild memberships for a channel_type so migrate-db can + * form platform_ids matching what the v2 adapter emits at runtime. Returns + * null on any failure (network, auth, rate limit, unsupported channel_type) + * — callers fall back to the v1-format platform_id, which works but may + * trigger v2's channel-registration flow on first message. + * + * Currently handles Discord. Extending to other channels: the function + * needs a "single-or-multi guild?" shape; for single-guild bots the caller + * can splice the guild id globally, for multi-guild a per-channel lookup + * is needed and the caller should probably bail (rate-limit risk). + */ +export async function fetchBotGuilds( + channelType: string, + v1EnvLookup: (key: string) => string | undefined, +): Promise<{ guildIds: string[] } | null> { + if (channelType !== 'discord') return null; + const token = v1EnvLookup('DISCORD_BOT_TOKEN'); + if (!token) return null; + try { + const resp = await fetch('https://discord.com/api/v10/users/@me/guilds', { + headers: { Authorization: `Bot ${token}` }, + }); + if (!resp.ok) return null; + const data = (await resp.json()) as Array<{ id?: string }>; + const guildIds = data.map((g) => g.id).filter((id): id is string => typeof id === 'string'); + return { guildIds }; + } catch { + return null; + } } // ── Trigger rules → engage mode (ports migration 010's backfill) ─────── From 1d73b2986a891629a7e220ae4668d7d8b1acf5e8 Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:13:38 +0000 Subject: [PATCH 05/23] =?UTF-8?q?feat:=20add=20migrate-v2.sh=20=E2=80=94?= =?UTF-8?q?=20standalone=20v1=20=E2=86=92=20v2=20migration=20script?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New entry point: `bash migrate-v2.sh` from the v2 checkout. Replaces the old setup-embedded migration flow with a standalone 4-phase script + rewritten Claude skill for the interactive parts. Phase 0: Bootstrap (Node/pnpm/deps via setup.sh) + find v1 Phase 1: Core state (env, DB, groups, sessions, tasks) Phase 2: Channels (clack multiselect, auth copy, code install) Phase 3: Infrastructure (OneCLI, auth, Docker, skills, container build) Service switchover: stop v1 → start v2 → test → keep or revert Phase 4: Handoff → exec claude "/migrate-from-v1" The skill handles: owner seeding, access policy, CLAUDE.local.md cleanup, container config validation, fork customization porting. Key fixes found during testing: - triggerToEngage: requires_trigger=0 must override non-empty pattern - unknown_sender_policy defaults to 'public' (strict drops all msgs before owner is seeded) - Service revert must stop v2 (parse unit name from step log, not early tsx one-liner that can fail) - Session continuity: copy JSONL from -workspace-group/ to -workspace-agent/ and write continuation:claude into outbound.db - container_config.additionalMounts written directly to container.json (same shape in v1 and v2) - EXIT trap writes handoff.json; explicit write_handoff before exec Includes migrate-v2-reset.sh for dev iteration and docs/migration-dev.md for testing/debugging reference. Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 243 ++++++--- docs/migration-dev.md | 139 +++++ migrate-v2-reset.sh | 69 +++ migrate-v2.sh | 641 ++++++++++++++++++++++++ setup/migrate-v1/shared.ts | 8 +- setup/migrate-v2/channel-auth.ts | 134 +++++ setup/migrate-v2/db.ts | 162 ++++++ setup/migrate-v2/env.ts | 81 +++ setup/migrate-v2/groups.ts | 120 +++++ setup/migrate-v2/select-channels.ts | 63 +++ setup/migrate-v2/sessions.ts | 181 +++++++ setup/migrate-v2/switchover-prompt.ts | 53 ++ setup/migrate-v2/tasks.ts | 158 ++++++ 13 files changed, 1976 insertions(+), 76 deletions(-) create mode 100644 docs/migration-dev.md create mode 100644 migrate-v2-reset.sh create mode 100644 migrate-v2.sh create mode 100644 setup/migrate-v2/channel-auth.ts create mode 100644 setup/migrate-v2/db.ts create mode 100644 setup/migrate-v2/env.ts create mode 100644 setup/migrate-v2/groups.ts create mode 100644 setup/migrate-v2/select-channels.ts create mode 100644 setup/migrate-v2/sessions.ts create mode 100644 setup/migrate-v2/switchover-prompt.ts create mode 100644 setup/migrate-v2/tasks.ts diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 810ae03..6362fe6 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -1,55 +1,69 @@ --- name: migrate-from-v1 -description: Finish migrating a NanoClaw v1 install into v2. Run this after `bash nanoclaw.sh` has completed its automated migration step. Seeds the owner user, applies v1 access defaults, fixes any migration sub-step that didn't finish, and interviews the user about custom v1 code to port forward. Triggers on "migrate from v1", "finish migration", "v1 migration", or automatically after setup when `logs/setup-migration/handoff.json` exists. +description: Finish migrating a NanoClaw v1 install into v2. Run after `bash migrate-v2.sh` completes. Seeds the owner, cleans up CLAUDE.local.md files, reconciles container configs, and helps port custom v1 code. Triggers on "migrate from v1", "finish migration", "v1 migration". --- -# Migrate from v1 to v2 +# Finish v1 → v2 migration -> ⚠️ **Experimental.** This skill and the setup migration step are early. Remind the user to back up `data/v2.db` + `groups/` before making destructive changes, and prefer small, reversible edits. Not recommended yet for high-stakes production installs. +`bash migrate-v2.sh` already ran the deterministic migration. It handled: -The setup flow's `migration` step (in `setup/migrate-v1.ts`) already ran a best-effort automated pass. Your job is to finish what it couldn't do automatically, then interview the user about any custom code they had in v1 and help port it forward. +- .env keys merged +- v2 DB seeded (agent_groups, messaging_groups, wiring) +- Group folders copied (v1 CLAUDE.md → v2 CLAUDE.local.md) +- Session data copied with conversation continuity +- Scheduled tasks ported +- Channel code installed +- Container skills copied +- Container image built -Read [docs/v1-to-v2-changes.md](../../../docs/v1-to-v2-changes.md) before doing anything — it's the vocabulary for where v1 things moved to in v2. +Your job is the parts that need human judgment: triage any failed steps, seed the owner, clean up CLAUDE.local.md files, reconcile configs, and port any fork customizations. -## What the automation did +Read `logs/setup-migration/handoff.json` first — it has `overall_status`, per-step results in `steps`, and a `followups` list. -The setup flow ran these sub-steps (each as its own progression-log entry): +## Phase 0: Triage failed steps -| Sub-step | What it did | -|----------|-------------| -| `migrate-detect` | Found v1 install on disk (scanned `~/nanoclaw`, `~/.nanoclaw`, `~/Code/nanoclaw`, etc., or `$NANOCLAW_V1_PATH`). | -| `migrate-validate` | Checked v1 DB has expected tables + required columns. | -| `migrate-db` | Seeded `agent_groups` + `messaging_groups` + `messaging_group_agents` from `registered_groups`. Mapped `trigger_pattern`/`requires_trigger` → `engage_mode`/`engage_pattern`. Did NOT seed `users`/`user_roles`. | -| `migrate-groups` | Copied v1 `groups//` to v2. v1 `CLAUDE.md` → v2 `CLAUDE.local.md`. v1 `container_config` JSON → `.v1-container-config.json` sidecar (don't silent-map to v2's `container.json`). | -| `migrate-env` | Merged v1 `.env` keys into v2 `.env` (never overwrote existing keys). | -| `migrate-channel-auth` | Copied non-env auth state per channel (Baileys keystore, matrix state, etc.) based on `CHANNEL_AUTH_REGISTRY` in `setup/migrate-v1/shared.ts`. | -| `migrate-channels` | Ran `setup/install-.sh` for each channel detected in `registered_groups`. | -| `migrate-tasks` | Ported active v1 `scheduled_tasks` into each session's `inbound.db` as `kind='task'` rows. Inactive tasks dumped to `inactive-tasks.json` for reference. | +Check `handoff.json` → `overall_status`. If `"success"`, skip to Phase 1. -## Artifacts to read first +If `"partial"`, walk `handoff.steps` — each has `status` and `log` (path to the raw log file). For each failed step: -- `logs/setup-migration/handoff.json` — **start here.** Structured summary of every sub-step: `status`, `fields`, `notes`, plus detected channels, group selection, and a top-level `followups` list. The top-level `overall_status` tells you at a glance what kind of session this is. -- `logs/setup.log` — the progression log. Each `migrate-*` sub-step has one entry with status, duration, and a pointer to its raw log. -- `logs/setup-steps/NN-migrate-*.log` — raw per-sub-step stdout+stderr. Read these when a step failed or you need to understand why. -- `logs/setup-migration/schema-mismatch.json` — only exists if `migrate-validate` rejected the v1 DB shape. Describes what was missing. -- `logs/setup-migration/inactive-tasks.json` — v1 scheduled tasks we didn't migrate (completed, stopped, or unmappable schedule types). +1. Read its log file at `handoff.step_logs_dir/.log`. +2. Explain what failed in one sentence. +3. Fix it if mechanical (re-run the step script, hand-write a DB insert, copy a missed file). The step scripts are at `setup/migrate-v2/.ts` and accept `` as the first argument. +4. Use `AskUserQuestion` when a judgment call is needed. -## Flow +Common failures: +- **1b-db failed**: JID couldn't be parsed. Ask the user for the channel type, insert `agent_groups` + `messaging_groups` manually. +- **1d-sessions failed**: v2 DB wasn't seeded yet. Re-run after fixing 1b. +- **1e-tasks failed**: session doesn't exist yet. Re-run after fixing 1d. +- **2c-install-\ failed**: `git fetch origin channels` may have failed (network). Try again, or ask the user to run manually. +- **3e-container-build failed**: Docker issue. Read the build log, suggest fixes. -### Phase A — always run: owner seeding + access policy +After resolving all failures, proceed to Phase 1. -The automation deliberately did not seed `users`, `user_roles`, or flip `messaging_groups.unknown_sender_policy`. v1 has no ground truth for who the owner is, and no single source for the "anyone can message / only known users" setting. Ask the user. +## Phase 1: Owner and access -1. Read `handoff.json` → `detected_channels` to know which channel(s) to address the user on. -2. Use `AskUserQuestion` to ask "Which handle on `` is yours?" with options pulled from context if you have any hints (e.g. recent v1 message senders), plus "Let me type it" and "Use a different channel." Build the user id as `:`. -3. Insert into v2 central DB (`data/v2.db`): - - `users(id, kind, display_name, created_at)` — use the channel_type as `kind`. - - `user_roles(user_id, role='owner', agent_group_id=NULL, granted_by=NULL, granted_at=now)`. -4. Ask "In v1, could anyone message your assistant, or only known users?" via `AskUserQuestion`: - - "Anyone could message it" → update every row in `messaging_groups` (for migrated channel_types) to `unknown_sender_policy='public'`. - - "Only known users" → leave `unknown_sender_policy='strict'`; walk the user through seeding `agent_group_members` rows for each trusted handle they name. +v2 auto-creates a `users` row for every sender it sees (via `extractAndUpsertUser` in `src/modules/permissions/index.ts`). By the time this skill runs, the owner's row likely already exists — it just needs the `owner` role granted. -Use the DB helpers in `src/db/agent-groups.ts`, `src/db/messaging-groups.ts`, and `src/db/user-roles.ts` rather than hand-rolling SQL — they keep the companion `agent_destinations` and indexes correct. Always init the central DB first: +**User ID format**: always `:`. Each channel populates this differently: +- **Telegram**: `telegram:` (e.g. `telegram:6037840640`) +- **Discord**: `discord:` (e.g. `discord:123456789012345678`) +- **WhatsApp**: `whatsapp:@s.whatsapp.net` (e.g. `whatsapp:14155551234@s.whatsapp.net`) +- **Slack**: `slack:` (e.g. `slack:U04ABCDEF`) +- **Others**: `:` + +**Steps:** + +1. Query `users` table: `SELECT id, kind, display_name FROM users`. +2. If exactly one user exists, confirm: `AskUserQuestion`: "Is `` (``) you?" — Yes / No, let me type it. +3. If multiple users exist, present them as options in `AskUserQuestion`. +4. If no users exist yet (service hasn't received a message), ask the user to send a test message first, then re-query. +5. Once confirmed, check `user_roles` — if the owner role already exists, skip. Otherwise insert: + ```sql + INSERT INTO user_roles (user_id, role, agent_group_id, granted_by, granted_at) + VALUES ('', 'owner', NULL, NULL, datetime('now')) + ``` + +Use the DB helpers in `src/db/user-roles.ts` — they keep indexes correct. Init the DB first: ```ts import { initDb } from '../src/db/connection.js'; @@ -60,61 +74,144 @@ const db = initDb(path.join(DATA_DIR, 'v2.db')); runMigrations(db); ``` -### Phase B — branch on `handoff.json: overall_status` +### Access policy -**If `overall_status === 'success'`** and `followups` is empty: go straight to Phase C (customization interview). +After seeding the owner, discuss the access policy. v2's `messaging_groups.unknown_sender_policy` controls who can interact with the bot. `migrate-v2.sh` set it to `public` so the bot would respond during the switchover test, but the user may want to tighten it. -**Otherwise (partial, failed, or non-empty followups)**: walk `handoff.steps` and `handoff.followups` top-to-bottom. For each entry: +Present the options via `AskUserQuestion`: -- Read the step's `fields` and `notes` and its raw log (`logs/setup-steps/NN-.log`). -- Explain the situation to the user in one sentence, then propose a fix. -- Do the fix yourself when it's mechanical (re-running an install script, seeding a missed `agent_destinations` row, re-copying a channel's auth files, manually translating an unsupported `schedule_type`). Use `AskUserQuestion` when a judgment call is needed (is this orphan channel worth keeping? is this v1 container_config still relevant?). +1. **Public** (current) — anyone can message the bot. Good for personal DM bots. +2. **Known users only** — only users in `agent_group_members` can trigger the bot. Others are silently dropped. +3. **Approval required** — unknown senders trigger an approval request to the owner. Good for group chats where you want to vet new members. -Common cases: +If the user picks option 2 or 3, seed the known users from v1's message history. The v1 database is at `/store/messages.db`. It has a `messages` table with `sender` and `sender_name` columns. For each group: -- **`migrate-validate` status=failed**: the v1 DB had an unexpected shape. Read `schema-mismatch.json`. If tables are missing, the user may have run a very old or customized v1 — ask before trying to salvage. If only columns are missing, you can often proceed by hand-writing the SELECT with the columns that exist. -- **`migrate-db` status=partial, SKIPPED>0**: some `registered_groups` rows didn't seed. The `notes` field of the step entry names each failed folder. Most commonly: a JID we couldn't parse. Ask the user whether to manually wire each. -- **`migrate-channels` status=partial, some entries `not_supported`**: v1 had channels v2 doesn't ship a skill for yet. Ask the user whether to keep the `messaging_groups` rows (they'll stay orphaned until v2 grows the adapter) or delete them. -- **`migrate-channel-auth` has `files_missing`**: for WhatsApp specifically, encryption sessions often can't survive the copy — tell the user a fresh pair may be needed via `/add-whatsapp`. -- **Per-folder `.v1-container-config.json` sidecars exist**: read each, discuss with the user, and translate to v2's `groups//container.json` format. +```sql +-- v1: unique senders per chat (excluding bot messages) +SELECT DISTINCT sender, sender_name +FROM messages +WHERE chat_jid = '' AND is_from_me = 0 AND sender IS NOT NULL +``` -### Phase C — customizations (fork-aware) +The `sender` value is a platform handle (e.g. `6037840640` for Telegram). Build the v2 user ID by inferring the channel type from the chat JID prefix (use `parseJid` from `setup/migrate-v1/shared.ts`) and combining: `:`. -NanoClaw recommends running on a fork, so most real v1 installs have at least some customizations. +For each sender: +1. Upsert into `users(id, kind, display_name)` if not already present. +2. Insert into `agent_group_members(user_id, agent_group_id)` for each agent group wired to that messaging group. -**Start with divergence detection.** In the v1 repo at `handoff.v1_path`: +Show the user the list of senders being imported and let them deselect any they don't want. + +Then update the messaging groups: +```sql +UPDATE messaging_groups SET unknown_sender_policy = '' +WHERE id IN (SELECT id FROM messaging_groups WHERE channel_type IN ()) +``` + +## Phase 2: Clean up CLAUDE.local.md + +The migration copied v1's entire CLAUDE.md into CLAUDE.local.md for each group. This file now contains v1 boilerplate that v2 handles through its own composed fragments (`container/CLAUDE.md` + `.claude-fragments/module-*.md`). The user's customizations are buried inside. + +For each group that has a `CLAUDE.local.md`: + +1. Read the file. +2. Read the v1 template it was based on. Determine which template by checking the v1 install: + - If the group had `is_main=1` in v1's `registered_groups`, the template was `groups/main/CLAUDE.md` + - Otherwise, the template was `groups/global/CLAUDE.md` + - The v1 path is in `handoff.json` → `v1_path` +3. Diff the file against the template. Identify sections that are: + - **Stock boilerplate** (identical to template) — remove. v2's fragments cover this. + - **User customizations** (added sections, modified sections) — keep. +4. The following v1 sections are now handled by v2 fragments and should be removed even if slightly modified: + - "What You Can Do" → v2 runtime system prompt + - "Communication" / "Internal thoughts" / "Sub-agents" → `container/CLAUDE.md` + `module-core.md` + - "Your Workspace" / workspace path references → `container/CLAUDE.md` + - "Memory" (the stock version) → `container/CLAUDE.md` + - "Message Formatting" → `container/CLAUDE.md` + - "Admin Context" → v2 uses `user_roles`, not is_main + - "Authentication" → v2 uses OneCLI + - "Container Mounts" → v2 mounts are different + - "Managing Groups" / "Finding Available Groups" / "Registered Groups Config" → v2 entity model, no IPC + - "Global Memory" → v2 has `.claude-shared.md` symlink + - "Scheduling for Other Groups" → `module-scheduling.md` + - "Task Scripts" → `module-scheduling.md` + - "Sender Allowlist" → v2 uses `unknown_sender_policy` + `user_roles` +5. Fix path references in kept sections: + - `/workspace/group/` → `/workspace/agent/` + - `/workspace/project/` → these paths don't exist in v2; discuss with the user + - `/workspace/ipc/` → gone; remove references + - `/workspace/extra/` → v2 uses `container.json` `additionalMounts`; keep but note the path may change +6. Keep the `# Name` heading and first paragraph (identity) — this is the user's agent personality. +7. Show the user the proposed new CLAUDE.local.md before writing it. Use `AskUserQuestion`: "Here's what I'd keep — look right?" with options to approve, edit, or keep the original. + +If a CLAUDE.local.md has no user customizations (pure template copy), write a minimal file with just the identity heading. + +## Phase 3: Container config + +`migrate-v2.sh` writes `container.json` directly from v1's `container_config` (the `additionalMounts` shape is identical). If the v1 config was unparseable, it falls back to a `.v1-container-config.json` sidecar. + +For each group, check: + +1. If `container.json` exists, read it and verify the `additionalMounts` host paths are still valid on this machine. Flag any that don't exist. +2. If `.v1-container-config.json` exists (parse failure fallback), read it, discuss with the user, and write a proper `container.json`. Then delete the sidecar. +3. Check for `env` or `packages` fields — `env` may overlap with OneCLI vault, `packages` (apt/npm) are portable. + +## Phase 4: Fork customizations + +Check whether the user's v1 install was a customized fork. ```bash cd -git remote -v # identify the upstream remote -git log --oneline /main..HEAD # commits ahead of upstream +git remote -v +git log --oneline /main..HEAD 2>/dev/null ``` -If the log is **empty**: stock v1. Tell the user "no customizations to port" and skip the rest of Phase C. +If no commits ahead of upstream: stock v1, skip this phase. -If the log has commits, show them to the user and offer a scope via `AskUserQuestion`: +If there are commits: -1. **Mechanical** (recommended) — copy the portable categories (skills, docs), stash the rest as reference. -2. **Full interview** — walk each commit with me, decide one-by-one. Use `Explore` sub-agents for diffs > 10 files. -3. **Reference only** — stash everything to `docs/v1-fork-reference/`, copy nothing now. - -**Portability rules of thumb:** -- **Portable**: `container/skills/*`, `.claude/skills/*`, `docs/*`, top-level config. Scan each with `scanForV1Patterns` (in `setup/migrate-v1/shared.ts`) before copying — clean ones land as-is, dirty ones get a followup. -- **Not portable**: `src/*` (host) and `container/agent-runner/src/*` (agent-runner). v2's architecture is fundamentally different (Node host with split session DBs vs v1's single process + IPC file queue). Stash to `docs/v1-fork-reference/` with a README explaining the v1→v2 mapping — **don't translate**. Mechanical translation is a trap; let the user rebuild the feature on v2 primitives. -- **Already handled**: `groups/*` — `migrate-groups` copied these and flagged v1 patterns. Don't redo in Phase C. -- **Case by case**: `package.json` deps — check whether v2 already has each; never add to v2's lockfile without approval (supply-chain `minimumReleaseAge` applies). - -When stashing, write `docs/v1-fork-reference/README.md` with commits list, stashed source files, and the suggested porting plan. +1. Show the commit list to the user. +2. `AskUserQuestion`: "How do you want to handle your v1 customizations?" + - **Copy portable items** (recommended) — copy `container/skills/*`, `.claude/skills/*`, `docs/*`. Scan each with `scanForV1Patterns` from `setup/migrate-v1/shared.ts`. + - **Full walkthrough** — go commit by commit, decide together. + - **Reference only** — stash to `docs/v1-fork-reference/` for later. +3. Source code (`src/*`, `container/agent-runner/src/*`) is NOT portable — v2's architecture is fundamentally different. Stash to `docs/v1-fork-reference/` with a README explaining what each file did. Don't translate. ## Principles -- **Never silently copy code.** Read, explain, propose, apply. Show diffs before applying when non-trivial. -- **Credentials are masked when displayed** (first 4 + `...` + last 4 characters). The handoff file doesn't contain values; keep it that way. -- **The v1 checkout is read-only.** We never delete or modify `~/nanoclaw`. If the user wants to retire it later, that's a separate conversation. -- **No migration re-runs.** The `migrate-*` sub-steps are idempotent, but re-running them from inside this skill is almost always the wrong move — finish by hand. Re-running is for when the user re-runs `bash nanoclaw.sh`. -- **`handoff.json` is source of truth across context compactions.** If the conversation gets compacted mid-work, re-read it and `git status` to recover state. Do not maintain a separate state file. +- **v1 checkout is read-only.** Never modify files under `handoff.v1_path`. +- **Show before writing.** Show diffs/proposed content before modifying CLAUDE.local.md or container.json. +- **Mask credentials** when displaying (first 4 + `...` + last 4 characters). +- **`handoff.json` is the recovery point.** If context gets compacted, re-read it and `git status` to recover state. -## When you're done +## Setup steps you can run -- Delete `logs/setup-migration/handoff.json` once every followup is cleared and the user confirms. The file is a working document, not a record — if the user wants a record, offer to move it to `docs/migration-.md` before deleting. -- Tell the user: if the service is running (check `launchctl list | grep nanoclaw` on macOS or `systemctl --user status nanoclaw*` on Linux), restart it so the seeded `users` / `user_roles` / any channel installs take effect. +The setup flow at `setup/index.ts` has individual steps you can invoke if something is missing or failed: + +```bash +pnpm exec tsx setup/index.ts --step +``` + +| Step | When to use | +|------|-------------| +| `onecli` | OneCLI not installed or not healthy | +| `auth` | No Anthropic credential in vault | +| `container` | Container image needs rebuild | +| `service` | Service not installed or not running | +| `mounts` | Mount allowlist missing | +| `verify` | End-to-end health check (run after everything else) | +| `environment` | System check (Node, dirs) | + +## When done + +1. Run the verify step to confirm everything works: + ```bash + pnpm exec tsx setup/index.ts --step verify + ``` +2. Delete `logs/setup-migration/handoff.json` — offer to save as `docs/migration-.md` first. +3. Restart the service if running so changes take effect: + ```bash + # Linux + systemctl --user restart nanoclaw-v2-* + # macOS + launchctl kickstart -k gui/$(id -u)/com.nanoclaw-v2-* + ``` diff --git a/docs/migration-dev.md b/docs/migration-dev.md new file mode 100644 index 0000000..60feb4e --- /dev/null +++ b/docs/migration-dev.md @@ -0,0 +1,139 @@ +# v1 → v2 Migration — Development Guide + +How to test, develop, and debug the migration flow. + +## Quick start + +```bash +# Full cycle: reset → migrate → Claude finishes +bash migrate-v2-reset.sh && bash migrate-v2.sh +``` + +## Architecture + +Two-part migration: + +1. **`migrate-v2.sh`** — deterministic bash script. Handles prerequisites, DB seeding, file copies, channel install, container build, service switchover. Writes `logs/setup-migration/handoff.json` then `exec`s into Claude. + +2. **`/migrate-from-v1` skill** — Claude-driven. Reads the handoff, seeds owner/roles, cleans up CLAUDE.local.md, validates container configs, ports fork customizations. + +## File layout + +``` +migrate-v2.sh # Entry point +migrate-v2-reset.sh # Wipe v2 state for re-testing +setup/migrate-v2/ + env.ts # Phase 1a: merge .env + db.ts # Phase 1b: seed v2 DB + groups.ts # Phase 1c: copy group folders + container.json + sessions.ts # Phase 1d: copy sessions + set continuation + tasks.ts # Phase 1e: port scheduled tasks + channel-auth.ts # Phase 2b: copy channel auth state + select-channels.ts # Phase 2a: clack multiselect + switchover-prompt.ts # Service switch prompts +setup/migrate-v1/shared.ts # Shared helpers (JID parsing, trigger mapping, etc.) +.claude/skills/migrate-from-v1/ # The Claude skill +logs/setup-migration/handoff.json # Written by migrate-v2.sh, read by skill +logs/migrate-steps/*.log # Per-step raw output +``` + +## Development loop + +```bash +# Reset v2 to clean state (keeps node_modules) +bash migrate-v2-reset.sh + +# Run migration with non-interactive channel selection +NANOCLAW_CHANNELS="telegram" bash migrate-v2.sh + +# Or run interactively (clack multiselect) +bash migrate-v2.sh +``` + +`migrate-v2-reset.sh` wipes: `data/`, `logs/`, `.env`, `groups/` (restores git-tracked), `container/skills/` (restores git-tracked), `src/channels/` (restores git-tracked). + +It does NOT wipe `node_modules/` (expensive to reinstall). + +## Testing individual steps + +Each step is a standalone TypeScript file: + +```bash +# Run a single step (after pnpm install) +pnpm exec tsx setup/migrate-v2/env.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/db.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/groups.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/sessions.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/tasks.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/channel-auth.ts /path/to/v1 telegram discord +``` + +Each prints `OK:
`, `SKIPPED:`, or errors to stdout. Exit 0 on success/skip, non-zero on failure. + +## Debugging + +### Check what was migrated + +```bash +# Agent groups +sqlite3 data/v2.db "SELECT * FROM agent_groups" + +# Messaging groups + wiring +sqlite3 data/v2.db "SELECT mg.id, mg.channel_type, mg.platform_id, mg.unknown_sender_policy, mga.engage_mode, mga.engage_pattern FROM messaging_groups mg JOIN messaging_group_agents mga ON mga.messaging_group_id = mg.id" + +# Sessions +sqlite3 data/v2.db "SELECT * FROM sessions" + +# Users and roles +sqlite3 data/v2.db "SELECT * FROM users" +sqlite3 data/v2.db "SELECT * FROM user_roles" + +# Session continuation (which Claude Code session will be resumed) +AG_ID=$(sqlite3 data/v2.db "SELECT id FROM agent_groups LIMIT 1") +SESS_ID=$(sqlite3 data/v2.db "SELECT id FROM sessions LIMIT 1") +sqlite3 data/v2-sessions/$AG_ID/$SESS_ID/outbound.db "SELECT * FROM session_state" + +# Scheduled tasks +sqlite3 data/v2-sessions/$AG_ID/$SESS_ID/inbound.db "SELECT id, kind, recurrence, status FROM messages_in WHERE kind='task'" +``` + +### Check handoff + +```bash +python3 -m json.tool logs/setup-migration/handoff.json +``` + +### Common issues + +**Bot doesn't respond after switchover:** +1. Check both services aren't running: `systemctl --user list-units 'nanoclaw*'` +2. Check error log: `tail logs/nanoclaw.error.log` +3. Check sender policy: `sqlite3 data/v2.db "SELECT unknown_sender_policy FROM messaging_groups"` — must be `public` before owner is seeded +4. Check engage pattern: `sqlite3 data/v2.db "SELECT engage_mode, engage_pattern FROM messaging_group_agents"` — should be `pattern` / `.` for respond-to-everything + +**Session not continuing from v1:** +1. Check continuation is set: see "Session continuation" query above +2. Check JSONL exists at the right path: `ls data/v2-sessions//.claude-shared/projects/-workspace-agent/` +3. The v1 session JSONL should be copied from `-workspace-group/` to `-workspace-agent/` (v2 container CWD is `/workspace/agent`) + +**Service switchover revert didn't work:** +1. The v2 service name is `nanoclaw-v2-` — find it: `systemctl --user list-units 'nanoclaw*'` +2. Manually stop: `systemctl --user stop && systemctl --user disable ` +3. Restart v1: `systemctl --user start nanoclaw` + +### Step logs + +Each step writes raw output to `logs/migrate-steps/.log`. Read these when a step fails: + +```bash +cat logs/migrate-steps/1b-db.log +cat logs/migrate-steps/1d-sessions.log +``` + +## Key decisions + +- `unknown_sender_policy` is set to `public` during migration so the bot responds immediately. The `/migrate-from-v1` skill tightens it after seeding the owner. +- `requires_trigger=0` in v1 takes priority over a non-empty `trigger_pattern` — it means "respond to everything." +- v1 `container_config.additionalMounts` is written directly to v2 `container.json` (same shape). +- v1 Claude Code sessions are copied from `-workspace-group/` to `-workspace-agent/` and the session ID is written to `outbound.db` as `continuation:claude` so the agent-runner resumes the same conversation. +- `exec claude "/migrate-from-v1"` at the end replaces the bash process — `write_handoff` is called explicitly before `exec` since EXIT traps don't fire on `exec`. diff --git a/migrate-v2-reset.sh b/migrate-v2-reset.sh new file mode 100644 index 0000000..b795745 --- /dev/null +++ b/migrate-v2-reset.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# +# migrate-v2-reset.sh — Wipe v2 migration state back to clean. +# +# For development iteration: +# bash migrate-v2-reset.sh && bash migrate-v2.sh +# +# What it removes: +# - data/ (v2 DBs, session state) +# - logs/ (migration + setup logs) +# - .env (merged env keys) +# - groups/*/ (non-git group folders copied from v1) +# +# What it restores: +# - groups/global/CLAUDE.md and groups/main/CLAUDE.md from git +# +# What it does NOT touch: +# - node_modules/ (expensive to reinstall, keep it) +# - The v1 install (read-only, never modified) + +set -euo pipefail + +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$PROJECT_ROOT" + +use_ansi() { [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; } +dim() { use_ansi && printf '\033[2m%s\033[0m' "$1" || printf '%s' "$1"; } +green() { use_ansi && printf '\033[32m%s\033[0m' "$1" || printf '%s' "$1"; } + +clean() { + local target=$1 label=$2 + if [ -e "$target" ]; then + rm -rf "$target" + printf '%s Removed %s\n' "$(green '✓')" "$label" + fi +} + +echo +printf '%s\n\n' "$(dim 'Resetting v2 migration state…')" + +clean "data" "data/" +clean "logs" "logs/" +clean ".env" ".env" + +# Remove all group folders, then restore the two git-tracked ones +if [ -d "groups" ]; then + rm -rf groups + printf '%s Removed %s\n' "$(green '✓')" "groups/" +fi +git checkout -- groups/ 2>/dev/null || true +printf '%s Restored %s\n' "$(green '✓')" "groups/ from git" + +# Restore container/skills/ to git state (remove v1-copied skills) +git checkout -- container/skills/ 2>/dev/null || true +# Remove any untracked skill dirs that were copied from v1 +for d in container/skills/*/; do + [ -d "$d" ] || continue + if ! git ls-files --error-unmatch "$d" >/dev/null 2>&1; then + rm -rf "$d" + fi +done +printf '%s Restored %s\n' "$(green '✓')" "container/skills/ from git" + +# Restore channel code (src/channels/) to git state +git checkout -- src/channels/ 2>/dev/null || true +printf '%s Restored %s\n' "$(green '✓')" "src/channels/ from git" + +echo +printf '%s\n\n' "$(dim 'Clean. Run: bash migrate-v2.sh')" diff --git a/migrate-v2.sh b/migrate-v2.sh new file mode 100644 index 0000000..325b491 --- /dev/null +++ b/migrate-v2.sh @@ -0,0 +1,641 @@ +#!/usr/bin/env bash +# +# migrate-v2.sh — Migrate a NanoClaw v1 install into this v2 checkout. +# +# Run from the v2 directory: +# bash migrate-v2.sh +# +# Finds v1 automatically (sibling directory, or $NANOCLAW_V1_PATH). +# Installs prerequisites (Node, pnpm, deps) via the existing setup.sh +# bootstrap, then runs the migration steps. +# +# Idempotent — safe to re-run. Use migrate-v2-reset.sh to wipe v2 state +# back to clean for development iteration. + +set -uo pipefail + +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$PROJECT_ROOT" + +LOGS_DIR="$PROJECT_ROOT/logs" +STEPS_DIR="$LOGS_DIR/migrate-steps" +MIGRATE_LOG="$LOGS_DIR/migrate-v2.log" + +# Defaults for variables that may not be set if we exit early +V1_PATH="" +V1_VERSION="unknown" +ONECLI_OK=false +SERVICE_SWITCHED=false +SELECTED_CHANNELS=() +ABORTED_AT="" + +# Write handoff.json on any exit so the skill can always read it +write_handoff() { + local handoff_dir="$LOGS_DIR/setup-migration" + mkdir -p "$handoff_dir" + + local has_failures=false + for step_name in "${!STEP_RESULTS[@]}"; do + [ "${STEP_RESULTS[$step_name]}" = "failed" ] && has_failures=true + done + + local overall="success" + $has_failures && overall="partial" + [ -n "$ABORTED_AT" ] && overall="failed" + + local steps_json="{" + for step_name in "${!STEP_RESULTS[@]}"; do + steps_json="${steps_json}\"${step_name}\": {\"status\": \"${STEP_RESULTS[$step_name]}\", \"log\": \"logs/migrate-steps/${step_name}.log\"}," + done + steps_json="${steps_json%,}}" + + cat > "$handoff_dir/handoff.json" </dev/null | sed 's/,$//')], + "onecli_healthy": $ONECLI_OK, + "service_switched": $SERVICE_SWITCHED, + "steps": $steps_json, + "step_logs_dir": "logs/migrate-steps", + "followups": [ + "Seed owner user and access policy", + "Review CLAUDE.local.md files for v1-specific patterns", + "Verify container.json mount paths are valid" + ] +} +HANDOFF_EOF +} + +trap write_handoff EXIT + +abort() { + ABORTED_AT="$1" + log "ABORTED at $1" + exit 1 +} + +# ─── output helpers ────────────────────────────────────────────────────── + +use_ansi() { [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; } +dim() { use_ansi && printf '\033[2m%s\033[0m' "$1" || printf '%s' "$1"; } +green() { use_ansi && printf '\033[32m%s\033[0m' "$1" || printf '%s' "$1"; } +red() { use_ansi && printf '\033[31m%s\033[0m' "$1" || printf '%s' "$1"; } +bold() { use_ansi && printf '\033[1m%s\033[0m' "$1" || printf '%s' "$1"; } +clear_line() { use_ansi && printf '\r\033[2K' || printf '\n'; } + +step_ok() { printf '%s %s\n' "$(green '✓')" "$1"; } +step_fail() { printf '%s %s\n' "$(red '✗')" "$1"; } +step_skip() { printf '%s %s\n' "$(dim '–')" "$1"; } +step_info() { printf '%s %s\n' "$(dim '·')" "$1"; } + +ts_utc() { date -u +%Y-%m-%dT%H:%M:%SZ; } + +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >> "$MIGRATE_LOG" +} + +# ─── init logs ─────────────────────────────────────────────────────────── + +mkdir -p "$STEPS_DIR" +{ + echo "## $(ts_utc) · migrate-v2.sh started" + echo " cwd: $PROJECT_ROOT" + echo "" +} > "$MIGRATE_LOG" + +echo +bold "NanoClaw v1 → v2 migration" +echo +echo + +# ─── phase 0a: bootstrap prerequisites ────────────────────────────────── + +step_info "Installing prerequisites (Node, pnpm, dependencies)…" + +BOOTSTRAP_RAW="$STEPS_DIR/01-bootstrap.log" +export NANOCLAW_BOOTSTRAP_LOG="$BOOTSTRAP_RAW" + +if bash "$PROJECT_ROOT/setup.sh" > "$BOOTSTRAP_RAW" 2>&1; then + # Parse the status block from setup.sh output + STATUS=$(grep '^STATUS:' "$BOOTSTRAP_RAW" | head -1 | sed 's/^STATUS: *//') + NODE_VERSION=$(grep '^NODE_VERSION:' "$BOOTSTRAP_RAW" | head -1 | sed 's/^NODE_VERSION: *//') + + if [ "$STATUS" = "success" ]; then + step_ok "Prerequisites ready $(dim "(node $NODE_VERSION)")" + log "Bootstrap succeeded: node=$NODE_VERSION" + else + step_fail "Bootstrap reported: $STATUS" + echo + dim " See: $BOOTSTRAP_RAW" + echo + abort "bootstrap" + fi +else + step_fail "Bootstrap failed" + echo + echo "$(dim '── last 20 lines ──')" + tail -20 "$BOOTSTRAP_RAW" 2>/dev/null || true + echo + dim " Full log: $BOOTSTRAP_RAW" + echo + abort "bootstrap" +fi + +# setup.sh may have installed pnpm to a prefix not on our PATH — replay +# the same lookup nanoclaw.sh does. +if ! command -v pnpm >/dev/null 2>&1 && command -v npm >/dev/null 2>&1; then + NPM_PREFIX="$(npm config get prefix 2>/dev/null)" + if [ -n "$NPM_PREFIX" ] && [ -x "$NPM_PREFIX/bin/pnpm" ]; then + export PATH="$NPM_PREFIX/bin:$PATH" + fi +fi + +if ! command -v pnpm >/dev/null 2>&1; then + step_fail "pnpm not found after bootstrap" + abort "pnpm-missing" +fi + +# ─── phase 0b: find v1 install ────────────────────────────────────────── + +find_v1() { + # Explicit override + if [ -n "${NANOCLAW_V1_PATH:-}" ]; then + if [ -f "$NANOCLAW_V1_PATH/store/messages.db" ]; then + echo "$NANOCLAW_V1_PATH" + return 0 + fi + step_fail "NANOCLAW_V1_PATH=$NANOCLAW_V1_PATH does not contain store/messages.db" + return 1 + fi + + # Scan sibling directories for anything claw-ish with a v1 DB + local parent + parent="$(dirname "$PROJECT_ROOT")" + for entry in "$parent"/*/; do + [ -d "$entry" ] || continue + # Skip ourselves + [ "$(cd "$entry" && pwd)" = "$PROJECT_ROOT" ] && continue + # Must have the v1 DB + [ -f "$entry/store/messages.db" ] || continue + # Must not be v2 (check package.json version) + if [ -f "$entry/package.json" ]; then + local ver + ver=$(grep '"version"' "$entry/package.json" 2>/dev/null | head -1 | sed -E 's/.*"([0-9]+)\..*/\1/') + [ "$ver" = "2" ] && continue + fi + echo "$(cd "$entry" && pwd)" + return 0 + done + + return 1 +} + +V1_PATH="" +if V1_PATH=$(find_v1); then + V1_VERSION=$(grep '"version"' "$V1_PATH/package.json" 2>/dev/null | head -1 | sed -E 's/.*"([^"]+)".*/\1/' || echo "unknown") + step_ok "Found v1 at $(dim "$V1_PATH") $(dim "(v$V1_VERSION)")" + log "v1 found: $V1_PATH (v$V1_VERSION)" +else + step_fail "No v1 install found" + echo + echo " $(dim 'Set NANOCLAW_V1_PATH to point at your v1 checkout:')" + echo " $(dim 'NANOCLAW_V1_PATH=~/nanoclaw bash migrate-v2.sh')" + echo + abort "v1-not-found" +fi + +# ─── phase 0c: validate v1 DB ─────────────────────────────────────────── + +V1_DB="$V1_PATH/store/messages.db" + +# Quick schema check — make sure the tables we need exist +TABLES=$(sqlite3 "$V1_DB" ".tables" 2>/dev/null || true) + +if echo "$TABLES" | grep -q "registered_groups"; then + step_ok "v1 database has registered_groups" +else + step_fail "v1 database missing registered_groups table" + abort "v1-db-invalid" +fi + +# Show what we found +GROUP_COUNT=$(sqlite3 "$V1_DB" "SELECT COUNT(*) FROM registered_groups" 2>/dev/null || echo 0) +TASK_COUNT=$(sqlite3 "$V1_DB" "SELECT COUNT(*) FROM scheduled_tasks WHERE status='active'" 2>/dev/null || echo 0) +ENV_KEYS=0 +if [ -f "$V1_PATH/.env" ]; then + ENV_KEYS=$(grep -c '=' "$V1_PATH/.env" 2>/dev/null || echo 0) +fi + +step_info "v1 state: $(bold "$GROUP_COUNT") groups, $(bold "$TASK_COUNT") active tasks, $(bold "$ENV_KEYS") env keys" + +echo +step_ok "Phase 0 complete — ready to migrate" +echo +log "Phase 0 complete: groups=$GROUP_COUNT tasks=$TASK_COUNT env_keys=$ENV_KEYS" + +export NANOCLAW_V1_PATH="$V1_PATH" +export NANOCLAW_V2_PATH="$PROJECT_ROOT" + +# ─── run_step helper ───────────────────────────────────────────────────── +# Runs a TypeScript migration step, captures output, reports success/failure. + +# Track step outcomes for handoff.json +declare -A STEP_RESULTS + +run_step() { + local name=$1 label=$2 script=$3 + shift 3 + local raw="$STEPS_DIR/${name}.log" + + if pnpm exec tsx "$script" "$@" > "$raw" 2>&1; then + local result + result=$(grep '^OK:' "$raw" | head -1 || true) + step_ok "$label $(dim "$result")" + log "$name: $result" + STEP_RESULTS[$name]="success" + elif grep -q '^SKIPPED:' "$raw" 2>/dev/null; then + local reason + reason=$(grep '^SKIPPED:' "$raw" | head -1 | sed 's/^SKIPPED://') + step_skip "$label $(dim "($reason)")" + log "$name: skipped ($reason)" + STEP_RESULTS[$name]="skipped" + else + step_fail "$label" + echo + tail -10 "$raw" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + echo + log "$name: FAILED (see $raw)" + STEP_RESULTS[$name]="failed" + fi +} + +# ─── phase 1: core state ──────────────────────────────────────────────── + +echo "$(bold 'Phase 1: Core state')" +echo + +run_step "1a-env" \ + "Merge .env" \ + "setup/migrate-v2/env.ts" "$V1_PATH" + +run_step "1b-db" \ + "Seed v2 database" \ + "setup/migrate-v2/db.ts" "$V1_PATH" + +run_step "1c-groups" \ + "Copy group folders" \ + "setup/migrate-v2/groups.ts" "$V1_PATH" + +run_step "1d-sessions" \ + "Copy session data" \ + "setup/migrate-v2/sessions.ts" "$V1_PATH" + +run_step "1e-tasks" \ + "Port scheduled tasks" \ + "setup/migrate-v2/tasks.ts" "$V1_PATH" + +echo +step_ok "Phase 1 complete" +echo + +# ─── phase 2: channels (interactive) ──────────────────────────────────── + +echo "$(bold 'Phase 2: Channels')" +echo + +# Channel selection — clack multiselect (interactive) or NANOCLAW_CHANNELS env var. +# NANOCLAW_CHANNELS accepts comma-separated channel names: "telegram,discord" +SELECTED_CHANNELS=() +CHANNEL_SELECT_OUT="$STEPS_DIR/2a-channels-selected.txt" + +pnpm exec tsx setup/migrate-v2/select-channels.ts "$CHANNEL_SELECT_OUT" || true + +if [ -f "$CHANNEL_SELECT_OUT" ]; then + while IFS= read -r ch; do + [ -n "$ch" ] && SELECTED_CHANNELS+=("$ch") + done < "$CHANNEL_SELECT_OUT" +fi + +if [ ${#SELECTED_CHANNELS[@]} -eq 0 ]; then + echo + step_skip "No channels selected" +else + echo + step_info "Selected: ${SELECTED_CHANNELS[*]}" + echo + + # 2b. Copy channel auth state + run_step "2b-channel-auth" \ + "Copy channel credentials" \ + "setup/migrate-v2/channel-auth.ts" "$V1_PATH" "${SELECTED_CHANNELS[@]}" + + # 2c. Install channel code + for ch in "${SELECTED_CHANNELS[@]}"; do + INSTALL_SCRIPT="setup/install-${ch}.sh" + if [ -f "$INSTALL_SCRIPT" ]; then + STEP_LOG="$STEPS_DIR/2c-install-${ch}.log" + if bash "$INSTALL_SCRIPT" > "$STEP_LOG" 2>&1; then + STATUS_LINE=$(grep '^STATUS:' "$STEP_LOG" | head -1 | sed 's/^STATUS: *//') + if [ "$STATUS_LINE" = "already-installed" ]; then + step_skip "Install $ch $(dim "(already installed)")" + else + step_ok "Install $ch" + fi + log "install-$ch: $STATUS_LINE" + else + step_fail "Install $ch" + tail -5 "$STEP_LOG" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "install-$ch: FAILED (see $STEP_LOG)" + fi + else + step_skip "Install $ch $(dim "(no install script)")" + fi + done +fi + +echo +step_ok "Phase 2 complete" +echo + +# ─── phase 3: infrastructure ──────────────────────────────────────────── + +echo "$(bold 'Phase 3: Infrastructure')" +echo + +# 3a. OneCLI — detect or install via setup step +ONECLI_OK=false +ONECLI_URL_FROM_ENV=$(grep '^ONECLI_URL=' .env 2>/dev/null | head -1 | sed 's/^ONECLI_URL=//') +ONECLI_URL_CHECK="${ONECLI_URL_FROM_ENV:-http://127.0.0.1:10254}" + +if curl -sf "${ONECLI_URL_CHECK}/health" >/dev/null 2>&1; then + step_ok "OneCLI running at $(dim "$ONECLI_URL_CHECK")" + ONECLI_OK=true + log "OneCLI: running at $ONECLI_URL_CHECK" +else + # Run the setup onecli step — it handles install, reuse, and health checks + step_info "Setting up OneCLI…" + ONECLI_LOG="$STEPS_DIR/3a-onecli.log" + ONECLI_ERR="$STEPS_DIR/3a-onecli.err" + if pnpm exec tsx setup/index.ts --step onecli > "$ONECLI_LOG" 2>"$ONECLI_ERR"; then + step_ok "OneCLI ready" + ONECLI_OK=true + STEP_RESULTS["3a-onecli"]="success" + log "OneCLI: installed/configured" + else + step_fail "OneCLI setup failed $(dim "(see $ONECLI_LOG)")" + STEP_RESULTS["3a-onecli"]="failed" + log "OneCLI: FAILED" + fi +fi + +# 3b. Anthropic credential — run the auth setup step if no credential found +if grep -qE '^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN)=' .env 2>/dev/null; then + step_ok "Anthropic credential found in .env" + log "Anthropic credential: found in .env" +elif [ "$ONECLI_OK" = "true" ]; then + step_info "Registering Anthropic credential…" + AUTH_LOG="$STEPS_DIR/3b-auth.log" + AUTH_ERR="$STEPS_DIR/3b-auth.err" + if pnpm exec tsx setup/index.ts --step auth > "$AUTH_LOG" 2>"$AUTH_ERR"; then + step_ok "Anthropic credential registered" + STEP_RESULTS["3b-auth"]="success" + log "Anthropic credential: registered via auth step" + else + step_fail "Auth setup failed $(dim "(see $AUTH_LOG)")" + STEP_RESULTS["3b-auth"]="failed" + log "Anthropic credential: FAILED" + fi +else + step_info "No Anthropic credential $(dim "(OneCLI not available — add manually to .env)")" + log "Anthropic credential: skipped (no OneCLI)" +fi + +# 3c. Docker check +if command -v docker >/dev/null 2>&1; then + DOCKER_V=$(docker --version 2>/dev/null | head -1) + step_ok "Docker available $(dim "($DOCKER_V)")" + log "Docker: $DOCKER_V" +else + step_fail "Docker not found" + step_info "$(dim "Install Docker: bash setup/install-docker.sh")" + log "Docker: not found" +fi + +# 3d. Copy container skills from v1 that v2 doesn't have +V1_SKILLS_DIR="$V1_PATH/container/skills" +V2_SKILLS_DIR="$PROJECT_ROOT/container/skills" + +if [ -d "$V1_SKILLS_DIR" ]; then + SKILLS_COPIED=0 + SKILLS_SKIPPED=0 + for skill_dir in "$V1_SKILLS_DIR"/*/; do + [ -d "$skill_dir" ] || continue + skill_name=$(basename "$skill_dir") + if [ -d "$V2_SKILLS_DIR/$skill_name" ]; then + SKILLS_SKIPPED=$((SKILLS_SKIPPED + 1)) + else + cp -r "$skill_dir" "$V2_SKILLS_DIR/$skill_name" + SKILLS_COPIED=$((SKILLS_COPIED + 1)) + fi + done + if [ $SKILLS_COPIED -gt 0 ]; then + step_ok "Copied $SKILLS_COPIED container skills $(dim "(skipped $SKILLS_SKIPPED already in v2)")" + else + step_skip "All v1 container skills already in v2 $(dim "($SKILLS_SKIPPED)")" + fi + log "Container skills: copied=$SKILLS_COPIED skipped=$SKILLS_SKIPPED" +else + step_skip "No v1 container skills" +fi + +# 3e. Build agent container image +if command -v docker >/dev/null 2>&1; then + step_info "Building agent container image…" + BUILD_LOG="$STEPS_DIR/3e-container-build.log" + if bash container/build.sh > "$BUILD_LOG" 2>&1; then + step_ok "Container image built" + log "Container build: success" + else + step_fail "Container build failed" + tail -10 "$BUILD_LOG" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "Container build: FAILED (see $BUILD_LOG)" + fi +else + step_skip "Container build $(dim "(no Docker)")" +fi + +echo +step_ok "Phase 3 complete" +echo + +# ─── service switchover ───────────────────────────────────────────────── + +echo "$(bold 'Service switchover')" +echo + +# Detect platform and service names +V1_SERVICE="" +V2_SERVICE="" +PLATFORM_SERVICE="" + +if [ "$(uname -s)" = "Darwin" ]; then + PLATFORM_SERVICE="launchd" + V1_SERVICE="com.nanoclaw" + # v2 uses install-slug for unique service names + V2_SERVICE=$(pnpm exec tsx -e "import{getLaunchdLabel}from'./src/install-slug.js';console.log(getLaunchdLabel())" 2>/dev/null || echo "") +elif [ "$(uname -s)" = "Linux" ]; then + PLATFORM_SERVICE="systemd" + V1_SERVICE="nanoclaw" + V2_SERVICE=$(pnpm exec tsx -e "import{getSystemdUnit}from'./src/install-slug.js';console.log(getSystemdUnit())" 2>/dev/null || echo "") +fi + +# Check if v1 service is running +V1_RUNNING=false +if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user is-active "$V1_SERVICE" >/dev/null 2>&1 && V1_RUNNING=true +elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl list "$V1_SERVICE" >/dev/null 2>&1 && V1_RUNNING=true +fi + +SERVICE_SWITCHED=false +if [ "$V1_RUNNING" = "true" ]; then + step_info "v1 service is running $(dim "($V1_SERVICE)")" + + # Ask user if they want to switch + SWITCH_ANSWER_FILE=$(mktemp) + pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --offer-switch "$SWITCH_ANSWER_FILE" || true + SWITCH_ANSWER=$(cat "$SWITCH_ANSWER_FILE" 2>/dev/null || echo "skip") + rm -f "$SWITCH_ANSWER_FILE" + + if [ "$SWITCH_ANSWER" = "switch" ]; then + # Stop v1 + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user stop "$V1_SERVICE" 2>/dev/null && step_ok "Stopped v1 service" || step_fail "Could not stop v1" + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl unload ~/Library/LaunchAgents/${V1_SERVICE}.plist 2>/dev/null && step_ok "Stopped v1 service" || step_fail "Could not stop v1" + fi + + # Install and start v2 service + V2_SERVICE_LOG="$STEPS_DIR/service-install.log" + V2_SERVICE_ERR="$STEPS_DIR/service-install.err" + if pnpm exec tsx setup/index.ts --step service > "$V2_SERVICE_LOG" 2>"$V2_SERVICE_ERR"; then + # Parse the actual unit name from the service step stdout (clean, no ANSI) + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + V2_SERVICE=$(grep '^SERVICE_UNIT:' "$V2_SERVICE_LOG" | head -1 | sed 's/^SERVICE_UNIT: *//') + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + V2_SERVICE=$(grep '^SERVICE_LABEL:' "$V2_SERVICE_LOG" | head -1 | sed 's/^SERVICE_LABEL: *//') + fi + step_ok "v2 service installed and started $(dim "($V2_SERVICE)")" + else + step_fail "Could not start v2 service $(dim "(see $V2_SERVICE_LOG)")" + fi + + SERVICE_SWITCHED=true + echo + step_info "v2 is running — send a test message to your bot" + echo + + # Ask: keep or revert? + KEEP_ANSWER_FILE=$(mktemp) + pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --keep-or-revert "$KEEP_ANSWER_FILE" || true + KEEP_ANSWER=$(cat "$KEEP_ANSWER_FILE" 2>/dev/null || echo "keep") + rm -f "$KEEP_ANSWER_FILE" + + if [ "$KEEP_ANSWER" = "revert" ]; then + # Stop v2 + if [ "$PLATFORM_SERVICE" = "systemd" ] && [ -n "$V2_SERVICE" ]; then + systemctl --user stop "$V2_SERVICE" 2>/dev/null || true + systemctl --user disable "$V2_SERVICE" 2>/dev/null || true + elif [ "$PLATFORM_SERVICE" = "launchd" ] && [ -n "$V2_SERVICE" ]; then + launchctl unload ~/Library/LaunchAgents/${V2_SERVICE}.plist 2>/dev/null || true + fi + + # Restart v1 + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user start "$V1_SERVICE" 2>/dev/null || true + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl load ~/Library/LaunchAgents/${V1_SERVICE}.plist 2>/dev/null || true + fi + + step_ok "Reverted to v1 service" + SERVICE_SWITCHED=false + else + step_ok "Keeping v2 service" + # Disable v1 from auto-starting + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user disable "$V1_SERVICE" 2>/dev/null || true + fi + fi + else + step_skip "Service switchover skipped" + fi +else + step_skip "v1 service not running — nothing to switch" +fi + +echo + +# ─── phase 4: handoff ─────────────────────────────────────────────────── +# handoff.json is written by the EXIT trap (write_handoff) — always, even on +# abort. Here we just print the summary. + +echo "$(bold 'Phase 4: Handoff')" +echo + +step_ok "Wrote handoff summary" + +# Summary +echo +echo "$(bold '── Migration complete ──')" +echo +echo " $(dim 'v1:') $V1_PATH" +echo " $(dim 'v2:') $PROJECT_ROOT" +echo +echo " $(bold 'What was done:')" +echo " $(green '✓') .env keys merged" +echo " $(green '✓') Database seeded (agent groups, messaging groups, wiring)" +echo " $(green '✓') Group folders copied (CLAUDE.md → CLAUDE.local.md)" +echo " $(green '✓') Session data copied" +echo " $(green '✓') Scheduled tasks ported" +if [ ${#SELECTED_CHANNELS[@]} -gt 0 ]; then +echo " $(green '✓') Channels installed: ${SELECTED_CHANNELS[*]}" +fi +echo " $(green '✓') Container skills copied" +echo " $(green '✓') Container image built" +echo +echo " $(bold 'What still needs a human:')" +if [ "$ONECLI_OK" = "false" ]; then +echo " $(dim '·') Set up OneCLI: pnpm exec tsx setup/index.ts --step onecli" +fi +if ! grep -qE '^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN)=' .env 2>/dev/null; then +echo " $(dim '·') Add Anthropic credential to .env or OneCLI vault" +fi +echo " $(dim '·') Run $(bold '/migrate-from-v1') in Claude to finish:" +echo " $(dim '- Seed your owner account')" +echo " $(dim '- Set access policies')" +echo " $(dim '- Port any custom v1 code')" +echo +echo " $(dim "Handoff: $LOGS_DIR/setup-migration/handoff.json")" +echo " $(dim "Full log: $MIGRATE_LOG")" +echo " $(dim "Step logs: $STEPS_DIR/")" +echo + +# ─── hand off to Claude ───────────────────────────────────────────────── + +if command -v claude >/dev/null 2>&1; then + write_handoff + trap - EXIT + exec claude "/migrate-from-v1" +fi diff --git a/setup/migrate-v1/shared.ts b/setup/migrate-v1/shared.ts index 4597fcf..4d2cd92 100644 --- a/setup/migrate-v1/shared.ts +++ b/setup/migrate-v1/shared.ts @@ -431,12 +431,14 @@ export function triggerToEngage(input: { if (pattern === '.' || pattern === '.*') { return { engage_mode: 'pattern', engage_pattern: '.' }; } - if (pattern) { - return { engage_mode: 'pattern', engage_pattern: pattern }; - } + // requires_trigger=0 means "respond to everything" regardless of pattern. + // The pattern was used for mention highlighting, not message gating. if (!requiresTrigger) { return { engage_mode: 'pattern', engage_pattern: '.' }; } + if (pattern) { + return { engage_mode: 'pattern', engage_pattern: pattern }; + } return { engage_mode: 'mention', engage_pattern: null }; } diff --git a/setup/migrate-v2/channel-auth.ts b/setup/migrate-v2/channel-auth.ts new file mode 100644 index 0000000..788ae9d --- /dev/null +++ b/setup/migrate-v2/channel-auth.ts @@ -0,0 +1,134 @@ +/** + * migrate-v2 step: channel-auth + * + * Copy channel auth state from v1 to v2 for selected channels. + * Handles both env keys and on-disk auth files (Baileys, Matrix, etc.) + * per the CHANNEL_AUTH_REGISTRY. + * + * Usage: pnpm exec tsx setup/migrate-v2/channel-auth.ts [channel2...] + */ +import fs from 'fs'; +import path from 'path'; + +import { CHANNEL_AUTH_REGISTRY } from '../migrate-v1/shared.js'; + +function parseEnv(filePath: string): Map { + const out = new Map(); + if (!fs.existsSync(filePath)) return out; + for (const line of fs.readFileSync(filePath, 'utf-8').split('\n')) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) continue; + const eq = trimmed.indexOf('='); + if (eq <= 0) continue; + out.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); + } + return out; +} + +function appendEnvKey(envPath: string, key: string, value: string): boolean { + const existing = parseEnv(envPath); + if (existing.has(key)) return false; + + let content = fs.existsSync(envPath) ? fs.readFileSync(envPath, 'utf-8') : ''; + if (content && !content.endsWith('\n')) content += '\n'; + content += `${key}=${value}\n`; + fs.writeFileSync(envPath, content); + return true; +} + +function copyGlob(v1Root: string, v2Root: string, relativePath: string): string[] { + const src = path.join(v1Root, relativePath); + if (!fs.existsSync(src)) return []; + + const copied: string[] = []; + const stat = fs.statSync(src); + + if (stat.isFile()) { + const dst = path.join(v2Root, relativePath); + if (!fs.existsSync(dst)) { + fs.mkdirSync(path.dirname(dst), { recursive: true }); + fs.copyFileSync(src, dst); + copied.push(relativePath); + } + } else if (stat.isDirectory()) { + const dst = path.join(v2Root, relativePath); + fs.mkdirSync(dst, { recursive: true }); + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + const sub = path.join(relativePath, entry.name); + copied.push(...copyGlob(v1Root, v2Root, sub)); + } + } + + return copied; +} + +function main(): void { + const args = process.argv.slice(2); + const v1Path = args[0]; + const channels = args.slice(1); + + if (!v1Path || channels.length === 0) { + console.error('Usage: tsx setup/migrate-v2/channel-auth.ts [channel2...]'); + process.exit(1); + } + + const v1EnvPath = path.join(v1Path, '.env'); + const v2EnvPath = path.join(process.cwd(), '.env'); + const v1Env = parseEnv(v1EnvPath); + + let envKeysCopied = 0; + let filesCopied = 0; + let channelsProcessed = 0; + const missing: string[] = []; + + for (const channel of channels) { + const spec = CHANNEL_AUTH_REGISTRY[channel]; + if (!spec) { + // Unknown channel — just try copying env keys with common naming + channelsProcessed++; + continue; + } + + // Copy env keys + for (const key of spec.v1EnvKeys) { + const value = v1Env.get(key); + if (value) { + if (appendEnvKey(v2EnvPath, key, value)) { + envKeysCopied++; + } + } + } + + // Check required v2 keys — report missing ones + const v2Env = parseEnv(v2EnvPath); + for (const req of spec.requiredV2Keys) { + if (!v2Env.has(req.key)) { + missing.push(`${channel}:${req.key} (${req.where})`); + } + } + + // Copy on-disk auth files + for (const candidate of spec.candidatePaths) { + const copied = copyGlob(v1Path, process.cwd(), candidate); + filesCopied += copied.length; + } + + channelsProcessed++; + } + + // Sync to data/env/env + if (fs.existsSync(v2EnvPath)) { + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + try { + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { /* non-fatal */ } + } + + console.log(`OK:channels=${channelsProcessed},env_keys=${envKeysCopied},files=${filesCopied}`); + if (missing.length > 0) { + console.log(`MISSING:${missing.join(',')}`); + } +} + +main(); diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts new file mode 100644 index 0000000..141b267 --- /dev/null +++ b/setup/migrate-v2/db.ts @@ -0,0 +1,162 @@ +/** + * migrate-v2 step: db + * + * Seed v2.db from v1's registered_groups table. + * Creates agent_groups, messaging_groups, and messaging_group_agents. + * + * Does NOT seed users/user_roles — the /migrate-from-v1 skill handles that. + * + * Idempotent: re-running skips rows that already exist. + * + * Usage: pnpm exec tsx setup/migrate-v2/db.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { createAgentGroup, getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { initDb } from '../../src/db/connection.js'; +import { + createMessagingGroup, + createMessagingGroupAgent, + getMessagingGroupAgentByPair, + getMessagingGroupByPlatform, +} from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { + generateId, + parseJid, + triggerToEngage, + JID_PREFIX_TO_CHANNEL, +} from '../migrate-v1/shared.js'; + +interface V1Group { + jid: string; + name: string; + folder: string; + trigger_pattern: string | null; + requires_trigger: number | null; + is_main: number | null; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/db.ts '); + process.exit(1); + } + + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + if (!fs.existsSync(v1DbPath)) { + console.error(`v1 DB not found: ${v1DbPath}`); + process.exit(1); + } + + // Read v1 groups + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + + // v1 schema varies — channel_name was a late addition. Query only the + // columns we know exist in all v1 installs. + const v1Groups = v1Db + .prepare('SELECT jid, name, folder, trigger_pattern, requires_trigger, is_main FROM registered_groups') + .all() as V1Group[]; + v1Db.close(); + + if (v1Groups.length === 0) { + console.log('SKIPPED:no registered groups in v1'); + process.exit(0); + } + + // Init v2 DB + fs.mkdirSync(path.join(process.cwd(), 'data'), { recursive: true }); + const v2Db = initDb(path.join(DATA_DIR, 'v2.db')); + runMigrations(v2Db); + + let created = 0; + let reused = 0; + let skipped = 0; + const errors: string[] = []; + + for (const g of v1Groups) { + const parsed = parseJid(g.jid); + if (!parsed) { + skipped++; + errors.push(`Could not parse JID: ${g.jid}`); + continue; + } + + const channelType = parsed.channel_type; + const platformId = parsed.raw.startsWith(`${channelType}:`) + ? parsed.raw + : `${channelType}:${parsed.id}`; + const createdAt = new Date().toISOString(); + + try { + // agent_group — one per folder + let ag = getAgentGroupByFolder(g.folder); + if (!ag) { + createAgentGroup({ + id: generateId('ag'), + name: g.name || g.folder, + folder: g.folder, + agent_provider: null, + created_at: createdAt, + }); + ag = getAgentGroupByFolder(g.folder)!; + } + + // messaging_group — one per (channel_type, platform_id) + let mg = getMessagingGroupByPlatform(channelType, platformId); + if (!mg) { + createMessagingGroup({ + id: generateId('mg'), + channel_type: channelType, + platform_id: platformId, + name: g.name || null, + is_group: 1, + unknown_sender_policy: 'public', + created_at: createdAt, + }); + mg = getMessagingGroupByPlatform(channelType, platformId)!; + } + + // messaging_group_agents — wire them + const existing = getMessagingGroupAgentByPair(mg.id, ag.id); + if (!existing) { + const engage = triggerToEngage({ + trigger_pattern: g.trigger_pattern, + requires_trigger: g.requires_trigger, + }); + createMessagingGroupAgent({ + id: generateId('mga'), + messaging_group_id: mg.id, + agent_group_id: ag.id, + engage_mode: engage.engage_mode, + engage_pattern: engage.engage_pattern, + sender_scope: 'all', + ignored_message_policy: 'drop', + session_mode: 'shared', + priority: 0, + created_at: createdAt, + }); + created++; + } else { + reused++; + } + } catch (err) { + skipped++; + errors.push(`${g.folder}: ${err instanceof Error ? err.message : String(err)}`); + } + } + + v2Db.close(); + + console.log(`OK:groups=${v1Groups.length},created=${created},reused=${reused},skipped=${skipped}`); + if (errors.length > 0) { + for (const e of errors) console.log(`ERROR:${e}`); + } +} + +main(); diff --git a/setup/migrate-v2/env.ts b/setup/migrate-v2/env.ts new file mode 100644 index 0000000..5ac52f6 --- /dev/null +++ b/setup/migrate-v2/env.ts @@ -0,0 +1,81 @@ +/** + * migrate-v2 step: env + * + * Copy every key from v1 .env into v2 .env. Never overwrites existing v2 + * keys. Idempotent — re-running skips keys already present. + * + * Usage: pnpm exec tsx setup/migrate-v2/env.ts + */ +import fs from 'fs'; +import path from 'path'; + +function parseEnv(text: string): Map { + const out = new Map(); + for (const raw of text.split('\n')) { + const line = raw.trimEnd(); + if (!line || line.startsWith('#')) continue; + const eq = line.indexOf('='); + if (eq <= 0) continue; + const key = line.slice(0, eq).trim(); + if (!/^[A-Z_][A-Z0-9_]*$/i.test(key)) continue; + out.set(key, line); + } + return out; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/env.ts '); + process.exit(1); + } + + const v1EnvPath = path.join(v1Path, '.env'); + if (!fs.existsSync(v1EnvPath)) { + console.log('SKIPPED:no v1 .env'); + process.exit(0); + } + + const v2EnvPath = path.join(process.cwd(), '.env'); + const v1Lines = parseEnv(fs.readFileSync(v1EnvPath, 'utf-8')); + const v2Text = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; + const v2Lines = parseEnv(v2Text); + + const copied: string[] = []; + const skipped: string[] = []; + const appended: string[] = []; + + const BLOCK_START = '# ── migrated from v1 ──'; + const alreadyMigrated = v2Text.includes(BLOCK_START); + + for (const [key, raw] of v1Lines) { + if (v2Lines.has(key)) { + skipped.push(key); + continue; + } + copied.push(key); + appended.push(raw); + } + + if (appended.length > 0) { + let result = v2Text; + if (result && !result.endsWith('\n')) result += '\n'; + if (!alreadyMigrated) result += `\n${BLOCK_START}\n`; + result += appended.join('\n') + '\n'; + fs.writeFileSync(v2EnvPath, result); + } + + // Sync to data/env/env (container reads from here) + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + try { + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { + // Non-fatal + } + + console.log(`OK:copied=${copied.length},skipped=${skipped.length}`); + if (copied.length > 0) console.log(`COPIED:${copied.join(',')}`); +} + +main(); diff --git a/setup/migrate-v2/groups.ts b/setup/migrate-v2/groups.ts new file mode 100644 index 0000000..beb88be --- /dev/null +++ b/setup/migrate-v2/groups.ts @@ -0,0 +1,120 @@ +/** + * migrate-v2 step: groups + * + * Copy v1 group folders into v2. + * - v1 CLAUDE.md → v2 CLAUDE.local.md (v2 composes CLAUDE.md at spawn) + * - v1 container_config → .v1-container-config.json sidecar + * - All other files copied (no overwrite) + * - Also copies global/ if it exists + * + * Idempotent — does not overwrite files that already exist in v2. + * + * Usage: pnpm exec tsx setup/migrate-v2/groups.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +const SKIP_NAMES = new Set(['CLAUDE.md', 'logs', '.git', '.DS_Store', 'node_modules']); + +/** Copy a directory tree, skipping SKIP_NAMES. Never overwrites existing files. */ +function copyTree(src: string, dst: string): number { + let written = 0; + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(dst, { recursive: true }); + + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + if (SKIP_NAMES.has(entry.name)) continue; + const s = path.join(src, entry.name); + const d = path.join(dst, entry.name); + + if (entry.isDirectory()) { + written += copyTree(s, d); + continue; + } + if (fs.existsSync(d)) continue; + fs.copyFileSync(s, d); + written += 1; + } + return written; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/groups.ts '); + process.exit(1); + } + + const v1GroupsDir = path.join(v1Path, 'groups'); + const v2GroupsDir = path.join(process.cwd(), 'groups'); + + if (!fs.existsSync(v1GroupsDir)) { + console.log('SKIPPED:no v1 groups/ directory'); + process.exit(0); + } + + // Get all folders from v1 DB to know which groups are registered + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + const registeredFolders = new Set(); + if (fs.existsSync(v1DbPath)) { + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + const rows = v1Db + .prepare('SELECT folder, container_config FROM registered_groups') + .all() as Array<{ folder: string; container_config: string | null }>; + const containerConfigs = new Map(); + for (const r of rows) { + registeredFolders.add(r.folder); + containerConfigs.set(r.folder, r.container_config); + } + v1Db.close(); + + // Write container.json from v1 container_config. + // The additionalMounts shape is identical between v1 and v2. + for (const [folder, config] of containerConfigs) { + if (!config) continue; + const v2Folder = path.join(v2GroupsDir, folder); + const containerJson = path.join(v2Folder, 'container.json'); + if (fs.existsSync(containerJson)) continue; + fs.mkdirSync(v2Folder, { recursive: true }); + try { + const parsed = JSON.parse(config) as Record; + fs.writeFileSync(containerJson, JSON.stringify(parsed, null, 2)); + } catch { + // Unparseable config — write as sidecar for the skill to handle + fs.writeFileSync(path.join(v2Folder, '.v1-container-config.json'), config); + } + } + } + + // Copy all v1 group folders (registered + global + any extras) + let foldersCopied = 0; + let claudesMigrated = 0; + let filesCopied = 0; + + for (const entry of fs.readdirSync(v1GroupsDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const folder = entry.name; + const v1Folder = path.join(v1GroupsDir, folder); + const v2Folder = path.join(v2GroupsDir, folder); + + fs.mkdirSync(v2Folder, { recursive: true }); + + // CLAUDE.md → CLAUDE.local.md + const v1Claude = path.join(v1Folder, 'CLAUDE.md'); + const v2Local = path.join(v2Folder, 'CLAUDE.local.md'); + if (fs.existsSync(v1Claude) && !fs.existsSync(v2Local)) { + fs.copyFileSync(v1Claude, v2Local); + claudesMigrated++; + } + + // Copy everything else + filesCopied += copyTree(v1Folder, v2Folder); + foldersCopied++; + } + + console.log(`OK:folders=${foldersCopied},claudes=${claudesMigrated},files=${filesCopied}`); +} + +main(); diff --git a/setup/migrate-v2/select-channels.ts b/setup/migrate-v2/select-channels.ts new file mode 100644 index 0000000..eecf1ab --- /dev/null +++ b/setup/migrate-v2/select-channels.ts @@ -0,0 +1,63 @@ +/** + * migrate-v2: interactive channel selection via clack multiselect. + * + * Writes selected channel names (one per line) to the file path given as + * the first argument. Clack renders to the terminal normally. + * + * If NANOCLAW_CHANNELS env var is set (comma-separated names), skips the + * prompt and writes those directly. + * + * Usage: pnpm exec tsx setup/migrate-v2/select-channels.ts + */ +import fs from 'fs'; + +import * as p from '@clack/prompts'; + +const CHANNELS = [ + { value: 'telegram', label: 'Telegram' }, + { value: 'discord', label: 'Discord' }, + { value: 'slack', label: 'Slack' }, + { value: 'whatsapp', label: 'WhatsApp' }, + { value: 'teams', label: 'Microsoft Teams' }, + { value: 'matrix', label: 'Matrix' }, + { value: 'imessage', label: 'iMessage' }, + { value: 'webex', label: 'Webex' }, + { value: 'gchat', label: 'Google Chat' }, + { value: 'resend', label: 'Resend (email)' }, + { value: 'github', label: 'GitHub' }, + { value: 'linear', label: 'Linear' }, + { value: 'whatsapp-cloud', label: 'WhatsApp Cloud API' }, +]; + +const VALID_NAMES = new Set(CHANNELS.map((c) => c.value)); + +async function main(): Promise { + const outFile = process.argv[2]; + if (!outFile) { + console.error('Usage: tsx setup/migrate-v2/select-channels.ts '); + process.exit(1); + } + + // Non-interactive: NANOCLAW_CHANNELS="telegram,discord" + const envChannels = process.env.NANOCLAW_CHANNELS?.trim(); + if (envChannels) { + const names = envChannels.split(',').map((s) => s.trim()).filter((s) => VALID_NAMES.has(s)); + fs.writeFileSync(outFile, names.join('\n') + '\n'); + return; + } + + const selected = await p.multiselect({ + message: 'Which channels do you want to set up?', + options: CHANNELS, + required: false, + }); + + if (p.isCancel(selected)) { + fs.writeFileSync(outFile, ''); + return; + } + + fs.writeFileSync(outFile, (selected as string[]).join('\n') + '\n'); +} + +main(); diff --git a/setup/migrate-v2/sessions.ts b/setup/migrate-v2/sessions.ts new file mode 100644 index 0000000..0299dec --- /dev/null +++ b/setup/migrate-v2/sessions.ts @@ -0,0 +1,181 @@ +/** + * migrate-v2 step: sessions + * + * For each v1 session folder, create a proper v2 session: + * 1. Create a sessions row in v2.db (via resolveSession) + * 2. Initialize the session folder (inbound.db, outbound.db, outbox/) + * 3. Write session routing so the container knows where to reply + * 4. Copy v1 .claude/ state into v2's .claude-shared/ directory + * + * v1: data/sessions//.claude/ (settings, conversation history, skills) + * v2: data/v2-sessions//.claude-shared/ + session folder + * + * v1's agent-runner-src/ is NOT copied — v2 uses a completely different + * Bun-based agent-runner. + * + * Idempotent — reuses existing sessions, does not overwrite files. + * + * Usage: pnpm exec tsx setup/migrate-v2/sessions.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb, closeDb } from '../../src/db/connection.js'; +import { getAllAgentGroups } from '../../src/db/agent-groups.js'; +import { getMessagingGroupsByAgentGroup } from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { + resolveSession, + writeSessionRouting, + outboundDbPath, +} from '../../src/session-manager.js'; + +const SKIP_NAMES = new Set(['.DS_Store']); + +/** Recursively copy, never overwriting existing files. */ +function copyTree(src: string, dst: string): number { + let written = 0; + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(dst, { recursive: true }); + + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + if (SKIP_NAMES.has(entry.name)) continue; + const s = path.join(src, entry.name); + const d = path.join(dst, entry.name); + + if (entry.isDirectory()) { + written += copyTree(s, d); + continue; + } + if (fs.existsSync(d)) continue; + fs.copyFileSync(s, d); + written += 1; + } + return written; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/sessions.ts '); + process.exit(1); + } + + const v1SessionsDir = path.join(v1Path, 'data', 'sessions'); + if (!fs.existsSync(v1SessionsDir)) { + console.log('SKIPPED:no v1 data/sessions/ directory'); + process.exit(0); + } + + // Init v2 central DB + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('v2.db not found — run db step first'); + process.exit(1); + } + + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + const agentGroups = getAllAgentGroups(); + const folderToAg = new Map(); + for (const ag of agentGroups) { + folderToAg.set(ag.folder, ag); + } + + let sessionsCreated = 0; + let sessionsReused = 0; + let sessionsSkipped = 0; + let filesCopied = 0; + + for (const entry of fs.readdirSync(v1SessionsDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const folder = entry.name; + + const ag = folderToAg.get(folder); + if (!ag) { + sessionsSkipped++; + continue; + } + + // Find the messaging groups wired to this agent group + const messagingGroups = getMessagingGroupsByAgentGroup(ag.id); + if (messagingGroups.length === 0) { + sessionsSkipped++; + continue; + } + + // Create a session for each messaging group (v1 had one session per + // folder, v2 has one per agent_group + messaging_group pair) + for (const mg of messagingGroups) { + const { session, created } = resolveSession(ag.id, mg.id, null, 'shared'); + + if (created) { + // Write routing so the container knows where to reply + writeSessionRouting(ag.id, session.id); + sessionsCreated++; + } else { + sessionsReused++; + } + } + + // Copy v1 .claude/ state into v2's .claude-shared/ directory + // This is per-agent-group, shared across all sessions for that group + const v1ClaudeDir = path.join(v1SessionsDir, folder, '.claude'); + if (fs.existsSync(v1ClaudeDir)) { + const v2ClaudeDir = path.join(DATA_DIR, 'v2-sessions', ag.id, '.claude-shared'); + filesCopied += copyTree(v1ClaudeDir, v2ClaudeDir); + + // v1 containers worked in /workspace/group, v2 works in /workspace/agent. + // Claude Code stores sessions under projects//. Copy the v1 + // project dir to the v2 path so Claude Code finds the conversation history. + const projectsDir = path.join(v2ClaudeDir, 'projects'); + const v1ProjectDir = path.join(projectsDir, '-workspace-group'); + const v2ProjectDir = path.join(projectsDir, '-workspace-agent'); + if (fs.existsSync(v1ProjectDir) && !fs.existsSync(v2ProjectDir)) { + filesCopied += copyTree(v1ProjectDir, v2ProjectDir); + } + + // Write the v1 Claude Code session ID as the continuation in outbound.db + // so the agent-runner resumes the exact same conversation. + // The session ID is the JSONL filename (without extension) under the + // project dir. + const sourceDir = fs.existsSync(v2ProjectDir) ? v2ProjectDir : v1ProjectDir; + if (fs.existsSync(sourceDir)) { + const jsonlFiles = fs.readdirSync(sourceDir).filter((f) => f.endsWith('.jsonl')); + if (jsonlFiles.length > 0) { + // Use the most recent JSONL file (by mtime from v1) + const v1SessionId = jsonlFiles + .map((f) => ({ + name: f.replace('.jsonl', ''), + mtime: fs.statSync(path.join(sourceDir, f)).mtimeMs, + })) + .sort((a, b) => b.mtime - a.mtime)[0].name; + + // Write into each v2 session's outbound.db for this agent group + const sessions = getMessagingGroupsByAgentGroup(ag.id); + for (const mg of sessions) { + const { session } = resolveSession(ag.id, mg.id, null, 'shared'); + const obPath = outboundDbPath(ag.id, session.id); + if (fs.existsSync(obPath)) { + const ob = new Database(obPath); + ob.prepare( + "INSERT OR REPLACE INTO session_state (key, value, updated_at) VALUES ('continuation:claude', ?, ?)", + ).run(v1SessionId, new Date().toISOString()); + ob.close(); + } + } + } + } + } + } + + closeDb(); + + console.log(`OK:created=${sessionsCreated},reused=${sessionsReused},skipped=${sessionsSkipped},files=${filesCopied}`); +} + +main(); diff --git a/setup/migrate-v2/switchover-prompt.ts b/setup/migrate-v2/switchover-prompt.ts new file mode 100644 index 0000000..996b302 --- /dev/null +++ b/setup/migrate-v2/switchover-prompt.ts @@ -0,0 +1,53 @@ +/** + * migrate-v2: service switchover prompts. + * + * Writes a single word to the output file: + * --offer-switch → "switch" | "skip" + * --keep-or-revert → "keep" | "revert" + * + * Clack renders to the terminal normally. + * + * Usage: pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --offer-switch + */ +import fs from 'fs'; + +import * as p from '@clack/prompts'; + +async function main(): Promise { + const mode = process.argv[2]; + const outFile = process.argv[3]; + + if (!outFile) { + console.error('Usage: tsx setup/migrate-v2/switchover-prompt.ts <--offer-switch|--keep-or-revert> '); + process.exit(1); + } + + if (mode === '--offer-switch') { + const answer = await p.select({ + message: 'Want to stop the v1 service and start v2 so you can test?', + options: [ + { value: 'switch', label: 'Yes, switch to v2 now', hint: 'you can switch back after' }, + { value: 'skip', label: 'No, skip for now', hint: 'start v2 manually later' }, + ], + }); + fs.writeFileSync(outFile, p.isCancel(answer) ? 'skip' : String(answer)); + return; + } + + if (mode === '--keep-or-revert') { + const answer = await p.select({ + message: 'Keep v2 running, or switch back to v1?', + options: [ + { value: 'keep', label: 'Keep v2', hint: 'v1 stays stopped' }, + { value: 'revert', label: 'Switch back to v1', hint: 'stop v2, restart v1' }, + ], + }); + fs.writeFileSync(outFile, p.isCancel(answer) ? 'revert' : String(answer)); + return; + } + + console.error('Usage: --offer-switch | --keep-or-revert'); + process.exit(1); +} + +main(); diff --git a/setup/migrate-v2/tasks.ts b/setup/migrate-v2/tasks.ts new file mode 100644 index 0000000..9ba570a --- /dev/null +++ b/setup/migrate-v2/tasks.ts @@ -0,0 +1,158 @@ +/** + * migrate-v2 step: tasks + * + * Port v1 scheduled_tasks into v2 session inbound DBs. + * + * v1: scheduled_tasks table (schedule_type, schedule_value, next_run) + * v2: messages_in rows with kind='task' in per-session inbound.db + * + * Requires: db step must have run first (agent_groups + messaging_groups seeded). + * + * Usage: pnpm exec tsx setup/migrate-v2/tasks.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb, closeDb } from '../../src/db/connection.js'; +import { getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { insertTask } from '../../src/modules/scheduling/db.js'; +import { openInboundDb, resolveSession } from '../../src/session-manager.js'; +import { parseJid, v2PlatformId } from '../migrate-v1/shared.js'; + +interface V1Task { + id: string; + group_folder: string; + chat_jid: string; + prompt: string; + schedule_type: string; + schedule_value: string; + next_run: string | null; + status: string; + context_mode: string | null; + script: string | null; +} + +function toCron(t: V1Task): { processAfter: string; recurrence: string | null } | null { + const now = new Date().toISOString(); + + if (t.schedule_type === 'cron') { + const fields = t.schedule_value.trim().split(/\s+/).length; + if (fields < 5 || fields > 6) return null; + return { processAfter: t.next_run || now, recurrence: t.schedule_value.trim() }; + } + + if (t.schedule_type === 'interval') { + const m = /^(\d+)([smhd])$/.exec(t.schedule_value.trim()); + if (!m) return null; + const n = parseInt(m[1], 10); + const unit = m[2]; + if (!n || n < 1) return null; + let cron: string | null = null; + if (unit === 'm' && n < 60) cron = `*/${n} * * * *`; + else if (unit === 'h' && n < 24) cron = `0 */${n} * * *`; + else if (unit === 'd' && n < 28) cron = `0 0 */${n} * *`; + if (!cron) return null; + return { processAfter: t.next_run || now, recurrence: cron }; + } + + if (t.schedule_type === 'once' || t.schedule_type === 'at') { + return { processAfter: t.next_run || t.schedule_value || now, recurrence: null }; + } + + return null; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/tasks.ts '); + process.exit(1); + } + + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + if (!fs.existsSync(v1DbPath)) { + console.log('SKIPPED:no v1 DB'); + process.exit(0); + } + + // Read v1 tasks + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + const allTasks = v1Db.prepare('SELECT * FROM scheduled_tasks').all() as V1Task[]; + v1Db.close(); + + const activeTasks = allTasks.filter((t) => t.status === 'active'); + if (activeTasks.length === 0) { + console.log('SKIPPED:no active tasks'); + process.exit(0); + } + + // Init v2 central DB + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('v2.db not found — run db step first'); + process.exit(1); + } + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + let migrated = 0; + let skipped = 0; + let failed = 0; + + for (const t of activeTasks) { + try { + const ag = getAgentGroupByFolder(t.group_folder); + if (!ag) { skipped++; continue; } + + const parsed = parseJid(t.chat_jid); + if (!parsed) { skipped++; continue; } + + const platformId = v2PlatformId(parsed.channel_type, t.chat_jid); + const mg = getMessagingGroupByPlatform(parsed.channel_type, platformId); + if (!mg) { skipped++; continue; } + + const scheduling = toCron(t); + if (!scheduling) { skipped++; continue; } + + const { session } = resolveSession(ag.id, mg.id, null, 'shared'); + const inboxDb = openInboundDb(ag.id, session.id); + try { + // Idempotence check + const existing = inboxDb + .prepare("SELECT id FROM messages_in WHERE id = ? AND kind = 'task'") + .get(t.id) as { id: string } | undefined; + if (existing) { skipped++; continue; } + + insertTask(inboxDb, { + id: t.id, + processAfter: scheduling.processAfter, + recurrence: scheduling.recurrence, + platformId, + channelType: parsed.channel_type, + threadId: null, + content: JSON.stringify({ + prompt: t.prompt, + script: t.script ?? null, + migrated_from_v1: { original_id: t.id, context_mode: t.context_mode ?? null }, + }), + }); + migrated++; + } finally { + inboxDb.close(); + } + } catch (err) { + failed++; + console.error(`TASK_ERROR:${t.id}:${err instanceof Error ? err.message : String(err)}`); + } + } + + closeDb(); + console.log(`OK:active=${activeTasks.length},migrated=${migrated},skipped=${skipped},failed=${failed}`); +} + +main(); From 67eb85d818f641cdfb5898eafe7c4d3ae1f4084b Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:20:06 +0000 Subject: [PATCH 06/23] chore: remove old setup-embedded migration steps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The old migration flow (detect → validate → db → groups → env → channel-auth → channels → tasks) ran inside `bash nanoclaw.sh` via setup/auto.ts. Replaced by the standalone `bash migrate-v2.sh` flow. Deleted: - setup/migrate-v1.ts (orchestrator) - setup/migrate-v1/{detect,validate,db,env,groups,channel-auth,channels,tasks}.ts Kept: - setup/migrate-v1/shared.ts (used by new migrate-v2/ steps) Co-Authored-By: Claude Opus 4.6 (1M context) --- setup/auto.ts | 14 +- setup/index.ts | 9 +- setup/migrate-v1.ts | 257 ------------------------- setup/migrate-v1/channel-auth.ts | 262 ------------------------- setup/migrate-v1/channels.ts | 172 ----------------- setup/migrate-v1/db.ts | 321 ------------------------------- setup/migrate-v1/detect.ts | 107 ----------- setup/migrate-v1/env.ts | 135 ------------- setup/migrate-v1/groups.ts | 230 ---------------------- setup/migrate-v1/tasks.ts | 307 ----------------------------- setup/migrate-v1/validate.ts | 213 -------------------- 11 files changed, 4 insertions(+), 2023 deletions(-) delete mode 100644 setup/migrate-v1.ts delete mode 100644 setup/migrate-v1/channel-auth.ts delete mode 100644 setup/migrate-v1/channels.ts delete mode 100644 setup/migrate-v1/db.ts delete mode 100644 setup/migrate-v1/detect.ts delete mode 100644 setup/migrate-v1/env.ts delete mode 100644 setup/migrate-v1/groups.ts delete mode 100644 setup/migrate-v1/tasks.ts delete mode 100644 setup/migrate-v1/validate.ts diff --git a/setup/auto.ts b/setup/auto.ts index ab0cbb4..147eed4 100644 --- a/setup/auto.ts +++ b/setup/auto.ts @@ -14,10 +14,8 @@ * "Terminal Agent". * NANOCLAW_SKIP comma-separated step names to skip * (environment|container|onecli|auth|mounts| - * service|cli-agent|timezone|migration|channel| + * service|cli-agent|timezone|channel| * verify|first-chat) - * NANOCLAW_V1_PATH explicit path to a v1 install to migrate - * from (default: scan common locations) * * Timezone is auto-detected after the CLI agent step. UTC resolves are * confirmed with the user, and free-text replies fall through to a @@ -41,7 +39,6 @@ import { runWhatsAppChannel } from './channels/whatsapp.js'; import { pingCliAgent, type PingResult } from './lib/agent-ping.js'; import { brightSelect } from './lib/bright-select.js'; import { offerClaudeAssist } from './lib/claude-assist.js'; -import { runMigrateV1 } from './migrate-v1.js'; import { applyToEnv, parseFlags, @@ -437,13 +434,8 @@ async function main(): Promise { await runTimezoneStep(); } - if (!skip.has('migration')) { - // Runs silently when there's no v1 install; otherwise orchestrates the - // detect → validate → db → groups → env → channel-auth → channels → - // tasks sub-steps and writes logs/setup-migration/handoff.json for the - // /migrate-from-v1 skill to pick up. - await runMigrateV1(); - } + // v1 → v2 migration is handled by `bash migrate-v2.sh`, not the setup flow. + // Users migrating from v1 run that script before (or instead of) setup. let channelChoice: ChannelChoice = 'skip'; diff --git a/setup/index.ts b/setup/index.ts index a6c66ec..ee02e45 100644 --- a/setup/index.ts +++ b/setup/index.ts @@ -14,6 +14,7 @@ const STEPS: Record< environment: () => import('./environment.js'), container: () => import('./container.js'), register: () => import('./register.js'), + 'pair-telegram': () => import('./pair-telegram.js'), groups: () => import('./groups.js'), 'whatsapp-auth': () => import('./whatsapp-auth.js'), 'signal-auth': () => import('./signal-auth.js'), @@ -23,14 +24,6 @@ const STEPS: Record< onecli: () => import('./onecli.js'), auth: () => import('./auth.js'), 'cli-agent': () => import('./cli-agent.js'), - 'migrate-detect': () => import('./migrate-v1/detect.js'), - 'migrate-validate': () => import('./migrate-v1/validate.js'), - 'migrate-db': () => import('./migrate-v1/db.js'), - 'migrate-groups': () => import('./migrate-v1/groups.js'), - 'migrate-env': () => import('./migrate-v1/env.js'), - 'migrate-channel-auth': () => import('./migrate-v1/channel-auth.js'), - 'migrate-channels': () => import('./migrate-v1/channels.js'), - 'migrate-tasks': () => import('./migrate-v1/tasks.js'), }; async function main(): Promise { diff --git a/setup/migrate-v1.ts b/setup/migrate-v1.ts deleted file mode 100644 index a3c1883..0000000 --- a/setup/migrate-v1.ts +++ /dev/null @@ -1,257 +0,0 @@ -/** - * v1 → v2 migration orchestrator. Called from setup/auto.ts after the - * timezone step and before the channel step. - * - * Silent happy path: if no v1 install is found, we emit one "skipped" step - * and return. Users on a fresh v2 install never see anything. - * - * When v1 IS found: detect → [confirm] → group-selection prompt → validate - * → db → groups → env → channel-auth → channels → tasks → handoff. - * Every sub-step is a separate entry in the progression log; failures never - * abort the chain (the handoff file records them for the skill to finish). - * - * After everything runs, a one-line note points the user at the - * `/migrate-from-v1` skill. - */ -import fs from 'fs'; -import path from 'path'; - -import * as p from '@clack/prompts'; -import Database from 'better-sqlite3'; -import k from 'kleur'; - -import { ensureAnswer, runQuietStep } from './lib/runner.js'; -import { wrapForGutter } from './lib/theme.js'; -import * as setupLog from './logs.js'; -import { - HANDOFF_PATH, - MIGRATION_DIR, - inferChannelType, - readHandoff, - v1PathsFor, - writeHandoff, -} from './migrate-v1/shared.js'; - -/** - * Count groups in v1's registered_groups, split by whether the channel_type - * can be inferred. Uses the same `inferChannelType` logic as migrate-db so - * the displayed count matches what will actually get seeded. Open-and-close - * because this runs in the orchestrator before migrate-db's child process. - */ -function countV1Groups(v1Root: string): { total: number; wired: number } { - const dbPath = v1PathsFor(v1Root).db; - try { - const db = new Database(dbPath, { readonly: true, fileMustExist: true }); - const rows = db - .prepare('SELECT jid, channel_name FROM registered_groups') - .all() as Array<{ jid: string; channel_name: string | null }>; - db.close(); - let wired = 0; - for (const r of rows) { - if (inferChannelType(r.jid, r.channel_name)) wired++; - } - return { total: rows.length, wired }; - } catch { - return { total: 0, wired: 0 }; - } -} - -async function askGroupSelection(counts: { total: number; wired: number }): Promise<'all' | 'wired-only' | 'cancel'> { - // Non-interactive escape hatch for CI / re-runs / scripted migrations. - // NANOCLAW_MIGRATE_SELECTION = 'all' | 'wired-only' | 'cancel'. - const envChoice = process.env.NANOCLAW_MIGRATE_SELECTION?.trim(); - if (envChoice === 'all' || envChoice === 'wired-only' || envChoice === 'cancel') { - setupLog.userInput('migrate_selection', `${envChoice} (from NANOCLAW_MIGRATE_SELECTION)`); - return envChoice; - } - // Most v1 installs accumulated many orphan folders. Default the user to - // wired-only (the ones we can actually route) — explicit opt-in for "all". - const choice = ensureAnswer( - await p.select({ - message: `Found ${counts.total} v1 group folders (${counts.wired} wired to a channel). Which to bring over?`, - options: [ - { - value: 'wired-only', - label: `Only the ${counts.wired} wired ones`, - hint: 'recommended — skips orphans', - }, - { - value: 'all', - label: `All ${counts.total} folders`, - hint: 'brings dead/orphan folders over too', - }, - { - value: 'cancel', - label: 'Skip migration', - hint: "I'll migrate later", - }, - ], - }), - ) as 'all' | 'wired-only' | 'cancel'; - setupLog.userInput('migrate_selection', choice); - return choice; -} - -/** - * Finalize the handoff record after every sub-step has run. Computes an - * overall status from per-step statuses: anything `failed` → partial; - * anything `partial` → partial; else success. - */ -function finalizeHandoff(): 'success' | 'partial' | 'failed' { - const h = readHandoff(); - const statuses = Object.values(h.steps).map((s) => s?.status); - const anyFailed = statuses.includes('failed'); - const anyPartial = statuses.includes('partial'); - const overall: 'success' | 'partial' | 'failed' = anyFailed - ? 'partial' // DB or files may have landed; the skill can pick up the rest - : anyPartial - ? 'partial' - : 'success'; - h.overall_status = overall; - writeHandoff(h); - return overall; -} - -function printHandoffNote(overall: 'success' | 'partial' | 'failed'): void { - const relHandoff = path.relative(process.cwd(), HANDOFF_PATH); - const lines: string[] = []; - if (overall === 'success') { - lines.push( - wrapForGutter( - 'Your v1 install has been migrated. Run `/migrate-from-v1` in Claude next — it will seed your owner account and help port any custom code you had.', - 4, - ), - ); - } else { - lines.push( - wrapForGutter( - 'Migration finished with some items for a human. Run `/migrate-from-v1` in Claude — it will read the handoff, finish the unfinished steps, and walk through custom code.', - 4, - ), - ); - } - lines.push(''); - lines.push(k.dim(` Handoff: ${relHandoff}`)); - lines.push(k.dim(` Full log: ${setupLog.progressLogPath}`)); - lines.push(k.dim(` Raw logs: ${setupLog.stepsDir}/`)); - p.note(lines.join('\n'), 'Migration handoff'); -} - -export async function runMigrateV1(): Promise<'proceeded' | 'skipped' | 'cancelled'> { - // 0. Ensure migration log dir exists before any sub-step writes to it. - fs.mkdirSync(MIGRATION_DIR, { recursive: true }); - - // 1. Detect. If nothing obvious, give the user one subtle chance to point - // us at a non-standard path — then accept silently. - const detect = await runQuietStep('migrate-detect', { - running: 'Checking for a previous NanoClaw install…', - done: 'Found a previous install.', - skipped: 'No previous install to migrate.', - }); - - const v1Found = detect.ok && detect.terminal?.fields.STATUS === 'success'; - - if (!v1Found) { - // Silent skip — the 99% case is a fresh install with no v1 anywhere. - // Prompting for a custom path on every fresh run is UX noise. Users - // with a v1 at a non-standard location use `NANOCLAW_V1_PATH= - // bash nanoclaw.sh` (documented in README + setup/auto.ts header). - return 'skipped'; - } - - // 2. Ask the user which groups to bring over. - const h = readHandoff(); - if (!h.v1_path) { - // Shouldn't happen — detect set it if v1Found. Guard anyway. - return 'skipped'; - } - - // Experimental warning — fires only when a v1 install is found, so stock - // v2 users (no v1 to migrate) never see it. Not a blocker; the user can - // still proceed. Skip when NANOCLAW_MIGRATE_SELECTION is set (scripted / - // CI runs have already accepted the risk by defining their selection). - if (!process.env.NANOCLAW_MIGRATE_SELECTION) { - p.log.warn( - wrapForGutter( - 'v1 → v2 migration is experimental. Back up your v2 state (data/v2.db, groups/) before continuing. Not recommended for high-stakes production installs — it does a best-effort port and a human still has to finish via /migrate-from-v1.', - 4, - ), - ); - } - - const counts = countV1Groups(h.v1_path); - const selection = await askGroupSelection(counts); - if (selection === 'cancel') { - // Mark the handoff so the skill can still see what would have happened. - const ho = readHandoff(); - ho.overall_status = 'skipped'; - writeHandoff(ho); - return 'cancelled'; - } - - // 3. Validate — if it fails, subsequent steps will short-circuit the - // DB-dependent parts. Groups + env still run. - await runQuietStep('migrate-validate', { - running: "Checking the v1 database's shape…", - done: 'v1 database looks good.', - failed: "v1 database didn't match what I expected.", - skipped: 'Skipped database validation.', - }); - - // 4. DB seeding — parameterized by the user's selection. - await runQuietStep( - 'migrate-db', - { - running: 'Seeding v2 agents and channels from v1…', - done: 'Seeded v2 database.', - skipped: 'Skipped database seeding.', - failed: "Couldn't seed the v2 database.", - }, - ['--selection', selection], - ); - - // 5. Group folders. - await runQuietStep('migrate-groups', { - running: 'Copying group folders…', - done: 'Group folders copied.', - skipped: 'Skipped group-folder copy.', - failed: "Couldn't copy some group folders.", - }); - - // 6. Env keys. - await runQuietStep('migrate-env', { - running: 'Merging v1 .env into v2 .env…', - done: 'Env keys migrated.', - skipped: 'No env keys to migrate.', - failed: "Couldn't merge .env.", - }); - - // 7. Non-env channel auth (Baileys keystore, matrix state, etc.). - await runQuietStep('migrate-channel-auth', { - running: 'Copying channel auth files…', - done: 'Channel auth copied.', - skipped: 'No channel auth to copy.', - failed: 'Some channel auth files need attention.', - }); - - // 8. Install v2 channel adapters for the detected channels. - await runQuietStep('migrate-channels', { - running: 'Installing v2 channel adapters…', - done: 'Channel adapters installed.', - skipped: 'No channels to install.', - failed: 'Some channel adapters need attention.', - }); - - // 9. Scheduled tasks. - await runQuietStep('migrate-tasks', { - running: 'Porting scheduled tasks…', - done: 'Scheduled tasks ported.', - skipped: 'No scheduled tasks to port.', - failed: 'Some scheduled tasks need attention.', - }); - - // 10. Finalize + hand off. - const overall = finalizeHandoff(); - printHandoffNote(overall); - return 'proceeded'; -} diff --git a/setup/migrate-v1/channel-auth.ts b/setup/migrate-v1/channel-auth.ts deleted file mode 100644 index 8c8a415..0000000 --- a/setup/migrate-v1/channel-auth.ts +++ /dev/null @@ -1,262 +0,0 @@ -/** - * Step: migrate-channel-auth - * - * For each channel detected in migrate-db, copy non-.env auth state from v1 - * to the matching v2 location. Env keys are handled by migrate-env (this - * step reads the registry to confirm they made it over, but doesn't rewrite - * them). Files are copied from the first matching candidate path in the - * registry — missing paths are recorded so the skill can prompt the user. - * - * Destination uses the same relative path on v2 (e.g. v1 has - * `data/sessions/baileys/` → v2 gets `data/sessions/baileys/`). If v2 already - * has a different file/dir at that path, we skip and flag it — never clobber. - */ -import fs from 'fs'; -import path from 'path'; - -import { emitStatus } from '../status.js'; -import { - CHANNEL_AUTH_REGISTRY, - autoResolveV2Keys, - readHandoff, - recordStep, - v1PathsFor, - writeHandoff, -} from './shared.js'; - -/** - * Copy file or directory tree from src to dst. `force: false` means existing - * files on the v2 side are never clobbered — important because we'd otherwise - * overwrite auth state the user may have set up on v2 directly. Returns a - * rough count of files copied (post-hoc walk of the destination). - */ -function copyRecursive(src: string, dst: string): number { - if (!fs.existsSync(src)) return 0; - fs.mkdirSync(path.dirname(dst), { recursive: true }); - fs.cpSync(src, dst, { recursive: true, force: false, errorOnExist: false }); - return countFilesUnder(dst); -} - -function countFilesUnder(p: string): number { - if (!fs.existsSync(p)) return 0; - if (fs.statSync(p).isFile()) return 1; - let n = 0; - for (const entry of fs.readdirSync(p, { withFileTypes: true })) { - n += countFilesUnder(path.join(p, entry.name)); - } - return n; -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-channel-auth', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_CHANNEL_AUTH', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const channels = h.detected_channels; - if (channels.length === 0) { - recordStep('migrate-channel-auth', { - status: 'skipped', - fields: { REASON: 'no-channels-detected' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_CHANNEL_AUTH', { STATUS: 'skipped', REASON: 'no_channels' }); - return; - } - - const v1Paths = v1PathsFor(h.v1_path); - const v1Env = fs.existsSync(v1Paths.env) ? fs.readFileSync(v1Paths.env, 'utf-8') : ''; - const v1EnvKeys = new Set( - v1Env - .split('\n') - .map((line) => line.trim()) - .filter((line) => line && !line.startsWith('#')) - .map((line) => line.split('=')[0].trim()) - .filter(Boolean), - ); - - const results: typeof h.channel_auth = []; - const followups: string[] = []; - let anyMissingRequired = false; - - for (const ch of channels) { - const spec = CHANNEL_AUTH_REGISTRY[ch.channel_type]; - if (!spec) { - // Unknown channel — give the skill enough context to drive a useful - // interview instead of a generic "we don't know." Scan v1's .env for - // keys that look related (substring match on channel name + common - // suffixes) and list v1 state directories the user should check. - const haystack = ch.channel_type.toLowerCase(); - const candidateEnvKeys = [...v1EnvKeys].filter((k) => { - const lk = k.toLowerCase(); - return ( - lk.includes(haystack) || - (haystack.length >= 3 && lk.includes(haystack.slice(0, 3))) - ); - }); - const v1DataDirs = ['data', 'store', 'data/sessions'] - .map((d) => path.join(h.v1_path, d)) - .filter((p) => fs.existsSync(p)); - - results.push({ - channel_type: ch.channel_type, - env_keys_copied: [], - files_copied: [], - files_missing: [], - notes: `Unknown channel (not in CHANNEL_AUTH_REGISTRY). Inferred via ${ch.source}. Candidate v1 env keys: ${candidateEnvKeys.join(', ') || 'none found'}. Check v1 dirs: ${v1DataDirs.join(', ') || '(none)'}.`, - }); - followups.push( - `Channel "${ch.channel_type}" (${ch.group_count} group(s), inferred via ${ch.source}) is not in the auth registry. ` + - `Candidate v1 env keys that may belong to it: ${candidateEnvKeys.length > 0 ? candidateEnvKeys.join(', ') : '(none obvious)'}. ` + - `Check v1 for on-disk auth state under ${v1DataDirs.join(', ') || '(no standard dirs found)'}. ` + - `The skill should interview the user, then add a registry entry to setup/migrate-v1/shared.ts for future migrations.`, - ); - continue; - } - - const envKeysPresentInV1 = spec.v1EnvKeys.filter((key) => v1EnvKeys.has(key)); - - // Check v2's .env for required keys the v2 adapter needs to boot. v1 - // may not have had all of them (e.g. v1's Discord used discord.js - // directly and never stored DISCORD_PUBLIC_KEY which v2's Chat SDK - // requires). Try to auto-resolve the gap by calling the channel's API - // with the v1 credential; fall through to a followup for anything we - // can't resolve. - const v2EnvPath = path.join(process.cwd(), '.env'); - const v1EnvMap = new Map(); - for (const line of v1Env.split('\n')) { - const trimmed = line.trim(); - if (!trimmed || trimmed.startsWith('#')) continue; - const eq = trimmed.indexOf('='); - if (eq <= 0) continue; - v1EnvMap.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); - } - - // Also let the resolver reach into v2's .env (migrate-env already merged - // v1 keys into v2). Either source is fine for derivation inputs. - const v2EnvPre = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; - const v2EnvPreMap = new Map(); - for (const line of v2EnvPre.split('\n')) { - const trimmed = line.trim(); - if (!trimmed || trimmed.startsWith('#')) continue; - const eq = trimmed.indexOf('='); - if (eq <= 0) continue; - v2EnvPreMap.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); - } - - const resolved = await autoResolveV2Keys( - ch.channel_type, - (key) => v1EnvMap.get(key) ?? v2EnvPreMap.get(key), - ); - const resolvedKeys = Object.keys(resolved); - if (resolvedKeys.length > 0) { - // Append to v2 .env (never overwriting existing values) + sync the - // container-side copy. Log keys, never values. - let text = v2EnvPre; - if (text && !text.endsWith('\n')) text += '\n'; - for (const [key, value] of Object.entries(resolved)) { - if (v2EnvPreMap.has(key)) continue; - text += `${key}=${value}\n`; - } - fs.writeFileSync(v2EnvPath, text); - try { - const containerEnvDir = path.join(process.cwd(), 'data', 'env'); - fs.mkdirSync(containerEnvDir, { recursive: true }); - fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); - } catch { - // Best-effort; service restart rehydrates it if needed. - } - } - - // Re-read v2 .env after possible resolution to compute the real gap. - const v2Env = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; - const v2EnvKeys = new Set( - v2Env - .split('\n') - .map((l) => l.trim()) - .filter((l) => l && !l.startsWith('#')) - .map((l) => l.split('=')[0].trim()) - .filter(Boolean), - ); - const missingRequired = spec.requiredV2Keys.filter((r) => !v2EnvKeys.has(r.key)); - if (missingRequired.length > 0) { - anyMissingRequired = true; - followups.push( - `Channel "${ch.channel_type}" is missing required v2 keys in .env: ${missingRequired - .map((r) => `${r.key} (${r.where})`) - .join('; ')}. The v2 adapter won't boot until these are set.`, - ); - } - - const filesCopied: string[] = []; - const filesMissing: string[] = []; - - for (const relPath of spec.candidatePaths) { - const src = path.join(h.v1_path, relPath); - if (!fs.existsSync(src)) continue; - - const dst = path.join(process.cwd(), relPath); - if (fs.existsSync(dst)) { - followups.push( - `Channel "${ch.channel_type}": v2 already has ${relPath} — left untouched. Reconcile manually if needed.`, - ); - filesMissing.push(`${relPath} (already exists in v2)`); - continue; - } - - try { - const count = copyRecursive(src, dst); - filesCopied.push(`${relPath} (${count} files)`); - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - filesMissing.push(`${relPath} (copy failed: ${message})`); - followups.push(`Channel "${ch.channel_type}": failed to copy ${relPath} — ${message}`); - } - } - - if (spec.candidatePaths.length > 0 && filesCopied.length === 0) { - filesMissing.push(`(no candidate paths existed under ${h.v1_path})`); - } - - results.push({ - channel_type: ch.channel_type, - env_keys_copied: [...envKeysPresentInV1, ...resolvedKeys.map((k) => `${k} (auto-resolved)`)], - files_copied: filesCopied, - files_missing: filesMissing, - notes: spec.note ?? '', - }); - } - - const handoffAfter = readHandoff(); - handoffAfter.channel_auth = results; - handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; - writeHandoff(handoffAfter); - - const anyFileMissing = results.some((r) => r.files_missing.length > 0); - const anyPartial = anyFileMissing || anyMissingRequired; - recordStep('migrate-channel-auth', { - status: anyPartial ? 'partial' : 'success', - fields: { - CHANNELS: channels.map((c) => c.channel_type).join(','), - FILES_COPIED: results.reduce((sum, r) => sum + r.files_copied.length, 0), - FILES_MISSING: results.reduce((sum, r) => sum + r.files_missing.length, 0), - }, - notes: followups, - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_CHANNEL_AUTH', { - STATUS: anyPartial ? 'partial' : 'success', - CHANNELS: channels.map((c) => c.channel_type).join(','), - FILES_COPIED: String(results.reduce((sum, r) => sum + r.files_copied.length, 0)), - FILES_MISSING: String(results.reduce((sum, r) => sum + r.files_missing.length, 0)), - }); -} diff --git a/setup/migrate-v1/channels.ts b/setup/migrate-v1/channels.ts deleted file mode 100644 index 89df966..0000000 --- a/setup/migrate-v1/channels.ts +++ /dev/null @@ -1,172 +0,0 @@ -/** - * Step: migrate-channels - * - * For each channel detected in migrate-db, run the corresponding v2 - * `setup/install-.sh` script in non-interactive mode. The script - * copies the adapter from the `channels` branch, installs the pinned - * dependency, and rebuilds. Credentials in v2 `.env` (migrate-env already - * copied them) are picked up automatically on the next service restart. - * - * This step does NOT run the pairing flow for each channel (that needs - * interactive prompts). The user is guided through pairing by the normal - * channel-selection step in setup/auto.ts, which happens immediately after - * migration. Installing the adapter first means that step won't have to - * re-install. - * - * Channels not supported in v2 are recorded in the handoff as - * `not_supported` so the skill can raise them with the user. - */ -import { spawn } from 'child_process'; -import fs from 'fs'; -import path from 'path'; - -import { log } from '../../src/log.js'; -import { emitStatus } from '../status.js'; -import { - installScriptForChannel, - readHandoff, - recordStep, - writeHandoff, -} from './shared.js'; - -function runScript(script: string): Promise<{ code: number; stdout: string; stderr: string }> { - return new Promise((resolve) => { - const child = spawn('bash', [script], { - stdio: ['ignore', 'pipe', 'pipe'], - env: { ...process.env, MIGRATION_NONINTERACTIVE: '1' }, - }); - // Capture both streams silently — the parent is under a clack spinner, - // and forwarding to stdout/stderr would break the spinner UI. The full - // transcript still lands in this step's raw log via the parent's tee - // (runner.ts: spawnStep writes this step's stdout/stderr to logs/setup- - // steps/NN-migrate-channels.log already). - let stdout = ''; - let stderr = ''; - child.stdout.on('data', (c: Buffer) => { - stdout += c.toString('utf-8'); - }); - child.stderr.on('data', (c: Buffer) => { - stderr += c.toString('utf-8'); - }); - child.on('close', (code) => - resolve({ code: code ?? 1, stdout, stderr }), - ); - child.on('error', () => - resolve({ code: 1, stdout, stderr: stderr || 'spawn_error' }), - ); - }); -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-channels', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_CHANNELS', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const channels = h.detected_channels; - if (channels.length === 0) { - recordStep('migrate-channels', { - status: 'skipped', - fields: { REASON: 'no-channels-detected' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_CHANNELS', { STATUS: 'skipped', REASON: 'no_channels' }); - return; - } - - const results: typeof h.channels_installed = []; - const followups: string[] = []; - - for (const ch of channels) { - const script = installScriptForChannel(ch.channel_type); - if (!script) { - results.push({ - channel_type: ch.channel_type, - status: 'not_supported', - }); - followups.push( - `Channel "${ch.channel_type}" has no v2 install script. The /migrate-from-v1 skill should ask the user whether to keep it as an orphan messaging_group or drop it.`, - ); - continue; - } - - const absoluteScript = path.join(process.cwd(), script); - if (!fs.existsSync(absoluteScript)) { - results.push({ - channel_type: ch.channel_type, - status: 'failed', - error: `install script missing at ${script}`, - }); - followups.push(`Install script for "${ch.channel_type}" missing at ${script} — this is a v2 repo issue, not a user issue.`); - continue; - } - - log.info('Running channel install script', { channel: ch.channel_type, script: absoluteScript }); - const { code, stdout, stderr } = await runScript(absoluteScript); - // Persist the install-script output to a sidecar so the skill can read it - // if diagnosis is needed. The parent's tee already captures our own - // stdout/stderr but the nested script's output is lost otherwise. - try { - const sidecar = path.join( - process.cwd(), - 'logs', - 'setup-migration', - `install-${ch.channel_type}.log`, - ); - fs.mkdirSync(path.dirname(sidecar), { recursive: true }); - fs.writeFileSync(sidecar, `# ${script}\n# exit ${code}\n\n=== stdout ===\n${stdout}\n=== stderr ===\n${stderr}\n`); - } catch { - // Sidecar is diagnostic-only — don't abort if the log dir is unwritable. - } - if (code === 0) { - results.push({ channel_type: ch.channel_type, status: 'success' }); - } else { - results.push({ - channel_type: ch.channel_type, - status: 'failed', - error: stderr.trim().slice(0, 400) || `exit ${code}`, - }); - followups.push( - `Installing "${ch.channel_type}" failed (exit ${code}). The /migrate-from-v1 skill should retry ${script} or walk the user through /add-${ch.channel_type}.`, - ); - } - } - - const handoffAfter = readHandoff(); - handoffAfter.channels_installed = results; - handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; - writeHandoff(handoffAfter); - - // `not_supported` is an expected/known outcome for channels whose v1 adapter - // has no v2 equivalent yet. It's a followup for the skill to raise — not a - // partial success. Only real install failures degrade status. - const anyFailed = results.some((r) => r.status === 'failed'); - const status: 'success' | 'partial' | 'failed' = anyFailed ? 'partial' : 'success'; - - recordStep('migrate-channels', { - status, - fields: { - INSTALLED: results.filter((r) => r.status === 'success').length, - FAILED: results.filter((r) => r.status === 'failed').length, - NOT_SUPPORTED: results.filter((r) => r.status === 'not_supported').length, - CHANNELS: results.map((r) => `${r.channel_type}=${r.status}`).join(','), - }, - notes: followups, - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_CHANNELS', { - STATUS: status, - INSTALLED: String(results.filter((r) => r.status === 'success').length), - FAILED: String(results.filter((r) => r.status === 'failed').length), - NOT_SUPPORTED: String(results.filter((r) => r.status === 'not_supported').length), - }); -} diff --git a/setup/migrate-v1/db.ts b/setup/migrate-v1/db.ts deleted file mode 100644 index d338214..0000000 --- a/setup/migrate-v1/db.ts +++ /dev/null @@ -1,321 +0,0 @@ -/** - * Step: migrate-db - * - * Seed v2.db with the essentials derived from v1's `registered_groups`: - * - agent_groups: one per v1 folder the user selected - * - messaging_groups: one per distinct (channel_type, platform_id) pair - * - messaging_group_agents: the wiring between them, with engage fields - * backfilled from v1's trigger_pattern / requires_trigger - * - * Does NOT seed users, user_roles, or agent_group_members. v1 has no ground - * truth for them — the /migrate-from-v1 skill interviews the user for the - * owner and seeds those tables. - * - * Idempotent: re-running skips any (folder) agent_group, (channel, platform_id) - * messaging_group, and (mg, ag) wiring that already exist. Safe to re-run - * after a partial failure. - * - * Expects `--selection ` where mode is 'all' | 'wired-only'. The - * orchestrator asks the user via clack and passes the result. - */ -import fs from 'fs'; -import path from 'path'; - -import Database from 'better-sqlite3'; - -import { DATA_DIR } from '../../src/config.js'; -import { createAgentGroup, getAgentGroupByFolder } from '../../src/db/agent-groups.js'; -import { initDb } from '../../src/db/connection.js'; -import { - createMessagingGroup, - createMessagingGroupAgent, - getMessagingGroupAgentByPair, - getMessagingGroupByPlatform, -} from '../../src/db/messaging-groups.js'; -import { runMigrations } from '../../src/db/migrations/index.js'; -import { log } from '../../src/log.js'; -import { emitStatus } from '../status.js'; -import { - fetchBotGuilds, - generateId, - inferChannelType, - readHandoff, - recordStep, - triggerToEngage, - v1PathsFor, - v2PlatformId, - writeHandoff, -} from './shared.js'; - -interface V1Group { - jid: string; - name: string; - folder: string; - trigger_pattern: string | null; - requires_trigger: number | null; - is_main: number | null; - channel_name: string | null; -} - -interface DbArgs { - selection: 'all' | 'wired-only'; -} - -function parseArgs(args: string[]): DbArgs { - let selection: 'all' | 'wired-only' = 'wired-only'; - for (let i = 0; i < args.length; i++) { - if (args[i] === '--selection') { - const v = args[++i]; - if (v === 'all' || v === 'wired-only') selection = v; - } - } - return { selection }; -} - -export async function run(args: string[]): Promise { - const parsed = parseArgs(args); - const h = readHandoff(); - - if (!h.v1_path) { - recordStep('migrate-db', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_DB', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const validate = h.steps['migrate-validate']; - if (validate && validate.status === 'failed') { - recordStep('migrate-db', { - status: 'skipped', - fields: { REASON: 'validate-failed' }, - notes: ['DB shape did not validate; skipping DB migration.'], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_DB', { STATUS: 'skipped', REASON: 'validate_failed' }); - return; - } - - const paths = v1PathsFor(h.v1_path); - let v1Db: Database.Database; - try { - v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - recordStep('migrate-db', { - status: 'failed', - fields: { REASON: 'v1-db-open-failed' }, - notes: [message], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_DB', { STATUS: 'failed', REASON: 'v1_db_open_failed', ERROR: message }); - return; - } - - const v1Groups = v1Db - .prepare( - 'SELECT jid, name, folder, trigger_pattern, requires_trigger, is_main, channel_name FROM registered_groups', - ) - .all() as V1Group[]; - v1Db.close(); - - // Filter by selection mode. "wired-only" keeps rows where we can confidently - // say which channel they belong to — either `channel_name` is set, or the - // JID prefix resolves to a known channel type. - const selected: V1Group[] = []; - const detectedChannels = new Map(); - - for (const g of v1Groups) { - const channelType = inferChannelType(g.jid, g.channel_name); - const source: 'channel_name' | 'jid_prefix' = g.channel_name?.trim() ? 'channel_name' : 'jid_prefix'; - if (!channelType) { - // Can't infer — skip in both modes; the skill raises it with the user. - continue; - } - if (parsed.selection === 'wired-only' && source === 'jid_prefix' && !channelType) { - continue; - } - selected.push(g); - const entry = detectedChannels.get(channelType) ?? { source, count: 0 }; - entry.count += 1; - // Prefer explicit channel_name as the source if any row had it. - if (source === 'channel_name') entry.source = 'channel_name'; - detectedChannels.set(channelType, entry); - } - - h.group_selection = { - mode: parsed.selection, - selected_folders: selected.map((g) => g.folder), - total_v1_groups: v1Groups.length, - wired_v1_groups: selected.length, - }; - h.detected_channels = [...detectedChannels.entries()].map(([channel_type, info]) => ({ - channel_type, - source: info.source, - group_count: info.count, - })); - writeHandoff(h); - - // For channels where v2's platform_id includes a component v1 didn't record - // (Discord's guild id), fetch the bot's guilds up-front. If the bot is in - // a single guild we can splice that id into every platform_id; otherwise - // fall back to the v1-format id (v2's channel-registration flow will repair - // on first message). Done ONCE per channel_type, not per-row, so this is - // cheap regardless of group count. - const v1EnvText = fs.existsSync(paths.env) ? fs.readFileSync(paths.env, 'utf-8') : ''; - const v1EnvMap = new Map(); - for (const line of v1EnvText.split('\n')) { - const t = line.trim(); - if (!t || t.startsWith('#')) continue; - const eq = t.indexOf('='); - if (eq <= 0) continue; - v1EnvMap.set(t.slice(0, eq).trim(), t.slice(eq + 1)); - } - const singleGuildByChannel = new Map(); - for (const channelType of detectedChannels.keys()) { - const info = await fetchBotGuilds(channelType, (k) => v1EnvMap.get(k)); - if (info && info.guildIds.length === 1) { - singleGuildByChannel.set(channelType, info.guildIds[0]); - } - } - - // Initialize v2.db (creates schema if not present — runMigrations is no-op - // when the schema is already current, so this is safe on a live v2 install). - fs.mkdirSync(path.join(process.cwd(), 'data'), { recursive: true }); - const v2Path = path.join(DATA_DIR, 'v2.db'); - const v2Db = initDb(v2Path); - runMigrations(v2Db); - - let agentGroupsCreated = 0; - let agentGroupsReused = 0; - let messagingGroupsCreated = 0; - let messagingGroupsReused = 0; - let wiringsCreated = 0; - let wiringsReused = 0; - let skipped = 0; - const followups: string[] = []; - - for (const g of selected) { - const channelType = inferChannelType(g.jid, g.channel_name); - if (!channelType) { - skipped += 1; - continue; - } - - const guildId = singleGuildByChannel.get(channelType); - const platformId = v2PlatformId(channelType, g.jid, { guildId }); - const createdAt = new Date().toISOString(); - - try { - // agent_group — one per folder - let ag = getAgentGroupByFolder(g.folder); - if (!ag) { - createAgentGroup({ - id: generateId('ag'), - name: g.name || g.folder, - folder: g.folder, - agent_provider: null, - created_at: createdAt, - }); - ag = getAgentGroupByFolder(g.folder)!; - agentGroupsCreated += 1; - } else { - agentGroupsReused += 1; - } - - // messaging_group — one per (channel_type, platform_id) - let mg = getMessagingGroupByPlatform(channelType, platformId); - if (!mg) { - createMessagingGroup({ - id: generateId('mg'), - channel_type: channelType, - platform_id: platformId, - name: g.name || null, - is_group: 1, // v1 didn't distinguish; default to group (safe for routing) - unknown_sender_policy: 'strict', // skill's interview flips this if v1 was "public" - created_at: createdAt, - }); - mg = getMessagingGroupByPlatform(channelType, platformId)!; - messagingGroupsCreated += 1; - } else { - messagingGroupsReused += 1; - } - - // messaging_group_agents — wire them if not already wired - const existingWiring = getMessagingGroupAgentByPair(mg.id, ag.id); - if (!existingWiring) { - const engage = triggerToEngage({ - trigger_pattern: g.trigger_pattern, - requires_trigger: g.requires_trigger, - }); - createMessagingGroupAgent({ - id: generateId('mga'), - messaging_group_id: mg.id, - agent_group_id: ag.id, - engage_mode: engage.engage_mode, - engage_pattern: engage.engage_pattern, - sender_scope: 'all', - ignored_message_policy: 'drop', - session_mode: 'shared', - priority: 0, - created_at: createdAt, - }); - wiringsCreated += 1; - } else { - wiringsReused += 1; - } - - if (g.is_main === 1) { - followups.push( - `Folder "${g.folder}" was the v1 main group (is_main=1). v2 has no is_main flag — the /migrate-from-v1 skill should grant this folder's channel to the owner user when it runs.`, - ); - } - } catch (err) { - skipped += 1; - const message = err instanceof Error ? err.message : String(err); - log.error('Failed to seed v1 group', { folder: g.folder, err: message }); - followups.push(`Folder "${g.folder}" failed to seed: ${message}`); - } - } - - v2Db.close(); - - const partial = skipped > 0; - const handoffAfter = readHandoff(); - handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; - writeHandoff(handoffAfter); - - recordStep('migrate-db', { - status: partial ? 'partial' : 'success', - fields: { - SELECTION: parsed.selection, - V1_GROUPS_TOTAL: v1Groups.length, - SELECTED: selected.length, - AGENT_GROUPS_CREATED: agentGroupsCreated, - AGENT_GROUPS_REUSED: agentGroupsReused, - MESSAGING_GROUPS_CREATED: messagingGroupsCreated, - MESSAGING_GROUPS_REUSED: messagingGroupsReused, - WIRINGS_CREATED: wiringsCreated, - WIRINGS_REUSED: wiringsReused, - SKIPPED: skipped, - CHANNELS: [...detectedChannels.keys()].join(','), - }, - notes: followups, - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_DB', { - STATUS: partial ? 'partial' : 'success', - SELECTION: parsed.selection, - V1_GROUPS_TOTAL: String(v1Groups.length), - SELECTED: String(selected.length), - AGENT_GROUPS_CREATED: String(agentGroupsCreated), - MESSAGING_GROUPS_CREATED: String(messagingGroupsCreated), - WIRINGS_CREATED: String(wiringsCreated), - SKIPPED: String(skipped), - CHANNELS: [...detectedChannels.keys()].join(',') || 'none', - }); -} diff --git a/setup/migrate-v1/detect.ts b/setup/migrate-v1/detect.ts deleted file mode 100644 index 983531d..0000000 --- a/setup/migrate-v1/detect.ts +++ /dev/null @@ -1,107 +0,0 @@ -/** - * Step: migrate-detect - * - * Find a v1 install on disk. Scans the standard candidate paths; if none - * matches, exits with a NOT_FOUND status (the orchestrator then offers a - * clack prompt so the user can point at a custom path). - * - * Never prompts — this step is pure discovery so it stays safe to run under - * NANOCLAW_SKIP= without blocking on stdin. - */ -import fs from 'fs'; -import path from 'path'; - -import { emitStatus } from '../status.js'; -import { - defaultV1Candidates, - looksLikeV1Install, - readHandoff, - recordStep, - v1PathsFor, - writeHandoff, -} from './shared.js'; - -interface DetectArgs { - /** Explicit path to check, skipping the default candidate list. */ - path?: string; -} - -function parseArgs(args: string[]): DetectArgs { - const out: DetectArgs = {}; - for (let i = 0; i < args.length; i++) { - if (args[i] === '--path') { - out.path = args[++i] || undefined; - } - } - return out; -} - -export async function run(args: string[]): Promise { - const parsed = parseArgs(args); - - // An explicit path — either from --path or $NANOCLAW_V1_PATH — is - // authoritative. If it doesn't validate, we don't fall through to - // the default candidate list. That keeps the user's explicit intent - // from being silently overridden. - const envOverride = process.env.NANOCLAW_V1_PATH?.trim(); - const explicit = parsed.path ?? envOverride ?? null; - const candidates = explicit ? [explicit] : defaultV1Candidates(); - - for (const candidate of candidates) { - const absolute = path.resolve(candidate); - // Don't self-match — if the candidate resolves to the v2 checkout we're - // running inside, skip it. Protects users who cloned v2 into `~/nanoclaw` - // after deleting v1. - if (absolute === path.resolve(process.cwd())) continue; - - const check = looksLikeV1Install(absolute); - if (!check.ok) continue; - - const paths = v1PathsFor(absolute); - let version = 'unknown'; - try { - const pkg = JSON.parse(fs.readFileSync(paths.packageJson, 'utf-8')) as { version?: string }; - version = pkg.version ?? 'unknown'; - } catch { - // Already sanity-checked by looksLikeV1Install — a failure here means - // the file changed under us between calls. Unlikely, not fatal. - } - - const h = readHandoff(); - h.v1_path = absolute; - h.v1_version = version; - writeHandoff(h); - - recordStep('migrate-detect', { - status: 'success', - fields: { V1_PATH: absolute, V1_VERSION: version }, - notes: [], - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_DETECT', { - STATUS: 'success', - V1_PATH: absolute, - V1_VERSION: version, - DB_PATH: paths.db, - ENV_PATH: paths.env, - GROUPS_PATH: paths.groups, - }); - return; - } - - // Nothing matched. Not an error — most v2 installs are fresh, not migrations. - const scanned = candidates.map((c) => path.resolve(c)).join(','); - recordStep('migrate-detect', { - status: 'skipped', - fields: { REASON: 'no-v1-install-found' }, - notes: [`Scanned: ${scanned}`], - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_DETECT', { - STATUS: 'skipped', - REASON: 'not_found', - CANDIDATES_SCANNED: String(candidates.length), - }); -} diff --git a/setup/migrate-v1/env.ts b/setup/migrate-v1/env.ts deleted file mode 100644 index e2530e0..0000000 --- a/setup/migrate-v1/env.ts +++ /dev/null @@ -1,135 +0,0 @@ -/** - * Step: migrate-env - * - * Copy every key from v1 `.env` to v2 `.env`. Preserves v2 values that - * already exist (never overwrites). Skips lines that don't look like a - * `KEY=value` pair. - * - * Why copy everything, not a curated list? v1 installs accumulate - * project-specific keys (custom MCP creds, feature flags, webhook tokens) - * that the migration can't enumerate ahead of time. The user explicitly - * asked for everything. We log what we carried so the skill can review. - * - * Security note: we do NOT log values here — only keys. The raw log already - * contains the file contents; we don't echo them to stdout. - */ -import fs from 'fs'; -import path from 'path'; - -import { emitStatus } from '../status.js'; -import { readHandoff, recordStep, v1PathsFor } from './shared.js'; - -interface EnvLine { - key: string; - value: string; - raw: string; -} - -function parseEnv(text: string): EnvLine[] { - const out: EnvLine[] = []; - for (const raw of text.split('\n')) { - const line = raw.trimEnd(); - if (!line) continue; - if (line.startsWith('#')) continue; - const eq = line.indexOf('='); - if (eq <= 0) continue; - const key = line.slice(0, eq).trim(); - if (!/^[A-Z_][A-Z0-9_]*$/i.test(key)) continue; - const value = line.slice(eq + 1); - out.push({ key, value, raw: line }); - } - return out; -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-env', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_ENV', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const paths = v1PathsFor(h.v1_path); - if (!fs.existsSync(paths.env)) { - recordStep('migrate-env', { - status: 'skipped', - fields: { REASON: 'v1-env-missing' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_ENV', { STATUS: 'skipped', REASON: 'v1_env_missing' }); - return; - } - - const v2EnvPath = path.join(process.cwd(), '.env'); - const v1Text = fs.readFileSync(paths.env, 'utf-8'); - const v1Lines = parseEnv(v1Text); - - let v2Text = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; - const v2Lines = parseEnv(v2Text); - const v2Keys = new Set(v2Lines.map((l) => l.key)); - - const copied: string[] = []; - const skipped: string[] = []; - const appended: string[] = []; - - // Tag the appended block so a later re-run can find it and not double-append. - const BLOCK_START = '# ── migrated from v1 ──'; - const alreadyMigrated = v2Text.includes(BLOCK_START); - - for (const line of v1Lines) { - if (v2Keys.has(line.key)) { - skipped.push(line.key); - continue; - } - copied.push(line.key); - appended.push(line.raw); - } - - if (appended.length > 0) { - const suffix = [ - v2Text.endsWith('\n') || v2Text === '' ? '' : '\n', - alreadyMigrated ? '' : `\n${BLOCK_START}\n`, - appended.join('\n'), - '\n', - ].join(''); - v2Text = v2Text + suffix; - fs.writeFileSync(v2EnvPath, v2Text); - } - - // Container reads from data/env/env (host mounts it). Keep it in sync. - const containerEnvDir = path.join(process.cwd(), 'data', 'env'); - try { - fs.mkdirSync(containerEnvDir, { recursive: true }); - fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); - } catch { - // Non-fatal; the service restart (later step) will rehydrate if needed. - } - - recordStep('migrate-env', { - status: 'success', - fields: { - KEYS_COPIED: copied.length, - KEYS_SKIPPED_EXISTING: skipped.length, - V1_ENV: paths.env, - V2_ENV: v2EnvPath, - }, - notes: [ - copied.length > 0 ? `Copied: ${copied.join(', ')}` : '', - skipped.length > 0 ? `Skipped (already in v2 .env): ${skipped.join(', ')}` : '', - ].filter(Boolean), - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_ENV', { - STATUS: 'success', - KEYS_COPIED: String(copied.length), - KEYS_SKIPPED_EXISTING: String(skipped.length), - COPIED_KEYS: copied.join(',') || 'none', - }); -} diff --git a/setup/migrate-v1/groups.ts b/setup/migrate-v1/groups.ts deleted file mode 100644 index 206f441..0000000 --- a/setup/migrate-v1/groups.ts +++ /dev/null @@ -1,230 +0,0 @@ -/** - * Step: migrate-groups - * - * Copy v1 group folders into v2. For each folder selected in migrate-db: - * - Create groups// in v2 if missing - * - Copy v1's CLAUDE.md to v2 as CLAUDE.local.md (v2 composes CLAUDE.md at - * container spawn — don't write directly to CLAUDE.md) - * - If v1 had a container_config JSON, write it to .v1-container-config.json - * for the /migrate-from-v1 skill to reconcile (v2's container.json shape - * has drifted enough that a silent 1:1 copy would be wrong) - * - Preserve any other non-standard files from the v1 folder (e.g. SOUL.md, - * personality.md, custom subdirs) — rsync-style, skipping destination files - * that already exist. - * - * Does not overwrite files already present in v2 — re-running is safe. - */ -import fs from 'fs'; -import path from 'path'; - -import Database from 'better-sqlite3'; - -import { log } from '../../src/log.js'; -import { emitStatus } from '../status.js'; -import { - readHandoff, - recordStep, - safeJsonStringify, - scanForV1Patterns, - v1PathsFor, - writeHandoff, -} from './shared.js'; - -const SKIP_NAMES = new Set(['CLAUDE.md', 'logs', '.git', '.DS_Store', 'node_modules']); - -/** - * Copy everything in src except SKIP_NAMES. CLAUDE.md is handled separately. - * Returns the count of files actually written (skipped-existing not counted). - */ -function copyTree(src: string, dst: string): number { - let written = 0; - if (!fs.existsSync(src)) return 0; - fs.mkdirSync(dst, { recursive: true }); - - for (const entry of fs.readdirSync(src, { withFileTypes: true })) { - if (SKIP_NAMES.has(entry.name)) continue; - const s = path.join(src, entry.name); - const d = path.join(dst, entry.name); - - if (entry.isDirectory()) { - written += copyTree(s, d); - continue; - } - // Don't clobber files v2 already has (e.g. CLAUDE.local.md that the - // operator already wrote). Append-only semantics for this step. - if (fs.existsSync(d)) continue; - fs.copyFileSync(s, d); - written += 1; - } - return written; -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-groups', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_GROUPS', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - if (h.group_selection.selected_folders.length === 0) { - recordStep('migrate-groups', { - status: 'skipped', - fields: { REASON: 'no-folders-selected' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_GROUPS', { STATUS: 'skipped', REASON: 'no_selection' }); - return; - } - - const paths = v1PathsFor(h.v1_path); - const v2GroupsDir = path.join(process.cwd(), 'groups'); - fs.mkdirSync(v2GroupsDir, { recursive: true }); - - // Pull container_config for each selected folder up-front so we can write - // the .v1-container-config.json sidecar without holding the DB open per-folder. - const containerConfigs = new Map(); - try { - const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); - const rows = v1Db - .prepare('SELECT folder, container_config FROM registered_groups WHERE folder IN (SELECT value FROM json_each(?))') - .all(JSON.stringify(h.group_selection.selected_folders)) as Array<{ folder: string; container_config: string | null }>; - for (const r of rows) containerConfigs.set(r.folder, r.container_config); - v1Db.close(); - } catch (err) { - // Older sqlite without json_each would break the query. Fall back to - // per-folder reads — slower but reliable. - log.info('Falling back to per-folder container_config lookup', { err }); - try { - const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); - const stmt = v1Db.prepare('SELECT container_config FROM registered_groups WHERE folder = ?'); - for (const folder of h.group_selection.selected_folders) { - const row = stmt.get(folder) as { container_config: string | null } | undefined; - containerConfigs.set(folder, row?.container_config ?? null); - } - v1Db.close(); - } catch { - // Give up — we still migrate files; the skill handles missing config. - } - } - - let foldersProcessed = 0; - let foldersSkippedMissing = 0; - let claudeMdMigrated = 0; - let claudeLocalPreserved = 0; - let containerConfigsStashed = 0; - let otherFilesCopied = 0; - const followups: string[] = []; - - for (const folder of h.group_selection.selected_folders) { - const v1Folder = path.join(paths.groups, folder); - const v2Folder = path.join(v2GroupsDir, folder); - - if (!fs.existsSync(v1Folder)) { - foldersSkippedMissing += 1; - followups.push( - `Folder "${folder}" was in v1's registered_groups but not on disk at ${v1Folder} — DB entry was seeded, no files to migrate.`, - ); - continue; - } - - fs.mkdirSync(v2Folder, { recursive: true }); - - // CLAUDE.md → CLAUDE.local.md. Don't write CLAUDE.md directly — v2's - // group-init.ts composes that file from shared + fragments + local. - const v1Claude = path.join(v1Folder, 'CLAUDE.md'); - const v2Local = path.join(v2Folder, 'CLAUDE.local.md'); - let claudeContent: string | null = null; - if (fs.existsSync(v1Claude)) { - if (fs.existsSync(v2Local)) { - claudeLocalPreserved += 1; - try { - claudeContent = fs.readFileSync(v2Local, 'utf-8'); - } catch { - claudeContent = null; - } - } else { - try { - claudeContent = fs.readFileSync(v1Claude, 'utf-8'); - fs.writeFileSync(v2Local, claudeContent); - claudeMdMigrated += 1; - } catch (err) { - followups.push(`Failed to copy CLAUDE.md for "${folder}": ${err instanceof Error ? err.message : err}`); - } - } - } - - // Scan the copied content for v1-specific infrastructure patterns. If we - // find any, add a followup so the /migrate-from-v1 skill can triage the - // file with the user. We DON'T edit the file — v1 CLAUDE.md can be - // author-specific and heuristic translation is worse than a flag. - if (claudeContent) { - const matches = scanForV1Patterns(claudeContent); - if (matches.length > 0) { - const summary = matches - .map((m) => `${m.description} (lines ${m.lines.join(',')})`) - .join('; '); - followups.push( - `Folder "${folder}" CLAUDE.local.md references v1-specific infrastructure: ${summary}. The skill should read the file and translate patterns using docs/v1-to-v2-changes.md.`, - ); - } - } - - // Stash container_config JSON so the skill can reconcile it. - const config = containerConfigs.get(folder); - if (config) { - const sidecar = path.join(v2Folder, '.v1-container-config.json'); - try { - // Pretty-print so humans can read it during reconciliation. - const parsed = JSON.parse(config) as unknown; - fs.writeFileSync(sidecar, safeJsonStringify(parsed)); - containerConfigsStashed += 1; - followups.push( - `Folder "${folder}" has a v1 container_config — stashed at ${path.relative(process.cwd(), sidecar)}. The /migrate-from-v1 skill will map it to v2's container.json shape.`, - ); - } catch { - // Non-JSON container_config — write raw so the skill can still read it. - fs.writeFileSync(sidecar, config); - containerConfigsStashed += 1; - } - } - - otherFilesCopied += copyTree(v1Folder, v2Folder); - foldersProcessed += 1; - } - - // Merge followups. - const handoffAfter = readHandoff(); - handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; - writeHandoff(handoffAfter); - - const partial = foldersSkippedMissing > 0; - recordStep('migrate-groups', { - status: partial ? 'partial' : 'success', - fields: { - FOLDERS_PROCESSED: foldersProcessed, - FOLDERS_SKIPPED_MISSING: foldersSkippedMissing, - CLAUDE_MD_MIGRATED: claudeMdMigrated, - CLAUDE_LOCAL_PRESERVED: claudeLocalPreserved, - CONTAINER_CONFIGS_STASHED: containerConfigsStashed, - OTHER_FILES_COPIED: otherFilesCopied, - }, - notes: followups, - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_GROUPS', { - STATUS: partial ? 'partial' : 'success', - FOLDERS_PROCESSED: String(foldersProcessed), - FOLDERS_SKIPPED_MISSING: String(foldersSkippedMissing), - CLAUDE_MD_MIGRATED: String(claudeMdMigrated), - CONTAINER_CONFIGS_STASHED: String(containerConfigsStashed), - OTHER_FILES_COPIED: String(otherFilesCopied), - }); -} diff --git a/setup/migrate-v1/tasks.ts b/setup/migrate-v1/tasks.ts deleted file mode 100644 index 836be7c..0000000 --- a/setup/migrate-v1/tasks.ts +++ /dev/null @@ -1,307 +0,0 @@ -/** - * Step: migrate-tasks - * - * Port v1's `scheduled_tasks` into v2's session inbound DBs. v1 had a - * dedicated table with its own scheduling grammar; v2 treats tasks as - * `messages_in` rows with `kind='task'`, `process_after`, and `recurrence` - * (cron string). See docs/v1-to-v2-changes.md "Scheduling". - * - * Flow per v1 row: - * 1. Resolve (agent_group_id, messaging_group_id) from v1 (group_folder, chat_jid) - * 2. resolveSession() — creates the session on demand if absent - * 3. insertTask() into the session's inbound.db - * - * Active v1 rows (status='active') are migrated. Completed/stopped rows get - * exported to logs/setup-migration/inactive-tasks.json for reference. - * - * v1's schedule_type / schedule_value are mapped to cron here. Known types: - * 'cron' → schedule_value is already a cron string - * 'interval' → e.g. '5m'/'1h' → cron equivalent (best effort) - * 'once' → no recurrence, process_after = schedule_value if parseable - * Unknown types go to inactive-tasks.json with a note. - */ -import fs from 'fs'; -import path from 'path'; - -import Database from 'better-sqlite3'; - -import { DATA_DIR } from '../../src/config.js'; -import { initDb, closeDb } from '../../src/db/connection.js'; -import { getAgentGroupByFolder } from '../../src/db/agent-groups.js'; -import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; -import { runMigrations } from '../../src/db/migrations/index.js'; -import { log } from '../../src/log.js'; -import { insertTask } from '../../src/modules/scheduling/db.js'; -import { openInboundDb, resolveSession } from '../../src/session-manager.js'; -import { emitStatus } from '../status.js'; -import { - INACTIVE_TASKS_PATH, - MIGRATION_DIR, - inferChannelType, - readHandoff, - recordStep, - safeJsonStringify, - v1PathsFor, - v2PlatformId, - writeHandoff, -} from './shared.js'; - -interface V1Task { - id: string; - group_folder: string; - chat_jid: string; - prompt: string; - schedule_type: string; - schedule_value: string; - next_run: string | null; - last_run: string | null; - status: string; - context_mode: string | null; - script: string | null; -} - -/** Convert v1 schedule_type + schedule_value into (processAfter, recurrence). */ -function toProcessAfterAndRecurrence(t: V1Task): { - processAfter: string; - recurrence: string | null; - note?: string; -} | null { - const now = new Date().toISOString(); - - if (t.schedule_type === 'cron') { - // Validate shape — 5 or 6 fields separated by whitespace. cron-parser is - // the runtime source of truth; here we just reject obvious garbage so - // we don't insert tasks that will explode on the first sweep tick. - const fields = t.schedule_value.trim().split(/\s+/).length; - if (fields < 5 || fields > 6) return null; - return { - processAfter: t.next_run || now, - recurrence: t.schedule_value.trim(), - }; - } - - if (t.schedule_type === 'interval') { - // '5m' → '*/5 * * * *'; '1h' → '0 * * * *'; '1d' → '0 0 * * *'. - // Best effort — any unit we don't recognize falls through to null. - const m = /^(\d+)([smhd])$/.exec(t.schedule_value.trim()); - if (!m) return null; - const n = parseInt(m[1], 10); - const unit = m[2]; - if (!n || n < 1) return null; - let cron: string | null = null; - if (unit === 'm' && n < 60) cron = `*/${n} * * * *`; - else if (unit === 'h' && n < 24) cron = `0 */${n} * * *`; - else if (unit === 'd' && n < 28) cron = `0 0 */${n} * *`; - if (!cron) return null; - return { processAfter: t.next_run || now, recurrence: cron }; - } - - if (t.schedule_type === 'once' || t.schedule_type === 'at') { - return { - processAfter: t.next_run || t.schedule_value || now, - recurrence: null, - }; - } - - return null; -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-tasks', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const validate = h.steps['migrate-validate']; - if (validate && validate.status === 'failed') { - recordStep('migrate-tasks', { - status: 'skipped', - fields: { REASON: 'validate-failed' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'validate_failed' }); - return; - } - - const paths = v1PathsFor(h.v1_path); - - // Read v1 tasks into memory so we can close the v1 DB before we open v2's - // central DB via initDb() (which is a module singleton and doesn't love - // having two files open through it). - let activeTasks: V1Task[] = []; - let inactiveTasks: V1Task[] = []; - try { - const v1Db = new Database(paths.db, { readonly: true, fileMustExist: true }); - const all = v1Db.prepare('SELECT * FROM scheduled_tasks').all() as V1Task[]; - v1Db.close(); - activeTasks = all.filter((t) => t.status === 'active'); - inactiveTasks = all.filter((t) => t.status !== 'active'); - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - recordStep('migrate-tasks', { - status: 'failed', - fields: { REASON: 'v1-read-failed' }, - notes: [message], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_TASKS', { STATUS: 'failed', REASON: 'v1_read_failed', ERROR: message }); - return; - } - - if (activeTasks.length === 0 && inactiveTasks.length === 0) { - recordStep('migrate-tasks', { - status: 'skipped', - fields: { REASON: 'no-v1-tasks' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_TASKS', { STATUS: 'skipped', REASON: 'no_v1_tasks' }); - return; - } - - // Dump inactive tasks for reference — always, even if there are no active ones. - if (inactiveTasks.length > 0) { - fs.mkdirSync(MIGRATION_DIR, { recursive: true }); - fs.writeFileSync(INACTIVE_TASKS_PATH, safeJsonStringify({ tasks: inactiveTasks })); - } - - // Connect to v2 central DB to resolve (folder → ag) and (channel+pid → mg). - const v2Path = path.join(DATA_DIR, 'v2.db'); - fs.mkdirSync(path.dirname(v2Path), { recursive: true }); - const v2Db = initDb(v2Path); - runMigrations(v2Db); - - const followups: string[] = []; - let migrated = 0; - let failed = 0; - let skipped = 0; - - for (const t of activeTasks) { - try { - const ag = getAgentGroupByFolder(t.group_folder); - if (!ag) { - skipped += 1; - followups.push( - `Task "${t.id}" (folder "${t.group_folder}"): agent_group not seeded in v2 — run migrate-db first or deselect the task.`, - ); - continue; - } - - const channelType = inferChannelType(t.chat_jid, null); - if (!channelType) { - skipped += 1; - followups.push(`Task "${t.id}": could not infer channel from chat_jid "${t.chat_jid}".`); - continue; - } - const platformId = v2PlatformId(channelType, t.chat_jid); - const mg = getMessagingGroupByPlatform(channelType, platformId); - if (!mg) { - skipped += 1; - followups.push( - `Task "${t.id}": messaging_group for (${channelType}, ${platformId}) not seeded. Add the channel then re-run this step.`, - ); - continue; - } - - const scheduling = toProcessAfterAndRecurrence(t); - if (!scheduling) { - skipped += 1; - followups.push( - `Task "${t.id}": schedule_type "${t.schedule_type}" / value "${t.schedule_value}" did not map to a v2 cron — exported to inactive-tasks.json for manual review.`, - ); - inactiveTasks.push(t); - continue; - } - - // resolveSession creates (ag, mg) session if not present; 'shared' mode - // matches v1 which had one session per group_folder. - const { session } = resolveSession(ag.id, mg.id, null, 'shared'); - const inboxDb = openInboundDb(ag.id, session.id); - try { - // Idempotence: skip if we've already migrated this task id. We use the - // v1 task id verbatim as the v2 messages_in.id (stable — lets users - // re-run migration without duplicate-key errors or shadow tasks). - const existing = inboxDb - .prepare("SELECT id FROM messages_in WHERE id = ? AND kind = 'task'") - .get(t.id) as { id: string } | undefined; - if (existing) { - skipped += 1; - continue; - } - - insertTask(inboxDb, { - id: t.id, - processAfter: scheduling.processAfter, - recurrence: scheduling.recurrence, - platformId, - channelType, - threadId: null, - content: JSON.stringify({ - prompt: t.prompt, - script: t.script ?? null, - migrated_from_v1: { original_id: t.id, context_mode: t.context_mode ?? null }, - }), - }); - } finally { - inboxDb.close(); - } - - log.info('Migrated v1 scheduled task', { taskId: t.id, session: session.id, mg: mg.id }); - migrated += 1; - } catch (err) { - failed += 1; - const message = err instanceof Error ? err.message : String(err); - followups.push(`Task "${t.id}" failed to migrate: ${message}`); - } - } - - // Re-dump inactive tasks in case scheduling-translation pushed any in. - if (inactiveTasks.length > 0) { - fs.writeFileSync(INACTIVE_TASKS_PATH, safeJsonStringify({ tasks: inactiveTasks })); - } - - closeDb(); - - const handoffAfter = readHandoff(); - handoffAfter.tasks = { - v1_active: activeTasks.length, - v1_inactive: inactiveTasks.length, - migrated, - failed, - skipped, - }; - handoffAfter.followups = [...new Set([...handoffAfter.followups, ...followups])]; - writeHandoff(handoffAfter); - - const partial = failed > 0 || skipped > 0; - recordStep('migrate-tasks', { - status: failed > 0 ? 'partial' : partial ? 'partial' : 'success', - fields: { - V1_ACTIVE: activeTasks.length, - V1_INACTIVE: inactiveTasks.length, - MIGRATED: migrated, - FAILED: failed, - SKIPPED: skipped, - INACTIVE_EXPORT: inactiveTasks.length > 0 ? INACTIVE_TASKS_PATH : '', - }, - notes: followups, - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_TASKS', { - STATUS: partial ? 'partial' : 'success', - V1_ACTIVE: String(activeTasks.length), - V1_INACTIVE: String(inactiveTasks.length), - MIGRATED: String(migrated), - FAILED: String(failed), - SKIPPED: String(skipped), - }); -} diff --git a/setup/migrate-v1/validate.ts b/setup/migrate-v1/validate.ts deleted file mode 100644 index 73cd377..0000000 --- a/setup/migrate-v1/validate.ts +++ /dev/null @@ -1,213 +0,0 @@ -/** - * Step: migrate-validate - * - * Before touching v1 data, assert the DB has the shape we expect. We know - * v1's schema (see docs/v1-to-v2-changes.md "Entity model") — different - * shapes happened over v1's development, but by v1.2.x the `registered_groups` - * columns and `scheduled_tasks` columns stabilized. If we see something else, - * we bail early so later steps don't write garbage to v2.db. - * - * Output: - * - `logs/setup-migration/schema-mismatch.json` on failure (read by the skill) - * - Status block MIGRATE_VALIDATE with OK/FAILED - * - Even on failure, subsequent steps still run — they'll short-circuit - * on their own if validate marked the DB unusable. This keeps env + group - * folder migration working when only the DB is broken. - */ -import fs from 'fs'; - -import Database from 'better-sqlite3'; - -import { emitStatus } from '../status.js'; -import { - SCHEMA_MISMATCH_PATH, - readHandoff, - recordStep, - safeJsonStringify, - v1PathsFor, -} from './shared.js'; - -const EXPECTED_TABLES = [ - 'registered_groups', - 'scheduled_tasks', - 'chats', - 'messages', - 'sessions', - 'router_state', -]; - -const REQUIRED_REGISTERED_GROUPS_COLUMNS = [ - 'jid', - 'name', - 'folder', - 'trigger_pattern', - 'added_at', - 'requires_trigger', -]; - -const REQUIRED_SCHEDULED_TASKS_COLUMNS = [ - 'id', - 'group_folder', - 'chat_jid', - 'prompt', - 'schedule_type', - 'schedule_value', - 'status', -]; - -interface TableInfo { - table: string; - columns: string[]; - missing_columns: string[]; -} - -export async function run(_args: string[]): Promise { - const h = readHandoff(); - if (!h.v1_path) { - recordStep('migrate-validate', { - status: 'skipped', - fields: { REASON: 'detect-not-run' }, - notes: [], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_VALIDATE', { STATUS: 'skipped', REASON: 'no_v1_path' }); - return; - } - - const paths = v1PathsFor(h.v1_path); - if (!fs.existsSync(paths.db)) { - recordStep('migrate-validate', { - status: 'failed', - fields: { REASON: 'db-missing', DB_PATH: paths.db }, - notes: ['v1 DB file does not exist at expected path'], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_VALIDATE', { - STATUS: 'failed', - REASON: 'db_missing', - DB_PATH: paths.db, - }); - return; - } - - let db: Database.Database; - try { - db = new Database(paths.db, { readonly: true, fileMustExist: true }); - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - recordStep('migrate-validate', { - status: 'failed', - fields: { REASON: 'db-open-failed' }, - notes: [message], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_VALIDATE', { - STATUS: 'failed', - REASON: 'db_open_failed', - ERROR: message, - }); - return; - } - - try { - const tableRows = db - .prepare("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name") - .all() as Array<{ name: string }>; - const tables = new Set(tableRows.map((r) => r.name)); - - const missingTables = EXPECTED_TABLES.filter((t) => !tables.has(t)); - const tableInfos: TableInfo[] = []; - - for (const t of EXPECTED_TABLES) { - if (!tables.has(t)) continue; - const cols = db.prepare(`PRAGMA table_info(${t})`).all() as Array<{ name: string }>; - const columnNames = cols.map((c) => c.name); - const missing = - t === 'registered_groups' - ? REQUIRED_REGISTERED_GROUPS_COLUMNS.filter((c) => !columnNames.includes(c)) - : t === 'scheduled_tasks' - ? REQUIRED_SCHEDULED_TASKS_COLUMNS.filter((c) => !columnNames.includes(c)) - : []; - tableInfos.push({ table: t, columns: columnNames, missing_columns: missing }); - } - - const columnMismatches = tableInfos.filter((t) => t.missing_columns.length > 0); - const groupCount = - tables.has('registered_groups') - ? ((db.prepare('SELECT COUNT(*) AS c FROM registered_groups').get() as { c: number }).c) - : 0; - const taskCount = - tables.has('scheduled_tasks') - ? ((db.prepare('SELECT COUNT(*) AS c FROM scheduled_tasks').get() as { c: number }).c) - : 0; - - db.close(); - - if (missingTables.length > 0 || columnMismatches.length > 0) { - const mismatch = { - v1_path: h.v1_path, - v1_version: h.v1_version, - present_tables: [...tables].sort(), - missing_tables: missingTables, - column_mismatches: columnMismatches, - scanned_at: new Date().toISOString(), - }; - fs.writeFileSync(SCHEMA_MISMATCH_PATH, safeJsonStringify(mismatch)); - - recordStep('migrate-validate', { - status: 'failed', - fields: { - MISSING_TABLES: missingTables.join(',') || 'none', - COLUMN_MISMATCHES: String(columnMismatches.length), - REPORT: SCHEMA_MISMATCH_PATH, - }, - notes: [ - missingTables.length > 0 ? `Missing tables: ${missingTables.join(', ')}` : '', - columnMismatches.length > 0 - ? `Column mismatches in: ${columnMismatches.map((c) => c.table).join(', ')}` - : '', - ].filter(Boolean), - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_VALIDATE', { - STATUS: 'failed', - REASON: 'schema_mismatch', - MISSING_TABLES: missingTables.join(',') || 'none', - COLUMN_MISMATCHES: String(columnMismatches.length), - REPORT: SCHEMA_MISMATCH_PATH, - }); - return; - } - - recordStep('migrate-validate', { - status: 'success', - fields: { - V1_GROUPS: groupCount, - V1_TASKS: taskCount, - }, - notes: [], - at: new Date().toISOString(), - }); - - emitStatus('MIGRATE_VALIDATE', { - STATUS: 'success', - V1_GROUPS: String(groupCount), - V1_TASKS: String(taskCount), - }); - } catch (err) { - db.close(); - const message = err instanceof Error ? err.message : String(err); - recordStep('migrate-validate', { - status: 'failed', - fields: { REASON: 'validate-error' }, - notes: [message], - at: new Date().toISOString(), - }); - emitStatus('MIGRATE_VALIDATE', { - STATUS: 'failed', - REASON: 'validate_error', - ERROR: message, - }); - } -} From f35be24aeff4845b9d57019e6e452c3e57271c04 Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:23:34 +0000 Subject: [PATCH 07/23] chore: move shared helpers to migrate-v2/, delete migrate-v1/ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extracted the helpers we use (JID parsing, trigger mapping, channel auth registry, generateId, v2PlatformId) into setup/migrate-v2/shared.ts. Deleted setup/migrate-v1/ entirely — no code references it anymore. Updated README, CLAUDE.md, docs/v1-to-v2-changes.md, and docs/migration-dev.md to reference the new paths and migrate-v2.sh entry point. Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 4 +- CLAUDE.md | 5 +- README.md | 2 +- docs/migration-dev.md | 2 +- docs/v1-to-v2-changes.md | 22 +- setup/migrate-v1/shared.ts | 731 ------------------------ setup/migrate-v2/channel-auth.ts | 2 +- setup/migrate-v2/db.ts | 3 +- setup/migrate-v2/shared.ts | 189 ++++++ setup/migrate-v2/tasks.ts | 2 +- 10 files changed, 210 insertions(+), 752 deletions(-) delete mode 100644 setup/migrate-v1/shared.ts create mode 100644 setup/migrate-v2/shared.ts diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 6362fe6..52d8293 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -93,7 +93,7 @@ FROM messages WHERE chat_jid = '' AND is_from_me = 0 AND sender IS NOT NULL ``` -The `sender` value is a platform handle (e.g. `6037840640` for Telegram). Build the v2 user ID by inferring the channel type from the chat JID prefix (use `parseJid` from `setup/migrate-v1/shared.ts`) and combining: `:`. +The `sender` value is a platform handle (e.g. `6037840640` for Telegram). Build the v2 user ID by inferring the channel type from the chat JID prefix (use `parseJid` from `setup/migrate-v2/shared.ts`) and combining: `:`. For each sender: 1. Upsert into `users(id, kind, display_name)` if not already present. @@ -171,7 +171,7 @@ If there are commits: 1. Show the commit list to the user. 2. `AskUserQuestion`: "How do you want to handle your v1 customizations?" - - **Copy portable items** (recommended) — copy `container/skills/*`, `.claude/skills/*`, `docs/*`. Scan each with `scanForV1Patterns` from `setup/migrate-v1/shared.ts`. + - **Copy portable items** (recommended) — copy `container/skills/*`, `.claude/skills/*`, `docs/*`. Scan each with `scanForV1Patterns` from `setup/migrate-v2/shared.ts`. - **Full walkthrough** — go commit by commit, decide together. - **Reference only** — stash to `docs/v1-fork-reference/` for later. 3. Source code (`src/*`, `container/agent-runner/src/*`) is NOT portable — v2's architecture is fundamentally different. Stash to `docs/v1-fork-reference/` with a README explaining what each file did. Don't translate. diff --git a/CLAUDE.md b/CLAUDE.md index e070bce..e65515a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,7 +77,7 @@ Exactly one writer per file — no cross-mount lock contention. Heartbeat is a f | `container/skills/` | Container skills mounted into every agent session | | `groups//` | Per-agent-group filesystem (CLAUDE.md, skills, per-group `agent-runner-src/` overlay) | | `scripts/init-first-agent.ts` | Bootstrap the first DM-wired agent (used by `/init-first-agent` skill) | -| `setup/migrate-v1.ts` + `setup/migrate-v1/` | v1→v2 migration (**experimental**); runs inside `setup/auto.ts` before channel pairing. Seeds `agent_groups` + `messaging_groups` + wirings from v1's `registered_groups`, copies group folders, merges `.env`, installs channel adapters, ports scheduled tasks. Writes `logs/setup-migration/handoff.json` for the `/migrate-from-v1` skill to pick up (owner seeding + fork customizations). | +| `migrate-v2.sh` + `setup/migrate-v2/` | v1→v2 migration. Standalone script: `bash migrate-v2.sh`. Seeds DB, copies groups/sessions, installs channels, builds container, offers service switchover, then hands off to `/migrate-from-v1` skill for owner setup and CLAUDE.md cleanup. See [docs/migration-dev.md](docs/migration-dev.md). | ## Channels and Providers (skill-installed) @@ -233,7 +233,8 @@ This project uses pnpm with `minimumReleaseAge: 4320` (3 days) in `pnpm-workspac | [docs/setup-wiring.md](docs/setup-wiring.md) | What's wired, what's open in the setup flow | | [docs/architecture-diagram.md](docs/architecture-diagram.md) | Diagram version of the architecture | | [docs/build-and-runtime.md](docs/build-and-runtime.md) | Runtime split (Node host + Bun container), lockfiles, image build surface, CI, key invariants | -| [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) | v1→v2 architecture diff — vocabulary for where v1 things moved. Referenced by the migration step and `/migrate-from-v1` skill | +| [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) | v1→v2 architecture diff — vocabulary for where v1 things moved | +| [docs/migration-dev.md](docs/migration-dev.md) | Migration development guide — testing, debugging, dev loop | ## Container Build Cache diff --git a/README.md b/README.md index 9228d40..1b485c9 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ bash nanoclaw.sh `nanoclaw.sh` walks you from a fresh machine to a named agent you can message. It installs Node, pnpm, and Docker if missing, registers your Anthropic credential with OneCLI, builds the agent container, and pairs your first channel (Telegram, Discord, WhatsApp, or a local CLI). If a step fails, Claude Code is invoked automatically to diagnose and resume from where it broke. -**Coming from v1?** `nanoclaw.sh` detects your old install (scans siblings of the v2 checkout + common `$HOME` locations) and migrates automatically; `/migrate-from-v1` in Claude finishes owner setup and helps port custom fork work. If v1 is at a non-standard path, run with `NANOCLAW_V1_PATH=/path/to/nanoclaw bash nanoclaw.sh`. See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). _Experimental — back up `data/v2.db` and `groups/` first; not recommended yet for high-stakes production installs._ +**Coming from v1?** Run `bash migrate-v2.sh` instead of `nanoclaw.sh`. It finds your v1 install (sibling directory or `NANOCLAW_V1_PATH`), migrates state, installs channels, and hands off to Claude for owner setup and custom code porting. See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) and [docs/migration-dev.md](docs/migration-dev.md). ## Philosophy diff --git a/docs/migration-dev.md b/docs/migration-dev.md index 60feb4e..35ef536 100644 --- a/docs/migration-dev.md +++ b/docs/migration-dev.md @@ -31,7 +31,7 @@ setup/migrate-v2/ channel-auth.ts # Phase 2b: copy channel auth state select-channels.ts # Phase 2a: clack multiselect switchover-prompt.ts # Service switch prompts -setup/migrate-v1/shared.ts # Shared helpers (JID parsing, trigger mapping, etc.) +setup/migrate-v2/shared.ts # Shared helpers (JID parsing, trigger mapping, etc.) .claude/skills/migrate-from-v1/ # The Claude skill logs/setup-migration/handoff.json # Written by migrate-v2.sh, read by skill logs/migrate-steps/*.log # Per-step raw output diff --git a/docs/v1-to-v2-changes.md b/docs/v1-to-v2-changes.md index f81fb58..082bf30 100644 --- a/docs/v1-to-v2-changes.md +++ b/docs/v1-to-v2-changes.md @@ -1,6 +1,6 @@ # NanoClaw v1 → v2 — what changed -Big-picture differences between NanoClaw v1 (the `~/nanoclaw` checkout you've been running) and v2 (this rewrite). Not a migration guide — that's what `bash nanoclaw.sh` and the `/migrate-from-v1` skill are for. This doc is the **vocabulary**: when something has moved or been renamed, find it here. +Big-picture differences between NanoClaw v1 (the `~/nanoclaw` checkout you've been running) and v2 (this rewrite). Not a migration guide — that's what `bash migrate-v2.sh` and the `/migrate-from-v1` skill are for. This doc is the **vocabulary**: when something has moved or been renamed, find it here. Read this before touching the migration code or porting customizations forward. @@ -37,7 +37,7 @@ Consequences: - v1 `requires_trigger=0` or pattern was `.`/`.*` → v2 `engage_mode='pattern'`, `engage_pattern='.'` (the "always" flavor) - no pattern and requires a trigger → v2 `engage_mode='mention'` - `sender_scope` and `ignored_message_policy` are new; defaults `all` / `drop` -- **JID decomposition.** v1's `jid` column stored `dc:12345` / `tg:67890`. v2 splits this into `channel_type` + `platform_id`. Concretely: `dc:12345` becomes `channel_type='discord'`, `platform_id='discord:12345'`. Prefix aliases (`dc` → `discord`, `tg` → `telegram`, `wa` → `whatsapp`) are in `setup/migrate-v1/shared.ts`. +- **JID decomposition.** v1's `jid` column stored `dc:12345` / `tg:67890`. v2 splits this into `channel_type` + `platform_id`. Concretely: `dc:12345` becomes `channel_type='discord'`, `platform_id='discord:12345'`. Prefix aliases (`dc` → `discord`, `tg` → `telegram`, `wa` → `whatsapp`) are in `setup/migrate-v2/shared.ts`. - **`channel_name` was unreliable in v1.** Many rows had it empty; the actual channel had to be guessed from the JID prefix. v2's `channel_type` is always explicit. --- @@ -99,7 +99,7 @@ Gotcha: auto-created agents default to `selective` secret mode — no secrets at Idempotent — re-running is a no-op. Pinned versions keep the supply chain honest. The automated migration detects which channels were wired in v1 (via distinct `channel_name` / JID prefix) and runs the matching `setup/install-.sh` for each. Channels in v1 that don't have a v2 skill (rare now, more common as v2 catches up) are recorded in the handoff file for the `/migrate-from-v1` skill to raise with the user. -**Channel auth beyond `.env`.** Some channels store session state on disk (Baileys WhatsApp keystore, Matrix sync state, iMessage tokens). The `channel-auth` sub-step has a per-channel registry (`setup/migrate-v1/shared.ts: CHANNEL_AUTH_REGISTRY`) that knows which file globs to copy alongside env keys. +**Channel auth beyond `.env`.** Some channels store session state on disk (Baileys WhatsApp keystore, Matrix sync state, iMessage tokens). The `channel-auth` step has a per-channel registry (`setup/migrate-v2/shared.ts: CHANNEL_AUTH_REGISTRY`) that knows which file globs to copy alongside env keys. --- @@ -162,11 +162,11 @@ Lockfiles: host uses `pnpm-lock.yaml`, agent-runner uses `bun.lock`. `minimumRel ## Migration surface — where the code lives -- `setup/migrate-v1.ts` — orchestrator called from `setup/auto.ts` between the timezone and channel steps. -- `setup/migrate-v1/.ts` — registered in `setup/index.ts` STEPS; each runs under `runQuietStep`. -- `logs/setup.log` — progression log. Each sub-step appends one entry. -- `logs/setup-steps/NN-migrate-.log` — raw per-sub-step stdout/stderr. -- `logs/setup-migration/handoff.json` — summary of what was migrated, what failed, what was deferred. Read by the `/migrate-from-v1` skill. -- `logs/setup-migration/schema-mismatch.json` — written only if `migrate-validate` finds a v1 DB that doesn't match the expected shape. The skill uses this to decide what to hand back to the user. -- `logs/setup-migration/inactive-tasks.json` — completed or stopped v1 `scheduled_tasks`, exported for reference. -- `.claude/skills/migrate-from-v1/SKILL.md` — tells Claude how to finish anything the automation couldn't and then interview the user about custom code changes. +- `migrate-v2.sh` — entry point: `bash migrate-v2.sh` from the v2 checkout. +- `setup/migrate-v2/*.ts` — individual migration steps (env, db, groups, sessions, tasks, channel-auth, select-channels, switchover-prompt). +- `setup/migrate-v2/shared.ts` — JID parsing, trigger mapping, channel auth registry. +- `logs/setup-migration/handoff.json` — written by `migrate-v2.sh`, read by the `/migrate-from-v1` skill. +- `logs/migrate-steps/*.log` — raw per-step stdout. +- `.claude/skills/migrate-from-v1/SKILL.md` — Claude skill for owner seeding, CLAUDE.md cleanup, container config validation, fork porting. +- `migrate-v2-reset.sh` — development helper to wipe v2 state for re-testing. +- See [docs/migration-dev.md](migration-dev.md) for the full development guide. diff --git a/setup/migrate-v1/shared.ts b/setup/migrate-v1/shared.ts deleted file mode 100644 index 4d2cd92..0000000 --- a/setup/migrate-v1/shared.ts +++ /dev/null @@ -1,731 +0,0 @@ -/** - * Shared types, constants, and helpers for the v1 → v2 migration. - * - * The migration is a sequence of small steps registered in setup/index.ts - * (migrate-detect, migrate-validate, migrate-db, …). Every step: - * - Reads state it needs from `logs/setup-migration/handoff.json` - * - Writes its own outcome back to that handoff file - * - Emits exactly one `=== NANOCLAW SETUP: MIGRATE_ ===` block on stdout - * - * No step aborts the chain on failure — the orchestrator in setup/migrate-v1.ts - * reads the handoff after each step to decide whether to continue, skip, or - * hand off to the Claude `/migrate-from-v1` skill. - */ -import fs from 'fs'; -import os from 'os'; -import path from 'path'; - -// ── Paths ────────────────────────────────────────────────────────────── - -export const MIGRATION_DIR = path.join('logs', 'setup-migration'); -export const HANDOFF_PATH = path.join(MIGRATION_DIR, 'handoff.json'); -export const SCHEMA_MISMATCH_PATH = path.join(MIGRATION_DIR, 'schema-mismatch.json'); -export const INACTIVE_TASKS_PATH = path.join(MIGRATION_DIR, 'inactive-tasks.json'); - -// ── V1 install discovery ─────────────────────────────────────────────── - -/** - * Default candidate paths to scan for a v1 install. Combines: - * - `$NANOCLAW_V1_PATH` (explicit override, takes priority) - * - Sibling directories of the v2 checkout whose name contains "nanoclaw" - * or "clawdbot" (most common layout — v1 lives next to v2) - * - Common checkout locations under $HOME - * - Common XDG-style state dirs (.nanoclaw, .clawdbot — v1's predecessor) - * - * Kept generic — don't bake specific usernames in. Deduped so a path that - * satisfies multiple rules only appears once. - */ -export function defaultV1Candidates(): string[] { - const home = os.homedir(); - const cwd = process.cwd(); - const cwdParent = path.dirname(cwd); - - const siblings: string[] = []; - try { - if (fs.existsSync(cwdParent)) { - for (const entry of fs.readdirSync(cwdParent, { withFileTypes: true })) { - if (!entry.isDirectory()) continue; - const lower = entry.name.toLowerCase(); - // Match anything claw-ish next to v2: "nanoclaw", "nanoclaw-v1", - // "clawdbot", user's fork name like "nanoclaw-prod". Excludes the - // v2 checkout we're running from so we don't self-match. - if (!lower.includes('claw')) continue; - const full = path.join(cwdParent, entry.name); - if (path.resolve(full) === path.resolve(cwd)) continue; - siblings.push(full); - } - } - } catch { - // Can't list parent — fall through to the fixed list. - } - - const fixed = [ - path.join(home, 'nanoclaw'), - path.join(home, '.nanoclaw'), - path.join(home, 'clawdbot'), - path.join(home, '.clawdbot'), - path.join(home, 'Code', 'nanoclaw'), - path.join(home, 'code', 'nanoclaw'), - path.join(home, 'projects', 'nanoclaw'), - path.join(home, 'Projects', 'nanoclaw'), - path.join(home, 'src', 'nanoclaw'), - path.join(home, 'dev', 'nanoclaw'), - path.join(home, 'workspace', 'nanoclaw'), - path.join(home, 'Documents', 'nanoclaw'), - path.join(home, 'GitHub', 'nanoclaw'), - path.join(home, 'github', 'nanoclaw'), - path.join(home, 'repos', 'nanoclaw'), - ]; - - // NANOCLAW_V1_PATH is handled authoritatively by detect.ts — if it's set, - // detect doesn't call this function at all. So we only build the - // auto-discovery list here. - const all = [...siblings, ...fixed]; - - // Dedupe by resolved path. A sibling "nanoclaw" and a fixed "$HOME/nanoclaw" - // often resolve to the same thing on single-user machines. - const seen = new Set(); - const out: string[] = []; - for (const p of all) { - const resolved = path.resolve(p); - if (seen.has(resolved)) continue; - seen.add(resolved); - out.push(p); - } - return out; -} - -export interface V1Paths { - root: string; - db: string; - env: string; - groups: string; - packageJson: string; -} - -/** - * Build the expected v1 file layout relative to a root. All paths are returned - * even if they don't exist — callers check existence on the ones they care about. - */ -export function v1PathsFor(root: string): V1Paths { - return { - root, - db: path.join(root, 'store', 'messages.db'), - env: path.join(root, '.env'), - groups: path.join(root, 'groups'), - packageJson: path.join(root, 'package.json'), - }; -} - -/** - * Quick "does this path look like a v1 install?" check — used by detect. - * - * Strategy: the strongest signal is `store/messages.db`, so that's required. - * The package.json check is a weaker corroboration — forks may rename - * `"name"` or strip it, so we allow: - * - `name` missing or non-string - * - `name` containing "nanoclaw" or "clawdbot" (case-insensitive) - * We reject only if `name` looks like a completely unrelated project, OR - * the version is 2.x (the v2 rewrite itself). - * - * This keeps stock + forked v1 installs detectable while filtering out - * unrelated repos that happen to have a `store/messages.db`. - */ -export function looksLikeV1Install(root: string): { ok: boolean; reason?: string } { - if (!fs.existsSync(root)) return { ok: false, reason: 'root_missing' }; - const { db, packageJson } = v1PathsFor(root); - if (!fs.existsSync(db)) return { ok: false, reason: 'db_missing' }; - - // package.json is optional — a user may have stripped it, or be running - // from a state-only dir (.nanoclaw). The DB shape is checked separately - // by migrate-validate, which is authoritative for "is this schema v1?" - if (!fs.existsSync(packageJson)) return { ok: true }; - - try { - const pkg = JSON.parse(fs.readFileSync(packageJson, 'utf-8')) as { name?: string; version?: string }; - const name = (pkg.name ?? '').toLowerCase(); - if (pkg.version && /^2\./.test(pkg.version)) return { ok: false, reason: 'already_v2' }; - if (name && !name.includes('nanoclaw') && !name.includes('clawdbot')) { - return { ok: false, reason: 'unrelated_project' }; - } - } catch { - // Broken package.json doesn't rule out v1 — DB presence is enough. - return { ok: true }; - } - return { ok: true }; -} - -// ── Handoff state (single source of truth across sub-steps) ──────────── - -/** - * Rich state shared between migration sub-steps. Each step reads the whole - * file, merges its section, and writes it back. Never hand-edit — it's - * consumed by the `/migrate-from-v1` skill too. - * - * All paths stored are ABSOLUTE, so subsequent steps don't need to guess - * about cwd. Relative paths would be a footgun once the skill reads this - * file later from a different cwd. - */ -export interface Handoff { - version: 1; - started_at: string; - v1_path: string | null; - v1_version: string | null; - - /** Overall status once migrate-handoff finalizes the run. */ - overall_status: 'pending' | 'success' | 'partial' | 'failed' | 'skipped'; - - steps: Partial>; - - /** Group folders the user chose to bring over (migrate-db populates). */ - group_selection: { - mode: 'all' | 'wired-only' | 'cancelled' | null; - selected_folders: string[]; - total_v1_groups: number; - wired_v1_groups: number; - }; - - /** Distinct channels inferred from v1 registered_groups. */ - detected_channels: Array<{ - channel_type: string; - source: 'channel_name' | 'jid_prefix'; - group_count: number; - }>; - - /** Per-channel auth copy results (migrate-channel-auth populates). */ - channel_auth: Array<{ - channel_type: string; - env_keys_copied: string[]; - files_copied: string[]; - files_missing: string[]; - notes: string; - }>; - - /** Result of each `setup/install-.sh` invocation. */ - channels_installed: Array<{ - channel_type: string; - status: 'success' | 'failed' | 'skipped' | 'not_supported'; - error?: string; - }>; - - /** Scheduled-task migration results (migrate-tasks populates). */ - tasks: { - v1_active: number; - v1_inactive: number; - migrated: number; - failed: number; - skipped: number; - }; - - /** Things the skill must finish manually. Always safe to append to. */ - followups: string[]; -} - -export type MigrateStep = - | 'migrate-detect' - | 'migrate-validate' - | 'migrate-db' - | 'migrate-groups' - | 'migrate-env' - | 'migrate-channel-auth' - | 'migrate-channels' - | 'migrate-tasks' - | 'migrate-handoff'; - -export interface StepOutcome { - status: 'success' | 'partial' | 'failed' | 'skipped'; - fields: Record; - notes: string[]; - at: string; -} - -function emptyHandoff(): Handoff { - return { - version: 1, - started_at: new Date().toISOString(), - v1_path: null, - v1_version: null, - overall_status: 'pending', - steps: {}, - group_selection: { - mode: null, - selected_folders: [], - total_v1_groups: 0, - wired_v1_groups: 0, - }, - detected_channels: [], - channel_auth: [], - channels_installed: [], - tasks: { v1_active: 0, v1_inactive: 0, migrated: 0, failed: 0, skipped: 0 }, - followups: [], - }; -} - -/** Read the handoff, creating an empty one if it doesn't exist yet. */ -export function readHandoff(): Handoff { - fs.mkdirSync(MIGRATION_DIR, { recursive: true }); - if (!fs.existsSync(HANDOFF_PATH)) return emptyHandoff(); - try { - const parsed = JSON.parse(fs.readFileSync(HANDOFF_PATH, 'utf-8')) as Handoff; - if (parsed.version !== 1) throw new Error(`unsupported handoff version ${parsed.version}`); - return parsed; - } catch { - // Broken handoff shouldn't wedge the migration — start fresh and let the - // step that called us re-record its outcome. - return emptyHandoff(); - } -} - -/** Persist a handoff mutation atomically (write-tmp + rename). */ -export function writeHandoff(h: Handoff): void { - fs.mkdirSync(MIGRATION_DIR, { recursive: true }); - const tmp = HANDOFF_PATH + '.tmp'; - fs.writeFileSync(tmp, JSON.stringify(h, null, 2)); - fs.renameSync(tmp, HANDOFF_PATH); -} - -/** Convenience: merge a step outcome into the handoff and persist. */ -export function recordStep(step: MigrateStep, outcome: StepOutcome): void { - const h = readHandoff(); - h.steps[step] = outcome; - writeHandoff(h); -} - -// ── JID parsing + channel inference ──────────────────────────────────── - -/** - * v1 stored chat identifiers as `:` in `registered_groups.jid`. - * The prefix was often a short code (`dc` for Discord, `tg` for Telegram) - * that doesn't match v2's `channel_type` names. This table normalizes them. - * - * Unknown prefixes fall through as-is (`channel_type = prefix`) so a channel - * we didn't anticipate still ends up with a distinct messaging_group per - * chat — the skill can reconcile it interactively. - */ -export const JID_PREFIX_TO_CHANNEL: Record = { - dc: 'discord', - discord: 'discord', - tg: 'telegram', - telegram: 'telegram', - wa: 'whatsapp', - whatsapp: 'whatsapp', - slack: 'slack', - matrix: 'matrix', - mx: 'matrix', - teams: 'teams', - imessage: 'imessage', - im: 'imessage', - email: 'email', - webex: 'webex', - gchat: 'gchat', - linear: 'linear', - github: 'github', -}; - -export interface ParsedJid { - raw: string; - prefix: string; - id: string; - channel_type: string; -} - -export function parseJid(raw: string): ParsedJid | null { - const colon = raw.indexOf(':'); - if (colon === -1) return null; - const prefix = raw.slice(0, colon).toLowerCase(); - const id = raw.slice(colon + 1); - if (!prefix || !id) return null; - return { - raw, - prefix, - id, - channel_type: JID_PREFIX_TO_CHANNEL[prefix] ?? prefix, - }; -} - -/** - * Prefer an explicit v1 `channel_name` when one is set; fall back to the JID - * prefix. v1 left `channel_name` empty on most rows (it was a late addition), - * so the JID prefix is often the only honest source. - */ -export function inferChannelType(jid: string, channelName: string | null): string | null { - if (channelName && channelName.trim()) return channelName.trim(); - const parsed = parseJid(jid); - return parsed?.channel_type ?? null; -} - -/** - * v2's messaging_groups.platform_id is always prefixed with the channel_type - * (see setup/register.ts:118-120). This helper normalizes v1's `jid` into - * that shape so router lookups at runtime find the right row. - * - * Some channels need extra structure on the id itself. Discord's Chat SDK - * emits `discord::` at runtime but v1 only stored - * `dc:` (no guild). Callers that know the guild (e.g. bot with - * a single guild) can pass it via `extra`; otherwise the returned id will - * be the v1-format `discord:` and will be repaired on first - * message via v2's channel-registration approval flow. - */ -export function v2PlatformId(channelType: string, jid: string, extra?: { guildId?: string }): string { - const parsed = parseJid(jid); - const id = parsed?.id ?? jid; - const prefixed = id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; - // For Discord: splice the guild id in between when we know it and the id - // isn't already in `:` form. - if (channelType === 'discord' && extra?.guildId) { - const body = prefixed.slice(`discord:`.length); - if (!body.includes(':')) return `discord:${extra.guildId}:${body}`; - } - return prefixed; -} - -/** - * Fetch the bot's guild memberships for a channel_type so migrate-db can - * form platform_ids matching what the v2 adapter emits at runtime. Returns - * null on any failure (network, auth, rate limit, unsupported channel_type) - * — callers fall back to the v1-format platform_id, which works but may - * trigger v2's channel-registration flow on first message. - * - * Currently handles Discord. Extending to other channels: the function - * needs a "single-or-multi guild?" shape; for single-guild bots the caller - * can splice the guild id globally, for multi-guild a per-channel lookup - * is needed and the caller should probably bail (rate-limit risk). - */ -export async function fetchBotGuilds( - channelType: string, - v1EnvLookup: (key: string) => string | undefined, -): Promise<{ guildIds: string[] } | null> { - if (channelType !== 'discord') return null; - const token = v1EnvLookup('DISCORD_BOT_TOKEN'); - if (!token) return null; - try { - const resp = await fetch('https://discord.com/api/v10/users/@me/guilds', { - headers: { Authorization: `Bot ${token}` }, - }); - if (!resp.ok) return null; - const data = (await resp.json()) as Array<{ id?: string }>; - const guildIds = data.map((g) => g.id).filter((id): id is string => typeof id === 'string'); - return { guildIds }; - } catch { - return null; - } -} - -// ── Trigger rules → engage mode (ports migration 010's backfill) ─────── - -/** - * Mirrors the backfill() logic in src/db/migrations/010-engage-modes.ts so - * rows written by the migration land in the same shape as rows written by - * setup/register.ts (which goes through migration 010 at boot). - */ -export function triggerToEngage(input: { - trigger_pattern: string | null; - requires_trigger: number | null; -}): { - engage_mode: 'pattern' | 'mention' | 'mention-sticky'; - engage_pattern: string | null; -} { - const pattern = input.trigger_pattern && input.trigger_pattern.trim().length > 0 ? input.trigger_pattern : null; - const requiresTrigger = input.requires_trigger !== 0; // NULL/1 → true; 0 → false - - if (pattern === '.' || pattern === '.*') { - return { engage_mode: 'pattern', engage_pattern: '.' }; - } - // requires_trigger=0 means "respond to everything" regardless of pattern. - // The pattern was used for mention highlighting, not message gating. - if (!requiresTrigger) { - return { engage_mode: 'pattern', engage_pattern: '.' }; - } - if (pattern) { - return { engage_mode: 'pattern', engage_pattern: pattern }; - } - return { engage_mode: 'mention', engage_pattern: null }; -} - -// ── Channel auth registry (non-.env state per channel) ───────────────── - -/** - * Describes the auth surface for a channel beyond `.env`. Each entry tells - * the channel-auth step: - * - * - `v1EnvKeys`: env keys we might find on the v1 side and carry over - * - `requiredV2Keys`: env keys v2's adapter REQUIRES to boot — if missing - * from v2's merged .env after migrate-env runs, a followup is emitted so - * the user knows exactly what to add (and where to get it). - * - `candidatePaths`: relative paths under the v1 root that may hold - * on-disk auth state (WhatsApp keystore, matrix sync state, etc.) - * - `note`: short human-readable hint surfaced to the user - * - * Unknown channels fall through as {v1EnvKeys:[], requiredV2Keys:[], - * candidatePaths:[]} — the skill asks the user how to proceed. - * - * Keep `requiredV2Keys` honest: list only what the v2 adapter actually - * refuses to boot without. False positives spam the followups; false - * negatives let the agent silently fail. Verify against the actual - * `@chat-adapter/` package when adding/updating entries. - */ -export interface ChannelAuthSpec { - v1EnvKeys: string[]; - requiredV2Keys: { key: string; where: string }[]; - candidatePaths: string[]; - note?: string; -} - -export const CHANNEL_AUTH_REGISTRY: Record = { - discord: { - v1EnvKeys: ['DISCORD_BOT_TOKEN', 'DISCORD_CLIENT_ID', 'DISCORD_GUILD_ID'], - // v1 used raw discord.js (bot token only). v2 uses Chat SDK which needs - // the interaction-verification public key + application id on top. - requiredV2Keys: [ - { key: 'DISCORD_BOT_TOKEN', where: 'Discord Developer Portal → Application → Bot → Token' }, - { key: 'DISCORD_APPLICATION_ID', where: 'Discord Developer Portal → Application → General → Application ID' }, - { key: 'DISCORD_PUBLIC_KEY', where: 'Discord Developer Portal → Application → General → Public Key' }, - ], - candidatePaths: [], - note: 'v1 used raw discord.js (bot token only). v2 uses Chat SDK and needs APPLICATION_ID + PUBLIC_KEY too.', - }, - 'discord-supervisor': { - v1EnvKeys: ['DISCORD_SUPERVISOR_BOT_TOKEN'], - requiredV2Keys: [], - candidatePaths: [], - note: 'v1-specific secondary bot. v2 does not have a native supervisor channel; the token is preserved in .env for the skill to reconcile.', - }, - telegram: { - v1EnvKeys: ['TELEGRAM_BOT_TOKEN', 'TELEGRAM_API_ID', 'TELEGRAM_API_HASH'], - requiredV2Keys: [ - { key: 'TELEGRAM_BOT_TOKEN', where: 'BotFather on Telegram → /mybots → Bot → API Token' }, - ], - candidatePaths: ['data/sessions/telegram', 'store/telegram-session'], - }, - whatsapp: { - v1EnvKeys: ['WHATSAPP_PHONE', 'WHATSAPP_OWNER'], - requiredV2Keys: [], - candidatePaths: [ - 'data/sessions/baileys', - 'data/baileys_auth', - 'store/auth_info_baileys', - 'store/baileys', - 'auth_info_baileys', - ], - note: 'Baileys keystore — copying is best-effort. Encryption sessions may still need a fresh pair via /add-whatsapp.', - }, - matrix: { - v1EnvKeys: ['MATRIX_HOMESERVER', 'MATRIX_USER_ID', 'MATRIX_ACCESS_TOKEN'], - requiredV2Keys: [ - { key: 'MATRIX_HOMESERVER', where: 'your Matrix homeserver URL (e.g. https://matrix.org)' }, - { key: 'MATRIX_ACCESS_TOKEN', where: 'Element → Settings → Help & About → Access Token (keep secret)' }, - ], - candidatePaths: ['data/matrix-store', 'store/matrix', 'data/sessions/matrix'], - }, - slack: { - v1EnvKeys: ['SLACK_BOT_TOKEN', 'SLACK_APP_TOKEN', 'SLACK_SIGNING_SECRET'], - requiredV2Keys: [ - { key: 'SLACK_BOT_TOKEN', where: 'Slack app → OAuth & Permissions → Bot User OAuth Token (xoxb-…)' }, - { key: 'SLACK_SIGNING_SECRET', where: 'Slack app → Basic Information → Signing Secret' }, - ], - candidatePaths: [], - }, - teams: { - v1EnvKeys: ['TEAMS_APP_ID', 'TEAMS_APP_PASSWORD', 'TEAMS_TENANT_ID'], - requiredV2Keys: [ - { key: 'TEAMS_APP_ID', where: 'Azure portal → App registration → Application (client) ID' }, - { key: 'TEAMS_APP_PASSWORD', where: 'Azure portal → App registration → Certificates & secrets' }, - ], - candidatePaths: [], - }, - imessage: { - v1EnvKeys: ['IMESSAGE_PHOTON_URL', 'IMESSAGE_PHOTON_TOKEN'], - requiredV2Keys: [], - candidatePaths: ['data/imessage', 'store/imessage'], - }, - webex: { - v1EnvKeys: ['WEBEX_BOT_TOKEN'], - requiredV2Keys: [{ key: 'WEBEX_BOT_TOKEN', where: 'Webex developer portal → Bot → Bot Access Token' }], - candidatePaths: [], - }, - gchat: { - v1EnvKeys: ['GCHAT_SERVICE_ACCOUNT', 'GCHAT_WEBHOOK_URL'], - requiredV2Keys: [], - candidatePaths: ['data/gchat-credentials.json', 'store/gchat-sa.json'], - }, - resend: { - v1EnvKeys: ['RESEND_API_KEY', 'RESEND_FROM'], - requiredV2Keys: [{ key: 'RESEND_API_KEY', where: 'resend.com → API Keys' }], - candidatePaths: [], - }, - github: { - v1EnvKeys: ['GITHUB_WEBHOOK_SECRET', 'GITHUB_APP_ID', 'GITHUB_PRIVATE_KEY_PATH'], - requiredV2Keys: [], - candidatePaths: [], - note: 'Webhook channel — secrets carry over, but GitHub webhook URLs are new per v2 install.', - }, - linear: { - v1EnvKeys: ['LINEAR_API_KEY', 'LINEAR_WEBHOOK_SECRET'], - requiredV2Keys: [{ key: 'LINEAR_API_KEY', where: 'Linear → Settings → API → Personal API keys' }], - candidatePaths: [], - }, -}; - -/** - * For channels where v2's adapter needs keys v1 never stored (e.g. Discord's - * Chat SDK wants DISCORD_APPLICATION_ID + DISCORD_PUBLIC_KEY, but v1 used - * raw discord.js with just the bot token), try to derive the missing keys - * from the v1 creds we already have by calling the channel's API. - * - * Returns a map of key → value for what we successfully resolved. - * Never throws; returns `{}` on any failure (network, auth, unexpected - * shape). The caller writes the resolved keys to v2 .env, then re-checks - * `requiredV2Keys` so the step reports `success` instead of `partial` when - * auto-resolution covered the gap. - * - * Adding a new channel resolver: pull the needed values from an endpoint - * that accepts only the v1-side credential (bot token, API key). Don't - * prompt, don't log values. If the endpoint has rate limits, keep this - * best-effort and fail silently. - */ -export async function autoResolveV2Keys( - channelType: string, - v1EnvLookup: (key: string) => string | undefined, -): Promise> { - if (channelType === 'discord') { - const token = v1EnvLookup('DISCORD_BOT_TOKEN'); - if (!token) return {}; - try { - const resp = await fetch('https://discord.com/api/v10/oauth2/applications/@me', { - headers: { Authorization: `Bot ${token}` }, - }); - if (!resp.ok) return {}; - const data = (await resp.json()) as { id?: string; verify_key?: string }; - const out: Record = {}; - if (typeof data.id === 'string' && data.id) out.DISCORD_APPLICATION_ID = data.id; - if (typeof data.verify_key === 'string' && data.verify_key) { - out.DISCORD_PUBLIC_KEY = data.verify_key; - } - return out; - } catch { - return {}; - } - } - - return {}; -} - -/** - * Map a v2 `channel_type` name to the corresponding `setup/install-.sh` - * script, if one exists. `null` means no v2 skill is available yet — the - * handoff lists the channel as "not supported" and the skill raises it with - * the user. - */ -export function installScriptForChannel(channelType: string): string | null { - const known = new Set([ - 'discord', - 'telegram', - 'whatsapp', - 'whatsapp-cloud', - 'teams', - 'slack', - 'matrix', - 'imessage', - 'webex', - 'gchat', - 'resend', - 'github', - 'linear', - ]); - if (!known.has(channelType)) return null; - return `setup/install-${channelType}.sh`; -} - -// ── Misc helpers ─────────────────────────────────────────────────────── - -export function generateId(prefix: string): string { - return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; -} - -// ── v1-specific pattern scan (for migrate-groups) ────────────────────── - -/** - * Tight set of v1-only infrastructure patterns. When one of these shows up - * in a copied CLAUDE.md, the content referencing v1 plumbing that is genuinely - * gone in v2 (IPC file queue, single-DB paths, v1 pr-factory conventions). - * - * Deliberately excludes portable patterns — `mcp__nanoclaw__*` tool names, - * `agent-browser`, generic `/workspace/` paths — which v2 supports the same - * way. The list is scan-only; the migration does NOT modify file content. It - * just adds a followup so the /migrate-from-v1 skill can triage each file - * with the user. - * - * Keep this list conservative: false positives spam the skill with noise, - * false negatives leave the user with silently-broken agents. When adding, - * include a comment naming the specific v1 thing each pattern points at. - */ -export interface V1PatternMatch { - pattern: string; - description: string; - lines: number[]; -} - -const V1_PATTERNS: Array<{ pattern: RegExp; description: string }> = [ - { - pattern: /\/workspace\/ipc\/tasks/, - description: "v1 IPC file queue (gone in v2 — agents talk to the host via session DBs, not JSON files)", - }, - { - pattern: /\/workspace\/extra\/project\/store\b/, - description: "v1-specific mount + store/ path (v2 mounts differ; state lives under data/)", - }, - { - pattern: /\bstore\/messages\.db\b/, - description: "v1 central DB path (v2 uses data/v2.db + data/v2-sessions//{inbound,outbound}.db)", - }, - { - pattern: /"clear_session"|"retrigger"/, - description: "v1 IPC task types (no v2 equivalent; use session lifecycle + the scheduling MCP tool instead)", - }, - { - pattern: /\[PR_CONTEXT:/, - description: "v1 pr-factory context-tag convention (specific to the supervisor group; needs reworking in v2)", - }, - { - pattern: /\brequires_trigger\b|\btrigger_pattern\b/, - description: "v1 column names on registered_groups (v2 uses engage_mode + engage_pattern on messaging_group_agents)", - }, - { - pattern: /\bchatJid\b(?!\s*[:=]\s*["']dc:)/, - description: "v1 routing key (v2 uses messaging_group_id or channel_type+platform_id)", - }, -]; - -/** Scan a CLAUDE.md-ish text blob for v1-specific infrastructure patterns. */ -export function scanForV1Patterns(text: string): V1PatternMatch[] { - const matches: V1PatternMatch[] = []; - const lines = text.split('\n'); - - for (const entry of V1_PATTERNS) { - const hitLines: number[] = []; - for (let i = 0; i < lines.length; i++) { - if (entry.pattern.test(lines[i])) { - hitLines.push(i + 1); - } - } - if (hitLines.length > 0) { - matches.push({ - pattern: entry.pattern.source, - description: entry.description, - // Cap to first 5 line numbers — we're generating a followup summary, - // not a code index. Full context is in the file itself. - lines: hitLines.slice(0, 5), - }); - } - } - - return matches; -} - -export function safeJsonStringify(value: unknown): string { - try { - return JSON.stringify(value, null, 2); - } catch { - return '{"error":"unserializable"}'; - } -} diff --git a/setup/migrate-v2/channel-auth.ts b/setup/migrate-v2/channel-auth.ts index 788ae9d..05d706d 100644 --- a/setup/migrate-v2/channel-auth.ts +++ b/setup/migrate-v2/channel-auth.ts @@ -10,7 +10,7 @@ import fs from 'fs'; import path from 'path'; -import { CHANNEL_AUTH_REGISTRY } from '../migrate-v1/shared.js'; +import { CHANNEL_AUTH_REGISTRY } from './shared.js'; function parseEnv(filePath: string): Map { const out = new Map(); diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts index 141b267..f33ec2b 100644 --- a/setup/migrate-v2/db.ts +++ b/setup/migrate-v2/db.ts @@ -29,8 +29,7 @@ import { generateId, parseJid, triggerToEngage, - JID_PREFIX_TO_CHANNEL, -} from '../migrate-v1/shared.js'; +} from './shared.js'; interface V1Group { jid: string; diff --git a/setup/migrate-v2/shared.ts b/setup/migrate-v2/shared.ts new file mode 100644 index 0000000..62f2236 --- /dev/null +++ b/setup/migrate-v2/shared.ts @@ -0,0 +1,189 @@ +/** + * Shared helpers for the v1 → v2 migration steps. + */ + +// ── JID parsing ───────────────────────────────────────────────────────── + +/** v1 JID prefix → v2 channel_type. Unknown prefixes pass through as-is. */ +export const JID_PREFIX_TO_CHANNEL: Record = { + dc: 'discord', + discord: 'discord', + tg: 'telegram', + telegram: 'telegram', + wa: 'whatsapp', + whatsapp: 'whatsapp', + slack: 'slack', + matrix: 'matrix', + mx: 'matrix', + teams: 'teams', + imessage: 'imessage', + im: 'imessage', + email: 'email', + webex: 'webex', + gchat: 'gchat', + linear: 'linear', + github: 'github', +}; + +export interface ParsedJid { + raw: string; + prefix: string; + id: string; + channel_type: string; +} + +export function parseJid(raw: string): ParsedJid | null { + const colon = raw.indexOf(':'); + if (colon === -1) return null; + const prefix = raw.slice(0, colon).toLowerCase(); + const id = raw.slice(colon + 1); + if (!prefix || !id) return null; + return { + raw, + prefix, + id, + channel_type: JID_PREFIX_TO_CHANNEL[prefix] ?? prefix, + }; +} + +/** + * Build a v2 platform_id from a v1 JID. v2's messaging_groups.platform_id + * is always `:`. + */ +export function v2PlatformId(channelType: string, jid: string): string { + const parsed = parseJid(jid); + const id = parsed?.id ?? jid; + return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; +} + +// ── Trigger mapping ───────────────────────────────────────────────────── + +/** + * Map v1's trigger_pattern + requires_trigger to v2's engage_mode + engage_pattern. + * + * Key rule: requires_trigger=0 means "respond to everything" regardless + * of the pattern value. The pattern was for mention highlighting, not gating. + */ +export function triggerToEngage(input: { + trigger_pattern: string | null; + requires_trigger: number | null; +}): { + engage_mode: 'pattern' | 'mention' | 'mention-sticky'; + engage_pattern: string | null; +} { + const pattern = input.trigger_pattern && input.trigger_pattern.trim().length > 0 ? input.trigger_pattern : null; + const requiresTrigger = input.requires_trigger !== 0; + + if (pattern === '.' || pattern === '.*') { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + if (!requiresTrigger) { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + if (pattern) { + return { engage_mode: 'pattern', engage_pattern: pattern }; + } + return { engage_mode: 'mention', engage_pattern: null }; +} + +// ── ID generation ─────────────────────────────────────────────────────── + +export function generateId(prefix: string): string { + return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; +} + +// ── Channel auth registry ─────────────────────────────────────────────── + +export interface ChannelAuthSpec { + v1EnvKeys: string[]; + requiredV2Keys: { key: string; where: string }[]; + candidatePaths: string[]; + note?: string; +} + +export const CHANNEL_AUTH_REGISTRY: Record = { + discord: { + v1EnvKeys: ['DISCORD_BOT_TOKEN', 'DISCORD_CLIENT_ID', 'DISCORD_GUILD_ID'], + requiredV2Keys: [ + { key: 'DISCORD_BOT_TOKEN', where: 'Discord Developer Portal → Application → Bot → Token' }, + { key: 'DISCORD_APPLICATION_ID', where: 'Discord Developer Portal → Application → General → Application ID' }, + { key: 'DISCORD_PUBLIC_KEY', where: 'Discord Developer Portal → Application → General → Public Key' }, + ], + candidatePaths: [], + note: 'v1 used raw discord.js (bot token only). v2 uses Chat SDK and needs APPLICATION_ID + PUBLIC_KEY too.', + }, + telegram: { + v1EnvKeys: ['TELEGRAM_BOT_TOKEN', 'TELEGRAM_API_ID', 'TELEGRAM_API_HASH'], + requiredV2Keys: [ + { key: 'TELEGRAM_BOT_TOKEN', where: 'BotFather on Telegram → /mybots → Bot → API Token' }, + ], + candidatePaths: ['data/sessions/telegram', 'store/telegram-session'], + }, + whatsapp: { + v1EnvKeys: ['WHATSAPP_PHONE', 'WHATSAPP_OWNER'], + requiredV2Keys: [], + candidatePaths: [ + 'data/sessions/baileys', + 'data/baileys_auth', + 'store/auth_info_baileys', + 'store/baileys', + 'auth_info_baileys', + ], + note: 'Baileys keystore — copying is best-effort. Encryption sessions may still need a fresh pair via /add-whatsapp.', + }, + matrix: { + v1EnvKeys: ['MATRIX_HOMESERVER', 'MATRIX_USER_ID', 'MATRIX_ACCESS_TOKEN'], + requiredV2Keys: [ + { key: 'MATRIX_HOMESERVER', where: 'your Matrix homeserver URL (e.g. https://matrix.org)' }, + { key: 'MATRIX_ACCESS_TOKEN', where: 'Element → Settings → Help & About → Access Token (keep secret)' }, + ], + candidatePaths: ['data/matrix-store', 'store/matrix', 'data/sessions/matrix'], + }, + slack: { + v1EnvKeys: ['SLACK_BOT_TOKEN', 'SLACK_APP_TOKEN', 'SLACK_SIGNING_SECRET'], + requiredV2Keys: [ + { key: 'SLACK_BOT_TOKEN', where: 'Slack app → OAuth & Permissions → Bot User OAuth Token (xoxb-…)' }, + { key: 'SLACK_SIGNING_SECRET', where: 'Slack app → Basic Information → Signing Secret' }, + ], + candidatePaths: [], + }, + teams: { + v1EnvKeys: ['TEAMS_APP_ID', 'TEAMS_APP_PASSWORD', 'TEAMS_TENANT_ID'], + requiredV2Keys: [ + { key: 'TEAMS_APP_ID', where: 'Azure portal → App registration → Application (client) ID' }, + { key: 'TEAMS_APP_PASSWORD', where: 'Azure portal → App registration → Certificates & secrets' }, + ], + candidatePaths: [], + }, + imessage: { + v1EnvKeys: ['IMESSAGE_PHOTON_URL', 'IMESSAGE_PHOTON_TOKEN'], + requiredV2Keys: [], + candidatePaths: ['data/imessage', 'store/imessage'], + }, + webex: { + v1EnvKeys: ['WEBEX_BOT_TOKEN'], + requiredV2Keys: [{ key: 'WEBEX_BOT_TOKEN', where: 'Webex developer portal → Bot → Bot Access Token' }], + candidatePaths: [], + }, + gchat: { + v1EnvKeys: ['GCHAT_SERVICE_ACCOUNT', 'GCHAT_WEBHOOK_URL'], + requiredV2Keys: [], + candidatePaths: ['data/gchat-credentials.json', 'store/gchat-sa.json'], + }, + resend: { + v1EnvKeys: ['RESEND_API_KEY', 'RESEND_FROM'], + requiredV2Keys: [{ key: 'RESEND_API_KEY', where: 'resend.com → API Keys' }], + candidatePaths: [], + }, + github: { + v1EnvKeys: ['GITHUB_WEBHOOK_SECRET', 'GITHUB_APP_ID', 'GITHUB_PRIVATE_KEY_PATH'], + requiredV2Keys: [], + candidatePaths: [], + note: 'Webhook channel — secrets carry over, but GitHub webhook URLs are new per v2 install.', + }, + linear: { + v1EnvKeys: ['LINEAR_API_KEY', 'LINEAR_WEBHOOK_SECRET'], + requiredV2Keys: [{ key: 'LINEAR_API_KEY', where: 'Linear → Settings → API → Personal API keys' }], + candidatePaths: [], + }, +}; diff --git a/setup/migrate-v2/tasks.ts b/setup/migrate-v2/tasks.ts index 9ba570a..6a7efbe 100644 --- a/setup/migrate-v2/tasks.ts +++ b/setup/migrate-v2/tasks.ts @@ -22,7 +22,7 @@ import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; import { runMigrations } from '../../src/db/migrations/index.js'; import { insertTask } from '../../src/modules/scheduling/db.js'; import { openInboundDb, resolveSession } from '../../src/session-manager.js'; -import { parseJid, v2PlatformId } from '../migrate-v1/shared.js'; +import { parseJid, v2PlatformId } from './shared.js'; interface V1Task { id: string; From 00a30e3effcf24178b1636c9b5c0e5ecd39c86aa Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:24:39 +0000 Subject: [PATCH 08/23] docs: update changelog, remove experimental label from migration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The migration is no longer experimental — it's been tested end-to-end with service switchover, session continuity, and revert. Updated the changelog entry to reflect the new migrate-v2.sh flow. Co-Authored-By: Claude Opus 4.6 (1M context) --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cf6992b..e297d3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ For detailed release notes, see the [full changelog on the documentation site](h ## [Unreleased] -- **v1 → v2 migration (experimental).** `bash nanoclaw.sh` now detects a v1 install (`~/nanoclaw`, `~/.nanoclaw`, siblings of the v2 checkout, or `$NANOCLAW_V1_PATH`) and runs a best-effort port before channel pairing: seeds `agent_groups`/`messaging_groups`/wirings from v1's `registered_groups` (with trigger_pattern → engage_mode/engage_pattern), copies group folders, merges `.env`, installs v2 channel adapters, and ports `scheduled_tasks`. Hands off to `/migrate-from-v1` for owner seeding and fork customizations. Experimental — back up `data/v2.db` and `groups/` first; see [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). +- **v1 → v2 migration.** Run `bash migrate-v2.sh` from the v2 checkout. Finds your v1 install (sibling directory or `NANOCLAW_V1_PATH`), merges `.env`, seeds the v2 DB from `registered_groups`, copies group folders (`CLAUDE.md` → `CLAUDE.local.md`), copies session data with conversation continuity, ports scheduled tasks, interactively selects and installs channels (clack multiselect), copies container skills, builds the agent container, and offers a service switchover to test. Hands off to Claude (`/migrate-from-v1`) for owner seeding, access policy, CLAUDE.md cleanup, and fork customization porting. See [docs/migration-dev.md](docs/migration-dev.md) and [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). ## [2.0.0] - 2026-04-22 From cf3fcc18d45fe9f73d30a5dd9c9dc8b1fbf98842 Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:28:04 +0000 Subject: [PATCH 09/23] fix: install Docker if missing, don't skip container build migrate-v2.sh now runs setup/install-docker.sh when Docker isn't found instead of just printing a message. The container build step reports failure (not skip) when Docker is unavailable so the skill can triage it. Co-Authored-By: Claude Opus 4.6 (1M context) --- migrate-v2.sh | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/migrate-v2.sh b/migrate-v2.sh index 325b491..edae888 100644 --- a/migrate-v2.sh +++ b/migrate-v2.sh @@ -420,15 +420,24 @@ else log "Anthropic credential: skipped (no OneCLI)" fi -# 3c. Docker check +# 3c. Docker — install if missing if command -v docker >/dev/null 2>&1; then DOCKER_V=$(docker --version 2>/dev/null | head -1) step_ok "Docker available $(dim "($DOCKER_V)")" log "Docker: $DOCKER_V" else - step_fail "Docker not found" - step_info "$(dim "Install Docker: bash setup/install-docker.sh")" - log "Docker: not found" + step_info "Installing Docker…" + DOCKER_LOG="$STEPS_DIR/3c-docker.log" + if bash setup/install-docker.sh > "$DOCKER_LOG" 2>&1; then + hash -r 2>/dev/null || true + step_ok "Docker installed" + STEP_RESULTS["3c-docker"]="success" + log "Docker: installed" + else + step_fail "Docker install failed $(dim "(see $DOCKER_LOG)")" + STEP_RESULTS["3c-docker"]="failed" + log "Docker: FAILED" + fi fi # 3d. Copy container skills from v1 that v2 doesn't have @@ -464,16 +473,20 @@ if command -v docker >/dev/null 2>&1; then BUILD_LOG="$STEPS_DIR/3e-container-build.log" if bash container/build.sh > "$BUILD_LOG" 2>&1; then step_ok "Container image built" + STEP_RESULTS["3e-build"]="success" log "Container build: success" else step_fail "Container build failed" + STEP_RESULTS["3e-build"]="failed" tail -10 "$BUILD_LOG" 2>/dev/null | while IFS= read -r line; do echo " $(dim "$line")" done log "Container build: FAILED (see $BUILD_LOG)" fi else - step_skip "Container build $(dim "(no Docker)")" + step_fail "Docker not available — cannot build container" + STEP_RESULTS["3e-build"]="failed" + log "Container build: skipped (no Docker)" fi echo From ce9f17523856359636154fdd6cfed58e086758c6 Mon Sep 17 00:00:00 2001 From: "exe.dev user" Date: Fri, 1 May 2026 20:28:45 +0000 Subject: [PATCH 10/23] =?UTF-8?q?fix:=20reorder=20phase=203=20=E2=80=94=20?= =?UTF-8?q?Docker=20before=20OneCLI?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OneCLI runs in a Docker container, so Docker must be installed first. Reordered: Docker (3a) → OneCLI (3b) → Auth (3c) → Skills (3d) → Build (3e). OneCLI install now skips with a clear message if Docker isn't available. Co-Authored-By: Claude Opus 4.6 (1M context) --- migrate-v2.sh | 67 +++++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 32 deletions(-) diff --git a/migrate-v2.sh b/migrate-v2.sh index edae888..d790c32 100644 --- a/migrate-v2.sh +++ b/migrate-v2.sh @@ -372,7 +372,27 @@ echo echo "$(bold 'Phase 3: Infrastructure')" echo -# 3a. OneCLI — detect or install via setup step +# 3a. Docker — install if missing (OneCLI needs it) +if command -v docker >/dev/null 2>&1; then + DOCKER_V=$(docker --version 2>/dev/null | head -1) + step_ok "Docker available $(dim "($DOCKER_V)")" + log "Docker: $DOCKER_V" +else + step_info "Installing Docker…" + DOCKER_LOG="$STEPS_DIR/3a-docker.log" + if bash setup/install-docker.sh > "$DOCKER_LOG" 2>&1; then + hash -r 2>/dev/null || true + step_ok "Docker installed" + STEP_RESULTS["3a-docker"]="success" + log "Docker: installed" + else + step_fail "Docker install failed $(dim "(see $DOCKER_LOG)")" + STEP_RESULTS["3a-docker"]="failed" + log "Docker: FAILED" + fi +fi + +# 3b. OneCLI — detect or install via setup step (requires Docker) ONECLI_OK=false ONECLI_URL_FROM_ENV=$(grep '^ONECLI_URL=' .env 2>/dev/null | head -1 | sed 's/^ONECLI_URL=//') ONECLI_URL_CHECK="${ONECLI_URL_FROM_ENV:-http://127.0.0.1:10254}" @@ -381,38 +401,41 @@ if curl -sf "${ONECLI_URL_CHECK}/health" >/dev/null 2>&1; then step_ok "OneCLI running at $(dim "$ONECLI_URL_CHECK")" ONECLI_OK=true log "OneCLI: running at $ONECLI_URL_CHECK" -else - # Run the setup onecli step — it handles install, reuse, and health checks +elif command -v docker >/dev/null 2>&1; then step_info "Setting up OneCLI…" - ONECLI_LOG="$STEPS_DIR/3a-onecli.log" - ONECLI_ERR="$STEPS_DIR/3a-onecli.err" + ONECLI_LOG="$STEPS_DIR/3b-onecli.log" + ONECLI_ERR="$STEPS_DIR/3b-onecli.err" if pnpm exec tsx setup/index.ts --step onecli > "$ONECLI_LOG" 2>"$ONECLI_ERR"; then step_ok "OneCLI ready" ONECLI_OK=true - STEP_RESULTS["3a-onecli"]="success" + STEP_RESULTS["3b-onecli"]="success" log "OneCLI: installed/configured" else step_fail "OneCLI setup failed $(dim "(see $ONECLI_LOG)")" - STEP_RESULTS["3a-onecli"]="failed" + STEP_RESULTS["3b-onecli"]="failed" log "OneCLI: FAILED" fi +else + step_fail "OneCLI needs Docker $(dim "(install Docker first)")" + STEP_RESULTS["3b-onecli"]="failed" + log "OneCLI: skipped (no Docker)" fi -# 3b. Anthropic credential — run the auth setup step if no credential found +# 3c. Anthropic credential — run the auth setup step if no credential found if grep -qE '^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN)=' .env 2>/dev/null; then step_ok "Anthropic credential found in .env" log "Anthropic credential: found in .env" elif [ "$ONECLI_OK" = "true" ]; then step_info "Registering Anthropic credential…" - AUTH_LOG="$STEPS_DIR/3b-auth.log" - AUTH_ERR="$STEPS_DIR/3b-auth.err" + AUTH_LOG="$STEPS_DIR/3c-auth.log" + AUTH_ERR="$STEPS_DIR/3c-auth.err" if pnpm exec tsx setup/index.ts --step auth > "$AUTH_LOG" 2>"$AUTH_ERR"; then step_ok "Anthropic credential registered" - STEP_RESULTS["3b-auth"]="success" + STEP_RESULTS["3c-auth"]="success" log "Anthropic credential: registered via auth step" else step_fail "Auth setup failed $(dim "(see $AUTH_LOG)")" - STEP_RESULTS["3b-auth"]="failed" + STEP_RESULTS["3c-auth"]="failed" log "Anthropic credential: FAILED" fi else @@ -420,26 +443,6 @@ else log "Anthropic credential: skipped (no OneCLI)" fi -# 3c. Docker — install if missing -if command -v docker >/dev/null 2>&1; then - DOCKER_V=$(docker --version 2>/dev/null | head -1) - step_ok "Docker available $(dim "($DOCKER_V)")" - log "Docker: $DOCKER_V" -else - step_info "Installing Docker…" - DOCKER_LOG="$STEPS_DIR/3c-docker.log" - if bash setup/install-docker.sh > "$DOCKER_LOG" 2>&1; then - hash -r 2>/dev/null || true - step_ok "Docker installed" - STEP_RESULTS["3c-docker"]="success" - log "Docker: installed" - else - step_fail "Docker install failed $(dim "(see $DOCKER_LOG)")" - STEP_RESULTS["3c-docker"]="failed" - log "Docker: FAILED" - fi -fi - # 3d. Copy container skills from v1 that v2 doesn't have V1_SKILLS_DIR="$V1_PATH/container/skills" V2_SKILLS_DIR="$PROJECT_ROOT/container/skills" From aec7ddd09932e6a1e86306c71ce898b51ff7bb34 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 14:32:34 +0300 Subject: [PATCH 11/23] fix(migrate-v2): correct JID parsing, Discord guildId lookup, silent failures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - shared.ts: parseJid now recognizes raw Baileys WhatsApp JIDs (`@s.whatsapp.net`, `@g.us`, etc.); v2PlatformId returns the raw JID for whatsapp to match what the runtime adapter emits. Without this, every WhatsApp group in a v1 install was silently skipped. - discord-resolver.ts: new helper that uses DISCORD_BOT_TOKEN to look up channelId → guildId via the Discord API, since v1 stored only the channel id but v2 needs `discord::`. Best-effort: on missing/invalid token or network error, returns empty resolver and the affected groups are skipped with the reason surfaced per channel. - db.ts, tasks.ts: route Discord groups through the resolver; other channels go through v2PlatformId unchanged. Resolver only built when at least one Discord group exists, so non-Discord installs incur no network. - db.ts: when every v1 group is skipped, exit non-zero with a FAIL line instead of `OK:groups=N,...,skipped=N`, so the wrapper doesn't hide total failure under a successful-looking summary. - migrate-v2.sh: run_step now surfaces ERROR: lines from successful steps (with count + first 3 + raw log path); phase 2c install loop populates STEP_RESULTS so install failures show in handoff.json instead of silently passing. - sessions.ts: copyTree skips dangling symlinks (e.g. v1's `.claude/debug/latest`) instead of crashing the entire step. Co-Authored-By: Claude Opus 4.7 (1M context) --- migrate-v2.sh | 20 +++- setup/migrate-v2/db.ts | 57 +++++++++- setup/migrate-v2/discord-resolver.test.ts | 115 +++++++++++++++++++++ setup/migrate-v2/discord-resolver.ts | 120 ++++++++++++++++++++++ setup/migrate-v2/sessions.ts | 2 + setup/migrate-v2/shared.ts | 22 +++- setup/migrate-v2/tasks.ts | 26 ++++- 7 files changed, 351 insertions(+), 11 deletions(-) create mode 100644 setup/migrate-v2/discord-resolver.test.ts create mode 100644 setup/migrate-v2/discord-resolver.ts diff --git a/migrate-v2.sh b/migrate-v2.sh index d790c32..38a0f0d 100644 --- a/migrate-v2.sh +++ b/migrate-v2.sh @@ -259,6 +259,18 @@ run_step() { step_ok "$label $(dim "$result")" log "$name: $result" STEP_RESULTS[$name]="success" + # Surface partial errors (rows skipped due to parse/lookup failures) + # even when the step exited successfully — they're easy to miss in the + # raw log and have caused silent migrations before. + if grep -q '^ERROR:' "$raw" 2>/dev/null; then + local err_count + err_count=$(grep -c '^ERROR:' "$raw") + echo " $(dim "${err_count} error(s) reported — see $raw")" + grep '^ERROR:' "$raw" | head -3 | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "$name: ${err_count} non-fatal errors" + fi elif grep -q '^SKIPPED:' "$raw" 2>/dev/null; then local reason reason=$(grep '^SKIPPED:' "$raw" | head -1 | sed 's/^SKIPPED://') @@ -340,14 +352,17 @@ else # 2c. Install channel code for ch in "${SELECTED_CHANNELS[@]}"; do INSTALL_SCRIPT="setup/install-${ch}.sh" + STEP_NAME="2c-install-${ch}" if [ -f "$INSTALL_SCRIPT" ]; then - STEP_LOG="$STEPS_DIR/2c-install-${ch}.log" + STEP_LOG="$STEPS_DIR/${STEP_NAME}.log" if bash "$INSTALL_SCRIPT" > "$STEP_LOG" 2>&1; then STATUS_LINE=$(grep '^STATUS:' "$STEP_LOG" | head -1 | sed 's/^STATUS: *//') if [ "$STATUS_LINE" = "already-installed" ]; then step_skip "Install $ch $(dim "(already installed)")" + STEP_RESULTS[$STEP_NAME]="skipped" else step_ok "Install $ch" + STEP_RESULTS[$STEP_NAME]="success" fi log "install-$ch: $STATUS_LINE" else @@ -356,9 +371,12 @@ else echo " $(dim "$line")" done log "install-$ch: FAILED (see $STEP_LOG)" + STEP_RESULTS[$STEP_NAME]="failed" fi else step_skip "Install $ch $(dim "(no install script)")" + log "install-$ch: no install script" + STEP_RESULTS[$STEP_NAME]="failed" fi done fi diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts index f33ec2b..fb15ab0 100644 --- a/setup/migrate-v2/db.ts +++ b/setup/migrate-v2/db.ts @@ -25,10 +25,13 @@ import { getMessagingGroupByPlatform, } from '../../src/db/messaging-groups.js'; import { runMigrations } from '../../src/db/migrations/index.js'; +import { readEnvFile } from '../../src/env.js'; +import { buildDiscordResolver, type DiscordResolver } from './discord-resolver.js'; import { generateId, parseJid, triggerToEngage, + v2PlatformId, } from './shared.js'; interface V1Group { @@ -40,7 +43,7 @@ interface V1Group { is_main: number | null; } -function main(): void { +async function main(): Promise { const v1Path = process.argv[2]; if (!v1Path) { console.error('Usage: tsx setup/migrate-v2/db.ts '); @@ -78,6 +81,24 @@ function main(): void { let skipped = 0; const errors: string[] = []; + // v1 stored Discord groups as `dc:` (no guildId). v2 needs + // `discord::`. If there are any Discord groups, use + // the bot token (carried forward by 1a-env) to look up each channel's + // guild via the Discord API. On any failure the resolver returns null + // for every channel and the affected groups skip with a clear warning. + let discordResolver: DiscordResolver | null = null; + const hasDiscord = v1Groups.some((g) => parseJid(g.jid)?.channel_type === 'discord'); + if (hasDiscord) { + const env = readEnvFile(['DISCORD_BOT_TOKEN']); + discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? ''); + const stats = discordResolver.stats(); + if (stats.reason) { + console.log(`WARN:discord resolver disabled: ${stats.reason}`); + } else { + console.log(`INFO:discord resolver: ${stats.guilds} guild(s), ${stats.channels} channel(s)`); + } + } + for (const g of v1Groups) { const parsed = parseJid(g.jid); if (!parsed) { @@ -87,9 +108,22 @@ function main(): void { } const channelType = parsed.channel_type; - const platformId = parsed.raw.startsWith(`${channelType}:`) - ? parsed.raw - : `${channelType}:${parsed.id}`; + let platformId: string; + if (channelType === 'discord') { + const resolved = discordResolver?.resolve(parsed.id) ?? null; + if (!resolved) { + const stats = discordResolver?.stats(); + const why = stats?.reason + ? `discord resolver unavailable (${stats.reason})` + : 'not found in any guild the bot can see — re-add the bot to that server and re-run, or rewire after migration'; + skipped++; + errors.push(`Discord channel ${parsed.id} (${g.folder}): ${why}`); + continue; + } + platformId = resolved; + } else { + platformId = v2PlatformId(channelType, parsed.raw); + } const createdAt = new Date().toISOString(); try { @@ -152,10 +186,23 @@ function main(): void { v2Db.close(); + // If every group was skipped, the migration didn't actually do anything. + // Treat that as failure so the wrapper script surfaces it instead of + // hiding it under an `OK:` line. + const totalDone = created + reused; + if (v1Groups.length > 0 && totalDone === 0) { + console.error(`FAIL:groups=${v1Groups.length},created=0,reused=0,skipped=${skipped}`); + for (const e of errors) console.error(`ERROR:${e}`); + process.exit(1); + } + console.log(`OK:groups=${v1Groups.length},created=${created},reused=${reused},skipped=${skipped}`); if (errors.length > 0) { for (const e of errors) console.log(`ERROR:${e}`); } } -main(); +main().catch((err) => { + console.error(`FAIL:${err instanceof Error ? err.message : String(err)}`); + process.exit(1); +}); diff --git a/setup/migrate-v2/discord-resolver.test.ts b/setup/migrate-v2/discord-resolver.test.ts new file mode 100644 index 0000000..31e63f7 --- /dev/null +++ b/setup/migrate-v2/discord-resolver.test.ts @@ -0,0 +1,115 @@ +import { describe, expect, it, vi } from 'vitest'; + +import { buildDiscordResolver } from './discord-resolver.js'; + +function mockFetch(handlers: Record): typeof fetch { + return vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + const match = Object.keys(handlers).find((k) => url.startsWith(k)); + if (!match) throw new Error(`unexpected fetch: ${url}`); + const body = handlers[match]; + if (body instanceof Error) throw body; + if (typeof body === 'object' && body !== null && 'status' in body && (body as { status?: number }).status) { + const r = body as { status: number; statusText?: string; body?: string }; + return new Response(r.body ?? '', { status: r.status, statusText: r.statusText ?? '' }); + } + return new Response(JSON.stringify(body), { status: 200 }); + }) as unknown as typeof fetch; +} + +describe('buildDiscordResolver', () => { + it('returns empty resolver when token is missing', async () => { + const r = await buildDiscordResolver(''); + expect(r.stats()).toMatchObject({ guilds: 0, channels: 0 }); + expect(r.stats().reason).toMatch(/no DISCORD_BOT_TOKEN/); + expect(r.resolve('any')).toBeNull(); + }); + + it('resolves channels to guild-prefixed platform ids', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [ + { id: 'g1', name: 'Guild 1' }, + { id: 'g2', name: 'Guild 2' }, + ], + 'https://discord.com/api/v10/guilds/g1/channels': [ + { id: 'c1' }, + { id: 'c2' }, + ], + 'https://discord.com/api/v10/guilds/g2/channels': [ + { id: 'c3' }, + ], + }); + + const r = await buildDiscordResolver('valid-token', fetchImpl); + + expect(r.stats()).toEqual({ guilds: 2, channels: 3 }); + expect(r.resolve('c1')).toBe('discord:g1:c1'); + expect(r.resolve('c2')).toBe('discord:g1:c2'); + expect(r.resolve('c3')).toBe('discord:g2:c3'); + expect(r.resolve('cX')).toBeNull(); + }); + + it('returns disabled resolver on 401', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': { + status: 401, + statusText: 'Unauthorized', + body: '{"message":"401: Unauthorized","code":0}', + }, + }); + + const r = await buildDiscordResolver('bad-token', fetchImpl); + expect(r.stats().guilds).toBe(0); + expect(r.stats().reason).toMatch(/401/); + expect(r.resolve('c1')).toBeNull(); + }); + + it('keeps partial results when one guild lookup fails', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [ + { id: 'g1', name: 'Good Guild' }, + { id: 'g2', name: 'Bad Guild' }, + ], + 'https://discord.com/api/v10/guilds/g1/channels': [{ id: 'c1' }], + 'https://discord.com/api/v10/guilds/g2/channels': { + status: 403, + statusText: 'Forbidden', + body: '{}', + }, + }); + + const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + const r = await buildDiscordResolver('valid-token', fetchImpl); + errSpy.mockRestore(); + + expect(r.resolve('c1')).toBe('discord:g1:c1'); + expect(r.stats().guilds).toBe(2); + expect(r.stats().channels).toBe(1); + }); + + it('paginates the guild list', async () => { + // First page: 200 guilds (g0..g199); second page: 1 guild (g200); third call would not happen. + const page1 = Array.from({ length: 200 }, (_, i) => ({ id: `g${i}`, name: `G${i}` })); + const page2 = [{ id: 'g200', name: 'G200' }]; + let call = 0; + const fetchImpl = vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (url.includes('/users/@me/guilds')) { + call++; + const body = call === 1 ? page1 : page2; + return new Response(JSON.stringify(body), { status: 200 }); + } + // Every guild has one channel named after itself + const m = /\/guilds\/([^/]+)\/channels/.exec(url); + const gid = m ? m[1] : ''; + return new Response(JSON.stringify([{ id: `c-${gid}` }]), { status: 200 }); + }) as unknown as typeof fetch; + + const r = await buildDiscordResolver('valid-token', fetchImpl); + + expect(r.stats().guilds).toBe(201); + expect(r.stats().channels).toBe(201); + expect(r.resolve('c-g0')).toBe('discord:g0:c-g0'); + expect(r.resolve('c-g200')).toBe('discord:g200:c-g200'); + }); +}); diff --git a/setup/migrate-v2/discord-resolver.ts b/setup/migrate-v2/discord-resolver.ts new file mode 100644 index 0000000..17b9a9f --- /dev/null +++ b/setup/migrate-v2/discord-resolver.ts @@ -0,0 +1,120 @@ +/** + * Discord channel → guild resolver for the v1 → v2 migration. + * + * v1 stored Discord groups as `dc:` — only the channel id, not + * the guild id. v2's `@chat-adapter/discord` encodes `platform_id` as + * `discord::`, so we can't reconstruct it from v1 data + * alone. Instead, we use the v1 bot token (carried forward by 1a-env) to + * query the Discord API and build a channelId → guildId map. + * + * Network calls are best-effort: on auth failure or network error, the + * resolver returns null for every channel and the caller falls back to + * skipping with a clear warning. + */ + +const DISCORD_API = 'https://discord.com/api/v10'; + +interface Guild { + id: string; + name: string; +} + +interface Channel { + id: string; + name?: string; +} + +export interface DiscordResolver { + /** Returns `discord::` or null if the channel isn't visible to the bot. */ + resolve(channelId: string): string | null; + /** Diagnostic info — guild count and total channel count discovered. */ + stats(): { guilds: number; channels: number; reason?: string }; +} + +/** A no-op resolver that returns null for every lookup with a stored reason. */ +function emptyResolver(reason: string): DiscordResolver { + return { + resolve: () => null, + stats: () => ({ guilds: 0, channels: 0, reason }), + }; +} + +type FetchFn = typeof fetch; + +async function getJson(url: string, token: string, fetchImpl: FetchFn): Promise { + const res = await fetchImpl(url, { + headers: { + Authorization: `Bot ${token}`, + 'User-Agent': 'NanoClaw-Migration (https://github.com/qwibitai/nanoclaw, 2.x)', + }, + }); + if (!res.ok) { + const body = await res.text().catch(() => ''); + throw new Error(`Discord API ${res.status} ${res.statusText}: ${body.slice(0, 200)}`); + } + return (await res.json()) as T; +} + +/** + * Build a Discord resolver by enumerating every guild the bot is in and + * every channel in those guilds. Returns an empty resolver on any error. + * + * Costs: 1 + N HTTP calls (N = guild count). Discord's global rate limit + * is 50 req/s; even installs with hundreds of guilds finish in under a + * second of network time. + */ +export async function buildDiscordResolver( + token: string, + fetchImpl: FetchFn = fetch, +): Promise { + if (!token) return emptyResolver('no DISCORD_BOT_TOKEN in .env'); + + // Page through guilds. Default page size is 200; loop until short page. + const guilds: Guild[] = []; + let after: string | null = null; + try { + while (true) { + const url = new URL(`${DISCORD_API}/users/@me/guilds`); + url.searchParams.set('limit', '200'); + if (after) url.searchParams.set('after', after); + const page = await getJson(url.toString(), token, fetchImpl); + guilds.push(...page); + if (page.length < 200) break; + after = page[page.length - 1].id; + } + } catch (err) { + return emptyResolver(`failed to list guilds: ${err instanceof Error ? err.message : String(err)}`); + } + + // Per-guild channel enumeration. + const channelToGuild = new Map(); + for (const guild of guilds) { + try { + const channels = await getJson( + `${DISCORD_API}/guilds/${guild.id}/channels`, + token, + fetchImpl, + ); + for (const ch of channels) { + channelToGuild.set(ch.id, guild.id); + } + } catch (err) { + // Skip this guild but keep going — partial results are still useful. + // The caller logs which channels couldn't be resolved. + console.error( + `WARN:discord-resolver: failed to enumerate guild ${guild.id} (${guild.name}): ${ + err instanceof Error ? err.message : String(err) + }`, + ); + } + } + + return { + resolve(channelId: string): string | null { + const guildId = channelToGuild.get(channelId); + if (!guildId) return null; + return `discord:${guildId}:${channelId}`; + }, + stats: () => ({ guilds: guilds.length, channels: channelToGuild.size }), + }; +} diff --git a/setup/migrate-v2/sessions.ts b/setup/migrate-v2/sessions.ts index 0299dec..7ca8516 100644 --- a/setup/migrate-v2/sessions.ts +++ b/setup/migrate-v2/sessions.ts @@ -50,6 +50,8 @@ function copyTree(src: string, dst: string): number { written += copyTree(s, d); continue; } + // Skip dangling symlinks (e.g. v1's .claude/debug/latest pointer). + if (entry.isSymbolicLink() && !fs.existsSync(s)) continue; if (fs.existsSync(d)) continue; fs.copyFileSync(s, d); written += 1; diff --git a/setup/migrate-v2/shared.ts b/setup/migrate-v2/shared.ts index 62f2236..ff219df 100644 --- a/setup/migrate-v2/shared.ts +++ b/setup/migrate-v2/shared.ts @@ -32,7 +32,19 @@ export interface ParsedJid { channel_type: string; } +/** WhatsApp (Baileys) JID hosts. v1 stored these raw, with no `wa:` prefix. */ +const WA_JID_HOSTS = new Set(['s.whatsapp.net', 'g.us', 'lid', 'broadcast', 'newsletter']); + +function isWhatsappJid(raw: string): boolean { + const at = raw.lastIndexOf('@'); + if (at === -1) return false; + return WA_JID_HOSTS.has(raw.slice(at + 1).toLowerCase()); +} + export function parseJid(raw: string): ParsedJid | null { + if (isWhatsappJid(raw)) { + return { raw, prefix: 'whatsapp', id: raw, channel_type: 'whatsapp' }; + } const colon = raw.indexOf(':'); if (colon === -1) return null; const prefix = raw.slice(0, colon).toLowerCase(); @@ -47,10 +59,16 @@ export function parseJid(raw: string): ParsedJid | null { } /** - * Build a v2 platform_id from a v1 JID. v2's messaging_groups.platform_id - * is always `:`. + * Build a v2 platform_id from a v1 JID, in the format the runtime adapter + * for that channel emits. WhatsApp uses the raw Baileys JID (`@`, + * no prefix). Other channels use `:`. */ export function v2PlatformId(channelType: string, jid: string): string { + if (channelType === 'whatsapp') { + // Strip any v1 `wa:`/`whatsapp:` prefix; otherwise pass through raw. + const parsed = parseJid(jid); + return parsed?.channel_type === 'whatsapp' ? parsed.id : jid; + } const parsed = parseJid(jid); const id = parsed?.id ?? jid; return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; diff --git a/setup/migrate-v2/tasks.ts b/setup/migrate-v2/tasks.ts index 6a7efbe..4d3b3b5 100644 --- a/setup/migrate-v2/tasks.ts +++ b/setup/migrate-v2/tasks.ts @@ -22,6 +22,8 @@ import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; import { runMigrations } from '../../src/db/migrations/index.js'; import { insertTask } from '../../src/modules/scheduling/db.js'; import { openInboundDb, resolveSession } from '../../src/session-manager.js'; +import { readEnvFile } from '../../src/env.js'; +import { buildDiscordResolver, type DiscordResolver } from './discord-resolver.js'; import { parseJid, v2PlatformId } from './shared.js'; interface V1Task { @@ -67,7 +69,7 @@ function toCron(t: V1Task): { processAfter: string; recurrence: string | null } return null; } -function main(): void { +async function main(): Promise { const v1Path = process.argv[2]; if (!v1Path) { console.error('Usage: tsx setup/migrate-v2/tasks.ts '); @@ -104,6 +106,14 @@ function main(): void { let skipped = 0; let failed = 0; + // Mirrors db.ts: Discord platform_id needs API lookup to recover guildId. + let discordResolver: DiscordResolver | null = null; + const hasDiscord = activeTasks.some((t) => parseJid(t.chat_jid)?.channel_type === 'discord'); + if (hasDiscord) { + const env = readEnvFile(['DISCORD_BOT_TOKEN']); + discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? ''); + } + for (const t of activeTasks) { try { const ag = getAgentGroupByFolder(t.group_folder); @@ -112,7 +122,14 @@ function main(): void { const parsed = parseJid(t.chat_jid); if (!parsed) { skipped++; continue; } - const platformId = v2PlatformId(parsed.channel_type, t.chat_jid); + let platformId: string; + if (parsed.channel_type === 'discord') { + const resolved = discordResolver?.resolve(parsed.id) ?? null; + if (!resolved) { skipped++; continue; } + platformId = resolved; + } else { + platformId = v2PlatformId(parsed.channel_type, t.chat_jid); + } const mg = getMessagingGroupByPlatform(parsed.channel_type, platformId); if (!mg) { skipped++; continue; } @@ -155,4 +172,7 @@ function main(): void { console.log(`OK:active=${activeTasks.length},migrated=${migrated},skipped=${skipped},failed=${failed}`); } -main(); +main().catch((err) => { + console.error(`FAIL:${err instanceof Error ? err.message : String(err)}`); + process.exit(1); +}); From 416c283dcbfdb8a4305e0ff2e1d7c6b448657a6f Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 14:50:21 +0300 Subject: [PATCH 12/23] fix(migrate-v2): bash 3.2 compatibility + reset coverage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit migrate-v2.sh Replace `declare -A STEP_RESULTS` with two parallel indexed arrays (STEP_NAMES + STEP_STATUSES) plus a `record_step` helper. macOS ships bash 3.2 which has no associative arrays — `declare -A` errored out silently and every `STEP_RESULTS["1a-env"]=...` triggered a fatal bash arithmetic error (interpreting "1a" as a number). Visible symptom: `steps: {}` in handoff.json. Latent symptom: phase 2c's install loop sometimes bailed mid-iteration before invoking the channel install script, leaving channel code uninstalled while reporting `overall_status: success`. migrate-v2-reset.sh Cover the gaps that left install side-effects in place between iterations: - Remove untracked adapter files in src/channels/ (mirror the pattern already used for container/skills/). - Restore tracked setup helpers that channel installs overwrite (setup/whatsapp-auth.ts, setup/pair-telegram.ts, setup/index.ts) and remove untracked ones they create (setup/groups.ts). - Restore package.json + pnpm-lock.yaml (channel installs add deps like @whiskeysockets/baileys). Setup/migrate-v2/* is intentionally not touched — that's where user WIP lives. Verified end-to-end: reset → migrate → all 9 steps reported in handoff.json with status "success", phase 2c install actually runs. Co-Authored-By: Claude Opus 4.7 (1M context) --- migrate-v2-reset.sh | 45 ++++++++++++++++++++++++++++------ migrate-v2.sh | 59 +++++++++++++++++++++++++++------------------ 2 files changed, 73 insertions(+), 31 deletions(-) diff --git a/migrate-v2-reset.sh b/migrate-v2-reset.sh index b795745..cd817d9 100644 --- a/migrate-v2-reset.sh +++ b/migrate-v2-reset.sh @@ -6,17 +6,27 @@ # bash migrate-v2-reset.sh && bash migrate-v2.sh # # What it removes: -# - data/ (v2 DBs, session state) -# - logs/ (migration + setup logs) -# - .env (merged env keys) -# - groups/*/ (non-git group folders copied from v1) +# - data/ (v2 DBs, session state) +# - logs/ (migration + setup logs) +# - .env (merged env keys) +# - groups/*/ (non-git group folders copied from v1) +# - container/skills/*/ (untracked skill dirs copied from v1) +# - src/channels/*.ts (untracked adapters copied from channels branch) +# - setup/groups.ts (untracked, copied by channel install scripts) # -# What it restores: -# - groups/global/CLAUDE.md and groups/main/CLAUDE.md from git +# What it restores from git: +# - groups/ (CLAUDE.md files etc.) +# - container/skills/ (tracked container skills) +# - src/channels/ (tracked bridge / registry code) +# - setup/whatsapp-auth.ts (channel installs may overwrite) +# - setup/pair-telegram.ts (channel installs may overwrite) +# - setup/index.ts (channel installs append entries) +# - package.json + pnpm-lock.yaml (channel installs add deps) # # What it does NOT touch: -# - node_modules/ (expensive to reinstall, keep it) -# - The v1 install (read-only, never modified) +# - node_modules/ (expensive to reinstall, kept on purpose) +# - setup/migrate-v2/* (the migration scripts themselves, plus user WIP) +# - The v1 install (read-only, never modified) set -euo pipefail @@ -63,7 +73,26 @@ printf '%s Restored %s\n' "$(green '✓')" "container/skills/ from git" # Restore channel code (src/channels/) to git state git checkout -- src/channels/ 2>/dev/null || true +# Remove any untracked channel adapters copied in by install-*.sh +for f in src/channels/*.ts; do + [ -f "$f" ] || continue + if ! git ls-files --error-unmatch "$f" >/dev/null 2>&1; then + rm -f "$f" + fi +done printf '%s Restored %s\n' "$(green '✓')" "src/channels/ from git" +# Restore tracked setup helpers that channel installs overwrite, and +# remove the untracked ones they create. Don't blanket-clean setup/ +# because user WIP (setup/migrate-v2/*) lives there too. +git checkout -- setup/whatsapp-auth.ts setup/pair-telegram.ts setup/index.ts 2>/dev/null || true +rm -f setup/groups.ts +printf '%s Restored %s\n' "$(green '✓')" "setup/ install helpers" + +# Restore package.json + lockfile (channel installs add deps like +# @whiskeysockets/baileys). node_modules/ is intentionally kept. +git checkout -- package.json pnpm-lock.yaml 2>/dev/null || true +printf '%s Restored %s\n' "$(green '✓')" "package.json + pnpm-lock.yaml" + echo printf '%s\n\n' "$(dim 'Clean. Run: bash migrate-v2.sh')" diff --git a/migrate-v2.sh b/migrate-v2.sh index 38a0f0d..cacdffc 100644 --- a/migrate-v2.sh +++ b/migrate-v2.sh @@ -29,14 +29,25 @@ SERVICE_SWITCHED=false SELECTED_CHANNELS=() ABORTED_AT="" +# Per-step status tracking. Parallel indexed arrays so this works on +# bash 3.2 (macOS default) which has no associative arrays. +STEP_NAMES=() +STEP_STATUSES=() + +record_step() { + STEP_NAMES+=("$1") + STEP_STATUSES+=("$2") +} + # Write handoff.json on any exit so the skill can always read it write_handoff() { local handoff_dir="$LOGS_DIR/setup-migration" mkdir -p "$handoff_dir" local has_failures=false - for step_name in "${!STEP_RESULTS[@]}"; do - [ "${STEP_RESULTS[$step_name]}" = "failed" ] && has_failures=true + local i + for ((i=0; i<${#STEP_NAMES[@]}; i++)); do + [ "${STEP_STATUSES[$i]}" = "failed" ] && has_failures=true done local overall="success" @@ -44,8 +55,10 @@ write_handoff() { [ -n "$ABORTED_AT" ] && overall="failed" local steps_json="{" - for step_name in "${!STEP_RESULTS[@]}"; do - steps_json="${steps_json}\"${step_name}\": {\"status\": \"${STEP_RESULTS[$step_name]}\", \"log\": \"logs/migrate-steps/${step_name}.log\"}," + for ((i=0; i<${#STEP_NAMES[@]}; i++)); do + local n="${STEP_NAMES[$i]}" + local s="${STEP_STATUSES[$i]}" + steps_json="${steps_json}\"${n}\": {\"status\": \"${s}\", \"log\": \"logs/migrate-steps/${n}.log\"}," done steps_json="${steps_json%,}}" @@ -245,8 +258,8 @@ export NANOCLAW_V2_PATH="$PROJECT_ROOT" # ─── run_step helper ───────────────────────────────────────────────────── # Runs a TypeScript migration step, captures output, reports success/failure. -# Track step outcomes for handoff.json -declare -A STEP_RESULTS +# Step outcomes are tracked via record_step() into STEP_NAMES/STEP_STATUSES +# (defined above, near write_handoff). run_step() { local name=$1 label=$2 script=$3 @@ -258,7 +271,7 @@ run_step() { result=$(grep '^OK:' "$raw" | head -1 || true) step_ok "$label $(dim "$result")" log "$name: $result" - STEP_RESULTS[$name]="success" + record_step "$name" "success" # Surface partial errors (rows skipped due to parse/lookup failures) # even when the step exited successfully — they're easy to miss in the # raw log and have caused silent migrations before. @@ -276,7 +289,7 @@ run_step() { reason=$(grep '^SKIPPED:' "$raw" | head -1 | sed 's/^SKIPPED://') step_skip "$label $(dim "($reason)")" log "$name: skipped ($reason)" - STEP_RESULTS[$name]="skipped" + record_step "$name" "skipped" else step_fail "$label" echo @@ -285,7 +298,7 @@ run_step() { done echo log "$name: FAILED (see $raw)" - STEP_RESULTS[$name]="failed" + record_step "$name" "failed" fi } @@ -359,10 +372,10 @@ else STATUS_LINE=$(grep '^STATUS:' "$STEP_LOG" | head -1 | sed 's/^STATUS: *//') if [ "$STATUS_LINE" = "already-installed" ]; then step_skip "Install $ch $(dim "(already installed)")" - STEP_RESULTS[$STEP_NAME]="skipped" + record_step "$STEP_NAME" "skipped" else step_ok "Install $ch" - STEP_RESULTS[$STEP_NAME]="success" + record_step "$STEP_NAME" "success" fi log "install-$ch: $STATUS_LINE" else @@ -371,12 +384,12 @@ else echo " $(dim "$line")" done log "install-$ch: FAILED (see $STEP_LOG)" - STEP_RESULTS[$STEP_NAME]="failed" + record_step "$STEP_NAME" "failed" fi else step_skip "Install $ch $(dim "(no install script)")" log "install-$ch: no install script" - STEP_RESULTS[$STEP_NAME]="failed" + record_step "$STEP_NAME" "failed" fi done fi @@ -401,11 +414,11 @@ else if bash setup/install-docker.sh > "$DOCKER_LOG" 2>&1; then hash -r 2>/dev/null || true step_ok "Docker installed" - STEP_RESULTS["3a-docker"]="success" + record_step "3a-docker" "success" log "Docker: installed" else step_fail "Docker install failed $(dim "(see $DOCKER_LOG)")" - STEP_RESULTS["3a-docker"]="failed" + record_step "3a-docker" "failed" log "Docker: FAILED" fi fi @@ -426,16 +439,16 @@ elif command -v docker >/dev/null 2>&1; then if pnpm exec tsx setup/index.ts --step onecli > "$ONECLI_LOG" 2>"$ONECLI_ERR"; then step_ok "OneCLI ready" ONECLI_OK=true - STEP_RESULTS["3b-onecli"]="success" + record_step "3b-onecli" "success" log "OneCLI: installed/configured" else step_fail "OneCLI setup failed $(dim "(see $ONECLI_LOG)")" - STEP_RESULTS["3b-onecli"]="failed" + record_step "3b-onecli" "failed" log "OneCLI: FAILED" fi else step_fail "OneCLI needs Docker $(dim "(install Docker first)")" - STEP_RESULTS["3b-onecli"]="failed" + record_step "3b-onecli" "failed" log "OneCLI: skipped (no Docker)" fi @@ -449,11 +462,11 @@ elif [ "$ONECLI_OK" = "true" ]; then AUTH_ERR="$STEPS_DIR/3c-auth.err" if pnpm exec tsx setup/index.ts --step auth > "$AUTH_LOG" 2>"$AUTH_ERR"; then step_ok "Anthropic credential registered" - STEP_RESULTS["3c-auth"]="success" + record_step "3c-auth" "success" log "Anthropic credential: registered via auth step" else step_fail "Auth setup failed $(dim "(see $AUTH_LOG)")" - STEP_RESULTS["3c-auth"]="failed" + record_step "3c-auth" "failed" log "Anthropic credential: FAILED" fi else @@ -494,11 +507,11 @@ if command -v docker >/dev/null 2>&1; then BUILD_LOG="$STEPS_DIR/3e-container-build.log" if bash container/build.sh > "$BUILD_LOG" 2>&1; then step_ok "Container image built" - STEP_RESULTS["3e-build"]="success" + record_step "3e-build" "success" log "Container build: success" else step_fail "Container build failed" - STEP_RESULTS["3e-build"]="failed" + record_step "3e-build" "failed" tail -10 "$BUILD_LOG" 2>/dev/null | while IFS= read -r line; do echo " $(dim "$line")" done @@ -506,7 +519,7 @@ if command -v docker >/dev/null 2>&1; then fi else step_fail "Docker not available — cannot build container" - STEP_RESULTS["3e-build"]="failed" + record_step "3e-build" "failed" log "Container build: skipped (no Docker)" fi From 2a915e8af09adef67092b6f6c54b402b7054f74f Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 15:24:57 +0300 Subject: [PATCH 13/23] fix(migrate-v2): infer is_group from JID format v1 didn't track is_group separately; db.ts hardcoded `is_group: 1` for every messaging_group. v2 uses is_group=0 to collapse DM sub-thread sessions and to drive routing decisions, so getting it wrong is latent risk on otherwise-working installs. New helper inferIsGroup(channelType, platformId) lives in shared.ts so tasks.ts and any future migration step can reuse it. Inferred per channel: - whatsapp: `@g.us` is a group, anything else is a DM - telegram: negative chat IDs are groups, positive are DMs - everything else: default to 1 (least surprising for chats v1 chose to register, where DM auto-create paths weren't used) Co-Authored-By: Claude Opus 4.7 (1M context) --- setup/migrate-v2/db.ts | 3 ++- setup/migrate-v2/shared.ts | 21 +++++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts index fb15ab0..da2bab7 100644 --- a/setup/migrate-v2/db.ts +++ b/setup/migrate-v2/db.ts @@ -29,6 +29,7 @@ import { readEnvFile } from '../../src/env.js'; import { buildDiscordResolver, type DiscordResolver } from './discord-resolver.js'; import { generateId, + inferIsGroup, parseJid, triggerToEngage, v2PlatformId, @@ -148,7 +149,7 @@ async function main(): Promise { channel_type: channelType, platform_id: platformId, name: g.name || null, - is_group: 1, + is_group: inferIsGroup(channelType, platformId), unknown_sender_policy: 'public', created_at: createdAt, }); diff --git a/setup/migrate-v2/shared.ts b/setup/migrate-v2/shared.ts index ff219df..58c0b16 100644 --- a/setup/migrate-v2/shared.ts +++ b/setup/migrate-v2/shared.ts @@ -74,6 +74,27 @@ export function v2PlatformId(channelType: string, jid: string): string { return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; } +/** + * Infer messaging_groups.is_group from a v2 platform_id, given a channel type. + * + * v1 didn't track is_group, but most channels encode it in the JID/id format: + * - whatsapp: `@g.us` is a group, `@s.whatsapp.net` / `@lid` is a DM + * - telegram: negative chat IDs are groups, positive are DMs + * - everything else: default to 1 (group/channel) — least-surprising guess + * for chats v1 chose to register, where DM auto-create paths weren't used + */ +export function inferIsGroup(channelType: string, platformId: string): number { + if (channelType === 'whatsapp') { + return platformId.endsWith('@g.us') ? 1 : 0; + } + if (channelType === 'telegram') { + // platform_id is `telegram:` — negative chatId means group/channel. + const chatId = platformId.replace(/^telegram:/, ''); + return chatId.startsWith('-') ? 1 : 0; + } + return 1; +} + // ── Trigger mapping ───────────────────────────────────────────────────── /** From dca02f5453e97ccdfc47935a90b069596fcf6f1e Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 15:39:36 +0300 Subject: [PATCH 14/23] feat(migrate-v2): resolve WhatsApp LIDs from store/auth, alias DMs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v1 stored every WhatsApp DM as `@s.whatsapp.net`. v2's WA adapter sometimes resolves the chat to `@lid` instead — when WhatsApp delivers via the LID protocol and Baileys hasn't yet learned a LID→phone mapping for that contact (cold cache after migration). The router then can't find the phone-keyed messaging_group and silently drops the message at router.ts:184. Baileys persists every LID↔phone pair it has ever learned to disk as `store/auth/lid-mapping-.json` (forward) and `lid-mapping-_reverse.json` (reverse). v1 will already have these populated for every contact it has talked to. New step 2d-whatsapp-lids parses the reverse files and writes paired LID-keyed `messaging_groups` + `messaging_group_agents` rows so both `@s.whatsapp.net` and `@lid` route to the same agent_group with the same engage rules. No Baileys boot, no WhatsApp connectivity required — pure filesystem read of files we've already copied via 2b-channel-auth. Step is no-op-on-skip if either store/auth or whatsapp DM rows are missing. Anything that slips through (a contact whose LID v1 never learned) falls back to the runtime approval flow once the WA adapter sets isMention=true on DMs — each unknown LID DM auto-creates an approval-required messaging_group and the owner gets a one-tap register prompt. Verified end-to-end on a 12-group v1 install: 3 DM rows aliased, inbound DM routed via the LID-keyed row. Co-Authored-By: Claude Opus 4.7 (1M context) --- migrate-v2.sh | 15 ++ setup/migrate-v2/whatsapp-resolve-lids.ts | 192 ++++++++++++++++++++++ 2 files changed, 207 insertions(+) create mode 100644 setup/migrate-v2/whatsapp-resolve-lids.ts diff --git a/migrate-v2.sh b/migrate-v2.sh index cacdffc..eb5a381 100644 --- a/migrate-v2.sh +++ b/migrate-v2.sh @@ -392,6 +392,21 @@ else record_step "$STEP_NAME" "failed" fi done + + # 2d. WhatsApp LID resolution. After whatsapp is installed (so Baileys + # is on disk) and auth files have been copied (so we can connect with + # the migrated identity), boot Baileys briefly to learn LID↔phone + # mappings during initial sync, then write paired LID-keyed + # messaging_groups. Best-effort: any failure degrades to runtime + # approval flow, which the WA adapter's isMention=true on DMs handles. + for ch in "${SELECTED_CHANNELS[@]}"; do + if [ "$ch" = "whatsapp" ]; then + run_step "2d-whatsapp-lids" \ + "Resolve WhatsApp LIDs for migrated DMs" \ + "setup/migrate-v2/whatsapp-resolve-lids.ts" + break + fi + done fi echo diff --git a/setup/migrate-v2/whatsapp-resolve-lids.ts b/setup/migrate-v2/whatsapp-resolve-lids.ts new file mode 100644 index 0000000..7a5eb8b --- /dev/null +++ b/setup/migrate-v2/whatsapp-resolve-lids.ts @@ -0,0 +1,192 @@ +/** + * migrate-v2 step: resolve WhatsApp LIDs for migrated DM messaging_groups. + * + * Why this exists + * ─────────────── + * v1 stored every WhatsApp DM as `@s.whatsapp.net`. v2's WA adapter + * sometimes resolves the chat to `@lid` instead — when WhatsApp + * delivers a message via the LID protocol and Baileys hasn't yet learned + * a LID→phone mapping for that contact (cold cache after migration). The + * router then can't find the phone-keyed messaging_group and silently + * drops the message at router.ts:184 — until the LID is learned (which + * happens lazily, message-by-message, via `chats.phoneNumberShare`). + * + * Baileys persists LID↔phone mappings to disk as + * `store/auth/lid-mapping-_reverse.json` (LID → phone) and + * `lid-mapping-.json` (phone → LID). v1 will already have populated + * these for every contact it talked to. This step parses the reverse + * files and writes paired LID-keyed `messaging_groups` + + * `messaging_group_agents` rows so both `@s.whatsapp.net` and + * `@lid` route to the same agent_group with the same engage rules. + * + * No Baileys boot, no network — pure filesystem read. If store/auth is + * missing or has no reverse mappings, exits 0 with a SKIPPED. Runtime + * fallback (WA adapter sets isMention=true on DMs → router auto-creates + * with `unknown_sender_policy=request_approval`) handles anything we + * miss. + * + * Usage: pnpm exec tsx setup/migrate-v2/whatsapp-resolve-lids.ts + */ +import fs from 'fs'; +import path from 'path'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb } from '../../src/db/connection.js'; +import { + createMessagingGroup, + createMessagingGroupAgent, + getMessagingGroupAgentByPair, + getMessagingGroupByPlatform, +} from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { generateId } from './shared.js'; + +interface RawMessagingGroup { + id: string; + channel_type: string; + platform_id: string; +} + +interface RawWiring { + id: string; + messaging_group_id: string; + agent_group_id: string; + engage_mode: string; + engage_pattern: string | null; + sender_scope: string; + ignored_message_policy: string; + session_mode: string; + priority: number; +} + +const REVERSE_FILE_RE = /^lid-mapping-(\d+)_reverse\.json$/; + +/** + * Read store/auth/lid-mapping-*_reverse.json into a Map. + * Returns an empty Map if the directory doesn't exist. + */ +function readReverseMappings(authDir: string): Map { + const out = new Map(); + if (!fs.existsSync(authDir)) return out; + for (const entry of fs.readdirSync(authDir)) { + const m = REVERSE_FILE_RE.exec(entry); + if (!m) continue; + const lidUser = m[1]; + try { + const raw = fs.readFileSync(path.join(authDir, entry), 'utf-8').trim(); + // The file content is a JSON-encoded string: `""` + const phoneUser = JSON.parse(raw); + if (typeof phoneUser !== 'string' || phoneUser.length === 0) continue; + out.set(lidUser, phoneUser); + } catch { + // Skip malformed entries — best-effort. + } + } + return out; +} + +function phoneUserOf(jid: string): string { + return jid.split('@')[0].split(':')[0]; +} + +function main(): void { + const authDir = path.join(process.cwd(), 'store', 'auth'); + const reverse = readReverseMappings(authDir); + + if (reverse.size === 0) { + console.log('SKIPPED:no lid-mapping-*_reverse.json files in store/auth'); + process.exit(0); + } + + // phoneUser → lidJid (the form we'll write to messaging_groups) + const phoneUserToLidJid = new Map(); + for (const [lidUser, phoneUser] of reverse) { + phoneUserToLidJid.set(phoneUser, `${lidUser}@lid`); + } + + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('FAIL:v2.db not found — run db step first'); + process.exit(1); + } + + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + const phoneRows = v2Db + .prepare( + `SELECT id, channel_type, platform_id FROM messaging_groups + WHERE channel_type='whatsapp' AND platform_id LIKE '%@s.whatsapp.net'`, + ) + .all() as RawMessagingGroup[]; + + if (phoneRows.length === 0) { + console.log('SKIPPED:no whatsapp DM messaging_groups to resolve'); + v2Db.close(); + process.exit(0); + } + + // Pull existing wirings so each new alias gets the same agent_group + + // engage rules as the phone-keyed row. + const placeholders = phoneRows.map(() => '?').join(','); + const wiringRows = v2Db + .prepare(`SELECT * FROM messaging_group_agents WHERE messaging_group_id IN (${placeholders})`) + .all(...phoneRows.map((r) => r.id)) as RawWiring[]; + + const wiringsByMg = new Map(); + for (const w of wiringRows) { + const arr = wiringsByMg.get(w.messaging_group_id) ?? []; + arr.push(w); + wiringsByMg.set(w.messaging_group_id, arr); + } + + let resolved = 0; + let aliased = 0; + const createdAt = new Date().toISOString(); + + for (const row of phoneRows) { + const phoneUser = phoneUserOf(row.platform_id); + const lidJid = phoneUserToLidJid.get(phoneUser); + if (!lidJid) continue; + resolved++; + + let lidMg = getMessagingGroupByPlatform('whatsapp', lidJid); + if (!lidMg) { + createMessagingGroup({ + id: generateId('mg'), + channel_type: 'whatsapp', + platform_id: lidJid, + name: null, + is_group: 0, + unknown_sender_policy: 'public', + created_at: createdAt, + }); + lidMg = getMessagingGroupByPlatform('whatsapp', lidJid)!; + } + + const wirings = wiringsByMg.get(row.id) ?? []; + for (const w of wirings) { + if (getMessagingGroupAgentByPair(lidMg.id, w.agent_group_id)) continue; + createMessagingGroupAgent({ + id: generateId('mga'), + messaging_group_id: lidMg.id, + agent_group_id: w.agent_group_id, + engage_mode: w.engage_mode as 'pattern' | 'mention' | 'mention-sticky', + engage_pattern: w.engage_pattern, + sender_scope: w.sender_scope as 'all' | 'admins', + ignored_message_policy: w.ignored_message_policy as 'drop' | 'queue', + session_mode: w.session_mode as 'shared' | 'thread', + priority: w.priority, + created_at: createdAt, + }); + aliased++; + } + } + + v2Db.close(); + console.log( + `OK:reverse_mappings=${reverse.size},phone_dms=${phoneRows.length},lids_resolved=${resolved},aliased=${aliased}`, + ); +} + +main(); From 8439a180be0c16b09ab8f7a28e36f80e57181fa3 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 18:22:50 +0300 Subject: [PATCH 15/23] docs(migrate-v2): collapsible README section + skill preflight MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README: replace the one-line v1 migration note with a collapsed
block. Quick Start stays compact for the common case (fresh install) while v1 users get the actual instructions. Calls out explicitly that the script must be run from a real terminal — not from inside a Claude session — so the channel-select / switchover prompts and the Node/pnpm/Docker bootstrap all work. migrate-from-v1 skill: add a Preflight section that aborts if logs/setup-migration/handoff.json is missing. Without this, invoking the skill before the script just leads Claude to start guessing / running shell commands. The new message redirects them to the script and tells them it'll hand back to Claude on completion. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 16 ++++++++++++++++ README.md | 23 ++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 52d8293..a6fc990 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -20,6 +20,22 @@ Your job is the parts that need human judgment: triage any failed steps, seed th Read `logs/setup-migration/handoff.json` first — it has `overall_status`, per-step results in `steps`, and a `followups` list. +## Preflight: was the script run? + +Before anything else, check that `logs/setup-migration/handoff.json` exists. If it doesn't, the user is invoking this skill before `migrate-v2.sh` ran. Stop and tell them, verbatim: + +> This skill finishes a migration that `migrate-v2.sh` started. Run that first, in your terminal — not from inside Claude: +> +> ```bash +> bash migrate-v2.sh +> ``` +> +> It needs interactive prompts (channel selection, service switchover) and runs Node/pnpm bootstrap, Docker, OneCLI setup, and a container build that don't fit inside a Claude session. When it finishes, it'll hand control back to Claude automatically — at which point this skill picks up. + +Do not attempt to run the script yourself, simulate its effects, or pick up the migration mid-stream. The deterministic side has dependencies on a real interactive shell. + +Once `handoff.json` exists, proceed to Phase 0. + ## Phase 0: Triage failed steps Check `handoff.json` → `overall_status`. If `"success"`, skip to Phase 1. diff --git a/README.md b/README.md index 1b485c9..69f9ea2 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,28 @@ bash nanoclaw.sh `nanoclaw.sh` walks you from a fresh machine to a named agent you can message. It installs Node, pnpm, and Docker if missing, registers your Anthropic credential with OneCLI, builds the agent container, and pairs your first channel (Telegram, Discord, WhatsApp, or a local CLI). If a step fails, Claude Code is invoked automatically to diagnose and resume from where it broke. -**Coming from v1?** Run `bash migrate-v2.sh` instead of `nanoclaw.sh`. It finds your v1 install (sibling directory or `NANOCLAW_V1_PATH`), migrates state, installs channels, and hands off to Claude for owner setup and custom code porting. See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) and [docs/migration-dev.md](docs/migration-dev.md). +
+Migrating from NanoClaw v1? + +Run from a fresh v2 checkout next to your v1 install: + +```bash +git clone https://github.com/qwibitai/nanoclaw.git nanoclaw-v2 +cd nanoclaw-v2 +bash migrate-v2.sh +``` + +`migrate-v2.sh` finds your v1 install (sibling directory, or `NANOCLAW_V1_PATH=/path/to/nanoclaw`), migrates state into the v2 checkout, then `exec`s into Claude Code to finish the parts that need judgment (owner seeding, CLAUDE.local.md cleanup, fork-customisation replay). + +Run the script directly, not from inside a Claude session — the deterministic side needs interactive prompts and real shell I/O for Node/pnpm bootstrap, Docker, OneCLI, and the container build. + +**What it does:** merges `.env`, seeds the v2 DB from `registered_groups`, copies group folders + session data + scheduled tasks, installs the channel adapters you select, copies channel auth state (including Baileys keystore + LID mappings for WhatsApp), builds the agent container. + +**What it doesn't:** flip the system service. Pick *"switch to v2"* at the prompt, or do it manually after testing — your v1 install is left untouched. + +See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) for what's different and [docs/migration-dev.md](docs/migration-dev.md) for development notes. + +
## Philosophy From 2617313f19cca785b037a2955935fdb4f5cc8983 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 18:28:46 +0300 Subject: [PATCH 16/23] docs(migrate-from-v1): blockers-first + smoke test before deeper work MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 0 used to be "triage every failed step before doing anything else", which front-loaded a bunch of fixes for things that don't actually block the user from proving v2 works. Restructure: - 0a — fix blockers only (1b/1d/2c/2d/3a/3b/3e). Defer non-blockers (1a, 1c, 1e, 2b, 3c) — most surface naturally in later phases. - 0b — smoke test: switch v1 → v2, send a real message, verify the routing chain in logs/nanoclaw.log. AskUserQuestion gates whether to continue. - Revert recipe (launchctl/systemctl) called out as always-available, not destructive — v1 process, data, and credentials are untouched. Up-front list of what the script handled now also mentions the WhatsApp LID resolution and Baileys keystore copy, so users see exactly what continuity they're getting. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 107 ++++++++++++++++++++---- 1 file changed, 91 insertions(+), 16 deletions(-) diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index a6fc990..50c6afa 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -10,9 +10,10 @@ description: Finish migrating a NanoClaw v1 install into v2. Run after `bash mig - .env keys merged - v2 DB seeded (agent_groups, messaging_groups, wiring) - Group folders copied (v1 CLAUDE.md → v2 CLAUDE.local.md) -- Session data copied with conversation continuity +- Session data copied with conversation continuity (incl. Claude Code memory + JSONL transcripts) - Scheduled tasks ported -- Channel code installed +- Channel code installed and auth state copied (incl. WhatsApp Baileys keystore) +- WhatsApp LIDs resolved from `store/auth` and aliased into `messaging_groups` - Container skills copied - Container image built @@ -36,25 +37,99 @@ Do not attempt to run the script yourself, simulate its effects, or pick up the Once `handoff.json` exists, proceed to Phase 0. -## Phase 0: Triage failed steps +## Phase 0: Get v2 routing real messages -Check `handoff.json` → `overall_status`. If `"success"`, skip to Phase 1. +Goal: get from "the script finished" to "the user sent a message and v2 answered" as fast as possible — *before* spending tokens on CLAUDE.local.md cleanup, fork customisations, or anything that requires deeper engagement. v1 is paused, not touched; flipping back is a one-line restart. -If `"partial"`, walk `handoff.steps` — each has `status` and `log` (path to the raw log file). For each failed step: +### 0a — Fix blockers (only the blockers) -1. Read its log file at `handoff.step_logs_dir/.log`. -2. Explain what failed in one sentence. -3. Fix it if mechanical (re-run the step script, hand-write a DB insert, copy a missed file). The step scripts are at `setup/migrate-v2/.ts` and accept `` as the first argument. -4. Use `AskUserQuestion` when a judgment call is needed. +Walk `handoff.steps`. A step is **blocking** only if its failure prevents v2 from routing a single message. Treat these as blockers: -Common failures: -- **1b-db failed**: JID couldn't be parsed. Ask the user for the channel type, insert `agent_groups` + `messaging_groups` manually. -- **1d-sessions failed**: v2 DB wasn't seeded yet. Re-run after fixing 1b. -- **1e-tasks failed**: session doesn't exist yet. Re-run after fixing 1d. -- **2c-install-\ failed**: `git fetch origin channels` may have failed (network). Try again, or ask the user to run manually. -- **3e-container-build failed**: Docker issue. Read the build log, suggest fixes. +| Step | Why blocking | +|------|--------------| +| `1b-db` | No `messaging_groups` → router has nothing to match | +| `1d-sessions` | No session → no inbound DB to write into | +| `2c-install-` | No adapter for the channel the user wants to test | +| `2d-whatsapp-lids` | WhatsApp DMs may arrive as `@lid` and miss migrated phone-keyed rows | +| `3a-docker` / `3e-build` | No container image → agent can't run | +| `3b-onecli` | Anthropic credentials not injected → first agent call 401s | -After resolving all failures, proceed to Phase 1. +**Defer** these — they don't block a smoke test, and most surface naturally in later phases: + +- `1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth` + +For each blocker: read `handoff.step_logs_dir/.log`, identify the cause, re-run the underlying script directly (`pnpm exec tsx setup/migrate-v2/.ts `) or hand-fix mechanically. Use `AskUserQuestion` for judgment calls. Don't simulate the script's work. + +Common blockers: +- **`1b-db` failed**: JID couldn't be parsed. Insert `agent_groups` + `messaging_groups` for the user's confirmed channel. +- **`2c-install-` failed**: `git fetch origin channels` issue. The user can run `bash setup/install-.sh` directly. +- **`3e-build` failed**: usually stale builder cache. `docker buildx prune -f && ./container/build.sh`. + +### 0b — Smoke test before any further migration work + +Tell the user, verbatim: + +> Before we touch CLAUDE.local.md or fork customisations, let's confirm v2 actually answers your real messages. **This is non-destructive — v1 is just paused, not touched.** v1 and v2 share your WhatsApp identity (we copied `store/auth/` over), so only one can be online at a time, but flipping back is instant. + +Find the v2 service unit (per-checkout hash): + +```bash +# macOS +launchctl list | grep nanoclaw +# Linux +systemctl --user list-units 'nanoclaw*' +``` + +Switch v1 → v2: + +```bash +# macOS +launchctl unload ~/Library/LaunchAgents/com.nanoclaw.plist +launchctl load ~/Library/LaunchAgents/com.nanoclaw-v2-.plist + +# Linux +systemctl --user stop nanoclaw +systemctl --user start nanoclaw-v2- +``` + +Tail the log and confirm clean boot: + +```bash +tail -f logs/nanoclaw.log +``` + +Watch for `NanoClaw running` plus `Channel adapter started` for each installed channel (and `Connected to ` for native adapters like WhatsApp). + +Ask the user to send a real test message — a DM to the bot, or a post in a known group from a non-bot account. A working route logs an inbound event → session resolution → container spawn → outbound delivery. + +`AskUserQuestion`: *"Did v2 respond? — Yes / No, here's what happened."* + +**If yes**: continue to Phase 1. + +**If no**: do not proceed. Read `logs/nanoclaw.log` + `logs/nanoclaw.error.log` and diagnose. Common patterns: +- WhatsApp DM with no routing chain in the log → check `SELECT platform_id FROM messaging_groups WHERE platform_id LIKE '%@lid'`. If empty, re-run `setup/migrate-v2/whatsapp-resolve-lids.ts`. +- Agent inside container fails on Anthropic 401 → OneCLI agents start in `selective` secret mode. `onecli agents set-secret-mode --id --mode all`. +- Channel disconnected silently → restart: `launchctl kickstart -k gui/$(id -u)/com.nanoclaw-v2-`. + +Re-test before continuing. + +### Reverting (anytime — not just now) + +```bash +# macOS — back to v1 +launchctl unload ~/Library/LaunchAgents/com.nanoclaw-v2-.plist +launchctl load ~/Library/LaunchAgents/com.nanoclaw.plist + +# Linux +systemctl --user stop nanoclaw-v2- +systemctl --user start nanoclaw +``` + +v1's process, data, credentials, and groups are untouched the whole time. Reverting is just a service restart. + +### Deferred non-blocker failures + +If you skipped non-blocker failures in 0a (`1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth`), they still need fixing — most surface naturally in later phases (`1c-groups` ↔ Phase 2 CLAUDE.local.md cleanup, `1e-tasks` ↔ task verification). Re-run any that don't get covered before declaring the migration done. ## Phase 1: Owner and access From 2bc1279a12823ef8276fd383657aaee8101b060b Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 18:31:23 +0300 Subject: [PATCH 17/23] docs(migrate-from-v1): trim Phase 0 to intent only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Previous version spelled out launchctl/systemctl commands, log lines to grep for, diagnostic recipes — the agent reading this skill knows all of that. Keep only the parts that aren't obvious from the rest of the codebase: which steps are blocking vs deferred, the smoke-test ordering, and the non-destructive framing for the user. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 94 +++---------------------- 1 file changed, 11 insertions(+), 83 deletions(-) diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 50c6afa..0440d8a 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -39,97 +39,25 @@ Once `handoff.json` exists, proceed to Phase 0. ## Phase 0: Get v2 routing real messages -Goal: get from "the script finished" to "the user sent a message and v2 answered" as fast as possible — *before* spending tokens on CLAUDE.local.md cleanup, fork customisations, or anything that requires deeper engagement. v1 is paused, not touched; flipping back is a one-line restart. +Before any deeper migration work, prove v2 actually answers messages on the user's real channels. v1 is paused, not touched — flipping back is a service restart. -### 0a — Fix blockers (only the blockers) +### 0a — Fix blockers only -Walk `handoff.steps`. A step is **blocking** only if its failure prevents v2 from routing a single message. Treat these as blockers: +A step is **blocking** if its failure stops the bot from routing one message. Re-run or hand-fix only these; defer everything else to its later phase: -| Step | Why blocking | -|------|--------------| -| `1b-db` | No `messaging_groups` → router has nothing to match | -| `1d-sessions` | No session → no inbound DB to write into | -| `2c-install-` | No adapter for the channel the user wants to test | -| `2d-whatsapp-lids` | WhatsApp DMs may arrive as `@lid` and miss migrated phone-keyed rows | -| `3a-docker` / `3e-build` | No container image → agent can't run | -| `3b-onecli` | Anthropic credentials not injected → first agent call 401s | +| Blocking | Deferred | +|---|---| +| `1b-db`, `1d-sessions`, `2c-install-`, `2d-whatsapp-lids`, `3a-docker`, `3b-onecli`, `3e-build` | `1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth` | -**Defer** these — they don't block a smoke test, and most surface naturally in later phases: +### 0b — Smoke test, then continue -- `1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth` +Tell the user the switch is non-destructive (v1 is paused, not modified; reverting is one command). Help them stop v1's service unit and start v2's, tail the host log for a clean boot, and have them send a real test message. Use `AskUserQuestion` to confirm the bot responded. -For each blocker: read `handoff.step_logs_dir/.log`, identify the cause, re-run the underlying script directly (`pnpm exec tsx setup/migrate-v2/.ts `) or hand-fix mechanically. Use `AskUserQuestion` for judgment calls. Don't simulate the script's work. +If yes, continue to Phase 1. If no, diagnose from `logs/nanoclaw.log` and re-test — don't proceed to deeper work on a broken router. -Common blockers: -- **`1b-db` failed**: JID couldn't be parsed. Insert `agent_groups` + `messaging_groups` for the user's confirmed channel. -- **`2c-install-` failed**: `git fetch origin channels` issue. The user can run `bash setup/install-.sh` directly. -- **`3e-build` failed**: usually stale builder cache. `docker buildx prune -f && ./container/build.sh`. +### Deferred failures -### 0b — Smoke test before any further migration work - -Tell the user, verbatim: - -> Before we touch CLAUDE.local.md or fork customisations, let's confirm v2 actually answers your real messages. **This is non-destructive — v1 is just paused, not touched.** v1 and v2 share your WhatsApp identity (we copied `store/auth/` over), so only one can be online at a time, but flipping back is instant. - -Find the v2 service unit (per-checkout hash): - -```bash -# macOS -launchctl list | grep nanoclaw -# Linux -systemctl --user list-units 'nanoclaw*' -``` - -Switch v1 → v2: - -```bash -# macOS -launchctl unload ~/Library/LaunchAgents/com.nanoclaw.plist -launchctl load ~/Library/LaunchAgents/com.nanoclaw-v2-.plist - -# Linux -systemctl --user stop nanoclaw -systemctl --user start nanoclaw-v2- -``` - -Tail the log and confirm clean boot: - -```bash -tail -f logs/nanoclaw.log -``` - -Watch for `NanoClaw running` plus `Channel adapter started` for each installed channel (and `Connected to ` for native adapters like WhatsApp). - -Ask the user to send a real test message — a DM to the bot, or a post in a known group from a non-bot account. A working route logs an inbound event → session resolution → container spawn → outbound delivery. - -`AskUserQuestion`: *"Did v2 respond? — Yes / No, here's what happened."* - -**If yes**: continue to Phase 1. - -**If no**: do not proceed. Read `logs/nanoclaw.log` + `logs/nanoclaw.error.log` and diagnose. Common patterns: -- WhatsApp DM with no routing chain in the log → check `SELECT platform_id FROM messaging_groups WHERE platform_id LIKE '%@lid'`. If empty, re-run `setup/migrate-v2/whatsapp-resolve-lids.ts`. -- Agent inside container fails on Anthropic 401 → OneCLI agents start in `selective` secret mode. `onecli agents set-secret-mode --id --mode all`. -- Channel disconnected silently → restart: `launchctl kickstart -k gui/$(id -u)/com.nanoclaw-v2-`. - -Re-test before continuing. - -### Reverting (anytime — not just now) - -```bash -# macOS — back to v1 -launchctl unload ~/Library/LaunchAgents/com.nanoclaw-v2-.plist -launchctl load ~/Library/LaunchAgents/com.nanoclaw.plist - -# Linux -systemctl --user stop nanoclaw-v2- -systemctl --user start nanoclaw -``` - -v1's process, data, credentials, and groups are untouched the whole time. Reverting is just a service restart. - -### Deferred non-blocker failures - -If you skipped non-blocker failures in 0a (`1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth`), they still need fixing — most surface naturally in later phases (`1c-groups` ↔ Phase 2 CLAUDE.local.md cleanup, `1e-tasks` ↔ task verification). Re-run any that don't get covered before declaring the migration done. +Re-visit anything you skipped in 0a before declaring the migration done. Most surface naturally in later phases (`1c-groups` ↔ Phase 2, `1e-tasks` ↔ task verification). ## Phase 1: Owner and access From 8c1b209aeb8cdd2776fcd1afa92569d9f16629e2 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 18:36:10 +0300 Subject: [PATCH 18/23] docs(migrate-from-v1): 2b-channel-auth and 3c-auth are blockers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 2b-channel-auth: copies the Baileys keystore + channel-specific env keys. Without it WhatsApp can't connect — saw this firsthand when the original candidatePaths bug left env_keys=0,files=0. 3c-auth: registers Anthropic credentials in OneCLI. 3b installs the gateway; 3c puts the secret in the vault. Without 3c every agent request 401s regardless of 3b's status. 1c-groups stays deferred — agent runs on stock CLAUDE.md without it, but routing works. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 0440d8a..04b3aee 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -47,7 +47,7 @@ A step is **blocking** if its failure stops the bot from routing one message. Re | Blocking | Deferred | |---|---| -| `1b-db`, `1d-sessions`, `2c-install-`, `2d-whatsapp-lids`, `3a-docker`, `3b-onecli`, `3e-build` | `1a-env`, `1c-groups`, `1e-tasks`, `2b-channel-auth`, `3c-auth` | +| `1b-db`, `1d-sessions`, `2b-channel-auth`, `2c-install-`, `2d-whatsapp-lids`, `3a-docker`, `3b-onecli`, `3c-auth`, `3e-build` | `1a-env`, `1c-groups`, `1e-tasks` | ### 0b — Smoke test, then continue From 7922a19af73f4a2cd99bbf1ef01107756e8faada Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 18:40:07 +0300 Subject: [PATCH 19/23] docs(migrate-from-v1): drop the blocker/deferred table Trust the agent to figure out which failed steps actually stop routing. The rule is the goal ("can the bot route one message?"), not a hardcoded list. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/migrate-from-v1/SKILL.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md index 04b3aee..e36a005 100644 --- a/.claude/skills/migrate-from-v1/SKILL.md +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -43,11 +43,7 @@ Before any deeper migration work, prove v2 actually answers messages on the user ### 0a — Fix blockers only -A step is **blocking** if its failure stops the bot from routing one message. Re-run or hand-fix only these; defer everything else to its later phase: - -| Blocking | Deferred | -|---|---| -| `1b-db`, `1d-sessions`, `2b-channel-auth`, `2c-install-`, `2d-whatsapp-lids`, `3a-docker`, `3b-onecli`, `3c-auth`, `3e-build` | `1a-env`, `1c-groups`, `1e-tasks` | +Walk `handoff.steps`. Fix only the failures that would stop the bot from routing one message; defer the rest to its later phase. ### 0b — Smoke test, then continue From 8181054bdbf675b71e6ff84f77f79d8dbe773988 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 16:04:39 +0000 Subject: [PATCH 20/23] fix(migrate-v2): resolve Discord DMs as discord:@me: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The resolver only enumerated guild channels, so any v1 install whose registered Discord chat was a DM (a common case for personal-bot installs) failed 1b-db with "not found in any guild" — leaving the migration without an agent_group or wiring, and the user with a bot that received messages but had nowhere to route them. Add an unresolved-channel classification pass: for any v1 channel id not found in a guild, GET /channels/ and emit discord:@me: when the type is DM (1) or GROUP_DM (3). Matches the runtime adapter's guild_id || "@me" encoding. Other types / 404 / 403 keep current skip-with-warning behavior. Caller passes the v1 channel id list (already on hand). Test coverage extends the existing mock-fetch pattern with DM, GROUP_DM, orphan, and dedupe cases. --- setup/migrate-v2/db.ts | 24 +++--- setup/migrate-v2/discord-resolver.test.ts | 92 +++++++++++++++++++++-- setup/migrate-v2/discord-resolver.ts | 90 +++++++++++++++++----- 3 files changed, 174 insertions(+), 32 deletions(-) diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts index da2bab7..c2b5b48 100644 --- a/setup/migrate-v2/db.ts +++ b/setup/migrate-v2/db.ts @@ -82,21 +82,27 @@ async function main(): Promise { let skipped = 0; const errors: string[] = []; - // v1 stored Discord groups as `dc:` (no guildId). v2 needs - // `discord::`. If there are any Discord groups, use - // the bot token (carried forward by 1a-env) to look up each channel's - // guild via the Discord API. On any failure the resolver returns null - // for every channel and the affected groups skip with a clear warning. + // v1 stored Discord groups as `dc:` with no guild/DM signal. + // v2 needs either `discord::` (guild) or + // `discord:@me:` (DM / group DM). Use the v1 bot token to + // enumerate guilds + channels and to classify any leftover ids as DMs. + // On any failure the resolver returns null for every channel and the + // affected groups skip with a clear warning. let discordResolver: DiscordResolver | null = null; - const hasDiscord = v1Groups.some((g) => parseJid(g.jid)?.channel_type === 'discord'); - if (hasDiscord) { + const discordChannelIds = v1Groups + .map((g) => parseJid(g.jid)) + .filter((p): p is NonNullable => p?.channel_type === 'discord') + .map((p) => p.id); + if (discordChannelIds.length > 0) { const env = readEnvFile(['DISCORD_BOT_TOKEN']); - discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? ''); + discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? '', discordChannelIds); const stats = discordResolver.stats(); if (stats.reason) { console.log(`WARN:discord resolver disabled: ${stats.reason}`); } else { - console.log(`INFO:discord resolver: ${stats.guilds} guild(s), ${stats.channels} channel(s)`); + console.log( + `INFO:discord resolver: ${stats.guilds} guild(s), ${stats.channels} guild channel(s), ${stats.dms} DM(s)`, + ); } } diff --git a/setup/migrate-v2/discord-resolver.test.ts b/setup/migrate-v2/discord-resolver.test.ts index 31e63f7..e74fc40 100644 --- a/setup/migrate-v2/discord-resolver.test.ts +++ b/setup/migrate-v2/discord-resolver.test.ts @@ -20,7 +20,7 @@ function mockFetch(handlers: Record): typeof fetch { describe('buildDiscordResolver', () => { it('returns empty resolver when token is missing', async () => { const r = await buildDiscordResolver(''); - expect(r.stats()).toMatchObject({ guilds: 0, channels: 0 }); + expect(r.stats()).toMatchObject({ guilds: 0, channels: 0, dms: 0 }); expect(r.stats().reason).toMatch(/no DISCORD_BOT_TOKEN/); expect(r.resolve('any')).toBeNull(); }); @@ -40,9 +40,9 @@ describe('buildDiscordResolver', () => { ], }); - const r = await buildDiscordResolver('valid-token', fetchImpl); + const r = await buildDiscordResolver('valid-token', [], fetchImpl); - expect(r.stats()).toEqual({ guilds: 2, channels: 3 }); + expect(r.stats()).toEqual({ guilds: 2, channels: 3, dms: 0 }); expect(r.resolve('c1')).toBe('discord:g1:c1'); expect(r.resolve('c2')).toBe('discord:g1:c2'); expect(r.resolve('c3')).toBe('discord:g2:c3'); @@ -58,7 +58,7 @@ describe('buildDiscordResolver', () => { }, }); - const r = await buildDiscordResolver('bad-token', fetchImpl); + const r = await buildDiscordResolver('bad-token', [], fetchImpl); expect(r.stats().guilds).toBe(0); expect(r.stats().reason).toMatch(/401/); expect(r.resolve('c1')).toBeNull(); @@ -79,7 +79,7 @@ describe('buildDiscordResolver', () => { }); const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); - const r = await buildDiscordResolver('valid-token', fetchImpl); + const r = await buildDiscordResolver('valid-token', [], fetchImpl); errSpy.mockRestore(); expect(r.resolve('c1')).toBe('discord:g1:c1'); @@ -105,11 +105,91 @@ describe('buildDiscordResolver', () => { return new Response(JSON.stringify([{ id: `c-${gid}` }]), { status: 200 }); }) as unknown as typeof fetch; - const r = await buildDiscordResolver('valid-token', fetchImpl); + const r = await buildDiscordResolver('valid-token', [], fetchImpl); expect(r.stats().guilds).toBe(201); expect(r.stats().channels).toBe(201); expect(r.resolve('c-g0')).toBe('discord:g0:c-g0'); expect(r.resolve('c-g200')).toBe('discord:g200:c-g200'); }); + + it('classifies unresolved ids as DMs and emits discord:@me:', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [{ id: 'g1', name: 'G1' }], + 'https://discord.com/api/v10/guilds/g1/channels': [{ id: 'guild-chan' }], + // dmId is a 1:1 DM (type=1) + 'https://discord.com/api/v10/channels/dmId': { id: 'dmId', type: 1 }, + // groupDmId is a multi-recipient DM (type=3) + 'https://discord.com/api/v10/channels/groupDmId': { id: 'groupDmId', type: 3 }, + }); + + const r = await buildDiscordResolver( + 'valid-token', + ['guild-chan', 'dmId', 'groupDmId'], + fetchImpl, + ); + + expect(r.stats()).toEqual({ guilds: 1, channels: 1, dms: 2 }); + expect(r.resolve('guild-chan')).toBe('discord:g1:guild-chan'); + expect(r.resolve('dmId')).toBe('discord:@me:dmId'); + expect(r.resolve('groupDmId')).toBe('discord:@me:groupDmId'); + }); + + it('leaves ids unresolved when classify returns 404 or non-DM type', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [], + // 404 — bot has no access (typical when bot was kicked from the guild) + 'https://discord.com/api/v10/channels/orphanId': { + status: 404, + statusText: 'Not Found', + body: '{"message":"Unknown Channel","code":10003}', + }, + // type=0 — guild text channel in a guild we no longer enumerate (shouldn't happen, + // but the fallback is conservative: only emit @me for type 1/3) + 'https://discord.com/api/v10/channels/leftoverGuildChan': { + id: 'leftoverGuildChan', + type: 0, + }, + }); + + const r = await buildDiscordResolver( + 'valid-token', + ['orphanId', 'leftoverGuildChan'], + fetchImpl, + ); + + expect(r.stats()).toEqual({ guilds: 0, channels: 0, dms: 0 }); + expect(r.resolve('orphanId')).toBeNull(); + expect(r.resolve('leftoverGuildChan')).toBeNull(); + }); + + it('skips classify for ids already found in a guild and dedupes input', async () => { + let dmCallCount = 0; + const fetchImpl = vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (url.includes('/users/@me/guilds')) { + return new Response(JSON.stringify([{ id: 'g1', name: 'G1' }]), { status: 200 }); + } + if (url.includes('/guilds/g1/channels')) { + return new Response(JSON.stringify([{ id: 'guild-chan' }]), { status: 200 }); + } + if (url.includes('/channels/dmId')) { + dmCallCount++; + return new Response(JSON.stringify({ id: 'dmId', type: 1 }), { status: 200 }); + } + throw new Error(`unexpected fetch: ${url}`); + }) as unknown as typeof fetch; + + // 'guild-chan' is in the guild map (skip classify); 'dmId' appears twice + // in the input (classify exactly once). + const r = await buildDiscordResolver( + 'valid-token', + ['guild-chan', 'dmId', 'dmId'], + fetchImpl, + ); + + expect(dmCallCount).toBe(1); + expect(r.resolve('guild-chan')).toBe('discord:g1:guild-chan'); + expect(r.resolve('dmId')).toBe('discord:@me:dmId'); + }); }); diff --git a/setup/migrate-v2/discord-resolver.ts b/setup/migrate-v2/discord-resolver.ts index 17b9a9f..ecc1d5e 100644 --- a/setup/migrate-v2/discord-resolver.ts +++ b/setup/migrate-v2/discord-resolver.ts @@ -1,11 +1,18 @@ /** - * Discord channel → guild resolver for the v1 → v2 migration. + * Discord channel → platform_id resolver for the v1 → v2 migration. * - * v1 stored Discord groups as `dc:` — only the channel id, not - * the guild id. v2's `@chat-adapter/discord` encodes `platform_id` as - * `discord::`, so we can't reconstruct it from v1 data - * alone. Instead, we use the v1 bot token (carried forward by 1a-env) to - * query the Discord API and build a channelId → guildId map. + * v1 stored Discord groups as `dc:` — only the channel id, with + * no signal for guild vs. DM. v2's `@chat-adapter/discord` encodes + * `platform_id` as either `discord::` (guild channel) + * or `discord:@me:` (DM / group DM) — see `guild_id || "@me"` + * in the runtime adapter. We can't reconstruct that from v1 data alone, so + * we use the v1 bot token (carried forward by 1a-env) to query Discord: + * 1. Enumerate every guild the bot is in and every channel in those + * guilds → channelId → guildId map. + * 2. For any v1 channel id NOT in that map, classify via `GET + * /channels/` — DM (type=1) and GROUP_DM (type=3) get + * `discord:@me:`. Anything else returns null and the caller + * skips with a warning. * * Network calls are best-effort: on auth failure or network error, the * resolver returns null for every channel and the caller falls back to @@ -14,6 +21,11 @@ const DISCORD_API = 'https://discord.com/api/v10'; +// Discord channel types we care about. See: +// https://discord.com/developers/docs/resources/channel#channel-object-channel-types +const CHANNEL_TYPE_DM = 1; +const CHANNEL_TYPE_GROUP_DM = 3; + interface Guild { id: string; name: string; @@ -24,18 +36,27 @@ interface Channel { name?: string; } +interface ChannelInfo { + id: string; + type: number; +} + export interface DiscordResolver { - /** Returns `discord::` or null if the channel isn't visible to the bot. */ + /** + * Returns the v2 `platform_id` for a v1 channel id, or null if the bot + * can't see it. Format is `discord::` for guild + * channels and `discord:@me:` for DMs / group DMs. + */ resolve(channelId: string): string | null; - /** Diagnostic info — guild count and total channel count discovered. */ - stats(): { guilds: number; channels: number; reason?: string }; + /** Diagnostic info — guild count, channel count, DM count, optional disable reason. */ + stats(): { guilds: number; channels: number; dms: number; reason?: string }; } /** A no-op resolver that returns null for every lookup with a stored reason. */ function emptyResolver(reason: string): DiscordResolver { return { resolve: () => null, - stats: () => ({ guilds: 0, channels: 0, reason }), + stats: () => ({ guilds: 0, channels: 0, dms: 0, reason }), }; } @@ -57,14 +78,20 @@ async function getJson(url: string, token: string, fetchImpl: FetchFn): Promi /** * Build a Discord resolver by enumerating every guild the bot is in and - * every channel in those guilds. Returns an empty resolver on any error. + * every channel in those guilds, then classifying any `unresolvedChannelIds` + * that didn't show up in a guild via `GET /channels/` (so DMs and + * group DMs can be encoded as `discord:@me:`). * - * Costs: 1 + N HTTP calls (N = guild count). Discord's global rate limit - * is 50 req/s; even installs with hundreds of guilds finish in under a - * second of network time. + * Returns an empty resolver on any error during guild enumeration. + * + * Costs: 1 + N + K HTTP calls — N = guild count (enumerated channels per + * guild), K = unresolved-channel classification calls. Discord's global + * rate limit is 50 req/s; even installs with hundreds of guilds finish in + * under a second of network time. */ export async function buildDiscordResolver( token: string, + unresolvedChannelIds: string[] = [], fetchImpl: FetchFn = fetch, ): Promise { if (!token) return emptyResolver('no DISCORD_BOT_TOKEN in .env'); @@ -109,12 +136,41 @@ export async function buildDiscordResolver( } } + // Classify any v1 channel ids that didn't surface in a guild — they're + // most likely DMs (type=1) or group DMs (type=3). Anything else (404, + // 403, type=0 in a guild the bot left) stays unresolved so the caller's + // existing skip-with-warning path fires. + const dmChannels = new Set(); + const seen = new Set(); + for (const channelId of unresolvedChannelIds) { + if (channelToGuild.has(channelId)) continue; + if (seen.has(channelId)) continue; + seen.add(channelId); + try { + const ch = await getJson( + `${DISCORD_API}/channels/${channelId}`, + token, + fetchImpl, + ); + if (ch.type === CHANNEL_TYPE_DM || ch.type === CHANNEL_TYPE_GROUP_DM) { + dmChannels.add(channelId); + } + } catch { + // Channel not visible to the bot — leave it unresolved. + } + } + return { resolve(channelId: string): string | null { const guildId = channelToGuild.get(channelId); - if (!guildId) return null; - return `discord:${guildId}:${channelId}`; + if (guildId) return `discord:${guildId}:${channelId}`; + if (dmChannels.has(channelId)) return `discord:@me:${channelId}`; + return null; }, - stats: () => ({ guilds: guilds.length, channels: channelToGuild.size }), + stats: () => ({ + guilds: guilds.length, + channels: channelToGuild.size, + dms: dmChannels.size, + }), }; } From 7dbedad9bde26650ae2bec60f84c989ab770bc52 Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 16:06:45 +0000 Subject: [PATCH 21/23] fix(migrate-v2): skip symlinks in group copyTree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fs.copyFileSync follows symlinks, so a single broken/dangling link in v1 (e.g. .claude-shared.md → /app/CLAUDE.md, a container-side path that doesn't resolve on the host) crashed the alphabetical traversal with ENOENT — preventing later folders, including the actual registered group, from being copied. Check entry.isSymbolicLink() and skip with a one-line log. v2 uses composed CLAUDE.md fragments, so v1's container-path symlinks have no v2 meaning and don't need to be carried forward. --- setup/migrate-v2/groups.ts | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/setup/migrate-v2/groups.ts b/setup/migrate-v2/groups.ts index beb88be..d1e9ed0 100644 --- a/setup/migrate-v2/groups.ts +++ b/setup/migrate-v2/groups.ts @@ -18,7 +18,16 @@ import Database from 'better-sqlite3'; const SKIP_NAMES = new Set(['CLAUDE.md', 'logs', '.git', '.DS_Store', 'node_modules']); -/** Copy a directory tree, skipping SKIP_NAMES. Never overwrites existing files. */ +/** + * Copy a directory tree, skipping SKIP_NAMES. Never overwrites existing files. + * + * Symlinks are skipped, not followed: v1 group folders sometimes contain + * container-side paths like `.claude-shared.md → /app/CLAUDE.md` that + * don't resolve on the host. Following them with `fs.copyFileSync` would + * crash ENOENT on a broken target and abort the rest of the traversal. + * v2 uses composed CLAUDE.md fragments anyway — these v1 symlinks have no + * v2 meaning and don't need to be carried forward. + */ function copyTree(src: string, dst: string): number { let written = 0; if (!fs.existsSync(src)) return 0; @@ -29,6 +38,10 @@ function copyTree(src: string, dst: string): number { const s = path.join(src, entry.name); const d = path.join(dst, entry.name); + if (entry.isSymbolicLink()) { + console.log(`SKIP:symlink ${path.relative(process.cwd(), s)}`); + continue; + } if (entry.isDirectory()) { written += copyTree(s, d); continue; From 3b5e5a24f43b50acd5280f4e873335511b5e76fc Mon Sep 17 00:00:00 2001 From: Gavriel Cohen Date: Sat, 2 May 2026 16:07:43 +0000 Subject: [PATCH 22/23] fix(migrate-v2): reset auto-created messaging_group policy on re-run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit If 1b-db is re-run after the v2 service has already started (e.g. recovering from an earlier failure), the messaging_group it would otherwise create may already exist — auto-created by the runtime router on the first inbound message, with the router's default unknown_sender_policy ('request_approval'), not the migration's intent ('public'). The previous reuse path skipped creation but never updated the policy, so re-runs left the bot hanging every message waiting for an approver that wasn't seeded yet. When reusing an existing row that has zero wired agent_groups (signal of a router auto-create), reset the policy to 'public'. Once any wiring exists, the user has had a chance to tighten via the skill — leave it. Also adds a CHANGELOG entry covering this and the two sibling fixes (Discord DM resolution, symlink skip in copyTree). --- CHANGELOG.md | 1 + setup/migrate-v2/db.ts | 18 +++++++++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e297d3a..2ec9fc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ For detailed release notes, see the [full changelog on the documentation site](h ## [Unreleased] - **v1 → v2 migration.** Run `bash migrate-v2.sh` from the v2 checkout. Finds your v1 install (sibling directory or `NANOCLAW_V1_PATH`), merges `.env`, seeds the v2 DB from `registered_groups`, copies group folders (`CLAUDE.md` → `CLAUDE.local.md`), copies session data with conversation continuity, ports scheduled tasks, interactively selects and installs channels (clack multiselect), copies container skills, builds the agent container, and offers a service switchover to test. Hands off to Claude (`/migrate-from-v1`) for owner seeding, access policy, CLAUDE.md cleanup, and fork customization porting. See [docs/migration-dev.md](docs/migration-dev.md) and [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). +- **Migration fixes.** `1b-db` now resolves Discord DMs as `discord:@me:` (previously skipped any v1 chat that wasn't a guild channel — a blocker for personal-bot installs). `1c-groups` skips symlinks instead of following them (a single broken `.claude-shared.md → /app/CLAUDE.md` no longer aborts the whole copy). When `1b-db` reuses an auto-created `messaging_group` with no wired agents, its `unknown_sender_policy` is now reconciled to the migration's `public` default. ## [2.0.0] - 2026-04-22 diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts index c2b5b48..ada4764 100644 --- a/setup/migrate-v2/db.ts +++ b/setup/migrate-v2/db.ts @@ -22,7 +22,9 @@ import { createMessagingGroup, createMessagingGroupAgent, getMessagingGroupAgentByPair, + getMessagingGroupAgents, getMessagingGroupByPlatform, + updateMessagingGroup, } from '../../src/db/messaging-groups.js'; import { runMigrations } from '../../src/db/migrations/index.js'; import { readEnvFile } from '../../src/env.js'; @@ -147,7 +149,15 @@ async function main(): Promise { ag = getAgentGroupByFolder(g.folder)!; } - // messaging_group — one per (channel_type, platform_id) + // messaging_group — one per (channel_type, platform_id). + // + // If the row already exists *and* has zero wired agent_groups, it + // was almost certainly auto-created by the runtime router on an + // inbound message (which uses 'request_approval' or similar — not + // the migration's 'public'). Reset its policy to match what the + // migration would have set if it had created the row first. Once + // any wiring exists, the user has had a chance to tighten the + // policy via the skill — leave it alone. let mg = getMessagingGroupByPlatform(channelType, platformId); if (!mg) { createMessagingGroup({ @@ -160,6 +170,12 @@ async function main(): Promise { created_at: createdAt, }); mg = getMessagingGroupByPlatform(channelType, platformId)!; + } else if ( + mg.unknown_sender_policy !== 'public' && + getMessagingGroupAgents(mg.id).length === 0 + ) { + updateMessagingGroup(mg.id, { unknown_sender_policy: 'public' }); + mg = getMessagingGroupByPlatform(channelType, platformId)!; } // messaging_group_agents — wire them From 82216b536dbf70e18e3af8e0a9370610a71e1189 Mon Sep 17 00:00:00 2001 From: gavrielc Date: Sat, 2 May 2026 21:19:23 +0300 Subject: [PATCH 23/23] Add /add-deltachat skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Skill files only — copied from PR #2192 (channels branch). Source adapter (src/channels/deltachat.ts) lives on the channels branch and is installed by the skill. Co-Authored-By: Axel McLaren Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/add-deltachat/REMOVE.md | 62 ++++++ .claude/skills/add-deltachat/SKILL.md | 254 +++++++++++++++++++++++++ .claude/skills/add-deltachat/VERIFY.md | 54 ++++++ 3 files changed, 370 insertions(+) create mode 100644 .claude/skills/add-deltachat/REMOVE.md create mode 100644 .claude/skills/add-deltachat/SKILL.md create mode 100644 .claude/skills/add-deltachat/VERIFY.md diff --git a/.claude/skills/add-deltachat/REMOVE.md b/.claude/skills/add-deltachat/REMOVE.md new file mode 100644 index 0000000..7cb2d31 --- /dev/null +++ b/.claude/skills/add-deltachat/REMOVE.md @@ -0,0 +1,62 @@ +# Remove DeltaChat + +## 1. Disable the adapter + +Comment out the import in `src/channels/index.ts`: + +```typescript +// import './deltachat.js'; +``` + +## 2. Remove credentials + +Remove the `DC_*` lines from `.env`: + +```bash +DC_EMAIL +DC_PASSWORD +DC_IMAP_HOST +DC_IMAP_PORT +DC_SMTP_HOST +DC_SMTP_PORT +``` + +## 3. Rebuild and restart + +```bash +pnpm run build + +# Linux +systemctl --user restart nanoclaw + +# macOS +launchctl kickstart -k gui/$(id -u)/com.nanoclaw +``` + +## 4. Remove account data (optional) + +To fully remove all account data including DeltaChat encryption keys: + +```bash +rm -rf dc-account/ +``` + +> **Warning:** This deletes the Autocrypt keys. Contacts who have verified your bot's key will need to re-verify if the same email address is re-used with a new account. + +To keep the account for later reinstall, leave `dc-account/` intact. + +## 5. Remove the package (optional) + +```bash +pnpm remove @deltachat/stdio-rpc-server +``` + +## Verification + +After removal, confirm the adapter is no longer starting: + +```bash +grep "deltachat" logs/nanoclaw.log | tail -5 +``` + +Expected: no `Channel adapter started` entry after the last restart. diff --git a/.claude/skills/add-deltachat/SKILL.md b/.claude/skills/add-deltachat/SKILL.md new file mode 100644 index 0000000..45aa416 --- /dev/null +++ b/.claude/skills/add-deltachat/SKILL.md @@ -0,0 +1,254 @@ +--- +name: add-deltachat +description: Add DeltaChat channel integration via @deltachat/stdio-rpc-server. Native adapter — no Chat SDK bridge. Email-based messaging with end-to-end encryption. +--- + +# Add DeltaChat Channel + +The adapter drives the `@deltachat/stdio-rpc-server` JSON-RPC subprocess directly — pure Node.js against the DeltaChat core library. Messages are delivered over email with Autocrypt/OpenPGP encryption. + +## Install + +### Pre-flight (idempotent) + +Skip to **Credentials** if all of these are already in place: + +- `src/channels/deltachat.ts` exists +- `src/channels/index.ts` contains `import './deltachat.js';` +- `@deltachat/stdio-rpc-server` is listed in `package.json` dependencies + +Otherwise continue. Every step below is safe to re-run. + +### 1. Fetch the channels branch + +```bash +git fetch origin channels +``` + +### 2. Copy the adapter + +```bash +git show origin/channels:src/channels/deltachat.ts > src/channels/deltachat.ts +``` + +### 3. Append the self-registration import + +Append to `src/channels/index.ts` (skip if already present): + +```typescript +import './deltachat.js'; +``` + +### 4. Install the adapter package (pinned) + +```bash +pnpm install @deltachat/stdio-rpc-server@2.49.0 +``` + +### 5. Build + +```bash +pnpm run build +``` + +## Account Setup + +A dedicated email account is strongly recommended — it will accumulate DeltaChat-formatted messages and store encryption keys. Not all providers work well with DeltaChat; check https://providers.delta.chat/ before picking one. + +**Default security modes:** IMAP uses SSL/TLS (port 993), SMTP uses STARTTLS (port 587). Both are configurable via `.env` — see Credentials below. + +To find the correct hostnames for a domain: + +```bash +node -e "require('dns').resolveMx('example.com', (e,r) => console.log(r))" +``` + +Most providers publish their IMAP/SMTP hostnames in their help docs under "manual setup" or "IMAP access." + +## Credentials + +Add to `.env`: + +```bash +DC_EMAIL=bot@example.com +DC_PASSWORD=your-app-password +DC_IMAP_HOST=imap.example.com +DC_IMAP_PORT=993 +DC_IMAP_SECURITY=1 # 1=SSL/TLS (default), 2=STARTTLS, 3=plain +DC_SMTP_HOST=smtp.example.com +DC_SMTP_PORT=587 +DC_SMTP_SECURITY=2 # 2=STARTTLS (default), 1=SSL/TLS, 3=plain +``` + +Security settings are applied on every startup, so changing them in `.env` and restarting takes effect without wiping the account. + +Sync to container: `mkdir -p data/env && cp .env data/env/env` + +### Optional settings + +The following are read from the process environment (not `.env`). To override them, add `Environment=` lines to the systemd service unit or your launchd plist: + +| Variable | Default | Description | +|----------|---------|-------------| +| `DC_ACCOUNT_DIR` | `dc-account` | Directory for DeltaChat account data (IMAP state, keys, blobs) | +| `DC_DISPLAY_NAME` | `NanoClaw` | Bot display name shown in DeltaChat | +| `DC_AVATAR_PATH` | _(none)_ | Absolute path to avatar image; set at startup only | + +The `/set-avatar` command (send an image with that caption) is the easiest way to set the avatar at runtime without modifying the service file. Only users with `owner` or global `admin` role can use it. + +### Restart + +```bash +# Linux +systemctl --user restart nanoclaw + +# macOS +launchctl kickstart -k gui/$(id -u)/com.nanoclaw +``` + +On first start the adapter configures the email account (IMAP/SMTP credentials, calls `configure()`). Subsequent starts skip straight to `startIo()`. Account data is stored in `dc-account/` in the project root (or your `DC_ACCOUNT_DIR`). + +## Wiring + +### DMs + +**DeltaChat contacts cannot be added by email alone** — to start a chat, the user must open the bot's invite link in their DeltaChat app or scan its QR code. This triggers the SecureJoin handshake. + +#### Step 1 — Get the invite link + +After the service starts, the adapter logs the invite URL and writes a QR SVG: + +```bash +grep "invite link" logs/nanoclaw.log | tail -1 +# url field contains the https://i.delta.chat/... invite link +# also written to dc-account/invite-qr.svg (or $DC_ACCOUNT_DIR/invite-qr.svg) +``` + +The invite URL is stable (tied to the bot's email and encryption keys) so it stays valid across restarts. + +#### Step 2 — Add the bot in DeltaChat + +Two options for the user to connect: + +- **Link**: Copy the `https://i.delta.chat/...` URL and open it on the device running DeltaChat. The app recognises it and shows a "Start chat" prompt. +- **QR code**: Open `dc-account/invite-qr.svg` in a browser or image viewer, display it on screen, and scan it from the DeltaChat app using the QR-scan button on the new-chat screen. + +After accepting, DeltaChat exchanges keys and creates the chat automatically. + +#### Step 3 — Wire the chat to an agent + +Once the first message arrives the router auto-creates a `messaging_groups` row. Look up the chat ID: + +```bash +sqlite3 data/v2.db \ + "SELECT platform_id, name FROM messaging_groups WHERE channel_type='deltachat' AND is_group=0 ORDER BY created_at DESC LIMIT 5" +``` + +Then run `/init-first-agent` — it creates the agent group, grants the user owner access, and wires the messaging group in one step: + +```bash +pnpm exec tsx scripts/init-first-agent.ts \ + --channel deltachat \ + --user-id deltachat:user@example.com \ + --platform-id \ + --display-name "Your Name" +``` + +### Groups + +Add the bot email to a DeltaChat group. When any member sends a message, the router creates a `messaging_groups` row with `is_group = 1`. Run `/manage-channels` to wire it to an agent group. + +## Next Steps + +If you're in the middle of `/setup`, return to the setup flow now. + +Otherwise, run `/init-first-agent` to create an agent and wire it to your DeltaChat DM (see Wiring above), or `/manage-channels` to wire this channel to an existing agent group. + +## Channel Info + +- **type**: `deltachat` +- **terminology**: DeltaChat calls them "chats" (1:1 DMs) and "groups" +- **supports-threads**: no — DeltaChat has no thread model +- **platform-id-format**: numeric chat ID as a string (e.g. `"12"`) — the DeltaChat core's internal chat identifier +- **user-id-format**: `deltachat:{email}` — the contact's email address +- **how-to-find-id**: Send a message from DeltaChat to the bot email, then query `messaging_groups` as shown above +- **typical-use**: Personal assistant over DeltaChat DMs; small groups where participants use DeltaChat +- **default-isolation**: One agent per bot identity. Multiple chats with the same operator can share an agent group; groups with other people should typically use `isolated` session mode + +### Features + +- File attachments — inbound and outbound; inbound waits up to 30 seconds for large-message download to complete +- Invite link logged on every startup — URL + QR SVG written to `dc-account/invite-qr.svg`; see Wiring for the bootstrap flow +- `/set-avatar` — send an image with this caption to change the bot's DeltaChat avatar (admin/owner only) +- Connectivity watchdog — restarts IO if IMAP goes quiet for 20 minutes or connectivity drops below threshold for two consecutive 5-minute checks +- Network nudge — `maybeNetwork()` called every 10 minutes to recover from prolonged idle + +Not supported: DeltaChat reactions, message editing/deletion, read receipts. + +### Connectivity model + +`isConnected()` returns `true` when the internal connectivity value is ≥ 3000: + +| Range | Meaning | +|-------|---------| +| 1000–1999 | Not connected | +| 2000–2999 | Connecting | +| 3000–3999 | Working (IMAP fetching) | +| ≥ 4000 | Fully connected (IMAP IDLE) | + +## Troubleshooting + +### Adapter not starting — credentials missing + +```bash +grep "Channel credentials missing" logs/nanoclaw.log | grep deltachat +``` + +All six required vars (`DC_EMAIL`, `DC_PASSWORD`, `DC_IMAP_HOST`, `DC_IMAP_PORT`, `DC_SMTP_HOST`, `DC_SMTP_PORT`) must be present in `.env`. + +### Account configure fails + +```bash +grep "DeltaChat" logs/nanoclaw.log | tail -20 +``` + +Common causes: +- Wrong IMAP/SMTP hostnames — double-check provider docs +- App password not generated — Gmail and some others require this when 2FA is enabled +- Port/security mismatch — defaults are port 993 + SSL/TLS for IMAP and port 587 + STARTTLS for SMTP; override with `DC_IMAP_PORT`/`DC_IMAP_SECURITY` or `DC_SMTP_PORT`/`DC_SMTP_SECURITY` in `.env` + +### Provider uses SMTP port 465 (SSL/TLS) instead of 587 + +Set `DC_SMTP_SECURITY=1` and `DC_SMTP_PORT=465` in `.env`, then restart. + +### Messages not arriving + +1. Check the service is running and the adapter started: `grep "Channel adapter started.*deltachat" logs/nanoclaw.log` +2. Check connectivity: `grep "DeltaChat: IO started" logs/nanoclaw.log` +3. Check the sender has been granted access — run `/init-first-agent` to create their user record and wire the chat +4. Verify the messaging group is wired: `sqlite3 data/v2.db "SELECT mg.platform_id, mga.agent_group_id FROM messaging_groups mg JOIN messaging_group_agents mga ON mg.id = mga.messaging_group_id WHERE mg.channel_type='deltachat'"` + +### Stale lock file after crash + +```bash +rm -f dc-account/accounts.lock +systemctl --user restart nanoclaw +``` + +### Bot not responding after restart + +The account is already configured — IO restarts automatically on service start. If the RPC subprocess is stuck, restart the service. Check for errors: + +```bash +grep "DeltaChat" logs/nanoclaw.error.log | tail -20 +``` + +### Messages received but agent not responding + +The messaging group exists but may not be wired to an agent group. Run: + +```bash +sqlite3 data/v2.db "SELECT id, platform_id, name FROM messaging_groups WHERE channel_type='deltachat'" +``` + +If the group has no entry in `messaging_group_agents`, wire it with `/manage-channels`. diff --git a/.claude/skills/add-deltachat/VERIFY.md b/.claude/skills/add-deltachat/VERIFY.md new file mode 100644 index 0000000..839fa85 --- /dev/null +++ b/.claude/skills/add-deltachat/VERIFY.md @@ -0,0 +1,54 @@ +# Verify DeltaChat + +## 1. Check the adapter started + +```bash +grep "Channel adapter started.*deltachat" logs/nanoclaw.log | tail -1 +``` + +Expected: `Channel adapter started { channel: 'deltachat', type: 'deltachat' }` + +## 2. Check IMAP/SMTP connectivity + +Replace with your provider's hostnames from `.env`: + +```bash +DC_IMAP=$(grep '^DC_IMAP_HOST=' .env | cut -d= -f2) +DC_SMTP=$(grep '^DC_SMTP_HOST=' .env | cut -d= -f2) + +bash -c "echo >/dev/tcp/$DC_IMAP/993" && echo "IMAP open" || echo "IMAP blocked" +bash -c "echo >/dev/tcp/$DC_SMTP/587" && echo "SMTP open" || echo "SMTP blocked" +``` + +## 3. End-to-end message test + +1. Open DeltaChat on your device +2. Add the bot email address as a contact +3. Send a message +4. The bot should respond within a few seconds + +If nothing arrives, check: + +```bash +grep "DeltaChat" logs/nanoclaw.log | tail -20 +grep "DeltaChat" logs/nanoclaw.error.log | tail -10 +``` + +## 4. Check messaging group was created + +```bash +sqlite3 data/v2.db \ + "SELECT id, platform_id, name FROM messaging_groups WHERE channel_type='deltachat' ORDER BY created_at DESC LIMIT 5" +``` + +If a row appears, the inbound routing is working. If not, the adapter isn't receiving the message — check logs for `DeltaChat: error handling incoming message`. + +## 5. Verify user access + +If the message arrived but the agent didn't respond, the sender may not have access: + +```bash +sqlite3 data/v2.db "SELECT id, display_name FROM users WHERE id LIKE 'deltachat:%'" +``` + +Grant access as shown in the SKILL.md "Grant user access" section.