diff --git a/.claude/skills/migrate-from-v1/SKILL.md b/.claude/skills/migrate-from-v1/SKILL.md new file mode 100644 index 0000000..e36a005 --- /dev/null +++ b/.claude/skills/migrate-from-v1/SKILL.md @@ -0,0 +1,232 @@ +--- +name: migrate-from-v1 +description: Finish migrating a NanoClaw v1 install into v2. Run after `bash migrate-v2.sh` completes. Seeds the owner, cleans up CLAUDE.local.md files, reconciles container configs, and helps port custom v1 code. Triggers on "migrate from v1", "finish migration", "v1 migration". +--- + +# Finish v1 → v2 migration + +`bash migrate-v2.sh` already ran the deterministic migration. It handled: + +- .env keys merged +- v2 DB seeded (agent_groups, messaging_groups, wiring) +- Group folders copied (v1 CLAUDE.md → v2 CLAUDE.local.md) +- Session data copied with conversation continuity (incl. Claude Code memory + JSONL transcripts) +- Scheduled tasks ported +- Channel code installed and auth state copied (incl. WhatsApp Baileys keystore) +- WhatsApp LIDs resolved from `store/auth` and aliased into `messaging_groups` +- Container skills copied +- Container image built + +Your job is the parts that need human judgment: triage any failed steps, seed the owner, clean up CLAUDE.local.md files, reconcile configs, and port any fork customizations. + +Read `logs/setup-migration/handoff.json` first — it has `overall_status`, per-step results in `steps`, and a `followups` list. + +## Preflight: was the script run? + +Before anything else, check that `logs/setup-migration/handoff.json` exists. If it doesn't, the user is invoking this skill before `migrate-v2.sh` ran. Stop and tell them, verbatim: + +> This skill finishes a migration that `migrate-v2.sh` started. Run that first, in your terminal — not from inside Claude: +> +> ```bash +> bash migrate-v2.sh +> ``` +> +> It needs interactive prompts (channel selection, service switchover) and runs Node/pnpm bootstrap, Docker, OneCLI setup, and a container build that don't fit inside a Claude session. When it finishes, it'll hand control back to Claude automatically — at which point this skill picks up. + +Do not attempt to run the script yourself, simulate its effects, or pick up the migration mid-stream. The deterministic side has dependencies on a real interactive shell. + +Once `handoff.json` exists, proceed to Phase 0. + +## Phase 0: Get v2 routing real messages + +Before any deeper migration work, prove v2 actually answers messages on the user's real channels. v1 is paused, not touched — flipping back is a service restart. + +### 0a — Fix blockers only + +Walk `handoff.steps`. Fix only the failures that would stop the bot from routing one message; defer the rest to its later phase. + +### 0b — Smoke test, then continue + +Tell the user the switch is non-destructive (v1 is paused, not modified; reverting is one command). Help them stop v1's service unit and start v2's, tail the host log for a clean boot, and have them send a real test message. Use `AskUserQuestion` to confirm the bot responded. + +If yes, continue to Phase 1. If no, diagnose from `logs/nanoclaw.log` and re-test — don't proceed to deeper work on a broken router. + +### Deferred failures + +Re-visit anything you skipped in 0a before declaring the migration done. Most surface naturally in later phases (`1c-groups` ↔ Phase 2, `1e-tasks` ↔ task verification). + +## Phase 1: Owner and access + +v2 auto-creates a `users` row for every sender it sees (via `extractAndUpsertUser` in `src/modules/permissions/index.ts`). By the time this skill runs, the owner's row likely already exists — it just needs the `owner` role granted. + +**User ID format**: always `:`. Each channel populates this differently: +- **Telegram**: `telegram:` (e.g. `telegram:6037840640`) +- **Discord**: `discord:` (e.g. `discord:123456789012345678`) +- **WhatsApp**: `whatsapp:@s.whatsapp.net` (e.g. `whatsapp:14155551234@s.whatsapp.net`) +- **Slack**: `slack:` (e.g. `slack:U04ABCDEF`) +- **Others**: `:` + +**Steps:** + +1. Query `users` table: `SELECT id, kind, display_name FROM users`. +2. If exactly one user exists, confirm: `AskUserQuestion`: "Is `` (``) you?" — Yes / No, let me type it. +3. If multiple users exist, present them as options in `AskUserQuestion`. +4. If no users exist yet (service hasn't received a message), ask the user to send a test message first, then re-query. +5. Once confirmed, check `user_roles` — if the owner role already exists, skip. Otherwise insert: + ```sql + INSERT INTO user_roles (user_id, role, agent_group_id, granted_by, granted_at) + VALUES ('', 'owner', NULL, NULL, datetime('now')) + ``` + +Use the DB helpers in `src/db/user-roles.ts` — they keep indexes correct. Init the DB first: + +```ts +import { initDb } from '../src/db/connection.js'; +import { runMigrations } from '../src/db/migrations/index.js'; +import { DATA_DIR } from '../src/config.js'; +import path from 'path'; +const db = initDb(path.join(DATA_DIR, 'v2.db')); +runMigrations(db); +``` + +### Access policy + +After seeding the owner, discuss the access policy. v2's `messaging_groups.unknown_sender_policy` controls who can interact with the bot. `migrate-v2.sh` set it to `public` so the bot would respond during the switchover test, but the user may want to tighten it. + +Present the options via `AskUserQuestion`: + +1. **Public** (current) — anyone can message the bot. Good for personal DM bots. +2. **Known users only** — only users in `agent_group_members` can trigger the bot. Others are silently dropped. +3. **Approval required** — unknown senders trigger an approval request to the owner. Good for group chats where you want to vet new members. + +If the user picks option 2 or 3, seed the known users from v1's message history. The v1 database is at `/store/messages.db`. It has a `messages` table with `sender` and `sender_name` columns. For each group: + +```sql +-- v1: unique senders per chat (excluding bot messages) +SELECT DISTINCT sender, sender_name +FROM messages +WHERE chat_jid = '' AND is_from_me = 0 AND sender IS NOT NULL +``` + +The `sender` value is a platform handle (e.g. `6037840640` for Telegram). Build the v2 user ID by inferring the channel type from the chat JID prefix (use `parseJid` from `setup/migrate-v2/shared.ts`) and combining: `:`. + +For each sender: +1. Upsert into `users(id, kind, display_name)` if not already present. +2. Insert into `agent_group_members(user_id, agent_group_id)` for each agent group wired to that messaging group. + +Show the user the list of senders being imported and let them deselect any they don't want. + +Then update the messaging groups: +```sql +UPDATE messaging_groups SET unknown_sender_policy = '' +WHERE id IN (SELECT id FROM messaging_groups WHERE channel_type IN ()) +``` + +## Phase 2: Clean up CLAUDE.local.md + +The migration copied v1's entire CLAUDE.md into CLAUDE.local.md for each group. This file now contains v1 boilerplate that v2 handles through its own composed fragments (`container/CLAUDE.md` + `.claude-fragments/module-*.md`). The user's customizations are buried inside. + +For each group that has a `CLAUDE.local.md`: + +1. Read the file. +2. Read the v1 template it was based on. Determine which template by checking the v1 install: + - If the group had `is_main=1` in v1's `registered_groups`, the template was `groups/main/CLAUDE.md` + - Otherwise, the template was `groups/global/CLAUDE.md` + - The v1 path is in `handoff.json` → `v1_path` +3. Diff the file against the template. Identify sections that are: + - **Stock boilerplate** (identical to template) — remove. v2's fragments cover this. + - **User customizations** (added sections, modified sections) — keep. +4. The following v1 sections are now handled by v2 fragments and should be removed even if slightly modified: + - "What You Can Do" → v2 runtime system prompt + - "Communication" / "Internal thoughts" / "Sub-agents" → `container/CLAUDE.md` + `module-core.md` + - "Your Workspace" / workspace path references → `container/CLAUDE.md` + - "Memory" (the stock version) → `container/CLAUDE.md` + - "Message Formatting" → `container/CLAUDE.md` + - "Admin Context" → v2 uses `user_roles`, not is_main + - "Authentication" → v2 uses OneCLI + - "Container Mounts" → v2 mounts are different + - "Managing Groups" / "Finding Available Groups" / "Registered Groups Config" → v2 entity model, no IPC + - "Global Memory" → v2 has `.claude-shared.md` symlink + - "Scheduling for Other Groups" → `module-scheduling.md` + - "Task Scripts" → `module-scheduling.md` + - "Sender Allowlist" → v2 uses `unknown_sender_policy` + `user_roles` +5. Fix path references in kept sections: + - `/workspace/group/` → `/workspace/agent/` + - `/workspace/project/` → these paths don't exist in v2; discuss with the user + - `/workspace/ipc/` → gone; remove references + - `/workspace/extra/` → v2 uses `container.json` `additionalMounts`; keep but note the path may change +6. Keep the `# Name` heading and first paragraph (identity) — this is the user's agent personality. +7. Show the user the proposed new CLAUDE.local.md before writing it. Use `AskUserQuestion`: "Here's what I'd keep — look right?" with options to approve, edit, or keep the original. + +If a CLAUDE.local.md has no user customizations (pure template copy), write a minimal file with just the identity heading. + +## Phase 3: Container config + +`migrate-v2.sh` writes `container.json` directly from v1's `container_config` (the `additionalMounts` shape is identical). If the v1 config was unparseable, it falls back to a `.v1-container-config.json` sidecar. + +For each group, check: + +1. If `container.json` exists, read it and verify the `additionalMounts` host paths are still valid on this machine. Flag any that don't exist. +2. If `.v1-container-config.json` exists (parse failure fallback), read it, discuss with the user, and write a proper `container.json`. Then delete the sidecar. +3. Check for `env` or `packages` fields — `env` may overlap with OneCLI vault, `packages` (apt/npm) are portable. + +## Phase 4: Fork customizations + +Check whether the user's v1 install was a customized fork. + +```bash +cd +git remote -v +git log --oneline /main..HEAD 2>/dev/null +``` + +If no commits ahead of upstream: stock v1, skip this phase. + +If there are commits: + +1. Show the commit list to the user. +2. `AskUserQuestion`: "How do you want to handle your v1 customizations?" + - **Copy portable items** (recommended) — copy `container/skills/*`, `.claude/skills/*`, `docs/*`. Scan each with `scanForV1Patterns` from `setup/migrate-v2/shared.ts`. + - **Full walkthrough** — go commit by commit, decide together. + - **Reference only** — stash to `docs/v1-fork-reference/` for later. +3. Source code (`src/*`, `container/agent-runner/src/*`) is NOT portable — v2's architecture is fundamentally different. Stash to `docs/v1-fork-reference/` with a README explaining what each file did. Don't translate. + +## Principles + +- **v1 checkout is read-only.** Never modify files under `handoff.v1_path`. +- **Show before writing.** Show diffs/proposed content before modifying CLAUDE.local.md or container.json. +- **Mask credentials** when displaying (first 4 + `...` + last 4 characters). +- **`handoff.json` is the recovery point.** If context gets compacted, re-read it and `git status` to recover state. + +## Setup steps you can run + +The setup flow at `setup/index.ts` has individual steps you can invoke if something is missing or failed: + +```bash +pnpm exec tsx setup/index.ts --step +``` + +| Step | When to use | +|------|-------------| +| `onecli` | OneCLI not installed or not healthy | +| `auth` | No Anthropic credential in vault | +| `container` | Container image needs rebuild | +| `service` | Service not installed or not running | +| `mounts` | Mount allowlist missing | +| `verify` | End-to-end health check (run after everything else) | +| `environment` | System check (Node, dirs) | + +## When done + +1. Run the verify step to confirm everything works: + ```bash + pnpm exec tsx setup/index.ts --step verify + ``` +2. Delete `logs/setup-migration/handoff.json` — offer to save as `docs/migration-.md` first. +3. Restart the service if running so changes take effect: + ```bash + # Linux + systemctl --user restart nanoclaw-v2-* + # macOS + launchctl kickstart -k gui/$(id -u)/com.nanoclaw-v2-* + ``` diff --git a/CHANGELOG.md b/CHANGELOG.md index ab2fd5a..2ec9fc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,11 @@ All notable changes to NanoClaw will be documented in this file. For detailed release notes, see the [full changelog on the documentation site](https://docs.nanoclaw.dev/changelog). +## [Unreleased] + +- **v1 → v2 migration.** Run `bash migrate-v2.sh` from the v2 checkout. Finds your v1 install (sibling directory or `NANOCLAW_V1_PATH`), merges `.env`, seeds the v2 DB from `registered_groups`, copies group folders (`CLAUDE.md` → `CLAUDE.local.md`), copies session data with conversation continuity, ports scheduled tasks, interactively selects and installs channels (clack multiselect), copies container skills, builds the agent container, and offers a service switchover to test. Hands off to Claude (`/migrate-from-v1`) for owner seeding, access policy, CLAUDE.md cleanup, and fork customization porting. See [docs/migration-dev.md](docs/migration-dev.md) and [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md). +- **Migration fixes.** `1b-db` now resolves Discord DMs as `discord:@me:` (previously skipped any v1 chat that wasn't a guild channel — a blocker for personal-bot installs). `1c-groups` skips symlinks instead of following them (a single broken `.claude-shared.md → /app/CLAUDE.md` no longer aborts the whole copy). When `1b-db` reuses an auto-created `messaging_group` with no wired agents, its `unknown_sender_policy` is now reconciled to the migration's `public` default. + ## [2.0.0] - 2026-04-22 Major version. NanoClaw v2 is a substantial architectural rewrite. Existing forks should run `/migrate-nanoclaw` (clean-base replay of customizations) or `/update-nanoclaw` (selective cherry-pick) before resuming work. diff --git a/CLAUDE.md b/CLAUDE.md index 1376a9e..e65515a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,6 +77,7 @@ Exactly one writer per file — no cross-mount lock contention. Heartbeat is a f | `container/skills/` | Container skills mounted into every agent session | | `groups//` | Per-agent-group filesystem (CLAUDE.md, skills, per-group `agent-runner-src/` overlay) | | `scripts/init-first-agent.ts` | Bootstrap the first DM-wired agent (used by `/init-first-agent` skill) | +| `migrate-v2.sh` + `setup/migrate-v2/` | v1→v2 migration. Standalone script: `bash migrate-v2.sh`. Seeds DB, copies groups/sessions, installs channels, builds container, offers service switchover, then hands off to `/migrate-from-v1` skill for owner setup and CLAUDE.md cleanup. See [docs/migration-dev.md](docs/migration-dev.md). | ## Channels and Providers (skill-installed) @@ -232,6 +233,8 @@ This project uses pnpm with `minimumReleaseAge: 4320` (3 days) in `pnpm-workspac | [docs/setup-wiring.md](docs/setup-wiring.md) | What's wired, what's open in the setup flow | | [docs/architecture-diagram.md](docs/architecture-diagram.md) | Diagram version of the architecture | | [docs/build-and-runtime.md](docs/build-and-runtime.md) | Runtime split (Node host + Bun container), lockfiles, image build surface, CI, key invariants | +| [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) | v1→v2 architecture diff — vocabulary for where v1 things moved | +| [docs/migration-dev.md](docs/migration-dev.md) | Migration development guide — testing, debugging, dev loop | ## Container Build Cache diff --git a/README.md b/README.md index de61f6d..69f9ea2 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,29 @@ bash nanoclaw.sh `nanoclaw.sh` walks you from a fresh machine to a named agent you can message. It installs Node, pnpm, and Docker if missing, registers your Anthropic credential with OneCLI, builds the agent container, and pairs your first channel (Telegram, Discord, WhatsApp, or a local CLI). If a step fails, Claude Code is invoked automatically to diagnose and resume from where it broke. +
+Migrating from NanoClaw v1? + +Run from a fresh v2 checkout next to your v1 install: + +```bash +git clone https://github.com/qwibitai/nanoclaw.git nanoclaw-v2 +cd nanoclaw-v2 +bash migrate-v2.sh +``` + +`migrate-v2.sh` finds your v1 install (sibling directory, or `NANOCLAW_V1_PATH=/path/to/nanoclaw`), migrates state into the v2 checkout, then `exec`s into Claude Code to finish the parts that need judgment (owner seeding, CLAUDE.local.md cleanup, fork-customisation replay). + +Run the script directly, not from inside a Claude session — the deterministic side needs interactive prompts and real shell I/O for Node/pnpm bootstrap, Docker, OneCLI, and the container build. + +**What it does:** merges `.env`, seeds the v2 DB from `registered_groups`, copies group folders + session data + scheduled tasks, installs the channel adapters you select, copies channel auth state (including Baileys keystore + LID mappings for WhatsApp), builds the agent container. + +**What it doesn't:** flip the system service. Pick *"switch to v2"* at the prompt, or do it manually after testing — your v1 install is left untouched. + +See [docs/v1-to-v2-changes.md](docs/v1-to-v2-changes.md) for what's different and [docs/migration-dev.md](docs/migration-dev.md) for development notes. + +
+ ## Philosophy **Small enough to understand.** One process, a few source files and no microservices. If you want to understand the full NanoClaw codebase, just ask Claude Code to walk you through it. diff --git a/docs/migration-dev.md b/docs/migration-dev.md new file mode 100644 index 0000000..35ef536 --- /dev/null +++ b/docs/migration-dev.md @@ -0,0 +1,139 @@ +# v1 → v2 Migration — Development Guide + +How to test, develop, and debug the migration flow. + +## Quick start + +```bash +# Full cycle: reset → migrate → Claude finishes +bash migrate-v2-reset.sh && bash migrate-v2.sh +``` + +## Architecture + +Two-part migration: + +1. **`migrate-v2.sh`** — deterministic bash script. Handles prerequisites, DB seeding, file copies, channel install, container build, service switchover. Writes `logs/setup-migration/handoff.json` then `exec`s into Claude. + +2. **`/migrate-from-v1` skill** — Claude-driven. Reads the handoff, seeds owner/roles, cleans up CLAUDE.local.md, validates container configs, ports fork customizations. + +## File layout + +``` +migrate-v2.sh # Entry point +migrate-v2-reset.sh # Wipe v2 state for re-testing +setup/migrate-v2/ + env.ts # Phase 1a: merge .env + db.ts # Phase 1b: seed v2 DB + groups.ts # Phase 1c: copy group folders + container.json + sessions.ts # Phase 1d: copy sessions + set continuation + tasks.ts # Phase 1e: port scheduled tasks + channel-auth.ts # Phase 2b: copy channel auth state + select-channels.ts # Phase 2a: clack multiselect + switchover-prompt.ts # Service switch prompts +setup/migrate-v2/shared.ts # Shared helpers (JID parsing, trigger mapping, etc.) +.claude/skills/migrate-from-v1/ # The Claude skill +logs/setup-migration/handoff.json # Written by migrate-v2.sh, read by skill +logs/migrate-steps/*.log # Per-step raw output +``` + +## Development loop + +```bash +# Reset v2 to clean state (keeps node_modules) +bash migrate-v2-reset.sh + +# Run migration with non-interactive channel selection +NANOCLAW_CHANNELS="telegram" bash migrate-v2.sh + +# Or run interactively (clack multiselect) +bash migrate-v2.sh +``` + +`migrate-v2-reset.sh` wipes: `data/`, `logs/`, `.env`, `groups/` (restores git-tracked), `container/skills/` (restores git-tracked), `src/channels/` (restores git-tracked). + +It does NOT wipe `node_modules/` (expensive to reinstall). + +## Testing individual steps + +Each step is a standalone TypeScript file: + +```bash +# Run a single step (after pnpm install) +pnpm exec tsx setup/migrate-v2/env.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/db.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/groups.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/sessions.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/tasks.ts /path/to/v1 +pnpm exec tsx setup/migrate-v2/channel-auth.ts /path/to/v1 telegram discord +``` + +Each prints `OK:
`, `SKIPPED:`, or errors to stdout. Exit 0 on success/skip, non-zero on failure. + +## Debugging + +### Check what was migrated + +```bash +# Agent groups +sqlite3 data/v2.db "SELECT * FROM agent_groups" + +# Messaging groups + wiring +sqlite3 data/v2.db "SELECT mg.id, mg.channel_type, mg.platform_id, mg.unknown_sender_policy, mga.engage_mode, mga.engage_pattern FROM messaging_groups mg JOIN messaging_group_agents mga ON mga.messaging_group_id = mg.id" + +# Sessions +sqlite3 data/v2.db "SELECT * FROM sessions" + +# Users and roles +sqlite3 data/v2.db "SELECT * FROM users" +sqlite3 data/v2.db "SELECT * FROM user_roles" + +# Session continuation (which Claude Code session will be resumed) +AG_ID=$(sqlite3 data/v2.db "SELECT id FROM agent_groups LIMIT 1") +SESS_ID=$(sqlite3 data/v2.db "SELECT id FROM sessions LIMIT 1") +sqlite3 data/v2-sessions/$AG_ID/$SESS_ID/outbound.db "SELECT * FROM session_state" + +# Scheduled tasks +sqlite3 data/v2-sessions/$AG_ID/$SESS_ID/inbound.db "SELECT id, kind, recurrence, status FROM messages_in WHERE kind='task'" +``` + +### Check handoff + +```bash +python3 -m json.tool logs/setup-migration/handoff.json +``` + +### Common issues + +**Bot doesn't respond after switchover:** +1. Check both services aren't running: `systemctl --user list-units 'nanoclaw*'` +2. Check error log: `tail logs/nanoclaw.error.log` +3. Check sender policy: `sqlite3 data/v2.db "SELECT unknown_sender_policy FROM messaging_groups"` — must be `public` before owner is seeded +4. Check engage pattern: `sqlite3 data/v2.db "SELECT engage_mode, engage_pattern FROM messaging_group_agents"` — should be `pattern` / `.` for respond-to-everything + +**Session not continuing from v1:** +1. Check continuation is set: see "Session continuation" query above +2. Check JSONL exists at the right path: `ls data/v2-sessions//.claude-shared/projects/-workspace-agent/` +3. The v1 session JSONL should be copied from `-workspace-group/` to `-workspace-agent/` (v2 container CWD is `/workspace/agent`) + +**Service switchover revert didn't work:** +1. The v2 service name is `nanoclaw-v2-` — find it: `systemctl --user list-units 'nanoclaw*'` +2. Manually stop: `systemctl --user stop && systemctl --user disable ` +3. Restart v1: `systemctl --user start nanoclaw` + +### Step logs + +Each step writes raw output to `logs/migrate-steps/.log`. Read these when a step fails: + +```bash +cat logs/migrate-steps/1b-db.log +cat logs/migrate-steps/1d-sessions.log +``` + +## Key decisions + +- `unknown_sender_policy` is set to `public` during migration so the bot responds immediately. The `/migrate-from-v1` skill tightens it after seeding the owner. +- `requires_trigger=0` in v1 takes priority over a non-empty `trigger_pattern` — it means "respond to everything." +- v1 `container_config.additionalMounts` is written directly to v2 `container.json` (same shape). +- v1 Claude Code sessions are copied from `-workspace-group/` to `-workspace-agent/` and the session ID is written to `outbound.db` as `continuation:claude` so the agent-runner resumes the same conversation. +- `exec claude "/migrate-from-v1"` at the end replaces the bash process — `write_handoff` is called explicitly before `exec` since EXIT traps don't fire on `exec`. diff --git a/docs/v1-to-v2-changes.md b/docs/v1-to-v2-changes.md new file mode 100644 index 0000000..082bf30 --- /dev/null +++ b/docs/v1-to-v2-changes.md @@ -0,0 +1,172 @@ +# NanoClaw v1 → v2 — what changed + +Big-picture differences between NanoClaw v1 (the `~/nanoclaw` checkout you've been running) and v2 (this rewrite). Not a migration guide — that's what `bash migrate-v2.sh` and the `/migrate-from-v1` skill are for. This doc is the **vocabulary**: when something has moved or been renamed, find it here. + +Read this before touching the migration code or porting customizations forward. + +--- + +## One-line summary + +v1 was one Node process with one SQLite file and native channel adapters. v2 is a host that spawns per-session Docker containers, splits state across a central DB + per-session DB pair, routes through an explicit entity model, and installs channels as skills from a sibling branch. + +--- + +## Entity model — the biggest shift + +**v1:** one flat table `registered_groups(jid, name, folder, trigger_pattern, requires_trigger, is_main, channel_name)`. A group folder is the unit of agent identity. A chat (JID) is wired to exactly one folder, and `trigger_pattern` is an opaque regex the router applies to every incoming message. + +**v2:** three tables, with a deliberate many-to-many in the middle: + +``` +agent_groups ─┐ + ├─ messaging_group_agents ─┬─ messaging_groups + │ (engage_mode, │ (channel_type, + │ engage_pattern, │ platform_id, + │ sender_scope, │ unknown_sender_policy) + │ ignored_message_policy, + │ session_mode, priority) +``` + +Consequences: + +- **One agent can answer on many chats, and one chat can fan out to many agents.** v1 couldn't do either. +- **No `is_main` flag.** Privilege is now explicit via `user_roles` (owner/admin, global or scoped). See below. +- **No `trigger_pattern` regex.** Replaced with four orthogonal columns. Mapping rule used by the automated migration and by the `/migrate-from-v1` skill: + - v1 `trigger_pattern` non-empty → v2 `engage_mode='pattern'`, `engage_pattern = ` + - v1 `requires_trigger=0` or pattern was `.`/`.*` → v2 `engage_mode='pattern'`, `engage_pattern='.'` (the "always" flavor) + - no pattern and requires a trigger → v2 `engage_mode='mention'` + - `sender_scope` and `ignored_message_policy` are new; defaults `all` / `drop` +- **JID decomposition.** v1's `jid` column stored `dc:12345` / `tg:67890`. v2 splits this into `channel_type` + `platform_id`. Concretely: `dc:12345` becomes `channel_type='discord'`, `platform_id='discord:12345'`. Prefix aliases (`dc` → `discord`, `tg` → `telegram`, `wa` → `whatsapp`) are in `setup/migrate-v2/shared.ts`. +- **`channel_name` was unreliable in v1.** Many rows had it empty; the actual channel had to be guessed from the JID prefix. v2's `channel_type` is always explicit. + +--- + +## Central DB vs session DBs + +**v1:** one SQLite file at `store/messages.db`. Every chat, message, registered group, scheduled task, and session lived there. Host and any agent processes all opened the same file. + +**v2:** three DB shapes. + +1. `data/v2.db` — **central**. Everything that isn't per-session: users, roles, agent groups, messaging groups, wirings, pending approvals, user DMs, schema migrations. +2. `data/v2-sessions//inbound.db` — **host writes, container reads**. `messages_in`, routing, destinations, pending questions, processing_ack. This is where scheduled tasks live (see "Scheduling" below). +3. `data/v2-sessions//outbound.db` — **container writes, host reads**. `messages_out`, session_state. + +Exactly one writer per file. No cross-mount lock contention. Heartbeat is a file touch at `/workspace/.heartbeat`, not a DB update. Host uses even `seq` numbers, container uses odd. + +Message history (v1 `messages` table, v1 `chats` table) is **not migrated**. The migration copies operationally important state forward (agents, channels, wirings, scheduled tasks, group folders) and leaves chat logs behind. + +--- + +## Scheduling + +**v1:** dedicated `scheduled_tasks` table in `store/messages.db` with its own columns (`schedule_type`, `schedule_value`, `next_run`, `last_run`, `context_mode`, `script`, `status`). A separate cron-ish scheduler process read from it. + +**v2:** scheduled tasks are **`messages_in` rows with `kind='task'`** in a session's `inbound.db`. Relevant columns: +- `process_after` (ISO8601) — host sweep wakes the container when `datetime(process_after) <= datetime('now')` +- `recurrence` — cron string; `NULL` = one-shot +- `series_id` — groups recurring occurrences; set to the task id on first insert +- `status` — `pending` | `processing` | `completed` | `failed` | `paused` + +The public API is `insertTask()` in `src/modules/scheduling/db.ts`. Recurrence is computed in the user's TZ via `cron-parser` (see `src/modules/scheduling/recurrence.ts`). The migration maps v1's `schedule_type`+`schedule_value` pair into a single cron string before calling `insertTask()`. + +Tasks can exist before a session is awake — the host sweep creates/wakes the container on the first due tick. + +--- + +## Credentials + +**v1:** `.env` — plain environment variables. `DISCORD_BOT_TOKEN`, `ANTHROPIC_API_KEY`, etc. The host read them directly and passed them in to any code that needed them. + +**v2:** OneCLI Agent Vault. A separate local service at `http://127.0.0.1:10254` holds secrets. Agents are *scoped* to specific secrets and the vault injects them into approved API requests as they leave the container. The container never sees the raw secret value. + +Gotcha: auto-created agents default to `selective` secret mode — no secrets attached, even if matching secrets exist in the vault. See the "auto-created agents start in selective secret mode" section of the root CLAUDE.md for the fix (`onecli agents set-secret-mode --mode all`). + +**What the automated migration does:** copies every v1 `.env` key verbatim into v2 `.env`, never overwriting existing v2 keys. The OneCLI vault migration is a separate step owned by the `/init-onecli` skill, which knows how to pull from `.env`. + +--- + +## Channel adapters + +**v1:** native adapters (e.g. `discord.js` used directly) imported in `src/channels/`. Installing a channel meant editing code, adding a dependency, and setting env vars. + +**v2:** channel adapters live on a sibling `channels` branch. Each `/add-` skill: +1. `git fetch origin channels` +2. `git show channels:src/channels/.ts > src/channels/.ts` +3. Appends `import './.js';` to `src/channels/index.ts` +4. `pnpm install @chat-adapter/@` +5. `pnpm run build` + +Idempotent — re-running is a no-op. Pinned versions keep the supply chain honest. The automated migration detects which channels were wired in v1 (via distinct `channel_name` / JID prefix) and runs the matching `setup/install-.sh` for each. Channels in v1 that don't have a v2 skill (rare now, more common as v2 catches up) are recorded in the handoff file for the `/migrate-from-v1` skill to raise with the user. + +**Channel auth beyond `.env`.** Some channels store session state on disk (Baileys WhatsApp keystore, Matrix sync state, iMessage tokens). The `channel-auth` step has a per-channel registry (`setup/migrate-v2/shared.ts: CHANNEL_AUTH_REGISTRY`) that knows which file globs to copy alongside env keys. + +--- + +## Privilege — from implicit to explicit + +**v1:** `registered_groups.is_main = 1` flagged one group as the privileged one. No `users` table. Permissions were conventions, not enforced. + +**v2:** explicit tables. +- `users(id = ":", kind, display_name)` — one row per messaging-platform identifier +- `user_roles(user_id, role ∈ {owner, admin}, agent_group_id nullable, granted_by, granted_at)` — owner is always global; admin can be global or scoped +- `agent_group_members(user_id, agent_group_id, ...)` — "known" membership for the `sender_scope='known'` gate + +Owner gets seeded during the `/migrate-from-v1` skill's interview phase ("Which handle is you?"). The automated migration doesn't guess — v1 has no source of truth for it. + +**Default access — "anyone can talk to the bot" vs "only known users".** v1 stored this implicitly (via trigger regex + `is_main`). v2 exposes it as `messaging_groups.unknown_sender_policy ∈ {'strict', 'request_approval', 'public'}`. The skill asks the user which mode v1 ran in and flips the migrated messaging groups accordingly. + +--- + +## Group folders on disk + +**v1:** `groups//CLAUDE.md` and optional `logs/`. `CLAUDE.md` was a plain instruction file, group-specific. + +**v2:** each group still lives at `groups//`, but the shape is richer: +- `CLAUDE.md` — **composed at container spawn** from `.claude-shared.md` (symlink to global) + `.claude-fragments/*.md` (module fragments) + `CLAUDE.local.md`. **Don't edit `CLAUDE.md` directly.** +- `CLAUDE.local.md` — per-group content. The migration writes v1's old `CLAUDE.md` here. +- `container.json` — optional per-group container config (apt deps, env, mounts). v1's `registered_groups.container_config` JSON is close but not identical — the migration stores the v1 payload at `groups//.v1-container-config.json` for the skill to reconcile, rather than silently mapping it. +- `.claude-fragments/` and `.claude-shared.md` are installed by `initGroupFilesystem()` the first time the host touches the group, so the migration only has to write `CLAUDE.local.md` and leave the scaffolding to the host. + +--- + +## Host process vs containers + +**v1:** single Node process. The "agent" was the same process as the router. + +**v2:** Node host at top, Bun-runtime Docker container per session. They communicate only via the two session DBs. No shared modules, no IPC, no stdin piping. If you wrote custom code that reached from the agent into host internals (or vice versa), that surface no longer exists — porting it is a `/migrate-from-v1` skill topic, not a mechanical copy. + +Lockfiles: host uses `pnpm-lock.yaml`, agent-runner uses `bun.lock`. `minimumReleaseAge: 4320` on the host side (3-day supply-chain wait); agent-runner has no release-age gate. + +--- + +## Self-modification and MCP tools + +**v1:** if you added MCP servers or self-modification plumbing, it was usually direct edits to the long-running process. + +**v2:** +- MCP servers register through `container/agent-runner/src/mcp-tools/*.ts` and load per-session. There's also `install_packages` and `add_mcp_server` self-mod tools that go through an admin-approval flow (`src/modules/self-mod/apply.ts`) before rebuilding the container image. +- Custom MCP tools you wrote in v1 map cleanly to the v2 tool registry, but the import paths, runtime (Bun vs Node), and SQL helper differences (`bun:sqlite` uses `$name`-prefixed params) may need adjustment. The skill walks through this. + +--- + +## Things that are gone or don't map + +- **`scheduled_tasks` as a separate table** — moved into session `inbound.db` under `kind='task'`. Migration ports active rows; inactive/completed are exported to `logs/setup-migration/inactive-tasks.json` for reference. +- **`messages` / `chats` tables (chat history)** — not migrated. Stay in the v1 checkout if you need them. +- **`router_state` (key/value)** — not migrated. v2 state lives in the explicit tables above. +- **`sessions` (v1 group→session_id)** — v1 sessions don't map; v2 sessions are keyed by `(agent_group_id, messaging_group_id, thread_id)` and are created on demand. +- **Raw access to the old `store/messages.db`** — the v1 DB is left in place and untouched. If migration goes wrong you can re-run it (the migration sub-steps are idempotent for agents/channels/wirings; folders use rsync semantics). + +--- + +## Migration surface — where the code lives + +- `migrate-v2.sh` — entry point: `bash migrate-v2.sh` from the v2 checkout. +- `setup/migrate-v2/*.ts` — individual migration steps (env, db, groups, sessions, tasks, channel-auth, select-channels, switchover-prompt). +- `setup/migrate-v2/shared.ts` — JID parsing, trigger mapping, channel auth registry. +- `logs/setup-migration/handoff.json` — written by `migrate-v2.sh`, read by the `/migrate-from-v1` skill. +- `logs/migrate-steps/*.log` — raw per-step stdout. +- `.claude/skills/migrate-from-v1/SKILL.md` — Claude skill for owner seeding, CLAUDE.md cleanup, container config validation, fork porting. +- `migrate-v2-reset.sh` — development helper to wipe v2 state for re-testing. +- See [docs/migration-dev.md](migration-dev.md) for the full development guide. diff --git a/migrate-v2-reset.sh b/migrate-v2-reset.sh new file mode 100644 index 0000000..cd817d9 --- /dev/null +++ b/migrate-v2-reset.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# +# migrate-v2-reset.sh — Wipe v2 migration state back to clean. +# +# For development iteration: +# bash migrate-v2-reset.sh && bash migrate-v2.sh +# +# What it removes: +# - data/ (v2 DBs, session state) +# - logs/ (migration + setup logs) +# - .env (merged env keys) +# - groups/*/ (non-git group folders copied from v1) +# - container/skills/*/ (untracked skill dirs copied from v1) +# - src/channels/*.ts (untracked adapters copied from channels branch) +# - setup/groups.ts (untracked, copied by channel install scripts) +# +# What it restores from git: +# - groups/ (CLAUDE.md files etc.) +# - container/skills/ (tracked container skills) +# - src/channels/ (tracked bridge / registry code) +# - setup/whatsapp-auth.ts (channel installs may overwrite) +# - setup/pair-telegram.ts (channel installs may overwrite) +# - setup/index.ts (channel installs append entries) +# - package.json + pnpm-lock.yaml (channel installs add deps) +# +# What it does NOT touch: +# - node_modules/ (expensive to reinstall, kept on purpose) +# - setup/migrate-v2/* (the migration scripts themselves, plus user WIP) +# - The v1 install (read-only, never modified) + +set -euo pipefail + +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$PROJECT_ROOT" + +use_ansi() { [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; } +dim() { use_ansi && printf '\033[2m%s\033[0m' "$1" || printf '%s' "$1"; } +green() { use_ansi && printf '\033[32m%s\033[0m' "$1" || printf '%s' "$1"; } + +clean() { + local target=$1 label=$2 + if [ -e "$target" ]; then + rm -rf "$target" + printf '%s Removed %s\n' "$(green '✓')" "$label" + fi +} + +echo +printf '%s\n\n' "$(dim 'Resetting v2 migration state…')" + +clean "data" "data/" +clean "logs" "logs/" +clean ".env" ".env" + +# Remove all group folders, then restore the two git-tracked ones +if [ -d "groups" ]; then + rm -rf groups + printf '%s Removed %s\n' "$(green '✓')" "groups/" +fi +git checkout -- groups/ 2>/dev/null || true +printf '%s Restored %s\n' "$(green '✓')" "groups/ from git" + +# Restore container/skills/ to git state (remove v1-copied skills) +git checkout -- container/skills/ 2>/dev/null || true +# Remove any untracked skill dirs that were copied from v1 +for d in container/skills/*/; do + [ -d "$d" ] || continue + if ! git ls-files --error-unmatch "$d" >/dev/null 2>&1; then + rm -rf "$d" + fi +done +printf '%s Restored %s\n' "$(green '✓')" "container/skills/ from git" + +# Restore channel code (src/channels/) to git state +git checkout -- src/channels/ 2>/dev/null || true +# Remove any untracked channel adapters copied in by install-*.sh +for f in src/channels/*.ts; do + [ -f "$f" ] || continue + if ! git ls-files --error-unmatch "$f" >/dev/null 2>&1; then + rm -f "$f" + fi +done +printf '%s Restored %s\n' "$(green '✓')" "src/channels/ from git" + +# Restore tracked setup helpers that channel installs overwrite, and +# remove the untracked ones they create. Don't blanket-clean setup/ +# because user WIP (setup/migrate-v2/*) lives there too. +git checkout -- setup/whatsapp-auth.ts setup/pair-telegram.ts setup/index.ts 2>/dev/null || true +rm -f setup/groups.ts +printf '%s Restored %s\n' "$(green '✓')" "setup/ install helpers" + +# Restore package.json + lockfile (channel installs add deps like +# @whiskeysockets/baileys). node_modules/ is intentionally kept. +git checkout -- package.json pnpm-lock.yaml 2>/dev/null || true +printf '%s Restored %s\n' "$(green '✓')" "package.json + pnpm-lock.yaml" + +echo +printf '%s\n\n' "$(dim 'Clean. Run: bash migrate-v2.sh')" diff --git a/migrate-v2.sh b/migrate-v2.sh new file mode 100644 index 0000000..eb5a381 --- /dev/null +++ b/migrate-v2.sh @@ -0,0 +1,703 @@ +#!/usr/bin/env bash +# +# migrate-v2.sh — Migrate a NanoClaw v1 install into this v2 checkout. +# +# Run from the v2 directory: +# bash migrate-v2.sh +# +# Finds v1 automatically (sibling directory, or $NANOCLAW_V1_PATH). +# Installs prerequisites (Node, pnpm, deps) via the existing setup.sh +# bootstrap, then runs the migration steps. +# +# Idempotent — safe to re-run. Use migrate-v2-reset.sh to wipe v2 state +# back to clean for development iteration. + +set -uo pipefail + +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$PROJECT_ROOT" + +LOGS_DIR="$PROJECT_ROOT/logs" +STEPS_DIR="$LOGS_DIR/migrate-steps" +MIGRATE_LOG="$LOGS_DIR/migrate-v2.log" + +# Defaults for variables that may not be set if we exit early +V1_PATH="" +V1_VERSION="unknown" +ONECLI_OK=false +SERVICE_SWITCHED=false +SELECTED_CHANNELS=() +ABORTED_AT="" + +# Per-step status tracking. Parallel indexed arrays so this works on +# bash 3.2 (macOS default) which has no associative arrays. +STEP_NAMES=() +STEP_STATUSES=() + +record_step() { + STEP_NAMES+=("$1") + STEP_STATUSES+=("$2") +} + +# Write handoff.json on any exit so the skill can always read it +write_handoff() { + local handoff_dir="$LOGS_DIR/setup-migration" + mkdir -p "$handoff_dir" + + local has_failures=false + local i + for ((i=0; i<${#STEP_NAMES[@]}; i++)); do + [ "${STEP_STATUSES[$i]}" = "failed" ] && has_failures=true + done + + local overall="success" + $has_failures && overall="partial" + [ -n "$ABORTED_AT" ] && overall="failed" + + local steps_json="{" + for ((i=0; i<${#STEP_NAMES[@]}; i++)); do + local n="${STEP_NAMES[$i]}" + local s="${STEP_STATUSES[$i]}" + steps_json="${steps_json}\"${n}\": {\"status\": \"${s}\", \"log\": \"logs/migrate-steps/${n}.log\"}," + done + steps_json="${steps_json%,}}" + + cat > "$handoff_dir/handoff.json" </dev/null | sed 's/,$//')], + "onecli_healthy": $ONECLI_OK, + "service_switched": $SERVICE_SWITCHED, + "steps": $steps_json, + "step_logs_dir": "logs/migrate-steps", + "followups": [ + "Seed owner user and access policy", + "Review CLAUDE.local.md files for v1-specific patterns", + "Verify container.json mount paths are valid" + ] +} +HANDOFF_EOF +} + +trap write_handoff EXIT + +abort() { + ABORTED_AT="$1" + log "ABORTED at $1" + exit 1 +} + +# ─── output helpers ────────────────────────────────────────────────────── + +use_ansi() { [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; } +dim() { use_ansi && printf '\033[2m%s\033[0m' "$1" || printf '%s' "$1"; } +green() { use_ansi && printf '\033[32m%s\033[0m' "$1" || printf '%s' "$1"; } +red() { use_ansi && printf '\033[31m%s\033[0m' "$1" || printf '%s' "$1"; } +bold() { use_ansi && printf '\033[1m%s\033[0m' "$1" || printf '%s' "$1"; } +clear_line() { use_ansi && printf '\r\033[2K' || printf '\n'; } + +step_ok() { printf '%s %s\n' "$(green '✓')" "$1"; } +step_fail() { printf '%s %s\n' "$(red '✗')" "$1"; } +step_skip() { printf '%s %s\n' "$(dim '–')" "$1"; } +step_info() { printf '%s %s\n' "$(dim '·')" "$1"; } + +ts_utc() { date -u +%Y-%m-%dT%H:%M:%SZ; } + +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >> "$MIGRATE_LOG" +} + +# ─── init logs ─────────────────────────────────────────────────────────── + +mkdir -p "$STEPS_DIR" +{ + echo "## $(ts_utc) · migrate-v2.sh started" + echo " cwd: $PROJECT_ROOT" + echo "" +} > "$MIGRATE_LOG" + +echo +bold "NanoClaw v1 → v2 migration" +echo +echo + +# ─── phase 0a: bootstrap prerequisites ────────────────────────────────── + +step_info "Installing prerequisites (Node, pnpm, dependencies)…" + +BOOTSTRAP_RAW="$STEPS_DIR/01-bootstrap.log" +export NANOCLAW_BOOTSTRAP_LOG="$BOOTSTRAP_RAW" + +if bash "$PROJECT_ROOT/setup.sh" > "$BOOTSTRAP_RAW" 2>&1; then + # Parse the status block from setup.sh output + STATUS=$(grep '^STATUS:' "$BOOTSTRAP_RAW" | head -1 | sed 's/^STATUS: *//') + NODE_VERSION=$(grep '^NODE_VERSION:' "$BOOTSTRAP_RAW" | head -1 | sed 's/^NODE_VERSION: *//') + + if [ "$STATUS" = "success" ]; then + step_ok "Prerequisites ready $(dim "(node $NODE_VERSION)")" + log "Bootstrap succeeded: node=$NODE_VERSION" + else + step_fail "Bootstrap reported: $STATUS" + echo + dim " See: $BOOTSTRAP_RAW" + echo + abort "bootstrap" + fi +else + step_fail "Bootstrap failed" + echo + echo "$(dim '── last 20 lines ──')" + tail -20 "$BOOTSTRAP_RAW" 2>/dev/null || true + echo + dim " Full log: $BOOTSTRAP_RAW" + echo + abort "bootstrap" +fi + +# setup.sh may have installed pnpm to a prefix not on our PATH — replay +# the same lookup nanoclaw.sh does. +if ! command -v pnpm >/dev/null 2>&1 && command -v npm >/dev/null 2>&1; then + NPM_PREFIX="$(npm config get prefix 2>/dev/null)" + if [ -n "$NPM_PREFIX" ] && [ -x "$NPM_PREFIX/bin/pnpm" ]; then + export PATH="$NPM_PREFIX/bin:$PATH" + fi +fi + +if ! command -v pnpm >/dev/null 2>&1; then + step_fail "pnpm not found after bootstrap" + abort "pnpm-missing" +fi + +# ─── phase 0b: find v1 install ────────────────────────────────────────── + +find_v1() { + # Explicit override + if [ -n "${NANOCLAW_V1_PATH:-}" ]; then + if [ -f "$NANOCLAW_V1_PATH/store/messages.db" ]; then + echo "$NANOCLAW_V1_PATH" + return 0 + fi + step_fail "NANOCLAW_V1_PATH=$NANOCLAW_V1_PATH does not contain store/messages.db" + return 1 + fi + + # Scan sibling directories for anything claw-ish with a v1 DB + local parent + parent="$(dirname "$PROJECT_ROOT")" + for entry in "$parent"/*/; do + [ -d "$entry" ] || continue + # Skip ourselves + [ "$(cd "$entry" && pwd)" = "$PROJECT_ROOT" ] && continue + # Must have the v1 DB + [ -f "$entry/store/messages.db" ] || continue + # Must not be v2 (check package.json version) + if [ -f "$entry/package.json" ]; then + local ver + ver=$(grep '"version"' "$entry/package.json" 2>/dev/null | head -1 | sed -E 's/.*"([0-9]+)\..*/\1/') + [ "$ver" = "2" ] && continue + fi + echo "$(cd "$entry" && pwd)" + return 0 + done + + return 1 +} + +V1_PATH="" +if V1_PATH=$(find_v1); then + V1_VERSION=$(grep '"version"' "$V1_PATH/package.json" 2>/dev/null | head -1 | sed -E 's/.*"([^"]+)".*/\1/' || echo "unknown") + step_ok "Found v1 at $(dim "$V1_PATH") $(dim "(v$V1_VERSION)")" + log "v1 found: $V1_PATH (v$V1_VERSION)" +else + step_fail "No v1 install found" + echo + echo " $(dim 'Set NANOCLAW_V1_PATH to point at your v1 checkout:')" + echo " $(dim 'NANOCLAW_V1_PATH=~/nanoclaw bash migrate-v2.sh')" + echo + abort "v1-not-found" +fi + +# ─── phase 0c: validate v1 DB ─────────────────────────────────────────── + +V1_DB="$V1_PATH/store/messages.db" + +# Quick schema check — make sure the tables we need exist +TABLES=$(sqlite3 "$V1_DB" ".tables" 2>/dev/null || true) + +if echo "$TABLES" | grep -q "registered_groups"; then + step_ok "v1 database has registered_groups" +else + step_fail "v1 database missing registered_groups table" + abort "v1-db-invalid" +fi + +# Show what we found +GROUP_COUNT=$(sqlite3 "$V1_DB" "SELECT COUNT(*) FROM registered_groups" 2>/dev/null || echo 0) +TASK_COUNT=$(sqlite3 "$V1_DB" "SELECT COUNT(*) FROM scheduled_tasks WHERE status='active'" 2>/dev/null || echo 0) +ENV_KEYS=0 +if [ -f "$V1_PATH/.env" ]; then + ENV_KEYS=$(grep -c '=' "$V1_PATH/.env" 2>/dev/null || echo 0) +fi + +step_info "v1 state: $(bold "$GROUP_COUNT") groups, $(bold "$TASK_COUNT") active tasks, $(bold "$ENV_KEYS") env keys" + +echo +step_ok "Phase 0 complete — ready to migrate" +echo +log "Phase 0 complete: groups=$GROUP_COUNT tasks=$TASK_COUNT env_keys=$ENV_KEYS" + +export NANOCLAW_V1_PATH="$V1_PATH" +export NANOCLAW_V2_PATH="$PROJECT_ROOT" + +# ─── run_step helper ───────────────────────────────────────────────────── +# Runs a TypeScript migration step, captures output, reports success/failure. + +# Step outcomes are tracked via record_step() into STEP_NAMES/STEP_STATUSES +# (defined above, near write_handoff). + +run_step() { + local name=$1 label=$2 script=$3 + shift 3 + local raw="$STEPS_DIR/${name}.log" + + if pnpm exec tsx "$script" "$@" > "$raw" 2>&1; then + local result + result=$(grep '^OK:' "$raw" | head -1 || true) + step_ok "$label $(dim "$result")" + log "$name: $result" + record_step "$name" "success" + # Surface partial errors (rows skipped due to parse/lookup failures) + # even when the step exited successfully — they're easy to miss in the + # raw log and have caused silent migrations before. + if grep -q '^ERROR:' "$raw" 2>/dev/null; then + local err_count + err_count=$(grep -c '^ERROR:' "$raw") + echo " $(dim "${err_count} error(s) reported — see $raw")" + grep '^ERROR:' "$raw" | head -3 | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "$name: ${err_count} non-fatal errors" + fi + elif grep -q '^SKIPPED:' "$raw" 2>/dev/null; then + local reason + reason=$(grep '^SKIPPED:' "$raw" | head -1 | sed 's/^SKIPPED://') + step_skip "$label $(dim "($reason)")" + log "$name: skipped ($reason)" + record_step "$name" "skipped" + else + step_fail "$label" + echo + tail -10 "$raw" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + echo + log "$name: FAILED (see $raw)" + record_step "$name" "failed" + fi +} + +# ─── phase 1: core state ──────────────────────────────────────────────── + +echo "$(bold 'Phase 1: Core state')" +echo + +run_step "1a-env" \ + "Merge .env" \ + "setup/migrate-v2/env.ts" "$V1_PATH" + +run_step "1b-db" \ + "Seed v2 database" \ + "setup/migrate-v2/db.ts" "$V1_PATH" + +run_step "1c-groups" \ + "Copy group folders" \ + "setup/migrate-v2/groups.ts" "$V1_PATH" + +run_step "1d-sessions" \ + "Copy session data" \ + "setup/migrate-v2/sessions.ts" "$V1_PATH" + +run_step "1e-tasks" \ + "Port scheduled tasks" \ + "setup/migrate-v2/tasks.ts" "$V1_PATH" + +echo +step_ok "Phase 1 complete" +echo + +# ─── phase 2: channels (interactive) ──────────────────────────────────── + +echo "$(bold 'Phase 2: Channels')" +echo + +# Channel selection — clack multiselect (interactive) or NANOCLAW_CHANNELS env var. +# NANOCLAW_CHANNELS accepts comma-separated channel names: "telegram,discord" +SELECTED_CHANNELS=() +CHANNEL_SELECT_OUT="$STEPS_DIR/2a-channels-selected.txt" + +pnpm exec tsx setup/migrate-v2/select-channels.ts "$CHANNEL_SELECT_OUT" || true + +if [ -f "$CHANNEL_SELECT_OUT" ]; then + while IFS= read -r ch; do + [ -n "$ch" ] && SELECTED_CHANNELS+=("$ch") + done < "$CHANNEL_SELECT_OUT" +fi + +if [ ${#SELECTED_CHANNELS[@]} -eq 0 ]; then + echo + step_skip "No channels selected" +else + echo + step_info "Selected: ${SELECTED_CHANNELS[*]}" + echo + + # 2b. Copy channel auth state + run_step "2b-channel-auth" \ + "Copy channel credentials" \ + "setup/migrate-v2/channel-auth.ts" "$V1_PATH" "${SELECTED_CHANNELS[@]}" + + # 2c. Install channel code + for ch in "${SELECTED_CHANNELS[@]}"; do + INSTALL_SCRIPT="setup/install-${ch}.sh" + STEP_NAME="2c-install-${ch}" + if [ -f "$INSTALL_SCRIPT" ]; then + STEP_LOG="$STEPS_DIR/${STEP_NAME}.log" + if bash "$INSTALL_SCRIPT" > "$STEP_LOG" 2>&1; then + STATUS_LINE=$(grep '^STATUS:' "$STEP_LOG" | head -1 | sed 's/^STATUS: *//') + if [ "$STATUS_LINE" = "already-installed" ]; then + step_skip "Install $ch $(dim "(already installed)")" + record_step "$STEP_NAME" "skipped" + else + step_ok "Install $ch" + record_step "$STEP_NAME" "success" + fi + log "install-$ch: $STATUS_LINE" + else + step_fail "Install $ch" + tail -5 "$STEP_LOG" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "install-$ch: FAILED (see $STEP_LOG)" + record_step "$STEP_NAME" "failed" + fi + else + step_skip "Install $ch $(dim "(no install script)")" + log "install-$ch: no install script" + record_step "$STEP_NAME" "failed" + fi + done + + # 2d. WhatsApp LID resolution. After whatsapp is installed (so Baileys + # is on disk) and auth files have been copied (so we can connect with + # the migrated identity), boot Baileys briefly to learn LID↔phone + # mappings during initial sync, then write paired LID-keyed + # messaging_groups. Best-effort: any failure degrades to runtime + # approval flow, which the WA adapter's isMention=true on DMs handles. + for ch in "${SELECTED_CHANNELS[@]}"; do + if [ "$ch" = "whatsapp" ]; then + run_step "2d-whatsapp-lids" \ + "Resolve WhatsApp LIDs for migrated DMs" \ + "setup/migrate-v2/whatsapp-resolve-lids.ts" + break + fi + done +fi + +echo +step_ok "Phase 2 complete" +echo + +# ─── phase 3: infrastructure ──────────────────────────────────────────── + +echo "$(bold 'Phase 3: Infrastructure')" +echo + +# 3a. Docker — install if missing (OneCLI needs it) +if command -v docker >/dev/null 2>&1; then + DOCKER_V=$(docker --version 2>/dev/null | head -1) + step_ok "Docker available $(dim "($DOCKER_V)")" + log "Docker: $DOCKER_V" +else + step_info "Installing Docker…" + DOCKER_LOG="$STEPS_DIR/3a-docker.log" + if bash setup/install-docker.sh > "$DOCKER_LOG" 2>&1; then + hash -r 2>/dev/null || true + step_ok "Docker installed" + record_step "3a-docker" "success" + log "Docker: installed" + else + step_fail "Docker install failed $(dim "(see $DOCKER_LOG)")" + record_step "3a-docker" "failed" + log "Docker: FAILED" + fi +fi + +# 3b. OneCLI — detect or install via setup step (requires Docker) +ONECLI_OK=false +ONECLI_URL_FROM_ENV=$(grep '^ONECLI_URL=' .env 2>/dev/null | head -1 | sed 's/^ONECLI_URL=//') +ONECLI_URL_CHECK="${ONECLI_URL_FROM_ENV:-http://127.0.0.1:10254}" + +if curl -sf "${ONECLI_URL_CHECK}/health" >/dev/null 2>&1; then + step_ok "OneCLI running at $(dim "$ONECLI_URL_CHECK")" + ONECLI_OK=true + log "OneCLI: running at $ONECLI_URL_CHECK" +elif command -v docker >/dev/null 2>&1; then + step_info "Setting up OneCLI…" + ONECLI_LOG="$STEPS_DIR/3b-onecli.log" + ONECLI_ERR="$STEPS_DIR/3b-onecli.err" + if pnpm exec tsx setup/index.ts --step onecli > "$ONECLI_LOG" 2>"$ONECLI_ERR"; then + step_ok "OneCLI ready" + ONECLI_OK=true + record_step "3b-onecli" "success" + log "OneCLI: installed/configured" + else + step_fail "OneCLI setup failed $(dim "(see $ONECLI_LOG)")" + record_step "3b-onecli" "failed" + log "OneCLI: FAILED" + fi +else + step_fail "OneCLI needs Docker $(dim "(install Docker first)")" + record_step "3b-onecli" "failed" + log "OneCLI: skipped (no Docker)" +fi + +# 3c. Anthropic credential — run the auth setup step if no credential found +if grep -qE '^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN)=' .env 2>/dev/null; then + step_ok "Anthropic credential found in .env" + log "Anthropic credential: found in .env" +elif [ "$ONECLI_OK" = "true" ]; then + step_info "Registering Anthropic credential…" + AUTH_LOG="$STEPS_DIR/3c-auth.log" + AUTH_ERR="$STEPS_DIR/3c-auth.err" + if pnpm exec tsx setup/index.ts --step auth > "$AUTH_LOG" 2>"$AUTH_ERR"; then + step_ok "Anthropic credential registered" + record_step "3c-auth" "success" + log "Anthropic credential: registered via auth step" + else + step_fail "Auth setup failed $(dim "(see $AUTH_LOG)")" + record_step "3c-auth" "failed" + log "Anthropic credential: FAILED" + fi +else + step_info "No Anthropic credential $(dim "(OneCLI not available — add manually to .env)")" + log "Anthropic credential: skipped (no OneCLI)" +fi + +# 3d. Copy container skills from v1 that v2 doesn't have +V1_SKILLS_DIR="$V1_PATH/container/skills" +V2_SKILLS_DIR="$PROJECT_ROOT/container/skills" + +if [ -d "$V1_SKILLS_DIR" ]; then + SKILLS_COPIED=0 + SKILLS_SKIPPED=0 + for skill_dir in "$V1_SKILLS_DIR"/*/; do + [ -d "$skill_dir" ] || continue + skill_name=$(basename "$skill_dir") + if [ -d "$V2_SKILLS_DIR/$skill_name" ]; then + SKILLS_SKIPPED=$((SKILLS_SKIPPED + 1)) + else + cp -r "$skill_dir" "$V2_SKILLS_DIR/$skill_name" + SKILLS_COPIED=$((SKILLS_COPIED + 1)) + fi + done + if [ $SKILLS_COPIED -gt 0 ]; then + step_ok "Copied $SKILLS_COPIED container skills $(dim "(skipped $SKILLS_SKIPPED already in v2)")" + else + step_skip "All v1 container skills already in v2 $(dim "($SKILLS_SKIPPED)")" + fi + log "Container skills: copied=$SKILLS_COPIED skipped=$SKILLS_SKIPPED" +else + step_skip "No v1 container skills" +fi + +# 3e. Build agent container image +if command -v docker >/dev/null 2>&1; then + step_info "Building agent container image…" + BUILD_LOG="$STEPS_DIR/3e-container-build.log" + if bash container/build.sh > "$BUILD_LOG" 2>&1; then + step_ok "Container image built" + record_step "3e-build" "success" + log "Container build: success" + else + step_fail "Container build failed" + record_step "3e-build" "failed" + tail -10 "$BUILD_LOG" 2>/dev/null | while IFS= read -r line; do + echo " $(dim "$line")" + done + log "Container build: FAILED (see $BUILD_LOG)" + fi +else + step_fail "Docker not available — cannot build container" + record_step "3e-build" "failed" + log "Container build: skipped (no Docker)" +fi + +echo +step_ok "Phase 3 complete" +echo + +# ─── service switchover ───────────────────────────────────────────────── + +echo "$(bold 'Service switchover')" +echo + +# Detect platform and service names +V1_SERVICE="" +V2_SERVICE="" +PLATFORM_SERVICE="" + +if [ "$(uname -s)" = "Darwin" ]; then + PLATFORM_SERVICE="launchd" + V1_SERVICE="com.nanoclaw" + # v2 uses install-slug for unique service names + V2_SERVICE=$(pnpm exec tsx -e "import{getLaunchdLabel}from'./src/install-slug.js';console.log(getLaunchdLabel())" 2>/dev/null || echo "") +elif [ "$(uname -s)" = "Linux" ]; then + PLATFORM_SERVICE="systemd" + V1_SERVICE="nanoclaw" + V2_SERVICE=$(pnpm exec tsx -e "import{getSystemdUnit}from'./src/install-slug.js';console.log(getSystemdUnit())" 2>/dev/null || echo "") +fi + +# Check if v1 service is running +V1_RUNNING=false +if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user is-active "$V1_SERVICE" >/dev/null 2>&1 && V1_RUNNING=true +elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl list "$V1_SERVICE" >/dev/null 2>&1 && V1_RUNNING=true +fi + +SERVICE_SWITCHED=false +if [ "$V1_RUNNING" = "true" ]; then + step_info "v1 service is running $(dim "($V1_SERVICE)")" + + # Ask user if they want to switch + SWITCH_ANSWER_FILE=$(mktemp) + pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --offer-switch "$SWITCH_ANSWER_FILE" || true + SWITCH_ANSWER=$(cat "$SWITCH_ANSWER_FILE" 2>/dev/null || echo "skip") + rm -f "$SWITCH_ANSWER_FILE" + + if [ "$SWITCH_ANSWER" = "switch" ]; then + # Stop v1 + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user stop "$V1_SERVICE" 2>/dev/null && step_ok "Stopped v1 service" || step_fail "Could not stop v1" + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl unload ~/Library/LaunchAgents/${V1_SERVICE}.plist 2>/dev/null && step_ok "Stopped v1 service" || step_fail "Could not stop v1" + fi + + # Install and start v2 service + V2_SERVICE_LOG="$STEPS_DIR/service-install.log" + V2_SERVICE_ERR="$STEPS_DIR/service-install.err" + if pnpm exec tsx setup/index.ts --step service > "$V2_SERVICE_LOG" 2>"$V2_SERVICE_ERR"; then + # Parse the actual unit name from the service step stdout (clean, no ANSI) + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + V2_SERVICE=$(grep '^SERVICE_UNIT:' "$V2_SERVICE_LOG" | head -1 | sed 's/^SERVICE_UNIT: *//') + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + V2_SERVICE=$(grep '^SERVICE_LABEL:' "$V2_SERVICE_LOG" | head -1 | sed 's/^SERVICE_LABEL: *//') + fi + step_ok "v2 service installed and started $(dim "($V2_SERVICE)")" + else + step_fail "Could not start v2 service $(dim "(see $V2_SERVICE_LOG)")" + fi + + SERVICE_SWITCHED=true + echo + step_info "v2 is running — send a test message to your bot" + echo + + # Ask: keep or revert? + KEEP_ANSWER_FILE=$(mktemp) + pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --keep-or-revert "$KEEP_ANSWER_FILE" || true + KEEP_ANSWER=$(cat "$KEEP_ANSWER_FILE" 2>/dev/null || echo "keep") + rm -f "$KEEP_ANSWER_FILE" + + if [ "$KEEP_ANSWER" = "revert" ]; then + # Stop v2 + if [ "$PLATFORM_SERVICE" = "systemd" ] && [ -n "$V2_SERVICE" ]; then + systemctl --user stop "$V2_SERVICE" 2>/dev/null || true + systemctl --user disable "$V2_SERVICE" 2>/dev/null || true + elif [ "$PLATFORM_SERVICE" = "launchd" ] && [ -n "$V2_SERVICE" ]; then + launchctl unload ~/Library/LaunchAgents/${V2_SERVICE}.plist 2>/dev/null || true + fi + + # Restart v1 + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user start "$V1_SERVICE" 2>/dev/null || true + elif [ "$PLATFORM_SERVICE" = "launchd" ]; then + launchctl load ~/Library/LaunchAgents/${V1_SERVICE}.plist 2>/dev/null || true + fi + + step_ok "Reverted to v1 service" + SERVICE_SWITCHED=false + else + step_ok "Keeping v2 service" + # Disable v1 from auto-starting + if [ "$PLATFORM_SERVICE" = "systemd" ]; then + systemctl --user disable "$V1_SERVICE" 2>/dev/null || true + fi + fi + else + step_skip "Service switchover skipped" + fi +else + step_skip "v1 service not running — nothing to switch" +fi + +echo + +# ─── phase 4: handoff ─────────────────────────────────────────────────── +# handoff.json is written by the EXIT trap (write_handoff) — always, even on +# abort. Here we just print the summary. + +echo "$(bold 'Phase 4: Handoff')" +echo + +step_ok "Wrote handoff summary" + +# Summary +echo +echo "$(bold '── Migration complete ──')" +echo +echo " $(dim 'v1:') $V1_PATH" +echo " $(dim 'v2:') $PROJECT_ROOT" +echo +echo " $(bold 'What was done:')" +echo " $(green '✓') .env keys merged" +echo " $(green '✓') Database seeded (agent groups, messaging groups, wiring)" +echo " $(green '✓') Group folders copied (CLAUDE.md → CLAUDE.local.md)" +echo " $(green '✓') Session data copied" +echo " $(green '✓') Scheduled tasks ported" +if [ ${#SELECTED_CHANNELS[@]} -gt 0 ]; then +echo " $(green '✓') Channels installed: ${SELECTED_CHANNELS[*]}" +fi +echo " $(green '✓') Container skills copied" +echo " $(green '✓') Container image built" +echo +echo " $(bold 'What still needs a human:')" +if [ "$ONECLI_OK" = "false" ]; then +echo " $(dim '·') Set up OneCLI: pnpm exec tsx setup/index.ts --step onecli" +fi +if ! grep -qE '^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN)=' .env 2>/dev/null; then +echo " $(dim '·') Add Anthropic credential to .env or OneCLI vault" +fi +echo " $(dim '·') Run $(bold '/migrate-from-v1') in Claude to finish:" +echo " $(dim '- Seed your owner account')" +echo " $(dim '- Set access policies')" +echo " $(dim '- Port any custom v1 code')" +echo +echo " $(dim "Handoff: $LOGS_DIR/setup-migration/handoff.json")" +echo " $(dim "Full log: $MIGRATE_LOG")" +echo " $(dim "Step logs: $STEPS_DIR/")" +echo + +# ─── hand off to Claude ───────────────────────────────────────────────── + +if command -v claude >/dev/null 2>&1; then + write_handoff + trap - EXIT + exec claude "/migrate-from-v1" +fi diff --git a/setup/auto.ts b/setup/auto.ts index c468abc..0095321 100644 --- a/setup/auto.ts +++ b/setup/auto.ts @@ -14,8 +14,8 @@ * "Terminal Agent". * NANOCLAW_SKIP comma-separated step names to skip * (environment|container|onecli|auth|mounts| - * service|cli-agent|timezone|channel|verify| - * first-chat) + * service|cli-agent|timezone|channel| + * verify|first-chat) * * Timezone is auto-detected after the CLI agent step. UTC resolves are * confirmed with the user, and free-text replies fall through to a @@ -434,7 +434,11 @@ async function main(): Promise { await runTimezoneStep(); } + // v1 → v2 migration is handled by `bash migrate-v2.sh`, not the setup flow. + // Users migrating from v1 run that script before (or instead of) setup. + let channelChoice: ChannelChoice = 'skip'; + if (!skip.has('channel')) { channelChoice = await askChannelChoice(); if (channelChoice !== 'skip') { diff --git a/setup/index.ts b/setup/index.ts index 200b9e2..ee02e45 100644 --- a/setup/index.ts +++ b/setup/index.ts @@ -14,6 +14,7 @@ const STEPS: Record< environment: () => import('./environment.js'), container: () => import('./container.js'), register: () => import('./register.js'), + 'pair-telegram': () => import('./pair-telegram.js'), groups: () => import('./groups.js'), 'whatsapp-auth': () => import('./whatsapp-auth.js'), 'signal-auth': () => import('./signal-auth.js'), diff --git a/setup/migrate-v2/channel-auth.ts b/setup/migrate-v2/channel-auth.ts new file mode 100644 index 0000000..05d706d --- /dev/null +++ b/setup/migrate-v2/channel-auth.ts @@ -0,0 +1,134 @@ +/** + * migrate-v2 step: channel-auth + * + * Copy channel auth state from v1 to v2 for selected channels. + * Handles both env keys and on-disk auth files (Baileys, Matrix, etc.) + * per the CHANNEL_AUTH_REGISTRY. + * + * Usage: pnpm exec tsx setup/migrate-v2/channel-auth.ts [channel2...] + */ +import fs from 'fs'; +import path from 'path'; + +import { CHANNEL_AUTH_REGISTRY } from './shared.js'; + +function parseEnv(filePath: string): Map { + const out = new Map(); + if (!fs.existsSync(filePath)) return out; + for (const line of fs.readFileSync(filePath, 'utf-8').split('\n')) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) continue; + const eq = trimmed.indexOf('='); + if (eq <= 0) continue; + out.set(trimmed.slice(0, eq).trim(), trimmed.slice(eq + 1)); + } + return out; +} + +function appendEnvKey(envPath: string, key: string, value: string): boolean { + const existing = parseEnv(envPath); + if (existing.has(key)) return false; + + let content = fs.existsSync(envPath) ? fs.readFileSync(envPath, 'utf-8') : ''; + if (content && !content.endsWith('\n')) content += '\n'; + content += `${key}=${value}\n`; + fs.writeFileSync(envPath, content); + return true; +} + +function copyGlob(v1Root: string, v2Root: string, relativePath: string): string[] { + const src = path.join(v1Root, relativePath); + if (!fs.existsSync(src)) return []; + + const copied: string[] = []; + const stat = fs.statSync(src); + + if (stat.isFile()) { + const dst = path.join(v2Root, relativePath); + if (!fs.existsSync(dst)) { + fs.mkdirSync(path.dirname(dst), { recursive: true }); + fs.copyFileSync(src, dst); + copied.push(relativePath); + } + } else if (stat.isDirectory()) { + const dst = path.join(v2Root, relativePath); + fs.mkdirSync(dst, { recursive: true }); + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + const sub = path.join(relativePath, entry.name); + copied.push(...copyGlob(v1Root, v2Root, sub)); + } + } + + return copied; +} + +function main(): void { + const args = process.argv.slice(2); + const v1Path = args[0]; + const channels = args.slice(1); + + if (!v1Path || channels.length === 0) { + console.error('Usage: tsx setup/migrate-v2/channel-auth.ts [channel2...]'); + process.exit(1); + } + + const v1EnvPath = path.join(v1Path, '.env'); + const v2EnvPath = path.join(process.cwd(), '.env'); + const v1Env = parseEnv(v1EnvPath); + + let envKeysCopied = 0; + let filesCopied = 0; + let channelsProcessed = 0; + const missing: string[] = []; + + for (const channel of channels) { + const spec = CHANNEL_AUTH_REGISTRY[channel]; + if (!spec) { + // Unknown channel — just try copying env keys with common naming + channelsProcessed++; + continue; + } + + // Copy env keys + for (const key of spec.v1EnvKeys) { + const value = v1Env.get(key); + if (value) { + if (appendEnvKey(v2EnvPath, key, value)) { + envKeysCopied++; + } + } + } + + // Check required v2 keys — report missing ones + const v2Env = parseEnv(v2EnvPath); + for (const req of spec.requiredV2Keys) { + if (!v2Env.has(req.key)) { + missing.push(`${channel}:${req.key} (${req.where})`); + } + } + + // Copy on-disk auth files + for (const candidate of spec.candidatePaths) { + const copied = copyGlob(v1Path, process.cwd(), candidate); + filesCopied += copied.length; + } + + channelsProcessed++; + } + + // Sync to data/env/env + if (fs.existsSync(v2EnvPath)) { + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + try { + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { /* non-fatal */ } + } + + console.log(`OK:channels=${channelsProcessed},env_keys=${envKeysCopied},files=${filesCopied}`); + if (missing.length > 0) { + console.log(`MISSING:${missing.join(',')}`); + } +} + +main(); diff --git a/setup/migrate-v2/db.ts b/setup/migrate-v2/db.ts new file mode 100644 index 0000000..ada4764 --- /dev/null +++ b/setup/migrate-v2/db.ts @@ -0,0 +1,231 @@ +/** + * migrate-v2 step: db + * + * Seed v2.db from v1's registered_groups table. + * Creates agent_groups, messaging_groups, and messaging_group_agents. + * + * Does NOT seed users/user_roles — the /migrate-from-v1 skill handles that. + * + * Idempotent: re-running skips rows that already exist. + * + * Usage: pnpm exec tsx setup/migrate-v2/db.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { createAgentGroup, getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { initDb } from '../../src/db/connection.js'; +import { + createMessagingGroup, + createMessagingGroupAgent, + getMessagingGroupAgentByPair, + getMessagingGroupAgents, + getMessagingGroupByPlatform, + updateMessagingGroup, +} from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { readEnvFile } from '../../src/env.js'; +import { buildDiscordResolver, type DiscordResolver } from './discord-resolver.js'; +import { + generateId, + inferIsGroup, + parseJid, + triggerToEngage, + v2PlatformId, +} from './shared.js'; + +interface V1Group { + jid: string; + name: string; + folder: string; + trigger_pattern: string | null; + requires_trigger: number | null; + is_main: number | null; +} + +async function main(): Promise { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/db.ts '); + process.exit(1); + } + + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + if (!fs.existsSync(v1DbPath)) { + console.error(`v1 DB not found: ${v1DbPath}`); + process.exit(1); + } + + // Read v1 groups + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + + // v1 schema varies — channel_name was a late addition. Query only the + // columns we know exist in all v1 installs. + const v1Groups = v1Db + .prepare('SELECT jid, name, folder, trigger_pattern, requires_trigger, is_main FROM registered_groups') + .all() as V1Group[]; + v1Db.close(); + + if (v1Groups.length === 0) { + console.log('SKIPPED:no registered groups in v1'); + process.exit(0); + } + + // Init v2 DB + fs.mkdirSync(path.join(process.cwd(), 'data'), { recursive: true }); + const v2Db = initDb(path.join(DATA_DIR, 'v2.db')); + runMigrations(v2Db); + + let created = 0; + let reused = 0; + let skipped = 0; + const errors: string[] = []; + + // v1 stored Discord groups as `dc:` with no guild/DM signal. + // v2 needs either `discord::` (guild) or + // `discord:@me:` (DM / group DM). Use the v1 bot token to + // enumerate guilds + channels and to classify any leftover ids as DMs. + // On any failure the resolver returns null for every channel and the + // affected groups skip with a clear warning. + let discordResolver: DiscordResolver | null = null; + const discordChannelIds = v1Groups + .map((g) => parseJid(g.jid)) + .filter((p): p is NonNullable => p?.channel_type === 'discord') + .map((p) => p.id); + if (discordChannelIds.length > 0) { + const env = readEnvFile(['DISCORD_BOT_TOKEN']); + discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? '', discordChannelIds); + const stats = discordResolver.stats(); + if (stats.reason) { + console.log(`WARN:discord resolver disabled: ${stats.reason}`); + } else { + console.log( + `INFO:discord resolver: ${stats.guilds} guild(s), ${stats.channels} guild channel(s), ${stats.dms} DM(s)`, + ); + } + } + + for (const g of v1Groups) { + const parsed = parseJid(g.jid); + if (!parsed) { + skipped++; + errors.push(`Could not parse JID: ${g.jid}`); + continue; + } + + const channelType = parsed.channel_type; + let platformId: string; + if (channelType === 'discord') { + const resolved = discordResolver?.resolve(parsed.id) ?? null; + if (!resolved) { + const stats = discordResolver?.stats(); + const why = stats?.reason + ? `discord resolver unavailable (${stats.reason})` + : 'not found in any guild the bot can see — re-add the bot to that server and re-run, or rewire after migration'; + skipped++; + errors.push(`Discord channel ${parsed.id} (${g.folder}): ${why}`); + continue; + } + platformId = resolved; + } else { + platformId = v2PlatformId(channelType, parsed.raw); + } + const createdAt = new Date().toISOString(); + + try { + // agent_group — one per folder + let ag = getAgentGroupByFolder(g.folder); + if (!ag) { + createAgentGroup({ + id: generateId('ag'), + name: g.name || g.folder, + folder: g.folder, + agent_provider: null, + created_at: createdAt, + }); + ag = getAgentGroupByFolder(g.folder)!; + } + + // messaging_group — one per (channel_type, platform_id). + // + // If the row already exists *and* has zero wired agent_groups, it + // was almost certainly auto-created by the runtime router on an + // inbound message (which uses 'request_approval' or similar — not + // the migration's 'public'). Reset its policy to match what the + // migration would have set if it had created the row first. Once + // any wiring exists, the user has had a chance to tighten the + // policy via the skill — leave it alone. + let mg = getMessagingGroupByPlatform(channelType, platformId); + if (!mg) { + createMessagingGroup({ + id: generateId('mg'), + channel_type: channelType, + platform_id: platformId, + name: g.name || null, + is_group: inferIsGroup(channelType, platformId), + unknown_sender_policy: 'public', + created_at: createdAt, + }); + mg = getMessagingGroupByPlatform(channelType, platformId)!; + } else if ( + mg.unknown_sender_policy !== 'public' && + getMessagingGroupAgents(mg.id).length === 0 + ) { + updateMessagingGroup(mg.id, { unknown_sender_policy: 'public' }); + mg = getMessagingGroupByPlatform(channelType, platformId)!; + } + + // messaging_group_agents — wire them + const existing = getMessagingGroupAgentByPair(mg.id, ag.id); + if (!existing) { + const engage = triggerToEngage({ + trigger_pattern: g.trigger_pattern, + requires_trigger: g.requires_trigger, + }); + createMessagingGroupAgent({ + id: generateId('mga'), + messaging_group_id: mg.id, + agent_group_id: ag.id, + engage_mode: engage.engage_mode, + engage_pattern: engage.engage_pattern, + sender_scope: 'all', + ignored_message_policy: 'drop', + session_mode: 'shared', + priority: 0, + created_at: createdAt, + }); + created++; + } else { + reused++; + } + } catch (err) { + skipped++; + errors.push(`${g.folder}: ${err instanceof Error ? err.message : String(err)}`); + } + } + + v2Db.close(); + + // If every group was skipped, the migration didn't actually do anything. + // Treat that as failure so the wrapper script surfaces it instead of + // hiding it under an `OK:` line. + const totalDone = created + reused; + if (v1Groups.length > 0 && totalDone === 0) { + console.error(`FAIL:groups=${v1Groups.length},created=0,reused=0,skipped=${skipped}`); + for (const e of errors) console.error(`ERROR:${e}`); + process.exit(1); + } + + console.log(`OK:groups=${v1Groups.length},created=${created},reused=${reused},skipped=${skipped}`); + if (errors.length > 0) { + for (const e of errors) console.log(`ERROR:${e}`); + } +} + +main().catch((err) => { + console.error(`FAIL:${err instanceof Error ? err.message : String(err)}`); + process.exit(1); +}); diff --git a/setup/migrate-v2/discord-resolver.test.ts b/setup/migrate-v2/discord-resolver.test.ts new file mode 100644 index 0000000..e74fc40 --- /dev/null +++ b/setup/migrate-v2/discord-resolver.test.ts @@ -0,0 +1,195 @@ +import { describe, expect, it, vi } from 'vitest'; + +import { buildDiscordResolver } from './discord-resolver.js'; + +function mockFetch(handlers: Record): typeof fetch { + return vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + const match = Object.keys(handlers).find((k) => url.startsWith(k)); + if (!match) throw new Error(`unexpected fetch: ${url}`); + const body = handlers[match]; + if (body instanceof Error) throw body; + if (typeof body === 'object' && body !== null && 'status' in body && (body as { status?: number }).status) { + const r = body as { status: number; statusText?: string; body?: string }; + return new Response(r.body ?? '', { status: r.status, statusText: r.statusText ?? '' }); + } + return new Response(JSON.stringify(body), { status: 200 }); + }) as unknown as typeof fetch; +} + +describe('buildDiscordResolver', () => { + it('returns empty resolver when token is missing', async () => { + const r = await buildDiscordResolver(''); + expect(r.stats()).toMatchObject({ guilds: 0, channels: 0, dms: 0 }); + expect(r.stats().reason).toMatch(/no DISCORD_BOT_TOKEN/); + expect(r.resolve('any')).toBeNull(); + }); + + it('resolves channels to guild-prefixed platform ids', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [ + { id: 'g1', name: 'Guild 1' }, + { id: 'g2', name: 'Guild 2' }, + ], + 'https://discord.com/api/v10/guilds/g1/channels': [ + { id: 'c1' }, + { id: 'c2' }, + ], + 'https://discord.com/api/v10/guilds/g2/channels': [ + { id: 'c3' }, + ], + }); + + const r = await buildDiscordResolver('valid-token', [], fetchImpl); + + expect(r.stats()).toEqual({ guilds: 2, channels: 3, dms: 0 }); + expect(r.resolve('c1')).toBe('discord:g1:c1'); + expect(r.resolve('c2')).toBe('discord:g1:c2'); + expect(r.resolve('c3')).toBe('discord:g2:c3'); + expect(r.resolve('cX')).toBeNull(); + }); + + it('returns disabled resolver on 401', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': { + status: 401, + statusText: 'Unauthorized', + body: '{"message":"401: Unauthorized","code":0}', + }, + }); + + const r = await buildDiscordResolver('bad-token', [], fetchImpl); + expect(r.stats().guilds).toBe(0); + expect(r.stats().reason).toMatch(/401/); + expect(r.resolve('c1')).toBeNull(); + }); + + it('keeps partial results when one guild lookup fails', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [ + { id: 'g1', name: 'Good Guild' }, + { id: 'g2', name: 'Bad Guild' }, + ], + 'https://discord.com/api/v10/guilds/g1/channels': [{ id: 'c1' }], + 'https://discord.com/api/v10/guilds/g2/channels': { + status: 403, + statusText: 'Forbidden', + body: '{}', + }, + }); + + const errSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + const r = await buildDiscordResolver('valid-token', [], fetchImpl); + errSpy.mockRestore(); + + expect(r.resolve('c1')).toBe('discord:g1:c1'); + expect(r.stats().guilds).toBe(2); + expect(r.stats().channels).toBe(1); + }); + + it('paginates the guild list', async () => { + // First page: 200 guilds (g0..g199); second page: 1 guild (g200); third call would not happen. + const page1 = Array.from({ length: 200 }, (_, i) => ({ id: `g${i}`, name: `G${i}` })); + const page2 = [{ id: 'g200', name: 'G200' }]; + let call = 0; + const fetchImpl = vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (url.includes('/users/@me/guilds')) { + call++; + const body = call === 1 ? page1 : page2; + return new Response(JSON.stringify(body), { status: 200 }); + } + // Every guild has one channel named after itself + const m = /\/guilds\/([^/]+)\/channels/.exec(url); + const gid = m ? m[1] : ''; + return new Response(JSON.stringify([{ id: `c-${gid}` }]), { status: 200 }); + }) as unknown as typeof fetch; + + const r = await buildDiscordResolver('valid-token', [], fetchImpl); + + expect(r.stats().guilds).toBe(201); + expect(r.stats().channels).toBe(201); + expect(r.resolve('c-g0')).toBe('discord:g0:c-g0'); + expect(r.resolve('c-g200')).toBe('discord:g200:c-g200'); + }); + + it('classifies unresolved ids as DMs and emits discord:@me:', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [{ id: 'g1', name: 'G1' }], + 'https://discord.com/api/v10/guilds/g1/channels': [{ id: 'guild-chan' }], + // dmId is a 1:1 DM (type=1) + 'https://discord.com/api/v10/channels/dmId': { id: 'dmId', type: 1 }, + // groupDmId is a multi-recipient DM (type=3) + 'https://discord.com/api/v10/channels/groupDmId': { id: 'groupDmId', type: 3 }, + }); + + const r = await buildDiscordResolver( + 'valid-token', + ['guild-chan', 'dmId', 'groupDmId'], + fetchImpl, + ); + + expect(r.stats()).toEqual({ guilds: 1, channels: 1, dms: 2 }); + expect(r.resolve('guild-chan')).toBe('discord:g1:guild-chan'); + expect(r.resolve('dmId')).toBe('discord:@me:dmId'); + expect(r.resolve('groupDmId')).toBe('discord:@me:groupDmId'); + }); + + it('leaves ids unresolved when classify returns 404 or non-DM type', async () => { + const fetchImpl = mockFetch({ + 'https://discord.com/api/v10/users/@me/guilds': [], + // 404 — bot has no access (typical when bot was kicked from the guild) + 'https://discord.com/api/v10/channels/orphanId': { + status: 404, + statusText: 'Not Found', + body: '{"message":"Unknown Channel","code":10003}', + }, + // type=0 — guild text channel in a guild we no longer enumerate (shouldn't happen, + // but the fallback is conservative: only emit @me for type 1/3) + 'https://discord.com/api/v10/channels/leftoverGuildChan': { + id: 'leftoverGuildChan', + type: 0, + }, + }); + + const r = await buildDiscordResolver( + 'valid-token', + ['orphanId', 'leftoverGuildChan'], + fetchImpl, + ); + + expect(r.stats()).toEqual({ guilds: 0, channels: 0, dms: 0 }); + expect(r.resolve('orphanId')).toBeNull(); + expect(r.resolve('leftoverGuildChan')).toBeNull(); + }); + + it('skips classify for ids already found in a guild and dedupes input', async () => { + let dmCallCount = 0; + const fetchImpl = vi.fn(async (input: string | URL | Request) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (url.includes('/users/@me/guilds')) { + return new Response(JSON.stringify([{ id: 'g1', name: 'G1' }]), { status: 200 }); + } + if (url.includes('/guilds/g1/channels')) { + return new Response(JSON.stringify([{ id: 'guild-chan' }]), { status: 200 }); + } + if (url.includes('/channels/dmId')) { + dmCallCount++; + return new Response(JSON.stringify({ id: 'dmId', type: 1 }), { status: 200 }); + } + throw new Error(`unexpected fetch: ${url}`); + }) as unknown as typeof fetch; + + // 'guild-chan' is in the guild map (skip classify); 'dmId' appears twice + // in the input (classify exactly once). + const r = await buildDiscordResolver( + 'valid-token', + ['guild-chan', 'dmId', 'dmId'], + fetchImpl, + ); + + expect(dmCallCount).toBe(1); + expect(r.resolve('guild-chan')).toBe('discord:g1:guild-chan'); + expect(r.resolve('dmId')).toBe('discord:@me:dmId'); + }); +}); diff --git a/setup/migrate-v2/discord-resolver.ts b/setup/migrate-v2/discord-resolver.ts new file mode 100644 index 0000000..ecc1d5e --- /dev/null +++ b/setup/migrate-v2/discord-resolver.ts @@ -0,0 +1,176 @@ +/** + * Discord channel → platform_id resolver for the v1 → v2 migration. + * + * v1 stored Discord groups as `dc:` — only the channel id, with + * no signal for guild vs. DM. v2's `@chat-adapter/discord` encodes + * `platform_id` as either `discord::` (guild channel) + * or `discord:@me:` (DM / group DM) — see `guild_id || "@me"` + * in the runtime adapter. We can't reconstruct that from v1 data alone, so + * we use the v1 bot token (carried forward by 1a-env) to query Discord: + * 1. Enumerate every guild the bot is in and every channel in those + * guilds → channelId → guildId map. + * 2. For any v1 channel id NOT in that map, classify via `GET + * /channels/` — DM (type=1) and GROUP_DM (type=3) get + * `discord:@me:`. Anything else returns null and the caller + * skips with a warning. + * + * Network calls are best-effort: on auth failure or network error, the + * resolver returns null for every channel and the caller falls back to + * skipping with a clear warning. + */ + +const DISCORD_API = 'https://discord.com/api/v10'; + +// Discord channel types we care about. See: +// https://discord.com/developers/docs/resources/channel#channel-object-channel-types +const CHANNEL_TYPE_DM = 1; +const CHANNEL_TYPE_GROUP_DM = 3; + +interface Guild { + id: string; + name: string; +} + +interface Channel { + id: string; + name?: string; +} + +interface ChannelInfo { + id: string; + type: number; +} + +export interface DiscordResolver { + /** + * Returns the v2 `platform_id` for a v1 channel id, or null if the bot + * can't see it. Format is `discord::` for guild + * channels and `discord:@me:` for DMs / group DMs. + */ + resolve(channelId: string): string | null; + /** Diagnostic info — guild count, channel count, DM count, optional disable reason. */ + stats(): { guilds: number; channels: number; dms: number; reason?: string }; +} + +/** A no-op resolver that returns null for every lookup with a stored reason. */ +function emptyResolver(reason: string): DiscordResolver { + return { + resolve: () => null, + stats: () => ({ guilds: 0, channels: 0, dms: 0, reason }), + }; +} + +type FetchFn = typeof fetch; + +async function getJson(url: string, token: string, fetchImpl: FetchFn): Promise { + const res = await fetchImpl(url, { + headers: { + Authorization: `Bot ${token}`, + 'User-Agent': 'NanoClaw-Migration (https://github.com/qwibitai/nanoclaw, 2.x)', + }, + }); + if (!res.ok) { + const body = await res.text().catch(() => ''); + throw new Error(`Discord API ${res.status} ${res.statusText}: ${body.slice(0, 200)}`); + } + return (await res.json()) as T; +} + +/** + * Build a Discord resolver by enumerating every guild the bot is in and + * every channel in those guilds, then classifying any `unresolvedChannelIds` + * that didn't show up in a guild via `GET /channels/` (so DMs and + * group DMs can be encoded as `discord:@me:`). + * + * Returns an empty resolver on any error during guild enumeration. + * + * Costs: 1 + N + K HTTP calls — N = guild count (enumerated channels per + * guild), K = unresolved-channel classification calls. Discord's global + * rate limit is 50 req/s; even installs with hundreds of guilds finish in + * under a second of network time. + */ +export async function buildDiscordResolver( + token: string, + unresolvedChannelIds: string[] = [], + fetchImpl: FetchFn = fetch, +): Promise { + if (!token) return emptyResolver('no DISCORD_BOT_TOKEN in .env'); + + // Page through guilds. Default page size is 200; loop until short page. + const guilds: Guild[] = []; + let after: string | null = null; + try { + while (true) { + const url = new URL(`${DISCORD_API}/users/@me/guilds`); + url.searchParams.set('limit', '200'); + if (after) url.searchParams.set('after', after); + const page = await getJson(url.toString(), token, fetchImpl); + guilds.push(...page); + if (page.length < 200) break; + after = page[page.length - 1].id; + } + } catch (err) { + return emptyResolver(`failed to list guilds: ${err instanceof Error ? err.message : String(err)}`); + } + + // Per-guild channel enumeration. + const channelToGuild = new Map(); + for (const guild of guilds) { + try { + const channels = await getJson( + `${DISCORD_API}/guilds/${guild.id}/channels`, + token, + fetchImpl, + ); + for (const ch of channels) { + channelToGuild.set(ch.id, guild.id); + } + } catch (err) { + // Skip this guild but keep going — partial results are still useful. + // The caller logs which channels couldn't be resolved. + console.error( + `WARN:discord-resolver: failed to enumerate guild ${guild.id} (${guild.name}): ${ + err instanceof Error ? err.message : String(err) + }`, + ); + } + } + + // Classify any v1 channel ids that didn't surface in a guild — they're + // most likely DMs (type=1) or group DMs (type=3). Anything else (404, + // 403, type=0 in a guild the bot left) stays unresolved so the caller's + // existing skip-with-warning path fires. + const dmChannels = new Set(); + const seen = new Set(); + for (const channelId of unresolvedChannelIds) { + if (channelToGuild.has(channelId)) continue; + if (seen.has(channelId)) continue; + seen.add(channelId); + try { + const ch = await getJson( + `${DISCORD_API}/channels/${channelId}`, + token, + fetchImpl, + ); + if (ch.type === CHANNEL_TYPE_DM || ch.type === CHANNEL_TYPE_GROUP_DM) { + dmChannels.add(channelId); + } + } catch { + // Channel not visible to the bot — leave it unresolved. + } + } + + return { + resolve(channelId: string): string | null { + const guildId = channelToGuild.get(channelId); + if (guildId) return `discord:${guildId}:${channelId}`; + if (dmChannels.has(channelId)) return `discord:@me:${channelId}`; + return null; + }, + stats: () => ({ + guilds: guilds.length, + channels: channelToGuild.size, + dms: dmChannels.size, + }), + }; +} diff --git a/setup/migrate-v2/env.ts b/setup/migrate-v2/env.ts new file mode 100644 index 0000000..5ac52f6 --- /dev/null +++ b/setup/migrate-v2/env.ts @@ -0,0 +1,81 @@ +/** + * migrate-v2 step: env + * + * Copy every key from v1 .env into v2 .env. Never overwrites existing v2 + * keys. Idempotent — re-running skips keys already present. + * + * Usage: pnpm exec tsx setup/migrate-v2/env.ts + */ +import fs from 'fs'; +import path from 'path'; + +function parseEnv(text: string): Map { + const out = new Map(); + for (const raw of text.split('\n')) { + const line = raw.trimEnd(); + if (!line || line.startsWith('#')) continue; + const eq = line.indexOf('='); + if (eq <= 0) continue; + const key = line.slice(0, eq).trim(); + if (!/^[A-Z_][A-Z0-9_]*$/i.test(key)) continue; + out.set(key, line); + } + return out; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/env.ts '); + process.exit(1); + } + + const v1EnvPath = path.join(v1Path, '.env'); + if (!fs.existsSync(v1EnvPath)) { + console.log('SKIPPED:no v1 .env'); + process.exit(0); + } + + const v2EnvPath = path.join(process.cwd(), '.env'); + const v1Lines = parseEnv(fs.readFileSync(v1EnvPath, 'utf-8')); + const v2Text = fs.existsSync(v2EnvPath) ? fs.readFileSync(v2EnvPath, 'utf-8') : ''; + const v2Lines = parseEnv(v2Text); + + const copied: string[] = []; + const skipped: string[] = []; + const appended: string[] = []; + + const BLOCK_START = '# ── migrated from v1 ──'; + const alreadyMigrated = v2Text.includes(BLOCK_START); + + for (const [key, raw] of v1Lines) { + if (v2Lines.has(key)) { + skipped.push(key); + continue; + } + copied.push(key); + appended.push(raw); + } + + if (appended.length > 0) { + let result = v2Text; + if (result && !result.endsWith('\n')) result += '\n'; + if (!alreadyMigrated) result += `\n${BLOCK_START}\n`; + result += appended.join('\n') + '\n'; + fs.writeFileSync(v2EnvPath, result); + } + + // Sync to data/env/env (container reads from here) + const containerEnvDir = path.join(process.cwd(), 'data', 'env'); + try { + fs.mkdirSync(containerEnvDir, { recursive: true }); + fs.copyFileSync(v2EnvPath, path.join(containerEnvDir, 'env')); + } catch { + // Non-fatal + } + + console.log(`OK:copied=${copied.length},skipped=${skipped.length}`); + if (copied.length > 0) console.log(`COPIED:${copied.join(',')}`); +} + +main(); diff --git a/setup/migrate-v2/groups.ts b/setup/migrate-v2/groups.ts new file mode 100644 index 0000000..d1e9ed0 --- /dev/null +++ b/setup/migrate-v2/groups.ts @@ -0,0 +1,133 @@ +/** + * migrate-v2 step: groups + * + * Copy v1 group folders into v2. + * - v1 CLAUDE.md → v2 CLAUDE.local.md (v2 composes CLAUDE.md at spawn) + * - v1 container_config → .v1-container-config.json sidecar + * - All other files copied (no overwrite) + * - Also copies global/ if it exists + * + * Idempotent — does not overwrite files that already exist in v2. + * + * Usage: pnpm exec tsx setup/migrate-v2/groups.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +const SKIP_NAMES = new Set(['CLAUDE.md', 'logs', '.git', '.DS_Store', 'node_modules']); + +/** + * Copy a directory tree, skipping SKIP_NAMES. Never overwrites existing files. + * + * Symlinks are skipped, not followed: v1 group folders sometimes contain + * container-side paths like `.claude-shared.md → /app/CLAUDE.md` that + * don't resolve on the host. Following them with `fs.copyFileSync` would + * crash ENOENT on a broken target and abort the rest of the traversal. + * v2 uses composed CLAUDE.md fragments anyway — these v1 symlinks have no + * v2 meaning and don't need to be carried forward. + */ +function copyTree(src: string, dst: string): number { + let written = 0; + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(dst, { recursive: true }); + + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + if (SKIP_NAMES.has(entry.name)) continue; + const s = path.join(src, entry.name); + const d = path.join(dst, entry.name); + + if (entry.isSymbolicLink()) { + console.log(`SKIP:symlink ${path.relative(process.cwd(), s)}`); + continue; + } + if (entry.isDirectory()) { + written += copyTree(s, d); + continue; + } + if (fs.existsSync(d)) continue; + fs.copyFileSync(s, d); + written += 1; + } + return written; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/groups.ts '); + process.exit(1); + } + + const v1GroupsDir = path.join(v1Path, 'groups'); + const v2GroupsDir = path.join(process.cwd(), 'groups'); + + if (!fs.existsSync(v1GroupsDir)) { + console.log('SKIPPED:no v1 groups/ directory'); + process.exit(0); + } + + // Get all folders from v1 DB to know which groups are registered + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + const registeredFolders = new Set(); + if (fs.existsSync(v1DbPath)) { + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + const rows = v1Db + .prepare('SELECT folder, container_config FROM registered_groups') + .all() as Array<{ folder: string; container_config: string | null }>; + const containerConfigs = new Map(); + for (const r of rows) { + registeredFolders.add(r.folder); + containerConfigs.set(r.folder, r.container_config); + } + v1Db.close(); + + // Write container.json from v1 container_config. + // The additionalMounts shape is identical between v1 and v2. + for (const [folder, config] of containerConfigs) { + if (!config) continue; + const v2Folder = path.join(v2GroupsDir, folder); + const containerJson = path.join(v2Folder, 'container.json'); + if (fs.existsSync(containerJson)) continue; + fs.mkdirSync(v2Folder, { recursive: true }); + try { + const parsed = JSON.parse(config) as Record; + fs.writeFileSync(containerJson, JSON.stringify(parsed, null, 2)); + } catch { + // Unparseable config — write as sidecar for the skill to handle + fs.writeFileSync(path.join(v2Folder, '.v1-container-config.json'), config); + } + } + } + + // Copy all v1 group folders (registered + global + any extras) + let foldersCopied = 0; + let claudesMigrated = 0; + let filesCopied = 0; + + for (const entry of fs.readdirSync(v1GroupsDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const folder = entry.name; + const v1Folder = path.join(v1GroupsDir, folder); + const v2Folder = path.join(v2GroupsDir, folder); + + fs.mkdirSync(v2Folder, { recursive: true }); + + // CLAUDE.md → CLAUDE.local.md + const v1Claude = path.join(v1Folder, 'CLAUDE.md'); + const v2Local = path.join(v2Folder, 'CLAUDE.local.md'); + if (fs.existsSync(v1Claude) && !fs.existsSync(v2Local)) { + fs.copyFileSync(v1Claude, v2Local); + claudesMigrated++; + } + + // Copy everything else + filesCopied += copyTree(v1Folder, v2Folder); + foldersCopied++; + } + + console.log(`OK:folders=${foldersCopied},claudes=${claudesMigrated},files=${filesCopied}`); +} + +main(); diff --git a/setup/migrate-v2/select-channels.ts b/setup/migrate-v2/select-channels.ts new file mode 100644 index 0000000..eecf1ab --- /dev/null +++ b/setup/migrate-v2/select-channels.ts @@ -0,0 +1,63 @@ +/** + * migrate-v2: interactive channel selection via clack multiselect. + * + * Writes selected channel names (one per line) to the file path given as + * the first argument. Clack renders to the terminal normally. + * + * If NANOCLAW_CHANNELS env var is set (comma-separated names), skips the + * prompt and writes those directly. + * + * Usage: pnpm exec tsx setup/migrate-v2/select-channels.ts + */ +import fs from 'fs'; + +import * as p from '@clack/prompts'; + +const CHANNELS = [ + { value: 'telegram', label: 'Telegram' }, + { value: 'discord', label: 'Discord' }, + { value: 'slack', label: 'Slack' }, + { value: 'whatsapp', label: 'WhatsApp' }, + { value: 'teams', label: 'Microsoft Teams' }, + { value: 'matrix', label: 'Matrix' }, + { value: 'imessage', label: 'iMessage' }, + { value: 'webex', label: 'Webex' }, + { value: 'gchat', label: 'Google Chat' }, + { value: 'resend', label: 'Resend (email)' }, + { value: 'github', label: 'GitHub' }, + { value: 'linear', label: 'Linear' }, + { value: 'whatsapp-cloud', label: 'WhatsApp Cloud API' }, +]; + +const VALID_NAMES = new Set(CHANNELS.map((c) => c.value)); + +async function main(): Promise { + const outFile = process.argv[2]; + if (!outFile) { + console.error('Usage: tsx setup/migrate-v2/select-channels.ts '); + process.exit(1); + } + + // Non-interactive: NANOCLAW_CHANNELS="telegram,discord" + const envChannels = process.env.NANOCLAW_CHANNELS?.trim(); + if (envChannels) { + const names = envChannels.split(',').map((s) => s.trim()).filter((s) => VALID_NAMES.has(s)); + fs.writeFileSync(outFile, names.join('\n') + '\n'); + return; + } + + const selected = await p.multiselect({ + message: 'Which channels do you want to set up?', + options: CHANNELS, + required: false, + }); + + if (p.isCancel(selected)) { + fs.writeFileSync(outFile, ''); + return; + } + + fs.writeFileSync(outFile, (selected as string[]).join('\n') + '\n'); +} + +main(); diff --git a/setup/migrate-v2/sessions.ts b/setup/migrate-v2/sessions.ts new file mode 100644 index 0000000..7ca8516 --- /dev/null +++ b/setup/migrate-v2/sessions.ts @@ -0,0 +1,183 @@ +/** + * migrate-v2 step: sessions + * + * For each v1 session folder, create a proper v2 session: + * 1. Create a sessions row in v2.db (via resolveSession) + * 2. Initialize the session folder (inbound.db, outbound.db, outbox/) + * 3. Write session routing so the container knows where to reply + * 4. Copy v1 .claude/ state into v2's .claude-shared/ directory + * + * v1: data/sessions//.claude/ (settings, conversation history, skills) + * v2: data/v2-sessions//.claude-shared/ + session folder + * + * v1's agent-runner-src/ is NOT copied — v2 uses a completely different + * Bun-based agent-runner. + * + * Idempotent — reuses existing sessions, does not overwrite files. + * + * Usage: pnpm exec tsx setup/migrate-v2/sessions.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb, closeDb } from '../../src/db/connection.js'; +import { getAllAgentGroups } from '../../src/db/agent-groups.js'; +import { getMessagingGroupsByAgentGroup } from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { + resolveSession, + writeSessionRouting, + outboundDbPath, +} from '../../src/session-manager.js'; + +const SKIP_NAMES = new Set(['.DS_Store']); + +/** Recursively copy, never overwriting existing files. */ +function copyTree(src: string, dst: string): number { + let written = 0; + if (!fs.existsSync(src)) return 0; + fs.mkdirSync(dst, { recursive: true }); + + for (const entry of fs.readdirSync(src, { withFileTypes: true })) { + if (SKIP_NAMES.has(entry.name)) continue; + const s = path.join(src, entry.name); + const d = path.join(dst, entry.name); + + if (entry.isDirectory()) { + written += copyTree(s, d); + continue; + } + // Skip dangling symlinks (e.g. v1's .claude/debug/latest pointer). + if (entry.isSymbolicLink() && !fs.existsSync(s)) continue; + if (fs.existsSync(d)) continue; + fs.copyFileSync(s, d); + written += 1; + } + return written; +} + +function main(): void { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/sessions.ts '); + process.exit(1); + } + + const v1SessionsDir = path.join(v1Path, 'data', 'sessions'); + if (!fs.existsSync(v1SessionsDir)) { + console.log('SKIPPED:no v1 data/sessions/ directory'); + process.exit(0); + } + + // Init v2 central DB + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('v2.db not found — run db step first'); + process.exit(1); + } + + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + const agentGroups = getAllAgentGroups(); + const folderToAg = new Map(); + for (const ag of agentGroups) { + folderToAg.set(ag.folder, ag); + } + + let sessionsCreated = 0; + let sessionsReused = 0; + let sessionsSkipped = 0; + let filesCopied = 0; + + for (const entry of fs.readdirSync(v1SessionsDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const folder = entry.name; + + const ag = folderToAg.get(folder); + if (!ag) { + sessionsSkipped++; + continue; + } + + // Find the messaging groups wired to this agent group + const messagingGroups = getMessagingGroupsByAgentGroup(ag.id); + if (messagingGroups.length === 0) { + sessionsSkipped++; + continue; + } + + // Create a session for each messaging group (v1 had one session per + // folder, v2 has one per agent_group + messaging_group pair) + for (const mg of messagingGroups) { + const { session, created } = resolveSession(ag.id, mg.id, null, 'shared'); + + if (created) { + // Write routing so the container knows where to reply + writeSessionRouting(ag.id, session.id); + sessionsCreated++; + } else { + sessionsReused++; + } + } + + // Copy v1 .claude/ state into v2's .claude-shared/ directory + // This is per-agent-group, shared across all sessions for that group + const v1ClaudeDir = path.join(v1SessionsDir, folder, '.claude'); + if (fs.existsSync(v1ClaudeDir)) { + const v2ClaudeDir = path.join(DATA_DIR, 'v2-sessions', ag.id, '.claude-shared'); + filesCopied += copyTree(v1ClaudeDir, v2ClaudeDir); + + // v1 containers worked in /workspace/group, v2 works in /workspace/agent. + // Claude Code stores sessions under projects//. Copy the v1 + // project dir to the v2 path so Claude Code finds the conversation history. + const projectsDir = path.join(v2ClaudeDir, 'projects'); + const v1ProjectDir = path.join(projectsDir, '-workspace-group'); + const v2ProjectDir = path.join(projectsDir, '-workspace-agent'); + if (fs.existsSync(v1ProjectDir) && !fs.existsSync(v2ProjectDir)) { + filesCopied += copyTree(v1ProjectDir, v2ProjectDir); + } + + // Write the v1 Claude Code session ID as the continuation in outbound.db + // so the agent-runner resumes the exact same conversation. + // The session ID is the JSONL filename (without extension) under the + // project dir. + const sourceDir = fs.existsSync(v2ProjectDir) ? v2ProjectDir : v1ProjectDir; + if (fs.existsSync(sourceDir)) { + const jsonlFiles = fs.readdirSync(sourceDir).filter((f) => f.endsWith('.jsonl')); + if (jsonlFiles.length > 0) { + // Use the most recent JSONL file (by mtime from v1) + const v1SessionId = jsonlFiles + .map((f) => ({ + name: f.replace('.jsonl', ''), + mtime: fs.statSync(path.join(sourceDir, f)).mtimeMs, + })) + .sort((a, b) => b.mtime - a.mtime)[0].name; + + // Write into each v2 session's outbound.db for this agent group + const sessions = getMessagingGroupsByAgentGroup(ag.id); + for (const mg of sessions) { + const { session } = resolveSession(ag.id, mg.id, null, 'shared'); + const obPath = outboundDbPath(ag.id, session.id); + if (fs.existsSync(obPath)) { + const ob = new Database(obPath); + ob.prepare( + "INSERT OR REPLACE INTO session_state (key, value, updated_at) VALUES ('continuation:claude', ?, ?)", + ).run(v1SessionId, new Date().toISOString()); + ob.close(); + } + } + } + } + } + } + + closeDb(); + + console.log(`OK:created=${sessionsCreated},reused=${sessionsReused},skipped=${sessionsSkipped},files=${filesCopied}`); +} + +main(); diff --git a/setup/migrate-v2/shared.ts b/setup/migrate-v2/shared.ts new file mode 100644 index 0000000..58c0b16 --- /dev/null +++ b/setup/migrate-v2/shared.ts @@ -0,0 +1,228 @@ +/** + * Shared helpers for the v1 → v2 migration steps. + */ + +// ── JID parsing ───────────────────────────────────────────────────────── + +/** v1 JID prefix → v2 channel_type. Unknown prefixes pass through as-is. */ +export const JID_PREFIX_TO_CHANNEL: Record = { + dc: 'discord', + discord: 'discord', + tg: 'telegram', + telegram: 'telegram', + wa: 'whatsapp', + whatsapp: 'whatsapp', + slack: 'slack', + matrix: 'matrix', + mx: 'matrix', + teams: 'teams', + imessage: 'imessage', + im: 'imessage', + email: 'email', + webex: 'webex', + gchat: 'gchat', + linear: 'linear', + github: 'github', +}; + +export interface ParsedJid { + raw: string; + prefix: string; + id: string; + channel_type: string; +} + +/** WhatsApp (Baileys) JID hosts. v1 stored these raw, with no `wa:` prefix. */ +const WA_JID_HOSTS = new Set(['s.whatsapp.net', 'g.us', 'lid', 'broadcast', 'newsletter']); + +function isWhatsappJid(raw: string): boolean { + const at = raw.lastIndexOf('@'); + if (at === -1) return false; + return WA_JID_HOSTS.has(raw.slice(at + 1).toLowerCase()); +} + +export function parseJid(raw: string): ParsedJid | null { + if (isWhatsappJid(raw)) { + return { raw, prefix: 'whatsapp', id: raw, channel_type: 'whatsapp' }; + } + const colon = raw.indexOf(':'); + if (colon === -1) return null; + const prefix = raw.slice(0, colon).toLowerCase(); + const id = raw.slice(colon + 1); + if (!prefix || !id) return null; + return { + raw, + prefix, + id, + channel_type: JID_PREFIX_TO_CHANNEL[prefix] ?? prefix, + }; +} + +/** + * Build a v2 platform_id from a v1 JID, in the format the runtime adapter + * for that channel emits. WhatsApp uses the raw Baileys JID (`@`, + * no prefix). Other channels use `:`. + */ +export function v2PlatformId(channelType: string, jid: string): string { + if (channelType === 'whatsapp') { + // Strip any v1 `wa:`/`whatsapp:` prefix; otherwise pass through raw. + const parsed = parseJid(jid); + return parsed?.channel_type === 'whatsapp' ? parsed.id : jid; + } + const parsed = parseJid(jid); + const id = parsed?.id ?? jid; + return id.startsWith(`${channelType}:`) ? id : `${channelType}:${id}`; +} + +/** + * Infer messaging_groups.is_group from a v2 platform_id, given a channel type. + * + * v1 didn't track is_group, but most channels encode it in the JID/id format: + * - whatsapp: `@g.us` is a group, `@s.whatsapp.net` / `@lid` is a DM + * - telegram: negative chat IDs are groups, positive are DMs + * - everything else: default to 1 (group/channel) — least-surprising guess + * for chats v1 chose to register, where DM auto-create paths weren't used + */ +export function inferIsGroup(channelType: string, platformId: string): number { + if (channelType === 'whatsapp') { + return platformId.endsWith('@g.us') ? 1 : 0; + } + if (channelType === 'telegram') { + // platform_id is `telegram:` — negative chatId means group/channel. + const chatId = platformId.replace(/^telegram:/, ''); + return chatId.startsWith('-') ? 1 : 0; + } + return 1; +} + +// ── Trigger mapping ───────────────────────────────────────────────────── + +/** + * Map v1's trigger_pattern + requires_trigger to v2's engage_mode + engage_pattern. + * + * Key rule: requires_trigger=0 means "respond to everything" regardless + * of the pattern value. The pattern was for mention highlighting, not gating. + */ +export function triggerToEngage(input: { + trigger_pattern: string | null; + requires_trigger: number | null; +}): { + engage_mode: 'pattern' | 'mention' | 'mention-sticky'; + engage_pattern: string | null; +} { + const pattern = input.trigger_pattern && input.trigger_pattern.trim().length > 0 ? input.trigger_pattern : null; + const requiresTrigger = input.requires_trigger !== 0; + + if (pattern === '.' || pattern === '.*') { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + if (!requiresTrigger) { + return { engage_mode: 'pattern', engage_pattern: '.' }; + } + if (pattern) { + return { engage_mode: 'pattern', engage_pattern: pattern }; + } + return { engage_mode: 'mention', engage_pattern: null }; +} + +// ── ID generation ─────────────────────────────────────────────────────── + +export function generateId(prefix: string): string { + return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`; +} + +// ── Channel auth registry ─────────────────────────────────────────────── + +export interface ChannelAuthSpec { + v1EnvKeys: string[]; + requiredV2Keys: { key: string; where: string }[]; + candidatePaths: string[]; + note?: string; +} + +export const CHANNEL_AUTH_REGISTRY: Record = { + discord: { + v1EnvKeys: ['DISCORD_BOT_TOKEN', 'DISCORD_CLIENT_ID', 'DISCORD_GUILD_ID'], + requiredV2Keys: [ + { key: 'DISCORD_BOT_TOKEN', where: 'Discord Developer Portal → Application → Bot → Token' }, + { key: 'DISCORD_APPLICATION_ID', where: 'Discord Developer Portal → Application → General → Application ID' }, + { key: 'DISCORD_PUBLIC_KEY', where: 'Discord Developer Portal → Application → General → Public Key' }, + ], + candidatePaths: [], + note: 'v1 used raw discord.js (bot token only). v2 uses Chat SDK and needs APPLICATION_ID + PUBLIC_KEY too.', + }, + telegram: { + v1EnvKeys: ['TELEGRAM_BOT_TOKEN', 'TELEGRAM_API_ID', 'TELEGRAM_API_HASH'], + requiredV2Keys: [ + { key: 'TELEGRAM_BOT_TOKEN', where: 'BotFather on Telegram → /mybots → Bot → API Token' }, + ], + candidatePaths: ['data/sessions/telegram', 'store/telegram-session'], + }, + whatsapp: { + v1EnvKeys: ['WHATSAPP_PHONE', 'WHATSAPP_OWNER'], + requiredV2Keys: [], + candidatePaths: [ + 'data/sessions/baileys', + 'data/baileys_auth', + 'store/auth_info_baileys', + 'store/baileys', + 'auth_info_baileys', + ], + note: 'Baileys keystore — copying is best-effort. Encryption sessions may still need a fresh pair via /add-whatsapp.', + }, + matrix: { + v1EnvKeys: ['MATRIX_HOMESERVER', 'MATRIX_USER_ID', 'MATRIX_ACCESS_TOKEN'], + requiredV2Keys: [ + { key: 'MATRIX_HOMESERVER', where: 'your Matrix homeserver URL (e.g. https://matrix.org)' }, + { key: 'MATRIX_ACCESS_TOKEN', where: 'Element → Settings → Help & About → Access Token (keep secret)' }, + ], + candidatePaths: ['data/matrix-store', 'store/matrix', 'data/sessions/matrix'], + }, + slack: { + v1EnvKeys: ['SLACK_BOT_TOKEN', 'SLACK_APP_TOKEN', 'SLACK_SIGNING_SECRET'], + requiredV2Keys: [ + { key: 'SLACK_BOT_TOKEN', where: 'Slack app → OAuth & Permissions → Bot User OAuth Token (xoxb-…)' }, + { key: 'SLACK_SIGNING_SECRET', where: 'Slack app → Basic Information → Signing Secret' }, + ], + candidatePaths: [], + }, + teams: { + v1EnvKeys: ['TEAMS_APP_ID', 'TEAMS_APP_PASSWORD', 'TEAMS_TENANT_ID'], + requiredV2Keys: [ + { key: 'TEAMS_APP_ID', where: 'Azure portal → App registration → Application (client) ID' }, + { key: 'TEAMS_APP_PASSWORD', where: 'Azure portal → App registration → Certificates & secrets' }, + ], + candidatePaths: [], + }, + imessage: { + v1EnvKeys: ['IMESSAGE_PHOTON_URL', 'IMESSAGE_PHOTON_TOKEN'], + requiredV2Keys: [], + candidatePaths: ['data/imessage', 'store/imessage'], + }, + webex: { + v1EnvKeys: ['WEBEX_BOT_TOKEN'], + requiredV2Keys: [{ key: 'WEBEX_BOT_TOKEN', where: 'Webex developer portal → Bot → Bot Access Token' }], + candidatePaths: [], + }, + gchat: { + v1EnvKeys: ['GCHAT_SERVICE_ACCOUNT', 'GCHAT_WEBHOOK_URL'], + requiredV2Keys: [], + candidatePaths: ['data/gchat-credentials.json', 'store/gchat-sa.json'], + }, + resend: { + v1EnvKeys: ['RESEND_API_KEY', 'RESEND_FROM'], + requiredV2Keys: [{ key: 'RESEND_API_KEY', where: 'resend.com → API Keys' }], + candidatePaths: [], + }, + github: { + v1EnvKeys: ['GITHUB_WEBHOOK_SECRET', 'GITHUB_APP_ID', 'GITHUB_PRIVATE_KEY_PATH'], + requiredV2Keys: [], + candidatePaths: [], + note: 'Webhook channel — secrets carry over, but GitHub webhook URLs are new per v2 install.', + }, + linear: { + v1EnvKeys: ['LINEAR_API_KEY', 'LINEAR_WEBHOOK_SECRET'], + requiredV2Keys: [{ key: 'LINEAR_API_KEY', where: 'Linear → Settings → API → Personal API keys' }], + candidatePaths: [], + }, +}; diff --git a/setup/migrate-v2/switchover-prompt.ts b/setup/migrate-v2/switchover-prompt.ts new file mode 100644 index 0000000..996b302 --- /dev/null +++ b/setup/migrate-v2/switchover-prompt.ts @@ -0,0 +1,53 @@ +/** + * migrate-v2: service switchover prompts. + * + * Writes a single word to the output file: + * --offer-switch → "switch" | "skip" + * --keep-or-revert → "keep" | "revert" + * + * Clack renders to the terminal normally. + * + * Usage: pnpm exec tsx setup/migrate-v2/switchover-prompt.ts --offer-switch + */ +import fs from 'fs'; + +import * as p from '@clack/prompts'; + +async function main(): Promise { + const mode = process.argv[2]; + const outFile = process.argv[3]; + + if (!outFile) { + console.error('Usage: tsx setup/migrate-v2/switchover-prompt.ts <--offer-switch|--keep-or-revert> '); + process.exit(1); + } + + if (mode === '--offer-switch') { + const answer = await p.select({ + message: 'Want to stop the v1 service and start v2 so you can test?', + options: [ + { value: 'switch', label: 'Yes, switch to v2 now', hint: 'you can switch back after' }, + { value: 'skip', label: 'No, skip for now', hint: 'start v2 manually later' }, + ], + }); + fs.writeFileSync(outFile, p.isCancel(answer) ? 'skip' : String(answer)); + return; + } + + if (mode === '--keep-or-revert') { + const answer = await p.select({ + message: 'Keep v2 running, or switch back to v1?', + options: [ + { value: 'keep', label: 'Keep v2', hint: 'v1 stays stopped' }, + { value: 'revert', label: 'Switch back to v1', hint: 'stop v2, restart v1' }, + ], + }); + fs.writeFileSync(outFile, p.isCancel(answer) ? 'revert' : String(answer)); + return; + } + + console.error('Usage: --offer-switch | --keep-or-revert'); + process.exit(1); +} + +main(); diff --git a/setup/migrate-v2/tasks.ts b/setup/migrate-v2/tasks.ts new file mode 100644 index 0000000..4d3b3b5 --- /dev/null +++ b/setup/migrate-v2/tasks.ts @@ -0,0 +1,178 @@ +/** + * migrate-v2 step: tasks + * + * Port v1 scheduled_tasks into v2 session inbound DBs. + * + * v1: scheduled_tasks table (schedule_type, schedule_value, next_run) + * v2: messages_in rows with kind='task' in per-session inbound.db + * + * Requires: db step must have run first (agent_groups + messaging_groups seeded). + * + * Usage: pnpm exec tsx setup/migrate-v2/tasks.ts + */ +import fs from 'fs'; +import path from 'path'; + +import Database from 'better-sqlite3'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb, closeDb } from '../../src/db/connection.js'; +import { getAgentGroupByFolder } from '../../src/db/agent-groups.js'; +import { getMessagingGroupByPlatform } from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { insertTask } from '../../src/modules/scheduling/db.js'; +import { openInboundDb, resolveSession } from '../../src/session-manager.js'; +import { readEnvFile } from '../../src/env.js'; +import { buildDiscordResolver, type DiscordResolver } from './discord-resolver.js'; +import { parseJid, v2PlatformId } from './shared.js'; + +interface V1Task { + id: string; + group_folder: string; + chat_jid: string; + prompt: string; + schedule_type: string; + schedule_value: string; + next_run: string | null; + status: string; + context_mode: string | null; + script: string | null; +} + +function toCron(t: V1Task): { processAfter: string; recurrence: string | null } | null { + const now = new Date().toISOString(); + + if (t.schedule_type === 'cron') { + const fields = t.schedule_value.trim().split(/\s+/).length; + if (fields < 5 || fields > 6) return null; + return { processAfter: t.next_run || now, recurrence: t.schedule_value.trim() }; + } + + if (t.schedule_type === 'interval') { + const m = /^(\d+)([smhd])$/.exec(t.schedule_value.trim()); + if (!m) return null; + const n = parseInt(m[1], 10); + const unit = m[2]; + if (!n || n < 1) return null; + let cron: string | null = null; + if (unit === 'm' && n < 60) cron = `*/${n} * * * *`; + else if (unit === 'h' && n < 24) cron = `0 */${n} * * *`; + else if (unit === 'd' && n < 28) cron = `0 0 */${n} * *`; + if (!cron) return null; + return { processAfter: t.next_run || now, recurrence: cron }; + } + + if (t.schedule_type === 'once' || t.schedule_type === 'at') { + return { processAfter: t.next_run || t.schedule_value || now, recurrence: null }; + } + + return null; +} + +async function main(): Promise { + const v1Path = process.argv[2]; + if (!v1Path) { + console.error('Usage: tsx setup/migrate-v2/tasks.ts '); + process.exit(1); + } + + const v1DbPath = path.join(v1Path, 'store', 'messages.db'); + if (!fs.existsSync(v1DbPath)) { + console.log('SKIPPED:no v1 DB'); + process.exit(0); + } + + // Read v1 tasks + const v1Db = new Database(v1DbPath, { readonly: true, fileMustExist: true }); + const allTasks = v1Db.prepare('SELECT * FROM scheduled_tasks').all() as V1Task[]; + v1Db.close(); + + const activeTasks = allTasks.filter((t) => t.status === 'active'); + if (activeTasks.length === 0) { + console.log('SKIPPED:no active tasks'); + process.exit(0); + } + + // Init v2 central DB + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('v2.db not found — run db step first'); + process.exit(1); + } + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + let migrated = 0; + let skipped = 0; + let failed = 0; + + // Mirrors db.ts: Discord platform_id needs API lookup to recover guildId. + let discordResolver: DiscordResolver | null = null; + const hasDiscord = activeTasks.some((t) => parseJid(t.chat_jid)?.channel_type === 'discord'); + if (hasDiscord) { + const env = readEnvFile(['DISCORD_BOT_TOKEN']); + discordResolver = await buildDiscordResolver(env.DISCORD_BOT_TOKEN ?? ''); + } + + for (const t of activeTasks) { + try { + const ag = getAgentGroupByFolder(t.group_folder); + if (!ag) { skipped++; continue; } + + const parsed = parseJid(t.chat_jid); + if (!parsed) { skipped++; continue; } + + let platformId: string; + if (parsed.channel_type === 'discord') { + const resolved = discordResolver?.resolve(parsed.id) ?? null; + if (!resolved) { skipped++; continue; } + platformId = resolved; + } else { + platformId = v2PlatformId(parsed.channel_type, t.chat_jid); + } + const mg = getMessagingGroupByPlatform(parsed.channel_type, platformId); + if (!mg) { skipped++; continue; } + + const scheduling = toCron(t); + if (!scheduling) { skipped++; continue; } + + const { session } = resolveSession(ag.id, mg.id, null, 'shared'); + const inboxDb = openInboundDb(ag.id, session.id); + try { + // Idempotence check + const existing = inboxDb + .prepare("SELECT id FROM messages_in WHERE id = ? AND kind = 'task'") + .get(t.id) as { id: string } | undefined; + if (existing) { skipped++; continue; } + + insertTask(inboxDb, { + id: t.id, + processAfter: scheduling.processAfter, + recurrence: scheduling.recurrence, + platformId, + channelType: parsed.channel_type, + threadId: null, + content: JSON.stringify({ + prompt: t.prompt, + script: t.script ?? null, + migrated_from_v1: { original_id: t.id, context_mode: t.context_mode ?? null }, + }), + }); + migrated++; + } finally { + inboxDb.close(); + } + } catch (err) { + failed++; + console.error(`TASK_ERROR:${t.id}:${err instanceof Error ? err.message : String(err)}`); + } + } + + closeDb(); + console.log(`OK:active=${activeTasks.length},migrated=${migrated},skipped=${skipped},failed=${failed}`); +} + +main().catch((err) => { + console.error(`FAIL:${err instanceof Error ? err.message : String(err)}`); + process.exit(1); +}); diff --git a/setup/migrate-v2/whatsapp-resolve-lids.ts b/setup/migrate-v2/whatsapp-resolve-lids.ts new file mode 100644 index 0000000..7a5eb8b --- /dev/null +++ b/setup/migrate-v2/whatsapp-resolve-lids.ts @@ -0,0 +1,192 @@ +/** + * migrate-v2 step: resolve WhatsApp LIDs for migrated DM messaging_groups. + * + * Why this exists + * ─────────────── + * v1 stored every WhatsApp DM as `@s.whatsapp.net`. v2's WA adapter + * sometimes resolves the chat to `@lid` instead — when WhatsApp + * delivers a message via the LID protocol and Baileys hasn't yet learned + * a LID→phone mapping for that contact (cold cache after migration). The + * router then can't find the phone-keyed messaging_group and silently + * drops the message at router.ts:184 — until the LID is learned (which + * happens lazily, message-by-message, via `chats.phoneNumberShare`). + * + * Baileys persists LID↔phone mappings to disk as + * `store/auth/lid-mapping-_reverse.json` (LID → phone) and + * `lid-mapping-.json` (phone → LID). v1 will already have populated + * these for every contact it talked to. This step parses the reverse + * files and writes paired LID-keyed `messaging_groups` + + * `messaging_group_agents` rows so both `@s.whatsapp.net` and + * `@lid` route to the same agent_group with the same engage rules. + * + * No Baileys boot, no network — pure filesystem read. If store/auth is + * missing or has no reverse mappings, exits 0 with a SKIPPED. Runtime + * fallback (WA adapter sets isMention=true on DMs → router auto-creates + * with `unknown_sender_policy=request_approval`) handles anything we + * miss. + * + * Usage: pnpm exec tsx setup/migrate-v2/whatsapp-resolve-lids.ts + */ +import fs from 'fs'; +import path from 'path'; + +import { DATA_DIR } from '../../src/config.js'; +import { initDb } from '../../src/db/connection.js'; +import { + createMessagingGroup, + createMessagingGroupAgent, + getMessagingGroupAgentByPair, + getMessagingGroupByPlatform, +} from '../../src/db/messaging-groups.js'; +import { runMigrations } from '../../src/db/migrations/index.js'; +import { generateId } from './shared.js'; + +interface RawMessagingGroup { + id: string; + channel_type: string; + platform_id: string; +} + +interface RawWiring { + id: string; + messaging_group_id: string; + agent_group_id: string; + engage_mode: string; + engage_pattern: string | null; + sender_scope: string; + ignored_message_policy: string; + session_mode: string; + priority: number; +} + +const REVERSE_FILE_RE = /^lid-mapping-(\d+)_reverse\.json$/; + +/** + * Read store/auth/lid-mapping-*_reverse.json into a Map. + * Returns an empty Map if the directory doesn't exist. + */ +function readReverseMappings(authDir: string): Map { + const out = new Map(); + if (!fs.existsSync(authDir)) return out; + for (const entry of fs.readdirSync(authDir)) { + const m = REVERSE_FILE_RE.exec(entry); + if (!m) continue; + const lidUser = m[1]; + try { + const raw = fs.readFileSync(path.join(authDir, entry), 'utf-8').trim(); + // The file content is a JSON-encoded string: `""` + const phoneUser = JSON.parse(raw); + if (typeof phoneUser !== 'string' || phoneUser.length === 0) continue; + out.set(lidUser, phoneUser); + } catch { + // Skip malformed entries — best-effort. + } + } + return out; +} + +function phoneUserOf(jid: string): string { + return jid.split('@')[0].split(':')[0]; +} + +function main(): void { + const authDir = path.join(process.cwd(), 'store', 'auth'); + const reverse = readReverseMappings(authDir); + + if (reverse.size === 0) { + console.log('SKIPPED:no lid-mapping-*_reverse.json files in store/auth'); + process.exit(0); + } + + // phoneUser → lidJid (the form we'll write to messaging_groups) + const phoneUserToLidJid = new Map(); + for (const [lidUser, phoneUser] of reverse) { + phoneUserToLidJid.set(phoneUser, `${lidUser}@lid`); + } + + const v2DbPath = path.join(DATA_DIR, 'v2.db'); + if (!fs.existsSync(v2DbPath)) { + console.error('FAIL:v2.db not found — run db step first'); + process.exit(1); + } + + const v2Db = initDb(v2DbPath); + runMigrations(v2Db); + + const phoneRows = v2Db + .prepare( + `SELECT id, channel_type, platform_id FROM messaging_groups + WHERE channel_type='whatsapp' AND platform_id LIKE '%@s.whatsapp.net'`, + ) + .all() as RawMessagingGroup[]; + + if (phoneRows.length === 0) { + console.log('SKIPPED:no whatsapp DM messaging_groups to resolve'); + v2Db.close(); + process.exit(0); + } + + // Pull existing wirings so each new alias gets the same agent_group + + // engage rules as the phone-keyed row. + const placeholders = phoneRows.map(() => '?').join(','); + const wiringRows = v2Db + .prepare(`SELECT * FROM messaging_group_agents WHERE messaging_group_id IN (${placeholders})`) + .all(...phoneRows.map((r) => r.id)) as RawWiring[]; + + const wiringsByMg = new Map(); + for (const w of wiringRows) { + const arr = wiringsByMg.get(w.messaging_group_id) ?? []; + arr.push(w); + wiringsByMg.set(w.messaging_group_id, arr); + } + + let resolved = 0; + let aliased = 0; + const createdAt = new Date().toISOString(); + + for (const row of phoneRows) { + const phoneUser = phoneUserOf(row.platform_id); + const lidJid = phoneUserToLidJid.get(phoneUser); + if (!lidJid) continue; + resolved++; + + let lidMg = getMessagingGroupByPlatform('whatsapp', lidJid); + if (!lidMg) { + createMessagingGroup({ + id: generateId('mg'), + channel_type: 'whatsapp', + platform_id: lidJid, + name: null, + is_group: 0, + unknown_sender_policy: 'public', + created_at: createdAt, + }); + lidMg = getMessagingGroupByPlatform('whatsapp', lidJid)!; + } + + const wirings = wiringsByMg.get(row.id) ?? []; + for (const w of wirings) { + if (getMessagingGroupAgentByPair(lidMg.id, w.agent_group_id)) continue; + createMessagingGroupAgent({ + id: generateId('mga'), + messaging_group_id: lidMg.id, + agent_group_id: w.agent_group_id, + engage_mode: w.engage_mode as 'pattern' | 'mention' | 'mention-sticky', + engage_pattern: w.engage_pattern, + sender_scope: w.sender_scope as 'all' | 'admins', + ignored_message_policy: w.ignored_message_policy as 'drop' | 'queue', + session_mode: w.session_mode as 'shared' | 'thread', + priority: w.priority, + created_at: createdAt, + }); + aliased++; + } + } + + v2Db.close(); + console.log( + `OK:reverse_mappings=${reverse.size},phone_dms=${phoneRows.length},lids_resolved=${resolved},aliased=${aliased}`, + ); +} + +main();