# Cutover Runbook — go fully self-sufficient, drop the GitHub mirror
**Goal:** Gluecron hosts, deploys, and backs up itself. GitHub is removed as
infrastructure. Owner-executed; each step is copy-pasteable on the VPS
(`66.42.121.161`, working tree `/opt/gluecron`).
**Sequencing rule:** durability (backups) goes in BEFORE anything is removed.
Do the phases in order. Do not delete the GitHub repo until Phase 4.
The code side is already merged (self-canonical bootstrap, offsite backup +
restore scripts, docs). What remains is operator config on the box.
---
## Phase 0 — Prerequisites (once)
```bash
# On the VPS, as the deploy user.
cd /opt/gluecron
# You need these on PATH:
which git rclone pg_dump tar curl # install missing: apt-get install -y postgresql-client rclone
# Confirm the live env file exists and holds DATABASE_URL (Neon).
sudo test -f /etc/gluecron.env && echo "env file present"
```
Mint a personal access token for pushing to the canonical remote (admin scope).
Run this OUTSIDE an AI session (it mints live admin creds):
```bash
EMERGENCY_PAT_USER=ccantynz bun run scripts/emergency-pat.ts
# copy the glc_... token it prints
```
---
## Phase 1 — Offsite backup FIRST (the safety net that replaces GitHub)
1. Create a bucket (Cloudflare R2 or S3) and configure an rclone remote:
```bash
rclone config # create a remote, e.g. named "r2", pointing at your bucket
rclone lsd r2: # verify it lists
```
2. Add backup env to `/etc/gluecron.env`:
```bash
sudo tee -a /etc/gluecron.env >/dev/null <<'ENV'
BACKUP_RCLONE_REMOTE=r2:gluecron-backups
RETAIN_DAYS=14
# optional dead-man's-switch (create a check at healthchecks.io):
# HEALTHCHECK_PING_URL=https://hc-ping.com/<uuid>
ENV
```
3. Run one backup by hand and confirm both artifacts land offsite:
```bash
set -a; . /etc/gluecron.env; set +a
bash scripts/backup-offsite.sh
rclone ls r2:gluecron-backups/repos/ # should show a repos-<ts>.tar.gz
rclone ls r2:gluecron-backups/db/ # should show a db-<ts>.dump
```
4. **Prove the restore works** (this is what makes it a backup, not a hope):
```bash
# Pull the newest repos snapshot back and extract to a scratch dir.
rclone copy r2:gluecron-backups/repos/ /tmp/rb/ --max-age 1d
bash scripts/restore.sh repos /tmp/rb/repos-<ts>.tar.gz --dest /tmp/verify
git -C /tmp/verify/repos/ccantynz/Gluecron.com.git log -1 # shows the latest commit
```
5. Install the daily timer:
```bash
sudo tee /etc/systemd/system/gluecron-backup.service >/dev/null <<'UNIT'
[Unit]
Description=Gluecron offsite backup
[Service]
Type=oneshot
EnvironmentFile=/etc/gluecron.env
WorkingDirectory=/opt/gluecron
ExecStart=/usr/bin/env bash scripts/backup-offsite.sh
UNIT
sudo tee /etc/systemd/system/gluecron-backup.timer >/dev/null <<'UNIT'
[Unit]
Description=Daily Gluecron offsite backup
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
[Install]
WantedBy=timers.target
UNIT
sudo systemctl daemon-reload
sudo systemctl enable --now gluecron-backup.timer
systemctl list-timers gluecron-backup.timer
```
**Do not proceed to Phase 3 until at least one restore drill has passed.**
---
## Phase 2 — Confirm self-deploy is truly cord-free
The one fact that can't be checked off-box: the deploy working-tree must pull
from the **local canonical repo**, not GitHub.
```bash
git -C /opt/gluecron remote -v
# origin should be the LOCAL bare repo (…/repos/ccantynz/Gluecron.com.git) or
# https://gluecron.com/ccantynz/Gluecron.com.git — NOT github.com.
# If it points at GitHub, repoint it:
# git -C /opt/gluecron remote set-url origin /opt/gluecron/repos/ccantynz/Gluecron.com.git
# Ensure the self-deploy trigger is armed:
grep -q '^SELF_HOST_REPO=' /etc/gluecron.env || \
echo 'SELF_HOST_REPO=ccantynz/Gluecron.com' | sudo tee -a /etc/gluecron.env
```
Repoint your **local dev clone** at the canonical remote too:
```bash
# On your workstation (this repo):
git remote set-url origin https://x:<glc_token>@gluecron.com/ccantynz/Gluecron.com.git
# or keep origin and add a dedicated remote:
# git remote add gluecron https://x:<glc_token>@gluecron.com/ccantynz/Gluecron.com.git
```
---
## Phase 3 — Deploy drill (push → deploy → rollback)
```bash
# 1. Push the already-merged self-sufficiency commits to the canonical remote.
git push origin main # (or: git push gluecron main)
# 2. Watch it deploy (~25s) — live step stream:
# https://gluecron.com/admin/deploys
# tail -f /var/log/gluecron-self-deploy.log
# 3. Rollback drill: deliberately break a deploy (e.g. push a commit that fails
# the smoke test on a throwaway branch flow) and confirm self-deploy.sh
# auto-reverts via `git reset --hard $PREV_SHA` + restart, and healthz recovers.
```
Green criteria: push deploys in ~25s, `/healthz` returns 200, smoke passes,
and a forced-fail auto-rolls-back to the previous SHA.
---
## Phase 4 — Retire GitHub (only after 7 green days)
Run the platform through Phases 1–3 and let it self-deploy for **7 days** with
no manual intervention. Then:
1. **Freeze** the GitHub repo `ccantynz-alt/Gluecron.com`: Settings → Archive
this repository (makes it read-only). Keep it as a cold archive for 30 days.
2. Neuter the old CI so a stray GitHub push can't deploy: change
`.github/workflows/hetzner-deploy.yml` trigger to `on: workflow_dispatch:`
only (do this via a Gluecron push, since GitHub is no longer canonical).
3. After **30 green days**: delete `ccantynz-alt/Gluecron.com` on GitHub and
delete `.github/workflows/hetzner-deploy.yml` from the repo.
---
## Rollback (if the cutover stumbles in week one)
The GitHub repo is still frozen-not-deleted until Phase 4 completes, so it
remains a last-resort source. But the primary recovery path is now the offsite
backup: `scripts/restore.sh repos <snapshot>` reconstructs every bare repo, and
`scripts/restore.sh db <dump> --target-url <db>` restores metadata. Test both in
Phase 1 so you trust them before you need them.
---
## Post-cutover verification (needs a running instance + PAT — do in a follow-up)
These were deferred from the audit because they need live creds:
- 60/60 MCP tool matrix against `https://gluecron.com/mcp`.
- Playwright journey: register → repo → push → issue → PR → merge → webhook.
- GateTest loop: push → `notifyGateTestOfPush` → results in `gate_runs`.
- Real GitHub-repo import via `/import` end-to-end (the blessed onboarding flow).
|