CodeIssuesDiscussionsWikiPull RequestsProjectsCommitsActionsReleasesContributorsPulse● GatesSecuritySettingsDeploymentsPipelineInsightsAgents✨ Explain✨ Ask AI✨ Workspace✨ Spec✨ Tests▓ Debt Map✨ NL Search🏛 Archaeology
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | # Cutover Runbook — go fully self-sufficient, drop the GitHub mirror
**Goal:** Gluecron hosts, deploys, and backs up itself. GitHub is removed as
infrastructure. Owner-executed; each step is copy-pasteable on the VPS
(`66.42.121.161`, working tree `/opt/gluecron`).
**Sequencing rule:** durability (backups) goes in BEFORE anything is removed.
Do the phases in order. Do not delete the GitHub repo until Phase 4.
The code side is already merged (self-canonical bootstrap, offsite backup +
restore scripts, docs). What remains is operator config on the box.
---
## Phase 0 — Prerequisites (once)
```bash
# On the VPS, as the deploy user.
cd /opt/gluecron
# You need these on PATH:
which git rclone pg_dump tar curl # install missing: apt-get install -y postgresql-client rclone
# Confirm the live env file exists and holds DATABASE_URL (Neon).
sudo test -f /etc/gluecron.env && echo "env file present"
```
Mint a personal access token for pushing to the canonical remote (admin scope).
Run this OUTSIDE an AI session (it mints live admin creds):
```bash
EMERGENCY_PAT_USER=ccantynz bun run scripts/emergency-pat.ts
# copy the glc_... token it prints
```
---
## Phase 1 — Offsite backup FIRST (the safety net that replaces GitHub)
1. Create a bucket (Cloudflare R2 or S3) and configure an rclone remote:
```bash
rclone config # create a remote, e.g. named "r2", pointing at your bucket
rclone lsd r2: # verify it lists
```
2. Add backup env to `/etc/gluecron.env`:
```bash
sudo tee -a /etc/gluecron.env >/dev/null <<'ENV'
BACKUP_RCLONE_REMOTE=r2:gluecron-backups
RETAIN_DAYS=14
# optional dead-man's-switch (create a check at healthchecks.io):
# HEALTHCHECK_PING_URL=https://hc-ping.com/<uuid>
ENV
```
3. Run one backup by hand and confirm both artifacts land offsite:
```bash
set -a; . /etc/gluecron.env; set +a
bash scripts/backup-offsite.sh
rclone ls r2:gluecron-backups/repos/ # should show a repos-<ts>.tar.gz
rclone ls r2:gluecron-backups/db/ # should show a db-<ts>.dump
```
4. **Prove the restore works** (this is what makes it a backup, not a hope):
```bash
# Pull the newest repos snapshot back and extract to a scratch dir.
rclone copy r2:gluecron-backups/repos/ /tmp/rb/ --max-age 1d
bash scripts/restore.sh repos /tmp/rb/repos-<ts>.tar.gz --dest /tmp/verify
git -C /tmp/verify/repos/ccantynz/Gluecron.com.git log -1 # shows the latest commit
```
5. Install the daily timer:
```bash
sudo tee /etc/systemd/system/gluecron-backup.service >/dev/null <<'UNIT'
[Unit]
Description=Gluecron offsite backup
[Service]
Type=oneshot
EnvironmentFile=/etc/gluecron.env
WorkingDirectory=/opt/gluecron
ExecStart=/usr/bin/env bash scripts/backup-offsite.sh
UNIT
sudo tee /etc/systemd/system/gluecron-backup.timer >/dev/null <<'UNIT'
[Unit]
Description=Daily Gluecron offsite backup
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
[Install]
WantedBy=timers.target
UNIT
sudo systemctl daemon-reload
sudo systemctl enable --now gluecron-backup.timer
systemctl list-timers gluecron-backup.timer
```
**Do not proceed to Phase 3 until at least one restore drill has passed.**
---
## Phase 2 — Confirm self-deploy is truly cord-free
The one fact that can't be checked off-box: the deploy working-tree must pull
from the **local canonical repo**, not GitHub.
```bash
git -C /opt/gluecron remote -v
# origin should be the LOCAL bare repo (…/repos/ccantynz/Gluecron.com.git) or
# https://gluecron.com/ccantynz/Gluecron.com.git — NOT github.com.
# If it points at GitHub, repoint it:
# git -C /opt/gluecron remote set-url origin /opt/gluecron/repos/ccantynz/Gluecron.com.git
# Ensure the self-deploy trigger is armed:
grep -q '^SELF_HOST_REPO=' /etc/gluecron.env || \
echo 'SELF_HOST_REPO=ccantynz/Gluecron.com' | sudo tee -a /etc/gluecron.env
```
Repoint your **local dev clone** at the canonical remote too:
```bash
# On your workstation (this repo):
git remote set-url origin https://x:<glc_token>@gluecron.com/ccantynz/Gluecron.com.git
# or keep origin and add a dedicated remote:
# git remote add gluecron https://x:<glc_token>@gluecron.com/ccantynz/Gluecron.com.git
```
---
## Phase 3 — Deploy drill (push → deploy → rollback)
```bash
# 1. Push the already-merged self-sufficiency commits to the canonical remote.
git push origin main # (or: git push gluecron main)
# 2. Watch it deploy (~25s) — live step stream:
# https://gluecron.com/admin/deploys
# tail -f /var/log/gluecron-self-deploy.log
# 3. Rollback drill: deliberately break a deploy (e.g. push a commit that fails
# the smoke test on a throwaway branch flow) and confirm self-deploy.sh
# auto-reverts via `git reset --hard $PREV_SHA` + restart, and healthz recovers.
```
Green criteria: push deploys in ~25s, `/healthz` returns 200, smoke passes,
and a forced-fail auto-rolls-back to the previous SHA.
---
## Phase 4 — Retire GitHub (only after 7 green days)
Run the platform through Phases 1–3 and let it self-deploy for **7 days** with
no manual intervention. Then:
1. **Freeze** the GitHub repo `ccantynz-alt/Gluecron.com`: Settings → Archive
this repository (makes it read-only). Keep it as a cold archive for 30 days.
2. Neuter the old CI so a stray GitHub push can't deploy: change
`.github/workflows/hetzner-deploy.yml` trigger to `on: workflow_dispatch:`
only (do this via a Gluecron push, since GitHub is no longer canonical).
3. After **30 green days**: delete `ccantynz-alt/Gluecron.com` on GitHub and
delete `.github/workflows/hetzner-deploy.yml` from the repo.
---
## Rollback (if the cutover stumbles in week one)
The GitHub repo is still frozen-not-deleted until Phase 4 completes, so it
remains a last-resort source. But the primary recovery path is now the offsite
backup: `scripts/restore.sh repos <snapshot>` reconstructs every bare repo, and
`scripts/restore.sh db <dump> --target-url <db>` restores metadata. Test both in
Phase 1 so you trust them before you need them.
---
## Post-cutover verification (needs a running instance + PAT — do in a follow-up)
These were deferred from the audit because they need live creds:
- 60/60 MCP tool matrix against `https://gluecron.com/mcp`.
- Playwright journey: register → repo → push → issue → PR → merge → webhook.
- GateTest loop: push → `notifyGateTestOfPush` → results in `gate_runs`.
- Real GitHub-repo import via `/import` end-to-end (the blessed onboarding flow).
|