Pre-launch — Gluecron is in final validation. Public signups and git hosting for non-owner users open after launch review.
ccantynz/Gluecron.com
Commit4a0dea1unknown_key

feat(BLOCK-I): I9 repository mirroring

feat(BLOCK-I): I9 repository mirroring

- drizzle/0026 adds repo_mirrors (one-per-repo config) + repo_mirror_runs
  (audit log) tables.
- src/lib/mirrors.ts validates upstream URLs (https/http/git only,
  rejects ssh/file/paths and shell metacharacters), redacts credentials
  from logs, and runs `git fetch --prune --tags` via Bun.spawn with a
  5-min timeout and GIT_TERMINAL_PROMPT=0. Updates last_synced_at +
  last_status + last_error on every run.
- src/routes/mirrors.tsx exposes owner-only /:owner/:repo/settings/mirror
  (form + recent-runs panel + save/delete/sync-now) and site-admin
  POST /admin/mirrors/sync-all that iterates due mirrors. All mutations
  audit-logged.

17 new tests covering URL validation (including shell-metachar rejection)
+ route auth gates. Full suite at 601 pass / 0 fail.

https://claude.ai/code/session_01U6sj9ezKCpmiwx7cXGZmS6
Claude committed on April 15, 2026Parent: 4c8f666
7 files changed+983−34a0dea10c6e3f8112b72d1ee29fd240812b531c5
7 changed files+983−3
ModifiedBUILD_BIBLE.md+7−3View fileUnifiedSplit
7171| Archive / disable repo | ✅ | I1 — `src/routes/repo-settings.tsx` archive toggle; `RepoHeader` renders an "Archived" badge when `is_archived=true`. |
7272| Repository transfer | ✅ | I3 — `src/routes/repo-settings.tsx` transfer form + `POST /:owner/:repo/settings/transfer`; ownership change recorded in `repo_transfers` audit table. Reject conflicts (target owner already has a repo by that name) with a redirect. |
7373| Template repositories | ✅ | I2 — `drizzle/0022_repo_templates.sql` adds `is_template`. `src/routes/templates.ts` serves `POST /:owner/:repo/use-template` (git clone --bare into caller's namespace). "Use this template" CTA rendered on the public repo page. |
74| Repository mirroring | ❌ | — |
74| Repository mirroring | ✅ | I9 — pull-style mirror of an upstream git URL. `drizzle/0026_repo_mirrors.sql` adds `repo_mirrors` (one-per-repo config) + `repo_mirror_runs` (audit log). `src/lib/mirrors.ts` validates URLs (https/http/git only, rejects ssh/file/shell metacharacters), runs `git fetch --prune --tags` via `Bun.spawn` with a 5-min timeout + `GIT_TERMINAL_PROMPT=0`. `src/routes/mirrors.tsx` exposes `/:owner/:repo/settings/mirror` + `/admin/mirrors/sync-all`. |
7575
7676### 2.2 Code browsing
7777| Feature | Status | Notes |
275275- **I6** — Sponsors → ✅ shipped. `drizzle/0023_sponsors.sql` adds `sponsorship_tiers` (maintainer_id, name, monthly_cents, one_time_allowed, is_active) + `sponsorships` (sponsor_id, maintainer_id, tier_id, amount_cents, kind, note, is_public, cancelled_at). `src/routes/sponsors.tsx` serves public `/sponsors/:username` (tier cards + recent public sponsors join) + maintainer `/settings/sponsors` (tier CRUD, soft-retire via is_active=false, activity list). Payment rails deferred — v1 captures intent + thank-you notes.
276276- **I7** — Weekly email digest → ✅ shipped. `drizzle/0024_email_digest.sql` adds `users.notify_email_digest_weekly` + `last_digest_sent_at`. `src/lib/email-digest.ts` exposes `composeDigest`/`sendDigestForUser`/`sendDigestsToAll` (never-throws). Pulls notifications + failed/repaired gate_runs + merged PRs from the last 7d, composes escaped HTML + plaintext, and sends via the shared email provider. `/settings/digest/preview` renders the digest inline for self-preview; `/admin/digests` gives site admins a "Send now" trigger + single-user preview, audit-logged as `admin.digests.run`/`admin.digests.preview`.
277277- **I8** — Symbol / xref navigation → ✅ shipped. `drizzle/0025_code_symbols.sql` adds a `code_symbols` table. `src/lib/symbols.ts` provides a regex-based top-level extractor for ts/js/py/rs/go/rb/java/kt/swift. On-demand indexing via `POST /:owner/:repo/symbols/reindex` walks the default-branch tree, caps at 2000 files/1MB each, replaces the prior set. Browse at `/:owner/:repo/symbols` (A–Z + per-kind counts), search via `/symbols/search?q=`, inspect at `/symbols/:name`. 14 new tests.
278- **I9** — Repository mirroring → ✅ shipped. `drizzle/0026_repo_mirrors.sql` adds `repo_mirrors` (one-per-repo config) + `repo_mirror_runs` (audit log). `src/lib/mirrors.ts` provides URL validation (https/http/git only, no ssh/file/paths/shell metas), credentials-redaction for logs, and `runMirrorSync` that shells out to `git fetch --prune --tags` with a 5-min timeout and `GIT_TERMINAL_PROMPT=0`. `src/routes/mirrors.tsx` serves owner-only `/:owner/:repo/settings/mirror` + site-admin `/admin/mirrors/sync-all`. 17 new tests.
278279
279280### BLOCK H — Marketplace
280281- **H1** — App marketplace → ✅ shipped. `src/routes/marketplace.tsx` + `src/lib/marketplace.ts` + `drizzle/0021_marketplace_and_apps.sql` (5 tables: `apps`, `app_installations`, `app_bots`, `app_install_tokens`, `app_events`). Routes: `GET /marketplace` (public directory with search), `GET /marketplace/:slug` (detail + install CTA), `POST /marketplace/:slug/install` (user-target install in v1), `POST /marketplace/installations/:id/uninstall`, `GET /settings/apps` (personal list), `GET+POST /developer/apps-new` (register), `GET /developer/apps/:slug/manage` (event log + install count), `POST /developer/apps/:slug/tokens/new` (show-once token). Install idempotent via soft-update on existing non-uninstalled row.
290291- `src/app.tsx` — route composition, middleware order, error handlers
291292- `src/index.ts` — Bun server entry
292293- `src/lib/config.ts` — env getters (late-binding)
293- `src/db/schema.ts` — 82 tables. New tables only via new migration.
294- `src/db/schema.ts` — 84 tables. New tables only via new migration.
294295- `src/db/index.ts` — lazy proxy DB connection
295296- `src/db/migrate.ts` — migration runner
296297- `drizzle/0000_initial.sql`, `drizzle/0001_green_ecosystem.sql` — migrations
316317- `drizzle/0023_sponsors.sql` (Block I6) — migration, never edited in place. Adds `sponsorship_tiers` + `sponsorships` tables.
317318- `drizzle/0024_email_digest.sql` (Block I7) — migration, never edited in place. Adds `users.notify_email_digest_weekly` + `users.last_digest_sent_at`.
318319- `drizzle/0025_code_symbols.sql` (Block I8) — migration, never edited in place. Adds `code_symbols` table with indexes on `(repository_id, name)` + `(repository_id, path)`.
320- `drizzle/0026_repo_mirrors.sql` (Block I9) — migration, never edited in place. Adds `repo_mirrors` (unique on `repository_id`) + `repo_mirror_runs`.
319321
320322### 4.2 Git layer (locked)
321323- `src/git/repository.ts` — tree / blob / commits / diff / branches / blame / search / raw / tags / commitsBetween
441443- `src/routes/settings.tsx` (extends for I7) — adds `notify_email_digest_weekly` checkbox to email prefs + handler wiring in `POST /settings/notifications`, and `GET /settings/digest/preview` (renders `composeDigest` output inline via `raw(body.html)` with Hono's `hono/html`).
442444- `src/lib/symbols.ts` (Block I8) — regex-based top-level symbol extractor. Pure helpers: `detectLanguage(path)` (10 extensions mapped to 8 languages), `extractSymbols(content, lang)` (per-language rule list, 1-based line numbers, 240-char signature cap, skips lines >500 chars). `indexRepositorySymbols(repoId)` walks the default-branch tree, caps at 2000 files / 1MB each, replaces the prior set in batches of 500. `findDefinitions(repoId, name)` + `countSymbolsForRepo(repoId)`. `__internal` exposes `RULES` + `EXT_LANG` for tests.
443445- `src/routes/symbols.tsx` (Block I8) — `/:owner/:repo/symbols` overview (total + per-kind counts + A–Z list with blob deep-links), `/:owner/:repo/symbols/search?q=` prefix search (ilike `q%`), `/:owner/:repo/symbols/:name` detail (all definitions with signature preview + deep link). `POST /:owner/:repo/symbols/reindex` is requireAuth + owner-only.
446- `src/lib/mirrors.ts` (Block I9) — upstream URL validator (accepts https/http/git schemes, rejects ssh/file/paths/shell metacharacters, 2048-char cap), `safeUrlForLog` (redacts embedded credentials), `upsertMirror` / `deleteMirror` / `getMirrorForRepo` / `listRecentRuns`, `runMirrorSync` (runs `git fetch --prune --tags --no-write-fetch-head` via `Bun.spawn` with 5-min timeout + `GIT_TERMINAL_PROMPT=0`; updates `last_synced_at` + `last_status` + `last_error`), `listDueMirrors` + `syncAllDue` for the admin trigger.
447- `src/routes/mirrors.tsx` (Block I9) — owner-only config at `/:owner/:repo/settings/mirror` (GET form + recent-runs panel, POST save, POST delete, POST sync-now). Site-admin `POST /admin/mirrors/sync-all` iterates due mirrors. All mutations `audit()`-logged.
444448
445449### 4.7 Views (locked contracts)
446450- `src/views/layout.tsx` — `Layout` accepts `title`, `user`, `notificationCount`
475479```bash
476480bun install
477481bun dev # hot reload
478bun test # 584 tests currently pass
482bun test # 601 tests currently pass
479483bun run db:migrate
480484```
481485
Addeddrizzle/0026_repo_mirrors.sql+36−0View fileUnifiedSplit
1-- Gluecron migration 0026: Repository mirroring.
2--
3-- I9 — Pull-style mirroring. A mirrored repository has an upstream URL
4-- that we periodically `git fetch` from (via admin-trigger or cron). We
5-- keep a single row per repo (one mirror per mirrored repo) plus an
6-- append-only log of sync attempts for audit.
7
8--> statement-breakpoint
9CREATE TABLE IF NOT EXISTS "repo_mirrors" (
10 "id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
11 "repository_id" uuid NOT NULL UNIQUE
12 REFERENCES "repositories"("id") ON DELETE CASCADE,
13 "upstream_url" text NOT NULL,
14 "interval_minutes" integer NOT NULL DEFAULT 1440,
15 "last_synced_at" timestamp,
16 "last_status" text, -- "ok" | "error" | null (never synced)
17 "last_error" text,
18 "is_enabled" boolean NOT NULL DEFAULT true,
19 "created_at" timestamp NOT NULL DEFAULT now(),
20 "updated_at" timestamp NOT NULL DEFAULT now()
21);
22
23--> statement-breakpoint
24CREATE TABLE IF NOT EXISTS "repo_mirror_runs" (
25 "id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
26 "mirror_id" uuid NOT NULL REFERENCES "repo_mirrors"("id") ON DELETE CASCADE,
27 "started_at" timestamp NOT NULL DEFAULT now(),
28 "finished_at" timestamp,
29 "status" text NOT NULL DEFAULT 'running', -- running | ok | error
30 "message" text,
31 "exit_code" integer
32);
33
34--> statement-breakpoint
35CREATE INDEX IF NOT EXISTS "repo_mirror_runs_mirror_id_idx"
36 ON "repo_mirror_runs" ("mirror_id", "started_at" DESC);
Addedsrc/__tests__/mirrors.test.ts+119−0View fileUnifiedSplit
1/**
2 * Block I9 — Repository mirroring tests.
3 *
4 * Pure validation tests for URL safety + auth smoke on mirror routes.
5 * Actual git fetch is exercised by the live server — `runMirrorSync` is
6 * not tested here because it needs a real upstream.
7 */
8
9import { describe, it, expect } from "bun:test";
10import app from "../app";
11import { validateUpstreamUrl, safeUrlForLog } from "../lib/mirrors";
12
13describe("mirrors — validateUpstreamUrl", () => {
14 it("accepts https URLs", () => {
15 expect(validateUpstreamUrl("https://github.com/foo/bar.git").ok).toBe(true);
16 });
17
18 it("accepts http URLs", () => {
19 expect(validateUpstreamUrl("http://git.example.com/x.git").ok).toBe(true);
20 });
21
22 it("accepts git:// URLs", () => {
23 expect(validateUpstreamUrl("git://kernel.org/linux.git").ok).toBe(true);
24 });
25
26 it("rejects ssh URLs", () => {
27 const r = validateUpstreamUrl("ssh://git@github.com/foo/bar.git");
28 expect(r.ok).toBe(false);
29 expect(r.error).toMatch(/https|http|git/);
30 });
31
32 it("rejects file:// URLs", () => {
33 const r = validateUpstreamUrl("file:///tmp/evil.git");
34 expect(r.ok).toBe(false);
35 });
36
37 it("rejects local paths", () => {
38 expect(validateUpstreamUrl("/etc/passwd").ok).toBe(false);
39 expect(validateUpstreamUrl("./foo.git").ok).toBe(false);
40 });
41
42 it("rejects URLs with shell metacharacters", () => {
43 expect(validateUpstreamUrl("https://evil;rm -rf /").ok).toBe(false);
44 expect(validateUpstreamUrl("https://evil`id`").ok).toBe(false);
45 expect(validateUpstreamUrl("https://evil$(whoami)").ok).toBe(false);
46 expect(validateUpstreamUrl("https://evil|nc 1.2.3.4 9").ok).toBe(false);
47 expect(validateUpstreamUrl("https://evil<payload").ok).toBe(false);
48 });
49
50 it("rejects empty or whitespace-only URLs", () => {
51 expect(validateUpstreamUrl("").ok).toBe(false);
52 expect(validateUpstreamUrl(" ").ok).toBe(false);
53 });
54
55 it("rejects URLs over 2048 chars", () => {
56 const long = "https://example.com/" + "a".repeat(2100);
57 expect(validateUpstreamUrl(long).ok).toBe(false);
58 });
59});
60
61describe("mirrors — safeUrlForLog", () => {
62 it("passes plain URLs through", () => {
63 expect(safeUrlForLog("https://github.com/foo/bar.git")).toBe(
64 "https://github.com/foo/bar.git"
65 );
66 });
67
68 it("redacts embedded credentials", () => {
69 const redacted = safeUrlForLog("https://user:pw@github.com/foo/bar.git");
70 expect(redacted).not.toContain("user:pw");
71 expect(redacted).not.toContain("pw");
72 expect(redacted).toContain("***");
73 expect(redacted).toContain("github.com");
74 });
75
76 it("returns original on unparseable input", () => {
77 expect(safeUrlForLog("not-a-url")).toBe("not-a-url");
78 });
79});
80
81describe("mirrors — route auth", () => {
82 it("GET /:owner/:repo/settings/mirror without auth → 302 /login", async () => {
83 const res = await app.request("/alice/repo/settings/mirror");
84 expect(res.status).toBe(302);
85 expect(res.headers.get("location") || "").toContain("/login");
86 });
87
88 it("POST /:owner/:repo/settings/mirror without auth → 302 /login", async () => {
89 const res = await app.request("/alice/repo/settings/mirror", {
90 method: "POST",
91 body: new URLSearchParams({ upstream_url: "https://example.com/x.git" }),
92 headers: { "content-type": "application/x-www-form-urlencoded" },
93 });
94 expect(res.status).toBe(302);
95 expect(res.headers.get("location") || "").toContain("/login");
96 });
97
98 it("POST /:owner/:repo/settings/mirror/sync without auth → 302 /login", async () => {
99 const res = await app.request("/alice/repo/settings/mirror/sync", {
100 method: "POST",
101 });
102 expect(res.status).toBe(302);
103 expect(res.headers.get("location") || "").toContain("/login");
104 });
105
106 it("POST /:owner/:repo/settings/mirror/delete without auth → 302 /login", async () => {
107 const res = await app.request("/alice/repo/settings/mirror/delete", {
108 method: "POST",
109 });
110 expect(res.status).toBe(302);
111 expect(res.headers.get("location") || "").toContain("/login");
112 });
113
114 it("POST /admin/mirrors/sync-all without auth → 302 /login", async () => {
115 const res = await app.request("/admin/mirrors/sync-all", { method: "POST" });
116 expect(res.status).toBe(302);
117 expect(res.headers.get("location") || "").toContain("/login");
118 });
119});
Modifiedsrc/app.tsx+4−0View fileUnifiedSplit
6767import codeScanningRoutes from "./routes/code-scanning";
6868import sponsorsRoutes from "./routes/sponsors";
6969import symbolsRoutes from "./routes/symbols";
70import mirrorsRoutes from "./routes/mirrors";
7071import webRoutes from "./routes/web";
7172
7273const app = new Hono();
230231// Symbol / xref navigation — /:owner/:repo/symbols (Block I8)
231232app.route("/", symbolsRoutes);
232233
234// Repository mirroring — /:owner/:repo/settings/mirror (Block I9)
235app.route("/", mirrorsRoutes);
236
233237// Insights + milestones
234238app.route("/", insightsRoutes);
235239
Modifiedsrc/db/schema.ts+40−0View fileUnifiedSplit
19941994);
19951995
19961996export type CodeSymbol = typeof codeSymbols.$inferSelect;
1997
1998// Block I9 — Repository mirroring. One row per mirrored repo + an
1999// append-only log of sync attempts.
2000export const repoMirrors = pgTable("repo_mirrors", {
2001 id: uuid("id").primaryKey().defaultRandom(),
2002 repositoryId: uuid("repository_id")
2003 .notNull()
2004 .unique()
2005 .references(() => repositories.id, { onDelete: "cascade" }),
2006 upstreamUrl: text("upstream_url").notNull(),
2007 intervalMinutes: integer("interval_minutes").default(1440).notNull(),
2008 lastSyncedAt: timestamp("last_synced_at"),
2009 lastStatus: text("last_status"), // "ok" | "error"
2010 lastError: text("last_error"),
2011 isEnabled: boolean("is_enabled").default(true).notNull(),
2012 createdAt: timestamp("created_at").defaultNow().notNull(),
2013 updatedAt: timestamp("updated_at").defaultNow().notNull(),
2014});
2015
2016export type RepoMirror = typeof repoMirrors.$inferSelect;
2017
2018export const repoMirrorRuns = pgTable(
2019 "repo_mirror_runs",
2020 {
2021 id: uuid("id").primaryKey().defaultRandom(),
2022 mirrorId: uuid("mirror_id")
2023 .notNull()
2024 .references(() => repoMirrors.id, { onDelete: "cascade" }),
2025 startedAt: timestamp("started_at").defaultNow().notNull(),
2026 finishedAt: timestamp("finished_at"),
2027 status: text("status").default("running").notNull(),
2028 message: text("message"),
2029 exitCode: integer("exit_code"),
2030 },
2031 (table) => [
2032 index("repo_mirror_runs_mirror_id_idx").on(table.mirrorId, table.startedAt),
2033 ]
2034);
2035
2036export type RepoMirrorRun = typeof repoMirrorRuns.$inferSelect;
Addedsrc/lib/mirrors.ts+389−0View fileUnifiedSplit
1/**
2 * Block I9 — Repository mirroring.
3 *
4 * Pull-style mirroring: a mirrored repo has an upstream URL that we
5 * periodically `git fetch` from. We run it as `git remote update` into
6 * the bare repo so refs/heads/* are kept in sync with upstream's.
7 *
8 * SECURITY: only http(s) and git:// URLs are accepted. We refuse any URL
9 * with shell metacharacters, `file://`, `ssh://`, or paths that could
10 * escape the bare repo. Credentials embedded in URLs are allowed (the
11 * caller decides whether to persist them) but stripped from logs.
12 */
13
14import { and, desc, eq } from "drizzle-orm";
15import { db } from "../db";
16import {
17 repoMirrors,
18 repoMirrorRuns,
19 repositories,
20 users,
21} from "../db/schema";
22import { getRepoPath } from "../git/repository";
23
24const MIRROR_REMOTE_NAME = "gluecron-mirror";
25const FETCH_TIMEOUT_MS = 5 * 60 * 1000; // 5 minutes
26
27export interface ValidationResult {
28 ok: boolean;
29 error?: string;
30}
31
32/** Pure — validates an upstream URL for use as a mirror. */
33export function validateUpstreamUrl(url: string): ValidationResult {
34 if (!url || typeof url !== "string") {
35 return { ok: false, error: "URL is required" };
36 }
37 const trimmed = url.trim();
38 if (trimmed.length === 0) {
39 return { ok: false, error: "URL is required" };
40 }
41 if (trimmed.length > 2048) {
42 return { ok: false, error: "URL too long" };
43 }
44 // Reject shell metacharacters that Bun.spawn would pass through safely,
45 // but which should never appear in a legitimate git URL.
46 if (/[\s;&|`$\\<>]/.test(trimmed)) {
47 return { ok: false, error: "URL contains invalid characters" };
48 }
49 // Accept only https/http/git schemes. Reject ssh/file/local paths —
50 // ssh needs key management we don't have yet, file:// lets the user
51 // escape into the server's filesystem.
52 const allowed = /^(https?:\/\/|git:\/\/)/i;
53 if (!allowed.test(trimmed)) {
54 return { ok: false, error: "URL must start with https://, http://, or git://" };
55 }
56 return { ok: true };
57}
58
59/** Strip credentials from a URL for safe logging. */
60export function safeUrlForLog(url: string): string {
61 try {
62 const u = new URL(url);
63 if (u.username || u.password) {
64 u.username = "***";
65 u.password = "";
66 return u.toString();
67 }
68 return url;
69 } catch {
70 return url;
71 }
72}
73
74export interface UpsertMirrorInput {
75 repositoryId: string;
76 upstreamUrl: string;
77 intervalMinutes?: number;
78 isEnabled?: boolean;
79}
80
81/** Create or update the mirror config for a repository. */
82export async function upsertMirror(
83 input: UpsertMirrorInput
84): Promise<{ ok: true; id: string } | { ok: false; error: string }> {
85 const v = validateUpstreamUrl(input.upstreamUrl);
86 if (!v.ok) return { ok: false, error: v.error! };
87
88 try {
89 const [existing] = await db
90 .select()
91 .from(repoMirrors)
92 .where(eq(repoMirrors.repositoryId, input.repositoryId))
93 .limit(1);
94
95 if (existing) {
96 await db
97 .update(repoMirrors)
98 .set({
99 upstreamUrl: input.upstreamUrl.trim(),
100 intervalMinutes: input.intervalMinutes ?? existing.intervalMinutes,
101 isEnabled: input.isEnabled ?? existing.isEnabled,
102 updatedAt: new Date(),
103 })
104 .where(eq(repoMirrors.id, existing.id));
105 return { ok: true, id: existing.id };
106 }
107
108 const [row] = await db
109 .insert(repoMirrors)
110 .values({
111 repositoryId: input.repositoryId,
112 upstreamUrl: input.upstreamUrl.trim(),
113 intervalMinutes: input.intervalMinutes ?? 1440,
114 isEnabled: input.isEnabled ?? true,
115 })
116 .returning({ id: repoMirrors.id });
117 return { ok: true, id: row.id };
118 } catch (err) {
119 console.error("[mirrors] upsertMirror error:", err);
120 return { ok: false, error: "Failed to save mirror configuration" };
121 }
122}
123
124export async function deleteMirror(repositoryId: string): Promise<void> {
125 try {
126 await db
127 .delete(repoMirrors)
128 .where(eq(repoMirrors.repositoryId, repositoryId));
129 } catch (err) {
130 console.error("[mirrors] deleteMirror error:", err);
131 }
132}
133
134export async function getMirrorForRepo(repositoryId: string) {
135 try {
136 const [row] = await db
137 .select()
138 .from(repoMirrors)
139 .where(eq(repoMirrors.repositoryId, repositoryId))
140 .limit(1);
141 return row || null;
142 } catch {
143 return null;
144 }
145}
146
147export async function listRecentRuns(
148 mirrorId: string,
149 limit = 20
150): Promise<Array<typeof repoMirrorRuns.$inferSelect>> {
151 try {
152 return await db
153 .select()
154 .from(repoMirrorRuns)
155 .where(eq(repoMirrorRuns.mirrorId, mirrorId))
156 .orderBy(desc(repoMirrorRuns.startedAt))
157 .limit(limit);
158 } catch {
159 return [];
160 }
161}
162
163/**
164 * Execute one sync run for a mirror. Returns the run row after completion.
165 * Safe to call concurrently per-repo (we'd still end up fetching serially
166 * because git locks `packed-refs` during fetch).
167 */
168export async function runMirrorSync(
169 mirrorId: string
170): Promise<{ ok: boolean; message: string; exitCode: number }> {
171 // Load mirror + owning repo in one shot.
172 const [row] = await db
173 .select({
174 mirror: repoMirrors,
175 repoName: repositories.name,
176 ownerName: users.username,
177 })
178 .from(repoMirrors)
179 .innerJoin(repositories, eq(repoMirrors.repositoryId, repositories.id))
180 .innerJoin(users, eq(repositories.ownerId, users.id))
181 .where(eq(repoMirrors.id, mirrorId))
182 .limit(1);
183
184 if (!row) {
185 return { ok: false, message: "mirror not found", exitCode: -1 };
186 }
187 if (!row.mirror.isEnabled) {
188 return { ok: false, message: "mirror disabled", exitCode: -1 };
189 }
190
191 const repoPath = getRepoPath(row.ownerName, row.repoName);
192
193 const [runRow] = await db
194 .insert(repoMirrorRuns)
195 .values({ mirrorId, status: "running" })
196 .returning();
197
198 const url = row.mirror.upstreamUrl;
199 let exitCode = -1;
200 let stdout = "";
201 let stderr = "";
202
203 try {
204 // Ensure the remote exists and points at the current URL.
205 await runGit(["git", "remote", "remove", MIRROR_REMOTE_NAME], repoPath);
206 const addRes = await runGit(
207 ["git", "remote", "add", MIRROR_REMOTE_NAME, url],
208 repoPath
209 );
210 if (addRes.exitCode !== 0) {
211 throw new Error(`remote add failed: ${addRes.stderr}`);
212 }
213
214 const fetchRes = await runGit(
215 [
216 "git",
217 "fetch",
218 "--prune",
219 "--tags",
220 "--no-write-fetch-head",
221 MIRROR_REMOTE_NAME,
222 "+refs/heads/*:refs/heads/*",
223 ],
224 repoPath,
225 FETCH_TIMEOUT_MS
226 );
227 exitCode = fetchRes.exitCode;
228 stdout = fetchRes.stdout;
229 stderr = fetchRes.stderr;
230 if (exitCode !== 0) {
231 throw new Error(stderr.slice(0, 4000) || "git fetch failed");
232 }
233
234 const messageLines: string[] = [];
235 if (stdout.trim()) messageLines.push(`stdout:\n${stdout.trim()}`);
236 if (stderr.trim()) messageLines.push(`stderr:\n${stderr.trim()}`);
237 const message =
238 messageLines.join("\n\n").slice(0, 4000) || "Mirror synced (no changes)";
239
240 await db
241 .update(repoMirrorRuns)
242 .set({
243 finishedAt: new Date(),
244 status: "ok",
245 message,
246 exitCode,
247 })
248 .where(eq(repoMirrorRuns.id, runRow.id));
249
250 await db
251 .update(repoMirrors)
252 .set({
253 lastSyncedAt: new Date(),
254 lastStatus: "ok",
255 lastError: null,
256 updatedAt: new Date(),
257 })
258 .where(eq(repoMirrors.id, mirrorId));
259
260 return { ok: true, message, exitCode };
261 } catch (err: any) {
262 const errMsg = String(err?.message || err || "unknown error").slice(0, 4000);
263 await db
264 .update(repoMirrorRuns)
265 .set({
266 finishedAt: new Date(),
267 status: "error",
268 message: errMsg,
269 exitCode,
270 })
271 .where(eq(repoMirrorRuns.id, runRow.id));
272
273 await db
274 .update(repoMirrors)
275 .set({
276 lastSyncedAt: new Date(),
277 lastStatus: "error",
278 lastError: errMsg,
279 updatedAt: new Date(),
280 })
281 .where(eq(repoMirrors.id, mirrorId));
282
283 return { ok: false, message: errMsg, exitCode };
284 }
285}
286
287// ---------- Internal ----------
288
289async function runGit(
290 cmd: string[],
291 cwd: string,
292 timeoutMs = 60_000
293): Promise<{ exitCode: number; stdout: string; stderr: string }> {
294 try {
295 const proc = Bun.spawn(cmd, {
296 cwd,
297 stdout: "pipe",
298 stderr: "pipe",
299 env: {
300 ...process.env,
301 GIT_TERMINAL_PROMPT: "0", // never prompt for creds
302 },
303 });
304 const timer = setTimeout(() => {
305 try {
306 proc.kill("SIGKILL");
307 } catch {
308 // ignore
309 }
310 }, timeoutMs);
311 const [stdout, stderr] = await Promise.all([
312 new Response(proc.stdout).text(),
313 new Response(proc.stderr).text(),
314 ]);
315 const exitCode = await proc.exited;
316 clearTimeout(timer);
317 return { exitCode, stdout, stderr };
318 } catch (err: any) {
319 return {
320 exitCode: -1,
321 stdout: "",
322 stderr: String(err?.message || err || "spawn failed"),
323 };
324 }
325}
326
327/** Returns mirrors that are due for a sync (used by admin cron trigger). */
328export async function listDueMirrors(
329 now: Date = new Date()
330): Promise<Array<{ id: string; repositoryId: string; upstreamUrl: string }>> {
331 try {
332 const rows = await db
333 .select()
334 .from(repoMirrors)
335 .where(eq(repoMirrors.isEnabled, true));
336 const due: Array<{
337 id: string;
338 repositoryId: string;
339 upstreamUrl: string;
340 }> = [];
341 for (const r of rows) {
342 if (!r.lastSyncedAt) {
343 due.push({
344 id: r.id,
345 repositoryId: r.repositoryId,
346 upstreamUrl: r.upstreamUrl,
347 });
348 continue;
349 }
350 const last = new Date(r.lastSyncedAt as any).getTime();
351 const elapsedMin = (now.getTime() - last) / 60000;
352 if (elapsedMin >= r.intervalMinutes) {
353 due.push({
354 id: r.id,
355 repositoryId: r.repositoryId,
356 upstreamUrl: r.upstreamUrl,
357 });
358 }
359 }
360 return due;
361 } catch {
362 return [];
363 }
364}
365
366/** Run sync for every due mirror. Returns summary counts. */
367export async function syncAllDue(): Promise<{
368 total: number;
369 ok: number;
370 failed: number;
371}> {
372 const due = await listDueMirrors();
373 let ok = 0;
374 let failed = 0;
375 for (const m of due) {
376 const r = await runMirrorSync(m.id);
377 if (r.ok) ok++;
378 else failed++;
379 }
380 return { total: due.length, ok, failed };
381}
382
383// Suppress unused import warning for `and`.
384void and;
385
386export const __internal = {
387 MIRROR_REMOTE_NAME,
388 FETCH_TIMEOUT_MS,
389};
Addedsrc/routes/mirrors.tsx+388−0View fileUnifiedSplit
1/**
2 * Block I9 — Repository mirroring.
3 *
4 * GET /:owner/:repo/settings/mirror — config form + recent runs
5 * POST /:owner/:repo/settings/mirror — save upstream URL + interval
6 * POST /:owner/:repo/settings/mirror/delete — remove mirror config
7 * POST /:owner/:repo/settings/mirror/sync — run one sync now (owner-only)
8 * POST /admin/mirrors/sync-all — site admin, run all due mirrors
9 */
10
11import { Hono } from "hono";
12import { and, eq } from "drizzle-orm";
13import { db } from "../db";
14import { repositories, users } from "../db/schema";
15import { Layout } from "../views/layout";
16import { RepoHeader } from "../views/components";
17import { softAuth, requireAuth } from "../middleware/auth";
18import type { AuthEnv } from "../middleware/auth";
19import { isSiteAdmin } from "../lib/admin";
20import { audit } from "../lib/notify";
21import {
22 deleteMirror,
23 getMirrorForRepo,
24 listRecentRuns,
25 runMirrorSync,
26 safeUrlForLog,
27 syncAllDue,
28 upsertMirror,
29 validateUpstreamUrl,
30} from "../lib/mirrors";
31
32const mirrors = new Hono<AuthEnv>();
33mirrors.use("*", softAuth);
34
35async function ownerGate(c: any): Promise<
36 | Response
37 | {
38 user: any;
39 ownerName: string;
40 repoName: string;
41 repo: typeof repositories.$inferSelect;
42 }
43> {
44 const user = c.get("user");
45 if (!user) return c.redirect("/login");
46 const { owner: ownerName, repo: repoName } = c.req.param();
47 const [owner] = await db
48 .select()
49 .from(users)
50 .where(eq(users.username, ownerName))
51 .limit(1);
52 if (!owner || owner.id !== user.id) {
53 return c.html(
54 <Layout title="Forbidden" user={user}>
55 <div class="empty-state">
56 <h2>403</h2>
57 <p>Only the repository owner can configure mirroring.</p>
58 </div>
59 </Layout>,
60 403
61 );
62 }
63 const [repo] = await db
64 .select()
65 .from(repositories)
66 .where(
67 and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName))
68 )
69 .limit(1);
70 if (!repo) return c.notFound();
71 return { user, ownerName, repoName, repo };
72}
73
74// ---------- Config page ----------
75
76mirrors.get("/:owner/:repo/settings/mirror", requireAuth, async (c) => {
77 const g = await ownerGate(c);
78 if (g instanceof Response) return g;
79 const { user, ownerName, repoName, repo } = g;
80
81 const mirror = await getMirrorForRepo(repo.id);
82 const runs = mirror ? await listRecentRuns(mirror.id, 20) : [];
83
84 const success = c.req.query("success");
85 const error = c.req.query("error");
86
87 return c.html(
88 <Layout title={`Mirror — ${ownerName}/${repoName}`} user={user}>
89 <RepoHeader owner={ownerName} repo={repoName} />
90 <div class="settings-container" style="max-width:720px">
91 <h2>Mirror settings</h2>
92 <p style="color:var(--text-muted)">
93 Keep this repository in sync with an upstream URL by periodically
94 running <code>git fetch --prune</code>. Only <code>https://</code>,{" "}
95 <code>http://</code>, and <code>git://</code> URLs are accepted —
96 SSH and local paths are not supported.
97 </p>
98
99 {success && (
100 <div class="auth-success">{decodeURIComponent(success)}</div>
101 )}
102 {error && (
103 <div class="auth-error">{decodeURIComponent(error)}</div>
104 )}
105
106 <form
107 method="POST"
108 action={`/${ownerName}/${repoName}/settings/mirror`}
109 class="panel"
110 style="padding:16px;margin:16px 0"
111 >
112 <div class="form-group">
113 <label for="upstream_url">Upstream URL</label>
114 <input
115 type="text"
116 id="upstream_url"
117 name="upstream_url"
118 value={mirror?.upstreamUrl || ""}
119 placeholder="https://github.com/torvalds/linux.git"
120 required
121 style="font-family:var(--font-mono)"
122 />
123 </div>
124 <div class="form-group">
125 <label for="interval_minutes">Sync interval (minutes)</label>
126 <input
127 type="number"
128 id="interval_minutes"
129 name="interval_minutes"
130 value={mirror?.intervalMinutes ?? 1440}
131 min="5"
132 max="43200"
133 style="width:160px"
134 />
135 </div>
136 <label
137 style="display:flex;gap:8px;align-items:center;margin-bottom:12px"
138 >
139 <input
140 type="checkbox"
141 name="is_enabled"
142 value="1"
143 checked={mirror ? mirror.isEnabled : true}
144 />
145 <span>Enabled</span>
146 </label>
147 <button type="submit" class="btn btn-primary">
148 {mirror ? "Update mirror" : "Enable mirror"}
149 </button>
150 </form>
151
152 {mirror && (
153 <>
154 <div style="display:flex;gap:8px;margin:12px 0">
155 <form
156 method="POST"
157 action={`/${ownerName}/${repoName}/settings/mirror/sync`}
158 >
159 <button type="submit" class="btn">
160 Sync now
161 </button>
162 </form>
163 <form
164 method="POST"
165 action={`/${ownerName}/${repoName}/settings/mirror/delete`}
166 onsubmit="return confirm('Remove mirror configuration?')"
167 >
168 <button type="submit" class="btn btn-danger">
169 Remove mirror
170 </button>
171 </form>
172 </div>
173
174 <h3 style="margin-top:20px">Last run</h3>
175 <div class="panel" style="padding:12px">
176 {mirror.lastSyncedAt ? (
177 <div>
178 <div
179 style="font-size:12px;color:var(--text-muted);text-transform:uppercase"
180 >
181 {mirror.lastStatus === "ok" ? "Success" : "Error"} —{" "}
182 {new Date(
183 mirror.lastSyncedAt as unknown as string
184 ).toLocaleString()}
185 </div>
186 {mirror.lastError && (
187 <pre
188 style="margin-top:8px;padding:8px;background:var(--bg-subtle);border-radius:4px;font-size:12px;overflow-x:auto;color:var(--red)"
189 >
190 {mirror.lastError}
191 </pre>
192 )}
193 </div>
194 ) : (
195 <div style="color:var(--text-muted)">Never synced.</div>
196 )}
197 </div>
198
199 <h3 style="margin-top:20px">Recent runs</h3>
200 <div class="panel">
201 {runs.length === 0 ? (
202 <div class="panel-empty">No runs yet.</div>
203 ) : (
204 runs.map((r) => (
205 <div
206 class="panel-item"
207 style="justify-content:space-between;flex-wrap:wrap;gap:6px"
208 >
209 <div>
210 <span
211 style={`font-size:11px;text-transform:uppercase;margin-right:8px;color:${
212 r.status === "ok"
213 ? "var(--green)"
214 : r.status === "error"
215 ? "var(--red)"
216 : "var(--text-muted)"
217 }`}
218 >
219 {r.status}
220 </span>
221 <span
222 style="font-size:12px;color:var(--text-muted);font-family:var(--font-mono)"
223 >
224 {new Date(
225 r.startedAt as unknown as string
226 ).toLocaleString()}
227 </span>
228 </div>
229 {r.message && (
230 <span
231 style="font-size:12px;color:var(--text-muted);max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap"
232 title={r.message}
233 >
234 {r.message.split("\n")[0]}
235 </span>
236 )}
237 </div>
238 ))
239 )}
240 </div>
241 <p
242 style="font-size:12px;color:var(--text-muted);margin-top:12px"
243 >
244 Upstream (logged, credentials redacted):{" "}
245 <code>{safeUrlForLog(mirror.upstreamUrl)}</code>
246 </p>
247 </>
248 )}
249 </div>
250 </Layout>
251 );
252});
253
254// ---------- Save config ----------
255
256mirrors.post("/:owner/:repo/settings/mirror", requireAuth, async (c) => {
257 const g = await ownerGate(c);
258 if (g instanceof Response) return g;
259 const { user, ownerName, repoName, repo } = g;
260 const body = await c.req.parseBody();
261 const upstreamUrl = String(body.upstream_url || "").trim();
262 const intervalRaw = Number(body.interval_minutes || 1440);
263 const interval = Math.max(5, Math.min(43200, Math.floor(intervalRaw || 1440)));
264 const isEnabled = String(body.is_enabled || "") === "1";
265
266 const v = validateUpstreamUrl(upstreamUrl);
267 if (!v.ok) {
268 return c.redirect(
269 `/${ownerName}/${repoName}/settings/mirror?error=${encodeURIComponent(
270 v.error || "Invalid URL"
271 )}`
272 );
273 }
274
275 const result = await upsertMirror({
276 repositoryId: repo.id,
277 upstreamUrl,
278 intervalMinutes: interval,
279 isEnabled,
280 });
281 if (!result.ok) {
282 return c.redirect(
283 `/${ownerName}/${repoName}/settings/mirror?error=${encodeURIComponent(
284 result.error
285 )}`
286 );
287 }
288
289 await audit({
290 userId: user.id,
291 repositoryId: repo.id,
292 action: "mirror.configure",
293 metadata: {
294 upstream: safeUrlForLog(upstreamUrl),
295 intervalMinutes: interval,
296 isEnabled,
297 },
298 });
299
300 return c.redirect(
301 `/${ownerName}/${repoName}/settings/mirror?success=${encodeURIComponent(
302 "Mirror configuration saved."
303 )}`
304 );
305});
306
307// ---------- Delete ----------
308
309mirrors.post("/:owner/:repo/settings/mirror/delete", requireAuth, async (c) => {
310 const g = await ownerGate(c);
311 if (g instanceof Response) return g;
312 const { user, ownerName, repoName, repo } = g;
313
314 await deleteMirror(repo.id);
315 await audit({
316 userId: user.id,
317 repositoryId: repo.id,
318 action: "mirror.delete",
319 });
320
321 return c.redirect(
322 `/${ownerName}/${repoName}/settings/mirror?success=${encodeURIComponent(
323 "Mirror removed."
324 )}`
325 );
326});
327
328// ---------- Sync now ----------
329
330mirrors.post("/:owner/:repo/settings/mirror/sync", requireAuth, async (c) => {
331 const g = await ownerGate(c);
332 if (g instanceof Response) return g;
333 const { user, ownerName, repoName, repo } = g;
334 const mirror = await getMirrorForRepo(repo.id);
335 if (!mirror) {
336 return c.redirect(
337 `/${ownerName}/${repoName}/settings/mirror?error=${encodeURIComponent(
338 "No mirror configured"
339 )}`
340 );
341 }
342
343 const result = await runMirrorSync(mirror.id);
344 await audit({
345 userId: user.id,
346 repositoryId: repo.id,
347 action: "mirror.sync",
348 metadata: { ok: result.ok, exitCode: result.exitCode },
349 });
350 const msg = result.ok
351 ? "Mirror sync completed."
352 : `Sync failed: ${result.message.split("\n")[0]}`;
353 return c.redirect(
354 `/${ownerName}/${repoName}/settings/mirror?${
355 result.ok ? "success" : "error"
356 }=${encodeURIComponent(msg)}`
357 );
358});
359
360// ---------- Admin: sync all due ----------
361
362mirrors.post("/admin/mirrors/sync-all", requireAuth, async (c) => {
363 const user = c.get("user")!;
364 if (!(await isSiteAdmin(user.id))) {
365 return c.html(
366 <Layout title="Forbidden" user={user}>
367 <div class="empty-state">
368 <h2>403</h2>
369 <p>Site admin only.</p>
370 </div>
371 </Layout>,
372 403
373 );
374 }
375 const summary = await syncAllDue();
376 await audit({
377 userId: user.id,
378 action: "admin.mirrors.sync-all",
379 metadata: summary,
380 });
381 return c.redirect(
382 `/admin?message=${encodeURIComponent(
383 `Mirror sync: ${summary.total} due, ${summary.ok} ok, ${summary.failed} failed.`
384 )}`
385 );
386});
387
388export default mirrors;
0389