feat(BLOCK-I): I8 symbol / xref navigation — ccantynz/Gluecron.com

ModifiedBUILD_BIBLE.md+7−3View fileUnified Split

@@ -86,7 +86,7 @@ Legend: ✅ shipped · 🟡 partial · ❌ not built

8686| Tag listing | ✅ | new this build |

8787| Code search (ILIKE) | ✅ | per-repo + global |

8888| Semantic / embedding search | ✅ | D1 — `code_chunks` table + lexical fallback, optional Voyage `voyage-code-3`; `src/lib/semantic-search.ts`, `src/routes/semantic-search.tsx` |

89| Symbol / xref navigation | ❌ | — |

89| Symbol / xref navigation | ✅ | I8 — `src/lib/symbols.ts` regex-based extractor for ts/js/py/rs/go/rb/java/kt/swift; on-demand indexer persists top-level definitions into `code_symbols` (0025). `src/routes/symbols.tsx` serves `/:owner/:repo/symbols` overview + A–Z list, `/:owner/:repo/symbols/search?q=` prefix search, `/:owner/:repo/symbols/:name` definition detail. Owner-only reindex. |

9090

9191### 2.3 Collaboration

9292| Feature | Status | Notes |

@@ -274,6 +274,7 @@ This is where GlueCron beats GitHub outright. **Priority: ship these loud.**

274274- **I5** — Code scanning UI → ✅ shipped. `src/routes/code-scanning.tsx` `GET /:owner/:repo/security` aggregates `gate_runs` matching `%scan%`/`%security%` (last 100), computes latest-per-gate status, renders failed/repaired/total summary cards + per-scanner status list + recent-runs table. Private-repo visibility enforced. Zero new tables — pure surfacing layer.

275275- **I6** — Sponsors → ✅ shipped. `drizzle/0023_sponsors.sql` adds `sponsorship_tiers` (maintainer_id, name, monthly_cents, one_time_allowed, is_active) + `sponsorships` (sponsor_id, maintainer_id, tier_id, amount_cents, kind, note, is_public, cancelled_at). `src/routes/sponsors.tsx` serves public `/sponsors/:username` (tier cards + recent public sponsors join) + maintainer `/settings/sponsors` (tier CRUD, soft-retire via is_active=false, activity list). Payment rails deferred — v1 captures intent + thank-you notes.

276276- **I7** — Weekly email digest → ✅ shipped. `drizzle/0024_email_digest.sql` adds `users.notify_email_digest_weekly` + `last_digest_sent_at`. `src/lib/email-digest.ts` exposes `composeDigest`/`sendDigestForUser`/`sendDigestsToAll` (never-throws). Pulls notifications + failed/repaired gate_runs + merged PRs from the last 7d, composes escaped HTML + plaintext, and sends via the shared email provider. `/settings/digest/preview` renders the digest inline for self-preview; `/admin/digests` gives site admins a "Send now" trigger + single-user preview, audit-logged as `admin.digests.run`/`admin.digests.preview`.

277- **I8** — Symbol / xref navigation → ✅ shipped. `drizzle/0025_code_symbols.sql` adds a `code_symbols` table. `src/lib/symbols.ts` provides a regex-based top-level extractor for ts/js/py/rs/go/rb/java/kt/swift. On-demand indexing via `POST /:owner/:repo/symbols/reindex` walks the default-branch tree, caps at 2000 files/1MB each, replaces the prior set. Browse at `/:owner/:repo/symbols` (A–Z + per-kind counts), search via `/symbols/search?q=`, inspect at `/symbols/:name`. 14 new tests.

277278

278279### BLOCK H — Marketplace

279280- **H1** — App marketplace → ✅ shipped. `src/routes/marketplace.tsx` + `src/lib/marketplace.ts` + `drizzle/0021_marketplace_and_apps.sql` (5 tables: `apps`, `app_installations`, `app_bots`, `app_install_tokens`, `app_events`). Routes: `GET /marketplace` (public directory with search), `GET /marketplace/:slug` (detail + install CTA), `POST /marketplace/:slug/install` (user-target install in v1), `POST /marketplace/installations/:id/uninstall`, `GET /settings/apps` (personal list), `GET+POST /developer/apps-new` (register), `GET /developer/apps/:slug/manage` (event log + install count), `POST /developer/apps/:slug/tokens/new` (show-once token). Install idempotent via soft-update on existing non-uninstalled row.

@@ -289,7 +290,7 @@ Everything below is committed, tested, and load-bearing. **Do not delete, rename

289290- `src/app.tsx` — route composition, middleware order, error handlers

290291- `src/index.ts` — Bun server entry

291292- `src/lib/config.ts` — env getters (late-binding)

292- `src/db/schema.ts` — 81 tables. New tables only via new migration.

293- `src/db/schema.ts` — 82 tables. New tables only via new migration.

293294- `src/db/index.ts` — lazy proxy DB connection

294295- `src/db/migrate.ts` — migration runner

295296- `drizzle/0000_initial.sql`, `drizzle/0001_green_ecosystem.sql` — migrations

@@ -314,6 +315,7 @@ Everything below is committed, tested, and load-bearing. **Do not delete, rename

314315- `drizzle/0022_repo_templates.sql` (Block I2+I3) — migration, never edited in place. Adds `repositories.is_template` (partial index where true) + `repo_transfers` audit table.

315316- `drizzle/0023_sponsors.sql` (Block I6) — migration, never edited in place. Adds `sponsorship_tiers` + `sponsorships` tables.

316317- `drizzle/0024_email_digest.sql` (Block I7) — migration, never edited in place. Adds `users.notify_email_digest_weekly` + `users.last_digest_sent_at`.

318- `drizzle/0025_code_symbols.sql` (Block I8) — migration, never edited in place. Adds `code_symbols` table with indexes on `(repository_id, name)` + `(repository_id, path)`.

317319

318320### 4.2 Git layer (locked)

319321- `src/git/repository.ts` — tree / blob / commits / diff / branches / blame / search / raw / tags / commitsBetween

@@ -437,6 +439,8 @@ Everything below is committed, tested, and load-bearing. **Do not delete, rename

437439- `src/lib/email-digest.ts` (Block I7) — `composeDigest(userId, since?)` (never throws, null on failure), `sendDigestForUser(userId)` (opt-out check + updates `last_digest_sent_at` on success), `sendDigestsToAll()` (iterates opted-in users). Pulls notifications + owned-repo gate_runs (failed/repaired) + merged PRs over last 7d. Builds text + escaped HTML body. Exports `__internal = { textToHtml, escapeHtml, fmtRange }` for tests.

438440- `src/routes/admin.tsx` (extends Block F3 for I7) — adds `GET /admin/digests` (opted-in count + recently sent list), `POST /admin/digests/run` (calls `sendDigestsToAll`, audit-logged with counts), `POST /admin/digests/preview` (sends to one user by username, audit-logged). New "Email digests" tile on the /admin dashboard grid.

439441- `src/routes/settings.tsx` (extends for I7) — adds `notify_email_digest_weekly` checkbox to email prefs + handler wiring in `POST /settings/notifications`, and `GET /settings/digest/preview` (renders `composeDigest` output inline via `raw(body.html)` with Hono's `hono/html`).

442- `src/lib/symbols.ts` (Block I8) — regex-based top-level symbol extractor. Pure helpers: `detectLanguage(path)` (10 extensions mapped to 8 languages), `extractSymbols(content, lang)` (per-language rule list, 1-based line numbers, 240-char signature cap, skips lines >500 chars). `indexRepositorySymbols(repoId)` walks the default-branch tree, caps at 2000 files / 1MB each, replaces the prior set in batches of 500. `findDefinitions(repoId, name)` + `countSymbolsForRepo(repoId)`. `__internal` exposes `RULES` + `EXT_LANG` for tests.

443- `src/routes/symbols.tsx` (Block I8) — `/:owner/:repo/symbols` overview (total + per-kind counts + A–Z list with blob deep-links), `/:owner/:repo/symbols/search?q=` prefix search (ilike `q%`), `/:owner/:repo/symbols/:name` detail (all definitions with signature preview + deep link). `POST /:owner/:repo/symbols/reindex` is requireAuth + owner-only.

440444

441445### 4.7 Views (locked contracts)

442446- `src/views/layout.tsx` — `Layout` accepts `title`, `user`, `notificationCount`

@@ -471,7 +475,7 @@ Everything below is committed, tested, and load-bearing. **Do not delete, rename

471475```bash

472476bun install

473477bun dev # hot reload

474bun test # 570 tests currently pass

478bun test # 584 tests currently pass

475479bun run db:migrate

476480```

477481

Addeddrizzle/0025_code_symbols.sql+31−0View fileUnified Split

@@ -0,0 +1,31 @@

1-- Gluecron migration 0025: Code symbol index.

2--

3-- I8 — Symbol / xref navigation. Stores top-level symbol definitions

4-- (functions, classes, interfaces, types, consts) extracted from a

5-- repository's HEAD via a regex-based parser. References are found at

6-- lookup-time by grepping content, so we only persist definitions.

7--

8-- On-demand index: owner clicks "Reindex" on /:owner/:repo/symbols. We

9-- keep only the most recent commit's symbols — older rows are overwritten

10-- by deleting the prior set before inserting the new one.

11

12--> statement-breakpoint

13CREATE TABLE IF NOT EXISTS "code_symbols" (

14 "id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),

15 "repository_id" uuid NOT NULL REFERENCES "repositories"("id") ON DELETE CASCADE,

16 "commit_sha" text NOT NULL,

17 "name" text NOT NULL,

18 "kind" text NOT NULL,

19 "path" text NOT NULL,

20 "line" integer NOT NULL,

21 "signature" text,

22 "created_at" timestamp NOT NULL DEFAULT now()

23);

24

25--> statement-breakpoint

26CREATE INDEX IF NOT EXISTS "code_symbols_repo_name_idx"

27 ON "code_symbols" ("repository_id", "name");

28

29--> statement-breakpoint

30CREATE INDEX IF NOT EXISTS "code_symbols_repo_path_idx"

31 ON "code_symbols" ("repository_id", "path");

Addedsrc/__tests__/symbols.test.ts+138−0View fileUnified Split

@@ -0,0 +1,138 @@

1/**

2 * Block I8 — Symbol / xref navigation tests.

3 *

4 * Pure tests for the regex-based extractor per language + auth smoke on

5 * the reindex endpoint.

6 */

7

8import { describe, it, expect } from "bun:test";

9import app from "../app";

10import { detectLanguage, extractSymbols } from "../lib/symbols";

11

12describe("symbols — detectLanguage", () => {

13 it("maps common extensions", () => {

14 expect(detectLanguage("src/foo.ts")).toBe("ts");

15 expect(detectLanguage("src/Foo.tsx")).toBe("ts");

16 expect(detectLanguage("script.js")).toBe("ts");

17 expect(detectLanguage("app.py")).toBe("py");

18 expect(detectLanguage("main.rs")).toBe("rs");

19 expect(detectLanguage("main.go")).toBe("go");

20 expect(detectLanguage("App.java")).toBe("java");

21 expect(detectLanguage("Main.kt")).toBe("kt");

22 expect(detectLanguage("App.swift")).toBe("swift");

23 expect(detectLanguage("helper.rb")).toBe("rb");

24 });

25

26 it("returns null for unknown extensions", () => {

27 expect(detectLanguage("README.md")).toBe(null);

28 expect(detectLanguage("styles.css")).toBe(null);

29 expect(detectLanguage("noext")).toBe(null);

30 });

31});

32

33describe("symbols — extractSymbols (ts)", () => {

34 it("finds exported functions", () => {

35 const src = `export function foo() {}\nexport async function bar() {}`;

36 const syms = extractSymbols(src, "ts");

37 expect(syms.find((s) => s.name === "foo")?.kind).toBe("function");

38 expect(syms.find((s) => s.name === "bar")?.kind).toBe("function");

39 });

40

41 it("finds classes + interfaces + types", () => {

42 const src = [

43 "export class Widget {}",

44 "export interface Opts {}",

45 "export type ID = string;",

46 ].join("\n");

47 const syms = extractSymbols(src, "ts");

48 expect(syms.find((s) => s.name === "Widget")?.kind).toBe("class");

49 expect(syms.find((s) => s.name === "Opts")?.kind).toBe("interface");

50 expect(syms.find((s) => s.name === "ID")?.kind).toBe("type");

51 });

52

53 it("finds arrow-function consts as functions", () => {

54 const src = `export const handler = async (req) => {};`;

55 const syms = extractSymbols(src, "ts");

56 expect(syms.find((s) => s.name === "handler")?.kind).toBe("function");

57 });

58

59 it("records 1-based line numbers", () => {

60 const src = `// comment\nexport function foo() {}`;

61 const [sym] = extractSymbols(src, "ts");

62 expect(sym.line).toBe(2);

63 });

64

65 it("skips minified / overly long lines", () => {

66 const long = "x".repeat(600);

67 const src = `${long}\nexport function real() {}`;

68 const syms = extractSymbols(src, "ts");

69 expect(syms.length).toBe(1);

70 expect(syms[0].name).toBe("real");

71 });

72

73 it("truncates signature to 240 chars", () => {

74 // keep the line under 500 chars (extractor skips minified lines)

75 const src = `export function foo(${"x: string, ".repeat(30)}) {}`;

76 const [sym] = extractSymbols(src, "ts");

77 expect(sym.signature.length).toBeLessThanOrEqual(240);

78 });

79});

80

81describe("symbols — extractSymbols (python)", () => {

82 it("finds def and class", () => {

83 const src = `def load():\n pass\n\nclass Widget:\n pass`;

84 const syms = extractSymbols(src, "py");

85 expect(syms.find((s) => s.name === "load")?.kind).toBe("function");

86 expect(syms.find((s) => s.name === "Widget")?.kind).toBe("class");

87 });

88

89 it("finds SCREAMING_CASE constants", () => {

90 const src = `MAX_ITEMS = 100\nfoo = 1`;

91 const syms = extractSymbols(src, "py");

92 expect(syms.find((s) => s.name === "MAX_ITEMS")?.kind).toBe("const");

93 // lowercase should not match const rule

94 expect(syms.find((s) => s.name === "foo")).toBeUndefined();

95 });

96});

97

98describe("symbols — extractSymbols (rust + go)", () => {

99 it("finds rust fn/struct/trait", () => {

100 const src = [

101 "pub fn run() {}",

102 "pub struct Config {}",

103 "pub trait Loader {}",

104 ].join("\n");

105 const syms = extractSymbols(src, "rs");

106 expect(syms.find((s) => s.name === "run")?.kind).toBe("function");

107 expect(syms.find((s) => s.name === "Config")?.kind).toBe("class");

108 expect(syms.find((s) => s.name === "Loader")?.kind).toBe("interface");

109 });

110

111 it("finds go func + type struct + type interface", () => {

112 const src = [

113 "func Handle() {}",

114 "type Config struct {",

115 "type Loader interface {",

116 ].join("\n");

117 const syms = extractSymbols(src, "go");

118 expect(syms.find((s) => s.name === "Handle")?.kind).toBe("function");

119 expect(syms.find((s) => s.name === "Config")?.kind).toBe("class");

120 expect(syms.find((s) => s.name === "Loader")?.kind).toBe("interface");

121 });

122});

123

124describe("symbols — extractSymbols (unknown language)", () => {

125 it("returns empty for unknown language", () => {

126 expect(extractSymbols("blah", "unknown")).toEqual([]);

127 });

128});

129

130describe("symbols — route auth", () => {

131 it("POST /:owner/:repo/symbols/reindex without auth → 302 /login", async () => {

132 const res = await app.request("/alice/repo/symbols/reindex", {

133 method: "POST",

134 });

135 expect(res.status).toBe(302);

136 expect(res.headers.get("location") || "").toContain("/login");

137 });

138});

Modifiedsrc/app.tsx+4−0View fileUnified Split

@@ -66,6 +66,7 @@ import marketplaceRoutes from "./routes/marketplace";

6666import templatesRoutes from "./routes/templates";

6767import codeScanningRoutes from "./routes/code-scanning";

6868import sponsorsRoutes from "./routes/sponsors";

69import symbolsRoutes from "./routes/symbols";

6970import webRoutes from "./routes/web";

7071

7172const app = new Hono();

@@ -226,6 +227,9 @@ app.route("/", codeScanningRoutes);

226227// Sponsors — /sponsors/:user + /settings/sponsors (Block I6)

227228app.route("/", sponsorsRoutes);

228229

230// Symbol / xref navigation — /:owner/:repo/symbols (Block I8)

231app.route("/", symbolsRoutes);

232

229233// Insights + milestones

230234app.route("/", insightsRoutes);

231235

Modifiedsrc/db/schema.ts+24−0View fileUnified Split

@@ -1970,3 +1970,27 @@ export const sponsorships = pgTable(

19701970);

19711971

19721972export type Sponsorship = typeof sponsorships.$inferSelect;

1973

1974// Block I8 — Code symbol index for xref navigation.

1975export const codeSymbols = pgTable(

1976 "code_symbols",

1977 {

1978 id: uuid("id").primaryKey().defaultRandom(),

1979 repositoryId: uuid("repository_id")

1980 .notNull()

1981 .references(() => repositories.id, { onDelete: "cascade" }),

1982 commitSha: text("commit_sha").notNull(),

1983 name: text("name").notNull(),

1985 path: text("path").notNull(),

1986 line: integer("line").notNull(),

1987 signature: text("signature"),

1988 createdAt: timestamp("created_at").defaultNow().notNull(),

1989 },

1990 (table) => [

1991 index("code_symbols_repo_name_idx").on(table.repositoryId, table.name),

1992 index("code_symbols_repo_path_idx").on(table.repositoryId, table.path),

1993 ]

1994);

1995

1996export type CodeSymbol = typeof codeSymbols.$inferSelect;

Addedsrc/lib/symbols.ts+319−0View fileUnified Split

@@ -0,0 +1,319 @@

1/**

2 * Block I8 — Symbol / xref navigation.

3 *

4 * A pragmatic regex-based top-level symbol extractor. Runs per-language,

5 * catches the common definition shapes (function / class / interface /

6 * type / const). References are computed at lookup-time by grepping the

7 * repository's tree for the symbol name, so this module persists only

8 * definitions. Never throws into request path.

9 */

10

11import { and, eq } from "drizzle-orm";

12import { db } from "../db";

13import { codeSymbols, repositories, users } from "../db/schema";

14import { getBlob, getTree, getDefaultBranch, resolveRef } from "../git/repository";

15import type { CodeSymbol } from "../db/schema";

16import type { GitTreeEntry } from "../git/repository";

17

18export type SymbolKind =

19 | "function"

20 | "class"

21 | "interface"

22 | "type"

23 | "const"

24 | "variable";

25

26export interface ExtractedSymbol {

27 name: string;

28 kind: SymbolKind;

29 line: number;

30 signature: string;

31}

32

33type Rule = { kind: SymbolKind; re: RegExp };

34

35// ---------- Language detection ----------

36

37const EXT_LANG: Record<string, string> = {

38 ts: "ts",

39 tsx: "ts",

40 js: "ts",

41 jsx: "ts",

42 mjs: "ts",

43 cjs: "ts",

44 py: "py",

45 rs: "rs",

46 go: "go",

47 rb: "rb",

48 java: "java",

49 kt: "kt",

50 swift: "swift",

51};

52

53export function detectLanguage(path: string): string | null {

54 const ext = path.split(".").pop()?.toLowerCase() || "";

55 return EXT_LANG[ext] ?? null;

56}

57

58// ---------- Per-language rules ----------

59

60const RULES: Record<string, Rule[]> = {

61 ts: [

62 {

63 kind: "function",

64 re: /^\s*(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+([A-Za-z_$][\w$]*)/,

65 },

66 {

67 kind: "function",

68 re: /^\s*(?:export\s+)?const\s+([A-Za-z_$][\w$]*)\s*=\s*(?:async\s*)?\(/,

69 },

70 {

71 kind: "function",

72 re: /^\s*(?:export\s+)?const\s+([A-Za-z_$][\w$]*)\s*:\s*[^=]+=\s*(?:async\s*)?\(/,

73 },

74 {

75 kind: "class",

76 re: /^\s*(?:export\s+)?(?:default\s+)?(?:abstract\s+)?class\s+([A-Za-z_$][\w$]*)/,

77 },

78 {

79 kind: "interface",

80 re: /^\s*(?:export\s+)?interface\s+([A-Za-z_$][\w$]*)/,

81 },

82 {

83 kind: "type",

84 re: /^\s*(?:export\s+)?type\s+([A-Za-z_$][\w$]*)\s*=/,

85 },

86 {

87 kind: "const",

88 re: /^\s*(?:export\s+)?(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*[:=]/,

89 },

90 ],

91 py: [

92 { kind: "function", re: /^\s*(?:async\s+)?def\s+([A-Za-z_][\w]*)\s*\(/ },

93 { kind: "class", re: /^\s*class\s+([A-Za-z_][\w]*)\s*[:(]/ },

94 { kind: "const", re: /^([A-Z_][A-Z0-9_]*)\s*=/ },

95 ],

96 rs: [

97 { kind: "function", re: /^\s*(?:pub(?:$[^)]*$)?\s+)?(?:async\s+)?fn\s+([A-Za-z_][\w]*)/ },

98 { kind: "class", re: /^\s*(?:pub(?:$[^)]*$)?\s+)?struct\s+([A-Za-z_][\w]*)/ },

99 { kind: "interface", re: /^\s*(?:pub(?:$[^)]*$)?\s+)?trait\s+([A-Za-z_][\w]*)/ },

100 { kind: "type", re: /^\s*(?:pub(?:$[^)]*$)?\s+)?type\s+([A-Za-z_][\w]*)\s*=/ },

101 { kind: "const", re: /^\s*(?:pub(?:$[^)]*$)?\s+)?const\s+([A-Z_][A-Z0-9_]*)\s*:/ },

102 ],

103 go: [

104 { kind: "function", re: /^\s*func(?:\s+$[^)]*$)?\s+([A-Za-z_][\w]*)\s*\(/ },

105 { kind: "class", re: /^\s*type\s+([A-Za-z_][\w]*)\s+struct\b/ },

106 { kind: "interface", re: /^\s*type\s+([A-Za-z_][\w]*)\s+interface\b/ },

107 { kind: "type", re: /^\s*type\s+([A-Za-z_][\w]*)\s+\w/ },

108 { kind: "const", re: /^\s*const\s+([A-Za-z_][\w]*)\s*=/ },

109 ],

110 rb: [

111 { kind: "function", re: /^\s*def\s+(?:self\.)?([A-Za-z_][\w?!=]*)/ },

112 { kind: "class", re: /^\s*class\s+([A-Z][\w]*)/ },

113 ],

114 java: [

115 {

116 kind: "class",

117 re: /^\s*(?:public|private|protected)?\s*(?:abstract\s+|final\s+)?class\s+([A-Z][\w]*)/,

118 },

119 {

120 kind: "interface",

121 re: /^\s*(?:public|private|protected)?\s*interface\s+([A-Z][\w]*)/,

122 },

123 ],

124 kt: [

125 { kind: "function", re: /^\s*(?:public|private|internal)?\s*fun\s+([A-Za-z_][\w]*)/ },

126 { kind: "class", re: /^\s*(?:public|private|internal)?\s*class\s+([A-Z][\w]*)/ },

127 ],

128 swift: [

129 { kind: "function", re: /^\s*(?:public|private|fileprivate|internal)?\s*func\s+([A-Za-z_][\w]*)/ },

130 { kind: "class", re: /^\s*(?:public|private|fileprivate|internal)?\s*class\s+([A-Z][\w]*)/ },

131 { kind: "interface", re: /^\s*(?:public|private|fileprivate|internal)?\s*protocol\s+([A-Z][\w]*)/ },

132 ],

133};

134

135// ---------- Extractor ----------

136

137/** Pure — extract top-level symbol definitions from a single file. */

138export function extractSymbols(

139 content: string,

140 lang: string

141): ExtractedSymbol[] {

142 const rules = RULES[lang];

143 if (!rules) return [];

144 const out: ExtractedSymbol[] = [];

145 const seen = new Set<string>();

146 const lines = content.split("\n");

147 for (let i = 0; i < lines.length; i++) {

148 const line = lines[i];

149 if (line.length > 500) continue; // skip minified lines

150 for (const rule of rules) {

151 const m = line.match(rule.re);

152 if (m && m[1]) {

153 const key = `${rule.kind}:${m[1]}:${i}`;

154 if (seen.has(key)) continue;

155 seen.add(key);

156 out.push({

157 name: m[1],

158 kind: rule.kind,

159 line: i + 1,

160 signature: line.trim().slice(0, 240),

161 });

162 break; // one match per line

163 }

164 }

165 }

166 return out;

167}

168

169// ---------- Indexer ----------

170

171const INDEXABLE_MAX_BYTES = 1_000_000; // skip files over 1MB

172const MAX_FILES = 2_000; // cap per reindex

173

174async function walkCodePaths(

175 owner: string,

176 repo: string,

177 ref: string,

178 maxFiles = MAX_FILES

179): Promise<Array<{ path: string; size?: number }>> {

180 const out: Array<{ path: string; size?: number }> = [];

181 const queue: string[] = [""];

182 while (queue.length && out.length < maxFiles) {

183 const dir = queue.shift()!;

184 let entries: GitTreeEntry[] = [];

185 try {

186 entries = await getTree(owner, repo, ref, dir);

187 } catch {

188 continue;

189 }

190 for (const e of entries) {

191 const p = dir ? `${dir}/${e.name}` : e.name;

192 if (e.type === "tree") {

193 const base = e.name.toLowerCase();

194 if (

195 base === "node_modules" ||

196 base === ".git" ||

197 base === "dist" ||

198 base === "build" ||

199 base === "vendor" ||

200 base === ".next" ||

201 base === ".turbo" ||

202 base === "target" ||

203 base === "__pycache__"

204 ) {

205 continue;

206 }

207 queue.push(p);

208 } else if (e.type === "blob") {

209 if (!detectLanguage(p)) continue;

210 if (e.size !== undefined && e.size > INDEXABLE_MAX_BYTES) continue;

211 out.push({ path: p, size: e.size });

212 if (out.length >= maxFiles) break;

213 }

214 }

215 }

216 return out;

217}

218

219/** Walks the repo tree at HEAD, extracts symbols, replaces the prior set. */

220export async function indexRepositorySymbols(

221 repositoryId: string

222): Promise<{ indexed: number; files: number; commitSha: string } | null> {

223 try {

224 const [repo] = await db

225 .select()

226 .from(repositories)

227 .where(eq(repositories.id, repositoryId))

228 .limit(1);

229 if (!repo) return null;

230

231 const [owner] = await db

232 .select({ username: users.username })

233 .from(users)

234 .where(eq(users.id, repo.ownerId))

235 .limit(1);

236 if (!owner) return null;

237

238 const defaultBranch =

239 (await getDefaultBranch(owner.username, repo.name)) || "main";

240 const head = await resolveRef(owner.username, repo.name, defaultBranch);

241 if (!head) return null;

242

243 const files = await walkCodePaths(owner.username, repo.name, head);

244

245 const rows: Array<Omit<CodeSymbol, "id" | "createdAt">> = [];

246 let processed = 0;

247

248 for (const f of files) {

249 const lang = detectLanguage(f.path);

250 if (!lang) continue;

251 try {

252 const blob = await getBlob(owner.username, repo.name, head, f.path);

253 if (!blob || blob.isBinary) continue;

254 const syms = extractSymbols(blob.content, lang);

255 for (const s of syms) {

256 rows.push({

257 repositoryId: repo.id,

258 commitSha: head,

259 name: s.name,

260 kind: s.kind,

261 path: f.path,

262 line: s.line,

263 signature: s.signature,

264 });

265 }

266 processed++;

267 } catch {

268 // skip unreadable files

269 }

270 }

271

272 // Replace the prior index (DELETE + batched INSERTs).

273 await db.delete(codeSymbols).where(eq(codeSymbols.repositoryId, repo.id));

274 const BATCH = 500;

275 for (let i = 0; i < rows.length; i += BATCH) {

276 await db.insert(codeSymbols).values(rows.slice(i, i + BATCH));

277 }

278

279 return { indexed: rows.length, files: processed, commitSha: head };

280 } catch (err) {

281 console.error("[symbols] indexRepositorySymbols error:", err);

282 return null;

283 }

284}

285

286/** Find definitions of a symbol name within a repo. */

287export async function findDefinitions(

288 repositoryId: string,

289 name: string

290): Promise<CodeSymbol[]> {

291 try {

292 return await db

293 .select()

294 .from(codeSymbols)

295 .where(

296 and(eq(codeSymbols.repositoryId, repositoryId), eq(codeSymbols.name, name))

297 );

298 } catch {

299 return [];

300 }

301}

302

303/** Count total indexed symbols for a repo (pagination helper). */

304export async function countSymbolsForRepo(

305 repositoryId: string

306): Promise<number> {

307 try {

308 const rows = await db

309 .select({ id: codeSymbols.id })

310 .from(codeSymbols)

311 .where(eq(codeSymbols.repositoryId, repositoryId));

312 return rows.length;

313 } catch {

314 return 0;

315 }

316}

317

318// Test-only hook

319export const __internal = { RULES, EXT_LANG };

Addedsrc/routes/symbols.tsx+369−0View fileUnified Split

@@ -0,0 +1,369 @@

1/**

2 * Block I8 — Symbol / xref navigation.

3 *

4 * GET /:owner/:repo/symbols — overview + "Reindex" button

5 * GET /:owner/:repo/symbols/search — search by name (prefix match)

6 * GET /:owner/:repo/symbols/:name — definitions list

7 * POST /:owner/:repo/symbols/reindex — owner-only, runs indexer

8 */

9

10import { Hono } from "hono";

11import { and, asc, eq, ilike, sql } from "drizzle-orm";

12import { db } from "../db";

13import { codeSymbols, repositories, users } from "../db/schema";

14import { Layout } from "../views/layout";

15import { RepoHeader, RepoNav } from "../views/components";

16import { softAuth, requireAuth } from "../middleware/auth";

17import type { AuthEnv } from "../middleware/auth";

18import { indexRepositorySymbols, findDefinitions } from "../lib/symbols";

19

20const symbols = new Hono<AuthEnv>();

21symbols.use("*", softAuth);

22

23async function loadRepo(ownerName: string, repoName: string) {

24 const [owner] = await db

25 .select()

26 .from(users)

27 .where(eq(users.username, ownerName))

28 .limit(1);

29 if (!owner) return null;

30 const [repo] = await db

31 .select()

32 .from(repositories)

33 .where(

34 and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName))

35 )

36 .limit(1);

37 if (!repo) return null;

38 return { owner, repo };

39}

40

41// ---------- Overview ----------

42

43symbols.get("/:owner/:repo/symbols", async (c) => {

44 const user = c.get("user");

45 const { owner: ownerName, repo: repoName } = c.req.param();

46 const ctx = await loadRepo(ownerName, repoName);

47 if (!ctx) return c.notFound();

48 const { owner, repo } = ctx;

49 if (repo.isPrivate && (!user || user.id !== repo.ownerId)) {

50 return c.notFound();

51 }

52

53 const [countRow] = await db

54 .select({ n: sql<number>`count(*)::int` })

55 .from(codeSymbols)

56 .where(eq(codeSymbols.repositoryId, repo.id));

57 const total = Number(countRow?.n || 0);

58

59 const byKindRaw = await db

60 .select({

61 kind: codeSymbols.kind,

62 n: sql<number>`count(*)::int`,

63 })

64 .from(codeSymbols)

65 .where(eq(codeSymbols.repositoryId, repo.id))

66 .groupBy(codeSymbols.kind);

67

68 const latest = await db

69 .select({

70 name: codeSymbols.name,

71 kind: codeSymbols.kind,

72 path: codeSymbols.path,

73 line: codeSymbols.line,

74 })

75 .from(codeSymbols)

76 .where(eq(codeSymbols.repositoryId, repo.id))

77 .orderBy(asc(codeSymbols.name))

78 .limit(50);

79

80 const isOwner = user && user.id === repo.ownerId;

81 const message = c.req.query("message");

82

83 return c.html(

84 <Layout title={`Symbols — ${ownerName}/${repoName}`} user={user}>

85 <RepoHeader owner={ownerName} repo={repoName} />

86 <RepoNav owner={ownerName} repo={repoName} active="code" />

87 <div class="settings-container">

88 <div style="display:flex;justify-content:space-between;align-items:center">

89 <h2 style="margin:0">Symbols</h2>

90 {isOwner && (

91 <form method="POST" action={`/${ownerName}/${repoName}/symbols/reindex`}>

92 <button type="submit" class="btn btn-primary btn-sm">

93 Reindex

94 </button>

95 </form>

96 )}

97 </div>

98 {message && (

99 <div class="auth-success" style="margin-top:12px">

100 {decodeURIComponent(message)}

101 </div>

102 )}

103 <p style="color:var(--text-muted);margin-top:8px">

104 Top-level definitions from the default branch. Click a symbol to

105 see all its definitions.

106 </p>

107

108 <form

109 method="GET"

110 action={`/${ownerName}/${repoName}/symbols/search`}

111 style="display:flex;gap:8px;margin:16px 0"

112 >

113 <input

114 type="text"

115 name="q"

116 placeholder="Search symbol name..."

117 required

118 style="flex:1"

119 />

120 <button type="submit" class="btn">

121 Search

122 </button>

123 </form>

124

125 <div

126 style="display:grid;grid-template-columns:repeat(auto-fit,minmax(140px,1fr));gap:8px;margin-bottom:16px"

127 >

128 <div class="panel" style="padding:12px;text-align:center">

129 <div style="font-size:20px;font-weight:700">{total}</div>

130 <div style="font-size:11px;color:var(--text-muted);text-transform:uppercase">

131 Symbols

132 </div>

133 </div>

134 {byKindRaw.map((r) => (

135 <div class="panel" style="padding:12px;text-align:center">

136 <div style="font-size:20px;font-weight:700">{Number(r.n)}</div>

137 <div style="font-size:11px;color:var(--text-muted);text-transform:uppercase">

138 {r.kind}

139 </div>

140 </div>

141 ))}

142 </div>

143

144 <h3>A–Z</h3>

145 {total === 0 ? (

146 <div class="panel-empty" style="padding:24px">

147 No symbols indexed yet.

148 {isOwner && " Click Reindex to scan the repository."}

149 </div>

150 ) : (

151 <div class="panel">

152 {latest.map((s) => (

153 <div class="panel-item" style="justify-content:space-between">

154 <div>

155 <a

156 href={`/${ownerName}/${repoName}/symbols/${encodeURIComponent(s.name)}`}

157 style="font-weight:600;font-family:var(--font-mono)"

158 >

159 {s.name}

160 </a>{" "}

161 <span

162 style="font-size:11px;color:var(--text-muted);text-transform:uppercase;margin-left:6px"

163 >

164 {s.kind}

165 </span>

166 </div>

167 <a

168 href={`/${ownerName}/${repoName}/blob/HEAD/${s.path}#L${s.line}`}

169 style="font-size:12px;color:var(--text-muted);font-family:var(--font-mono)"

170 >

171 {s.path}:{s.line}

172 </a>

173 </div>

174 ))}

175 </div>

176 )}

177 </div>

178 </Layout>

179 );

180});

181

182// ---------- Search ----------

183

184symbols.get("/:owner/:repo/symbols/search", async (c) => {

185 const user = c.get("user");

186 const { owner: ownerName, repo: repoName } = c.req.param();

187 const ctx = await loadRepo(ownerName, repoName);

188 if (!ctx) return c.notFound();

189 const { repo } = ctx;

190 if (repo.isPrivate && (!user || user.id !== repo.ownerId)) {

191 return c.notFound();

192 }

193

194 const q = (c.req.query("q") || "").trim();

195 const results = q

196 ? await db

197 .select({

198 name: codeSymbols.name,

199 kind: codeSymbols.kind,

200 path: codeSymbols.path,

201 line: codeSymbols.line,

202 })

203 .from(codeSymbols)

204 .where(

205 and(

206 eq(codeSymbols.repositoryId, repo.id),

207 ilike(codeSymbols.name, `${q}%`)

208 )

209 )

210 .orderBy(asc(codeSymbols.name))

211 .limit(200)

212 : [];

213

214 return c.html(

215 <Layout title={`Symbol search — ${ownerName}/${repoName}`} user={user}>

216 <RepoHeader owner={ownerName} repo={repoName} />

217 <RepoNav owner={ownerName} repo={repoName} active="code" />

218 <div class="settings-container">

219 <h2>Symbol search</h2>

220 <form

221 method="GET"

222 action={`/${ownerName}/${repoName}/symbols/search`}

223 style="display:flex;gap:8px;margin:12px 0"

224 >

225 <input

226 type="text"

227 name="q"

228 value={q}

229 placeholder="Search symbol name..."

230 required

231 style="flex:1"

232 />

233 <button type="submit" class="btn">

234 Search

235 </button>

236 <a href={`/${ownerName}/${repoName}/symbols`} class="btn">

237 Back

238 </a>

239 </form>

240 {q === "" ? (

241 <p style="color:var(--text-muted)">Enter a prefix to search.</p>

242 ) : results.length === 0 ? (

243 <p style="color:var(--text-muted)">No symbols match "{q}".</p>

244 ) : (

245 <div class="panel">

246 {results.map((s) => (

247 <div class="panel-item" style="justify-content:space-between">

248 <div>

249 <a

250 href={`/${ownerName}/${repoName}/symbols/${encodeURIComponent(s.name)}`}

251 style="font-weight:600;font-family:var(--font-mono)"

252 >

253 {s.name}

254 </a>{" "}

255 <span

256 style="font-size:11px;color:var(--text-muted);text-transform:uppercase;margin-left:6px"

257 >

258 {s.kind}

259 </span>

260 </div>

261 <a

262 href={`/${ownerName}/${repoName}/blob/HEAD/${s.path}#L${s.line}`}

263 style="font-size:12px;color:var(--text-muted);font-family:var(--font-mono)"

264 >

265 {s.path}:{s.line}

266 </a>

267 </div>

268 ))}

269 </div>

270 )}

271 </div>

272 </Layout>

273 );

274});

275

276// ---------- Symbol detail ----------

277

278symbols.get("/:owner/:repo/symbols/:name", async (c) => {

279 const user = c.get("user");

280 const { owner: ownerName, repo: repoName, name } = c.req.param();

281 const ctx = await loadRepo(ownerName, repoName);

282 if (!ctx) return c.notFound();

283 const { repo } = ctx;

284 if (repo.isPrivate && (!user || user.id !== repo.ownerId)) {

285 return c.notFound();

286 }

287

288 const defs = await findDefinitions(repo.id, decodeURIComponent(name));

289

290 return c.html(

291 <Layout title={`${name} — ${ownerName}/${repoName}`} user={user}>

292 <RepoHeader owner={ownerName} repo={repoName} />

293 <RepoNav owner={ownerName} repo={repoName} active="code" />

294 <div class="settings-container">

295 <h2 style="font-family:var(--font-mono)">{decodeURIComponent(name)}</h2>

296 <p style="color:var(--text-muted)">

297 {defs.length} definition{defs.length === 1 ? "" : "s"}

298 </p>

299 {defs.length === 0 ? (

300 <div class="panel-empty" style="padding:24px">

301 No definitions found.{" "}

302 <a href={`/${ownerName}/${repoName}/symbols`}>Back to symbols</a>

303 </div>

304 ) : (

305 <div class="panel">

306 {defs.map((d) => (

307 <div class="panel-item" style="flex-direction:column;align-items:stretch;gap:4px">

308 <div style="display:flex;justify-content:space-between;align-items:center">

309 <span

310 style="font-size:11px;color:var(--text-muted);text-transform:uppercase"

311 >

312 {d.kind}

313 </span>

314 <a

315 href={`/${ownerName}/${repoName}/blob/HEAD/${d.path}#L${d.line}`}

316 style="font-size:12px;font-family:var(--font-mono)"

317 >

318 {d.path}:{d.line}

319 </a>

320 </div>

321 {d.signature && (

322 <pre

323 style="margin:0;padding:8px;background:var(--bg-subtle);border-radius:4px;font-size:12px;overflow-x:auto"

324 >

325 {d.signature}

326 </pre>

327 )}

328 </div>

329 ))}

330 </div>

331 )}

332 <p style="margin-top:20px">

333 <a href={`/${ownerName}/${repoName}/symbols`}>← Back to symbols</a>

334 </p>

335 </div>

336 </Layout>

337 );

338});

339

340// ---------- Reindex ----------

341

342symbols.post("/:owner/:repo/symbols/reindex", requireAuth, async (c) => {

343 const user = c.get("user")!;

344 const { owner: ownerName, repo: repoName } = c.req.param();

345 const ctx = await loadRepo(ownerName, repoName);

346 if (!ctx) return c.notFound();

347 const { repo } = ctx;

348 if (user.id !== repo.ownerId) {

349 return c.html(

350 <Layout title="Forbidden" user={user}>

351 <div class="empty-state">

352 <h2>403</h2>

353 <p>Only the repository owner can reindex symbols.</p>

354 </div>

355 </Layout>,

356 403

357 );

358 }

359

360 const result = await indexRepositorySymbols(repo.id);

361 const msg = result

362 ? `Indexed ${result.indexed} symbols across ${result.files} files.`

363 : "Indexing failed — see server logs.";

364 return c.redirect(

365 `/${ownerName}/${repoName}/symbols?message=${encodeURIComponent(msg)}`

366 );

367});

368

369export default symbols;

0370