Marginalia is a Retrieval-Augmented Generation app that lets you chat with your documents. Every response cites the exact passages it was drawn from — click a 1 mark to read the source in the margin.
Retrieval-Augmented Generation grounds language model answers in real documents. Rather than relying on training-time knowledge, the model reads the passages most relevant to your question before writing its response.
Documents are split into overlapping passages, converted to embedding vectors, and stored in a local SQLite database.
Your question is embedded with the same model, then ranked against every stored passage by cosine similarity.
The numbered source passages are injected into a system prompt, and Gemini Flash streams a cited answer token by token.
Click any node to inspect its inputs, outputs, configuration, and source file.
File.text()Raw UTF-8 string of extracted text, passed to the chunker.
getDocumentProxy(buffer) — loads PDF via PDF.js WASMextractText(pdf, {mergePages: true}) — joins all pagesPlain text string, whitespace-normalized.
start in TARGET-char windows\n\n (paragraph). / ? (sentence)Separator-based chunking keeps coherent semantic units intact — a sentence about one topic won't be split across chunks. The 200-char overlap ensures context isn't lost at boundaries and that retrieval can match a phrase even when it straddles a chunk edge.
Array of overlapping text strings, each ≤ 1400 chars.
Every vector is L2-normalized after embedding, so cosine similarity reduces
to a simple dot product at query time — no division needed.
Stored as Float32Array → BLOB in SQLite.
Float32Array per chunk, stored as raw binary BLOB.
documents — id, name, mime, size, chunk_countchunks — id, document_id, idx, content, embedding BLOBconversations — id, title, timestampsmessages — id, conversation_id, role, content, sources JSONZero external service — one file on disk. Prisma's type-safe client and migration system replace raw SQL with a schema that TypeScript understands end-to-end. The LibSQL adapter connects the Prisma 7 runtime to the local SQLite file.
Run npm run db:studio to open a GUI browser over your documents, chunks, conversations, and messages — great for debugging retrieval.
conversationId for follow-upsThe API receives the message and an optional conversation ID. If provided, the last 10 messages are fetched and passed to Gemini as conversation history so follow-up questions have context.
Using the matching task type is critical — the model produces embeddings optimized for the asymmetric query↔document retrieval use case.
RETRIEVAL_DOCUMENT produces embeddings that represent "what this passage
contains." RETRIEVAL_QUERY produces embeddings that represent "what
information I'm looking for." The model is trained so these two spaces align — without
matching task types, retrieval quality degrades significantly.
Brute-force JS dot products over all chunks. At 768 floats per chunk: a 500-document library (~2,500 chunks) processes in under 5ms. The bottleneck is always the Gemini embedding API call, not the local ranking.
Scores range 0→1. Displayed as % relevance in the sources panel. 0.35 eliminates clearly unrelated passages.
{
"type": "meta",
"conversationId": "…",
"title": "…",
"sources": [{ "n":1, "documentName":…,
"excerpt":…, "score":… }]
}
The meta event carries all retrieved sources before a single token
is generated. This means 1 citation marks in the streamed
answer are immediately clickable — the sources panel populates in full the moment
the response begins.
After streaming, only sources actually cited ([n] appears in answer)
are saved to the database.
[n], never invent
The last 10 messages from the current conversation are fetched from SQLite
and prepended as user/model turns so follow-up
questions like "what else does it say about that?" resolve correctly.
The first 64 chars of the first message become the conversation title — no extra LLM call needed.
{type:"meta", sources:[…]} — sources, conv ID, title{type:"token", text:"…"} — one per streamed chunk{type:"done", messageId:"…"} — signals completion{type:"error", message:"…"} — on any failure
A minimal markdown renderer (Markdown.tsx) handles inline
bold, italic, code, headings, and lists.
Citation marks n are rendered as interactive buttons
that open the relevant source passage in the margin panel.
Hover a card to see why each technology was chosen over alternatives.
ReadableStream support.runtime = "nodejs" routes allow the Prisma LibSQL adapter to connect to the local database, while the route handler's Response streaming maps perfectly to NDJSON event delivery.generateContentStream with a custom system instruction.prisma/schema.prisma) provides type safety, migration history, and a GUI via Prisma Studio — none of which raw SQL gives you.mergePages: true gives a single string to pass directly to the chunker.suppressHydrationWarning on the root html element, combined with next-themes' class strategy, avoids the hydration mismatch that naive localStorage theme solutions produce in SSR.@custom-variant dark for class-based theming via .dark selector.@theme inline block maps CSS custom properties to utility classes, letting the same token names work in both component styles and one-off utilities without duplication.The codebase is organized around three concerns: data access, API routes, and React UI.
All routes run on the Node.js runtime to allow SQLite and native modules. Chat and upload responses stream as NDJSON.
| Method | Route | Description |
|---|---|---|
| GET | /api/documents | List all indexed documents with id, name, size, chunk count, created date. |
| POST | /api/documents | Upload and index a document. Accepts multipart/form-data with a file field.NDJSON stream — emits stage, done, or error events. |
| DELETE | /api/documents/[id] | Remove a document and all its chunks (cascades via SQLite foreign key). |
| GET | /api/conversations | List conversations ordered by most recently updated. |
| GET | /api/conversations/[id] | Fetch all messages for a conversation with parsed sources arrays. |
| DELETE | /api/conversations/[id] | Delete a conversation and all its messages. |
| POST | /api/chat | Send a message. Body: { message, conversationId? }.NDJSON stream — emits meta (sources + conv ID) → token× N → done. |
Key constants that control retrieval quality, chunking behaviour, and model selection. All live as named constants near the top of their respective files.
gemini-embedding-001. Stored as raw Float32 BLOBs (3 KB/chunk).