---
title: "Canvas Progress"
canonical_url: "https://scholarxiv.com/developers/docs/canvas-progress"
markdown_url: "https://scholarxiv.com/developers/docs/canvas-progress.md"
---

> For the complete documentation index, see [llms.txt](/llms.txt).

# Canvas Progress
URL: /developers/docs/canvas-progress
LLM index: /llms.txt

# Canvas graph: progress log

One entry per stage of the plan in `docs/canvas-audit.md`.

**How each stage was checked:** with unit tests, `svelte-check` (filtered to the canvas files) and Prettier. The repo has about 136 type errors in unrelated files that predate this work; the canvas files add none. The dev server and the logged-in UI could not be exercised in this environment, so the UI acceptance checks still need a manual pass. The dev server currently stops at startup because `PUBLIC_GO_PRODUCT_ID` is missing from `.env`.

## Stage 1: Directional ports and connection rules

**What changed**
- New pure module `src/lib/canvas/graph.ts`:
  - `portsFor(type)` says which handles each node type has.
  - `checkConnection` rejects self-links, wrong direction (output → output, input → input, or into a node with no input), duplicates and loops. A loop is rejected with the message "That would create a loop".
  - `normalizeConnections` brings saved boards in line with these rules.
- **Handles:** each node shows an input on the left and an output on the right, and the top and bottom handles are gone. Note, Attachment, Paper and Clip have an output only. The AI node has one input that accepts many connections, plus one output. A drag can start from either handle: from an output it connects to the input of the node you drop on, and from an input it connects from that node's output.
- **Edges:** each edge runs from the right middle of its source to the left middle of its target, with the arrowhead on the target end as before.
- **Rejected connections:** the board checks the rules before calling the API and shows a toast that doesn't block anything. `createCanvasConnection` repeats the same check on the server.
- **Old canvases** (`getCanvasBoard`):
  - Every existing edge already stored its direction (`fromId → toId`) and none stored handles, so valid edges load unchanged.
  - AI → Note edges are kept, drawn dashed, and flagged `legacy` for Stage 3.
  - Other edges that break the rules are hidden from the view and logged as a warning, but not deleted.
  - Loops in saved data are broken at their newest edge. That edge is deleted, and a notice is stored on the canvas (`canvases.notices`) naming the two nodes and the reason. It shows as a banner until it's dismissed (`PATCH /api/canvases/[id] { dismissNotice }`).
- Deleted `src/components/canvas/__narrowing_repro.svelte`.

**Tests:** `src/lib/canvas/graph.test.ts` (10 tests) covers ports, loop detection, the connection rules, legacy edges, hidden edges and loop breaking.

**Skipped / open:** none. The two existing type errors in the AI node markup (payload narrowing inside closures) were left for Stage 2, which rewrites that block.

## Stage 2: Graph run engine

**What changed**
- **Pure engine** (`src/lib/canvas/engine.ts`): topological sort with loop rejection, Merkle-style input hashes, derived staleness, cache hits, run planning and a dependency-aware scheduler with a concurrency cap.
  - The scheduler is generic: it takes its execute step as a parameter, so tests swap in a fake.
  - A node's hash covers its type, its config (instruction and *requested* model) and the hashes of its non-empty inputs.
  - "Stale" is not stored: a done node is stale whenever its current hash differs from `runtime.inputHash`. So editing the first Note dims everything downstream immediately, and undoing the edit brings the nodes back to done.
- **Values and run state:** typed values and `NodeRuntime` live in `src/lib/canvas/values.ts`, as approved, plus two optional fields: `warnings` and `model` (the model that actually answered).
  - Run state is stored in `canvas_items.runtime`, separate from `payload`.
  - Older AI nodes show their `payload.result` as a finished run with no hash. They display as done, and the first Run all refreshes them once.
- **Server run** (`src/lib/server/canvas_run.ts`, `POST /api/canvases/[id]/run`): streams server-sent events (`plan`, `status`, `delta`, `runtime`, `cancelled`, `error`, `end`).
  - **What runs:** with no targets it's Run all. A single node's Run (or Retry) sends `targets` plus `force` for that node, so its upstream AI nodes run only when they aren't done or are stale.
  - Up to 3 nodes run at once.
  - **Model calls:** the model is resolved through the existing router (`resolveModel`) and called with `streamText`, so keys stay on the server.
  - **Stop** aborts the request, which cancels the node running at that moment and everything still queued.
  - **Failures:** a failed node blocks the nodes downstream of it. They go back to idle, marked "Upstream failed". Unrelated branches finish.
  - The old route `items/[itemId]/run` is removed.
- **Inputs:**
  - **Attachments** now send their extracted text through a new `extractAttachmentTexts` in `attachment_parts.ts`. It shares the PDF, DOCX and plain-text readers with chat; chat's output is unchanged.
  - **Context budget:** the selected model's context window from the model catalog sets the input budget, minus 8k tokens kept for the answer, and capped at 200k tokens.
  - **Over budget:** inputs are trimmed so short ones stay whole (`src/lib/canvas/prompt.ts`), and the node shows "Inputs truncated to fit the model's context: … (N% kept)".
  - **Empty inputs** (a blank Note, an upstream node with no output) are skipped. The node warns about them before you run it.
  - **Labels:** each input is labeled in the prompt (Note, Attachment: name, Paper: title, PDF clip from …, Output of "…").
- **Per-node model:** each AI node saves `model` and `modelName`. The node's picker is the existing `SelectModel`, given a proxy of the shared model state, so it reads and writes the node's own model. New nodes start from the app's current model.
- **UI:**
  - Run all and Stop buttons in the top bar.
  - Status on each AI node:
    - queued: grey border and "Queued"
    - running: green border, a spinner and the model name
    - done: a check and the run time
    - error: red border, the message and a Retry button
    - stale: dimmed, "Inputs changed"
  - Text streams into the node while it runs.
  - Pending text saves are flushed before a run starts.
- **Limits:** each model call is charged to the existing per-user rate limiter (`checkAndIncrementUserRateLimit` with the plan's `api.rateLimit`, the same pool as API and MCP requests). When it's exceeded, the node fails with "Rate limit exceeded. Try again in N minutes."

**Tests:**
- `engine.test.ts` (13): topological order, loop rejection, ignoring legacy edges, Run all order, cache hits (nothing re-runs), single-node runs with stale ancestors, stale propagation from a Note, staleness on instruction, model and connection changes, skipping blank inputs, error propagation that leaves unrelated branches running, the concurrency cap, dependency order, and Stop.
- `prompt.test.ts` (3).
- The full suite shows the same 10 failures before and after this stage, in `hooks.server` and the search route and util tests, all unrelated.

**Skipped / open**
- **Open question, chat usage budget:** chat has no usage budget in the codebase today. `/api/usage` only reports token totals, and nothing enforces a cap. So decision 5's "same budget as chat" has nothing to plug into yet. Canvas calls go through one function, `chargeModelCall` in `canvas_run.ts`, so a shared budget can be added in one place. The "same message chat shows" also has no chat equivalent; the node shows the rate limiter's message instead.
- **Images:** image attachments are passed as "(Image file: its contents cannot be read as text.)" rather than sent to vision models. Sending them as images would need the per-node model to support image input.
- Output nodes arrive in Stage 3, so Stage 2's Note → AI → AI → Output check was verified as Note → AI → AI, through the engine tests.

## Stage 3: AI output display and chaining

**What changed**
- **AI node output:** the result shows below the instruction in `canvas_result_panel.svelte`. It's a scrollable, rendered-markdown area with Copy and Expand (a dialog), on a light sky-tinted background with the caption "Output · model · time". The model in the caption is the one that actually answered, after Auto routing. While the node runs, the caption reads "writing…" and text streams in.
- **AI node titles:** each AI node has an editable title, used in downstream prompt labels, e.g. `Output of 'Extract methods'`. Every input is labeled by its node type and title (Note, Attachment: name, Paper: title, PDF clip from …, Output of '…').
- **Chaining:** AI → AI already worked through the engine. The upstream text is the downstream node's labeled input.
- **Output node** (tools panel → Output, type `output`): one input, no output, read-only.
  - It shows each connected input: text as markdown, papers as a table, files by name, lists as sections.
  - Copy, Download as `.md`, and Expand.
  - If an input is stale, it's dimmed and says so.
  - Live text shows while an upstream node is running.
- **Save to collection** goes through the existing Collections API and `collectionState`. You can add to an existing collection, or create one with a name you choose. It reports "Added N, M already there".
- **Notes stay user-only.** A Note with a legacy AI → Note edge shows a one-time hint. "Add Output node" places an Output node next to the Note and connects the same AI nodes to it. "Dismiss" hides the hint. Either way the Note's text is never touched, and the dismissal is stored as `outputHintDismissed` on the Note.
- New pure helpers:
  - `node_value.ts`: values, titles, and the papers upstream of a node.
  - `markdown.ts`: papers table, sections, file names.
  - `collections.ts`: saving papers to a collection.

**Tests:** `markdown.test.ts` (5): the papers table with escaping, list sections, section headings, file names, and collecting papers upstream. Canvas suite total: 31.

**Skipped / open**
- **Collections hold papers only, not text.** So "Save to collection" saves the papers the output came from: paper cards and, from Stage 4, Search results upstream of it. When there are none, the button is disabled with an explanation. Saving the Output's text itself would need a new kind of collection entry. I left that out, since you asked not to build new systems.

## Stage 4: Live Search node

**What changed**
- **New `search` node type** (tools panel → Search node). It runs in the graph like an AI node: it has a status, goes stale and is cached by input hash.
  - **Fields:** query, sources (default all), year range, max results (default 10, max 50) and sort (relevance or newest). Sources, years, results and sort sit behind a filters toggle.
  - **Validation:** payloads are checked by `parseSearchPayload` on create and update.
- **Optional input:** connected text from a Note, a clip or an AI output becomes the query, or extends it when the node has its own query. Each text input contributes at most 300 characters. The node shows the query it actually ran when that differs from the typed one.
- **Backend:**
  - Paid plans: `runFederatedSearch`, run server-side inside the run engine.
  - Free plans: arXiv only, through the existing `searchPapers` mapped with `arxivToFederated`. The node shows "arXiv only on your plan · Upgrade for all sources", linking to `/pricing`.
  - Either way the results are a typed `papers` value with namespaced ids.
- **Results list:** title, first author and year, with a "N of M selected" count.
  - Unticking a paper stores its id in `excluded`. Excluded papers are removed from everything downstream: the server's input gathering, Output nodes and paper collection.
  - `excluded` feeds the hash that downstream nodes see, not the Search's own (`outputModifier` in the engine). So unticking a paper re-runs only what comes after the Search.
- **Static paper cards still work.** "+" on a Search result places that paper as a card, and the tools panel's "Find a paper" (the old one-shot search) still adds cards. Cards now store the abstract and output `papers` of length 1.
- **Papers into AI nodes:** each paper becomes its own labeled input ("Paper: title") with title, authors, year, URL and abstract.
  - **Full text** comes from the existing `getPaperFullText` (arXiv HTML/PDF, or a federated paper's PDF, with its cache) when 8 or fewer papers feed the node. It is fetched 4 at a time.
  - **Over budget:** the context-budget trimming cuts the longest texts first and says so on the node.
  - **Notes on the node:** "Full text was available for N of M papers", or "Using abstracts for all N papers" above 8.
- Run all and Run status now cover Search nodes. The Run, status, error/Retry and warning markup is shared in `canvas_run_status.svelte`.

**Tests:**
- Engine (3 new): Search runs before its consumer; a query or filter change stales the Search and what follows it; excluding a paper stales only the downstream node and re-runs only that.
- `search_payload.test.ts` (2).
- Canvas suite total: 36.

**Skipped / open**
- **Search rate limits:** Search runs aren't charged to the model-call rate limiter, since they aren't model calls. The browser search limiter (`checkBrowserRateLimit`) is keyed by IP address and needs the request event, so it isn't applied inside the run engine either.
- **"From Library":** the canvas Library lists uploaded files, not papers, so there was no Library paper to place as a card. Library files stay Attachment nodes.

## Stage 5: For-each on AI nodes

**What changed**
- **"Run for each item" toggle on AI nodes** (`payload.forEach`, part of the node's hash).
  - It appears when exactly one input is list-like: a Search or other `papers` value with more than one paper, or a `list` from another for-each. A Search that hasn't run yet counts, so the pipeline can be wired up before the first Run all.
  - With the toggle on and no list, or several lists, the node explains what to connect.
- **Execution** (`executeForEach` in `canvas_run.ts`):
  - The instruction runs once per item, 3 at a time inside the node. Each call goes through the same rate-limited `callModel`.
  - All other inputs are passed to every call as shared context.
  - Each paper item gets its full text when available (for-each reads full text for up to 100 items, since each call holds one paper). Each call is fitted to the context budget separately.
  - The output is a `list<text>`. Each item is keyed by its source (the paper id) and labeled by the paper title.
- **Per-item cache:**
  - Each item's hash covers the node's config, the shared inputs and the item itself, stored in `runtime.items`. A re-run reuses any item whose hash matches and that didn't fail.
  - So retrying re-runs only the failed items, and a Search that now returns one new paper costs one call. The node notes "Reused N unchanged items".
  - A node with failed items is never a cache hit.
- **Upstream output changes now reach downstream nodes** (needed for the per-item cache):
  - Every run stores an `outputHash`, which feeds its consumers' hashes. If an upstream re-run produced something different, consumers go stale.
  - Consumers of a node that will run are planned too. The server re-checks the cache once their inputs are final and skips them (a `skipped` event) if nothing changed.
  - Each stored `inputHash` is computed when the node finishes, from the upstream results it actually read.
- **UI:**
  - "7 / 12 done" with a progress bar, then a collapsible list keyed by paper title. Items stream in as they finish.
  - A failed item shows its error and a Retry button, which re-runs only the failed items.
  - The list sits in the tinted output panel, with Copy and Expand on the whole list.
- **Downstream:** an AI node that receives a `list<text>` gets every item as its own labeled input ("Output of 'Extract' · Paper title"). The Output node renders a list as sections. When every item uses the same headings ("## Method", "**Dataset:** …" or "Main result: …"), it offers a Table view with one row per item (`listAsTable`).
- **Confirmation:** before any run (Run all or a single node), the client estimates model calls with the engine's own `planRun`: one per AI node, or one per item for a for-each node. An un-run Search counts at its max-results setting. Above 20 calls, the user has to confirm.

**Acceptance pipeline:** Note (topic) → Search (10 papers) → AI for-each ("extract method, dataset, main result") → AI ("build a comparison table and summarize trends") → Output runs with one Run all. The Note's text becomes the Search query, the for-each makes 10 calls, and the summary node receives 10 labeled items. It's an estimated 11 calls, so no confirmation is needed. This was checked through the engine and helper tests; the live run is part of the manual pass.

**Tests:**
- Engine (3 new): consumers of a forced re-run are planned; a changed upstream output stales its consumers; failed for-each items prevent a cache hit.
- `for_each.test.ts` (3): what counts as list-like, an un-run Search as a list input, and the call estimate.
- `markdown.test.ts` (3 new): the headings table, multi-line fields, and mismatched headings.
- Canvas suite total: 45.

**Skipped / open**
- **Concurrency:** for-each concurrency (3) applies inside a node. Two for-each nodes on independent branches can therefore make up to 6 calls at once, within the engine's 3-node cap. Every call is still charged to the rate limiter.
- **Confirmation dialog:** this uses the browser's `confirm()`. It's simple and blocking; a styled dialog can replace it in Stage 6.

## Stage 6: Polish, saving and templates

**What changed**
- **Autosave (debounced and batched):**
  - Every item edit (moves, note text, AI and Search settings) updates the board at once and joins a per-item queue. Edits to the same item are merged.
  - After 700 ms of quiet, the queue goes out as one `PATCH /api/canvases/[id]/items { updates: [...] }`, which validates each update with the shared `parseItemPatch` and writes them in one `bulkWrite`. Viewport changes are queued in the same way.
  - The top bar shows Edited / Saving… / Saved, or "Not saved · retrying". A failed batch goes back underneath any newer edits and is retried after 3 s.
  - Runs flush the queue first. Closing the page sends what's left with `keepalive`, and the queue is also flushed when you leave the canvas.
- **Rename:** click the title in the top bar, as before. Canvases are listed and reopenable on `/canvas`, as before.
- **Reload restores outputs:** every node's last output and status are saved in `runtime` (Stage 2), so reloading brings back the whole board with its results.
- **Run history:** each run is stored in `canvas_runs` (indexed, last 10 kept per canvas, deleted with the canvas).
  - Each record holds the start and end times, every planned node with its outcome (done, error with message, blocked, cancelled or skipped), the model calls used, whether the run was stopped, and the output each node produced.
  - The top bar's history menu lists the runs. Picking one shows that run's outputs on every node, and "Back to latest outputs" undoes it.
  - Each AI and Search node also has its own "Latest output / Run of …" picker. Viewing an earlier output never changes what downstream nodes receive.
- **Templates:** "Start from template" on `/canvas` (`POST /api/canvases { template }`, built on the server in `createCanvasFromTemplate` from `src/lib/canvas/templates.ts`).
  - **Literature review:** the Stage 5 pipeline, with a sample topic Note.
  - **Compare two papers:** two example arXiv cards (Attention Is All You Need, BERT) → AI compare → Output. Swap in your own cards.
  - **Summarize my uploads:** up to 5 of your most recent non-image Library files → AI for-each summarize → AI synthesize → Output. With fewer than two files, a Note explains how to add them.
  - To support this template, "Run for each item" now also iterates when a node has no list input but two or more attached files (on both server and client).
- **Keyboard:** Cmd/Ctrl+Enter runs the selected node, Shift+Cmd/Ctrl+Enter runs all, and Delete/Backspace removes the selection when no text field has focus. The listener uses the capture phase, so the run shortcuts work from inside an instruction box too.
- **Empty state:** "Add a Search node or a Note, then connect it to an AI node", plus the Run all shortcut and a pointer to templates.
- **Performance with 50 nodes:**
  - Input hashes are now memoized on each item's payload and runtime, plus the connection list. Dragging a node replaces only its item object, so a drag no longer re-hashes the board on every pointer move.
  - Saves are batched, as above.
  - Engine test: hashing and planning a 50-node graph takes well under 50 ms.

**Tests:** `templates.test.ts` (7):
- every template is a valid graph: no hidden edges, no loops, every edge accepted by `checkConnection`, and something for Run all to run
- the literature-review run order
- uploads iterated as a list
- the empty-Library hint
- the 50-node timing

Canvas suite total: 52.

**Skipped / open**
- **Manual checks still needed:** "All three templates run successfully with Run all" was checked as graph structure and run plans. A live run needs a model, and the uploads template needs Library files.
- **Large confirmations:** these still use `window.confirm` rather than a styled dialog.
- **50-node frame rate:** this was reasoned about and covered by the engine timing test, not profiled in a browser.

## Stage 7: Extract to table node

**What changed**
- **New `extract` node** (tools panel → Extract to table). It takes papers from a Search, paper cards or (Stage 8) a Checkpoint. Any other connected inputs, such as a Note describing the study, are passed as extra context to every row.
- **Columns** are edited in the node: a name, plus a short description of what to extract.
  - The 4 starter columns (Method, Dataset, Main result, Limitations) can be edited or deleted. Up to 12 columns.
  - Settings are validated by `parseExtractPayload`.
  - The node has its own model picker, sharing `nodeModelPicker` with AI nodes (`src/lib/canvas/model_picker.ts`).
- **Execution** (`executeExtract` in `canvas_run.ts`):
  - One model call per paper, 3 at a time, through the same rate-limited call as for-each. The model is asked for JSON keyed by column id: `{ value, snippet }` per cell.
  - The system prompt forbids guessing: unsupported cells must be exactly "not reported".
  - Answers are validated by `parseExtraction` (code fences and stray text around the JSON are tolerated). A missing key, wrong shape or invalid JSON triggers one retry. If that fails too, the row is marked failed with the reason, and Retry re-runs only failed rows.
  - Blank values and "Not reported." are normalized to "not reported", with no snippet.
- **Sources:** the abstract plus full text when available, via the same `getPaperFullText`, for up to 100 papers. Each row records whether it was extracted from full text or the abstract only. The node notes "N of M rows were extracted from the abstract only".
- **Output is a typed `table` value** (`columns` + `rows`, each row linked to its paper id), now part of the value types in `values.ts`.
  - Downstream AI nodes get it as a markdown table.
  - The Output node shows a sortable table (rows that are "not reported" or failed always sort last), with Export CSV (UTF-8 with BOM, quoted, with paper id and "extracted from" columns), Copy and Download .md.
- **In the node:**
  - A preview of the first 5 rows, with an Expand dialog for the full table.
  - Hovering a value shows the sentence it came from.
  - Rows from the abstract only are marked.
  - Rows progress live ("7 / 10 rows").
  - Per-node history (Stage 6) also works here.
- **Caching: per cell.** I picked per cell over per row because, with the for-each machinery already in place, it was barely more code.
  - Each cell's hash covers the paper (id, title, abstract), that column's name and description, the shared context and the model. Hashes are stored in `runtime.cells`.
  - On a re-run, each row's single call asks only for the columns whose cells aren't cached. So editing one column's description extracts only that column, adding a column extracts only the new one, and unchanged rows make no call at all.
  - The node itself goes stale on any column change, because its config hash includes the columns.

**Tests:**
- `table.test.ts` (8): parsing fenced JSON and snippets; normalizing "not reported"; invalid JSON and missing columns; the instruction asking only for the requested keys; markdown without failed rows; CSV quoting and provenance; settings validation (12 columns at most, names required, starter columns by default); and the node going stale on a column description change.
- Canvas suite total: 60.

**Skipped / open**
- **Live run not done:** the acceptance run (Search with 10 papers → Extract → Output, giving 10 rows with CSV) needs a model and wasn't run here. The parsing, validation, caching and rendering behind it are covered by tests.
- **"Library" papers:** as in Stage 4, there are none to connect. Library items are files, not papers.

## Stage 8: Checkpoint node

**What changed**
- **New `checkpoint` node** (tools panel → Checkpoint), with one input and one output. It passes its input through unchanged, except for what the user edits.
- **Engine:**
  - A new `waiting` status.
  - A Checkpoint's executor returns `'waiting'`. The scheduler then marks everything below it in the plan as `held`: not run, and not marked as failed. Independent branches keep running.
  - A waiting Checkpoint is never a cache hit, so the next Run all reviews it again with fresh data.
- **Review UI:**
  - Papers, table rows and list items are shown as a checkbox list with "8 of 12 selected" and All / None.
  - Text gets an editable text area.
  - The node and its border turn amber while waiting.
- **Continue** (`POST /api/canvases/[id]/checkpoints/[itemId] { action: 'continue' }`):
  - Flushes pending saves first.
  - Checks that the data above hasn't changed since the pause. If it has, it asks you to run again to review the latest.
  - Marks the Checkpoint done, storing its applied output, input hash and output hash.
  - Returns the held nodes, which the client then runs as targets. Their ancestors are cached, so only the held steps run.
- **Stop here** (`{ action: 'stop' }`) ends the paused run. The node goes idle with "Run stopped at this checkpoint", and the data is kept so it can be reviewed later.
- **What downstream nodes receive:** `visibleOutput` gives nothing while the Checkpoint waits, then its input with the review applied (`applyCheckpoint`).
  - Unticked items are stored by id in `payload.excluded`. They change the hash consumers see (as `outputModifier`, like Search exclusions), so edits made after Continue reach downstream on the next run.
- **When upstream data changes later:** the Checkpoint goes stale and reviews again on the next run.
  - Exclusions are re-applied wherever the same paper ids come back.
  - A text edit is tied to the input hash it was written against (`editedFor`). If the text above changes, the edit isn't applied, and the node says so rather than silently dropping it.
- **Several Checkpoints:** each one holds only its own descendants. The top bar shows "Waiting for review (N)", and clicking it pans to the first waiting Checkpoint and selects it. Held nodes show "Paused for review above".
- **Reload:** the waiting state, the data under review (`runtime.pending`) and the held nodes (`runtime.heldDownstream`, written when the run ends) are all stored on the Checkpoint. A reload during a pause shows the same waiting Checkpoint, and Continue still resumes the held steps.

**Tests:** `checkpoint.test.ts` (7):
- unticked papers and rows dropped by id
- exclusions re-applied to refreshed data
- text edits applied only to their own input
- tickable items listed
- the scheduler holding everything below a waiting Checkpoint while an unrelated branch finishes
- waiting persists through `displayStatus`, passes nothing downstream, and is never cached
- reviewed papers pass downstream after Continue

Canvas suite total: 67.

**Skipped / open**
- **Acceptance check:** the pipeline (Search → Checkpoint → Extract: untick 3, Continue, table has only the kept papers) is covered by the engine and helper tests. The live run needs a model.
- **Housekeeping:** I accidentally ran Prettier over all of `src/lib/server` while working on this stage. I reverted every file except the two canvas server files before committing.

## Stage 9: Send-to nodes (Write, Collection)

**How Write stores documents (read first, as asked):**
- Write is a LaTeX editor. A project is a `paper_projects` row with a `rootFile` (`main.tex`). Its files are `paper_files` rows holding text content. They're compiled in a sandbox, and the editor saves files through `PATCH /api/write/[projectId]/files`.
- **Citations** are standard BibTeX. Every Write template ships a `refs.bib` and ends with `\bibliographystyle{plainnat}` and `\bibliography{refs}`, using natbib (`\usepackage[numbers]{natbib}`).
- **Completions:** the editor's `\cite`/`\citep`/`\citet` completions (`src/lib/write/completions.ts`) offer every key found in any `.bib` file in the project.
- **Existing BibTeX generator:** the app already has one for papers, `formatExternalCitation('bibtex', …)` in `src/lib/federated/citation.ts`.

**How canvas content maps into Write** (`src/lib/canvas/latex.ts`):
- **Where it goes:** each Send to Write node owns one section file in the project, `canvas/<section-slug>-<node>.tex`. It's pulled into the document with one `\input{…}` line, added once to the root file before the bibliography (or before `\end{document}`). The user's own text in `main.tex` is never rewritten.
- **The section file:**
  - A `\section{…}` heading.
  - A visible line linking back to the canvas: *From the ScholarXIV canvas "…" (scholarxiv.com/canvas/…)*. The full URL is also in a comment.
  - The content.
  - A "Sources: `\cite{…}`" line listing every paper it drew on.
- **Text** (AI output markdown) → LaTeX:
  - headings become `\subsection`/`\subsubsection`/`\paragraph`, nested under the section
  - **bold**/*italic* → `\textbf`/`\emph`, and `code` → `\texttt`
  - links become the link text with the URL in `\texttt`
  - bullet and numbered lists → `itemize`/`enumerate`
  - fenced code → `verbatim`
  - markdown pipe tables → booktabs tables
  - all LaTeX specials are escaped
- **Tables** (Extract to table) → a booktabs `table` with wrapping `p{}` columns. Each row's paper label carries `~\cite{key}`, and unsupported cells read "not reported".
- **Papers** → an `itemize` list of titles, each with `~\cite{key}`.
- **Citations:**
  - Every paper in the content, and every paper upstream of a text result, gets a BibTeX entry built with the existing `formatExternalCitation('bibtex', …)`. It's added to the bibliography file the root file actually reads (`\bibliography{x}` or `\addbibresource{x.bib}`, otherwise `refs.bib`).
  - Citations use `\cite`, which works with natbib, biblatex and plain BibTeX.
  - The canvas marks each entry it writes with `% scholarxiv-canvas-paper:<id>`. Later sends reuse those keys instead of adding duplicates, and new keys never collide with the user's own (a/b/… suffixes).
  - The keys then appear in Write's own `\cite` completions.
  - A project with no bibliography command gets `\bibliographystyle{plain}` and `\bibliography{refs}`, so the citations resolve.
- **New documents** get a small `main.tex` (article, geometry, booktabs, array, natbib) that inputs the section, plus a `refs.bib`.

**What changed**
- **Send to Write** (`send_write`, input only; `src/lib/server/canvas_send.ts`):
  - It accepts text, tables, papers and lists.
  - The target is either "A new Write document" (created on the first send; later sends update that same document) or "Append to …" an existing project.
  - The node shows the destination's name, an Open button (`/write/<id>`), the last send time and the result ("Created …", "Updated …, adding N references", "… is already up to date").
- **Send to Collection** (`send_collection`, input only):
  - It accepts papers, or a table (using each row's linked paper id).
  - It adds them to a chosen collection, or creates one with the name you give.
  - It skips duplicates and reports "Added 7, 3 already there" (plus "N not found" for papers missing from the catalogue).
  - Open goes to `/collections/<id>`.
- **Reused, not duplicated:**
  - The add-paper and create-collection logic moved out of the Collections API routes into `src/lib/server/collection_papers.ts`. The routes now call it too, with the same responses and access rules: only owners and editors can add.
  - Project creation moved from `POST /api/write` into `createProject` in `src/lib/server/write/projects.ts`, which the route also uses.
- **Idempotent:** both nodes are runnable and take part in Run all.
  - Unchanged input is a cache hit, so nothing is sent again.
  - When a re-run does happen, each node remembers where the last send went (`runtime.sent`), so it never creates a second document or collection. Unchanged section text isn't rewritten, existing BibTeX entries aren't duplicated, the `\input` line is added only once, and papers already in the collection are skipped.
- **Edits made in Write since the last send:** the node stores a hash of the section file as it wrote it. If the file differs on the next send, the node stops with "… was edited in Write since the last send". **Overwrite** re-runs it with permission to replace the file (`overwrite` on the run request). **Keep my edits and open** leaves the file alone.

**Tests:** `latex.test.ts` (10):
- markdown headings, emphasis, lists and escaping
- pipe tables → booktabs
- verbatim code
- every LaTeX special escaped
- BibTeX keys reused on later sends and never colliding with the user's keys
- table rows cited, with "not reported" cells
- `\input` added once, before the bibliography
- finding the bibliography file
- the section file is deterministic for the same input
- a sent node is a cache hit, so Run all twice doesn't send again

Canvas suite total: 77.

**Skipped / open**
- **Not tested live:** the acceptance runs (Extract → Send to Write creating a document with the table and citations; Run all twice creating nothing new) need a database and a signed-in session, so they weren't run here. They're covered by the conversion and caching tests above. A LaTeX compile of the generated section wasn't run either.
- **No cite keys inside prose:** model-written text isn't rewritten to insert `\cite` at the exact sentence where a paper is discussed. Papers are cited in the "Sources" line, and per row in tables.

## Follow-up: collections take papers, not text results

- **Changed:** collections are groups of papers on a topic, so a text result no longer offers to save to one. This replaces the Stage 3 note that the Output node saves "the papers the output came from".
  - The Output node shows **Save to collection** only when what it displays is papers (Search results, paper cards) or an Extract table. An Output showing an AI summary has no button.
  - **Send to Collection** collects only the papers in its own input (`papersIn` in `src/lib/canvas/papers.ts`). Connected to a text result, it stops with "Collections hold papers…".
  - **Send to Write** is unchanged: a text section still cites the papers behind it (`citedPapers`).
- **Tests:** `src/lib/canvas/papers.test.ts` (3 tests). All 80 canvas tests pass.

## Sitemap

See the full [sitemap](/sitemap.md) for all pages.
Docs-scoped sitemap: [/docs/sitemap.md](/docs/sitemap.md).
Well-known sitemap: [/.well-known/sitemap.md](/.well-known/sitemap.md).