Skip to content

Remine

The deterministic matrix router is literal: it fills a cell only from an exact-tag match or spine-side membership. A note that genuinely belongs to a dimension but was never tagged for it is simply invisible to the grid. Remine is the agent-semantic tier that closes that gap — a note-centric, batched LLM classifier that reads a row's scoped notes and the schema's dimensions and returns, per note, the dimension keys it belongs to plus a verbatim quote for each assignment. It composes with the deterministic tier; it never replaces it.

Why it exists

A grid built from tags alone under-reports. Notes accrete meaning faster than anyone tags them, so the honest membership of a dimension is almost always larger than its exact-tag set. Remine reads meaning, not surface keywords: it judges each note against the dimension menu and, crucially, never forces a fit — a note that matches nothing is surfaced, not shoehorned. The output is not authoritative on its own; it is a proposal an author reviews before anything is written.

The stages

Remine is a ladder of capabilities, each opt-in and each degrading safely to the one below it.

INDUCE synthesize columns from corpus + intent PROPOSE classify notes into a grid + residual bucket SUBFACETS sub-cluster a column into children (depth ≥ 1) RECONCILE reuse prior dimension keys by member overlap APPLY promote approved grid → spine-member edges writes
A ladder of opt-in capabilities, each degrading safely to the one below. Only APPLY writes to the corpus; every stage above it is a read-only proposal.

INDUCE (induce_dimensions()) handles the no-schema case. Given an intent like "make a matrix for X" and no fixed columns, it reads a compact digest of the corpus and proposes a small set of orthogonal facet columns, optionally grounded by embedding clustering so the facets reflect real attribute groupings rather than the agent's priors alone. Induced columns are flagged so the UI can show they were synthesized.

PROPOSE (propose_remine()) is the read-only workhorse. It gathers one cell per dimension over the row-axis scope with the classifier enabled, so each cell carries its deterministic members plus the classifier's inferred members (provenance reviewer-inferred, with confidence and a verified verbatim quote). Alongside the grid it returns a residual bucket: the in-scope notes that landed in no dimension and were not already tagged. Nothing is written.

SUBFACETS (induce_subfacets(), depth >= 1, induced columns only) grows a bounded subtree under an emergent column when its cell notes fall into cohesive sub-clusters. Cohesion is the required signal — with no embedding index there is no basis to claim a column splits, so it stays flat; a sufficient-clusters gate keeps noise from sprouting a spurious tree. Survivors are named into child facets through the same call/parse/slug seam as induction, and recursion is bounded by depth (Stage 4: recursive emergent depth (sub-cluster detector + matrix-collapse)). This is the depth complement to emergent rows, the earlier stage that made rows themselves inducible (Stage 2 done: emergent rows (hybrid embeddings-propose + agent-name)).

RECONCILE keeps identity stable across re-mines. When a spine definition already has a promoted spine, read_prior_members() reads each dimension node's current spine-member membership, and the induced facets are matched against it by member-set overlap (via the shared reconcile_ids primitive; see matrices). A facet whose members overlap a prior dimension reuses that prior column key, so an agent rename or reorder does not orphan the durable node or its edges. Greenfield (no prior spine) mints every key fresh, byte-identical to a first run (Optimal matching + dashboard reconcile parity (closes the two follow-ups)).

APPLY (apply_remine()) is the only stage that writes. It takes an approved proposed grid and materializes it through the existing promote-attach seam — promote_organization(..., proposed_grid=...) — so there is no bespoke edge-writer. Membership is reconciled, not add-only: a re-run prunes members no longer classified and adds new ones, and it is idempotent. The optional tag_stamp is the one opt-in that mutates base notes, stamping each dimension's tag onto its classified members so a future deterministic build routes them without the classifier.

BACKFILL (backfill_spine()) is the degenerate case, used internally by the dashboard rather than exposed as an MCP action. When both rows and columns are already fixed — a persona spine with a reverse-engineered schema — induction collapses to pure classify-and-attach: the classifier runs over the persona's own source-note scope and the results attach additively onto the existing dimension nodes through the same promote path, preserving each node's ported synthesized body.

The residual bucket

Notes that fit no dimension are surfaced, never dropped. This is a hard requirement: silently discarding an in-scope note would violate the corpus's no-silent-loss guarantee. The residual (also carried as unassigned) is the set of in-scope notes that landed in no cell — neither deterministic nor agent routing claimed them — computed straight off the gathered grid. The dashboard turns it into an actionable "needs attribution" surface where each leftover note can be assigned to its rightful owner, and those decisions persist as corrections so nothing is lost across rebuilds (Render residual fix: EXCLUDE ambiguous rows + actionable per-row attribution (corrections overlay), not drift-flag-only).

Cost discipline

The classifier is careful about how many notes reach the LLM, in strict order:

  1. Exact-tag short-circuit — a note already carrying a dimension's tag is already routed deterministically and is never re-sent to the LLM.
  2. Embedding pre-filter — an optional embedding index gates which remaining notes reach the LLM: only notes near a dimension's deterministic centroid (or a tagged seed) survive. The threshold is deliberately loose — recall matters more than precision, because the LLM is the precision step — and a note with no vector is retained rather than dropped.
  3. One batched call — the surviving candidates are classified against the whole dimension set in a single LLM call, and the parse is gated so an assignment is accepted only for a note that actually survived that dimension's pre-filter.

Constraints

Two rules keep remine safe. First, induced grids are preview-only. remine(action="apply") refuses an induced (intent-proposed) grid: its columns are synthesized from the corpus, not the spine definition's fixed schema, so promoting one onto a spine definition with a different schema would corrupt it. The guard lives where the row cells are consumed, so it cannot be bypassed by stripping the grid's columns — a grid whose row cells carry any key absent from the target spine definition's fixed schema is also refused. A legitimate registry-schema grid, whose cell keys are exactly the spine definition's column keys, applies unchanged.

Second, apply is the only writing action. propose is always available and writes nothing; apply is refused at runtime when the server is write-free. This is the approve-to-apply gate: propose, review, then apply.

For how the grids remine produces are rendered and compared, see matrices; for the promotion machinery apply reuses, see Spines → The spine definition.

Design notes

The decisions behind this, drawn as a slice of the memory tree.

  • D Stage 2 done: emergent rows (hybrid embeddings-propose + agent-name)active
    Graph 2 (run-fe52a3e5).
  • D Stage 4: recursive emergent depth (sub-cluster detector + matrix-collapse)active
    Stage 4 adds recursive emergent DEPTH to re-mining: a deterministic, cohesion-gated sub-cluster detector that grows a bounded subtree under an emergent column, plus matrix-collapse polish.
  • D Render residual fix: EXCLUDE ambiguous rows + actionable per-row attribution (corrections overlay), not drift-flag-onlyactive
    Critic proved (P0) the drift-flag-only render fix leaks: build path (_target_match) was unchanged, so a foreign blank-edge colliding row still renders; regenerate rebuilds the same leaking grid and self-clears the stale …
  • D Optimal matching + dashboard reconcile parity (closes the two follow-ups)active
    Resolved both follow-ups left from the identity-stability hardening.
  • R Angelo
    • P zettelkastenactive
      • P Phase 2: Literature Review Systemactive
        • P Workshop tab: Reading Syllabus (learn) + claim-centric Review/Outline (write)active
          • P Organizations: project-level lenses + spines (matrix builder redesign)active
            • P Port legacy structures (default views + extraction spines) into the spine directoryactive
              • P F follow-up: close 2 deferred blank-edge bare-id residuals (cross-project port + render leak)active
                • D Render residual fix: EXCLUDE ambiguous rows + actionable per-row attribution (corrections overlay), not drift-flag-onlyactive
            • P Re-mining: Schema-Aware Structure Induction over existing notes (constraint lattice)active
              • D Stage 2 done: emergent rows (hybrid embeddings-propose + agent-name)active
              • D Stage 4: recursive emergent depth (sub-cluster detector + matrix-collapse)active
                • N Stage 4 review/critic findings — fixed + re-verified
                • N Stage 4 known P2 follow-up (not fixed) — see todo-4ea7c2bb
                • C Re-mining build COMPLETE — all 4 stages shipped + verified (Stage 4 closing checkpoint)
              • P Hardening: identity-stability invariant (overlap reconcile)active
                • D Optimal matching + dashboard reconcile parity (closes the two follow-ups)active

Semantically related entries from the memory graph.