zettelkasten.dashboard.backend.routes.projects

zettelkasten.dashboard.backend.routes.projects

Project CRUD, connect, cross, project-graph, search, and framing routes.

Split out of the former flat routes.py; behaviour is unchanged.

ProjectDrilldownRequest

Bases: BaseModel

POST body for a project drill (Approach B: membership in the payload).

Source code in zettelkasten/dashboard/backend/routes/projects.py

class ProjectDrilldownRequest(BaseModel):
    """POST body for a project drill (Approach B: membership in the payload)."""
    group_id: str
    # The drilled glyph's EXACT build-time member id set, carried in the LOD payload
    # and echoed back here so drill resolution needs NO server-side store lookup.
    member_ids: "list[str] | None" = None

get_federation

get_federation() -> dict

Configured federated repos plus non-fatal validation errors.

The UI uses this to group/badge federated (read-only) graphs and to surface misconfigured repos. Federation is configured by hand-editing a federated_repos list in .zettelkasten/config.yaml.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.get("/federation")
def get_federation() -> dict:
    """Configured federated repos plus non-fatal validation errors.

    The UI uses this to group/badge federated (read-only) graphs and to surface
    misconfigured repos. Federation is configured by hand-editing a
    ``federated_repos`` list in ``.zettelkasten/config.yaml``.
    """
    repos, errors = federation.configured_repos()
    return {
        "repos": [
            {"id": r.id, "name": r.name, "path": str(r.path), "read_only": True}
            for r in repos
        ],
        "errors": errors,
    }

create_project_route

create_project_route(req: CreateProjectRequest) -> dict

Create a new LOCAL project, optionally as the UNION of selected source graphs and/or existing projects.

sources/cross are taken verbatim; each id in projects is resolved to its manifest and its sources/cross folded in. Every ref must be LOCAL — a federated (repo:-namespaced) ref is rejected up front, because there is no federated write path and a union can only be written to the local store. De-duplication is order-stable (first occurrence wins). Mirrors the MCP create_project tool: a slug collision maps to 409, an invalid name to 400.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/projects")
def create_project_route(req: CreateProjectRequest) -> dict:
    """Create a new LOCAL project, optionally as the UNION of selected source
    graphs and/or existing projects.

    ``sources``/``cross`` are taken verbatim; each id in ``projects`` is resolved
    to its manifest and its ``sources``/``cross`` folded in. Every ref must be
    LOCAL — a federated (``repo:``-namespaced) ref is rejected up front, because
    there is no federated write path and a union can only be written to the local
    store. De-duplication is order-stable (first occurrence wins). Mirrors the MCP
    ``create_project`` tool: a slug collision maps to 409, an invalid name to 400.
    """
    import zettelkasten.server as zk_server

    if zk_server._WRITE_DISABLED:
        raise HTTPException(
            status_code=403,
            detail="Graph writes are disabled (ZK_DISABLE_WRITE); this action is unavailable.",
        )

    def _reject_ns(kind: str, ref: str) -> None:
        if federation.split_namespace(ref) is not None:
            raise HTTPException(
                status_code=400,
                detail=f"Federated (read-only) {kind} '{ref}' cannot be part of a new project.",
            )

    sources: list[str] = []
    cross: list[str] = []

    def _add(seq: list[str], value: str) -> None:
        if value and value not in seq:
            seq.append(value)

    for s in req.sources:
        _reject_ns("source", s)
        if not str(s).startswith("_"):
            _add(sources, s)
    for c in req.cross:
        _reject_ns("cross note", c)
        _add(cross, c)

    for pname in req.projects:
        _reject_ns("project", pname)
        _local, repo = _resolve_ref(pname)
        if repo is not None:
            raise HTTPException(
                status_code=400,
                detail=f"Federated (read-only) project '{pname}' cannot be part of a new project.",
            )
        pdata = _load_project_or_404(pname)
        for s in pdata.get("sources", []) or []:
            if not str(s).startswith("_"):
                _add(sources, s)
        for c in pdata.get("cross", []) or []:
            _add(cross, c)

    result = json.loads(
        zk_server.create_project(
            name=req.name,
            description=req.description,
            sources=sources,
            cross=cross,
        )
    )
    err_type = result.get("type")
    if err_type == "AlreadyExists":
        raise HTTPException(status_code=409, detail=result.get("error", "Project already exists."))
    if err_type == "ValidationError":
        raise HTTPException(status_code=400, detail=result.get("error", "Invalid project name."))
    if result.get("error"):
        raise HTTPException(status_code=400, detail=result["error"])

    # Return the freshly-written manifest so the client can select it immediately.
    return _load_project_or_404(req.name)

connect_project

connect_project(name: str, req: ConnectProjectRequest) -> dict

Connect a project's sources to one another (LOCAL-only).

A freshly-unioned project has no edges between its newly-combined sources — this creates cross-source structure. Two modes:

• mesh: author direct related edges between semantically similar notes from different sources (dense mesh). Fully previewable via dry_run and reversible (every edge is stamped origin='link-mesh').
• hubs: concept-hub synthesis. dry_run returns cross-source cluster suggestions; apply mints one landscape _cross hub per new_hub cluster (idempotent, ownership-safe via materialize_theme).

Federated (read-only) projects and the global write barrier are refused up front, exactly like every other project write.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/projects/{name}/connect")
def connect_project(name: str, req: ConnectProjectRequest) -> dict:
    """Connect a project's sources to one another (LOCAL-only).

    A freshly-unioned project has no edges between its newly-combined sources —
    this creates cross-source structure. Two modes:

    - ``mesh``: author direct ``related`` edges between semantically similar notes
      from different sources (dense mesh). Fully previewable via ``dry_run`` and
      reversible (every edge is stamped ``origin='link-mesh'``).
    - ``hubs``: concept-hub synthesis. ``dry_run`` returns cross-source cluster
      suggestions; apply mints one landscape ``_cross`` hub per ``new_hub``
      cluster (idempotent, ownership-safe via ``materialize_theme``).

    Federated (read-only) projects and the global write barrier are refused up
    front, exactly like every other project write.
    """
    import zettelkasten.server as zk_server

    _reject_federated_write(name)
    _load_project_or_404(name)

    mode = (req.mode or "mesh").strip().lower()

    if mode == "mesh":
        return json.loads(zk_server.link_mesh(
            project=name,
            similarity_threshold=req.similarity_threshold,
            per_note_top_k=req.per_note_top_k,
            max_links=req.max_links,
            dry_run=req.dry_run,
        ))

    if mode == "hubs":
        suggestions_res = json.loads(zk_server.suggest_concept_hubs(
            name,
            similarity_threshold=req.similarity_threshold if req.similarity_threshold is not None else 0.7,
            min_sources=req.min_sources,
        ))
        if "error" in suggestions_res:
            raise HTTPException(status_code=404, detail=suggestions_res["error"])
        suggestions = suggestions_res.get("suggestions", [])
        if req.dry_run:
            return {**suggestions_res, "project": name, "mode": "hubs", "dry_run": True}

        created: list[dict] = []
        for s in suggestions:
            if s.get("action") != "new_hub":
                continue
            notes = s.get("notes", []) or []
            members = [
                {"id": n["id"], "graph": n.get("source_graph", "")}
                for n in notes
                if n.get("id")
            ]
            title = s.get("suggested_title") or (notes[0]["title"] if notes else "")
            if not title or not members:
                continue
            res = zk_server.suggest(kind="concept-hubs", accept={"title": title, "members": members})
            if isinstance(res, dict) and "error" not in res:
                created.append(res)
        return {
            "project": name,
            "mode": "hubs",
            "dry_run": False,
            "created_hubs": created,
            "created": len(created),
            "attach_suggestions": [s for s in suggestions if s.get("action") == "attach"],
        }

    raise HTTPException(
        status_code=400,
        detail=f"Unknown connect mode '{req.mode}'. Use 'mesh' or 'hubs'.",
    )

project_signature

project_signature(name: str) -> 'str | None'

Stable, process-independent signature of a project's SOURCE files.

Folds in exactly the inputs the project graph/cluster payloads derive from — each real source box's note-mtime fingerprint (:func:_graph_signature), the _cross box, each source's _meta.yaml (:func:_meta_signature), the global _citations store (:func:_citations_signature), and the project manifest — reduced with the process-INDEPENDENT digests (:func:_sig_digest / :func:_stable_int) so identical content yields the same token on every run and across machines (mirrors the reasoning in :func:_sig_digest). Unlike the route's in-memory cache key (which uses salted hash() and reads _graph_sigs populated by a structural load), this computes the per-box fingerprint DIRECTLY from the box directory — pure glob+stat, no note parsing — so the dashboard can check bundle freshness without touching the native path. Returns None for an unknown/unsafe project.

Source code in zettelkasten/dashboard/backend/routes/projects.py

def project_signature(name: str) -> "str | None":
    """Stable, process-independent signature of a project's SOURCE files.

    Folds in exactly the inputs the project graph/cluster payloads derive from —
    each real source box's note-mtime fingerprint (:func:`_graph_signature`), the
    ``_cross`` box, each source's ``_meta.yaml`` (:func:`_meta_signature`), the
    global ``_citations`` store (:func:`_citations_signature`), and the project
    manifest — reduced with the process-INDEPENDENT digests (:func:`_sig_digest` /
    :func:`_stable_int`) so identical content yields the same token on every run
    and across machines (mirrors the reasoning in :func:`_sig_digest`). Unlike the
    route's in-memory cache key (which uses salted ``hash()`` and reads
    ``_graph_sigs`` populated by a structural load), this computes the per-box
    fingerprint DIRECTLY from the box directory — pure glob+stat, no note parsing —
    so the dashboard can check bundle freshness without touching the native path.
    Returns ``None`` for an unknown/unsafe project.
    """
    try:
        _local, repo = _resolve_ref(name)
        project_data = _load_project_or_404(name)
    except Exception:
        return None
    gdir = repo.zettel_dir if repo is not None else GRAPHS_DIR
    sources = list(project_data.get("sources", []))
    parts: list[str] = []
    for source_name in sources:
        if source_name.startswith("_"):
            continue
        display = federation.namespace_id(repo.id, source_name) if repo is not None else source_name
        try:
            box_path = _build_graph(display).path
        except Exception:
            continue
        parts.append(f"{display}:{_sig_digest(_graph_signature(box_path))}")
    if (gdir / "_cross").is_dir():
        cross_display = federation.namespace_id(repo.id, "_cross") if repo is not None else "_cross"
        try:
            cross_path = _build_graph(cross_display).path
            parts.append(f"{cross_display}:{_sig_digest(_graph_signature(cross_path))}")
        except Exception:
            pass
    real_sources = [s for s in sources if not s.startswith("_")]
    meta_sig = _meta_signature(real_sources, gdir)
    cit_sig = _citations_signature(gdir)
    manifest_sig = _stable_int(json.dumps(project_data, sort_keys=True, default=str))
    from zettelkasten.bundle import BUNDLE_VERSION

    components = sorted(parts) + [
        f"meta:{meta_sig}",
        f"cit:{cit_sig}",
        f"manifest:{manifest_sig}",
        f"v:{BUNDLE_VERSION}",
    ]
    return _sig_digest(tuple(components))

build_project_bundle_payload

build_project_bundle_payload(name: str) -> 'dict | None'

Assemble the durable bundle payload for name (structure + layouts).

Force-warms each source box's embedding index in THIS process (eager load → parse + embed + persist .kgl) so the cluster build below produces a real layout instead of deferring with embeddings_status="warming". Then assembles the full composite structure (max_nodes=0) and the base + fine semantic layouts (t-SNE + clustering). Synchronous and native-heavy: run in a worker subprocess / at write time, NEVER on the web request thread.

Returns {"structure", "layout", "granularities"} (layout is None when the embedding stack is unavailable, so the reader still gets a re-parse- free structure), or None for an unknown/unsafe project.

Source code in zettelkasten/dashboard/backend/routes/projects.py

def build_project_bundle_payload(name: str) -> "dict | None":
    """Assemble the durable bundle payload for ``name`` (structure + layouts).

    Force-warms each source box's embedding index in THIS process (eager load →
    parse + embed + persist ``.kgl``) so the cluster build below produces a real
    layout instead of deferring with ``embeddings_status="warming"``. Then
    assembles the full composite structure (``max_nodes=0``) and the base + fine
    semantic layouts (t-SNE + clustering). Synchronous and native-heavy: run in a
    worker subprocess / at write time, NEVER on the web request thread.

    Returns ``{"structure", "layout", "granularities"}`` (``layout`` is ``None``
    when the embedding stack is unavailable, so the reader still gets a re-parse-
    free structure), or ``None`` for an unknown/unsafe project.
    """
    try:
        _local, repo = _resolve_ref(name)
        project_data = _load_project_or_404(name)
    except Exception:
        return None
    gdir = repo.zettel_dir if repo is not None else GRAPHS_DIR
    sources = list(project_data.get("sources", []))
    displays: list[str] = []
    for source_name in sources:
        if source_name.startswith("_"):
            continue
        displays.append(
            federation.namespace_id(repo.id, source_name) if repo is not None else source_name
        )
    if (gdir / "_cross").is_dir():
        displays.append(
            federation.namespace_id(repo.id, "_cross") if repo is not None else "_cross"
        )
    # Force-warm each box's embedding index (parse + embed + persist .kgl). Eager
    # here is safe and desired: this runs off the request thread, and warming the
    # boxes now is exactly what lets the cluster build below return a real layout.
    for display in displays:
        try:
            _get_graph(display, eager_embeddings=True)
        except Exception:
            logger.warning("Bundle warm failed for box %r", display, exc_info=True)

    structure = get_project_graph(name, max_nodes=0)
    base = _project_cluster_result(name, _LOD_BASE_GRANULARITY)
    fine = _project_cluster_result(name, _LOD_FINE_GRANULARITY) if base else None
    return {
        "structure": structure,
        "layout": {"base": base, "fine": fine} if base else None,
        "granularities": {"base": _LOD_BASE_GRANULARITY, "fine": _LOD_FINE_GRANULARITY},
    }

load_fresh_project_bundle

load_fresh_project_bundle(name: str) -> 'tuple[dict, str] | None'

Return (bundle, signature) for name IFF a bundle built from the CURRENT sources exists on disk, else None.

The web request-path entry point for serving a project from its pre-built bundle. Cheap and native-free: computes the stable source signature (:func:project_signature — pure glob+stat, no note parse) and reads the signature-gated bundle. A None result (no bundle, or one built from now- stale sources) means the caller falls through to the live assembly/compute — the bundle is a pure accelerator, never a source of truth.

Source code in zettelkasten/dashboard/backend/routes/projects.py

def load_fresh_project_bundle(name: str) -> "tuple[dict, str] | None":
    """Return ``(bundle, signature)`` for ``name`` IFF a bundle built from the
    CURRENT sources exists on disk, else ``None``.

    The web request-path entry point for serving a project from its pre-built
    bundle. Cheap and native-free: computes the stable source signature
    (:func:`project_signature` — pure glob+stat, no note parse) and reads the
    signature-gated bundle. A ``None`` result (no bundle, or one built from now-
    stale sources) means the caller falls through to the live assembly/compute —
    the bundle is a pure accelerator, never a source of truth.
    """
    try:
        from zettelkasten import bundle as _bundle
    except Exception:  # pragma: no cover - import wiring
        return None
    sig = project_signature(name)
    if sig is None:
        return None
    data = _bundle.load_bundle(name, sig)
    if not data:
        return None
    return data, sig

get_project_graph

get_project_graph(name: str, max_nodes: int = Query(0, ge=0, description='LOD cap: 0 = full graph; >0 keeps the top-N note nodes by link_count and collapses the rest (source/subsource hubs always pinned)'), bbox: str = Query(None, description="Viewport bbox 'x0,y0,x1,y1' in t-SNE space; with max_nodes>0, on-screen notes are kept in full detail (server-side viewport LOD)"), exclude_types: str = Query(None, description="Comma-separated note types to omit from the payload + LOD (e.g. 'quote'); they stay reachable via the note-detail API"), request: Request = None, response: Response = None) -> dict

Returns composite graph data for a project: nodes from all sources + cross + citations.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.get("/projects/{name}/graph")
def get_project_graph(
    name: str,
    max_nodes: int = Query(0, ge=0, description="LOD cap: 0 = full graph; >0 keeps the top-N note nodes by link_count and collapses the rest (source/subsource hubs always pinned)"),
    bbox: str = Query(None, description="Viewport bbox 'x0,y0,x1,y1' in t-SNE space; with max_nodes>0, on-screen notes are kept in full detail (server-side viewport LOD)"),
    exclude_types: str = Query(None, description="Comma-separated note types to omit from the payload + LOD (e.g. 'quote'); they stay reachable via the note-detail API"),
    request: Request = None,
    response: Response = None,
) -> dict:
    """Returns composite graph data for a project: nodes from all sources + cross + citations."""
    # The LOD cap only engages for an int > 0 over HTTP (direct callers pass the
    # ``Query(...)`` sentinel), so full (0) and LOD (>0) payloads are DISTINCT
    # entries.
    lod_key = max_nodes if isinstance(max_nodes, int) and max_nodes > 0 else 0
    # Note types to drop from the payload (and thus from LOD ranking/collapse).
    # ``()`` when absent, so the default request stays byte-identical below.
    excluded_types = _parse_exclude_types(exclude_types if isinstance(exclude_types, str) else None)
    # Viewport-aware LOD only applies alongside a max_nodes cap. Snap the bbox to
    # the tile grid so it keys the cache stably AND filters detail identically.
    qbbox = _quantize_bbox(_parse_bbox(bbox)) if lod_key else None

    # ── Pre-built bundle fast path ──
    # A fresh per-project bundle carries the FULL composite structure exactly as
    # assembled below at ``max_nodes=0``, so serve it straight from disk — no note
    # parse, no source resolution. Signature-gated (a stale bundle is ignored). On
    # a MISS for a real project, schedule a background (process-pool) build so the
    # next poll is an instant bundle read, then fall through to the live path for
    # this request. The full (non-LOD) request is served entirely from the bundle;
    # the LOD path still runs the ranked collapse below, but its expensive semantic
    # layout is itself bundle-served (see :func:`_project_cluster_result` →
    # ``get_project_clusters``), so the only residual LOD cost is the cheap
    # structural parse + Python ranking.
    # The pre-built bundle is assembled with the FULL note set (evidence
    # included), so it cannot serve a type-excluded request — skip the fast path
    # entirely and fall through to live assembly, which applies the exclusion.
    _bsig = None if excluded_types else project_signature(name)
    if _bsig is not None:
        _bundle_data = _load_project_bundle(name, _bsig)
        if _bundle_data is not None:
            _structure = _bundle_data.get("structure")
            if _structure:
                if lod_key == 0:
                    not_modified = _conditional_etag(
                        request, response, ("bundle", "project-graph", name, 0, _bsig)
                    )
                    if not_modified is not None:
                        return not_modified
                    return _structure
                # LOD FROM BUNDLE: collapse the pre-assembled full structure with no
                # note parse on the request thread. The semantic layout used by the
                # collapse is itself bundle-served (``_project_cluster_result`` →
                # ``get_project_clusters``), so on a warm bundle nothing native runs
                # here — only the cheap Python rank/collapse. Keyed by the stable
                # bundle signature (busts on any source edit) plus the viewport tile
                # and the cheap cluster-readiness probe (mirrors the live LOD key so
                # a momentary structural fallback can't 304 back once clusters warm).
                _bkey = (
                    "bundle-lod",
                    name,
                    _bsig,
                    lod_key,
                    qbbox,
                    int(_project_clusters_ready_cheap(name)),
                )
                not_modified = _conditional_etag(request, response, _bkey)
                if not_modified is not None:
                    return not_modified
                cached = _graph_cache_get(_bkey)
                if cached is not None:
                    return cached
                lod_result, member_sets = _apply_project_semantic_lod(
                    name, _structure, max_nodes, qbbox
                )
                _graph_cache_put(_bkey, lod_result, member_sets=member_sets)
                return lod_result
        else:
            _schedule_bundle_build(name)

    _, repo = _resolve_ref(name)
    project_data = _load_project_or_404(name)
    gdir = repo.zettel_dir if repo is not None else GRAPHS_DIR
    _src = _src_resolver(repo)

    sources = project_data.get("sources", [])
    source_set = set(sources)

    # Serve an unchanged project from the payload cache. The signature is built
    # BEFORE the (expensive) assembly and folds in every input the response is
    # derived from, so a HIT is byte-identical to a fresh build and any relevant
    # on-disk edit invalidates it:
    #   * each real source's note-mtime sig (``_graph_sigs``) — resolved first so
    #     an out-of-band note edit refreshes the sig here (mirrors
    #     ``get_project_clusters`` / ``project_papers``); an edit in ANY source
    #     invalidates;
    #   * the ``_cross`` graph's sig (its claimed notes are part of the payload);
    #   * each source's ``_meta.yaml`` (``_meta_signature``) — source hub fields
    #     and source-level relation edges;
    #   * the global ``_citations`` store (``_citations_signature``) — citation
    #     node titles;
    #   * the project manifest itself (source list + cross membership), embedded
    #     verbatim as ``result["project"]``.
    # The LOD cap only engages for an int > 0 over HTTP (direct callers pass the
    # ``Query(...)`` sentinel), so full (0) and LOD (>0) payloads are DISTINCT
    # entries.
    real_sources: list[str] = []
    sig_parts: list[str] = []
    for source_name in sources:
        if source_name.startswith("_"):
            continue
        display = federation.namespace_id(repo.id, source_name) if repo is not None else source_name
        _try_get_graph(display)  # refresh _graph_sigs from disk before keying (structural: no embedding build)
        real_sources.append(source_name)
        sig_parts.append(f"{display}:{hash(_graph_sigs.get(display, ()))}")
    if (gdir / "_cross").is_dir():
        cross_display = federation.namespace_id(repo.id, "_cross") if repo is not None else "_cross"
        _try_get_graph(cross_display)
        sig_parts.append(f"{cross_display}:{hash(_graph_sigs.get(cross_display, ()))}")
    meta_sig = _meta_signature(real_sources, gdir)
    cit_sig = _citations_signature(gdir)
    manifest_sig = hash(json.dumps(project_data, sort_keys=True, default=str))
    sig_component = hash(("|".join(sig_parts), meta_sig, cit_sig, manifest_sig))
    # The LOD collapse groups semantically ONLY when ``_project_cluster_result``
    # returns real clusters; otherwise it falls back to structural grouping. So the
    # LOD cache key's embedding component must key on THAT SAME signal — not the
    # per-source ``_embeddings_ready`` loop, which can read True (e.g. an eager load
    # warmed a box) while ``_project_cluster_result`` still reports warming/None,
    # caching a structural-fallback payload under ``emb=1`` that never busts once
    # the clusters warm (a stale structural view).
    #
    # FIX-2 perf: derive that component from the CHEAP in-memory readiness probe
    # (``_project_clusters_ready_cheap``) rather than eagerly calling
    # ``_project_cluster_result`` here — the eager call forced a synchronous t-SNE
    # on a cold cluster cache even for a would-be 304/cache HIT. The probe reads
    # only the cluster cache (no t-SNE); the actual layout is computed below ONLY on
    # the fresh-build path. It flips False→True on the SAME cold→warm transition, so
    # a structural-fallback payload is never served from the LRU after the clusters
    # warm. Mirrors ``graph_data.py::get_graph_data``.
    force_semantic_rebuild = False
    if lod_key:
        # LOD path only: the quantized viewport tile keys each pan tile separately,
        # and cluster-readiness busts the key on a cold→warm transition.
        emb_component = int(_project_clusters_ready_cheap(name))
        cache_key = ("project", name, sig_component, lod_key, qbbox, emb_component)
        # FIX-2: break the stale-STRUCTURAL deadlock (project twin of
        # ``graph_data.get_graph_data``). A structural payload cached while warming
        # (emb=0) stays stuck once the boxes warm, because the cheap probe only
        # flips True after a cluster build has cached clusters — which only happens
        # on a fresh build (payload-cache MISS), never re-entered while the emb=0
        # key keeps HITting. When all note-boxes are now warm BUT no project cluster
        # build has been attempted (``_project_clusters_attempted`` False — distinct
        # from "attempted, genuinely no clusters", which must NOT thrash), bypass
        # the 304/payload-cache serve and rebuild ONCE so the SEMANTIC payload
        # appears and the probe settles True. Cheap (readiness flag + dict lookup,
        # never a t-SNE) and NEVER fires on a genuine warm HIT (emb=1 there).
        force_semantic_rebuild = (
            emb_component == 0
            and _project_embeddings_ready(name)
            and not _project_clusters_attempted(name)
        )
    else:
        # Full (non-LOD) payload never touches clusters, so its key — and the ETag
        # derived from it — stays BYTE-IDENTICAL to the committed viewport-LOD work
        # (``lod_key, qbbox`` with qbbox=None). Adding a readiness component here
        # would needlessly invalidate every client's cached full-graph ETag.
        cache_key = ("project", name, sig_component, lod_key, qbbox)
    # Only extend the key when types are actually excluded, so a request WITHOUT
    # ``exclude_types`` keeps its historical key/ETag byte-identical.
    if excluded_types:
        cache_key = cache_key + ("ex",) + excluded_types
    if not force_semantic_rebuild:
        # ETag / conditional-GET on the FRESH signature: the ETag is derived from
        # THIS cache key (which folds in every source's note-mtime sig + the meta/
        # citation/manifest signatures), so a matching If-None-Match returns a
        # bodyless 304 — skipping BOTH the composite node/edge assembly below AND
        # FastAPI's JSON serialization of the (large) project payload. Skipped on a
        # forced rebuild so a stale structural ETag can't 304 back to the stale body.
        not_modified = _conditional_etag(request, response, cache_key)
        if not_modified is not None:
            return not_modified
        cached = _graph_cache_get(cache_key)
        if cached is not None:
            return cached

    nodes: list[dict] = []
    edges: list[dict] = []
    source_meta: list[dict] = []
    seen_ids: set[str] = set()

    project_note_ids: set[str] = set()
    referenced_citations: set[str] = set()
    referenced_cross: set[str] = set()

    def _emit_links(note, owner_graph: str) -> None:
        for link in note.links:
            target_graph = link.graph or owner_graph
            edge: dict[str, Any] = {
                "source": note.id,
                "target": link.target,
                "relation": link.relation,
                "direction": link.direction,
                "source_graph": owner_graph,
            }
            if link.graph and link.graph != owner_graph:
                edge["target_graph"] = link.graph
                edge["cross_graph"] = True
            edges.append(edge)
            if target_graph == "_citations":
                referenced_citations.add(link.target)
            elif target_graph == "_cross":
                referenced_cross.add(link.target)

    for source_name in sources:
        # Special folders (``_cross``/``_citations``/...) are not real sources;
        # ignore a legacy manifest that listed ``_cross`` in ``sources`` so it
        # does not drag the whole global cross folder into the composite.
        if source_name.startswith("_"):
            continue
        zg = _src(source_name)
        if zg is None or not zg.path.is_dir():
            continue
        source_meta.append({
            "name": source_name,
            "note_count": len(zg.notes),
        })
        meta = load_source_meta(source_name, graphs_dir=gdir)
        source_id = f"source:{source_name}"
        # Source hub node: a first-class node representing the source itself.
        nodes.append({
            "id": source_id,
            "title": meta.get("title") or source_name,
            "type": "source",
            "tags": [],
            "chapter": "",
            "link_count": len(zg.notes),
            "source_graph": source_name,
            "doc_type": meta.get("doc_type", ""),
            "authors": meta.get("authors", []),
            "year": meta.get("year"),
            "note_count": len(zg.notes),
        })
        # Chapter sub-hubs: derived from each note's source.chapter. They give
        # the source view an intermediate grouping layer so notes cluster by the
        # chapter/section they came from. A note with no chapter hangs directly
        # off the source hub, as before. Chapter hub ids are namespaced by source
        # so they never collide across sources or with note ids.
        chapter_hub_ids: dict[str, str] = {}

        def _chapter_hub(chapter: str) -> str:
            hub_id = chapter_hub_ids.get(chapter)
            if hub_id is None:
                hub_id = f"chapter:{source_name}:{chapter}"
                chapter_hub_ids[chapter] = hub_id
            return hub_id

        chapter_counts: dict[str, int] = {}
        for note in zg.notes.values():
            chapter = (note.source.get("chapter") or "").strip()
            if chapter:
                chapter_counts[chapter] = chapter_counts.get(chapter, 0) + 1

        for chapter, count in chapter_counts.items():
            hub_id = _chapter_hub(chapter)
            nodes.append({
                "id": hub_id,
                "title": chapter,
                "type": "subsource",
                "tags": [],
                "chapter": chapter,
                "link_count": count,
                "source_graph": source_name,
                "note_count": count,
            })
            # Membership edge from the source hub down to the chapter hub.
            edges.append({
                "source": source_id,
                "target": hub_id,
                "relation": "contains",
                "direction": "outgoing",
                "source_graph": source_name,
                "membership": True,
            })

        for note in zg.notes.values():
            seen_ids.add(note.id)
            project_note_ids.add(note.id)
            chapter = (note.source.get("chapter") or "").strip()
            nodes.append({
                "id": note.id,
                "title": note.title,
                "type": note.type,
                "tags": note.tags,
                "chapter": chapter,
                "link_count": len(note.links) + len(zg.get_backlinks(note.id)),
                "source_graph": source_name,
                # See list_notes: paper-anchored evidence (non-empty source or a
                # grounding block) is kept out of the Terms/Concepts glossary.
                "grounded": bool(note.grounding) or bool(note.source),
            })
            # Membership edge: route through the chapter hub when the note has a
            # chapter, otherwise straight from the source hub.
            edges.append({
                "source": _chapter_hub(chapter) if chapter else source_id,
                "target": note.id,
                "relation": "contains",
                "direction": "outgoing",
                "source_graph": source_name,
                "membership": True,
            })
            _emit_links(note, source_name)

        # Source-level relations (_meta.yaml) become edges between source hubs.
        for rel in meta.get("relations", []) or []:
            if not isinstance(rel, dict):
                continue
            target = rel.get("target")
            relation = rel.get("relation")
            if not target or not relation:
                continue
            edges.append({
                "source": source_id,
                "target": f"source:{target}",
                "relation": relation,
                "direction": "outgoing",
                "source_graph": source_name,
                "target_graph": target,
                "source_level": True,
                "external": target not in source_set,
            })

    # _cross synthesis notes: EXPLICIT membership (manifest ``cross:``) is
    # authoritative for inclusion. Link-connected-but-unclaimed notes are demoted
    # to the ``suggested_cross`` bucket for one-click curation, never auto-shown.
    claimed_cross = project_cross_ids(project_data)
    claimed_set = set(claimed_cross)
    suggested_cross: list[dict] = []
    cross_dir = gdir / "_cross"
    cross_graph = _src("_cross") if cross_dir.is_dir() else None
    if cross_graph is not None:
        included_cross = 0
        for note in cross_graph.notes.values():
            if note.id in claimed_set:
                included_cross += 1
                seen_ids.add(note.id)
                nodes.append({
                    "id": note.id,
                    "title": note.title,
                    "type": note.type,
                    "tags": note.tags,
                    "chapter": "",
                    "link_count": len(note.links) + len(cross_graph.get_backlinks(note.id)),
                    "source_graph": "_cross",
                    # See list_notes: paper-anchored evidence (non-empty source or
                    # a grounding block) is kept out of the Terms/Concepts glossary.
                    "grounded": bool(note.grounding) or bool(note.source),
                })
                _emit_links(note, "_cross")
            elif cross_note_connects(note, source_set, project_note_ids, referenced_cross):
                suggested_cross.append({
                    "id": note.id,
                    "title": note.title,
                    "type": note.type,
                    "tags": note.tags,
                })
        if included_cross:
            source_meta.append({"name": "_cross", "note_count": included_cross})

    citations_data = load_citations(graphs_dir=gdir)
    citation_nodes = []
    for cid in referenced_citations:
        cdata = citations_data.get(cid)
        if cdata is None or cid in seen_ids:
            continue
        citation_nodes.append({
            "id": cid,
            "title": cdata.get("title", cid),
            "type": "citation",
            "tags": [],
            "chapter": "",
            "link_count": 0,
            "source_graph": "_citations",
        })

    # Drop excluded note types (e.g. supporting ``quote`` evidence) from the
    # composite payload BEFORE namespacing + LOD, so evidence never enters the
    # ranking/collapse and no super-node's ``type_mix`` is dominated by it. Runs
    # on RAW ids (pre-namespace) — the excluded ids and their edges share the same
    # raw namespace here. No-op when ``exclude_types`` is absent.
    nodes, edges = _apply_exclude_types(nodes, edges, excluded_types)

    result = {
        "nodes": nodes,
        "citation_nodes": citation_nodes,
        "edges": edges,
        "sources": source_meta,
        "cross": claimed_cross,
        "suggested_cross": suggested_cross,
        "project": project_data,
    }
    if repo is not None:
        _namespace_project_graph(result, repo.id)
        result["read_only"] = True
        result["repo_id"] = repo.id
        result["repo_name"] = repo.name
    # ``isinstance`` guard: FastAPI resolves ``max_nodes`` to an int over HTTP,
    # but direct callers (tests) get the ``Query(...)`` sentinel — treat that as
    # "no LOD" so the default path stays byte-identical.
    if isinstance(max_nodes, int) and max_nodes > 0:
        # SEMANTIC LOD (fresh-build path ONLY — reached past the 304/cache lookup):
        # resolve the embedding clusters HERE, not before the cache check, so a
        # cache hit never pays for t-SNE (FIX-2). Collapsed notes group by topical
        # cluster (super-nodes coloured/labelled like the cluster), on-screen notes
        # stay in full detail, and every super-node is stamped with its members'
        # centroid. The cluster payload is keyed by the SAME (raw) note ids the
        # graph payload carries. A warming/empty box returns nothing → we fall back
        # to plain structural rank-based LOD (graceful, never blocks the response).
        # This same call warms the cluster cache, so the NEXT request's cheap probe
        # agrees with this build's readiness.
        # Applied after namespacing so a federated super-node's ``source_graph``
        # matches its (namespaced) members'. Hubs are pinned; citations kept only
        # while a surviving edge references them. The collapse — including the base/
        # fine cluster resolution and per-super member-set capture — is the shared
        # :func:`_apply_project_semantic_lod` helper (identical to the bundle path).
        result, member_sets = _apply_project_semantic_lod(name, result, max_nodes, qbbox)
        if force_semantic_rebuild:
            # Project twin of ``graph_data.get_graph_data``: the forced rebuild was
            # keyed under emb=0 (the PRE-build cheap probe), but the build above just
            # ran ``_project_cluster_result``, so re-key the PUT under the component
            # the NEXT poll will compute — so that poll HITs instead of MISSing under
            # a different key and rebuilding the same semantic payload a second time.
            cache_key = (
                "project", name, sig_component, lod_key, qbbox,
                int(_project_clusters_ready_cheap(name)),
            )
        _graph_cache_put(cache_key, result, member_sets=member_sets)
    else:
        _graph_cache_put(cache_key, result)
    return result

get_project_drilldown

get_project_drilldown(name: str, group_id: str = Query(..., description='The super-node id (or its group_id) to drill into'), member_ids: 'list[str] | None' = Query(None, description="Approach B: the drilled glyph's exact build-time member id set (from the LOD payload), so drill resolution needs no server store. Prefer the POST route for large sets."), request: Request = None, response: Response = None) -> dict

Recursive drill-down for a project-graph LOD super-node.

Given a collapsed super-node's id, returns that group's CONNECTED sub-graph: its member notes PLUS their REAL edges (inter-member connectivity preserved, never stripped), re-clustered into finer semantic sub-supers at the next granularity tier — or the real members when the group is small / cannot subdivide (leaf=True). Degrades to an empty sub-graph while embeddings are warming (semantic grouping needs the cluster map).

When member_ids is supplied (Approach B) membership resolves directly from that set with no server-side store lookup; otherwise it degrades to the store/live re-derivation for back-compat with old clients.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.get("/projects/{name}/graph-drilldown")
def get_project_drilldown(
    name: str,
    group_id: str = Query(..., description="The super-node id (or its group_id) to drill into"),
    member_ids: "list[str] | None" = Query(
        None,
        description="Approach B: the drilled glyph's exact build-time member id set (from the LOD payload), so drill resolution needs no server store. Prefer the POST route for large sets.",
    ),
    request: Request = None,
    response: Response = None,
) -> dict:
    """Recursive drill-down for a project-graph LOD super-node.

    Given a collapsed super-node's id, returns that group's CONNECTED sub-graph:
    its member notes PLUS their REAL edges (inter-member connectivity preserved,
    never stripped), re-clustered into finer semantic sub-supers at the next
    granularity tier — or the real members when the group is small / cannot
    subdivide (``leaf=True``). Degrades to an empty sub-graph while embeddings are
    warming (semantic grouping needs the cluster map).

    When ``member_ids`` is supplied (Approach B) membership resolves directly from
    that set with no server-side store lookup; otherwise it degrades to the
    store/live re-derivation for back-compat with old clients.
    """
    # ``isinstance`` guard (mirrors ``get_graph_drilldown``): a direct caller that
    # omits ``member_ids`` gets the ``Query(...)`` sentinel, not a list — treat that
    # as "not supplied" so the back-compat path stays byte-identical.
    supplied = member_ids if isinstance(member_ids, list) else None
    return _project_drilldown(name, group_id, supplied)

post_project_drilldown

post_project_drilldown(name: str, req: ProjectDrilldownRequest, request: Request = None, response: Response = None) -> dict

POST twin of :func:get_project_drilldown — carries the glyph's member_ids in the body (preferred for large member sets that would bloat a query string).

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/projects/{name}/graph-drilldown")
def post_project_drilldown(
    name: str,
    req: ProjectDrilldownRequest,
    request: Request = None,
    response: Response = None,
) -> dict:
    """POST twin of :func:`get_project_drilldown` — carries the glyph's ``member_ids``
    in the body (preferred for large member sets that would bloat a query string)."""
    return _project_drilldown(name, req.group_id, req.member_ids)

add_project_cross

add_project_cross(name: str, req: ProjectCrossRequest) -> dict

Claim a _cross note for a project (append to its cross: list).

Explicit, curated membership — the same note id may be claimed by several projects (one physical note, many references). LOCAL-ONLY; idempotent.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/projects/{name}/cross")
def add_project_cross(name: str, req: ProjectCrossRequest) -> dict:
    """Claim a ``_cross`` note for a project (append to its ``cross:`` list).

    Explicit, curated membership — the same note id may be claimed by several
    projects (one physical note, many references). LOCAL-ONLY; idempotent.
    """
    import zettelkasten.server as zk_server

    _reject_federated_write(name)
    _load_project_or_404(name)
    return json.loads(zk_server.add_cross_to_project(project=name, cross_id=req.cross_id))

remove_project_cross

remove_project_cross(name: str, cross_id: str) -> dict

Release a _cross note from a project's cross: list (LOCAL-ONLY).

Removes only this project's reference; the note and every other project's claim on it are untouched. Idempotent.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.delete("/projects/{name}/cross/{cross_id}")
def remove_project_cross(name: str, cross_id: str) -> dict:
    """Release a ``_cross`` note from a project's ``cross:`` list (LOCAL-ONLY).

    Removes only this project's reference; the note and every other project's
    claim on it are untouched. Idempotent.
    """
    import zettelkasten.server as zk_server

    _reject_federated_write(name)
    _load_project_or_404(name)
    return json.loads(zk_server.remove_cross_from_project(project=name, cross_id=cross_id))

project_search

project_search(name: str, req: ProjectSearchRequest) -> list[dict]

Hybrid search across all sources in a project.

Fuses a semantic (embedding cosine) ranking with a keyword ranking via Reciprocal Rank Fusion, so exact term matches surface even when their embedding similarity is modest.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/projects/{name}/search")
def project_search(name: str, req: ProjectSearchRequest) -> list[dict]:
    """Hybrid search across all sources in a project.

    Fuses a semantic (embedding cosine) ranking with a keyword ranking via
    Reciprocal Rank Fusion, so exact term matches surface even when their
    embedding similarity is modest.
    """
    if not req.query.strip():
        return []

    from memory.ranking import rrf_order

    _, repo = _resolve_ref(name)
    project_data = _load_project_or_404(name)
    gdir = repo.zettel_dir if repo is not None else GRAPHS_DIR

    source_names = list(project_data.get("sources", []))
    if (gdir / "_cross").is_dir():
        source_names.append("_cross")

    # ``display`` carries the namespace for federated sources so source_graph in
    # the results (and the follow-up _get_graph) route back to the right repo.
    def _display(src: str) -> str:
        return federation.namespace_id(repo.id, src) if repo is not None else src

    q_lower = req.query.lower()
    # Pull a generous semantic pool so most keyword hits also carry a cosine.
    sem_pool = max(req.top_k * 4, 40)

    score_by_key: dict[str, float] = {}
    meta_by_key: dict[str, tuple[str, str]] = {}  # key -> (display, note_id)
    semantic_scored: list[tuple[str, float]] = []
    keyword_scored: list[tuple] = []  # (tier, global_pos, key)
    pos = 0

    for source_name in source_names:
        display = _display(source_name)
        # Structural resolve so a cold box is never eagerly indexed on the request
        # thread; the semantic channel below runs only when the box is warm.
        zg = _try_get_graph_structural(display)
        if zg is None or not zg.path.is_dir() or not zg.notes:
            continue

        # Semantic channel for this source — ONLY when its embedding index is
        # already warm. A box still warming contributes keyword hits only (below)
        # rather than blocking the request on a native kglite build; run_warmup
        # owns the build. When every box is warm this is byte-identical to the
        # prior hybrid ranking.
        if _embeddings_ready(zg):
            for note_id, score in zg.embeddings.search(req.query, top_k=sem_pool):
                if note_id not in zg.notes:
                    continue
                key = f"{display}\x00{note_id}"
                meta_by_key[key] = (display, note_id)
                score_by_key[key] = score
                semantic_scored.append((key, score))
        else:
            # Cold box with notes: schedule an idempotent background rebuild (off
            # the request thread) so a later search gains the semantic channel.
            # This request still returns keyword hits below rather than blocking.
            _maybe_rewarm(display, zg)

        # Keyword channel for this source (title -> tag/alias -> body), over the
        # cached pre-lowercased corpus.
        for note_id, title_l, tags_l, aliases_l, body_l in _box_keyword_corpus(display, zg):
            title_match = q_lower in title_l
            tag_match = any(q_lower in t for t in tags_l) or any(q_lower in a for a in aliases_l)
            body_match = q_lower in body_l
            if title_match or tag_match or body_match:
                key = f"{display}\x00{note_id}"
                meta_by_key.setdefault(key, (display, note_id))
                tier = 0 if title_match else (1 if tag_match else 2)
                keyword_scored.append((tier, pos, key))
            pos += 1

    semantic_scored.sort(key=lambda x: x[1], reverse=True)
    semantic_rank = [k for k, _ in semantic_scored]
    keyword_scored.sort(key=lambda t: (t[0], t[1]))
    keyword_rank = [k for _, _, k in keyword_scored]

    fused = rrf_order([semantic_rank, keyword_rank])

    results = []
    for key in fused:
        meta = meta_by_key.get(key)
        if meta is None:
            continue
        display, note_id = meta
        zg = _get_graph_structural(display)  # reads notes only; no embedding build
        note = zg.notes.get(note_id)
        if note is None:
            continue
        results.append({
            "id": note_id,
            "title": note.title,
            "type": note.type,
            "source_graph": display,
            "similarity": round(score_by_key.get(key, 0.0), 3),
            "body_preview": note.body[:150] + "..." if len(note.body) > 150 else note.body,
            "tags": note.tags,
        })
        if len(results) >= req.top_k:
            break

    return results

frame_question

frame_question(req: FrameQuestionRequest) -> dict

Reframe the corpus along a research question (read-only lens).

Decomposes the question into facets (landscape hubs or caller-supplied), projects the relevant notes per facet, labels each established/contested/ thin, and bundles per-facet citations. Mirrors the MCP frame_question.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/frame")
def frame_question(req: FrameQuestionRequest) -> dict:
    """Reframe the corpus along a research question (read-only lens).

    Decomposes the question into facets (landscape hubs or caller-supplied),
    projects the relevant notes per facet, labels each established/contested/
    thin, and bundles per-facet citations. Mirrors the MCP `frame_question`.
    """
    from zettelkasten import framing

    # Resolve a federated project to its repo so framing reads that repo's
    # sources/citations and routes source names back to it (read-only). Local
    # projects pass through with the defaults unchanged.
    local_project, repo = _resolve_ref(req.project) if req.project else (req.project, None)
    # When the project ref is federated, route it through the same subset-rule
    # 404 guard as ``/projects/{name}/*`` so a hidden or partly-disallowed
    # federated project cannot be framed (an existence oracle inconsistent with
    # every other project endpoint). Local projects (repo is None) are unaffected.
    if repo is not None:
        _load_project_or_404(req.project)
    graphs_dir = repo.zettel_dir if repo is not None else None
    namespace = (lambda s: federation.namespace_id(repo.id, s)) if repo is not None else None

    result = framing.frame_question(
        req.question, get_graph=_get_graph, project=local_project, facets=req.facets,
        max_facets=req.max_facets, per_facet_k=req.per_facet_k,
        thin_min_results=req.thin_min_results, thin_min_similarity=req.thin_min_similarity,
        graphs_dir=graphs_dir, namespace=namespace,
    )
    if "error" in result:
        raise HTTPException(status_code=400, detail=result["error"])
    return result

frame_answer

frame_answer(req: FrameAnswerRequest) -> dict

Optional LLM-synthesized narrative answer over a frame (dashboard-only).

Recomputes the deterministic frame for the same request params (so the grounding evidence and cache signature are derived server-side), then layers a short grounded paragraph via :mod:zettelkasten.frame_answer. Returns {"answer": str | None} — null when no LLM is configured or the frame has no established/contested facet, so the UI simply renders no paragraph. The MCP frame_question tool is intentionally left answer-free.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/frame/answer")
def frame_answer(req: FrameAnswerRequest) -> dict:
    """Optional LLM-synthesized narrative answer over a frame (dashboard-only).

    Recomputes the deterministic frame for the same request params (so the
    grounding evidence and cache signature are derived server-side), then layers a
    short grounded paragraph via :mod:`zettelkasten.frame_answer`. Returns
    ``{"answer": str | None}`` — ``null`` when no LLM is configured or the frame
    has no established/contested facet, so the UI simply renders no paragraph. The
    MCP ``frame_question`` tool is intentionally left answer-free.
    """
    from zettelkasten import framing, frame_answer as fa

    local_project, repo = _resolve_ref(req.project) if req.project else (req.project, None)
    if repo is not None:
        _load_project_or_404(req.project)
    graphs_dir = repo.zettel_dir if repo is not None else None
    namespace = (lambda s: federation.namespace_id(repo.id, s)) if repo is not None else None

    result = framing.frame_question(
        req.question, get_graph=_get_graph, project=local_project, facets=req.facets,
        max_facets=req.max_facets, per_facet_k=req.per_facet_k,
        thin_min_results=req.thin_min_results, thin_min_similarity=req.thin_min_similarity,
        graphs_dir=graphs_dir, namespace=namespace,
    )
    if "error" in result:
        raise HTTPException(status_code=400, detail=result["error"])
    return {"answer": fa.synthesize_answer(result, use_cache=req.use_cache)}

frame_spine_suggest

frame_spine_suggest(req: FrameSpineSuggestRequest) -> dict

Propose a NEW spine (comparison matrix) seeded by a frame — READ-ONLY.

Recomputes the deterministic frame for the same request params (so the grounding is derived server-side, never trusting a client-supplied frame), then assembles a write-free spine PROPOSAL via :mod:zettelkasten.spine_suggest: candidate rows from the frame's facets/hubs and the suggest kind=structure clusters, dimension columns drafted from the question by a tool-free LLM (deterministic fallback when none), thin facets as gaps, plus a schema spec validated write-free. Phase 1 only PROPOSES — it does not reorganize notes or extract. Returns {"proposal", "validated_spec", "valid"}.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/frame/spine-suggest")
def frame_spine_suggest(req: FrameSpineSuggestRequest) -> dict:
    """Propose a NEW spine (comparison matrix) seeded by a frame — READ-ONLY.

    Recomputes the deterministic frame for the same request params (so the
    grounding is derived server-side, never trusting a client-supplied frame),
    then assembles a write-free spine PROPOSAL via
    :mod:`zettelkasten.spine_suggest`: candidate rows from the frame's facets/hubs
    and the ``suggest kind=structure`` clusters, dimension columns drafted from
    the question by a tool-free LLM (deterministic fallback when none), thin
    facets as gaps, plus a schema spec validated write-free. Phase 1 only
    PROPOSES — it does not reorganize notes or extract. Returns
    ``{"proposal", "validated_spec", "valid"}``.
    """
    from zettelkasten import framing, spine_suggest

    local_project, repo = _resolve_ref(req.project) if req.project else (req.project, None)
    if repo is not None:
        _load_project_or_404(req.project)
    graphs_dir = repo.zettel_dir if repo is not None else None
    namespace = (lambda s: federation.namespace_id(repo.id, s)) if repo is not None else None

    result = framing.frame_question(
        req.question, get_graph=_get_graph, project=local_project, facets=req.facets,
        max_facets=req.max_facets, per_facet_k=req.per_facet_k,
        thin_min_results=req.thin_min_results, thin_min_similarity=req.thin_min_similarity,
        graphs_dir=graphs_dir, namespace=namespace,
    )
    if "error" in result:
        raise HTTPException(status_code=400, detail=result["error"])
    return spine_suggest.suggest_spine(
        result, _get_graph, project=local_project, use_cache=req.use_cache,
    )

frame_spine_create

frame_spine_create(req: FrameSpineCreateRequest) -> dict

Materialize a spine by REORGANIZING existing notes (Phase 2 — WRITE).

Recomputes the deterministic frame server-side (exactly like /frame/spine-suggest), runs :func:spine_suggest.suggest_spine to get the write-free proposal, then BUILDS the spine by reusing the existing org machinery — no bespoke edge-writer, no grounded extraction in the default path:

1. Resolve every proposal row's members to {note_id, graph} (facet rows via the frame evidence source graph; cluster rows via title→id resolution over the named source graphs). Unresolvable members are dropped.
2. :func:organizations.save_organization a lens org from the proposal, keyed on a stable org_id derived from the question+project (the idempotency guard — a repeat click re-promotes the same org).
3. Attach each row's resolved members to the FIRST column's cell and :func:organizations.promote_organization — the ONLY edge-writer, idempotent (reconcile-based) so a re-promote reuses the same spine_ref.
4. When extract is true, ADDITIONALLY re-classify the already-attached notes into the spine's dimension cells across the SAME multi-row grid (:func:_enrich_grid_dimensions over :func:remine.build_classify_fn) and re-promote the FULL enriched grid — every reorganize row hub is preserved, only dimension membership among EXISTING notes is added. This reuses the re-mine classifier without the single-row :func:remine.backfill_spine collapse, and never reads source documents.

LOCAL-ONLY: federated (read-only) owners and the global ZK_DISABLE_WRITE barrier are refused up front (mirrors _spine_write_guard). Returns the created spine reference so the frontend can navigate to it.

Source code in zettelkasten/dashboard/backend/routes/projects.py

@router.post("/frame/spine/create")
def frame_spine_create(req: FrameSpineCreateRequest) -> dict:
    """Materialize a spine by REORGANIZING existing notes (Phase 2 — WRITE).

    Recomputes the deterministic frame server-side (exactly like
    ``/frame/spine-suggest``), runs :func:`spine_suggest.suggest_spine` to get the
    write-free proposal, then BUILDS the spine by reusing the existing org
    machinery — no bespoke edge-writer, no grounded extraction in the default
    path:

    1. Resolve every proposal row's members to ``{note_id, graph}`` (facet rows
       via the frame evidence source graph; cluster rows via title→id resolution
       over the named source graphs). Unresolvable members are dropped.
    2. :func:`organizations.save_organization` a lens org from the proposal, keyed
       on a stable ``org_id`` derived from the question+project (the idempotency
       guard — a repeat click re-promotes the same org).
    3. Attach each row's resolved members to the FIRST column's cell and
       :func:`organizations.promote_organization` — the ONLY edge-writer,
       idempotent (reconcile-based) so a re-promote reuses the same ``spine_ref``.
    4. When ``extract`` is true, ADDITIONALLY re-classify the already-attached
       notes into the spine's dimension cells across the SAME multi-row grid
       (:func:`_enrich_grid_dimensions` over :func:`remine.build_classify_fn`) and
       re-promote the FULL enriched grid — every reorganize row hub is preserved,
       only dimension membership among EXISTING notes is added. This reuses the
       re-mine classifier without the single-row :func:`remine.backfill_spine`
       collapse, and never reads source documents.

    LOCAL-ONLY: federated (read-only) owners and the global ``ZK_DISABLE_WRITE``
    barrier are refused up front (mirrors ``_spine_write_guard``). Returns the
    created spine reference so the frontend can navigate to it.
    """
    from zettelkasten import framing, spine_suggest

    if not (req.project or "").strip():
        raise HTTPException(
            status_code=400,
            detail="Creating a spine requires a project owner (question framing has no owner).",
        )

    # Reject federated (read-only) owners and the global write barrier BEFORE any
    # compute, mirroring every other spine write route.
    local_project, _name = _spine_owner(req.project)
    _spine_write_guard()

    result = framing.frame_question(
        req.question, get_graph=_get_graph, project=local_project, facets=req.facets,
        max_facets=req.max_facets, per_facet_k=req.per_facet_k,
        thin_min_results=req.thin_min_results, thin_min_similarity=req.thin_min_similarity,
    )
    if "error" in result:
        raise HTTPException(status_code=400, detail=result["error"])

    suggestion = spine_suggest.suggest_spine(result, _get_graph, project=local_project)
    proposal = suggestion.get("proposal") or {}
    prop_rows = proposal.get("rows") or []
    prop_cols = proposal.get("columns") or []

    if not prop_rows:
        raise HTTPException(
            status_code=422,
            detail="The frame produced no rows to organize into a spine.",
        )
    if not prop_cols:
        raise HTTPException(
            status_code=422,
            detail="The proposal has no dimension columns to build a spine from.",
        )

    # Lens columns from the proposal (tag → schema_tag column; label = desc).
    columns = [
        {
            "key": _spine_create_slug(str(c.get("tag") or "")) or f"col-{i + 1}",
            "label": str(c.get("desc") or c.get("tag") or "").strip() or str(c.get("tag") or ""),
            "type": "extractive",
            "backing": "schema_tag",
            "ref": str(c.get("tag") or ""),
            "prompt": "",
        }
        for i, c in enumerate(prop_cols)
    ]
    first_col_key = columns[0]["key"]

    # Resolve members and build the proposed grid (rows with 0 resolved members
    # contribute no edges, so drop them — an empty hub is noise).
    evidence = _frame_evidence_index(result)
    grid: list[dict] = []
    row_labels: list[str] = []
    total_members = 0
    for row in prop_rows:
        row_id, members = _resolve_row_members(row, evidence, _get_graph)
        if not members:
            continue
        label = str(row.get("label") or "").strip() or row_id
        total_members += len(members)
        row_labels.append(label)
        grid.append({
            "id": row_id,
            "label": label,
            "source_graph": "",
            "cells": {
                first_col_key: {
                    "summary": "",
                    "members": members,
                    "gap": False,
                    "value": "",
                    "note_ids": [m["note_id"] for m in members],
                }
            },
        })

    if total_members == 0:
        # Every row resolved to zero real members — promote's empty-spine guard
        # would abort. Return a clear, structured 4xx instead of a 500.
        raise HTTPException(
            status_code=422,
            detail=(
                "No notes could be resolved from the proposed rows, so there is "
                "nothing to organize into a spine."
            ),
        )

    review, table_id, org_id = _spine_create_org_id(req.question, local_project)

    raw = {
        "id": org_id,
        "title": str(proposal.get("title") or "").strip() or org_id,
        "state": "lens",
        "spine_ref": "",
        "row_axis": {
            "kind": "group",
            "strategy": "field",
            "ref": "",
            "include": row_labels,
        },
        "columns": columns,
        "review": review,
        "table_id": table_id,
    }

    # IDEMPOTENCY (rapid double-click, P2): do the "does it exist?" check and the
    # first lens write ATOMICALLY under the OWNER lock — the SAME lock
    # ``save_organization`` takes — so a concurrent second call can never observe a
    # stale "absent" and overwrite an already-created (or already-promoted) org back
    # to a raw lens. If the org already exists (a repeat click) its persisted
    # definition is left intact so promote re-promotes it (reusing ``spine_ref`` +
    # the stashed ``lens_definition``). ``_save_organization_locked`` is the
    # lock-free core exposed for exactly this kind of composite critical section;
    # the owner lock is DISTINCT from the per-org spine lock ``promote_organization``
    # takes below, so this never deadlocks with the promote.
    from zettelkasten.commit import review_write_lock

    with review_write_lock(
        organizations._owner_lock_name("project", local_project), graphs_dir=GRAPHS_DIR
    ):
        existing = organizations.load_organization(
            "project", local_project, org_id, graphs_dir=GRAPHS_DIR, migrate=False
        )
        if existing is None:
            organizations._save_organization_locked(
                GRAPHS_DIR,
                organizations.normalize_organization(
                    raw, owner_type="project", owner_name=local_project, org_id=org_id
                ),
            )

    try:
        promoted = organizations.promote_organization(
            "project", local_project, org_id,
            get_graph=_get_graph, graphs_dir=GRAPHS_DIR,
            proposed_grid=grid,
        )
    except ValueError as exc:
        raise HTTPException(status_code=422, detail=str(exc))

    extracted = False
    attached_edges = int(promoted.get("attached_edges") or 0)
    if req.extract:
        # OPT-IN grounded ENRICH: PRESERVE every reorganize row and only ADD
        # dimension membership among the EXISTING (already-attached) notes. Reuse
        # the re-mine classifier (:func:`remine.build_classify_fn` over the spine's
        # dimension columns) to re-sort the already-resolved members into dimension
        # cells across the SAME multi-row grid, then re-promote the FULL enriched
        # grid — so no row hub is pruned and the reorganization the user approved is
        # NEVER collapsed. We deliberately do NOT call ``remine.backfill_spine``
        # here: that is a SINGLE-ROW persona-spine tool whose one-row grid would
        # orphan-prune every reorganize row hub on re-promote.
        #
        # Best-effort: a missing LLM / classify failure degrades to
        # ``extracted=False`` with the reorganize spine intact (the failure is
        # caught before any re-promote, so the spine is never pruned).
        try:
            dims = [
                {"key": c["key"], "tag": c["ref"] or c["key"], "title": c["label"], "desc": c["label"]}
                for c in columns
            ]
            classify_fn = remine.build_classify_fn(dims)
            enriched_grid, added = _enrich_grid_dimensions(grid, columns, classify_fn, _get_graph)
            if added > 0:
                reorg_hubs = len(promoted.get("hub_nodes") or {})
                promoted2 = organizations.promote_organization(
                    "project", local_project, org_id,
                    get_graph=_get_graph, graphs_dir=GRAPHS_DIR,
                    proposed_grid=enriched_grid,
                )
                # ``extracted`` is true ONLY when the enrich GREW the spine and did
                # not shrink it. The enriched grid is a strict superset of the
                # reorganize grid, so a successful re-promote writes >0 NEW
                # dimension edges and keeps every reorganize row hub. If either
                # gate fails, keep the reorganize result and report extracted=False.
                new_edges = int(promoted2.get("attached_edges") or 0)
                kept_hubs = len(promoted2.get("hub_nodes") or {}) >= reorg_hubs
                if new_edges > 0 and kept_hubs:
                    attached_edges += new_edges
                    extracted = True
        except Exception:  # noqa: BLE001 — grounded enrich is opt-in + best-effort
            logger.warning("spine create '%s': grounded extraction enrich failed", org_id, exc_info=True)
            extracted = False

    return {
        "owner_type": "project",
        "owner_name": local_project,
        "org_id": org_id,
        "spine_ref": promoted.get("spine_ref", ""),
        "apex_id": promoted.get("apex_id", ""),
        "attached_edges": attached_edges,
        "extracted": extracted,
    }