zettelkasten.organizations

zettelkasten.organizations

Organizations: first-class, owner-scoped matrix configurations.

An organization is a named, persistent matrix definition — a row axis plus an ordered set of typed columns — that lives at the owner (project or graph) level rather than buried inside a single review's .tables.json. Decoupling the definition from the review means the same "lens" on a corpus can be reused across reviews, listed in a project switcher, and (later) promoted from a live read-only projection into a materialized graph.

An organization has two states:

• lens — a live, read-only projection. The grid is derived on demand from the org's row axis + columns; nothing is written back to the corpus.
• spine — a materialized structure. spine_ref names the apex graph the organization writes back into. (Promotion is Phase 2; 1a only models the field.)

Storage mirrors the project/review artifact discipline but keys by owner:

<base>/_organizations/<owner_type>/<owner_name>/<id>.json   # one org per file
<base>/_organizations/<owner_type>/<owner_name>/index.json  # derived listing
<base>/_organizations/<owner_type>/<owner_name>/_migrated.json  # migration marker

The per-file layout gives each organization an independent, git-trackable artifact (clean diffs, no whole-store write contention) while index.json is a derived read-accelerator — rebuilt by scanning the directory on every write, so it can never drift from the source-of-truth files. All writes go through the same M3a substrate as reviews (review_write_lock → atomic_write_text → _schedule_zettel_commit), keyed by a per-owner lock.

A lazy, idempotent migration imports each legacy _reviews/<name>.tables.json into organizations owned by that review's scope, so existing matrices appear as organizations the first time an owner is listed — without re-importing a table a user later deletes (a per-owner marker records which reviews were already absorbed).

normalize_organization

normalize_organization(raw: Any, *, owner_type: str, owner_name: str, org_id: str = '') -> dict[str, Any]

Canonicalize an organization dict to its on-disk shape.

Reuses the matrix engine's :func:normalize_row_axis / :func:normalize_columns so an organization's definition is byte-for-byte compatible with what build_matrix consumes. Timestamps are preserved when present (so a re-save keeps created_at); :func:save_organization fills the blanks.

Source code in zettelkasten/organizations.py

def normalize_organization(
    raw: Any,
    *,
    owner_type: str,
    owner_name: str,
    org_id: str = "",
) -> dict[str, Any]:
    """Canonicalize an organization dict to its on-disk shape.

    Reuses the matrix engine's :func:`normalize_row_axis` / :func:`normalize_columns`
    so an organization's definition is byte-for-byte compatible with what
    ``build_matrix`` consumes. Timestamps are preserved when present (so a re-save
    keeps ``created_at``); :func:`save_organization` fills the blanks.
    """
    ot, on = _validate_owner(owner_type, owner_name)
    data = raw if isinstance(raw, dict) else {}
    oid = (org_id or str(data.get("id") or "")).strip()
    if not oid:
        raise ValueError("organization id is required")
    validate_id(oid, kind="organization id", for_filename=True)

    state = str(data.get("state") or "lens")
    if state not in ORG_STATES:
        state = "lens"

    title = str(data.get("title") or "").strip() or oid

    return {
        "id": oid,
        "owner": {"type": ot, "name": on},
        "title": title,
        "state": state,
        "row_axis": normalize_row_axis(data.get("row_axis")),
        "columns": normalize_columns(data.get("columns")),
        # The matrix VIEW config: how the spine TREE (§11.3) flattens into the
        # 2-axis grid — which ``component-of`` level is the columns cut, and
        # whether a column cell rolls up its subtree. The default (``cols_level``
        # None, ``rollup`` True) reproduces the historical flat depth-1 grid, so an
        # org with no ``matrix_view`` renders exactly as before V2b.
        "matrix_view": normalize_matrix_view(data.get("matrix_view")),
        # The author's matrix corrections (member adds/removes + cell overrides),
        # stored as ID-references and re-applied onto each fresh build by
        # ``tables.apply_overlay`` (the lens flow). Normalized to the full shape so
        # a partial/None block loads cleanly and a rebuild never drops corrections.
        "overlay": _normalize_table_overlay(data.get("overlay")),
        # The apex graph a promoted (``spine``) org writes back into. Empty for a lens.
        "spine_ref": str(data.get("spine_ref") or "").strip(),
        # The spine_ref a DEMOTE cleared, stashed so a later re-promote re-binds the
        # SAME synthesis graph (whose nodes/edges demote preserved) instead of
        # minting a fresh one. Empty for a never-promoted lens or a still-live spine;
        # cleared once a re-promote rebinds. Makes ``state=='spine'`` + non-empty
        # ``spine_ref`` the SINGLE unambiguous "materialized" signal (a demoted org
        # is ``state=='lens'`` with a blank ``spine_ref``). Forward-compatible: an org
        # file written before this field existed loads as "" with no rewrite.
        "last_spine_ref": str(data.get("last_spine_ref") or "").strip(),
        # The pre-promotion (agent-driven) row axis + columns, stashed on promote so
        # ``demote``/``delete_spine`` can restore the lens definition. ``None`` until
        # the org is promoted.
        "lens_definition": (
            data["lens_definition"]
            if isinstance(data.get("lens_definition"), dict)
            else None
        ),
        # The EMBEDDED extraction schema (the full EXPANDED rubric) a ``spine``-state
        # org carries so the scribe can re-source extraction from the spine itself
        # rather than a separately-named schema file (design §3/§5). Generated on
        # promotion by ``tables.schema_from_columns`` (the inverse of
        # ``suggest_columns``) from the lens columns + materialized synthesis block.
        # EMPTY ``{}`` for a lens org. Forward-compatible: an org file written before
        # this field existed loads as ``{}`` with no rewrite, and the field is purely
        # additive so every existing caller keeps working.
        "schema": data["schema"] if isinstance(data.get("schema"), dict) else {},
        # The review this org was authored in, and the review-local table id that
        # keys its grid cache + cell-note anchors.
        "review": str(data.get("review") or "").strip(),
        "table_id": str(data.get("table_id") or "").strip(),
        # TRUE migration provenance: ``ensure_migrated`` (and ONLY it) stamps this
        # when importing a legacy ``_reviews/*.tables.json`` table into an org.
        # ``review`` + ``table_id`` are NOT a reliable migration signal —
        # ``upsert_definition`` sets BOTH on every live review-table build/rename —
        # so identity that must distinguish "a migrated legacy VIEW of synthesis
        # graph G" from "an ordinary review table that incidentally references G in
        # a column" keys on THIS flag (see :func:`_org_synthesis_graphs`). Defaults
        # to ``False`` (forward-compatible: an org file written before this field
        # loads as not-migrated, and a live ``upsert_definition`` rebuild clears it).
        "migrated": bool(data.get("migrated")),
        # Derived grid freshness mirrored from the table cache (the grid itself
        # stays in ``_reviews/<review>.tables.json``; orgs hold only the metadata
        # so a listing can show staleness without loading every grid).
        "signature": str(data.get("signature") or "").strip(),
        # Drift stamp captured at the last promote/resync: ``hash(corpus generation
        # signature + edge-affecting overlay fingerprint)`` (see
        # ``_synced_fingerprint``). Lets a Generate over an already-materialized spine
        # tell "nothing changed → readback" from "corpus/correction drifted → resync"
        # (``spine_is_synced``). Distinct from ``signature`` (the grid-cache key the
        # render re-stamps with link-form columns) because this must survive that
        # re-stamp and be recomputable byte-identically at gate time — so it folds in
        # only the pure corpus signature + overlay, NOT the columns. Absent on a legacy
        # pre-field spine → loads as ``""`` (never equals a real recompute → that spine
        # resyncs once, then stamps this and takes the fast path thereafter).
        "synced_sig": str(data.get("synced_sig") or "").strip(),
        "generated_at": str(data.get("generated_at") or "").strip(),
        "created_at": str(data.get("created_at") or "").strip(),
        "updated_at": str(data.get("updated_at") or "").strip(),
    }

save_organization

save_organization(raw: dict[str, Any], *, owner_type: str, owner_name: str, org_id: str = '', graphs_dir: 'Path | None' = None) -> dict[str, Any]

Create or update an organization; returns the persisted (normalized) dict.

created_at is preserved across updates and stamped on first write; updated_at is refreshed every save. The derived index is rebuilt after the write so a subsequent :func:list_organizations is consistent.

Source code in zettelkasten/organizations.py

def save_organization(
    raw: dict[str, Any],
    *,
    owner_type: str,
    owner_name: str,
    org_id: str = "",
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Create or update an organization; returns the persisted (normalized) dict.

    ``created_at`` is preserved across updates and stamped on first write;
    ``updated_at`` is refreshed every save. The derived index is rebuilt after the
    write so a subsequent :func:`list_organizations` is consistent.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    org = normalize_organization(raw, owner_type=ot, owner_name=on, org_id=org_id)
    with review_write_lock(_owner_lock_name(ot, on), graphs_dir=base):
        _save_organization_locked(base, org)
    return org

load_organization

load_organization(owner_type: str, owner_name: str, org_id: str, *, graphs_dir: 'Path | None' = None, migrate: bool = True) -> 'dict[str, Any] | None'

Load one organization by id, or None if it doesn't exist.

Source code in zettelkasten/organizations.py

def load_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    graphs_dir: "Path | None" = None,
    migrate: bool = True,
) -> "dict[str, Any] | None":
    """Load one organization by id, or ``None`` if it doesn't exist."""
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    if migrate:
        ensure_migrated(ot, on, graphs_dir=base)
    raw = _read_org_file(_org_file(base, ot, on, org_id))
    if raw is None:
        return None
    try:
        return normalize_organization(raw, owner_type=ot, owner_name=on, org_id=org_id)
    except ValueError:
        return None

list_organizations

list_organizations(owner_type: str, owner_name: str, *, graphs_dir: 'Path | None' = None, migrate: bool = True) -> list[dict[str, Any]]

All organizations for an owner (full definitions), newest-updated first.

Triggers a lazy, idempotent migration of any legacy review tables scoped to this owner the first time it runs (disable with migrate=False).

Source code in zettelkasten/organizations.py

def list_organizations(
    owner_type: str,
    owner_name: str,
    *,
    graphs_dir: "Path | None" = None,
    migrate: bool = True,
) -> list[dict[str, Any]]:
    """All organizations for an owner (full definitions), newest-updated first.

    Triggers a lazy, idempotent migration of any legacy review tables scoped to
    this owner the first time it runs (disable with ``migrate=False``).
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    if migrate:
        ensure_migrated(ot, on, graphs_dir=base)
        # Lazy spine porting fires from the SAME first-listing hook as the
        # review-table migration, but only for PROJECT owners on the GLOBAL store:
        # ``discover_spines`` reads the live graph universe via ``GRAPHS_DIR``, so a
        # custom ``graphs_dir`` (tests) must drive ``port_spines`` directly with its
        # own ``get_graph`` rather than mis-reading the global store. Best-effort
        # (``_maybe_port_spines`` never raises); runs AFTER ``ensure_migrated``
        # releases the owner lock, so the two locked sections never nest.
        if ot == "project" and base == GRAPHS_DIR:
            _maybe_port_spines(on, base)
    return _sort_orgs(_scan_owner_orgs(_owner_dir(base, ot, on), ot, on))

delete_organization

delete_organization(owner_type: str, owner_name: str, org_id: str, *, graphs_dir: 'Path | None' = None) -> bool

Remove an organization file. Returns True if it existed.

The migration marker is intentionally untouched: deleting a migrated org must NOT cause it to be re-imported on the next listing.

Source code in zettelkasten/organizations.py

def delete_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    graphs_dir: "Path | None" = None,
) -> bool:
    """Remove an organization file. Returns True if it existed.

    The migration marker is intentionally untouched: deleting a migrated org must
    NOT cause it to be re-imported on the next listing.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    with review_write_lock(_owner_lock_name(ot, on), graphs_dir=base):
        return _delete_organization_locked(ot, on, org_id, base)

org_id_for

org_id_for(review: str, table_id: str) -> str

The registry id a (review, table_id) maps to (shared with migration).

Source code in zettelkasten/organizations.py

def org_id_for(review: str, table_id: str) -> str:
    """The registry id a ``(review, table_id)`` maps to (shared with migration)."""
    return _migrated_org_id(review, table_id)

find_owner

find_owner(org_id: str, *, graphs_dir: 'Path | None' = None) -> 'tuple[str, str] | None'

Locate which owner currently holds an org with org_id (stable identity).

Scans both owner trees for <org_id>.json. Because an org id is owner-independent (<review>-<table_id>), this recovers the owner an org was created under even after the originating review's manifest scope was edited or the manifest was deleted entirely. The registry — not the mutable review manifest — is the source of truth for an existing org's location, so rename/delete must resolve ownership through here before falling back to the manifest (otherwise a re-scoped or deleted review silently orphans the org).

Deterministic: owner types are scanned in sorted order; in practice a given id exists under at most one owner.

Source code in zettelkasten/organizations.py

def find_owner(org_id: str, *, graphs_dir: "Path | None" = None) -> "tuple[str, str] | None":
    """Locate which owner currently holds an org with ``org_id`` (stable identity).

    Scans both owner trees for ``<org_id>.json``. Because an org id is
    owner-independent (``<review>-<table_id>``), this recovers the owner an org
    was *created* under even after the originating review's manifest scope was
    edited or the manifest was deleted entirely. The registry — not the mutable
    review manifest — is the source of truth for an existing org's location, so
    rename/delete must resolve ownership through here before falling back to the
    manifest (otherwise a re-scoped or deleted review silently orphans the org).

    Deterministic: owner types are scanned in sorted order; in practice a given
    id exists under at most one owner.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    try:
        validate_id(org_id, kind="organization id", for_filename=True)
    except ValueError:
        return None
    root = _org_root(base)
    for ot in sorted(OWNER_TYPES):
        type_dir = root / ot
        if not type_dir.is_dir():
            continue
        for owner_dir in sorted(type_dir.iterdir()):
            if not owner_dir.is_dir():
                continue
            if (owner_dir / f"{org_id}.json").exists():
                return ot, owner_dir.name
    return None

upsert_definition

upsert_definition(owner_type: str, owner_name: str, *, review: str, table_id: str, title: str = '', row_axis: Any = None, columns: Any = None, state: 'str | None' = None, spine_ref: 'str | None' = None, overlay: Any = None, signature: str = '', generated_at: str = '', graphs_dir: 'Path | None' = None) -> dict[str, Any]

Create/refresh the organization mirroring a review's table definition.

The authored definition (title / row axis / columns) is overwritten, but the lifecycle fields (state / spine_ref) and the author's corrections overlay are PRESERVED across rebuilds unless explicitly overridden — so regenerating a grid never silently demotes a promoted spine back to a lens nor loses editorial corrections. created_at is preserved too.

A promoted spine also carries two lifecycle fields that are not part of the authored definition and must survive a live rebuild the same way: the stashed pre-promotion lens_definition (what demote/delete/re-promote restore and re-key durable node ids from) and the embedded schema (the spine's own extraction rubric). Preserving them here keeps a routine grid regenerate — the _mirror_upsert path fired on every build_matrix — from silently dropping them on a promoted org (which would break demote reversibility and re-source extraction). normalize_organization would otherwise reset a lens_definition to None and a schema to {}.

Source code in zettelkasten/organizations.py

def upsert_definition(
    owner_type: str,
    owner_name: str,
    *,
    review: str,
    table_id: str,
    title: str = "",
    row_axis: Any = None,
    columns: Any = None,
    state: "str | None" = None,
    spine_ref: "str | None" = None,
    overlay: Any = None,
    signature: str = "",
    generated_at: str = "",
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Create/refresh the organization mirroring a review's table definition.

    The authored definition (title / row axis / columns) is overwritten, but the
    lifecycle fields (``state`` / ``spine_ref``) and the author's corrections
    ``overlay`` are PRESERVED across rebuilds unless explicitly overridden — so
    regenerating a grid never silently demotes a promoted spine back to a lens nor
    loses editorial corrections. ``created_at`` is preserved too.

    A promoted spine also carries two lifecycle fields that are *not* part of the
    authored definition and must survive a live rebuild the same way: the stashed
    pre-promotion ``lens_definition`` (what demote/delete/re-promote restore and
    re-key durable node ids from) and the embedded ``schema`` (the spine's own
    extraction rubric). Preserving them here keeps a routine grid regenerate — the
    ``_mirror_upsert`` path fired on every ``build_matrix`` — from silently dropping
    them on a promoted org (which would break demote reversibility and re-source
    extraction). ``normalize_organization`` would otherwise reset a lens_definition
    to ``None`` and a schema to ``{}``.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    oid = org_id_for(review, table_id)
    existing = load_organization(ot, on, oid, graphs_dir=base, migrate=False)
    return save_organization(
        {
            "id": oid,
            "title": title or (existing["title"] if existing else table_id),
            "state": state if state is not None else (existing["state"] if existing else "lens"),
            "spine_ref": spine_ref if spine_ref is not None else (existing["spine_ref"] if existing else ""),
            "overlay": overlay if overlay is not None else (existing.get("overlay") if existing else None),
            # Preserve the promoted-spine lifecycle fields across a live rebuild
            # (same discipline as state/spine_ref/overlay above). A brand-new lens
            # org has neither, so a first build correctly leaves them empty.
            "lens_definition": (existing.get("lens_definition") if existing else None),
            "schema": (existing.get("schema") if existing else None),
            # Preserve the readback drift stamp across the live rebuild the render
            # fires right after a promote/resync. This upsert re-stamps ``signature``
            # (the grid-cache key) with the link-form columns, but ``synced_sig`` is a
            # SEPARATE materialize-time drift stamp (corpus + overlay) that must
            # survive so ``spine_is_synced`` can gate a later readback. Not preserving
            # it here would reset it to ``""`` on every regenerate, defeating the gate.
            "synced_sig": (existing.get("synced_sig") if existing else ""),
            "row_axis": row_axis,
            "columns": columns,
            "review": review,
            "table_id": str(table_id),
            # PRESERVE the TRUE migration-provenance flag across live rebuilds, the
            # same way ``state``/``spine_ref``/``created_at`` are preserved. A live
            # ``upsert_definition`` (the ``_mirror_upsert`` rebuild path) must NEVER
            # set it (it is stamped ONLY by ``ensure_migrated``), but it must also
            # not CLEAR it: defaulting to ``False`` here flipped a migrated table-
            # org's flag off on the next routine grid regenerate, so
            # ``_org_synthesis_graphs`` stopped recognizing it as the synthesis
            # graph's overlay → ``port_spines`` minted a SECOND overlay and delete-
            # time reversibility broke (re-critic P1-2). Preserving ``existing``'s
            # value keeps a migrated org migrated across rebuilds while a brand-new
            # live table (no existing org) correctly stays not-migrated.
            "migrated": (existing.get("migrated") if existing else False),
            "signature": signature,
            "generated_at": generated_at,
            "created_at": existing["created_at"] if existing else "",
        },
        owner_type=ot,
        owner_name=on,
        org_id=oid,
        graphs_dir=base,
    )

rename_definition

rename_definition(owner_type: str, owner_name: str, *, review: str, table_id: str, title: str, graphs_dir: 'Path | None' = None) -> bool

Patch a mirrored org's display title. Returns True if the org existed.

Source code in zettelkasten/organizations.py

def rename_definition(
    owner_type: str,
    owner_name: str,
    *,
    review: str,
    table_id: str,
    title: str,
    graphs_dir: "Path | None" = None,
) -> bool:
    """Patch a mirrored org's display title. Returns True if the org existed."""
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    oid = org_id_for(review, table_id)
    existing = load_organization(ot, on, oid, graphs_dir=base, migrate=False)
    if existing is None:
        return False
    existing["title"] = (title or "").strip() or existing.get("table_id") or oid
    save_organization(existing, owner_type=ot, owner_name=on, org_id=oid, graphs_dir=base)
    return True

retitle_spine_apex

retitle_spine_apex(owner_type: str, owner_name: str, org_id: str, *, get_graph: Any, graphs_dir: 'Path | None' = None) -> bool

Re-title a materialized spine's apex node to the org's current display title.

A matrix rename updates the mirrored org title (rename_definition) but does NOT touch the already-materialized apex NODE, so the apex would keep showing the title it was minted with until the next re-promote. This re-titles it in place. The apex is keyed by the durable spine.APEX_NODE_ID sentinel, so this re-titles the SAME node (never mints a second apex) and preserves every edge on it (including the primary sub-spine link). A no-op — returns False — for a lens/unmaterialized org, a blank title, or a spine with no resolvable apex.

Callers that rename a matrix should call this (with get_graph) so the rename reaches the apex immediately; tables.generate_matrix also invokes it on every readback/resync so a rename propagates on the next regenerate without a route change.

Source code in zettelkasten/organizations.py

def retitle_spine_apex(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
) -> bool:
    """Re-title a materialized spine's apex node to the org's current display title.

    A matrix rename updates the mirrored org title (``rename_definition``) but does
    NOT touch the already-materialized apex NODE, so the apex would keep showing the
    title it was minted with until the next re-promote. This re-titles it in place.
    The apex is keyed by the durable ``spine.APEX_NODE_ID`` sentinel, so this
    re-titles the SAME node (never mints a second apex) and preserves every edge on
    it (including the primary sub-spine link). A no-op — returns ``False`` — for a
    lens/unmaterialized org, a blank title, or a spine with no resolvable apex.

    Callers that rename a matrix should call this (with ``get_graph``) so the rename
    reaches the apex immediately; ``tables.generate_matrix`` also invokes it on every
    readback/resync so a rename propagates on the next regenerate without a route
    change.
    """
    from zettelkasten import spine

    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    org = load_organization(ot, on, org_id, graphs_dir=base, migrate=False)
    if org is None or org.get("state") != "spine" or not org.get("spine_ref"):
        return False
    title = str(org.get("title") or "").strip()
    if not title:
        return False
    spine_graph = str(org["spine_ref"])
    ops = spine.SpineGraphOps(get_graph, graphs_dir=base)
    apex_id = spine.find_apex_id(ops, spine_graph, get_graph)
    if not apex_id:
        return False
    # ``ensure_apex_node`` tier 1 finds the apex by the sentinel and re-titles it
    # ONLY when the title actually differs (byte-identical otherwise); type/body are
    # untouched on the re-title path. Never mints (an apex already exists here).
    ops.ensure_apex_node(spine_graph, spine.APEX_NODE_ID, title, "synthesis", "", spine.APEX_TAGS)
    return True

delete_definition

delete_definition(owner_type: str, owner_name: str, *, review: str, table_id: str, graphs_dir: 'Path | None' = None) -> bool

Remove the org mirroring a (review, table_id). Returns True if it existed.

Source code in zettelkasten/organizations.py

def delete_definition(
    owner_type: str,
    owner_name: str,
    *,
    review: str,
    table_id: str,
    graphs_dir: "Path | None" = None,
) -> bool:
    """Remove the org mirroring a ``(review, table_id)``. Returns True if it existed."""
    return delete_organization(owner_type, owner_name, org_id_for(review, table_id), graphs_dir=graphs_dir)

spine_is_synced

spine_is_synced(org: dict[str, Any], *, graphs_dir: 'Path | None' = None) -> bool

True when a materialized spine already reflects the corpus + corrections.

A spine is synced when neither its in-scope corpus nor its edge-affecting overlay corrections have changed since the last promote/resync stamp. When it is, a Generate can READ BACK the grid over the existing spine-member edges instead of re-materializing (re-routing + re-synthesizing + edge writes) — "open a matrix I already built" becomes O(read), not an agent round-trip.

Recomputes :func:_synced_fingerprint and compares it to the synced_sig stamp. Deliberately conservative — an empty/missing stamp, an empty recompute, or any hiccup returns False so we resync rather than risk skipping a needed materialize. A legacy spine (no synced_sig) therefore resyncs once (stamping the field) and takes the fast path only thereafter.

Source code in zettelkasten/organizations.py

def spine_is_synced(org: dict[str, Any], *, graphs_dir: "Path | None" = None) -> bool:
    """True when a materialized spine already reflects the corpus + corrections.

    A spine is *synced* when neither its in-scope corpus nor its edge-affecting
    overlay corrections have changed since the last promote/resync stamp. When it is,
    a Generate can READ BACK the grid over the existing ``spine-member`` edges instead
    of re-materializing (re-routing + re-synthesizing + edge writes) — "open a matrix
    I already built" becomes O(read), not an agent round-trip.

    Recomputes :func:`_synced_fingerprint` and compares it to the ``synced_sig``
    stamp. Deliberately conservative — an empty/missing stamp, an empty recompute, or
    any hiccup returns ``False`` so we resync rather than risk skipping a needed
    materialize. A legacy spine (no ``synced_sig``) therefore resyncs once (stamping
    the field) and takes the fast path only thereafter.
    """
    if not isinstance(org, dict) or org.get("state") != "spine":
        return False
    stored = str(org.get("synced_sig") or "").strip()
    if not stored:
        return False
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    current = _synced_fingerprint(org, base=base)
    return bool(current) and current == stored

promote_organization

promote_organization(owner_type: str, owner_name: str, org_id: str, *, get_graph: Any, graphs_dir: 'Path | None' = None, attach_relation: 'str | None' = None, row_relation: 'str | None' = None, link_relation: 'str | None' = None, group_fn: Any = None, proposed_grid: 'list[dict[str, Any]] | None' = None, tag_stamp: bool = False) -> dict[str, Any]

Materialize an org lens into a spine graph + bulk-attach its members.

Builds the spine scaffold (apex + one dimension per materializable column + one hub per row), bulk-attaches the current cell members to their dimension + hub nodes, sets state='spine' + spine_ref, and rewrites the org's row axis + columns to their LINK-FORM so future builds route deterministically via the materialized edges. The pre-promotion definition is stashed in lens_definition for demote/delete_spine to restore.

Membership comes from the reconciled grid the live build route persisted (so a semantic/agent lens materializes the agent-routed membership the author saw, not an empty deterministic grid); group_fn is the agent seam a caller can thread for a semantic lens with no persisted grid. The whole load→materialize→ save runs under a per-org lock so concurrent promote/resync/delete can't clobber spine_ref/lens_definition or double-write edges.

The synthesis graph is NOT registered as a project source (spines are additive, selectable overlays — not corpus members).

A re-mining attach reuses this exact path by passing proposed_grid — an already-classified, approved schema-matrix grid (rows whose cells carry members with provenance='reviewer-inferred' + confidence/verified/quote). Promotion then materializes it through the SAME scaffold/attach/reconcile machinery (membership is reconciled, never add-only, so a re-run prunes members no longer classified and adds new ones — idempotent). Members are ADDITIVE to existing dimension nodes: a node's synthesized body (e.g. a ported persona spine's portrait) is never overwritten. tag_stamp (default OFF) is the one opt-in that mutates base notes — stamping each materializable column's dimension tag onto its classified members so a future deterministic build routes them without the classifier.

Source code in zettelkasten/organizations.py

def promote_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
    attach_relation: "str | None" = None,
    row_relation: "str | None" = None,
    link_relation: "str | None" = None,
    group_fn: Any = None,
    proposed_grid: "list[dict[str, Any]] | None" = None,
    tag_stamp: bool = False,
) -> dict[str, Any]:
    """Materialize an org lens into a spine graph + bulk-attach its members.

    Builds the spine scaffold (apex + one dimension per materializable column +
    one hub per row), bulk-attaches the current cell members to their
    dimension + hub nodes, sets ``state='spine'`` + ``spine_ref``, and rewrites
    the org's row axis + columns to their LINK-FORM so future builds route
    deterministically via the materialized edges. The pre-promotion definition is
    stashed in ``lens_definition`` for ``demote``/``delete_spine`` to restore.

    Membership comes from the reconciled grid the live build route persisted (so a
    semantic/agent lens materializes the agent-routed membership the author saw,
    not an empty deterministic grid); ``group_fn`` is the agent seam a caller can
    thread for a semantic lens with no persisted grid. The whole load→materialize→
    save runs under a per-org lock so concurrent promote/resync/delete can't
    clobber ``spine_ref``/``lens_definition`` or double-write edges.

    The synthesis graph is NOT registered as a project source (spines are
    additive, selectable overlays — not corpus members).

    A re-mining attach reuses this exact path by passing ``proposed_grid`` — an
    already-classified, approved schema-matrix grid (rows whose cells carry
    members with ``provenance='reviewer-inferred'`` + confidence/verified/quote).
    Promotion then materializes it through the SAME scaffold/attach/reconcile
    machinery (membership is reconciled, never add-only, so a re-run prunes
    members no longer classified and adds new ones — idempotent). Members are
    ADDITIVE to existing dimension nodes: a node's synthesized body (e.g. a ported
    persona spine's portrait) is never overwritten. ``tag_stamp`` (default OFF) is
    the one opt-in that mutates base notes — stamping each materializable column's
    dimension tag onto its classified members so a future deterministic build
    routes them without the classifier.
    """
    from zettelkasten import spine

    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    # Hold the canonical ``box::{spine_graph}`` lock across the WHOLE materialize
    # read-modify-write so a concurrent MCP-server / matrix-build write to the same
    # spine nodes can't drop an update or mint a duplicate (P1). The locked graph
    # name is handed to the ops as ``held_box_locks`` so its per-node saves/deletes
    # don't re-acquire the same non-reentrant lock.
    with _spine_rmw_lock(ot, on, org_id, base) as spine_graph_locked:
        return _promote_locked(
            ot, on, org_id, get_graph=get_graph, base=base,
            attach_relation=attach_relation, row_relation=row_relation,
            link_relation=link_relation, group_fn=group_fn,
            proposed_grid=proposed_grid, tag_stamp=tag_stamp,
            held_graph=spine_graph_locked,
        )

demote_organization

demote_organization(owner_type: str, owner_name: str, org_id: str, *, graphs_dir: 'Path | None' = None) -> dict[str, Any]

Stop treating a spine as materialized: state='lens', KEEP nodes/edges.

Non-destructive — the synthesis graph, its nodes, and the attach edges all survive (so a later re-promote / resync still finds them). spine_ref IS cleared and its old value stashed in last_spine_ref: state=='spine' with a non-empty spine_ref is the SINGLE source of truth for "materialized", so a demoted org must read as a lens (state=='lens', blank spine_ref) while last_spine_ref lets a later re-promote re-bind the SAME preserved graph. The link-form row axis + columns are reverted to the stashed lens_definition so the org reads as the agent-driven lens again.

Source code in zettelkasten/organizations.py

def demote_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Stop treating a spine as materialized: ``state='lens'``, KEEP nodes/edges.

    Non-destructive — the synthesis graph, its nodes, and the attach edges all
    survive (so a later re-promote / resync still finds them). ``spine_ref`` IS
    cleared and its old value stashed in ``last_spine_ref``: ``state=='spine'`` with
    a non-empty ``spine_ref`` is the SINGLE source of truth for "materialized", so a
    demoted org must read as a lens (``state=='lens'``, blank ``spine_ref``) while
    ``last_spine_ref`` lets a later re-promote re-bind the SAME preserved graph. The
    link-form row axis + columns are reverted to the stashed ``lens_definition`` so
    the org reads as the agent-driven lens again.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    # Serialize the ``spine_ref`` pointer flip on the SAME ``box::{spine_graph}`` key
    # promote/resync/delete hold, so a concurrent lifecycle op on this org can't
    # interleave its read of ``spine_ref`` with this clear.
    with _spine_rmw_lock(ot, on, org_id, base):
        org = load_organization(ot, on, org_id, graphs_dir=base, migrate=False)
        if org is None:
            raise ValueError(f"organization '{org_id}' not found for {ot} '{on}'")

        lens = org.get("lens_definition")
        if isinstance(lens, dict):
            if lens.get("row_axis") is not None:
                org["row_axis"] = lens["row_axis"]
            if lens.get("columns") is not None:
                org["columns"] = lens["columns"]
        org["state"] = "lens"
        # NULL spine_ref (so "materialized" is unambiguous) but stash its old value
        # so a re-promote re-binds the SAME graph demote preserved. The graph +
        # nodes + edges are NOT touched here (non-destructive), only the pointer.
        old_ref = (org.get("spine_ref") or "").strip()
        if old_ref:
            org["last_spine_ref"] = old_ref
        org["spine_ref"] = ""
        saved = save_organization(org, owner_type=ot, owner_name=on, org_id=org_id, graphs_dir=base)
    return {"organization": saved}

resync_organization

resync_organization(owner_type: str, owner_name: str, org_id: str, *, get_graph: Any, graphs_dir: 'Path | None' = None, attach_relation: 'str | None' = None, row_relation: 'str | None' = None, link_relation: 'str | None' = None, group_fn: Any = None) -> dict[str, Any]

Incrementally re-route NEW/unrouted in-scope notes into an existing spine.

Re-runs the (lens-form) fill to recover the current membership, then adds any MISSING attach edges (idempotent — existing edges are deduped). Corrections are honored, not undone: a member_remove'd note is never re-attached, and a member_add'ed note is attached like any other member. Refreshes the org's drift signature. Unlike promote (which may reuse the persisted reconciled grid), resync RE-ROUTES through group_fn via a fresh gather_rows so a drifted corpus's new notes + overlay member_adds reach an EXISTING semantic group instead of being silently dropped by a stale grid; it runs under the per-org lock.

Source code in zettelkasten/organizations.py

def resync_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
    attach_relation: "str | None" = None,
    row_relation: "str | None" = None,
    link_relation: "str | None" = None,
    group_fn: Any = None,
) -> dict[str, Any]:
    """Incrementally re-route NEW/unrouted in-scope notes into an existing spine.

    Re-runs the (lens-form) fill to recover the current membership, then adds any
    MISSING attach edges (idempotent — existing edges are deduped). Corrections
    are honored, not undone: a ``member_remove``'d note is never re-attached, and
    a ``member_add``'ed note is attached like any other member. Refreshes the
    org's drift signature. Unlike promote (which may reuse the persisted reconciled
    grid), resync RE-ROUTES through ``group_fn`` via a fresh ``gather_rows`` so a
    drifted corpus's new notes + overlay member_adds reach an EXISTING semantic
    group instead of being silently dropped by a stale grid; it runs under the
    per-org lock.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    # Hold ``box::{spine_graph}`` across the whole re-route read-modify-write (P1).
    with _spine_rmw_lock(ot, on, org_id, base) as spine_graph_locked:
        return _resync_locked(
            ot, on, org_id, get_graph=get_graph, base=base,
            attach_relation=attach_relation, row_relation=row_relation,
            link_relation=link_relation, group_fn=group_fn,
            held_graph=spine_graph_locked,
        )

materialize_themes

materialize_themes(owner_type: str, owner_name: str, org_id: str, themes: 'list[dict[str, Any]] | None', *, get_graph: Any, graphs_dir: 'Path | None' = None, link_relation: 'str | None' = None, row_relation: 'str | None' = None) -> dict[str, Any]

Materialize approved THEMES as row-hubs under a materialized spine.

Design §8.1/§8.2: a kept theme is a ROW-HUB grouping its member sources under the single spine — additive over the default per-source rows. Each theme becomes one hub node (tagged :data:_THEME_HUB_TAG, keyed on the durable :func:_theme_row_id), linked from the apex, with a spine-member edge to every member note. Membership is RECONCILED (set_member_targets) so a regenerate that drops a note prunes its edge; hub identity is durable (ensure_hub re-titles the SAME hub on a relabel rather than orphaning it).

themes is a list of {"label": str, "note_ids": ["<graph>::<id>", …]} (the shape :func:tables.suggest_from_description emits). When it is empty / None this is a NO-OP that PRESERVES any existing theme hubs (an empty list means "no new theme layer", NOT "wipe the themes") — the flat source-row grid is today's behavior. When it is non-empty the passed set is authoritative: theme hubs whose row id is not in it are pruned, mirroring the source-row defunct-hub prune, and any live overlay correction keyed on a pruned theme hub is SURFACED (never silently dropped) in orphaned_corrections.

North-star invariant: base notes are NEVER mutated — every edge is a spine-side outgoing spine-member / structural link FROM the theme hub in the spine graph. Runs under the per-org spine lock so it can't race a concurrent promote/resync/delete.

Returns {"theme_hubs": {label: hub_id}, "attached_edges": int, "orphaned_corrections": [...]}. A degrade (org missing / not a spine) returns empties rather than raising, so Generate never fails just because a theme could not be materialized.

Source code in zettelkasten/organizations.py

def materialize_themes(
    owner_type: str,
    owner_name: str,
    org_id: str,
    themes: "list[dict[str, Any]] | None",
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
    link_relation: "str | None" = None,
    row_relation: "str | None" = None,
) -> dict[str, Any]:
    """Materialize approved THEMES as row-hubs under a materialized spine.

    Design §8.1/§8.2: a kept theme is a ROW-HUB grouping its member sources under
    the single spine — additive over the default per-source rows. Each theme
    becomes one hub node (tagged :data:`_THEME_HUB_TAG`, keyed on the durable
    :func:`_theme_row_id`), linked from the apex, with a ``spine-member`` edge to
    every member note. Membership is RECONCILED (``set_member_targets``) so a
    regenerate that drops a note prunes its edge; hub identity is durable
    (``ensure_hub`` re-titles the SAME hub on a relabel rather than orphaning it).

    ``themes`` is a list of ``{"label": str, "note_ids": ["<graph>::<id>", …]}``
    (the shape :func:`tables.suggest_from_description` emits). When it is empty /
    ``None`` this is a NO-OP that PRESERVES any existing theme hubs (an empty list
    means "no new theme layer", NOT "wipe the themes") — the flat source-row grid
    is today's behavior. When it is non-empty the passed set is authoritative:
    theme hubs whose row id is not in it are pruned, mirroring the source-row
    defunct-hub prune, and any live overlay correction keyed on a pruned theme hub
    is SURFACED (never silently dropped) in ``orphaned_corrections``.

    North-star invariant: base notes are NEVER mutated — every edge is a
    spine-side outgoing ``spine-member`` / structural link FROM the theme hub in
    the spine graph. Runs under the per-org spine lock so it can't race a
    concurrent promote/resync/delete.

    Returns ``{"theme_hubs": {label: hub_id}, "attached_edges": int,
    "orphaned_corrections": [...]}``. A degrade (org missing / not a spine)
    returns empties rather than raising, so Generate never fails just because a
    theme could not be materialized.
    """
    from zettelkasten import spine

    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    kept = [t for t in (themes or []) if isinstance(t, dict) and str(t.get("label") or "").strip()]
    # Hold ``box::{spine_graph}`` across the whole theme-hub read-modify-write (P1).
    with _spine_rmw_lock(ot, on, org_id, base) as spine_graph_locked:
        org = load_organization(ot, on, org_id, graphs_dir=base, migrate=False)
        if org is None or org.get("state") != "spine" or not org.get("spine_ref"):
            return {"theme_hubs": {}, "attached_edges": 0, "orphaned_corrections": []}
        # An empty themes list is a deliberate no-op: it means "no theme layer",
        # not "delete existing themes" — so we never touch already-materialized
        # theme hubs unless the caller supplies an authoritative non-empty set.
        if not kept:
            return {"theme_hubs": {}, "attached_edges": 0, "orphaned_corrections": []}

        link_relation = (link_relation or spine.DEFAULT_LINK_RELATION).strip()
        row_relation = (row_relation or spine.MEMBERSHIP_RELATION).strip()
        spine_graph = org["spine_ref"]

        # The caller holds ``box::{spine_graph}`` across this whole RMW (P1).
        ops = spine.SpineGraphOps(
            get_graph, graphs_dir=base,
            held_box_locks={spine_graph_locked} if spine_graph_locked else set(),
        )
        apex_id = spine.find_apex_id(ops, spine_graph, get_graph)

        theme_hubs: dict[str, str] = {}
        attached = 0
        live_row_ids: set[str] = set()
        for theme in kept:
            label = str(theme.get("label") or "").strip()
            row_id = _theme_row_id(label)
            live_row_ids.add(row_id)
            hub_id = ops.ensure_hub(
                spine_graph, row_id, label,
                f"Theme '{label}' of spine '{org.get('title') or org_id}'.",
                tags=(_THEME_HUB_TAG,),
            )
            theme_hubs[label] = hub_id
            if apex_id and hub_id:
                ops.ensure_link(spine_graph, apex_id, hub_id, link_relation)
            # A theme's members are corpus notes addressed ``<home>::<note_id>``.
            desired: set[tuple[str, str]] = set()
            for raw in theme.get("note_ids") or []:
                uid = str(raw or "").strip()
                if "::" not in uid:
                    continue
                home, _, nid = uid.partition("::")
                home, nid = home.strip(), nid.strip()
                if not home or not nid:
                    continue
                desired.add((nid, home))
                if hub_id and ops.ensure_link(
                    spine_graph, hub_id, nid, row_relation, target_graph=home,
                ):
                    attached += 1
            # RECONCILE the hub's membership to EXACTLY the theme's set so a
            # regenerate that drops a note prunes its edge (never add-only).
            if hub_id:
                ops.set_member_targets(spine_graph, hub_id, row_relation, desired)

        # Prune theme hubs no longer in the authoritative set (reconcile by row
        # id). Surface — never silently drop — any live overlay correction keyed
        # on a pruned theme hub's row id (mirrors the source-row defunct-hub
        # prune in ``_resync_locked``).
        stored_overlay = _normalize_table_overlay(org.get("overlay"))
        orphaned_corrections: list[dict[str, Any]] = []
        try:
            for nid in ops.structural_node_ids(spine_graph, (_THEME_HUB_TAG,)):
                row_id = ops.node_row_id(spine_graph, nid)
                if row_id in live_row_ids:
                    continue
                if row_id:
                    for e in stored_overlay["member_adds"]:
                        if e["row_id"] == row_id:
                            orphaned_corrections.append({
                                "kind": "member_add",
                                "row_id": e["row_id"],
                                "col_key": e["col_key"],
                                "note_id": e["note_id"],
                                "ref": f"{e['row_id']}/{e['col_key']}/{e['note_id']}",
                            })
                    for e in stored_overlay["cell_overrides"]:
                        if e["row_id"] == row_id:
                            orphaned_corrections.append({
                                "kind": "cell_override",
                                "row_id": e["row_id"],
                                "col_key": e["col_key"],
                                "ref": f"{e['row_id']}/{e['col_key']}",
                            })
                ops.delete_node(spine_graph, nid)
        except Exception:  # noqa: BLE001 — theme prune must never fail a generate
            logger.warning("theme hub cleanup failed for spine '%s'", spine_graph, exc_info=True)

        if orphaned_corrections:
            logger.warning(
                "materialize_themes '%s': %d human correction(s) orphaned by a "
                "theme prune (surfaced, not applied): %s",
                org_id, len(orphaned_corrections),
                ", ".join(c["ref"] for c in orphaned_corrections),
            )

    return {
        "theme_hubs": theme_hubs,
        "attached_edges": attached,
        "orphaned_corrections": orphaned_corrections,
    }

delete_spine

delete_spine(owner_type: str, owner_name: str, org_id: str, *, get_graph: Any, graphs_dir: 'Path | None' = None) -> dict[str, Any]

Teardown of a spine: delete the spine graph folder, revert org to a lens.

Membership is stored spine-side, so teardown is trivial — deleting the spine graph removes every spine-member edge with it and base note files stay pristine (dangling edges are structurally impossible). A best-effort legacy base-side scrub handles spines materialized under the old model. Then clear spine_ref + revert to lens + clear any dangling default-spine pointer. The org itself is kept (as a lens) — only the materialization is removed.

Source code in zettelkasten/organizations.py

def delete_spine(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Teardown of a spine: delete the spine graph folder, revert org to a lens.

    Membership is stored spine-side, so teardown is trivial — deleting the spine
    graph removes every ``spine-member`` edge with it and base note files stay
    pristine (dangling edges are structurally impossible). A best-effort legacy
    base-side scrub handles spines materialized under the old model. Then clear
    ``spine_ref`` + revert to ``lens`` + clear any dangling default-spine pointer.
    The org itself is kept (as a lens) — only the materialization is removed.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)
    from zettelkasten import spine

    ops = spine.SpineGraphOps(get_graph, graphs_dir=base)
    # W4-B step 1 — LOCK-FREE PRE-SCAN. Resolve this spine's graph name and find
    # EVERY child sub-spine (any owner, any state) whose apex carries a
    # ``component-of`` edge into it. We do this WITHOUT any lock so we can then
    # acquire the parent's spine lock together with EVERY child's spine lock up
    # front, in one global sorted order (see below). The authoritative re-read of
    # each child's edges happens under those held locks in
    # :func:`_reparent_children_on_delete`, so a stale pre-scan can only ever cause
    # an idempotent no-op (edge already gone) or a documented, benign TOCTOU (a
    # brand-new child that connects AFTER the scan — see that function's docstring).
    pre = load_organization(ot, on, org_id, graphs_dir=base, migrate=False)
    spine_graph_pre = (
        (pre.get("spine_ref") or pre.get("last_spine_ref") or "").strip() if pre else ""
    )
    candidates = _scan_child_candidates(
        ops, spine_graph_pre, get_graph, base, ot, on, org_id
    )

    # DEADLOCK-FREE LOCK ACQUISITION (W4-B step 2). ``connect_sub_spine`` and
    # ``disconnect_sub_spine`` acquire ``{lock(child), lock(parent)}`` in SORTED
    # key order. If ``delete_spine`` held only ``lock(P)`` and then reached for a
    # child's ``lock(C)`` where ``C`` sorts BEFORE ``P``, a concurrent
    # ``connect_sub_spine(C, P)`` holding ``lock(C)`` and blocking on ``lock(P)``
    # would close an ABBA cycle. We avoid this the textbook way: acquire the WHOLE
    # lock set — ``lock(P)`` plus every candidate child's ``lock(C)`` — in the SAME
    # global sorted order every other op uses. Since all acquirers (connect,
    # disconnect, delete) take their locks in one shared total order, no cycle can
    # form. The set is de-duped (``P`` never appears among its own children — a
    # child's graph can't equal the parent graph — and the non-reentrant lock is
    # taken at most once per key, so no self-deadlock). Holding every child lock for
    # the whole critical section is also what makes the reparent lost-update-free:
    # a concurrent connect/disconnect/move on any child we touch is serialized
    # behind us and reads our post-reparent state, never a stale snapshot.
    # Hold the CANONICAL ``box::{spine_graph}`` lock for the parent spine AND every
    # child sub-spine graph we may reparent — all in ONE global sorted order so no
    # ABBA cycle can form with a concurrent connect/disconnect (which take their
    # ``box::`` locks in the SAME sorted order). Blank graph names are dropped; the
    # set is de-duped (the parent's graph can't equal a child's). The graphs we lock
    # are handed to ``_delete_spine_locked`` as ``held_box_locks`` so the reparent's
    # per-node child writes don't re-acquire the same NON-reentrant lock.
    held_graphs = {
        g for g in ([spine_graph_pre] + [cg for (_c, _o, _i, cg) in candidates]) if g
    }
    lock_keys = sorted(f"box::{g}" for g in held_graphs)
    with contextlib.ExitStack() as stack:
        for key in lock_keys:
            stack.enter_context(review_write_lock(key, graphs_dir=base))
        return _delete_spine_locked(
            ot, on, org_id, candidates=candidates, get_graph=get_graph, base=base,
            held_graphs=held_graphs,
        )

connect_sub_spine

connect_sub_spine(child_owner_type: str, child_owner_name: str, child_org_id: str, parent_owner_type: str, parent_owner_name: str, parent_org_id: str, *, get_graph: Any, graphs_dir: 'Path | None' = None) -> dict[str, Any]

Nest the CHILD spine beneath the PARENT spine (a primary sub-spine edge).

Resolves each org's synthesis graph + apex node and writes the durable primary child-apex --component-of--> parent-apex cross-graph edge on the child apex (idempotent — a re-connect to the same parent is a no-op). A child has exactly ONE primary parent, so connecting a child that already hangs under a DIFFERENT parent MOVES it (the stale primary edge is removed first). This is an explicit hand-authoring action, so it always wins over any prior edge — the stability rule only protects a hand-authored edge from a BUILD-TIME seed, not from a later explicit connect.

Returns {connected, added, moved_from, child_graph, child_apex, parent_graph, parent_apex}.

Source code in zettelkasten/organizations.py

def connect_sub_spine(
    child_owner_type: str,
    child_owner_name: str,
    child_org_id: str,
    parent_owner_type: str,
    parent_owner_name: str,
    parent_org_id: str,
    *,
    get_graph: Any,
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Nest the CHILD spine beneath the PARENT spine (a primary sub-spine edge).

    Resolves each org's synthesis graph + apex node and writes the durable primary
    ``child-apex --component-of--> parent-apex`` cross-graph edge on the child apex
    (idempotent — a re-connect to the same parent is a no-op). A child has exactly
    ONE primary parent, so connecting a child that already hangs under a DIFFERENT
    parent MOVES it (the stale primary edge is removed first). This is an explicit
    hand-authoring action, so it always wins over any prior edge — the stability
    rule only protects a hand-authored edge from a BUILD-TIME seed, not from a
    later explicit connect.

    Returns ``{connected, added, moved_from, child_graph, child_apex, parent_graph,
    parent_apex}``.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    cot, con = _validate_owner(child_owner_type, child_owner_name)
    pot, pon = _validate_owner(parent_owner_type, parent_owner_name)
    from zettelkasten import spine

    # Hold BOTH spine graph locks up front so no concurrent ``delete_spine(parent)``
    # (which holds ``box::{parent_graph}``, deletes the parent graph, and reverts the
    # parent org to a lens) can interleave between resolving the parent apex and
    # writing the edge — which would leave a dangling
    # ``child --component-of--> parent@deletedGraph`` edge while reporting
    # connected=True. Pre-resolve each org's materialized spine graph so we hold the
    # CANONICAL ``box::{graph}`` lock for both (sorted + de-duped: the child==parent
    # self case collapses to ONE key — no double-acquire on the non-reentrant lock;
    # a concurrent delete/connect/disconnect takes its ``box::`` locks in the SAME
    # sorted order → no ABBA). The authoritative apex resolve happens under the lock.
    child_graph_pre = _materialized_spine_graph(cot, con, child_org_id, base)
    parent_graph_pre = _materialized_spine_graph(pot, pon, parent_org_id, base)
    held_graphs = {g for g in (child_graph_pre, parent_graph_pre) if g}
    lock_keys = sorted(f"box::{g}" for g in held_graphs)
    with contextlib.ExitStack() as stack:
        for key in lock_keys:
            stack.enter_context(review_write_lock(key, graphs_dir=base))
        ops = spine.SpineGraphOps(get_graph, graphs_dir=base, held_box_locks=held_graphs)
        child_graph, child_apex = _resolve_spine_apex(ops, cot, con, child_org_id, get_graph, base)
        parent_graph, parent_apex = _resolve_spine_apex(ops, pot, pon, parent_org_id, get_graph, base)
        if child_graph == parent_graph:
            raise ValueError(
                f"cannot nest spine '{child_org_id}' beneath itself "
                f"(both resolve to graph '{child_graph}')"
            )
        result = spine.connect_sub_spine_edge(
            ops,
            child_graph=child_graph,
            child_apex_id=child_apex,
            parent_graph=parent_graph,
            parent_apex_id=parent_apex,
            respect_existing=False,
        )
    return {
        "connected": result["connected"],
        "added": result["added"],
        "moved_from": result.get("moved_from"),
        "child_graph": child_graph,
        "child_apex": child_apex,
        "parent_graph": parent_graph,
        "parent_apex": parent_apex,
    }

disconnect_sub_spine

disconnect_sub_spine(child_owner_type: str, child_owner_name: str, child_org_id: str, *, parent_owner_type: str = '', parent_owner_name: str = '', parent_org_id: str = '', get_graph: Any, graphs_dir: 'Path | None' = None) -> dict[str, Any]

Detach the CHILD spine from its parent (remove the primary sub-spine edge).

With no parent org given, removes whatever primary parent edge the child apex currently carries. When a parent org IS named, the edge is removed only if it is the child's current primary parent (so a stale request never scrubs the wrong edge). Idempotent: detaching an unnested child reports removed=False.

Returns {removed, parent, child_graph, child_apex}.

Source code in zettelkasten/organizations.py

def disconnect_sub_spine(
    child_owner_type: str,
    child_owner_name: str,
    child_org_id: str,
    *,
    parent_owner_type: str = "",
    parent_owner_name: str = "",
    parent_org_id: str = "",
    get_graph: Any,
    graphs_dir: "Path | None" = None,
) -> dict[str, Any]:
    """Detach the CHILD spine from its parent (remove the primary sub-spine edge).

    With no parent org given, removes whatever primary parent edge the child apex
    currently carries. When a parent org IS named, the edge is removed only if it
    is the child's current primary parent (so a stale request never scrubs the
    wrong edge). Idempotent: detaching an unnested child reports ``removed=False``.

    Returns ``{removed, parent, child_graph, child_apex}``.
    """
    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    cot, con = _validate_owner(child_owner_type, child_owner_name)
    from zettelkasten import spine

    # Lock the child spine, and — only when a parent is actually resolved — the
    # parent spine too, so a concurrent ``delete_spine(parent)`` cannot interleave
    # between resolving the parent apex and the edge write. Same sorted, de-duped
    # acquisition as ``connect_sub_spine`` (ABBA-safe; self case locks once). When
    # no parent org is named, only the child lock is taken (nothing to resolve).
    pot = pon = ""
    # Pre-resolve the child (and, when named, parent) materialized spine graph so we
    # hold the CANONICAL ``box::{graph}`` lock(s) — sorted + de-duped, the SAME
    # global order connect/delete use (ABBA-safe). When no parent org is named only
    # the child graph is locked (nothing else to resolve).
    child_graph_pre = _materialized_spine_graph(cot, con, child_org_id, base)
    held_graphs = {child_graph_pre} if child_graph_pre else set()
    if parent_org_id:
        pot, pon = _validate_owner(
            parent_owner_type or child_owner_type,
            parent_owner_name or child_owner_name,
        )
        parent_graph_pre = _materialized_spine_graph(pot, pon, parent_org_id, base)
        if parent_graph_pre:
            held_graphs.add(parent_graph_pre)
    lock_keys = sorted(f"box::{g}" for g in held_graphs)

    with contextlib.ExitStack() as stack:
        for key in lock_keys:
            stack.enter_context(review_write_lock(key, graphs_dir=base))
        ops = spine.SpineGraphOps(get_graph, graphs_dir=base, held_box_locks=held_graphs)
        child_graph, child_apex = _resolve_spine_apex(ops, cot, con, child_org_id, get_graph, base)
        parent_graph = ""
        parent_apex = ""
        if parent_org_id:
            parent_graph, parent_apex = _resolve_spine_apex(
                ops, pot, pon, parent_org_id, get_graph, base
            )
        result = spine.disconnect_sub_spine_edge(
            ops,
            child_graph=child_graph,
            child_apex_id=child_apex,
            parent_graph=parent_graph,
            parent_apex_id=parent_apex,
        )
    return {
        "removed": result["removed"],
        "parent": result.get("parent"),
        "child_graph": child_graph,
        "child_apex": child_apex,
    }

verify_organization

verify_organization(owner_type: str, owner_name: str, org_id: str, *, graphs_dir: 'Path | None' = None, verify_fn: 'Callable[[str, list[dict[str, Any]]], dict[str, Any]] | None' = None) -> dict[str, Any]

Read-only synthesis audit of an organization's grid (the synth-auditor).

Verifies that every SYNTHESIZED cell summary stays within the notes routed into that cell — the synthesis-layer mirror of source grounding's verify. Reads the persisted reconciled grid (the membership + summaries the live build last wrote) with the corrections overlay applied, so author locked / human cells are treated as authoritative and exempt. It NEVER builds or writes; if no grid has been generated yet there is nothing to audit.

Deterministic checks (no agent required):

• ungrounded_summary (error) — a member-bearing column cell carries a genuine synthesized summary but routed ZERO members: synthesis resting on nothing.
• empty_with_members (warn) — a cell has members but its summary was cleared: unfinished synthesis (the grid analogue of a scaffold node).
• synthesis_over_ungrounded_members (warn) — a 2+-member synthesized cell whose members are ALL ungrounded (no quote / unverified): the synthesis may be sound but its footing is not grounded.
• unbacked_value (info) — a prompt (agent free-text) cell carries a value but cites no member; can't be grounded-checked deterministically.

Semantic check (only when verify_fn is supplied):

• overreach (error) — for a 2+-member genuinely-synthesized cell, verify_fn(summary, members) returns {"supported": bool, "issues": [str, ...]}; an unsupported verdict flags assertions the members don't back. The verify_fn is the agent seam, injected exactly like build_matrix's summarize_fn — when None the pass is deterministic-only.

Returns a structured report: verified (no error issues), per-cell issues, counts, and per-column coverage. built: False signals no grid exists yet.

Source code in zettelkasten/organizations.py

def verify_organization(
    owner_type: str,
    owner_name: str,
    org_id: str,
    *,
    graphs_dir: "Path | None" = None,
    verify_fn: "Callable[[str, list[dict[str, Any]]], dict[str, Any]] | None" = None,
) -> dict[str, Any]:
    """Read-only synthesis audit of an organization's grid (the synth-auditor).

    Verifies that every SYNTHESIZED cell summary stays within the notes routed
    into that cell — the synthesis-layer mirror of source grounding's ``verify``.
    Reads the persisted reconciled grid (the membership + summaries the live build
    last wrote) with the corrections overlay applied, so author ``locked`` /
    ``human`` cells are treated as authoritative and exempt. It NEVER builds or
    writes; if no grid has been generated yet there is nothing to audit.

    Deterministic checks (no agent required):

    * ``ungrounded_summary`` (error) — a member-bearing column cell carries a
      genuine synthesized summary but routed ZERO members: synthesis resting on
      nothing.
    * ``empty_with_members`` (warn) — a cell has members but its summary was
      cleared: unfinished synthesis (the grid analogue of a ``scaffold`` node).
    * ``synthesis_over_ungrounded_members`` (warn) — a 2+-member synthesized cell
      whose members are ALL ungrounded (no quote / unverified): the synthesis may
      be sound but its footing is not grounded.
    * ``unbacked_value`` (info) — a ``prompt`` (agent free-text) cell carries a
      value but cites no member; can't be grounded-checked deterministically.

    Semantic check (only when ``verify_fn`` is supplied):

    * ``overreach`` (error) — for a 2+-member genuinely-synthesized cell,
      ``verify_fn(summary, members)`` returns ``{"supported": bool,
      "issues": [str, ...]}``; an unsupported verdict flags assertions the
      members don't back. The ``verify_fn`` is the agent seam, injected exactly
      like ``build_matrix``'s ``summarize_fn`` — when ``None`` the pass is
      deterministic-only.

    Returns a structured report: ``verified`` (no ``error`` issues), per-cell
    ``issues``, counts, and per-column ``coverage``. ``built: False`` signals no
    grid exists yet.
    """
    from zettelkasten import tables

    base = Path(graphs_dir) if graphs_dir is not None else GRAPHS_DIR
    ot, on = _validate_owner(owner_type, owner_name)

    org = load_organization(ot, on, org_id, graphs_dir=base, migrate=False)
    if org is None:
        raise ValueError(f"organization '{org_id}' not found for {ot} '{on}'")

    _, _, name = _owner_build_scope(org)
    table_id = (org.get("table_id") or "").strip()

    base_report = {
        "organization": org_id,
        "owner": {"type": ot, "name": on},
        "title": org["title"],
        "state": org["state"],
        "verify_fn_used": verify_fn is not None,
        "checked_at": _now(),
    }

    persisted = _persisted_grid(base, name, table_id)
    if persisted is None:
        # No grid has been built — nothing to audit. A non-error "clean" report so
        # a caller can distinguish "no synthesis yet" from "synthesis is sound".
        return {
            **base_report,
            "built": False,
            "verified": True,
            "checked_cells": 0,
            "summarized_cells": 0,
            "exempt_cells": 0,
            "gaps": 0,
            "issues": [],
            "coverage": {},
        }

    # Use the GRID's own columns/axis (what was actually built) and apply the
    # corrections overlay so locked/human edits are honored — a read-only view.
    cols = normalize_columns(persisted.get("columns") or org["columns"])
    table = {
        "rows": persisted.get("rows") or [],
        "columns": cols,
        "row_axis": normalize_row_axis(persisted.get("row_axis") or org["row_axis"]),
    }
    view = tables.apply_overlay(table, org.get("overlay"))

    cols_by_key = {c["key"]: c for c in cols}
    coverage: dict[str, dict[str, int]] = {
        c["key"]: {"label": c["label"], "backing": c["backing"], "filled": 0, "gap": 0, "summarized": 0}
        for c in cols
        if c["backing"] != "identity"
    }

    issues: list[dict[str, Any]] = []
    checked = 0
    summarized = 0
    exempt = 0
    gaps = 0

    def _flag(row: dict[str, Any], col: dict[str, Any], cell: dict[str, Any], kind: str, severity: str, detail: str) -> None:
        members = cell.get("members") or []
        issues.append({
            "row_id": row.get("id", ""),
            "row_label": row.get("label", ""),
            "column_key": col["key"],
            "column_label": col["label"],
            "kind": kind,
            "severity": severity,
            "detail": detail,
            "summary": (cell.get("summary") or cell.get("value") or "").strip(),
            "member_count": len(members),
            "grounded_members": sum(1 for m in members if m.get("quote") or m.get("verified")),
        })

    for row in view.get("rows") or []:
        cells = row.get("cells") or {}
        for key, cell in cells.items():
            col = cols_by_key.get(key)
            if col is None or col["backing"] == "identity":
                continue  # metadata columns carry no synthesis
            if not isinstance(cell, dict):
                continue
            cov = coverage.get(key)

            # Author-authoritative cells are exempt from synthesis grounding.
            if cell.get("locked") or cell.get("source_kind") == "human":
                exempt += 1
                continue

            members = cell.get("members") or []
            summary = (cell.get("summary") or cell.get("value") or "").strip()
            backing = col["backing"]

            # An honest gap (no value, no members) is not an error.
            if cell.get("gap") and not summary and not members:
                gaps += 1
                if cov is not None:
                    cov["gap"] += 1
                continue

            checked += 1
            if cov is not None and (members or summary):
                cov["filled"] += 1

            real_synth = _is_real_synthesis(summary, members)
            if real_synth:
                summarized += 1
                if cov is not None:
                    cov["summarized"] += 1

            if backing in _SYNTHESIS_BACKINGS:
                if summary and not members:
                    _flag(row, col, cell, "ungrounded_summary", _AUDIT_ERROR,
                          "cell carries a synthesized summary but routed zero members")
                    continue
                if members and not summary:
                    _flag(row, col, cell, "empty_with_members", _AUDIT_WARN,
                          "cell has members but no summary (unfinished synthesis)")
                    continue
                if real_synth and len(members) >= 2:
                    grounded = [m for m in members if m.get("quote") or m.get("verified")]
                    if not grounded:
                        _flag(row, col, cell, "synthesis_over_ungrounded_members", _AUDIT_WARN,
                              "synthesis rests on members that are all ungrounded (no quote / unverified)")
                    if verify_fn is not None:
                        verdict = _run_verify_fn(verify_fn, summary, members)
                        if verdict is not None and not verdict.get("supported", True):
                            detail = "; ".join(str(i) for i in (verdict.get("issues") or [])) or "summary asserts beyond its members"
                            _flag(row, col, cell, "overreach", _AUDIT_ERROR, detail)
            elif backing == "prompt":
                if summary and not members:
                    _flag(row, col, cell, "unbacked_value", _AUDIT_INFO,
                          "agent free-text value cites no member note")
                elif verify_fn is not None and real_synth and len(members) >= 2:
                    verdict = _run_verify_fn(verify_fn, summary, members)
                    if verdict is not None and not verdict.get("supported", True):
                        detail = "; ".join(str(i) for i in (verdict.get("issues") or [])) or "summary asserts beyond its members"
                        _flag(row, col, cell, "overreach", _AUDIT_ERROR, detail)

    verified = not any(i["severity"] == _AUDIT_ERROR for i in issues)
    return {
        **base_report,
        "built": True,
        "verified": verified,
        "checked_cells": checked,
        "summarized_cells": summarized,
        "exempt_cells": exempt,
        "gaps": gaps,
        "issues": issues,
        "coverage": coverage,
    }