Per-project view bundles — pre-built, content-addressed render artifacts.
A bundle is the single durable artifact the dashboard needs to render a project
WITHOUT running any native compute (markdown parse, embedding, t-SNE, clustering)
on the request path. It captures the two expensive-to-derive layers for a project:
- structure — the full composite node/edge payload (every source's notes +
chapter/source hubs + claimed
_cross notes + referenced citations), exactly
as :func:projects.get_project_graph assembles it at max_nodes=0; and
- layout — the semantic cluster result (t-SNE
node_positions +
node_clusters + cluster metadata) at the base and fine granularities, exactly
as :func:graph_data.get_project_clusters computes it.
The bundle is content-addressed by the project's source-fileset signature
(each source box's note-mtime fingerprint + the _meta/_citations/manifest
signatures) — the same inputs the routes' own cache keys fold in. A persisted
bundle is served IFF its recorded signature still matches the live sources; any
mismatch, corruption, or version bump is a miss that falls back to a rebuild. So a
bundle is a pure accelerator, never a source of truth — identical in spirit to the
memory dashboard's signature-gated memory.kgl snapshot.
Who builds it. :func:build_project_bundle runs the whole expensive pipeline
SYNCHRONOUSLY in the calling process. It is meant to run OFF the web request
thread — in a worker subprocess (its own native heap, so kglite/BLAS parallelize
safely across cores) or from the CLI/agent pipeline at write time — never inline on
uvicorn's threadpool. The web server only ever reads a bundle
(:func:load_bundle) and slices it by LOD/bbox.
Storage lives beside the per-box .kgl caches under ANGELO_DIR/zettel/bundles
(gitignored, disposable). The .sig sidecar is written AFTER the .json body,
so a signature match always implies a complete bundle; a torn write leaves the sig
absent/stale → a clean miss → rebuild.
bundles_dir
Directory holding per-project bundle files (beside the .kgl caches).
Source code in zettelkasten/bundle.py
| def bundles_dir() -> Path:
"""Directory holding per-project bundle files (beside the ``.kgl`` caches)."""
from zettelkasten.embeddings import _cache_dir
return _cache_dir() / "bundles"
|
bundle_path
bundle_path(project: str) -> Path
Path of a project's bundle body (<project>.json).
Source code in zettelkasten/bundle.py
| def bundle_path(project: str) -> Path:
"""Path of a project's bundle body (``<project>.json``)."""
return bundles_dir() / f"{_safe_name(project)}.json"
|
sig_path
sig_path(project: str) -> Path
Path of a project's bundle signature sidecar (<project>.sig).
Source code in zettelkasten/bundle.py
| def sig_path(project: str) -> Path:
"""Path of a project's bundle signature sidecar (``<project>.sig``)."""
return bundles_dir() / f"{_safe_name(project)}.sig"
|
load_bundle
load_bundle(project: str, signature: str) -> 'dict | None'
Return the persisted bundle for project IFF it matches signature.
Returns None on any miss — either file absent, a recorded-signature
mismatch (sources changed since the build), unreadable/corrupt JSON, or a
version this reader doesn't understand. A None result means "rebuild";
the bundle is an accelerator, so a failure is never fatal.
Source code in zettelkasten/bundle.py
| def load_bundle(project: str, signature: str) -> "dict | None":
"""Return the persisted bundle for ``project`` IFF it matches ``signature``.
Returns ``None`` on any miss — either file absent, a recorded-signature
mismatch (sources changed since the build), unreadable/corrupt JSON, or a
``version`` this reader doesn't understand. A ``None`` result means "rebuild";
the bundle is an accelerator, so a failure is never fatal.
"""
bp = bundle_path(project)
sp = sig_path(project)
try:
if not bp.exists() or not sp.exists():
return None
if sp.read_text(encoding="utf-8") != signature:
return None
data = json.loads(bp.read_text(encoding="utf-8"))
except Exception:
logger.warning("Bundle load failed for project %r; ignoring", project, exc_info=True)
return None
if not isinstance(data, dict) or data.get("version") != BUNDLE_VERSION:
return None
# Cross-process tear guard: the ``.sig`` sidecar and the body are two separate
# atomic writes with no cross-process lock, so two builders racing the same
# project can interleave as (A.body, B.body, B.sig, A.sig) — leaving B's body
# paired with A's ``.sig``. The sidecar then matches a reader's signature while
# the body was actually built under a DIFFERENT one. The body carries the
# signature it was built under, so a disagreement means a torn pairing → miss
# (→ rebuild). A single writer always agrees, so this never rejects a clean
# bundle. (Also invalidates pre-v2 bundles, which lack the field.)
if data.get(_SIG_FIELD) != signature:
return None
return data
|
write_bundle
write_bundle(project: str, signature: str, payload: 'dict') -> Path
Persist payload as project's bundle, then stamp signature.
The body is written and atomically renamed FIRST, the .sig sidecar SECOND,
so a reader's signature match always implies a complete body (a crash between
the two just leaves a stale/absent sig → a safe miss → rebuild). payload is
stored verbatim under the given signature and stamped with
:data:BUNDLE_VERSION if not already present.
The signature is ALSO embedded in the body (:data:_SIG_FIELD) so a torn
cross-process pairing — where the surviving .sig came from a different
build than the surviving body — is caught on load (see :func:load_bundle).
Source code in zettelkasten/bundle.py
| def write_bundle(project: str, signature: str, payload: "dict") -> Path:
"""Persist ``payload`` as ``project``'s bundle, then stamp ``signature``.
The body is written and atomically renamed FIRST, the ``.sig`` sidecar SECOND,
so a reader's signature match always implies a complete body (a crash between
the two just leaves a stale/absent sig → a safe miss → rebuild). ``payload`` is
stored verbatim under the given ``signature`` and stamped with
:data:`BUNDLE_VERSION` if not already present.
The signature is ALSO embedded in the body (:data:`_SIG_FIELD`) so a torn
cross-process pairing — where the surviving ``.sig`` came from a different
build than the surviving body — is caught on load (see :func:`load_bundle`).
"""
from zettelkasten.graph_io import atomic_write_text
body = dict(payload)
body.setdefault("version", BUNDLE_VERSION)
body[_SIG_FIELD] = signature
bp = bundle_path(project)
atomic_write_text(bp, json.dumps(body, separators=(",", ":"), default=str))
atomic_write_text(sig_path(project), signature)
return bp
|
clear_bundle
clear_bundle(project: str) -> None
Best-effort removal of a project's bundle + sidecar (e.g. on a hard reset).
Source code in zettelkasten/bundle.py
| def clear_bundle(project: str) -> None:
"""Best-effort removal of a project's bundle + sidecar (e.g. on a hard reset)."""
for p in (sig_path(project), bundle_path(project)):
try:
p.unlink(missing_ok=True)
except OSError:
logger.debug("Could not remove bundle file %s", p, exc_info=True)
|
build_project_bundle
build_project_bundle(project: str) -> 'dict | None'
Build, persist, and return the view bundle for project (synchronous).
Runs the FULL expensive pipeline in the calling process: force-warms every
source box's embedding index (parse + embed, persisting each .kgl),
assembles the composite structure payload, and computes the base + fine
semantic layouts (t-SNE + clustering). Intended for a worker subprocess or the
write-time pipeline — NOT the web request thread.
Returns the built bundle (also written to disk), or None when the project
is unknown/unsafe or the embedding stack is unavailable so no meaningful layout
could be produced (structure-only degradation is still persisted so the reader
at least skips the re-parse).
Source code in zettelkasten/bundle.py
| def build_project_bundle(project: str) -> "dict | None":
"""Build, persist, and return the view bundle for ``project`` (synchronous).
Runs the FULL expensive pipeline in the calling process: force-warms every
source box's embedding index (parse + embed, persisting each ``.kgl``),
assembles the composite structure payload, and computes the base + fine
semantic layouts (t-SNE + clustering). Intended for a worker subprocess or the
write-time pipeline — NOT the web request thread.
Returns the built bundle (also written to disk), or ``None`` when the project
is unknown/unsafe or the embedding stack is unavailable so no meaningful layout
could be produced (structure-only degradation is still persisted so the reader
at least skips the re-parse).
"""
# Lazily import the route-layer helpers: they own project resolution
# (federation, manifest), the box warm, the composite assembly, and the
# cluster compute. Importing here (not at module load) keeps ``bundle`` usable
# from the CLI without eagerly dragging in the FastAPI app, and avoids an
# import cycle (the routes import ``bundle`` for the read path).
from zettelkasten.dashboard.backend.routes import projects as P
sig = P.project_signature(project)
if sig is None:
logger.info("Bundle build skipped: project %r is unknown/unsafe", project)
return None
payload = P.build_project_bundle_payload(project)
if payload is None:
return None
write_bundle(project, sig, payload)
logger.info(
"Built project bundle %r: %d nodes, %d edges, layout=%s",
project,
len(payload.get("structure", {}).get("nodes", [])),
len(payload.get("structure", {}).get("edges", [])),
"yes" if payload.get("layout") else "structure-only",
)
return {**payload, "version": BUNDLE_VERSION}
|