zettelkasten.enrich

zettelkasten.enrich

OpenAlex citation-metrics enrichment (metadata-only refresh).

Fills/refreshes the two citation metrics that power the Papers view's Influence (cohort-normalized cited_by_count) and citation Recency (counts_by_year) axes on already-known works — each owned source's _meta.yaml and each _citations/<id>.yaml — by resolving them through OpenAlex. Resolution tries the most reliable identifier first: exact DOI, then exact arXiv id, then a STRICT title+year+author match. The fallbacks exist because DOI-less arXiv preprints (common in ML/finance corpora) would otherwise stay pinned at cited_by_count == 0 and drag down the Influence axis; the title match is deliberately strict (rejecting same-title impostors) so a DOI-less work is matched to the right paper or left untouched.

Unlike :func:zettelkasten.server.expand_citations, this performs no citation-graph expansion: it mints no new references/citing works and never touches the note graph. It only updates cited_by_count + counts_by_year on works that already exist in the corpus, which is exactly the gap that leaves Influence pinned at its neutral default and citation Recency dark when a corpus was ingested without those fields.

Conservative on failure: a DOI that does not resolve (offline, 404, no DOI) is a no-op for that work — its stored YAML is left untouched — so a partial/offline run never corrupts the store. Resolved works are refreshed (overwritten) since cited_by_count/counts_by_year are objective, monotonically-growing OpenAlex metrics rather than curated fields.

enrich_citation_metrics

enrich_citation_metrics(*, project: str = '', graph: str = '', graphs_dir: Path | None = None, dry_run: bool = False) -> dict[str, Any]

Refresh cited_by_count + counts_by_year for a scope's works.

Scope resolution:

• graph — that single owned source plus the citations it references / is cited by.
• project — the project's owned sources plus their referenced citations.
• neither — every owned source plus all citation entities.

With dry_run no network call or write happens; the report lists how many sources/citations carry a DOI and how many DOI-less ones will be attempted via the arXiv/title fallback. A live run resolves each work (DOI → arXiv id → strict title match) through OpenAlex and writes the two metrics back atomically (resolved works only — an unresolved work is left untouched), reporting a by_method breakdown of how each was matched.

Source code in zettelkasten/enrich.py

def enrich_citation_metrics(
    *,
    project: str = "",
    graph: str = "",
    graphs_dir: Path | None = None,
    dry_run: bool = False,
) -> dict[str, Any]:
    """Refresh ``cited_by_count`` + ``counts_by_year`` for a scope's works.

    Scope resolution:

    * ``graph`` — that single owned source plus the citations it references /
      is cited by.
    * ``project`` — the project's owned sources plus their referenced citations.
    * neither — every owned source plus *all* citation entities.

    With ``dry_run`` no network call or write happens; the report lists how many
    sources/citations carry a DOI and how many DOI-less ones will be attempted via
    the arXiv/title fallback. A live run resolves each work (DOI → arXiv id →
    strict title match) through OpenAlex and writes the two metrics back atomically
    (resolved works only — an unresolved work is left untouched), reporting a
    ``by_method`` breakdown of how each was matched.
    """
    base = graphs_dir or GRAPHS_DIR
    sources = _scope_sources(project, graph, base)
    scoped = bool(project or graph)

    all_citations = load_citations(graphs_dir=base)

    # Gather the citation ids in scope: those referenced by the scoped sources,
    # or every citation when no scope is given.
    citation_ids: set[str] = set()
    source_meta: dict[str, dict[str, Any]] = {}
    for sname in sources:
        meta = load_source_meta(sname, graphs_dir=base)
        source_meta[sname] = meta
        if scoped:
            for cid in (meta.get("references") or []):
                citation_ids.add(str(cid))
            for cid in (meta.get("cited_by") or []):
                citation_ids.add(str(cid))
    if not scoped:
        citation_ids = set(all_citations.keys())
    # Only ids that actually have a citation file.
    citation_ids &= set(all_citations.keys())

    sources_with_doi = [s for s in sources if str(source_meta[s].get("doi") or "").strip()]
    citations_with_doi = [
        cid for cid in citation_ids if str((all_citations.get(cid) or {}).get("doi") or "").strip()
    ]
    # DOI-less works are no longer dead ends: they are resolved via arXiv id /
    # strict title match. "Resolvable" counts every work carrying any identifier.
    sources_resolvable = [s for s in sources if _has_identifier(source_meta[s])]
    citations_resolvable = [
        cid for cid in citation_ids if _has_identifier(all_citations.get(cid) or {})
    ]

    if dry_run:
        return {
            "dry_run": True,
            "scope": {"project": project, "graph": graph} if scoped else {"all": True},
            "sources_in_scope": len(sources),
            "citations_in_scope": len(citation_ids),
            "sources_with_doi": len(sources_with_doi),
            "citations_with_doi": len(citations_with_doi),
            # Works without a DOI that will be attempted via arXiv id / title.
            "sources_without_doi": len(sources) - len(sources_with_doi),
            "citations_without_doi": len(citation_ids) - len(citations_with_doi),
            "would_enrich": len(sources_resolvable) + len(citations_resolvable),
        }

    citations_dir = base / "_citations"
    sources_enriched = 0
    sources_skipped = 0
    citations_enriched = 0
    citations_skipped = 0
    # How each enriched work was resolved (doi / arxiv / title), so the caller can
    # see how many were rescued by the DOI-less fallback.
    by_method: dict[str, int] = {"doi": 0, "arxiv": 0, "title": 0}
    report: list[dict[str, Any]] = []

    # 1) Owned sources — refresh metrics on their _meta.yaml.
    for sname in sources:
        meta = source_meta[sname]
        metrics, method = _resolve_metrics_for(meta)
        if metrics is None:
            sources_skipped += 1
            continue
        meta["cited_by_count"] = metrics["cited_by_count"]
        meta["counts_by_year"] = metrics["counts_by_year"]
        write_yaml_atomic(base / sname / "_meta.yaml", meta)
        sources_enriched += 1
        by_method[method] = by_method.get(method, 0) + 1
        report.append({
            "kind": "source",
            "id": sname,
            "title": str(meta.get("title") or sname),
            "cited_by_count": metrics["cited_by_count"],
            "resolved_by": method,
        })

    # 2) Citation entities — refresh metrics on their _citations/<id>.yaml.
    for cid in citation_ids:
        cdata = dict(all_citations.get(cid) or {})
        metrics, method = _resolve_metrics_for(cdata)
        if metrics is None:
            citations_skipped += 1
            continue
        cdata["cited_by_count"] = metrics["cited_by_count"]
        cdata["counts_by_year"] = metrics["counts_by_year"]
        write_yaml_atomic(citations_dir / f"{cid}.yaml", cdata)
        citations_enriched += 1
        by_method[method] = by_method.get(method, 0) + 1
        report.append({
            "kind": "citation",
            "id": cid,
            "title": str(cdata.get("title") or cid),
            "cited_by_count": metrics["cited_by_count"],
            "resolved_by": method,
        })

    report.sort(key=lambda r: r["cited_by_count"], reverse=True)
    enriched_total = sources_enriched + citations_enriched
    rescued = by_method.get("arxiv", 0) + by_method.get("title", 0)
    return {
        "dry_run": False,
        "scope": {"project": project, "graph": graph} if scoped else {"all": True},
        "sources_enriched": sources_enriched,
        "sources_skipped": sources_skipped,
        "citations_enriched": citations_enriched,
        "citations_skipped": citations_skipped,
        "enriched": enriched_total,
        "skipped": sources_skipped + citations_skipped,
        "by_method": by_method,
        "top_works": report[:20],
        "message": (
            f"Refreshed citation metrics for {enriched_total} work(s) "
            f"({sources_enriched} source(s), {citations_enriched} citation(s)); "
            f"{rescued} resolved without a DOI (arXiv/title); "
            f"{sources_skipped + citations_skipped} unresolved."
        ),
    }