feat: discussion artifacts as wiki pages, in two new layers
A discussion leaves behind a directory of markdown somewhere outside this repo, and the only durable home for it is the Gitea wiki. Getting it there by hand means re-deriving the same three things every time: what each file should be called, where it goes, and whether the page already exists. Two skills wrap that, along the split the repo already uses. `skills/page` is domain, offline, stdlib-only, and knows nothing about Gitea. It imports a directory into a space under `tmp/wiki/`, titles every file, records the result in `.pages.json`, and writes the index. `skills/wiki` is the bridge — `wikimap.py` translates, and the transport is `_gitea.py`, the same one the issue side uses. There is no second transport, and `tea` has no wiki subcommand to offer one. The Gitea wiki is flat, and that fact shapes everything There are no directories. A title of `a/b` is stored as one file named `a%2Fb.md`, and Gitea escapes it by rules of its own: space becomes `-`, `/` becomes `%2F`, and a literal `-` forces a trailing `.-` marker so the two stay distinct. `Chain decisions — DC` under two levels of prefix comes back as `Simple-Chains%2FParked%2FChain-decisions-%E2%80%94-DC`. So `sub_url` is the identity, it is read back from whatever the API returned, and it is never constructed. One built by hand that is almost right does not fail — it creates a second page and abandons the first. And a real subdirectory committed into a wiki's git repository is a ghost: the file exists, the API and the web UI do not see it. `folder/page.md` in this repo's own wiki is one. Nothing here clones a wiki repo. A title is a decision, not a derivation Titles come from the first heading, because there is no mechanical route from `03-q-01-do-we-know-the-chain-participant-by-name.md` to `Q-01. Do We Know the Chain Participant by Name`. But they are derived exactly once. A re-import replaces bodies and keeps titles, so editing a heading cannot rename a published page — which would not rename it, it would publish a second one. `--retitle` opts in. It finds the prior entry by `source` rather than by path, because the path is derived from the title and a retitle moves it; looked up by path the page would read as new and the next push would duplicate it. The old file goes, `sub_url` comes along, and `pushed` is cleared — a rename can leave the body byte-identical, and push decides by body hash alone, so a stale hash would skip the rename forever. Ordering is a `NN-` file-name prefix and never reaches the title. `00-` means "this is the directory's own page", and that page is named for the directory, not for its own heading: a child's title has to extend its parent's exactly, and `ideas/00-intro.md` opens with "Ideas for chain business requirements". Path collisions are reported and never resolved. Picking a winner is how a discussion loses a document. The index is navigation, not decoration Nothing draws a tree from flat titles. `page_index.py` writes one as an ordinary page, nested by title depth rather than by manifest path order — those disagree, since on disk `Top/System.md` sorts before `Top/Ideas/Scale.md` while in the hierarchy System is a child and Scale a grandchild. A parent with no page of its own still gets a node, so its children are not hidden. Links use `sub_url` when there is one and Gitea's `[[Title|label]]` syntax when there is not, so the order is push, rebuild, push. The same stances as the issue store, for the same reasons Pull overwrites, push is additive and never deletes, change detection is one hash and there is no drift model. A page with no `sub_url` has never been published, and that is a durable state. Issues gain a `wiki:` field holding page titles — titles, not URLs, so the reference stays in the domain. It already round-trips as a foreign key; this documents it. Verified against a live Gitea 1.26.1: create, update with a message, unchanged-skip, prefix-filtered pull, byte-identical round trip, and the per-page revision history carrying the operator's own words. The probe pages were deleted afterwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
page_import.py — pull a directory of markdown into a space. Offline.
|
||||
|
||||
This is the "wiki organization" step, and it is the only step where a page gets
|
||||
its name. A discussion produces artifacts wherever the discussion happened:
|
||||
|
||||
~/…/mpns/feat/simple-chains/tmp/simple-chains/
|
||||
handoff.md scope.md
|
||||
ideas/00-intro.md ideas/02-chain-core.md
|
||||
questions/03-q-01-do-we-know-the-chain-participant-by-name.md
|
||||
|
||||
Import copies that tree into a space under `tmp/wiki/`, gives every file a
|
||||
title, and records both in the manifest. Nothing here talks to a wiki; the
|
||||
result is a complete, readable, greppable tree whether or not it is ever
|
||||
published.
|
||||
|
||||
page_import.py --from DIR --space claude-skills/tea --prefix "Simple Chains"
|
||||
|
||||
Simple-Chains/Handoff.md Simple Chains/Handoff
|
||||
Simple-Chains/Ideas.md Simple Chains/Ideas
|
||||
Simple-Chains/Ideas/Chain-core.md Simple Chains/Ideas/Chain core
|
||||
|
||||
Re-importing is safe and is the normal way to refresh: a page already in the
|
||||
manifest keeps its title (a title is a decision, not a derivation) and only its
|
||||
body is replaced. `--retitle` opts into re-deriving titles, which is a rename
|
||||
and, for pages already published, will orphan the old ones — so it is never the
|
||||
default.
|
||||
|
||||
Usage:
|
||||
page_import.py --from DIR [--space SPACE] [--prefix TITLE]
|
||||
[--retitle] [--dry-run] [--out DIR]
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import page # noqa: E402
|
||||
|
||||
|
||||
def die(msg, code=1):
|
||||
sys.stderr.write("%s: %s\n" % (os.path.basename(sys.argv[0]), msg))
|
||||
sys.exit(code)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--from", dest="src", required=True,
|
||||
help="directory of markdown to import")
|
||||
ap.add_argument("--space", default="local",
|
||||
help="space to import into (default: local)")
|
||||
ap.add_argument("--prefix", default="",
|
||||
help="title every imported page hangs under")
|
||||
ap.add_argument("--retitle", action="store_true",
|
||||
help="re-derive titles of pages already in the manifest "
|
||||
"(a rename; orphans published pages)")
|
||||
ap.add_argument("--dry-run", action="store_true")
|
||||
ap.add_argument("--out", help="wiki cache root (default: <repo>/tmp/wiki)")
|
||||
a = ap.parse_args()
|
||||
|
||||
src = os.path.abspath(a.src)
|
||||
if not os.path.isdir(src):
|
||||
die("not a directory: %s" % a.src)
|
||||
|
||||
root = a.out or page.WIKI_ROOT
|
||||
prefix = page.sanitize_title(a.prefix) if a.prefix else ""
|
||||
|
||||
pages, collisions = page.plan_import(src, prefix)
|
||||
if not pages:
|
||||
die("no markdown found under %s" % src)
|
||||
if collisions:
|
||||
for path, titles in collisions:
|
||||
sys.stderr.write("collision: %s <- %s\n" % (path, " | ".join(titles)))
|
||||
die("%d path collision(s); rename the sources and retry" % len(collisions))
|
||||
|
||||
manifest = page.load_manifest(a.space, root)
|
||||
known = manifest["pages"]
|
||||
dest_root = page.space_root(a.space, root)
|
||||
# Asked before anything is written: nothing should create a space as a
|
||||
# silent side effect of a write, and saying so on stderr is how the
|
||||
# operator learns a typo in --space made a second one.
|
||||
created = not os.path.isdir(dest_root)
|
||||
|
||||
new = changed = same = moved = 0
|
||||
for p in pages:
|
||||
# Looked up by SOURCE, not by path: a retitle moves the path, and a
|
||||
# lookup that missed would treat the page as new and publish a
|
||||
# duplicate beside the one it was meant to rename.
|
||||
prior_path, prior = page.find_by_source(manifest, p["rel"], prefix)
|
||||
if prior is None:
|
||||
prior_path, prior = p["path"], known.get(p["path"])
|
||||
|
||||
# A title already in the manifest is a decision that was made once.
|
||||
# Re-deriving it on every import would let an edited heading silently
|
||||
# rename a published page — which does not rename it, it creates a
|
||||
# second one and abandons the first.
|
||||
title = p["title"] if (a.retitle or not prior) else prior["title"]
|
||||
relpath = page.path_for_title(title)
|
||||
dest = os.path.join(dest_root, relpath)
|
||||
|
||||
state = "new"
|
||||
if prior and relpath != prior_path:
|
||||
state = "moved"
|
||||
elif prior and os.path.isfile(dest):
|
||||
with open(dest, encoding="utf-8") as f:
|
||||
state = "same" if f.read() == p["text"] else "changed"
|
||||
elif prior:
|
||||
state = "changed"
|
||||
|
||||
new += state == "new"
|
||||
changed += state == "changed"
|
||||
same += state == "same"
|
||||
moved += state == "moved"
|
||||
|
||||
print("%-7s %-44s %s" % (state, relpath, title))
|
||||
if a.dry_run:
|
||||
continue
|
||||
|
||||
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||
shutil.copyfile(p["source"], dest)
|
||||
# Passthrough keys survive: a re-import must not cost a page its
|
||||
# sub_url, or the next push would publish a duplicate.
|
||||
e = dict(prior or {})
|
||||
e.update(page.entry(title, p["order"], p["rel"]))
|
||||
if state == "moved":
|
||||
# The old copy goes, the entry moves with its bookkeeping intact.
|
||||
# The page in the wiki is still at its old sub_url; the next push
|
||||
# sends the new title, which is what renames it there.
|
||||
old = os.path.join(dest_root, prior_path)
|
||||
if os.path.isfile(old):
|
||||
os.remove(old)
|
||||
known.pop(prior_path, None)
|
||||
# A rename can leave the body byte-identical, and push decides by
|
||||
# body hash alone. Clearing it is what makes the next push send the
|
||||
# new title instead of skipping the page as unchanged.
|
||||
e.pop("pushed", None)
|
||||
known[relpath] = e
|
||||
|
||||
if a.dry_run:
|
||||
print("\ndry run — nothing written")
|
||||
return 0
|
||||
|
||||
path = page.save_manifest(manifest, root)
|
||||
if created:
|
||||
sys.stderr.write("created space %s\n" % dest_root)
|
||||
print("\n%d new, %d changed, %d unchanged%s -> %s"
|
||||
% (new, changed, same,
|
||||
", %d renamed" % moved if moved else "", os.path.dirname(path)))
|
||||
if moved:
|
||||
sys.stderr.write(
|
||||
"%d page(s) renamed. A published page is renamed in the wiki by "
|
||||
"the next push, not by this import.\n" % moved)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user