claude-skills/marketplace
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

feat: discussion artifacts as wiki pages, in two new layers

Browse Source 
A discussion leaves behind a directory of markdown somewhere outside this
repo, and the only durable home for it is the Gitea wiki. Getting it
there by hand means re-deriving the same three things every time: what
each file should be called, where it goes, and whether the page already
exists. Two skills wrap that, along the split the repo already uses.

`skills/page` is domain, offline, stdlib-only, and knows nothing about
Gitea. It imports a directory into a space under `tmp/wiki/`, titles
every file, records the result in `.pages.json`, and writes the index.
`skills/wiki` is the bridge — `wikimap.py` translates, and the transport
is `_gitea.py`, the same one the issue side uses. There is no second
transport, and `tea` has no wiki subcommand to offer one.

The Gitea wiki is flat, and that fact shapes everything

There are no directories. A title of `a/b` is stored as one file named
`a%2Fb.md`, and Gitea escapes it by rules of its own: space becomes `-`,
`/` becomes `%2F`, and a literal `-` forces a trailing `.-` marker so the
two stay distinct. `Chain decisions — DC` under two levels of prefix
comes back as `Simple-Chains%2FParked%2FChain-decisions-%E2%80%94-DC`.

So `sub_url` is the identity, it is read back from whatever the API
returned, and it is never constructed. One built by hand that is almost
right does not fail — it creates a second page and abandons the first.

And a real subdirectory committed into a wiki's git repository is a ghost:
the file exists, the API and the web UI do not see it. `folder/page.md`
in this repo's own wiki is one. Nothing here clones a wiki repo.

A title is a decision, not a derivation

Titles come from the first heading, because there is no mechanical route
from `03-q-01-do-we-know-the-chain-participant-by-name.md` to
`Q-01. Do We Know the Chain Participant by Name`. But they are derived
exactly once. A re-import replaces bodies and keeps titles, so editing a
heading cannot rename a published page — which would not rename it, it
would publish a second one.

`--retitle` opts in. It finds the prior entry by `source` rather than by
path, because the path is derived from the title and a retitle moves it;
looked up by path the page would read as new and the next push would
duplicate it. The old file goes, `sub_url` comes along, and `pushed` is
cleared — a rename can leave the body byte-identical, and push decides by
body hash alone, so a stale hash would skip the rename forever.

Ordering is a `NN-` file-name prefix and never reaches the title. `00-`
means "this is the directory's own page", and that page is named for the
directory, not for its own heading: a child's title has to extend its
parent's exactly, and `ideas/00-intro.md` opens with "Ideas for chain
business requirements".

Path collisions are reported and never resolved. Picking a winner is how
a discussion loses a document.

The index is navigation, not decoration

Nothing draws a tree from flat titles. `page_index.py` writes one as an
ordinary page, nested by title depth rather than by manifest path order —
those disagree, since on disk `Top/System.md` sorts before
`Top/Ideas/Scale.md` while in the hierarchy System is a child and Scale a
grandchild. A parent with no page of its own still gets a node, so its
children are not hidden.

Links use `sub_url` when there is one and Gitea's `[[Title|label]]`
syntax when there is not, so the order is push, rebuild, push.

The same stances as the issue store, for the same reasons

Pull overwrites, push is additive and never deletes, change detection is
one hash and there is no drift model. A page with no `sub_url` has never
been published, and that is a durable state.

Issues gain a `wiki:` field holding page titles — titles, not URLs, so
the reference stays in the domain. It already round-trips as a foreign
key; this documents it.

Verified against a live Gitea 1.26.1: create, update with a message,
unchanged-skip, prefix-filtered pull, byte-identical round trip, and the
per-page revision history carrying the operator's own words. The probe
pages were deleted afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
naudachu
2026-08-10 16:29:55 +05:00
parent 257c547e22
commit 1815d91cdf
16 changed files with 2319 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/page/SKILL.md
+88
View File
@@ -0,0 +1,88 @@
---
name: page
description: Organize a discussion's artifacts into a named, ordered tree of wiki pages — import a directory of markdown, give every file a title, build the index, see what a space holds. Entirely offline; pages are local markdown files and need no wiki. Load when the user asks to turn notes/artifacts into wiki pages, organize or re-title a page tree, or rebuild a table of contents. For fetching from or publishing to a Gitea wiki, load /tea:wiki instead.
---
# /tea:page — discussion artifacts as a page tree
A discussion produces artifacts wherever the discussion happened — a directory
of markdown with numbered files and subdirectories. This skill turns that into
a **space**: a named, ordered tree of pages with a manifest, living under
`tmp/wiki/`.
**Nothing here touches the network.** No `tea`, no Gitea, no login. A space that
never leaves this machine is a finished thing, not a draft waiting for an
upload. Publishing is a separate, optional layer — `/tea:wiki`.
Read [`references/pages.md`](references/pages.md) before importing or
re-titling. It is the single source of truth for titles, ordering, paths, the
manifest, and the index.
## Identity: the title
`Simple Chains/Ideas/Chain core`. The `/` is the only hierarchy there is — the
wiki this feeds is flat and has no directories. The local path is derived from
the title (`Simple-Chains/Ideas/Chain-core.md`); the reverse never happens.
A title is chosen **once**, at import or at pull, and then it is a fact in the
manifest. Editing a heading does not rename a page. Renaming is `--retitle`,
and on a published page it orphans the old one.
## Scripts
All offline, all in `<skill-base-dir>/scripts/`.
| Script | What it does |
|---|---|
| `page_import.py --from DIR [--space S] [--prefix T]` | copy a directory of markdown into a space, titling every file |
| `page_index.py [--space S] [--prefix T]` | write the table-of-contents page — the navigation the flat wiki cannot provide |
| `page_ls.py [--space S] [--prefix T]` | the tree, the titles, and one sync-state tag per page |
| `page.py` | the domain module the others import — not a command |
```
tmp/wiki/claude-skills/tea/ a space
.pages.json the manifest — titles, order, sync bookkeeping
Simple-Chains.md the index page
Simple-Chains/Ideas.md title: Simple Chains/Ideas
Simple-Chains/Ideas/Chain-core.md title: Simple Chains/Ideas/Chain core
```
## The usual run
```bash
python3 scripts/page_import.py \
--from ~/proj/tmp/simple-chains \
--space claude-skills/tea --prefix "Simple Chains" --dry-run
```
`--dry-run` first, always: it prints every path and the title it would get, and
that listing is the only chance to catch a heading that titles a page badly
before the name becomes a decision. Drop the flag to write.
Then the index, then look at it:
```bash
python3 scripts/page_index.py --space claude-skills/tea --prefix "Simple Chains"
python3 scripts/page_ls.py --space claude-skills/tea --prefix "Simple Chains"
```
`page_ls.py` tags each page `local` (never published), `synced` (published and
unchanged), or `ahead` (edited since it was published). `local` is a complete
state.
## Where the cache is
`<repo root>/tmp/wiki` — **not** `tmp/wiki` relative to wherever you are
standing. The scripts resolve it by walking up from their own file to the
nearest `.git` or `AGENTS.md`, so they all see one cache no matter which
directory they are run from.
`--out` overrides that and is taken **literally**: an absolute path is used as
given, a relative one stays relative to the current directory.
## Re-importing is the normal refresh
The discussion continues, the artifacts change, run the same import again.
Bodies are replaced, titles are kept, `sub_url` and the rest of the wiki
bookkeeping survive — so the next push updates the pages that already exist
instead of publishing a second copy of each.
skills/page/references/pages.md
+173
View File
@@ -0,0 +1,173 @@
# The page-tree format
Canonical. Everything about how a discussion's artifacts become named, ordered,
navigable pages lives here. The scripts implement this document; when they
disagree, this document is right.
## The one fact that shapes everything: the wiki is flat
Gitea's wiki has no directories. It has a list of pages, each stored as one
file whose name Gitea escapes from the title:
| title | file Gitea writes | `sub_url` |
|---|---|---|
| `Abstract Issue` | `Abstract-Issue.md` | `Abstract-Issue` |
| `zz-probe/child` | `zz-probe%2Fchild.-.md` | `zz-probe%2Fchild.-` |
| `Simple Chains/Parked/Chain decisions — DC` | `Simple-Chains%2FParked%2FChain-decisions-%E2%80%94-DC.md` | same, minus `.md` |
Three rules are visible in that table, and all three are Gitea's to change:
space becomes `-`; `/` becomes `%2F`; a **literal** `-` in the title forces a
trailing `.-` marker so it stays distinguishable from a space.
Two consequences run through the whole design.
**Hierarchy lives in the title and nowhere else.** `/` inside a title is the
only nesting there is. A real subdirectory committed into the wiki's git
repository — `folder/page.md` — is invisible to the API and to the web UI. It
is a ghost file. Never create one.
**`sub_url` is identity and is never constructed.** It is read back from
whatever the API returned and stored in the manifest. A hand-built one that is
almost right does not fail loudly; it creates a second page and abandons the
first.
## The space
```
tmp/wiki/claude-skills/tea/ a SPACE
.pages.json the manifest
Simple-Chains.md title: Simple Chains (the index)
Simple-Chains/
Ideas.md title: Simple Chains/Ideas
Ideas/
Chain-core.md title: Simple Chains/Ideas/Chain core
```
A space is a directory holding a page tree and one manifest. Its name is
normally the `owner/repo` it syncs with, and to the domain layer that is an
opaque relative path — `--space docs` and `--space a/b/c` are equally valid.
The path is `<repo root>/tmp/wiki`, resolved from `page.py`'s own location and
not from the working directory. `--out` overrides it and is used exactly as
typed. Nothing creates a space as a side effect of a write: the scripts say so
on stderr when they make one.
## The manifest
`.pages.json`, one entry per page, keyed by the file's path inside the space.
```json
{
"space": "claude-skills/tea",
"pages": {
"Simple-Chains/Ideas/Chain-core.md": {
"title": "Simple Chains/Ideas/Chain core",
"order": 2,
"pushed": "9a1ab2e3bfd45f7c7ba323d9d8cd59642d6f0540",
"remote-updated": "2026-08-10T11:15:39Z",
"sha": "fc8ec1779d910850f49bfef60dd5a0e737bbdc8a",
"sub_url": "Simple-Chains%2FIdeas%2FChain-core",
"synced": "2026-08-10T11:15:39Z",
"url": "https://git.noodles.cam/…/wiki/Simple-Chains%2FIdeas%2FChain-core"
}
}
}
```
| key | owner | meaning |
|---|---|---|
| `title` | domain | the page's name; `/` is hierarchy |
| `order` | domain | sort key from a `NN-` file-name prefix; absent when there was none |
| `sub_url` | wiki | Gitea's address for the page — **the identity** |
| `pushed` | wiki | sha1 of the bytes last published; the whole of change detection |
| `sha` | wiki | the wiki commit the local copy came from |
| `synced` | wiki | when this copy was fetched or pushed |
| `url` | wiki | browser link |
| `remote-updated` | wiki | the page's last commit date in the wiki |
The domain layer writes `title` and `order`, carries everything else through
load and save verbatim, and never reads it. A page with no `sub_url` has never
been published — a complete state, not a pending one, exactly as `origin: local`
is for an issue.
## How a source file gets its title
Applied at import, once. Three rules, in order:
1. **`order 0`, or a file literally named `index` / `readme`, is the page for
the directory it sits in.** `ideas/00-intro.md` becomes `…/Ideas`, not a
child of it. Its title comes from the **directory name**, never from its own
heading — a child's title has to extend its parent's exactly, and that file
opens with "Ideas for chain business requirements", which no child would
ever be prefixed by.
2. **Otherwise the file's first markdown heading**, sanitized. It is what a
human wrote for a human: there is no mechanical route from
`03-q-01-do-we-know-the-chain-participant-by-name.md` to
`Q-01. Do We Know the Chain Participant by Name`.
3. **No heading: the file name**, made readable — `NN-` stripped, `-` and `_`
to spaces, first letter raised. Only the first letter: title-casing would
wreck `Q-01`, `sqlc`, and `APNs`.
Sanitizing a title drops markdown markup (`` ` ``, `*`, `_` — a page list does
not render markdown) and turns `/` into `-`, because a slash inside a heading
would silently invent a level of hierarchy the author did not ask for.
### A title is a decision, not a derivation
Once a page is in the manifest its title stays put. Re-importing replaces the
body and leaves the title alone, so editing a heading cannot rename a page —
which matters because renaming a **published** page does not move it, it
creates a second one and orphans the first. `--retitle` opts into that
explicitly.
The reverse direction does not exist. A path is derived from a title; a title
is never derived from a path. `02-chain-core` proves why: those dashes are
real, and undoing "space became dash" would eat them.
## Ordering
A leading `NN-` on a file name is sort order and nothing else — it never
reaches the title. `00` is special and means "this is the directory's own
page". Pages with an order sort before pages without one: an explicit `NN-` is
a decision, its absence is not.
The wiki cannot hold ordering, so `order` is local-only and survives a pull.
## Paths
A path is one component per title segment, spaces to `-`, with characters a
shell has to quote dropped — apostrophes and quotes and commas. `Don't send to
this one` keeps its apostrophe in the title and loses it in
`Dont-send-to-this-one.md`.
Two titles can land on one path. That is reported and never resolved
automatically: picking a winner is how a discussion loses a document. Rename a
source, or rename the page in the wiki, and run it again.
## The index page
The wiki will not draw a tree from titles, so an index page is the navigation,
not a nicety. `page_index.py` writes one as an ordinary page in the space — it
is pushed by the same command as everything else.
Nesting follows the **titles**, not the manifest's path order; those two
disagree, because on disk `Simple-Chains/System.md` sorts before
`Simple-Chains/Ideas/Scale.md` while in the hierarchy System is a child and
Scale a grandchild. A parent with no page of its own still gets a node, so its
children are not hidden.
Links: a published page is linked by its `sub_url`, the only address Gitea
guarantees. A page that has never been pushed gets Gitea's `[[Title|label]]`
wiki-link syntax, which resolves the escaping on the server at render time.
Rebuilding the index after a push upgrades those links to exact ones — so the
order is **push, rebuild the index, push again**.
## What the sync does not do
- **No merge.** A pull overwrites the local body. `synced` tells you how old
your copy is; re-pull when it matters.
- **No drift tracking.** `pushed` answers one question — is the local file
different from what was published — and answers it with a hash.
- **No deletes.** Pushing is additive. A page removed locally stays in the
wiki; removing a published page is an explicit act, done in the web UI or
with a `DELETE` through `/tea:use`.
skills/page/scripts/page.py
+523
View File
@@ -0,0 +1,523 @@
#!/usr/bin/env python3
r"""
page.py — what a PAGE TREE is. The domain layer for wiki artifacts.
Not a command; the module the other page scripts build on. It knows how a
directory of markdown becomes a named, ordered tree of pages, and it knows
NOTHING about any wiki: no Gitea, no `tea`, no logins, no HTTP, no `sub_url`.
The layering rule is mechanically checkable — every import in this directory is
stdlib, and `subprocess` is not among them:
grep -rh '^import \|^from ' skills/page/scripts/ | sort -u
Delete skills/wiki/ entirely and this layer keeps working: a discussion's
artifacts organized into a tree on this machine are a finished thing, not a
draft waiting for an upload.
tmp/wiki/claude-skills/tea/ <- a SPACE
.pages.json <- the manifest
Simple-Chains/
Ideas.md title: Simple Chains/Ideas
Ideas/
Chain-core.md title: Simple Chains/Ideas/Chain core
A space is a directory holding a page tree and one manifest. The space's name
("claude-skills/tea") is an opaque relative path to this module — it happens to
be an owner/repo pair, and this layer never learns that.
Why a manifest at all
---------------------
Because the wiki's own page identity is not derivable from a file path, and
guessing at it is how you get duplicate pages. The manifest is the record of
what each local file IS, written once at import or pull and never re-derived.
Domain keys in a manifest entry are `title` and `order`. Everything else —
`sub_url`, `sha`, `synced`, `pushed` — is written by the wiki layer, carried
through load/save verbatim, and never read here. That passthrough is what lets
one manifest describe both a local-only tree and a published one without the
domain learning a second vocabulary.
Titles
------
The title is the identity that matters, and `/` inside it is the ONLY
hierarchy there is — the wiki this feeds has no directories. A local path is
derived from the title, never the reverse:
title "Simple Chains/Ideas/Chain core"
path "Simple-Chains/Ideas/Chain-core.md"
That direction is deliberate. Deriving a title back from a path would have to
undo `-`-for-space, and `02-chain-core` proves it cannot: the dashes there are
real. So a title is chosen ONCE, at import or at pull, and then it is a fact in
the manifest. Renaming is an explicit act, not a side effect of editing a
heading.
Ordering
--------
A leading `NN-` on a file name is sort order and nothing else — it never
reaches the title. `order 0` is special: it is the directory's own page, so
`ideas/00-intro.md` becomes the page "…/Ideas" rather than a child of it.
"""
import hashlib
import json
import os
import re
# --------------------------------------------------------------------------
# where the cache lives
# --------------------------------------------------------------------------
# `<repo root>/tmp/wiki`, absolute, resolved once at import — the same anchoring
# rule the issue store uses, and for the same reason: a script's own location is
# a fact about the installation, cwd is a fact about the last `cd`. Walking up
# from __file__ hands every script in both layers one answer no matter where it
# is invoked from.
#
# The twenty lines below are duplicated from the issue domain rather than
# imported from it. Two domains that do not know about each other is worth more
# than the duplication is worth saving: skills/page must keep working with
# skills/issue deleted, exactly as skills/issue keeps working with skills/sync
# deleted.
STORE_PARTS = ("tmp", "wiki")
# `.git` is a directory in a normal clone and a FILE in a worktree — hence
# exists(), not isdir(). AGENTS.md is the fallback for a plugin copied out of
# git; the agents-sync hook only ever puts one at a repository root.
REPO_MARKERS = (".git", "AGENTS.md")
_HERE = os.path.dirname(os.path.abspath(__file__))
MANIFEST = ".pages.json"
# Written here; read here. Everything else in an entry belongs to the wiki
# layer and is passed through untouched.
DOMAIN_KEYS = ("title", "order", "source")
def repo_root(start):
"""Nearest ancestor of `start` (inclusive) carrying a repo marker, or None."""
d = os.path.abspath(start)
while True:
if any(os.path.exists(os.path.join(d, m)) for m in REPO_MARKERS):
return d
parent = os.path.dirname(d)
if parent == d:
return None
d = parent
def store_root(start=None):
"""Absolute path of the wiki cache root.
`start` overrides the anchor so the resolution can be exercised against a
scratch tree. Outside a repository, cwd gets a turn, then the historical
cwd-relative location stands — made absolute so an error can name the
directory it really looked in."""
for anchor in ([start] if start is not None else [_HERE, os.getcwd()]):
root = repo_root(anchor)
if root:
return os.path.join(root, *STORE_PARTS)
return os.path.abspath(os.path.join(*STORE_PARTS))
WIKI_ROOT = store_root()
def space_root(space, root=None):
"""Directory of one space. `space` is an opaque relative path — it may
contain `/` (it usually does) and is used as typed."""
return os.path.join(root or WIKI_ROOT, *space.split("/"))
# --------------------------------------------------------------------------
# names, titles, order
# --------------------------------------------------------------------------
# Characters a title may not carry into a path. `/` is absent on purpose: it is
# the hierarchy separator and is split on before this ever applies.
_UNSAFE = re.compile(r'[\\:*?"<>|\x00-\x1f]+')
# Inline code in a heading is markup, not a name: `Inventory — \`P-NN\`` is a
# page called "Inventory — P-NN", and a page list does not render markdown.
_MARKUP = re.compile(r"[`*_]+")
# Dropped from a PATH but kept in a title. An apostrophe in "Don't send to this
# one" belongs in the name and does not belong in something a shell has to
# quote.
_PATH_NOISE = re.compile(r"['‘’\"“”,]+")
_DASHES = re.compile(r"-{2,}")
_ORDER = re.compile(r"^(\d+)[-_. ]+(.*)$")
_HEADING = re.compile(r"^\s{0,3}#{1,6}\s+(.+?)\s*#*\s*$")
def order_of(name):
"""The `NN-` sort key on a file or directory name, or None.
`00-intro.md` -> 0, `02-chain-core.md` -> 2, `handoff.md` -> None. Zero is
a real answer and not None; callers distinguish them."""
m = _ORDER.match(strip_ext(name))
return int(m.group(1)) if m else None
def strip_ext(name):
stem, ext = os.path.splitext(name)
return stem if ext.lower() in (".md", ".markdown") else name
def strip_order(name):
"""`02-chain-core` -> `chain-core`; a name that is only digits is left
alone, because stripping it would leave nothing to call the page."""
m = _ORDER.match(strip_ext(name))
return m.group(2) if m and m.group(2) else strip_ext(name)
def title_from_name(name):
"""Fallback title: the file or directory name made readable.
`02-chain-core.md` -> `Chain core`. Only the first letter is raised —
title-casing would wreck `Q-01`, `sqlc`, `APNs`, and every other name that
already knows how it is spelled."""
t = strip_order(name).replace("_", " ").replace("-", " ").strip()
t = re.sub(r"\s+", " ", t)
return t[:1].upper() + t[1:] if t else t
def title_from_body(text):
"""The document's first markdown heading, or None.
Preferred over the file name because it is what a human wrote for a human:
`03-q-01-do-we-know-the-chain-participant-by-name.md` opens with
`## Q-01. Do We Know the Chain Participant by Name`, and there is no
mechanical route from the first string to the second. Only the first
heading is consulted, and only before any prose — a heading further down is
a section, not a name."""
for line in text.splitlines():
if not line.strip():
continue
m = _HEADING.match(line)
return m.group(1).strip() if m else None
return None
def sanitize_title(title):
"""Make a string safe to be one title SEGMENT.
`/` becomes `-`: a slash inside a heading would silently invent a level of
hierarchy that the author did not ask for, and inventing structure is worse
than losing a slash."""
t = _MARKUP.sub("", _UNSAFE.sub("", title.replace("/", "-")))
return re.sub(r"\s+", " ", t).strip(" .-") or "untitled"
def join_title(*parts):
"""Join title segments with the hierarchy separator, dropping empties."""
return "/".join(p for p in parts if p)
def path_segment(segment):
"""One title segment as one path component."""
s = _PATH_NOISE.sub("", _MARKUP.sub("", _UNSAFE.sub("", segment)))
s = re.sub(r"\s+", "-", s.replace("/", "-").strip())
return _DASHES.sub("-", s).strip("-.") or "untitled"
def path_for_title(title):
"""Relative path, inside a space, for a title. Always ends in `.md`."""
parts = [path_segment(p) for p in title.split("/") if p.strip()]
if not parts:
parts = ["untitled"]
return os.path.join(*parts) + ".md"
# --------------------------------------------------------------------------
# the manifest
# --------------------------------------------------------------------------
def blank_manifest(space):
return {"space": space, "pages": {}}
def manifest_path(space, root=None):
return os.path.join(space_root(space, root), MANIFEST)
def load_manifest(space, root=None):
"""The space's manifest, or a blank one.
A missing manifest and an empty one are the same thing to every caller here
— but they are NOT the same thing to a caller deciding whether to print
"no such space". That distinction is `os.path.isdir(space_root(...))`, and
the commands make it themselves rather than reading it out of a dict."""
p = manifest_path(space, root)
if not os.path.isfile(p):
return blank_manifest(space)
with open(p, encoding="utf-8") as f:
m = json.load(f)
m.setdefault("space", space)
m.setdefault("pages", {})
return m
def save_manifest(manifest, root=None):
"""Write the manifest, keys sorted, one page per line-block.
Sorted and indented because this file lands in a diff every time anything
syncs, and a diff nobody can read is a diff nobody checks."""
p = manifest_path(manifest["space"], root)
os.makedirs(os.path.dirname(p), exist_ok=True)
ordered = {"space": manifest["space"], "pages": {}}
for path, e in sorted(manifest.get("pages", {}).items()):
ordered["pages"][path] = {k: e[k] for k in DOMAIN_KEYS if k in e}
ordered["pages"][path].update(
{k: v for k, v in sorted(e.items()) if k not in DOMAIN_KEYS})
with open(p, "w", encoding="utf-8") as f:
json.dump(ordered, f, ensure_ascii=False, indent=2, sort_keys=False)
f.write("\n")
return p
def entry(title, order=None, source=None, **extra):
"""A manifest entry. Domain keys first, passthrough after — the same
render order the issue layer uses, for the same reason: it makes a diff of
the file readable."""
e = {"title": title}
if order is not None:
e["order"] = order
if source is not None:
e["source"] = source
e.update({k: v for k, v in extra.items() if v is not None})
return e
def find_by_source(manifest, source, prefix=""):
"""(relpath, entry) for the page imported from this source file, or
(None, None).
The path is derived from the title, so a retitle moves it — and looking a
page up by its new path would find nothing, treat it as new, and publish a
duplicate beside the page it was meant to rename. Source is the one link
that survives a rename, which is why it is recorded at all.
Scoped by title prefix, so importing the same directory twice under two
prefixes gives two independent trees rather than one fighting over itself.
"""
for path, e in manifest.get("pages", {}).items():
if e.get("source") != source:
continue
if prefix and not (e.get("title", "") == prefix
or e.get("title", "").startswith(prefix + "/")):
continue
return path, e
return None, None
def sort_key(relpath, e):
"""Order a tree for display and for an index.
Directory by directory, `order` first and unnumbered pages after — an
explicit `NN-` is a decision, its absence is not. Ties break on title so
the output is stable."""
d = os.path.dirname(relpath)
o = e.get("order")
return (d, 0 if o is not None else 1, o if o is not None else 0,
e.get("title", relpath))
def sorted_pages(manifest):
"""[(relpath, entry)] in tree order."""
return sorted(manifest.get("pages", {}).items(),
key=lambda kv: sort_key(kv[0], kv[1]))
def by_title(manifest):
return {e["title"]: (p, e) for p, e in manifest.get("pages", {}).items()
if e.get("title")}
def children_of(manifest, prefix):
"""Every page at or under a title prefix.
The wiki this feeds is flat, so "children" is a prefix test on the title
and nothing more — there is no tree to walk, only a naming convention to
trust."""
out = []
for p, e in sorted_pages(manifest):
t = e.get("title", "")
if t == prefix or t.startswith(prefix + "/"):
out.append((p, e))
return out
def body_hash(text):
"""sha1 of the exact bytes a page would be published as.
This is the whole of change detection: a page is worth pushing when what is
on disk hashes differently from what was pushed last. No timestamps, no
drift model — the same stance the issue store takes."""
if isinstance(text, str):
text = text.encode("utf-8")
return hashlib.sha1(text).hexdigest()
# --------------------------------------------------------------------------
# importing a directory of markdown
# --------------------------------------------------------------------------
SKIP_DIRS = {".git", ".svn", "__pycache__", "node_modules"}
MD_EXT = (".md", ".markdown")
def walk_markdown(src):
"""Every markdown file under `src`, as paths relative to it, depth first
and sorted so an import is reproducible."""
out = []
for dirpath, dirnames, filenames in os.walk(src):
dirnames[:] = sorted(d for d in dirnames
if d not in SKIP_DIRS and not d.startswith("."))
rel = os.path.relpath(dirpath, src)
rel = "" if rel == "." else rel
for f in sorted(filenames):
if f.lower().endswith(MD_EXT) and not f.startswith("."):
out.append(os.path.join(rel, f) if rel else f)
return out
def title_for_source(relpath, text, prefix=""):
"""The title a source file gets on import.
Three rules, in this order, and the reference doc spells out why:
1. `order 0` (`00-intro.md`, or a literal `index`/`readme`) is the page for
the directory it sits in. Its title comes from the DIRECTORY name, not
from its own heading — a child's title must extend its parent's exactly,
and `ideas/00-intro.md` opens with "Ideas for chain business
requirements", which no child would ever be prefixed by.
2. Any other file takes its first heading, sanitized.
3. No heading: the file name, made readable.
"""
parts = relpath.replace(os.sep, "/").split("/")
name = parts[-1]
dirs = [sanitize_title(title_from_name(d)) for d in parts[:-1]]
stem = strip_ext(name).lower()
if order_of(name) == 0 or stem in ("index", "readme"):
# The directory's own page. At the root of the import that is the
# prefix itself.
return join_title(prefix, *dirs)
own = title_from_body(text)
own = sanitize_title(own) if own else sanitize_title(title_from_name(name))
return join_title(prefix, *dirs, own)
def plan_import(src, prefix="", read=None):
"""Work out what an import would produce, without writing anything.
Returns (pages, collisions):
pages [{"source", "path", "title", "order", "text"}] in tree order
collisions [(path, [title, title, ...])] — two sources landing on one
file. Reported, never resolved: the wiki would end up with
two pages fighting over one local copy, and picking a winner
for the operator is how a discussion loses a document."""
def default_read(p):
with open(p, encoding="utf-8") as f:
return f.read()
read = read or default_read
pages, seen = [], {}
for rel in walk_markdown(src):
source = os.path.join(src, rel)
text = read(source)
title = title_for_source(rel, text, prefix)
path = path_for_title(title)
seen.setdefault(path, []).append(title)
# `source` is kept relative to the import root, not absolute: it is the
# only durable link between a file on the far side and the page it
# became, and it has to survive the artifacts directory being moved.
pages.append({"source": source, "rel": rel.replace(os.sep, "/"),
"path": path, "title": title,
"order": order_of(os.path.basename(rel)), "text": text})
pages.sort(key=lambda p: sort_key(p["path"], p))
collisions = [(p, t) for p, t in sorted(seen.items()) if len(t) > 1]
return pages, collisions
# --------------------------------------------------------------------------
# rendering
# --------------------------------------------------------------------------
def title_tree(manifest, prefix=""):
"""Group pages into a parent -> children map keyed by title.
Built from the titles, not from the manifest's path order. Those two
disagree: on disk `Simple-Chains/System.md` sorts before
`Simple-Chains/Ideas/Scale.md`, while in the hierarchy Scale is a
grandchild of Simple Chains and System is a child. Nesting has to follow
the titles, because the titles are the only hierarchy there is.
A parent with no page of its own still gets a node: `Simple Chains/Parked`
can have children while nothing is published at that title, and dropping
its children because it is missing would hide them entirely."""
kids, entries = {}, {}
for _, e in manifest.get("pages", {}).items():
title = e.get("title")
if not title:
continue
if prefix and not (title == prefix or title.startswith(prefix + "/")):
continue
entries[title] = e
parts = title.split("/")
# Every ancestor gets a node, so a gap in the chain does not orphan a
# subtree.
for i in range(len(parts), 0, -1):
kids.setdefault("/".join(parts[:i - 1]), set()).add("/".join(parts[:i]))
return kids, entries
def render_index(manifest, prefix="", heading=None):
"""A table-of-contents page for a space or a subtree.
Nested markdown list, indented by title depth. The wiki is flat and will
not draw this for you, so the index IS the navigation.
Links: a published page is linked by its `sub_url`, which is the only
address Gitea guarantees. A page that has never been pushed has no sub_url
yet, so it gets Gitea's own `[[Title]]` wiki-link syntax — which resolves
the escaping itself, at render time, on the server. Rebuilding the index
after a push upgrades those links to exact ones."""
kids, entries = title_tree(manifest, prefix)
lines = ["# %s" % (heading or prefix or "Contents"), ""]
def order_key(title):
e = entries.get(title) or {}
o = e.get("order")
return (0 if o is not None else 1, o if o is not None else 0, title)
def walk(node, depth):
for child in sorted(kids.get(node, ()), key=order_key):
e = entries.get(child) or {}
label = child.split("/")[-1]
sub = e.get("sub_url")
link = "[%s](%s)" % (label, sub) if sub else "[[%s|%s]]" % (child, label)
lines.append("%s- %s" % (" " * depth, link))
walk(child, depth + 1)
walk(prefix, 0)
lines.append("")
return "\n".join(lines)
def tree_lines(manifest, mark=None):
"""The space as an ascii tree, for a terminal.
`mark(relpath, entry)` returns a short state tag shown after the title —
the wiki layer passes sync state through it, and this module stays unaware
of what the tags mean."""
out, last_dir = [], None
for path, e in sorted_pages(manifest):
d = os.path.dirname(path)
if d != last_dir:
out.append("%s/" % d if d else ".")
last_dir = d
tag = mark(path, e) if mark else ""
out.append(" %-40s %s%s" % (os.path.basename(path),
e.get("title", ""),
(" " + tag) if tag else ""))
return out
skills/page/scripts/page_import.py
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""
page_import.py — pull a directory of markdown into a space. Offline.
This is the "wiki organization" step, and it is the only step where a page gets
its name. A discussion produces artifacts wherever the discussion happened:
~/…/mpns/feat/simple-chains/tmp/simple-chains/
handoff.md scope.md
ideas/00-intro.md ideas/02-chain-core.md
questions/03-q-01-do-we-know-the-chain-participant-by-name.md
Import copies that tree into a space under `tmp/wiki/`, gives every file a
title, and records both in the manifest. Nothing here talks to a wiki; the
result is a complete, readable, greppable tree whether or not it is ever
published.
page_import.py --from DIR --space claude-skills/tea --prefix "Simple Chains"
Simple-Chains/Handoff.md Simple Chains/Handoff
Simple-Chains/Ideas.md Simple Chains/Ideas
Simple-Chains/Ideas/Chain-core.md Simple Chains/Ideas/Chain core
Re-importing is safe and is the normal way to refresh: a page already in the
manifest keeps its title (a title is a decision, not a derivation) and only its
body is replaced. `--retitle` opts into re-deriving titles, which is a rename
and, for pages already published, will orphan the old ones — so it is never the
default.
Usage:
page_import.py --from DIR [--space SPACE] [--prefix TITLE]
[--retitle] [--dry-run] [--out DIR]
"""
import argparse
import os
import shutil
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import page # noqa: E402
def die(msg, code=1):
sys.stderr.write("%s: %s\n" % (os.path.basename(sys.argv[0]), msg))
sys.exit(code)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--from", dest="src", required=True,
help="directory of markdown to import")
ap.add_argument("--space", default="local",
help="space to import into (default: local)")
ap.add_argument("--prefix", default="",
help="title every imported page hangs under")
ap.add_argument("--retitle", action="store_true",
help="re-derive titles of pages already in the manifest "
"(a rename; orphans published pages)")
ap.add_argument("--dry-run", action="store_true")
ap.add_argument("--out", help="wiki cache root (default: <repo>/tmp/wiki)")
a = ap.parse_args()
src = os.path.abspath(a.src)
if not os.path.isdir(src):
die("not a directory: %s" % a.src)
root = a.out or page.WIKI_ROOT
prefix = page.sanitize_title(a.prefix) if a.prefix else ""
pages, collisions = page.plan_import(src, prefix)
if not pages:
die("no markdown found under %s" % src)
if collisions:
for path, titles in collisions:
sys.stderr.write("collision: %s <- %s\n" % (path, " | ".join(titles)))
die("%d path collision(s); rename the sources and retry" % len(collisions))
manifest = page.load_manifest(a.space, root)
known = manifest["pages"]
dest_root = page.space_root(a.space, root)
# Asked before anything is written: nothing should create a space as a
# silent side effect of a write, and saying so on stderr is how the
# operator learns a typo in --space made a second one.
created = not os.path.isdir(dest_root)
new = changed = same = moved = 0
for p in pages:
# Looked up by SOURCE, not by path: a retitle moves the path, and a
# lookup that missed would treat the page as new and publish a
# duplicate beside the one it was meant to rename.
prior_path, prior = page.find_by_source(manifest, p["rel"], prefix)
if prior is None:
prior_path, prior = p["path"], known.get(p["path"])
# A title already in the manifest is a decision that was made once.
# Re-deriving it on every import would let an edited heading silently
# rename a published page — which does not rename it, it creates a
# second one and abandons the first.
title = p["title"] if (a.retitle or not prior) else prior["title"]
relpath = page.path_for_title(title)
dest = os.path.join(dest_root, relpath)
state = "new"
if prior and relpath != prior_path:
state = "moved"
elif prior and os.path.isfile(dest):
with open(dest, encoding="utf-8") as f:
state = "same" if f.read() == p["text"] else "changed"
elif prior:
state = "changed"
new += state == "new"
changed += state == "changed"
same += state == "same"
moved += state == "moved"
print("%-7s %-44s %s" % (state, relpath, title))
if a.dry_run:
continue
os.makedirs(os.path.dirname(dest), exist_ok=True)
shutil.copyfile(p["source"], dest)
# Passthrough keys survive: a re-import must not cost a page its
# sub_url, or the next push would publish a duplicate.
e = dict(prior or {})
e.update(page.entry(title, p["order"], p["rel"]))
if state == "moved":
# The old copy goes, the entry moves with its bookkeeping intact.
# The page in the wiki is still at its old sub_url; the next push
# sends the new title, which is what renames it there.
old = os.path.join(dest_root, prior_path)
if os.path.isfile(old):
os.remove(old)
known.pop(prior_path, None)
# A rename can leave the body byte-identical, and push decides by
# body hash alone. Clearing it is what makes the next push send the
# new title instead of skipping the page as unchanged.
e.pop("pushed", None)
known[relpath] = e
if a.dry_run:
print("\ndry run — nothing written")
return 0
path = page.save_manifest(manifest, root)
if created:
sys.stderr.write("created space %s\n" % dest_root)
print("\n%d new, %d changed, %d unchanged%s -> %s"
% (new, changed, same,
", %d renamed" % moved if moved else "", os.path.dirname(path)))
if moved:
sys.stderr.write(
"%d page(s) renamed. A published page is renamed in the wiki by "
"the next push, not by this import.\n" % moved)
return 0
if __name__ == "__main__":
sys.exit(main())
skills/page/scripts/page_index.py
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""
page_index.py — write a table-of-contents page into a space. Offline.
The wiki this feeds is flat: a title like `Simple Chains/Ideas/Chain core` has
hierarchy in its name and nowhere else, and Gitea will not draw you a tree from
it. An index page is therefore not a nicety, it is the navigation.
Written as an ordinary page in the space, so it is pushed by the same command
as everything else and needs no special case anywhere downstream. Links are
written by TITLE rather than by URL — the wiki resolves those itself, and a
link written that way survives every filename-escaping rule this layer
deliberately refuses to model.
page_index.py --space claude-skills/tea --prefix "Simple Chains"
-> Simple-Chains.md, title `Simple Chains`
page_index.py --space claude-skills/tea --title Home
-> Home.md, title `Home`, listing the whole space
Usage:
page_index.py [--space SPACE] [--prefix TITLE] [--title TITLE]
[--dry-run] [--out DIR]
"""
import argparse
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import page # noqa: E402
def die(msg, code=1):
sys.stderr.write("%s: %s\n" % (os.path.basename(sys.argv[0]), msg))
sys.exit(code)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--space", default="local")
ap.add_argument("--prefix", default="",
help="index only this subtree; also the index's own title")
ap.add_argument("--title", help="title for the index page "
"(default: --prefix, else Home)")
ap.add_argument("--dry-run", action="store_true")
ap.add_argument("--out", help="wiki cache root (default: <repo>/tmp/wiki)")
a = ap.parse_args()
root = a.out or page.WIKI_ROOT
space_dir = page.space_root(a.space, root)
if not os.path.isdir(space_dir):
die("no such space: %s (looked in %s)" % (a.space, space_dir))
manifest = page.load_manifest(a.space, root)
prefix = page.sanitize_title(a.prefix) if a.prefix else ""
title = a.title or prefix or "Home"
body = page.render_index(manifest, prefix, heading=title)
relpath = page.path_for_title(title)
if a.dry_run:
sys.stdout.write(body)
print("-> %s (%s)" % (relpath, title))
return 0
dest = os.path.join(space_dir, relpath)
os.makedirs(os.path.dirname(dest), exist_ok=True)
with open(dest, "w", encoding="utf-8") as f:
f.write(body)
# Carries the entry's passthrough keys forward: rebuilding an index must
# update the page that is already published, never publish a second one.
prior = manifest["pages"].get(relpath, {})
e = dict(prior)
e.update(page.entry(title, prior.get("order")))
manifest["pages"][relpath] = e
page.save_manifest(manifest, root)
n = len(page.children_of(manifest, prefix) if prefix
else page.sorted_pages(manifest))
print("%s -> %s (%d entr%s)" % (title, relpath, n - 1,
"y" if n - 1 == 1 else "ies"))
return 0
if __name__ == "__main__":
sys.exit(main())
skills/page/scripts/page_ls.py
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""
page_ls.py — show what a space holds. Offline.
The tree, the titles, and one state tag per page. The tag is the only place
this layer acknowledges that a wiki exists, and it reads it the way the issue
index reads `origin:` — as an opaque fact recorded by somebody else:
local never published; a complete state, not a pending one
synced published, and the file matches what was pushed
ahead published, and the local file has changed since
? published, but nothing recorded what was pushed
Usage:
page_ls.py [--space SPACE] [--prefix TITLE] [--titles] [--out DIR]
"""
import argparse
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import page # noqa: E402
def die(msg, code=1):
sys.stderr.write("%s: %s\n" % (os.path.basename(sys.argv[0]), msg))
sys.exit(code)
def state_of(space_dir, relpath, e):
if not e.get("sub_url"):
return "local"
pushed = e.get("pushed")
if not pushed:
return "?"
full = os.path.join(space_dir, relpath)
if not os.path.isfile(full):
return "missing"
with open(full, encoding="utf-8") as f:
return "synced" if page.body_hash(f.read()) == pushed else "ahead"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--space", default="local")
ap.add_argument("--prefix", default="", help="only titles at or under this")
ap.add_argument("--titles", action="store_true",
help="print one title per line and nothing else")
ap.add_argument("--out", help="wiki cache root (default: <repo>/tmp/wiki)")
a = ap.parse_args()
root = a.out or page.WIKI_ROOT
space_dir = page.space_root(a.space, root)
# "Does not exist" and "is empty" are different answers and get different
# messages — an empty space is a space somebody made on purpose.
if not os.path.isdir(space_dir):
die("no such space: %s (looked in %s)" % (a.space, space_dir))
manifest = page.load_manifest(a.space, root)
pages = (page.children_of(manifest, a.prefix) if a.prefix
else page.sorted_pages(manifest))
if not pages:
print("space %s is empty" % a.space if not a.prefix
else "nothing at or under %r" % a.prefix)
return 0
if a.titles:
for _, e in pages:
print(e.get("title", ""))
return 0
sub = {p: e for p, e in pages}
view = dict(manifest, pages=sub)
for line in page.tree_lines(view, mark=lambda p, e: state_of(space_dir, p, e)):
print(line)
counts = {}
for p, e in pages:
s = state_of(space_dir, p, e)
counts[s] = counts.get(s, 0) + 1
print("\n%d page(s): %s" % (len(pages),
", ".join("%d %s" % (v, k)
for k, v in sorted(counts.items()))))
return 0
if __name__ == "__main__":
sys.exit(main())