feat: evict closed issues from the local store
The store is a working set, not an archive. Until now nothing removed a closed issue from it: #10 put a filter on the write and said so explicitly ("existing store files are not cleaned"), and the migration was never anybody's job. The only way out was rm past every script, followed by rebuilding INDEX.md by hand. issue_evict.py removes <id>.md and every sidecar under that slug for an issue that is state: closed AND carries an origin: naming a tracker, then rebuilds INDEX.md. --dry-run prints and writes nothing at all. Two conditions, and the second one is the whole safety argument. An origin: local issue IS the work — there is no other copy — so it is never evicted, in any state, not even when named on the command line: it is reported and kept. The only files that go are ones whose own metadata says pull.py <n> brings them back, which is the trade push.py already makes when it drops a file the tracker just confirmed. The command lives in the domain layer, and the layering rule decides that rather than convenience: state: and origin: are domain fields and the answer is already on disk, so eviction needs no network, no login and no tea. The domain also gains issue.slug_files — every file the store holds under one slug, which is all_ids' "a slug has no dot in it" read the other way round, and lets the domain remove an issue completely without learning what a comment thread is. skills/sync/scripts/evict.py is the bridge form, and it exists because a local state: is only as fresh as the last pull: an issue closed in the web UI still reads open here. It refreshes state: from Gitea, then calls issue_evict.run — one implementation of "what may be evicted", in the layer that owns the fields it reads. Same gate as push, one step earlier: every candidate's state is fetched before anything is removed, each answer must be an object carrying the number asked about and a state the domain recognizes (confirmed_state, the counterpart of confirmed_number), and a failed or unconfirmed call evicts nothing — not even the candidates whose answers had already arrived, and no refreshed state: is written back either. A candidate is an issue with a gitea: handle; origin: local has none, is never asked about, and is never removed. .remote.json is deliberately not pruned. It is the number -> slug ledger, its entries are supposed to outlive the files they name, and an evicted issue is in exactly the state a pushed one is. AGENTS.md gains the rule the tracker side never wrote down: pull by number fetches an issue in any state — an address is not a query. Eviction does not revoke it, so a closed issue pulled after a cleanup is on disk again, and that is the tracker answering what it was asked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
naudachu
@@ -38,6 +38,7 @@ All offline, all in `<skill-base-dir>/scripts/`.
|
||||
| `issue_check.py [id…]` | validate against the canonical format; exit 1 on errors |
|
||||
| `issue_ac.py <id> [--check N\|TEXT]` | list the body's checkboxes; tick or untick one |
|
||||
| `issue_tree.py [id…]` | draw the dependency graph from `depends:` |
|
||||
| `issue_evict.py [id…] [--dry-run]` | remove closed issues from the store; **never** an `origin: local` one |
|
||||
| `issue_index.py` | rebuild `tmp/issues/INDEX.md` |
|
||||
| `issue.py` | the domain module the others import — not a command |
|
||||
|
||||
@@ -205,6 +206,50 @@ on `tmp/issues/<id>.md`, and this layer does not know the difference. Getting
|
||||
the rewritten body into the tracker is a separate decision — `push.py --update`
|
||||
in `/tea:sync` — and is no part of this.
|
||||
|
||||
## Evicting closed issues
|
||||
|
||||
The store is a working set, not an archive. A closed issue is not a unit of
|
||||
work any more, and one command takes it out — no `rm`, no rebuilding `INDEX.md`
|
||||
by hand:
|
||||
|
||||
```bash
|
||||
python3 <skill-base-dir>/scripts/issue_evict.py --dry-run # what would go
|
||||
python3 <skill-base-dir>/scripts/issue_evict.py # every closed one
|
||||
python3 <skill-base-dir>/scripts/issue_evict.py old-thing # just this one
|
||||
```
|
||||
|
||||
Two conditions, both read off the file, and the second one is the whole safety
|
||||
argument:
|
||||
|
||||
| `state:` | `origin:` | what eviction does |
|
||||
|---|---|---|
|
||||
| `closed` | a tracker | removes `<id>.md` and every sidecar under that slug |
|
||||
| `closed` | `local` | **keeps it, always**, and says why |
|
||||
| `open` | anything | keeps it |
|
||||
|
||||
**`origin: local` is never evicted, in any state, not even when you name it on
|
||||
the command line.** That file *is* the issue; there is no copy to fetch back.
|
||||
Only a file whose own metadata says the work lives somewhere else may go — the
|
||||
same trade `push.py` makes when it drops a file the tracker just confirmed.
|
||||
|
||||
- `--dry-run` prints what would go and writes nothing at all, `INDEX.md`
|
||||
included.
|
||||
- `INDEX.md` is rebuilt afterwards, so the table and the directory agree. It is
|
||||
rebuilt only when something was actually removed.
|
||||
- `.remote.json` is **not** pruned, deliberately: it is the number → slug
|
||||
ledger, and its entries are supposed to outlive the files they name (that is
|
||||
what makes `pull.py <n>` land on the same slug after a push). An evicted issue
|
||||
is in exactly the state a pushed one is.
|
||||
- **This is not a one-off migration.** `pull.py <n>` fetches an issue in any
|
||||
state — a number is an address, not a query — so a closed issue pulled after
|
||||
an eviction lands on disk again. Not a regression: evict it again when you are
|
||||
done reading it.
|
||||
|
||||
This command is offline and decides from `state:` in the file, which is only as
|
||||
fresh as the last pull. To have the tracker's answer instead — an issue closed
|
||||
in the web UI five minutes ago — use `/tea:sync`'s `evict.py`, which refreshes
|
||||
`state:` first and then calls exactly this decision.
|
||||
|
||||
## Dependency graph
|
||||
|
||||
`depends:` is the authoritative edge list; the body's `## Depends on` section
|
||||
|
||||
@@ -82,19 +82,24 @@ represent a local issue and a synced one without a second format.
|
||||
leaves this machine is valid and finished work; pushing it is optional and
|
||||
nothing here treats it as a draft.
|
||||
|
||||
It is not a *permanent* state, and this is the one place where the file's fate
|
||||
depends on it:
|
||||
It is not a *permanent* state, and it is what the file's fate depends on:
|
||||
|
||||
| `origin:` | what the file is | what a push does to it |
|
||||
|---|---|---|
|
||||
| `local` | the issue itself — the only copy there is | creates it in the tracker, then deletes the file |
|
||||
| a tracker | a working copy of something the tracker already has | updates the tracker, then deletes the file |
|
||||
| `origin:` | what the file is | what a push does to it | what eviction does to it |
|
||||
|---|---|---|---|
|
||||
| `local` | the issue itself — the only copy there is | creates it in the tracker, then deletes the file | **nothing, ever** — in any state, named or not |
|
||||
| a tracker | a working copy of something the tracker already has | updates the tracker, then deletes the file | removes it once `state: closed` |
|
||||
|
||||
**A successful push deletes `tmp/issues/<id>.md`** (and `<id>.comments.md`), on
|
||||
create and on `--update` alike. What is in the store is what has not left this
|
||||
machine; everything else is fetched again when it is needed. The rule, its
|
||||
safety conditions, and how the slug survives are `/tea:sync`'s to state.
|
||||
|
||||
**A closed issue is evicted from the store** by `issue_evict.py` — same trade,
|
||||
one condition more: the work is done *and* it exists somewhere else. An
|
||||
`origin: local` issue is never evicted, because there is nowhere to fetch it
|
||||
back from. The store is a working set, not an archive; `pull.py <n>` fetches a
|
||||
closed issue again whenever it is wanted.
|
||||
|
||||
The `id` never changes across that round trip, which is why `depends:` in other
|
||||
issues keeps working. That is the format's promise; the mechanism is not.
|
||||
|
||||
|
||||
@@ -635,6 +635,35 @@ def all_ids(root):
|
||||
and "." not in f[:-3])
|
||||
|
||||
|
||||
def slug_files(root, id):
|
||||
"""Every file the store holds under one slug — the issue and its sidecars.
|
||||
|
||||
`<id>.md` is the issue. Anything named `<id>.<something>` beside it is a
|
||||
companion another layer parked there (`<id>.comments.md` is the one that
|
||||
exists today). `all_ids` already refuses to read those as issues because a
|
||||
slug has no dot in it; this is the same rule read the other way round.
|
||||
|
||||
Which is how the domain can remove an issue *completely* without learning
|
||||
what any of those companions are: it does not need to know that a comment
|
||||
thread exists to know that a file named after this issue belongs to it and
|
||||
goes when it goes. The issue's own file comes first — it is the headline of
|
||||
any receipt printed from this list.
|
||||
|
||||
A missing store is an empty list, not an error: nothing is there to remove.
|
||||
"""
|
||||
if not os.path.isdir(root):
|
||||
return []
|
||||
own, sidecars = [], []
|
||||
for name in sorted(os.listdir(root)):
|
||||
if not name.startswith("%s." % id):
|
||||
continue
|
||||
p = os.path.join(root, name)
|
||||
if not os.path.isfile(p):
|
||||
continue
|
||||
(own if name == "%s.md" % id else sidecars).append(p)
|
||||
return own + sidecars
|
||||
|
||||
|
||||
def load(root, id):
|
||||
with open(path_of(root, id)) as f:
|
||||
return Issue.from_text(f.read(), id=id)
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
issue_evict.py — closed issues leave the store. Offline.
|
||||
|
||||
issue_evict.py every closed issue that is not origin: local
|
||||
issue_evict.py old-thing … only these
|
||||
issue_evict.py --dry-run print what would go; touch nothing
|
||||
|
||||
The store is a working set, not an archive. A closed issue is not a unit of
|
||||
work any more, and `pull.py` has kept new ones out of filter mode for a while —
|
||||
but the files already on disk were nobody's job, so the only way to remove one
|
||||
was `rm` past every script, followed by rebuilding `INDEX.md` by hand. This is
|
||||
that job.
|
||||
|
||||
WHAT IS EVICTED, and it is two conditions, both read off the file:
|
||||
|
||||
state: closed the work is done
|
||||
origin: <tracker> the work is somewhere else too
|
||||
|
||||
TWO CONDITIONS, AND THE SECOND ONE IS THE WHOLE SAFETY ARGUMENT. `origin:
|
||||
local` means this file IS the issue — there is no other copy and deleting it
|
||||
deletes the work. It is therefore never evicted, in any state, not even when
|
||||
named explicitly on the command line: a closed local issue is reported and
|
||||
kept. The only files that go are ones whose own metadata says the work can be
|
||||
fetched back (`pull.py <n>`), which is the same trade `push.py` makes when it
|
||||
drops a file the tracker has just confirmed.
|
||||
|
||||
That parallel is exact except for where the confirmation comes from. Push has
|
||||
to ask Gitea, because it is Gitea that just changed. Eviction asks the file,
|
||||
because `state:` and `origin:` are domain fields and the answer is already in
|
||||
the store — which is why this command lives in the domain layer and needs no
|
||||
network, no login, and no `tea`. See `skills/sync/scripts/evict.py` for the
|
||||
variant that refreshes `state:` from the tracker first; it makes the deletion
|
||||
decision by calling `run()` below, so there is exactly one implementation of
|
||||
"what may be evicted" and it is this one.
|
||||
|
||||
NOT A ONE-OFF MIGRATION. `pull.py <n>` fetches an issue in any state — a number
|
||||
is an address, not a query — so a closed issue pulled after an eviction lands on
|
||||
disk again. That is the tracker being asked a direct question, not a regression,
|
||||
and the answer is to evict again when you are done with it.
|
||||
|
||||
`.remote.json` is deliberately NOT pruned. It is the local number -> slug
|
||||
ledger, its entries outlive the files they name (that is what makes `pull.py
|
||||
<n>` land on the same slug after a push deleted the file), and an evicted issue
|
||||
is in exactly that state. `INDEX.md` is rebuilt, because it *is* a view of the
|
||||
directory.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import issue # noqa: E402
|
||||
import issue_index # noqa: E402
|
||||
|
||||
CLOSED = "closed"
|
||||
|
||||
# Why an issue was kept, in the receipt. `LOCAL_REASON` is the one that matters:
|
||||
# it is printed whether or not the issue was named, because "this closed thing
|
||||
# is still here" needs an answer every time.
|
||||
LOCAL_REASON = "origin: %s — this file IS the issue" % issue.LOCAL
|
||||
|
||||
|
||||
def classify(issues, ids=None):
|
||||
"""Split the store into (evict, protected, still_open).
|
||||
|
||||
Pure — it reads the loaded issues and decides; nothing here touches disk.
|
||||
|
||||
evict closed, and lives in a tracker too: safe to remove
|
||||
protected closed, but `origin: local`: the only copy of the work
|
||||
still_open not closed
|
||||
|
||||
`ids` restricts the question to those issues; without it the whole store is
|
||||
considered. A protected issue is returned as such even when it was named
|
||||
explicitly — naming a file does not make deleting it safe.
|
||||
"""
|
||||
chosen = list(ids) if ids else sorted(issues)
|
||||
evict, protected, still_open = [], [], []
|
||||
for id in chosen:
|
||||
iss = issues[id]
|
||||
if iss.state != CLOSED:
|
||||
still_open.append(id)
|
||||
elif iss.is_local:
|
||||
protected.append(id)
|
||||
else:
|
||||
evict.append(id)
|
||||
return evict, protected, still_open
|
||||
|
||||
|
||||
def remove(root, id):
|
||||
"""Delete everything the store holds under one slug; return the paths.
|
||||
|
||||
Deliberately dumb, and for the same reason `push.drop_local` is: it takes an
|
||||
id, not a decision. Whether an issue may go is settled by `classify` before
|
||||
this is reached, so the dangerous half of the operation has no branches in
|
||||
it at all. There is exactly one call site.
|
||||
"""
|
||||
gone = []
|
||||
for p in issue.slug_files(root, id):
|
||||
os.remove(p)
|
||||
gone.append(p)
|
||||
return gone
|
||||
|
||||
|
||||
def run(root, issues, ids=None, dry_run=False, out=None):
|
||||
"""Classify, report, remove, rebuild the index. Returns (gone, kept).
|
||||
|
||||
The one implementation of eviction, called both by `main` below and by the
|
||||
sync layer's `evict.py` — which does nothing to this decision except hand
|
||||
over issues whose `state:` it has just refreshed from the tracker.
|
||||
|
||||
`gone` is {id: [paths]} and is empty on a dry run; `kept` is
|
||||
[(id, why)] for everything considered and not removed.
|
||||
"""
|
||||
out = out or sys.stdout
|
||||
evict, protected, still_open = classify(issues, ids)
|
||||
|
||||
gone, kept = {}, []
|
||||
for id in evict:
|
||||
paths = issue.slug_files(root, id) if dry_run else remove(root, id)
|
||||
if not dry_run:
|
||||
gone[id] = paths
|
||||
out.write("%-11s %s\n" % ("would evict" if dry_run else "evicted", id))
|
||||
for p in paths:
|
||||
out.write(" %s\n" % p)
|
||||
for id in protected:
|
||||
kept.append((id, LOCAL_REASON))
|
||||
out.write("%-11s %s closed, %s\n" % ("kept", id, LOCAL_REASON))
|
||||
# An open issue is the normal case and says nothing worth a line — unless
|
||||
# the operator named it, in which case they are owed the reason.
|
||||
for id in still_open:
|
||||
kept.append((id, "state: %s" % issues[id].state))
|
||||
if ids:
|
||||
out.write("%-11s %s state: %s\n" % ("kept", id, issues[id].state))
|
||||
|
||||
if dry_run:
|
||||
out.write("%d issue(s) would be evicted, %d kept — nothing was touched\n"
|
||||
% (len(evict), len(kept)))
|
||||
return gone, kept
|
||||
|
||||
out.write("%d issue(s) evicted, %d kept\n" % (len(gone), len(kept)))
|
||||
# Only when something actually went: the index is a view of the directory,
|
||||
# and rewriting it after a run that changed nothing is a write nobody asked
|
||||
# for.
|
||||
if gone:
|
||||
path, n = issue_index.build(root)
|
||||
out.write("index: %s — %d issue(s)\n" % (path, n))
|
||||
return gone, kept
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Evict closed issues from the local store (offline)")
|
||||
ap.add_argument("ids", nargs="*",
|
||||
help="issue ids (default: every closed issue in the store)")
|
||||
ap.add_argument("--dry-run", action="store_true",
|
||||
help="print what would be removed; touch nothing")
|
||||
ap.add_argument("--out", default=issue.ISSUE_ROOT,
|
||||
help="store root (default: <repo>/tmp/issues)")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
root = args.out
|
||||
if not issue.store_exists(root):
|
||||
sys.exit("issue_evict.py: store %s does not exist — nothing to evict" % root)
|
||||
|
||||
issues = issue.load_all(root)
|
||||
missing = [i for i in args.ids if i not in issues]
|
||||
if missing:
|
||||
sys.exit("issue_evict.py: no such issue(s) in the store: %s"
|
||||
% ", ".join(missing))
|
||||
|
||||
run(root, issues, args.ids, args.dry_run)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user