0.6.11: Auto-apply exact-tag format upgrades; self-heal stale pending rows
Two gaps left obvious cases stuck in the manual dedup queue: 1. Format-upgrade pairs found by Pass 2/3 (case-insensitive/normalized tag match) still required manual confirm even though the pass itself already proved same song via identical tags, and rank_file() already guarantees FLAC beats any other format regardless of size/dirty-name. A directory-casing difference (e.g. "All About This Ep" vs "...EP") meant these never matched the narrower same-directory numbered-twin rule from 0.6.10. New is_format_upgrade() auto-deletes any Pass 2/3 pair whose extensions differ, gated on pass_name so Pass 4 (cross-album fuzzy -- different masters, DJ-mix edits, genuinely ambiguous) is untouched and still requires a human. 2. A pending row can go stale without ever being confirmed -- some other action (a later auto-apply, upgrade-mp3-to-flac, a manual fix) already resolved one side of the pair -- and nothing pruned it, so an already-fixed duplicate kept surfacing in the queue indefinitely. scan() now sweeps and auto-resolves any pending candidate whose keep_path or delete_path no longer exists before persisting new ones, so the queue reflects current reality on every run instead of accumulating dead entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -27,6 +27,38 @@ def _script_for_pass(pass_name: str) -> str:
|
||||
return _FUZZY_SCRIPT if pass_name == _FUZZY_PASS else _SCRIPT
|
||||
|
||||
|
||||
def _prune_stale_pending(db) -> int:
|
||||
"""Mark pending candidates whose delete_path or keep_path no longer
|
||||
exists as resolved, instead of leaving them to surface forever.
|
||||
|
||||
A pending row can go stale without ever being confirmed through the UI --
|
||||
e.g. a later scan's numbered-twin/format-upgrade auto-apply already
|
||||
removed one side, or upgrade-mp3-to-flac replaced it, or someone deleted
|
||||
it by hand. confirm_and_apply() already does this "already_gone" check
|
||||
for candidates a user explicitly acts on; this generalizes it to run at
|
||||
the start of every scan, so the pending list reflects current reality
|
||||
instead of stale groups that something else already resolved."""
|
||||
now = time.time()
|
||||
rows = db.execute(
|
||||
select(DedupCandidate).where(
|
||||
DedupCandidate.applied == False, # noqa: E712
|
||||
DedupCandidate.confirmed == False, # noqa: E712
|
||||
DedupCandidate.ignored == False, # noqa: E712
|
||||
)
|
||||
).scalars().all()
|
||||
pruned = 0
|
||||
for c in rows:
|
||||
if not Path(c.delete_path).exists() or not Path(c.keep_path).exists():
|
||||
c.confirmed = True
|
||||
c.confirmed_by = "auto:stale"
|
||||
c.confirmed_at = now
|
||||
c.applied = True
|
||||
pruned += 1
|
||||
if pruned:
|
||||
db.commit()
|
||||
return pruned
|
||||
|
||||
|
||||
def _is_pair_ignored(db, path_a: str, path_b: str) -> bool:
|
||||
"""True if this path pair was ever marked 'keep both', regardless of
|
||||
which path was on the keep/delete side that time -- a later scan can
|
||||
@@ -78,6 +110,7 @@ async def _run_scan(
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
_prune_stale_pending(db)
|
||||
now = time.time()
|
||||
dedup_run = DedupRun(
|
||||
started_at=job_run.started_at,
|
||||
@@ -158,15 +191,19 @@ async def scan(triggered_by: str = "manual") -> DedupRun | None:
|
||||
"""Dry-run dedup-library.sh --json (plus --auto-apply-numbered), persist
|
||||
every candidate deletion into a fresh dedup_runs/dedup_candidates pair.
|
||||
|
||||
Everything except identical numbered-sibling twins (same dir, same name,
|
||||
same extension, differing only by the ".N" collision suffix) stays a
|
||||
dry-run candidate awaiting manual confirm_and_apply() below -- that's the
|
||||
false-negative-biased preference for tag-based/fuzzy matches, where a
|
||||
wrong auto-delete could take out a genuinely different version (a
|
||||
different album pressing, a DJ-mix edit, etc.). Numbered twins have none
|
||||
of that ambiguity -- it's the same download landing twice -- so
|
||||
dedup-library.sh deletes those unconditionally itself and reports them
|
||||
here already applied, purely for audit visibility."""
|
||||
Two kinds of match need no human judgment and get deleted unconditionally
|
||||
by dedup-library.sh itself (see is_numbered_twin/is_format_upgrade there):
|
||||
identical numbered-sibling twins (same dir, same name, same extension,
|
||||
differing only by the ".N" collision suffix), and format upgrades within
|
||||
an EXACT tag match from Pass 2/3 (case-insensitive/normalized already
|
||||
proved same song via identical tags, so a differing extension there is
|
||||
just "better format vs. worse"). Everything else -- including Pass 4's
|
||||
cross-album fuzzy matches, where a wrong auto-delete could take out a
|
||||
genuinely different version (a different album pressing, a DJ-mix edit,
|
||||
etc.) -- stays a dry-run candidate awaiting manual confirm_and_apply()
|
||||
below; that's the false-negative-biased preference for anything with real
|
||||
ambiguity. Auto-applied deletions are reported here already applied,
|
||||
purely for audit visibility."""
|
||||
return await _run_scan("dedup:scan", _SCRIPT, triggered_by, extra_args=("--auto-apply-numbered",))
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user