9d1eee33375b12893b8bbeade25c7336f1bd1a60
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 andrew
|
c87ddc2649 |
Dashboard detail, library filtering/pagination, simplified playlist times,
job descriptions Dashboard: format breakdown (FLAC/MP3/etc. counts), approximate library size (bitrate*length/8, same approximation `beet stats` itself uses -- verified to match its "100.5 GiB" output exactly against the real library), and a credential auth-state summary per scope (configured y/n plus, for Bandcamp, cookie expiry parsed locally from the stored cookie jar -- deliberately not a live network probe like pipeline-status.sh's Qobuz check, since this renders on every dashboard load). Library: was one unfiltered page dumping all 4072 tracks. Added search (artist/title/album), playlist and format filter dropdowns, and real SQL-level pagination (LIMIT/OFFSET, not fetch-everything-then-slice). beets_service gained count_items()/distinct_formats() to support this. Playlists: cron_expr is still the stored/scheduled representation, but the UI now shows and edits a plain daily time picker instead of raw cron syntax -- every playlist schedule today is a simple daily HH:MM anyway. playlist_service.cron_to_time()/time_to_cron() convert at the router boundary; verified round-trip against all real playlist cron values. Jobs: each maintenance job now shows a short one-line description of what it actually does (MAINTENANCE_JOB_DESCRIPTIONS), and "next run" is formatted consistently with playlists' time style (HH:MM, with a day qualifier for non-today runs -- maintenance jobs can be weekly/monthly, unlike playlists' plain daily schedule). Verified end-to-end against the real production data (4072 tracks, 100.5 GiB, real credentials): stats/format-breakdown/search/pagination all correct, all 7 credential scopes report configured with Bandcamp's real cookie expiry (325 days), and a full authenticated page-render sweep (dashboard, library with every filter combination, playlist detail, jobs) all returned 200 with no template errors. |
||
|
andrew
|
2075d6cf66 |
Add dedup_review_service and genre_review_service
dedup-library.sh: additive --json (one NDJSON line per candidate deletion to stdout, alongside the unchanged human log) and --only-paths FILE (in --apply mode, only actually delete entries whose path is in FILE; without it, --apply deletes everything as before -- existing direct callers are unaffected). Without --only-paths the safety re-verification is free: --apply --only-paths re-runs all 4 passes from scratch on every invocation, so if a group's ranking changed since a scan (e.g. the old keep_path is gone), the fresh pass assigns the previously-"delete" path the KEEP role instead and the only-paths allowlist naming it is simply never consulted -- no duplicate ranking logic needed in the review service. spotify-genre.py: additive --json emitting one JSON line per genre change (dry-run or --apply) for genre_review_service to persist. pipeline_runner.run_job_capture(): like run_job() but captures stdout as text (still under the same shared lock, still writes a job_runs row) for callers that need to parse structured output rather than just log it. dedup_review_service.scan() persists dry-run candidates into dedup_runs/dedup_candidates. confirm_and_apply() re-checks confirmed candidates still exist before invoking --apply --only-paths, so nothing is ever deleted without an explicit confirm -- matches the false-negative- biased dedup preference. scheduler_service's maintenance:dedup job now goes through this (still dry-run only, every day). genre_review_service.run() wraps spotify-genre.py for both dry-run preview and the real scheduled --apply run, persisting every run's diff into genre_runs/genre_candidates either way -- genre writes keep their current auto-apply behavior (low-risk, reversible, GENRE_LOCK-protected) but are now reviewable after the fact. lock_artist_genre() gives a one-click revert path when a run gets something wrong. Added minimal routers+templates for /dedup (scan, review, confirm-and- delete) and /genres (preview, review, lock-old-genre). Verified end-to-end against REAL duplicate files (not mocked): built an actual FLAC+MP3 duplicate pair in a real beets library, ran dedup-library.sh --json and confirmed correct JSON output, verified --apply --only-paths with an empty confirm list deletes nothing and with the real confirmed path deletes exactly that file (DB + disk) while preserving the FLAC, and ran the full dedup_review_service scan->confirm->apply flow through the same fixture. genre_review_service and spotify-genre.py --json verified against mocked/direct output (spotify-genre.py's own artist-genre lookup needs a live Spotify API call, out of reach in this sandbox). Confirmed the full app boots with all five routers registered. |
||
|
andrew
|
4e0cfb8463 |
Add scheduler_service and pipeline_runner
pipeline_runner: subprocess execution wrapper replacing flock -n with a single asyncio.Lock (deliberately one global lock, not per-resource -- matches the old flock's all-jobs-share-one-lock behavior rather than over-engineering it). Records one job_runs row per invocation (started/finished/status/exit_code/summary/log_path/triggered_by). If the lock is already held, records status='skipped_lock' immediately instead of silently dropping the run. Found and fixed a real concurrency bug via testing: the obvious asyncio.wait_for(lock.acquire(), timeout=0) idiom for a non-blocking try-acquire is broken in asyncio -- the wrapping Task's first iteration and the timeout-0 callback race with no guaranteed ordering, so it timed out on literally every call, including the very first uncontended one. Fixed using lock.locked() + acquire(), relying on acquire()'s fast path never suspending when uncontended. scheduler_service: AsyncIOScheduler with the default in-memory job store (NOT SQLAlchemyJobStore -- it pickles job functions to persist them, and the _lib()/_beet() factory closures here aren't picklable; MemoryJobStore avoids this since register_all_jobs() rebuilds the whole schedule from code + DB on every startup anyway). All 19 maintenance jobs ported 1:1 from /etc/cron.d/sldl-maintenance with their exact cron expressions; dedup deliberately has no --apply (dry-run only on schedule, per the false-negative-biased dedup preference). Playlist jobs are synced live from the playlists table (sync_playlist_jobs), so add/remove-in-the-UI takes effect with no redeploy -- wired into playlist_service's create/update/ delete. Maintenance job enable/disable persists to the scheduled_jobs table so a paused job stays paused across a restart despite the in-memory job store. trigger_now() supports "Run now" by invoking a job's function immediately with triggered_by='manual', bypassing its schedule. Wired into main.py's lifespan (start on boot, shutdown on exit). Verified via integration tests: 36 jobs register correctly (17 seeded playlists + 19 maintenance) with correct cron triggers; live playlist create/delete correctly adds/removes its scheduler job; maintenance enable/disable persists to the DB and takes effect live; trigger_now correctly bypasses the schedule; pipeline_runner correctly handles success/failure/timeout/concurrent-lock-contention with real subprocesses; and the full app boots with the scheduler running and shuts down cleanly. |