Spotify's February 2026 changes did not just move /playlists/{id}/tracks to
/items: for apps on the new behavior the per-entry payload key was renamed
from "track" to "item" (tracks.tracks.track -> items.items.item). Extended
Quota Mode (grandfathered) apps keep the old key, which is why this never
reproduced locally. Every parser in alembic read only "track", so on a new
app each entry looked like a null/local track and was silently skipped: the
CSV came out header-only, sldl no-opped with exit 0, and the playlist page
showed an empty tracklist. Confirmed live on a new app against a playlist
the connected account owns.
All /items consumers (spotify_client.py, spotify-playlist-csv.py,
spotify-retag.py) now parse both key names, and the fields query filter is
gone: it selects by key name, so filtering on track(...) is itself what
returned empty pages on the renamed shape.
Also per the migration guide, new apps only receive playlist contents for
playlists the connected account owns or collaborates on; other playlists
return metadata with no items field at all (public is no longer enough).
That case now raises a pointed error (UI and CSV fetch) instead of reading
as an empty playlist, run-playlist.sh logs the fetched track count and warns
loudly when it is zero, and the README guidance is updated to match.
New tests pin get_playlist_tracks against both response shapes, the
metadata-only error, and pagination.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
connect.py's OAuth scope request (playlist-read-private
playlist-read-collaborative) didn't match what sldl's own built-in login
flow requests (same two, plus user-library-read). The narrower scope alembic
granted let sldl refresh a valid access token, but Spotify then returned 403
Forbidden loading the playlist regardless of ownership, which sldl turns
into an unhandled-exception crash (exit 134) instead of a clean scope error.
Found by reproducing a friend's crash: his playlist was his own, connected
via his own OAuth, correct redirect URI -- ruling out the ownership/
visibility explanation and pointing at the scope mismatch instead.
Also:
- run-playlist.sh: detect exit 134 specifically and log a pointed hint
(distinguishing the Forbidden-loading-playlist case from any other
unhandled sldl exception) instead of just "sldl finished with exit code
134" with no context.
- README: document that anyone who connected Spotify before this version
needs to click Connect Spotify again -- existing refresh tokens carry the
old, narrower scope and don't upgrade themselves.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Add "Connect Spotify" OAuth flow (/connect/spotify) so newly created Spotify
apps can read playlists: Spotify now requires a user token for playlist
reads, which client-credentials alone can no longer provide. The stored
refresh token is rendered as spotify-refresh into each playlist's sldl
.conf -- sldl's own docs confirm supplying it skips its interactive login
flow, which is what was causing multi-hour hangs on a stuck sync.
Grandfathered apps keep working unchanged if no account is connected.
- Fix the connect flow's redirect_uri: request.url_for() reflected the raw
connection scheme (http) rather than what the reverse proxy actually
served over, which Spotify's exact-match redirect_uri check rejected.
Force https rather than trust the unproxied scheme.
- Add an AzuraCast base URL credential field alongside the API key, with a
keyfile fallback so the scheduled enrich-buy-url.py job can actually reach
AzuraCast (previously it had no way to receive a base URL at all when run
from the scheduler, so the reprocess call was silently always skipped).
- Fix build-fingerprint-index.py exiting non-zero on any single fingerprint
failure instead of only when nothing was written.
- Guard enrich-buy-url.py's AzuraCast reprocess against an empty
--azuracast-base (raised ValueError since PD2 removed the hardcoded base).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Run now (jobs.run_now): playlists without a daily sync time are never
registered as scheduler jobs, so the old registered-job check 404'd. Fall
back to running the playlist directly via pipeline_runner for any
playlist:<name> key that isn't registered.
- Spotify removed GET /playlists/{id}/tracks in its February 2026 API changes
in favor of /items (identical response shape). New apps are already 403'd on
the old endpoint. Migrate both callers (app/services/spotify_client.py and
pipeline/lib/spotify-retag.py) to /items. (The vendored sldl downloader uses
its own bundled Spotify library and would need an upstream update when
/tracks is fully removed.)
- Friendlier on-screen error for a Spotify 403 (check credentials + playlist
must be public) and a README troubleshooting entry covering the 403, the
public-playlist requirement, and the org-only extended-quota reality.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Navidrome password is now passed to curl via stdin (--data-urlencode "p@-")
in navidrome-scan.sh and pipeline-status.sh, so it no longer appears in
ps/proc. Verified the query sent is identical and a live scan still triggers.
- MIN_ARTIST_DIRS (the share-health gate) is now a setting, threaded through to
the pipeline env, so a user with a small library can lower it instead of the
scan/sync being permanently blocked by the hardcoded 500.
- /auth/logout is now POST-only (with a nav form + aria-label), so a drive-by
GET can't log the user out; enforced allowed_email already landed separately.
- view_log now confirms the run's log_path resolves under the logs dir before
serving it (defense in depth).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- R11: pin requirements.txt to the versions running in the image, so a rebuild
can't pull a breaking upstream release. Documented how to upgrade.
- R12: log rotation now also prunes job_runs rows older than 90 days (nulling
the manual_imports FK reference first), so the table doesn't grow without
bound. dedup/genre candidates are review state and left alone.
- U5: credential form shows non-secret fields (URLs, usernames, prefs) as plain
text so they can be verified while typing; only real secrets stay masked.
Dashboard IP card reworded from gluetun-specific to generic "Outbound IP".
- S9: enforce ALLOWED_EMAIL at the OIDC callback, before any user row or session
is created, instead of only on later requests.
Verified: pinned build resolves; prune deletes only old rows and keeps the
manual_imports record; credentials page renders text+password inputs; a
disallowed email gets 403 with no user row, an allowed one succeeds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Run now (jobs and playlists), dedup scan, genre preview/apply, and manual
import now dispatch via BackgroundTasks and redirect immediately, instead
of awaiting a job that can run for the better part of an hour and hang the
browser or reverse proxy. Progress shows in the Jobs runs table (which
already polls); if the pipeline is busy the run records skipped_lock there.
- Fix the import banner, which claimed work was ongoing after the request
had actually blocked to completion; it now reflects the backgrounded start.
- Add confirmation prompts to the dedup per-row and bulk delete and to
"Fix genres now", matching the existing confirms on library and playlist
deletes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Security (P0):
- Remove committed session-secret default; auto-generate and persist a
random secret to the config volume when SESSION_SECRET is unset
(prevents forgeable session cookies / auth bypass).
- Validate playlist names to a safe charset and render sldl configs via
literal Python substitution instead of sed (closes a command-injection
and path-traversal path through playlist names).
- shlex-quote credential values written to shell-sourced env files, and
strip newlines from values patched into .conf files.
- Render playlist .conf files 0600; warn at startup if the master key is
co-located with the config volume; document keeping it separate.
Portability:
- Configurable timezone via TZ (default UTC) instead of hardcoded Edmonton.
- Remove personal defaults (navidrome user "andrew", ephemeral.club URLs).
- Ship generic example seeds; move the cross-album dedup keep-list and the
legacy playlist import to editable config files; drop the personal
_upgrade.csv.
- Generic VPN reference in docker-compose.snippet.yml.
First-run experience:
- Redirect to /setup instead of 500 when OIDC is unconfigured; surface a
missing master key inline; entrypoint exits with an actionable message
when the config folder is not writable.
- Add unauthenticated /health (JSON) and /setup (checklist) diagnostics.
Docs:
- Write docs/ARCHITECTURE.md and docs/MIGRATION.md (previously referenced
but missing); expand README with ownership, backups, advanced settings,
and migration guidance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
genre_review_service.run() and dedup_review_service._run_scan() both
unconditionally created a new run row even when the underlying job never
actually executed (skipped_lock, or any other non-success status). For
genres this was directly user-visible: the review page always shows the
most recent run, so a skipped click created an empty run that displaced
the real previous preview, making it look like every pending change had
vanished. Both now return None (persisting nothing) when the job didn't
succeed, and both routers surface a "didn't run, something else was
using the pipeline" notice instead of silently redirecting. Cleaned up
the one phantom empty genre_runs row already sitting in production,
restoring the real 20-change preview.
Also added a "Run now" button to each row on the Playlists list page
(previously only on a playlist's own detail page), for the common case
of adding new tracks and wanting to sync immediately.
Playlist detail page's status section (and the new two-column compare
view) was never rendering: the router awaited status_service.playlist_status(),
but that function is plain sync, so awaiting its dict return raised
"object dict can't be used in 'await' expression" on every load, silently
caught and shown as a generic fetch error. Removed the erroneous await.
find-fuzzy-dupes.py's --apply and --apply-pairs both deleted by a `path::`
regex query even though the beets id was already available in scope --
same mixed-path-storage issue (pre/post beets-2.11-upgrade items store
absolute vs. library-relative paths) dedup-library.sh already worked
around by deleting via id instead. This silently failed to match for most
confirmed fuzzy-audio deletions (verified: 19 of 20 in one run). Added a
delete_id column to dedup_candidates, threaded through the scan/apply
pipeline, and switched both delete call sites to `id:`.
Dedup scans also never checked whether a pair was already sitting in the
pending list, so every re-scan (including the daily schedule) added a new
row for the same unreviewed duplicate -- cleaned up 38 redundant rows
already in production and added a check so future scans skip a pair
that's already pending.
Dashboard gets a Public IP stat card (cached 10 min, fetched via ipify)
as a quick confidence check that outbound traffic is actually routed
through gluetun's VPN and not the home connection.
Credentials page now shows a fact-checked one-line description per
service and marks Spotify/Soulseek/Navidrome as Required (no toggle --
they're load-bearing for every playlist sync). The four optional
integrations (Bandcamp, AzuraCast, Qobuz, Telegram) get a real on/off
switch: Bandcamp pauses its scheduled job (same mechanism as the Jobs
page), the other three just remove their rendered credential file, which
the scripts that read them already treat as "not configured, skip" --
no script changes needed. Disabled optional scopes also drop off the
dashboard's credential status row.
New Settings > Artist Casing page to view/add/remove entries in
artist-canonical.list without shelling in -- adding a name that already
matches case-insensitively replaces its casing in place instead of
duplicating.
Playlist detail page's status section is now two side-by-side columns:
the original Spotify tracklist on the left, and a tabbed Downloaded /
Waiting-to-download view on the right (client-side tabs via Alpine),
both with an internal scroll area so long playlists don't pull the
columns out of sync. Uses the existing status_service reconciliation
(sldl index + beets + quarantine) -- no backend changes needed there.
Settings is now one top-nav entry with Credentials and Jobs as sidebar
sub-pages (jobs moved from /jobs to /settings/jobs). Manual import page
replaces the bare file input and free-text filename field with a drag-drop
dropzone and a proper file-picker table. Genres page drops the "force"
checkbox for two explicit buttons (Preview / Fix genres now). Dedup's
"Scan now" runs both the file-naming and acoustic passes together, and the
table labels which pass caught each candidate instead of showing the raw
pass name. .btn-ghost gets a visible border so it reads as a real button
next to Delete/Danger actions instead of looking unaligned. Job names
throughout the UI are now human-readable instead of raw job_key strings.
Also includes the fpcalc exit-code fix from earlier (fingerprint index
was discarding valid fingerprints on files with a benign decode warning).
confirm_and_apply no longer marks candidates confirmed before their apply
job actually succeeds (a stuck/skipped_lock run was silently vanishing
confirmed deletes without deleting anything), switches the fuzzy pass to
--apply-pairs to avoid a full rescan on every confirm, and adds a job
timeout so a hung subprocess can't hold the global pipeline lock forever.
dedup-library.sh's Pass 2-4 use gawk-only features (gensub, POSIX interval
regex) but were running under mawk (Debian's default awk) in the deployed
image, throwing silent syntax errors on every run -- switched those
invocations to gawk explicitly and added it to the Dockerfile.
Also: per-track delete button on the library list/detail pages, and two
CSS fixes (the primary button's hover gradient was getting clobbered by
the base .btn:hover rule's plain background, and the select dropdown
arrow had no background-size so it rendered oversized).
Lets a candidate be marked ignored instead of confirmed/applied; future
scans skip re-flagging the same path pair once it's been ignored, since
a rescan can flip which side ranks as keep/delete without changing the
user's earlier decision that the pair is fine as two copies. Also adds
--apply-pairs to find-fuzzy-dupes.py to apply pre-confirmed pairs
without a full rescan.
Playlist M3U regen now writes to a temp file and renames it into place,
so a stray wrong-owner leftover file (root:root, from pre-migration
host-cron runs) can't block nightly writes the way it did last night.
Dashboard now lists tracks added to the beets library in the last 24h
with playlist and format, sourced from a new beets_service.recently_added().
- Strip the common /data/music/Library/ prefix from displayed keep/delete
paths (every track lives there, so it added width without adding
information); full path is still available via a hover title.
- Table now uses a colgroup with fixed column widths so headers stay
aligned with their content regardless of how long a given path is,
instead of drifting with the widest cell in the column.
- Add a per-row Delete button (self-contained form per cell) alongside
the existing checkbox + bulk "Delete selected" flow -- the checkboxes
now target an external form via the HTML `form` attribute instead of
wrapping the table in a form, since a form can't legally nest another
form for the per-row buttons.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- beets_service: filter params now use truthy checks instead of `is not
None`, since a real <select> left on "(any)" submits an empty string,
not an absent param -- the filter form silently matched zero rows for
any real submission. Also match grouping tokens within "; "-joined
multi-playlist values instead of requiring an exact string match.
- dashboard: credential status renders as compact dot indicators instead
of badge+text chips, so long scope names (telegram) stay on one line.
- credentials page: two-column grid layout instead of one long column.
- find-fuzzy-dupes.py: add --json/--only-paths flags matching
dedup-library.sh's convention, so it can plug into the same review
queue.
- dedup_review_service: add scan_fuzzy() and route confirm_and_apply()
to the correct underlying script (dedup-library.sh vs
find-fuzzy-dupes.py) per candidate's pass_name, since tag-based passes
miss duplicates whose tags differ even when the audio is identical
(e.g. a remix credited to different artists between two copies).
- dedup page: add a "Scan for audio duplicates" trigger alongside the
existing tag-based scan.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
job descriptions
Dashboard: format breakdown (FLAC/MP3/etc. counts), approximate library
size (bitrate*length/8, same approximation `beet stats` itself uses --
verified to match its "100.5 GiB" output exactly against the real
library), and a credential auth-state summary per scope (configured y/n
plus, for Bandcamp, cookie expiry parsed locally from the stored cookie
jar -- deliberately not a live network probe like pipeline-status.sh's
Qobuz check, since this renders on every dashboard load).
Library: was one unfiltered page dumping all 4072 tracks. Added search
(artist/title/album), playlist and format filter dropdowns, and real
SQL-level pagination (LIMIT/OFFSET, not fetch-everything-then-slice).
beets_service gained count_items()/distinct_formats() to support this.
Playlists: cron_expr is still the stored/scheduled representation, but the
UI now shows and edits a plain daily time picker instead of raw cron
syntax -- every playlist schedule today is a simple daily HH:MM anyway.
playlist_service.cron_to_time()/time_to_cron() convert at the router
boundary; verified round-trip against all real playlist cron values.
Jobs: each maintenance job now shows a short one-line description of what
it actually does (MAINTENANCE_JOB_DESCRIPTIONS), and "next run" is
formatted consistently with playlists' time style (HH:MM, with a day
qualifier for non-today runs -- maintenance jobs can be weekly/monthly,
unlike playlists' plain daily schedule).
Verified end-to-end against the real production data (4072 tracks, 100.5
GiB, real credentials): stats/format-breakdown/search/pagination all
correct, all 7 credential scopes report configured with Bandcamp's real
cookie expiry (325 days), and a full authenticated page-render sweep
(dashboard, library with every filter combination, playlist detail,
jobs) all returned 200 with no template errors.
Vendored htmx.min.js (1.9.12) and alpine.min.js (3.14.1) into static/ --
self-hosted per the no-CDN/CSP requirement, included in base.html.
routers/playlists.py: list/add/remove playlists, a detail page showing
live per-playlist status via status_service (Spotify vs. beets vs. sldl
index vs. quarantine), edit form for active/cron_expr/no_m3u/notes, and a
"Run now" button wired to the jobs run-now endpoint.
routers/credentials.py: write-only credential forms per scope (a saved
secret is never sent back to the browser -- fields just show "(set)" and
leaving a field blank keeps its current value, matching
credential_service.set_credentials' partial-update semantics).
routers/jobs.py: maintenance job list with run-now/enable-disable, and a
recent-runs table that auto-refreshes via HTMX polling
(hx-trigger="every 5s") against a partial-only endpoint -- the one place
in the UI where avoiding a full-page reload actually matters, since job
status changes while the page is sitting open. Per-run log viewing.
Dashboard now shows real data (beets stats, playlist counts, recent job
runs) instead of the placeholder from Task 2.
Verified all 10 authenticated pages render successfully (200, no template
errors) via TestClient with require_auth overridden, including the
playlist detail page's graceful-degradation path when Spotify credentials
aren't configured yet (renders a "couldn't fetch live status" message
instead of a 500).
dedup-library.sh: additive --json (one NDJSON line per candidate deletion
to stdout, alongside the unchanged human log) and --only-paths FILE (in
--apply mode, only actually delete entries whose path is in FILE; without
it, --apply deletes everything as before -- existing direct callers are
unaffected). Without --only-paths the safety re-verification is free:
--apply --only-paths re-runs all 4 passes from scratch on every invocation,
so if a group's ranking changed since a scan (e.g. the old keep_path is
gone), the fresh pass assigns the previously-"delete" path the KEEP role
instead and the only-paths allowlist naming it is simply never consulted --
no duplicate ranking logic needed in the review service.
spotify-genre.py: additive --json emitting one JSON line per genre change
(dry-run or --apply) for genre_review_service to persist.
pipeline_runner.run_job_capture(): like run_job() but captures stdout as
text (still under the same shared lock, still writes a job_runs row) for
callers that need to parse structured output rather than just log it.
dedup_review_service.scan() persists dry-run candidates into
dedup_runs/dedup_candidates. confirm_and_apply() re-checks confirmed
candidates still exist before invoking --apply --only-paths, so nothing is
ever deleted without an explicit confirm -- matches the false-negative-
biased dedup preference. scheduler_service's maintenance:dedup job now
goes through this (still dry-run only, every day).
genre_review_service.run() wraps spotify-genre.py for both dry-run preview
and the real scheduled --apply run, persisting every run's diff into
genre_runs/genre_candidates either way -- genre writes keep their current
auto-apply behavior (low-risk, reversible, GENRE_LOCK-protected) but are
now reviewable after the fact. lock_artist_genre() gives a one-click revert
path when a run gets something wrong.
Added minimal routers+templates for /dedup (scan, review, confirm-and-
delete) and /genres (preview, review, lock-old-genre).
Verified end-to-end against REAL duplicate files (not mocked): built an
actual FLAC+MP3 duplicate pair in a real beets library, ran dedup-library.sh
--json and confirmed correct JSON output, verified --apply --only-paths
with an empty confirm list deletes nothing and with the real confirmed path
deletes exactly that file (DB + disk) while preserving the FLAC, and ran
the full dedup_review_service scan->confirm->apply flow through the same
fixture. genre_review_service and spotify-genre.py --json verified against
mocked/direct output (spotify-genre.py's own artist-genre lookup needs a
live Spotify API call, out of reach in this sandbox). Confirmed the full
app boots with all five routers registered.
library_edit.update_track_fields: single-track tag edits via `beet modify
-y -m` (subprocess with an argument list, not beets.library.Library
in-process -- an arg-list subprocess has no shell-injection surface to
avoid in the first place, and correctly reuses beets' own configured
path-format/move logic rather than reimplementing it against a hand-built
Library object with guessed config). Every changed field gets a
manual_fix_audit row with old/new values; triggers a Navidrome rescan
afterward if anything changed. Uses "genres" (plural) as the field name,
matching this deployment's actual beets column and every existing script's
convention, not the generic beets docs' singular "genre".
genre_fix.set_artist_genre and retag.retag_from_url wrap the existing
fix-genre.sh and fix-track-metadata.py as callable services rather than
reimplementing their per-format mutagen tag-writing logic.
manual_import: ties pipeline_runner.run_import_track to the manual_imports
table, mirrors import-track.sh's own argument-count-based disambiguation
(0/1/2 args), plus import-me/ listing and upload-with-path-traversal-guard.
routers/library.py + routers/import_.py: browsable library list, a track
detail/edit page (tag edit, artist-wide genre override, retag-from-URL
forms), and the import page (upload + trigger). Minimal templates for now,
Task 9 covers full UI polish.
Verified update_track_fields end-to-end against a REAL beets library (not
mocked): generated a tagged FLAC with ffmpeg/metaflac, imported it via a
real `beet import`, edited artist+title through the service, and confirmed
the file was physically moved to the new path-template location, tags were
rewritten on disk, the DB updated, and both changes landed correctly in
manual_fix_audit. Also verified the full app boots with the new routers
registered and all four new routes correctly redirect to login when
unauthenticated.
- settings.py: pydantic-settings config for the two mount points
(MUSIC_DATA_DIR, ALEMBIC_CONFIG_DIR), OIDC client config, encryption key path
- db.py: SQLite engine with WAL mode, plain-SQL schema migration runner
(schema/*.sql, tracked by schema_version -- not the alembic migration tool)
- models.py: SQLAlchemy ORM models matching schema/0001_init.sql
- security/crypto.py: Fernet encrypt/decrypt for the secrets table
- security/oidc.py + routers/auth.py: Pocket ID OIDC login/callback/logout
- security/deps.py: require_auth dependency (redirect-to-login on no session)
- main.py: app factory, lifespan (init_db + beets WAL enable), session
middleware, minimal dashboard route
Verified boots end-to-end via TestClient: unauthenticated GET / redirects to
/auth/login, static files serve, and all 13 schema tables get created on
first run.
andrew