The 2026-07-08 path rewrite changed beets' displayed paths from the
transitional /music/... mount to real /data/music/Library/... paths.
dedup-library.sh was fixed in 0.6.8; three more scripts still assumed
the old prefix, each failing silently:
- replace-with-better.sh resolved every library match to a doubled
nonexistent path, logged [stale-db], skipped the in-place replace,
and then imported the staged FLAC as a NEW track via the
leftover-import step. Net effect: the mp3-to-flac upgrade flow
produced flac+mp3 twin pairs in the library (surfacing in the dedup
queue) instead of replacing the MP3 in place.
- fix-track-metadata.py queried beets only by the legacy /music/...
form, matched nothing, and silently skipped beet update/move after
retagging.
- export-laptop-playlists.py filtered out every library track (no
displayed path starts with /music/ anymore), producing empty
exports.
All three now use the real displayed path and keep the /music/... form
only as a legacy fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two bugs conspired to fill the library with .1.flac duplicates on the
night of 2026-07-15/16:
1. sldl derives its skip-index folder from the input: the Spotify
playlist display name before 0.6.2, the CSV filename after. The
switch stranded every playlist's download history in the old
emoji-named subfolder, so sldl started from an empty index and
re-downloaded every playlist in full. Fix: pin index-path in
_template.conf, seed the pinned file from all stranded indexes
(merge-sldl-indexes.py, called from run-playlist.sh whenever the
pinned index is missing or older than a stranded one). Recovered
rows keep the CSV-era identity fields but are marked downloaded, so
tracks that failed last night yet exist in the library since months
ago are not fetched again.
2. The safety net that should have flagged the flood, dedup-library.sh,
has silently reported 0 groups since the 2026-07-08 path rewrite:
host_path() still assumed /music/... display paths and prepended the
library dir to already-correct /data/music/Library/... paths, so
every candidate failed the -f check. Fix: pass real paths through
unchanged, keep the /music prefix swap only for legacy entries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
slskd is not part of the pipeline -- sldl is its own Soulseek client and
nothing in alembic calls slskd's API; the digest's reachability ping was its
only mention. It keeps running on the host purely as the user's own
file-sharing presence, so checking it here just risked a permanent bogus
warning line for something alembic doesn't depend on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Iterating on the 0.6.5 HTML digest on a real phone: <pre> column layout
forces fixed-width lines that wrap into unreadable fragments ("54 tracks
in / M3U"), and one <pre> per section rendered as a stack of copy-button
widgets. Every row is now plain proportional text -- colored dot, bold
label, "-- value" -- which reflows cleanly at any screen width; breakdowns
(added-today per playlist, library by format) become bullet lists. Header
banner (brand line, dot tally, all-clear/N-issues blockquote) unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The daily digest is now formatted with Telegram HTML: bold section headers
with <pre>-aligned columns, colored circle emoji as status dots, a branded
header line, and a top blockquote callout (all clear vs N issues). All free
text is &/</> escaped so a stray angle bracket in a log snippet can't break
the parse and swallow the whole message. notify-telegram.sh gains an --html
flag (parse_mode=HTML) used by the digest send; plain-text callers are
unchanged. Truncation is now tag-safe in HTML mode: cut on a line boundary,
re-close an open <pre>, and use an escaped marker -- the old literal
"...<truncated>" tail would itself have 400'd the send.
Two real bugs surfaced while testing:
- pipeline-status.sh pins its own PATH, which lacked /opt/venv/bin where
beet lives, so every beet probe ("added today", "Library by format", the
mp3-now count) has been silently empty behind 2>/dev/null since the cron
migration. PATH now includes the venv; both sections show real numbers.
- strip-watermark-art.py aborted its entire weekly run when metaflac stalled
on ONE file (seen today: a healthy 190KB cover took >15s under disk
contention, TimeoutExpired killed the job). Per-file timeout is now 60s
and a timeout skips that file with a warning instead of failing the run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Telegram digest reported healthy weekly jobs as "never run yet": its
maintenance section still globbed for cron-era per-script log filenames
(clean-index-*.log etc.) that jobs run through pipeline_runner no longer
write. It now reads job_runs, the scheduler's own record, so it reports the
last success (age + summary snippet from that run's log), flags a newer
failed or lock-skipped attempt on the same line, and distinguishes "never
succeeded (last attempt: skipped, lock busy)" from genuinely never
scheduled. Also matched the clear-bad-genres snippet grep to the script's
current summary wording, and dropped the now-unused newest_log helper.
The DB also showed WHY two jobs had never run: on Sunday the 08:30
strip-mb-tags run took 10m19s while holding the shared pipeline lock, so
strip-watermark-art (08:35) and scrub-watermark-text (08:40) hit flock-style
instant skip and lost their only slot of the week -- the 08:40 job missed by
19 seconds. Scheduled runs now wait up to 30 minutes for the lock
(SCHEDULED_LOCK_WAIT_SECONDS) and only then record skipped_lock, so
fixed-time blocks queue instead of starving; manual "Run now" keeps the
instant skip since a person expects an immediate answer.
Tests: scheduled-run queueing, lock-wait timeout, and manual instant-skip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spotify's February 2026 changes did not just move /playlists/{id}/tracks to
/items: for apps on the new behavior the per-entry payload key was renamed
from "track" to "item" (tracks.tracks.track -> items.items.item). Extended
Quota Mode (grandfathered) apps keep the old key, which is why this never
reproduced locally. Every parser in alembic read only "track", so on a new
app each entry looked like a null/local track and was silently skipped: the
CSV came out header-only, sldl no-opped with exit 0, and the playlist page
showed an empty tracklist. Confirmed live on a new app against a playlist
the connected account owns.
All /items consumers (spotify_client.py, spotify-playlist-csv.py,
spotify-retag.py) now parse both key names, and the fields query filter is
gone: it selects by key name, so filtering on track(...) is itself what
returned empty pages on the renamed shape.
Also per the migration guide, new apps only receive playlist contents for
playlists the connected account owns or collaborates on; other playlists
return metadata with no items field at all (public is no longer enough).
That case now raises a pointed error (UI and CSV fetch) instead of reading
as an empty playlist, run-playlist.sh logs the fetched track count and warns
loudly when it is zero, and the README guidance is updated to match.
New tests pin get_playlist_tracks against both response shapes, the
metadata-only error, and pagination.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sldl's vendored Spotify client still calls GET /playlists/{id}/tracks, which
Spotify removed in its February 2026 API changes. Grandfathered apps still get
a pass; apps created after the change get a hard 403 there no matter how they
authenticate (client-credentials or a correctly-scoped OAuth user token), so
sldl can never load a playlist for a new app regardless of what we hand it.
Instead of depending on sldl's Spotify client at all, run-playlist.sh now reads
the playlist itself via the still-working /items endpoint (new
spotify-playlist-csv.py, creds sourced from _spotify.env) and invokes sldl with
the CSV + --input-type csv, which override the conf's input lines while -c still
supplies Soulseek login, paths, and quality settings (verified live). The same
CSV format upgrade-mp3-to-flac.sh already feeds sldl. All artists are
comma-joined in the CSV so multi-artist tracks search no worse than before, and
a 403/404 on the fetch prints the account-visibility hint instead of a bare
traceback.
Since sldl no longer talks to Spotify, playlist .confs no longer carry
spotify-id/spotify-secret/spotify-refresh: _template.conf drops them,
render_playlist_confs stops injecting them, and a Spotify credential save no
longer re-renders confs (only _spotify.env). The retag step in run-playlist.sh
reuses the sourced _spotify.env creds instead of scraping the conf lines that
no longer exist. Confs are also re-rendered once at app startup so
already-deployed confs converge on upgrade (and shed the stale secret lines)
without waiting for a playlist or credential change. Playlist delete now also
removes the generated .csv.
Tests updated: conf rendering must NOT contain Spotify creds but must keep the
input URL line; new coverage for _spotify.env rendering (quoting, refresh-token
presence/blank, 0600).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
connect.py's OAuth scope request (playlist-read-private
playlist-read-collaborative) didn't match what sldl's own built-in login
flow requests (same two, plus user-library-read). The narrower scope alembic
granted let sldl refresh a valid access token, but Spotify then returned 403
Forbidden loading the playlist regardless of ownership, which sldl turns
into an unhandled-exception crash (exit 134) instead of a clean scope error.
Found by reproducing a friend's crash: his playlist was his own, connected
via his own OAuth, correct redirect URI -- ruling out the ownership/
visibility explanation and pointing at the scope mismatch instead.
Also:
- run-playlist.sh: detect exit 134 specifically and log a pointed hint
(distinguishing the Forbidden-loading-playlist case from any other
unhandled sldl exception) instead of just "sldl finished with exit code
134" with no context.
- README: document that anyone who connected Spotify before this version
needs to click Connect Spotify again -- existing refresh tokens carry the
old, narrower scope and don't upgrade themselves.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Previously described the Connect Spotify step in soft "one click, optional
either way" language and never used the word OAuth at all, even though it's
now a required step for any Spotify app created after Spotify's 2025/2026
API change. Rewrote "What you need before you start", the credentials
table, and the connect section to say plainly that keys alone no longer
work and walk through the OAuth login as a required numbered step.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Add "Connect Spotify" OAuth flow (/connect/spotify) so newly created Spotify
apps can read playlists: Spotify now requires a user token for playlist
reads, which client-credentials alone can no longer provide. The stored
refresh token is rendered as spotify-refresh into each playlist's sldl
.conf -- sldl's own docs confirm supplying it skips its interactive login
flow, which is what was causing multi-hour hangs on a stuck sync.
Grandfathered apps keep working unchanged if no account is connected.
- Fix the connect flow's redirect_uri: request.url_for() reflected the raw
connection scheme (http) rather than what the reverse proxy actually
served over, which Spotify's exact-match redirect_uri check rejected.
Force https rather than trust the unproxied scheme.
- Add an AzuraCast base URL credential field alongside the API key, with a
keyfile fallback so the scheduled enrich-buy-url.py job can actually reach
AzuraCast (previously it had no way to receive a base URL at all when run
from the scheduler, so the reprocess call was silently always skipped).
- Fix build-fingerprint-index.py exiting non-zero on any single fingerprint
failure instead of only when nothing was written.
- Guard enrich-buy-url.py's AzuraCast reprocess against an empty
--azuracast-base (raised ValueError since PD2 removed the hardcoded base).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Testing a brand-new app confirmed /tracks returns 403 and /items returns 401
'valid user authentication required' for app-only (client-credentials) access.
Update the prerequisite, the credentials table, and Troubleshooting so new
self-hosters know they need a grandfathered app's Client ID/Secret rather than
creating their own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Run now (jobs.run_now): playlists without a daily sync time are never
registered as scheduler jobs, so the old registered-job check 404'd. Fall
back to running the playlist directly via pipeline_runner for any
playlist:<name> key that isn't registered.
- Spotify removed GET /playlists/{id}/tracks in its February 2026 API changes
in favor of /items (identical response shape). New apps are already 403'd on
the old endpoint. Migrate both callers (app/services/spotify_client.py and
pipeline/lib/spotify-retag.py) to /items. (The vendored sldl downloader uses
its own bundled Spotify library and would need an upstream update when
/tracks is fully removed.)
- Friendlier on-screen error for a Spotify 403 (check credentials + playlist
must be public) and a README troubleshooting entry covering the 403, the
public-playlist requirement, and the org-only extended-quota reality.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point the install at the published image (git.kretzer.club/andrew/alembic:0.5)
now that v0.5 is on the registry: docker pull instead of clone+build, the
compose example uses the registry image, and Updating is docker compose
pull/up instead of git pull + build. Setup, credentials, and day-to-day
sections are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Security (P0):
- Remove committed session-secret default; auto-generate and persist a
random secret to the config volume when SESSION_SECRET is unset
(prevents forgeable session cookies / auth bypass).
- Validate playlist names to a safe charset and render sldl configs via
literal Python substitution instead of sed (closes a command-injection
and path-traversal path through playlist names).
- shlex-quote credential values written to shell-sourced env files, and
strip newlines from values patched into .conf files.
- Render playlist .conf files 0600; warn at startup if the master key is
co-located with the config volume; document keeping it separate.
Portability:
- Configurable timezone via TZ (default UTC) instead of hardcoded Edmonton.
- Remove personal defaults (navidrome user "andrew", ephemeral.club URLs).
- Ship generic example seeds; move the cross-album dedup keep-list and the
legacy playlist import to editable config files; drop the personal
_upgrade.csv.
- Generic VPN reference in docker-compose.snippet.yml.
First-run experience:
- Redirect to /setup instead of 500 when OIDC is unconfigured; surface a
missing master key inline; entrypoint exits with an actionable message
when the config folder is not writable.
- Add unauthenticated /health (JSON) and /setup (checklist) diagnostics.
Docs:
- Write docs/ARCHITECTURE.md and docs/MIGRATION.md (previously referenced
but missing); expand README with ownership, backups, advanced settings,
and migration guidance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Nav references were stale (Manage credentials/playlists -> Settings >
Credentials, Playlists), the credentials table was missing the
required/optional split and enable-disable toggles, and AzuraCast's
description was wrong (it's not a buy-link source, that's Bandcamp/Qobuz --
AzuraCast's key just triggers an immediate reprocess). Added the playlist
compare view, dashboard public-IP check, and Artist Casing page to the
feature list and day-to-day usage section.
README covers what alembic does, prerequisites, a full deploy quickstart,
credential setup, and a page-by-page usage tour, written for someone
running Docker but not necessarily comfortable coding or sysadmin work.
Favicon/icons were prepared in a separate theme scratch dir but never
wired in; moved them into app/static and pointed base.html at the full
icon set (SVG, PNG fallbacks, apple-touch-icon, manifest) instead of the
single inline data-URI SVG it had before.
andrew