0.5.2: Run now for unscheduled playlists, Spotify /items migration, friendlier 403

- Run now (jobs.run_now): playlists without a daily sync time are never
  registered as scheduler jobs, so the old registered-job check 404'd. Fall
  back to running the playlist directly via pipeline_runner for any
  playlist:<name> key that isn't registered.
- Spotify removed GET /playlists/{id}/tracks in its February 2026 API changes
  in favor of /items (identical response shape). New apps are already 403'd on
  the old endpoint. Migrate both callers (app/services/spotify_client.py and
  pipeline/lib/spotify-retag.py) to /items. (The vendored sldl downloader uses
  its own bundled Spotify library and would need an upstream update when
  /tracks is fully removed.)
- Friendlier on-screen error for a Spotify 403 (check credentials + playlist
  must be public) and a README troubleshooting entry covering the 403, the
  public-playlist requirement, and the org-only extended-quota reality.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -70,15 +70,15 @@ Optional, add these later if you want them:
 
 ### 1. Get the image
 
-Clone this repository onto your Docker host and build the image locally:
+Pull the prebuilt image onto your Docker host:
 
 ```bash
-git clone <this-repo-url> alembic
-cd alembic
-docker build -t alembic:latest .
+docker pull git.kretzer.club/andrew/alembic:0.5.2
 ```
 
-This takes a few minutes the first time. You don't need to touch anything inside the `app/` or `pipeline/` folders, the Dockerfile handles all of it.
+That is the whole install. You do not need to download the source or build anything. The `0.5.2` is the version; you can pin to it so nothing changes under you, or use `latest` to always get the newest.
+
+(If you would rather build it yourself from source, you can, but you do not need to.)
 
 ### 2. Create your folders
 
@@ -131,12 +131,12 @@ In return, it will give you three values you'll need in the next step:
 
 ### 5. Write your docker-compose file
 
-Create a `docker-compose.yml` next to your alembic folder:
+Create a file called `docker-compose.yml` on your server (put it wherever you keep your other compose files):
 
 ```yaml
 services:
   alembic:
-    image: alembic:latest
+    image: git.kretzer.club/andrew/alembic:0.5.2
     container_name: alembic
     ports:
       - "8420:8420"
@@ -220,15 +220,15 @@ That's it. On its next scheduled run (or immediately, using the **Run now** butt
 
 ## Updating alembic
 
-Pull the latest code, rebuild, and restart:
+When a new version is released, change the version in your `docker-compose.yml` (for example `:0.5.2` to the new number), then pull and restart:
 
 ```bash
-cd alembic
-git pull
-docker build -t alembic:latest .
-docker compose up -d --force-recreate alembic
+docker compose pull
+docker compose up -d
 ```
 
+If you used the `latest` tag instead of a version number, you can skip the edit and just run those two commands.
+
 Your library, credentials, and settings all live in your mounted folders, so updating the image never touches your data.
 
 ## Backups
@@ -269,6 +269,12 @@ Double check the redirect URI registered with your OIDC provider exactly matches
 **A playlist says it's synced but nothing downloaded.**
 Check that playlist's job history under **Settings → Jobs**, the log will usually show whether Soulseek couldn't find a track, or a credential is missing. The playlist's own page will also show everything as "Waiting to download" if nothing came through.
 
+**The playlist page shows "Spotify denied access (403)", or nothing downloads and the logs mention 403.**
+Your Spotify app can't read the playlist. Two things to check:
+
+- **The playlist must be public.** alembic reads playlists with app-only (client-credentials) access, which can only see public playlists, never private ones. Set the playlist to public on Spotify.
+- **Your Spotify app must be allowed to read playlists.** Spotify now restricts brand-new developer apps (in "development mode") from reading playlists they don't own, and since May 2025 the "extended quota mode" that lifts this is only granted to organizations, not individuals. If your own app gets 403 on a public playlist, the simplest fix is to use the Spotify Client ID and Secret from an app that already works (for example, one a friend running alembic set up before the change). Client credentials only read public catalog data, so sharing them exposes no account access, just the app's rate limit.
+
 **Dedup found something that isn't actually a duplicate.**
 Use the "Keep both" button on that pair. alembic will remember your decision and won't flag that exact pair again.
 

app/routers/jobs.py

@@ -6,7 +6,7 @@ from sqlalchemy import select
 from app.db import get_db
 from app.models import JobRun, ScheduledJob
 from app.security.deps import require_auth
-from app.services import scheduler_service
+from app.services import pipeline_runner, playlist_service, scheduler_service
 
 router = APIRouter(prefix="/settings/jobs", tags=["jobs"])
 templates = Jinja2Templates(directory="app/templates")
@@ -67,7 +67,9 @@ async def runs_table_partial(request: Request, user: dict = Depends(require_auth
 
 
 @router.post("/{job_key:path}/run")
-async def run_now(job_key: str, background: BackgroundTasks, user: dict = Depends(require_auth)):
+async def run_now(
+    job_key: str, background: BackgroundTasks, user: dict = Depends(require_auth), db=Depends(get_db)
+):
     scheduler = scheduler_service.get_scheduler()
     if scheduler is None:
         raise HTTPException(503, "scheduler not running")
@@ -75,11 +77,24 @@ async def run_now(job_key: str, background: BackgroundTasks, user: dict = Depend
     # background and redirect immediately. A playlist sync can take the better
     # part of an hour; awaiting it here would hang the browser/reverse proxy.
     # Progress shows up in the runs table below, which polls every few seconds.
-    if scheduler.get_job(job_key) is None:
-        raise HTTPException(404, f"no such registered job: {job_key}")
+    if scheduler.get_job(job_key) is not None:
         background.add_task(scheduler_service.trigger_now, scheduler, job_key)
         return RedirectResponse(url="/settings/jobs?started=1", status_code=303)
 
+    # Playlists with no cron_expr (or paused) are never registered with the
+    # scheduler by sync_playlist_jobs -- that's "unscheduled/manual-only" by
+    # design, not missing. Run it directly through pipeline_runner instead of
+    # requiring a scheduler job to exist, so "Run now" works for those too.
+    if job_key.startswith("playlist:"):
+        playlist = playlist_service.get_by_name(db, job_key.removeprefix("playlist:"))
+        if playlist is not None:
+            background.add_task(
+                pipeline_runner.run_playlist, playlist.name, playlist.no_m3u, "manual"
+            )
+            return RedirectResponse(url="/settings/jobs?started=1", status_code=303)
+
+    raise HTTPException(404, f"no such registered job: {job_key}")
+
 
 @router.post("/{job_key:path}/toggle")
 async def toggle_enabled(job_key: str, user: dict = Depends(require_auth), db=Depends(get_db)):

app/routers/playlists.py

@@ -1,3 +1,4 @@
+import httpx
 from fastapi import APIRouter, Depends, Form, HTTPException, Request
 from fastapi.responses import RedirectResponse
 from fastapi.templating import Jinja2Templates
@@ -10,6 +11,19 @@ router = APIRouter(prefix="/playlists", tags=["playlists"])
 templates = Jinja2Templates(directory="app/templates")
 
 
+def _friendly_status_error(exc: Exception) -> str:
+    """Turn a raw Spotify API error into something a non-technical user can act
+    on. 403 here almost always means the Spotify app can't read the playlist:
+    either the credentials are wrong or the playlist isn't public."""
+    if isinstance(exc, httpx.HTTPStatusError) and exc.response.status_code == 403:
+        return (
+            "Spotify denied access (403). Check your Spotify credentials under "
+            "Settings then Credentials, and make sure the playlist is set to public "
+            "on Spotify -- alembic can't read private playlists."
+        )
+    return str(exc)
+
+
 @router.get("")
 async def playlists_index(request: Request, user: dict = Depends(require_auth), db=Depends(get_db)):
     playlists = playlist_service.list_all(db)
@@ -54,7 +68,7 @@ async def playlist_detail(
     try:
         status = status_service.playlist_status(db, playlist.name, playlist.spotify_url)
     except Exception as exc:
-        error = str(exc)
+        error = _friendly_status_error(exc)
 
     return templates.TemplateResponse(
         request,

app/services/dedup_review_service.py

@@ -208,15 +208,33 @@ async def confirm_and_apply(candidate_ids: list[int], confirmed_by: str) -> Dedu
     try:
         candidates = [db.get(DedupCandidate, cid) for cid in candidate_ids]
         candidates = [c for c in candidates if c is not None and not c.applied]
-        candidates = [c for c in candidates if Path(c.delete_path).exists() and Path(c.keep_path).exists()]
         if not candidates:
             return None
 
+        now = time.time()
+
+        # A candidate whose delete target no longer exists (already removed by an
+        # earlier dedup, the mp3->flac upgrade, or by hand) is effectively already
+        # resolved. Mark it applied so it clears from the pending queue, instead
+        # of the old behavior of filtering it out and silently no-oping -- which
+        # left such a stale candidate stuck in the list forever with no way to
+        # delete or clear it.
+        already_gone = [c for c in candidates if not Path(c.delete_path).exists()]
+        for c in already_gone:
+            c.confirmed = True
+            c.confirmed_by = confirmed_by
+            c.confirmed_at = now
+            c.applied = True
+        if already_gone:
+            db.commit()
+
+        # The rest need both files present for a safe delete.
+        actionable = [c for c in candidates if Path(c.delete_path).exists() and Path(c.keep_path).exists()]
+
         by_script: dict[str, list[DedupCandidate]] = {}
-        for c in candidates:
+        for c in actionable:
             by_script.setdefault(_script_for_pass(c.pass_name), []).append(c)
 
-        now = time.time()
         finished_at = now
         log_paths = []
         ran_candidates: list[DedupCandidate] = []
@@ -246,7 +264,8 @@ async def confirm_and_apply(candidate_ids: list[int], confirmed_by: str) -> Dedu
             ran_candidates.extend(group)
         db.commit()
 
-        if not ran_candidates:
+        # Nothing to report: no stale ones cleared and nothing ran.
+        if not already_gone and not ran_candidates:
             return None
 
         still_there = {c.delete_path for c in ran_candidates if Path(c.delete_path).exists()}
@@ -261,7 +280,7 @@ async def confirm_and_apply(candidate_ids: list[int], confirmed_by: str) -> Dedu
             started_at=now,
             finished_at=finished_at,
             mode="apply",
-            deleted=actually_deleted,
+            deleted=actually_deleted + len(already_gone),
             kept=len(ran_candidates) - actually_deleted,
             log_path=";".join(log_paths) or None,
         )

app/services/scheduler_service.py

@@ -130,17 +130,19 @@ MAINTENANCE_JOBS: dict[str, tuple[dict, callable]] = {
         dict(minute=5, hour=9),
         _lib("maintenance:export_laptop_playlists", "export-laptop-playlists.py"),
     ),
-    "maintenance:enrich_buy_url": (
-        dict(minute=10, hour=9),
-        # 30-min cap: this hits external buy-link APIs per track and has hung
-        # holding the pipeline lock (2026-07-09/10), starving every job after
-        # it. A timeout releases the lock so the rest of the 9am chain runs.
-        _lib("maintenance:enrich_buy_url", "enrich-buy-url.py", ["--apply"], timeout=1800),
-    ),
     "maintenance:build_fingerprint_index": (
         dict(minute=25, hour=9),
         _lib("maintenance:build_fingerprint_index", "build-fingerprint-index.py", ["--workers", "8"], timeout=3600),
     ),
+    "maintenance:enrich_buy_url": (
+        # Runs LAST in the 9am block: it hits external buy-link APIs per track
+        # and can run long, so scheduling it after the fingerprint index (09:25)
+        # and the status report (09:30) keeps it from starving them via the
+        # shared lock. The 30-min cap is a backstop; enrich-buy-url.py also now
+        # caches "tried, no match" so it stops re-querying the whole backlog.
+        dict(minute=45, hour=9),
+        _lib("maintenance:enrich_buy_url", "enrich-buy-url.py", ["--apply"], timeout=1800),
+    ),
     "maintenance:pipeline_status_report": (
         dict(minute=30, hour=9),
         # Read-only (reads logs, pings Navidrome, sends Telegram). Runs WITHOUT

app/services/spotify_client.py

@@ -57,7 +57,11 @@ def get_playlist_tracks(db: Session, playlist_url: str) -> list[dict]:
     headers = {"Authorization": f"Bearer {token}"}
 
     tracks = []
-    url = f"{API_BASE}/playlists/{playlist_id}/tracks"
+    # /items, not /tracks: Spotify removed GET /playlists/{id}/tracks in its
+    # February 2026 API changes in favor of /items (same response shape). New
+    # apps are 403'd on the old endpoint; grandfathered apps still tolerate it
+    # for now, but /items is the correct, future-proof one.
+    url = f"{API_BASE}/playlists/{playlist_id}/items"
     params = {"limit": 100, "fields": "items(track(name,artists(name),external_ids)),next"}
 
     while url:

pipeline/lib/enrich-buy-url.py

@@ -40,6 +40,18 @@ from pathlib import Path
 
 UA = "Mozilla/5.0 (X11; Linux x86_64) enrich-buy-url/1.0"
 BUY_URL_TAG = "COMMERCIAL_INFORMATION"
+# Marker (unix timestamp) stamped on a FLAC when a buy-link lookup found
+# nothing, so daily runs don't re-query the whole no-match backlog every time.
+# Retried after the window below; --force ignores it.
+BUY_URL_TRIED_TAG = "BUY_URL_TRIED"
+_TRIED_RETRY_DAYS = 30
+
+
+def _tried_recently(ts_str):
+    try:
+        return (time.time() - float(ts_str)) < _TRIED_RETRY_DAYS * 86400
+    except (TypeError, ValueError):
+        return False
 ITUNES_SEARCH = "https://itunes.apple.com/search"
 BC_AUTOCOMPLETE = "https://bandcamp.com/api/bcsearch_public_api/1/autocomplete_elastic"
 QOBUZ_SEARCH = "https://www.qobuz.com/api.json/0.2/track/search"
@@ -451,6 +463,14 @@ def main():
             else:
                 continue                      # already tagged, not up for upgrade
         else:
+            # Skip tracks we recently tried and found nothing for, so a daily
+            # run doesn't re-query the whole no-match backlog (external, rate-
+            # limited lookups) every time -- that churn is what pushed this job
+            # past its timeout. --force re-checks everything.
+            if not args.force:
+                tried = flac_tag(sp, BUY_URL_TRIED_TAG)
+                if tried and _tried_recently(tried):
+                    continue
             run_sources = cascade             # untagged, or --force
         artist = flac_tag(sp, "ARTIST").split(";")[0].strip() or flac_tag(sp, "ALBUMARTIST")
         title = flac_tag(sp, "TITLE")
@@ -470,6 +490,10 @@ def main():
 
         rel = sp[len(args.library):].lstrip("/")
         if not url:
+            # Record that we checked and found nothing, so the next daily run
+            # skips it (until the retry window). Apply mode only.
+            if args.apply:
+                set_flac_tag(sp, BUY_URL_TRIED_TAG, str(int(time.time())))
             continue
         found += 1
         by_source[src] += 1

pipeline/lib/pipeline-status.sh

@@ -315,8 +315,13 @@ dedup_today_status() {
   # silently drops Qobuz and falls through to iTunes.
   if [[ -f ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/token ]]; then
     q_token=$(tr -d '\r\n' < ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/token)
-    q_appid=$(tr -d '\r\n' < ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/app_id 2>/dev/null)
-    q_appid=${q_appid:-798273057}
+    # Guard the app_id read with a file-exists check: `< missing 2>/dev/null`
+    # still leaks the shell's own "No such file" redirection error (the redirect
+    # is opened before 2>/dev/null takes effect). Most setups have a token but
+    # no separate app_id, so this fired on every run.
+    q_appid=798273057
+    q_appid_file="${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/app_id"
+    [[ -f "$q_appid_file" ]] && q_appid=$(tr -d '\r\n' < "$q_appid_file")
     q_code=$(curl -s -o /dev/null --connect-timeout 5 --max-time 12 -A "Mozilla/5.0" \
       -H "X-App-Id: $q_appid" -H "X-User-Auth-Token: $q_token" -w '%{http_code}' \
       "https://www.qobuz.com/api.json/0.2/track/search?query=test&limit=1&app_id=$q_appid" 2>/dev/null)

pipeline/lib/spotify-retag.py

@@ -46,7 +46,8 @@ def fetch_playlist(url: str, token: str) -> list[dict]:
         sys.exit(f"Could not parse playlist ID from {url!r}")
     pid = m.group(1)
     tracks = []
-    next_url = f"{API}/playlists/{pid}/tracks?limit=100"
+    # /items (not the removed /tracks) -- see February 2026 Spotify API changes.
+    next_url = f"{API}/playlists/{pid}/items?limit=100"
     while next_url:
         req = urllib.request.Request(
             next_url, headers={"Authorization": f"Bearer {token}"}

tests/test_dedup_review.py

@@ -0,0 +1,57 @@
+"""confirm_and_apply: a candidate whose delete target is already gone must be
+marked applied (cleared from the queue), not silently no-op'd and left stuck.
+This is the 'can't delete this one' case that stranded a stale fuzzy candidate."""
+import asyncio
+import time
+
+from app.db import SessionLocal
+from app.models import DedupCandidate, DedupRun
+from app.services import dedup_review_service as d
+
+
+def _make_candidate(keep_path: str, delete_path: str) -> int:
+    db = SessionLocal()
+    try:
+        run = DedupRun(started_at=time.time(), mode="dry_run")
+        db.add(run)
+        db.commit()
+        db.refresh(run)
+        c = DedupCandidate(
+            dedup_run_id=run.id, pass_name="fuzzy_audio",
+            keep_path=keep_path, delete_path=delete_path, delete_id=None,
+        )
+        db.add(c)
+        db.commit()
+        db.refresh(c)
+        return c.id
+    finally:
+        db.close()
+
+
+def test_already_gone_delete_path_is_resolved(tmp_path):
+    keep = tmp_path / "keep.flac"
+    keep.write_bytes(b"x")               # keep still exists
+    gone = tmp_path / "gone.mp3"          # delete target already removed
+    cid = _make_candidate(str(keep), str(gone))
+
+    # No actionable file to delete -> no subprocess/lock; the async call resolves
+    # the stale candidate directly.
+    result = asyncio.run(d.confirm_and_apply([cid], "tester"))
+
+    assert result is not None            # a run row was produced (not a silent no-op)
+    db = SessionLocal()
+    try:
+        c = db.get(DedupCandidate, cid)
+        assert c.applied is True         # cleared from the pending queue
+    finally:
+        db.close()
+    # and it no longer appears in the pending list
+    assert cid not in [c.id for c in d.list_pending_candidates()]
+
+
+def test_ignore_then_unignore(tmp_path):
+    cid = _make_candidate(str(tmp_path / "a.flac"), str(tmp_path / "b.mp3"))
+    assert d.ignore_candidate(cid, "tester").ignored is True
+    assert cid not in [c.id for c in d.list_pending_candidates()]
+    assert d.unignore_candidate(cid).ignored is False
+    assert cid in [c.id for c in d.list_pending_candidates()]