Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8ecff44811 | |||
| 2e3210cc01 | |||
| 3fbf580b88 |
@@ -73,10 +73,10 @@ Optional, add these later if you want them:
|
||||
Pull the prebuilt image onto your Docker host:
|
||||
|
||||
```bash
|
||||
docker pull git.kretzer.club/andrew/alembic:0.6
|
||||
docker pull git.kretzer.club/andrew/alembic:0.6.3
|
||||
```
|
||||
|
||||
That is the whole install. You do not need to download the source or build anything. The `0.6` is the version; you can pin to it so nothing changes under you, or use `latest` to always get the newest.
|
||||
That is the whole install. You do not need to download the source or build anything. The `0.6.3` is the version; you can pin to it so nothing changes under you, or use `latest` to always get the newest.
|
||||
|
||||
(If you would rather build it yourself from source, you can, but you do not need to.)
|
||||
|
||||
@@ -136,7 +136,7 @@ Create a file called `docker-compose.yml` on your server (put it wherever you ke
|
||||
```yaml
|
||||
services:
|
||||
alembic:
|
||||
image: git.kretzer.club/andrew/alembic:0.6
|
||||
image: git.kretzer.club/andrew/alembic:0.6.3
|
||||
container_name: alembic
|
||||
ports:
|
||||
- "8420:8420"
|
||||
@@ -211,9 +211,11 @@ Steps:
|
||||
|
||||
That's it, done once. alembic stores the resulting OAuth refresh token (encrypted, same as your other credentials) and uses it to silently mint a fresh access token whenever it needs one — you never have to repeat this unless you disconnect or revoke access on Spotify's side.
|
||||
|
||||
One more limit that comes with a new Spotify app: Spotify only returns a playlist's contents to the account that owns it (or collaborates on it), even after you connect. So sync playlists that belong to the account you connect, and if someone shares a playlist with you, save your own copy of it first (or have them add you as a collaborator).
|
||||
|
||||
If you happen to be running a Spotify app created *before* the 2025/2026 change, plain Client ID/Secret still works and this step is optional — but connecting is harmless either way, so there's no reason not to.
|
||||
|
||||
> **If you connected Spotify before version 0.6.1:** click **Connect Spotify** again. Versions before 0.6.1 requested a narrower OAuth scope than sldl actually needs, which could get you a token that refreshes fine but then gets 403 Forbidden loading a playlist (see Troubleshooting). Reconnecting grants the correct scope; your old connection doesn't upgrade itself.
|
||||
> **If you connected Spotify before version 0.6.1:** click **Connect Spotify** again. Versions before 0.6.1 requested a narrower OAuth scope than playlist reads actually need, which could get you a token that refreshes fine but then gets 403 Forbidden loading a playlist (see Troubleshooting). Reconnecting grants the correct scope; your old connection doesn't upgrade itself.
|
||||
|
||||
## Adding your first playlist
|
||||
|
||||
@@ -290,11 +292,13 @@ Your Spotify app can't read the playlist with app-only (client-credentials) acce
|
||||
|
||||
The fix: go to **Settings → Credentials** and click **Connect Spotify** to complete the OAuth login (see "Connecting your Spotify account (OAuth)" above). This mints a user token that works regardless of when your app was created. If you haven't already, add the redirect URI `https://<your-host>/connect/spotify/callback` in your Spotify app's dashboard first, or the connect step itself will fail with a redirect_uri mismatch.
|
||||
|
||||
**A playlist's job log shows `sldl crashed (exit 134)` mentioning a 403 Forbidden, even though you've connected Spotify.**
|
||||
Two possible causes, in order of likelihood:
|
||||
**A playlist's job log shows `Spotify returned 403 loading this playlist`, `Spotify returned this playlist without its contents`, or `0 visible tracks` even though you've connected Spotify.** (On versions before 0.6.2 these showed up as `sldl crashed (exit 134)` mentioning a 403 Forbidden; on 0.6.2 a playlist you don't own could finish "successfully" while downloading nothing.)
|
||||
Possible causes, in order of likelihood:
|
||||
|
||||
1. **You connected before version 0.6.1.** Earlier versions requested a narrower OAuth scope than sldl actually needs, so the token refreshes fine but then gets 403 loading a playlist regardless of ownership. Fix: go to **Settings → Credentials** and click **Connect Spotify** again to grant the correct scope.
|
||||
2. **The connected account genuinely can't see this playlist.** Once connected, playlist reads are scoped to what that account can actually see — its own playlists, ones it collaborates on, or ones marked Public — not any arbitrary playlist by ID the way app-only access could. Fix: make sure the playlist is owned by the connected account, or set it to Public on Spotify (Playlist → ⋯ → Make public).
|
||||
1. **The playlist isn't owned by the connected account.** For Spotify apps created after the 2025/2026 API change, Spotify only returns a playlist's contents to its owner or a collaborator on it. Being public is NOT enough. Fix: recreate the playlist under the account you connected via **Connect Spotify**, or have the owner make it collaborative and add you.
|
||||
2. **You connected before version 0.6.1.** Earlier versions requested a narrower OAuth scope than playlist reads actually need, so the token refreshes fine but then gets 403 loading a playlist regardless of ownership. Fix: go to **Settings → Credentials** and click **Connect Spotify** again to grant the correct scope.
|
||||
|
||||
(If your Spotify app is old enough to be grandfathered, both public playlists and your own keep working like before.)
|
||||
|
||||
Either way, re-run the playlist after fixing it.
|
||||
|
||||
|
||||
+20
-1
@@ -8,7 +8,7 @@ from fastapi.templating import Jinja2Templates
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
from starlette.middleware.sessions import SessionMiddleware
|
||||
|
||||
from app.db import enable_beets_db_wal, init_db, mark_interrupted_runs
|
||||
from app.db import SessionLocal, enable_beets_db_wal, init_db, mark_interrupted_runs
|
||||
from app.routers import artist_casing, auth, connect, credentials, dashboard, dedup, genres, health, import_, jobs, library, playlists
|
||||
from app.security.session import resolve_session_secret
|
||||
from app.services import scheduler_service
|
||||
@@ -48,12 +48,31 @@ def _warn_if_key_colocated_with_config() -> None:
|
||||
)
|
||||
|
||||
|
||||
def _render_confs_on_startup() -> None:
|
||||
"""Re-render every playlist .conf from the current template once per boot,
|
||||
so confs rendered by an older version converge on upgrade without waiting
|
||||
for a playlist or credential change. Concretely: 0.6.2 dropped the Spotify
|
||||
credential lines from confs (sldl no longer talks to Spotify), and this is
|
||||
what scrubs those secrets from already-deployed confs. Best-effort -- a
|
||||
render failure shouldn't stop the app from starting."""
|
||||
from app.services import credential_service
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
credential_service.render_playlist_confs(db)
|
||||
except Exception:
|
||||
log.exception("Startup playlist .conf render failed; continuing")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
_warn_if_key_colocated_with_config()
|
||||
init_db()
|
||||
mark_interrupted_runs()
|
||||
enable_beets_db_wal()
|
||||
_render_confs_on_startup()
|
||||
|
||||
scheduler = scheduler_service.create_scheduler()
|
||||
scheduler_service.register_all_jobs(scheduler)
|
||||
|
||||
@@ -14,12 +14,16 @@ templates = Jinja2Templates(directory="app/templates")
|
||||
def _friendly_status_error(exc: Exception) -> str:
|
||||
"""Turn a raw Spotify API error into something a non-technical user can act
|
||||
on. 403 here almost always means the Spotify app can't read the playlist:
|
||||
either the credentials are wrong or the playlist isn't public."""
|
||||
wrong credentials, or the connected account can't see it. (The other
|
||||
can't-read case, contents hidden for playlists the account doesn't own,
|
||||
arrives as a RuntimeError from spotify_client with its own message.)"""
|
||||
if isinstance(exc, httpx.HTTPStatusError) and exc.response.status_code == 403:
|
||||
return (
|
||||
"Spotify denied access (403). Check your Spotify credentials under "
|
||||
"Settings then Credentials, and make sure the playlist is set to public "
|
||||
"on Spotify -- alembic can't read private playlists."
|
||||
"Settings then Credentials, and make sure the account connected via "
|
||||
"Connect Spotify can see this playlist. For newly created Spotify "
|
||||
"apps the connected account must own the playlist or be a "
|
||||
"collaborator on it."
|
||||
)
|
||||
return str(exc)
|
||||
|
||||
|
||||
@@ -116,8 +116,10 @@ def get_scope(db: Session, scope: str) -> dict[str, str]:
|
||||
_SAFE_CONF_NAME = re.compile(r"^[A-Za-z0-9 _-]{1,64}$")
|
||||
|
||||
# The credential lines the renderer fills in from the encrypted store. Everything
|
||||
# else in a rendered .conf comes verbatim from _template.conf.
|
||||
_CONF_CRED_KEYS = ("user", "pass", "spotify-id", "spotify-secret", "spotify-refresh")
|
||||
# else in a rendered .conf comes verbatim from _template.conf. Spotify creds are
|
||||
# NOT here: sldl never talks to Spotify itself (see _template.conf and
|
||||
# run-playlist.sh) -- the scripts that do read them from _spotify.env instead.
|
||||
_CONF_CRED_KEYS = ("user", "pass")
|
||||
|
||||
|
||||
def _set_conf_field(text: str, key: str, value: str) -> str:
|
||||
@@ -134,27 +136,24 @@ def _set_conf_field(text: str, key: str, value: str) -> str:
|
||||
|
||||
def render_playlist_confs(db: Session) -> None:
|
||||
"""Render every playlist's sldl .conf directly from _template.conf, with
|
||||
the path placeholders and the Soulseek/Spotify credentials substituted in
|
||||
one pass, written 0600.
|
||||
the path placeholders and the Soulseek credentials substituted in one
|
||||
pass, written 0600. No Spotify credentials here -- sldl never talks to
|
||||
Spotify itself (see _template.conf); those live only in _spotify.env.
|
||||
|
||||
This is the single source of .conf rendering. It replaces the older
|
||||
DB -> playlists.json -> regen.sh -> regex-patch-back chain: the app owns
|
||||
every input (playlists in its DB, credentials in its encrypted store), so
|
||||
there is no reason to round-trip through an intermediate file and a
|
||||
subprocess. Called on any playlist change (playlist_service) and on any
|
||||
Spotify/Soulseek credential change (render_scope)."""
|
||||
Soulseek credential change (render_scope)."""
|
||||
from app.services import playlist_service
|
||||
|
||||
template = (settings.pipeline_dir / "configs" / "_template.conf").read_text()
|
||||
dropbox_root = str(settings.music_data_dir / "sldl-dropbox")
|
||||
spotify = get_scope(db, "spotify")
|
||||
soulseek = get_scope(db, "soulseek")
|
||||
creds = {
|
||||
"user": soulseek.get("username", ""),
|
||||
"pass": soulseek.get("password", ""),
|
||||
"spotify-id": spotify.get("client_id", ""),
|
||||
"spotify-secret": spotify.get("client_secret", ""),
|
||||
"spotify-refresh": spotify.get("refresh_token", ""),
|
||||
}
|
||||
# Path placeholders are substituted as LITERAL text in a single left-to-right
|
||||
# pass (never re-scanned), so a value can't be reinterpreted as another
|
||||
@@ -193,13 +192,14 @@ def render_scope(db: Session, scope: str) -> None:
|
||||
if scope == "spotify":
|
||||
cid = values.get("client_id", "")
|
||||
csec = values.get("client_secret", "")
|
||||
# _spotify.env is read by the pipeline python scripts (spotify-genre,
|
||||
# spotify-retag, fix-track-metadata). SPOTIFY_REFRESH_TOKEN is only set
|
||||
# once an account is connected via /connect/spotify -- its presence is
|
||||
# what switches every reader from client-credentials to a user token
|
||||
# (see _spotify_auth.get_token). The rendered playlist .conf gets the
|
||||
# same refresh token too (spotify-refresh, in render_playlist_confs),
|
||||
# which is what lets sldl itself use a user token.
|
||||
# _spotify.env is the ONLY place Spotify creds are rendered -- sldl
|
||||
# itself never talks to Spotify (see _template.conf), so playlist
|
||||
# .confs don't need them. Read by run-playlist.sh (to generate the
|
||||
# search CSV) and the pipeline python scripts (spotify-retag,
|
||||
# spotify-genre, fix-track-metadata). SPOTIFY_REFRESH_TOKEN is only
|
||||
# set once an account is connected via /connect/spotify -- its
|
||||
# presence is what switches every reader from client-credentials to
|
||||
# a user token (see _spotify_auth.get_token).
|
||||
_write_env_file(
|
||||
settings.pipeline_config_dir / "_spotify.env",
|
||||
{
|
||||
@@ -208,8 +208,6 @@ def render_scope(db: Session, scope: str) -> None:
|
||||
"SPOTIFY_REFRESH_TOKEN": values.get("refresh_token", ""),
|
||||
},
|
||||
)
|
||||
# Re-render every playlist .conf so the new Spotify creds land in them.
|
||||
render_playlist_confs(db)
|
||||
|
||||
elif scope == "soulseek":
|
||||
render_playlist_confs(db)
|
||||
|
||||
@@ -115,7 +115,8 @@ def update(db: Session, playlist_id: int, **fields) -> Playlist:
|
||||
|
||||
|
||||
def delete(db: Session, playlist_id: int) -> None:
|
||||
"""Remove the playlist from the DB and delete its rendered .conf file.
|
||||
"""Remove the playlist from the DB and delete its rendered .conf and
|
||||
generated search .csv files.
|
||||
Does NOT touch anything already downloaded/imported for it — that's a
|
||||
library decision, not a playlist-definition one."""
|
||||
playlist = db.get(Playlist, playlist_id)
|
||||
@@ -126,6 +127,9 @@ def delete(db: Session, playlist_id: int) -> None:
|
||||
db.commit()
|
||||
conf_path = settings.pipeline_config_dir / f"{name}.conf"
|
||||
conf_path.unlink(missing_ok=True)
|
||||
# The search CSV run-playlist.sh generates next to the conf each run.
|
||||
csv_path = settings.pipeline_config_dir / f"{name}.csv"
|
||||
csv_path.unlink(missing_ok=True)
|
||||
|
||||
from app.services import scheduler_service
|
||||
|
||||
|
||||
@@ -77,18 +77,34 @@ def get_playlist_tracks(db: Session, playlist_url: str) -> list[dict]:
|
||||
|
||||
tracks = []
|
||||
# /items, not /tracks: Spotify removed GET /playlists/{id}/tracks in its
|
||||
# February 2026 API changes in favor of /items (same response shape). New
|
||||
# apps are 403'd on the old endpoint; grandfathered apps still tolerate it
|
||||
# for now, but /items is the correct, future-proof one.
|
||||
# February 2026 API changes. The response shape ALSO changed, but only for
|
||||
# apps on the new behavior: each entry's payload moved from "track" to
|
||||
# "item" (tracks.tracks.track -> items.items.item). Extended Quota Mode
|
||||
# (grandfathered) apps keep the old "track" key, so parse both. No
|
||||
# `fields` filter: it selects by key name, so on the renamed shape a
|
||||
# track(...) filter silently returns empty pages -- exactly the failure
|
||||
# we're avoiding.
|
||||
url = f"{API_BASE}/playlists/{playlist_id}/items"
|
||||
params = {"limit": 100, "fields": "items(track(name,artists(name),external_ids)),next"}
|
||||
params = {"limit": 100}
|
||||
|
||||
while url:
|
||||
resp = httpx.get(url, params=params, headers=headers, timeout=15)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
for item in data.get("items", []):
|
||||
track = item.get("track")
|
||||
if "items" not in data:
|
||||
# New-behavior apps get metadata only (no items field at all) for
|
||||
# playlists the connected account doesn't own or collaborate on --
|
||||
# public is no longer sufficient. Same message style the playlist
|
||||
# page shows for a 403.
|
||||
raise RuntimeError(
|
||||
"Spotify returned this playlist without its contents. For newly "
|
||||
"created Spotify apps, the account connected via Connect Spotify "
|
||||
"must own the playlist (or be a collaborator on it) -- ask the "
|
||||
"owner to share it as collaborative, or recreate it under the "
|
||||
"connected account."
|
||||
)
|
||||
for item in data["items"]:
|
||||
track = item.get("track") or item.get("item")
|
||||
if not track:
|
||||
continue # local files / removed tracks show up as null
|
||||
artists = track.get("artists") or []
|
||||
|
||||
@@ -25,6 +25,8 @@ PLAYLIST_NAME="${1:?Usage: $0 [--no-m3u] <playlist_name>}"
|
||||
|
||||
# ==== Paths ====
|
||||
CONFIG_FILE=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/${PLAYLIST_NAME}.conf
|
||||
SPOTIFY_ENV=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/_spotify.env
|
||||
CSV_FILE=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/${PLAYLIST_NAME}.csv
|
||||
DROPBOX=${MUSIC_DATA_DIR:-/data/music}/sldl-dropbox/${PLAYLIST_NAME}
|
||||
PLAYLISTS_DIR=${MUSIC_DATA_DIR:-/data/music}/playlists
|
||||
QUARANTINE_DIR=${MUSIC_DATA_DIR:-/data/music}/Songs/untagged
|
||||
@@ -54,6 +56,46 @@ if [[ ! -f "$CONFIG_FILE" ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ==== Read the playlist from Spotify into a CSV sldl can search from ====
|
||||
# sldl's own vendored Spotify client still calls GET /playlists/{id}/tracks,
|
||||
# which Spotify removed in its February 2026 API changes. Grandfathered apps
|
||||
# still get a pass there; apps created after that change get a hard 403
|
||||
# regardless of auth method (client-credentials, or even a correctly-scoped,
|
||||
# correctly-owned OAuth user token -- confirmed live, exit 134 unhandled
|
||||
# APIException: Forbidden from Extractors.Spotify.GetPlaylist). sldl has no
|
||||
# fallback to the still-working /items endpoint, so it can never read a
|
||||
# playlist for a new app no matter what alembic hands it. Reading the
|
||||
# playlist ourselves (via /items) and handing sldl a plain CSV instead
|
||||
# sidesteps sldl's Spotify client entirely -- works the same for
|
||||
# grandfathered and new apps alike.
|
||||
SPOTIFY_URL=$(sed -n 's/^input *= *//p' "$CONFIG_FILE" | tr -d ' ')
|
||||
[[ -f "$SPOTIFY_ENV" ]] && source "$SPOTIFY_ENV"
|
||||
|
||||
if [[ -z "$SPOTIFY_URL" || -z "${SPOTIFY_CLIENT_ID:-}" || -z "${SPOTIFY_CLIENT_SECRET:-}" ]]; then
|
||||
log "ERROR: missing playlist URL in $CONFIG_FILE or Spotify credentials in $SPOTIFY_ENV"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log "Fetching playlist from Spotify: $SPOTIFY_URL"
|
||||
if ! SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
|
||||
SPOTIFY_REFRESH_TOKEN="${SPOTIFY_REFRESH_TOKEN:-}" \
|
||||
python3 "${PIPELINE_DIR:-/app/pipeline}/lib/spotify-playlist-csv.py" "$SPOTIFY_URL" "$CSV_FILE" >> "$LOG" 2>&1; then
|
||||
log "ERROR: failed to fetch playlist from Spotify — see $LOG"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Surface the fetched count in the timestamped summary. Zero tracks from a
|
||||
# successful fetch is almost never a truly empty playlist -- it usually means
|
||||
# Spotify hid the contents from the connected account (see the pointed errors
|
||||
# in spotify-playlist-csv.py), so call it out instead of letting sldl no-op
|
||||
# with a clean exit 0.
|
||||
TRACK_COUNT=$(($(wc -l < "$CSV_FILE") - 1))
|
||||
if [[ "$TRACK_COUNT" -le 0 ]]; then
|
||||
log "WARNING: Spotify returned 0 visible tracks for this playlist -- nothing to download. If the playlist isn't actually empty, the connected Spotify account can't see its contents (it must own or collaborate on the playlist for newly created Spotify apps)."
|
||||
else
|
||||
log "Fetched $TRACK_COUNT track(s) from Spotify"
|
||||
fi
|
||||
|
||||
# ==== Run sldl (vendored binary, subprocess of this container) ====
|
||||
log "Running sldl for: $PLAYLIST_NAME"
|
||||
|
||||
@@ -62,37 +104,24 @@ log "Running sldl for: $PLAYLIST_NAME"
|
||||
# download. A non-zero exit here is logged but not fatal.
|
||||
#
|
||||
# Hard timeout: without it a stuck sldl hangs FOREVER holding the shared lock,
|
||||
# which silently skips every later-scheduled playlist for the night. (Seen
|
||||
# 2026-06-18: a transient Spotify client-creds failure made sldl fall back to
|
||||
# interactive OAuth — "manually open: https://accounts.spotify.com/..." — and
|
||||
# it waited 7h on a browser callback that never comes. Fixed by always
|
||||
# rendering `spotify-refresh` into the conf once a Spotify account is
|
||||
# connected via /connect/spotify — sldl's own docs confirm supplying a
|
||||
# refresh token skips the login-flow requirement entirely. This timeout stays
|
||||
# as a backstop for any other cause of a stuck run.) timeout TERMs the run
|
||||
# and KILLs after a grace period; there's no leftover container to clean up
|
||||
# now that sldl is a plain subprocess instead of a docker-compose service.
|
||||
# which silently skips every later-scheduled playlist for the night. timeout
|
||||
# TERMs the run and KILLs after a grace period; there's no leftover container
|
||||
# to clean up now that sldl is a plain subprocess instead of a docker-compose
|
||||
# service.
|
||||
#
|
||||
# -c "$CONFIG_FILE" still supplies Soulseek login, download path, quality
|
||||
# settings, port, etc. -- the CSV positional arg + --input-type csv override
|
||||
# just the input source (confirmed: sldl ignores the conf's own `input =`/
|
||||
# `input-type =` lines when both are given).
|
||||
SLDL_TIMEOUT="${SLDL_TIMEOUT:-2700}" # 45 min; override via env
|
||||
SLDL_EXIT=0
|
||||
timeout --kill-after=30s "$SLDL_TIMEOUT" \
|
||||
"$SLDL_BIN" -c "$CONFIG_FILE" >> "$LOG" 2>&1 \
|
||||
"$SLDL_BIN" "$CSV_FILE" --input-type csv -c "$CONFIG_FILE" >> "$LOG" 2>&1 \
|
||||
|| SLDL_EXIT=$?
|
||||
if [[ "$SLDL_EXIT" -eq 124 || "$SLDL_EXIT" -eq 137 ]]; then
|
||||
log "ERROR: sldl exceeded ${SLDL_TIMEOUT}s and was killed (likely a hang)"
|
||||
elif [[ "$SLDL_EXIT" -eq 134 ]]; then
|
||||
# sldl aborts (SIGABRT) on ANY unhandled .NET exception rather than failing
|
||||
# cleanly. The one case we've actually seen in the wild: a valid, freshly-
|
||||
# refreshed user token still gets 403 Forbidden loading the playlist,
|
||||
# because Spotify scopes OAuth playlist reads to what the CONNECTED account
|
||||
# can see (its own playlists, ones it collaborates on, or ones marked
|
||||
# Public) -- unlike client-credentials, which could read any public
|
||||
# playlist regardless of whose app it was. Give a pointed hint when that's
|
||||
# the visible cause; otherwise just flag that sldl crashed outright.
|
||||
if grep -q "APIException: Forbidden" "$LOG"; then
|
||||
log "ERROR: sldl crashed (exit 134) -- Spotify returned 403 loading this playlist. The connected Spotify account can't see it: make sure it's owned by, or shared/followed by, whichever account is connected via /connect/spotify, or set it to Public on Spotify."
|
||||
else
|
||||
log "ERROR: sldl crashed (exit 134, unhandled exception) -- see the log above for the exception detail"
|
||||
fi
|
||||
log "ERROR: sldl crashed (exit 134, unhandled exception) -- see the log above for the exception detail"
|
||||
fi
|
||||
log "sldl finished with exit code $SLDL_EXIT"
|
||||
|
||||
@@ -125,27 +154,19 @@ log "Cleaning up .incomplete files in dropbox"
|
||||
find "$DROPBOX" -name "*.incomplete" -type f -delete 2>>"$LOG" || true
|
||||
|
||||
# ==== Rewrite tags from Spotify (source of truth for matched tracks) ====
|
||||
# Reads Spotify creds + playlist URL from the same conf sldl used. For each file
|
||||
# in the dropbox that matches a Spotify playlist track, overwrites ARTIST
|
||||
# Uses the playlist URL and _spotify.env credentials already loaded for the
|
||||
# CSV fetch above (confs no longer carry Spotify creds). For each file in the
|
||||
# dropbox that matches a Spotify playlist track, overwrites ARTIST
|
||||
# (semicolon-joined), ALBUMARTIST (primary), ALBUM, TITLE, TRACKNUMBER,
|
||||
# DISCNUMBER, DATE. GROUPING is preserved. Files that don't match are left for
|
||||
# the fallback step below.
|
||||
SPOTIFY_URL=$(sed -n 's/^input *= *//p' "$CONFIG_FILE" | tr -d ' ')
|
||||
SPOTIFY_CLIENT_ID=$(sed -n 's/^spotify-id *= *//p' "$CONFIG_FILE" | tr -d ' ')
|
||||
SPOTIFY_CLIENT_SECRET=$(sed -n 's/^spotify-secret *= *//p' "$CONFIG_FILE" | tr -d ' ')
|
||||
SPOTIFY_REFRESH_TOKEN=$(sed -n 's/^spotify-refresh *= *//p' "$CONFIG_FILE" | tr -d ' ')
|
||||
|
||||
if [[ -n "$SPOTIFY_URL" && -n "$SPOTIFY_CLIENT_ID" && -n "$SPOTIFY_CLIENT_SECRET" ]]; then
|
||||
log "Rewriting tags from Spotify for files in $DROPBOX"
|
||||
if SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
|
||||
SPOTIFY_REFRESH_TOKEN="$SPOTIFY_REFRESH_TOKEN" \
|
||||
python3 ${PIPELINE_DIR:-/app/pipeline}/lib/spotify-retag.py "$SPOTIFY_URL" "$DROPBOX" >> "$LOG" 2>&1; then
|
||||
log "Spotify retag complete"
|
||||
else
|
||||
log "WARNING: Spotify retag failed (exit $?) — continuing with sldl-supplied tags"
|
||||
fi
|
||||
log "Rewriting tags from Spotify for files in $DROPBOX"
|
||||
if SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
|
||||
SPOTIFY_REFRESH_TOKEN="${SPOTIFY_REFRESH_TOKEN:-}" \
|
||||
python3 ${PIPELINE_DIR:-/app/pipeline}/lib/spotify-retag.py "$SPOTIFY_URL" "$DROPBOX" >> "$LOG" 2>&1; then
|
||||
log "Spotify retag complete"
|
||||
else
|
||||
log "Skipping Spotify retag — missing input/spotify-id/spotify-secret in $CONFIG_FILE"
|
||||
log "WARNING: Spotify retag failed (exit $?) — continuing with sldl-supplied tags"
|
||||
fi
|
||||
|
||||
# ==== Quarantine any files still missing essential tags ====
|
||||
|
||||
@@ -14,19 +14,15 @@
|
||||
user = SOULSEEK_USER
|
||||
pass = SOULSEEK_PASS
|
||||
|
||||
# ==== Spotify API credentials ====
|
||||
spotify-id = SPOTIFY_CLIENT_ID
|
||||
spotify-secret = SPOTIFY_CLIENT_SECRET
|
||||
# Set once an account is connected via /connect/spotify (blank otherwise).
|
||||
# sldl's own docs confirm supplying a refresh token skips its interactive
|
||||
# login-flow requirement entirely -- this is what lets newly created Spotify
|
||||
# apps (which Spotify blocks from reading playlists with client-credentials
|
||||
# alone) work without sldl ever trying to open a browser.
|
||||
spotify-refresh = SPOTIFY_REFRESH_TOKEN
|
||||
|
||||
# ==== Input (Spotify playlist URL) ====
|
||||
# sldl itself never reads Spotify at all -- its vendored Spotify client still
|
||||
# calls GET /playlists/{id}/tracks, which Spotify removed in its February 2026
|
||||
# API changes (new apps get a hard 403 there regardless of auth method; see
|
||||
# spotify-playlist-csv.py). run-playlist.sh reads the playlist itself (via the
|
||||
# still-working /items endpoint) and hands sldl a CSV via --input-type csv on
|
||||
# the command line, which overrides this line entirely -- it's kept only so
|
||||
# run-playlist.sh has somewhere to read the playlist URL from per-playlist.
|
||||
input = SPOTIFY_URL
|
||||
input-type = spotify
|
||||
|
||||
# ==== Output paths ====
|
||||
# SLDL_DROPBOX_ROOT is substituted at render time from $MUSIC_DATA_DIR
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
spotify-playlist-csv.py — dump a Spotify playlist to a CSV sldl can search from.
|
||||
|
||||
sldl's own vendored Spotify client still calls Spotify's GET /playlists/{id}/tracks
|
||||
endpoint, which Spotify removed in its February 2026 API changes (see
|
||||
https://developer.spotify.com/documentation/web-api/references/changes/february-2026)
|
||||
in favor of /items. Grandfathered apps still get a pass on /tracks for now, but
|
||||
apps created after that change get a hard 403 there regardless of auth method --
|
||||
client-credentials, or a correctly-scoped, correctly-owned OAuth user token, none
|
||||
of it matters, because the endpoint itself is gone for them. sldl has no separate
|
||||
code path to fall back to /items, so it can never read a playlist for a new app
|
||||
no matter what alembic hands it.
|
||||
|
||||
Rather than depend on sldl's Spotify client at all, this script reads the
|
||||
playlist itself (via /items, which alembic's own code already migrated to) and
|
||||
writes the same Artist,Title,Album,Length CSV format upgrade-mp3-to-flac.sh
|
||||
already feeds to sldl via --input-type csv. run-playlist.sh runs this first and
|
||||
then points sldl at the CSV instead of the playlist URL, sidestepping sldl's
|
||||
Spotify client entirely -- for grandfathered and new apps alike.
|
||||
|
||||
Usage:
|
||||
spotify-playlist-csv.py <playlist_url> <out_csv>
|
||||
|
||||
Credentials are read from env vars SPOTIFY_CLIENT_ID and SPOTIFY_CLIENT_SECRET.
|
||||
SPOTIFY_REFRESH_TOKEN, if set, mints a user token instead of client-credentials
|
||||
(required for newly created Spotify apps -- see _spotify_auth.get_token).
|
||||
"""
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from _spotify_auth import get_token
|
||||
|
||||
API = "https://api.spotify.com/v1"
|
||||
|
||||
|
||||
def fetch_playlist(url: str, token: str) -> list[dict]:
|
||||
m = re.search(r"playlist[/:]([A-Za-z0-9]+)", url)
|
||||
if not m:
|
||||
sys.exit(f"Could not parse playlist ID from {url!r}")
|
||||
pid = m.group(1)
|
||||
tracks = []
|
||||
# No `fields` filter: Spotify's February 2026 changes renamed each entry's
|
||||
# payload from "track" to "item" for apps on the new behavior (extended
|
||||
# quota / grandfathered apps keep "track"), and a fields filter selects by
|
||||
# key name -- filtering on track(...) silently returns EMPTY pages on the
|
||||
# renamed shape. Fetch unfiltered and parse both key names below.
|
||||
next_url = f"{API}/playlists/{pid}/items?limit=100"
|
||||
while next_url:
|
||||
req = urllib.request.Request(next_url, headers={"Authorization": f"Bearer {token}"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
data = json.loads(r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code in (403, 404):
|
||||
sys.exit(
|
||||
f"Spotify returned {e.code} loading this playlist. The connected "
|
||||
"Spotify account can't see it: it must own the playlist, be a "
|
||||
"collaborator on it, or (for grandfathered apps) the playlist "
|
||||
"must be Public. Check which account is connected via "
|
||||
"/connect/spotify."
|
||||
)
|
||||
raise
|
||||
if "items" not in data:
|
||||
# New-behavior apps get metadata only (no items field) for
|
||||
# playlists the connected account doesn't own or collaborate on.
|
||||
sys.exit(
|
||||
"Spotify returned this playlist without its contents. For newly "
|
||||
"created Spotify apps, the connected account must OWN the playlist "
|
||||
"or be a collaborator on it (public is no longer enough). Ask the "
|
||||
"owner to make it collaborative and add you, or recreate the "
|
||||
"playlist under the connected account."
|
||||
)
|
||||
for item in data["items"]:
|
||||
t = item.get("track") or item.get("item")
|
||||
if not t or t.get("is_local"):
|
||||
continue
|
||||
# All artists, comma-joined -- matches what sldl's own Spotify
|
||||
# extractor fed its search, so multi-artist tracks match no worse
|
||||
# than they did before the CSV detour.
|
||||
artists = [a["name"] for a in (t.get("artists") or []) if a.get("name")]
|
||||
tracks.append(
|
||||
{
|
||||
"artist": ", ".join(artists),
|
||||
"title": t.get("name", ""),
|
||||
"album": (t.get("album") or {}).get("name", ""),
|
||||
"length": round((t.get("duration_ms") or 0) / 1000),
|
||||
}
|
||||
)
|
||||
next_url = data.get("next")
|
||||
return tracks
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) != 3:
|
||||
sys.exit(f"Usage: {sys.argv[0]} <playlist_url> <out_csv>")
|
||||
url, out_path = sys.argv[1], sys.argv[2]
|
||||
|
||||
cid = os.environ.get("SPOTIFY_CLIENT_ID")
|
||||
csec = os.environ.get("SPOTIFY_CLIENT_SECRET")
|
||||
refresh = os.environ.get("SPOTIFY_REFRESH_TOKEN") or None
|
||||
if not cid or not csec:
|
||||
sys.exit("SPOTIFY_CLIENT_ID and SPOTIFY_CLIENT_SECRET must be set")
|
||||
|
||||
token = get_token(cid, csec, refresh)
|
||||
tracks = fetch_playlist(url, token)
|
||||
|
||||
with open(out_path, "w", newline="") as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(["Artist", "Title", "Album", "Length"])
|
||||
for t in tracks:
|
||||
writer.writerow([t["artist"], t["title"], t["album"], t["length"]])
|
||||
|
||||
print(f"[spotify-playlist-csv] wrote {len(tracks)} tracks to {out_path}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -56,8 +56,11 @@ def fetch_playlist(url: str, token: str) -> list[dict]:
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
data = json.loads(r.read())
|
||||
for item in data["items"]:
|
||||
t = item.get("track")
|
||||
# Entries carry "track" (grandfathered apps) or "item" (apps on the
|
||||
# February 2026 renamed shape); items is absent entirely when the
|
||||
# connected account can't read the playlist's contents.
|
||||
for item in data.get("items", []):
|
||||
t = item.get("track") or item.get("item")
|
||||
if not t or t.get("is_local"):
|
||||
continue
|
||||
tracks.append(
|
||||
|
||||
@@ -43,22 +43,41 @@ def test_render_playlist_confs_injects_creds_safely(db, config_dir):
|
||||
text = conf.read_text()
|
||||
assert "user = seek" in text
|
||||
assert "pass = p'a$(id)ss" in text
|
||||
assert "spotify-id = CID" in text
|
||||
assert "spotify-secret = CSECRET" in text
|
||||
# no account connected yet -- rendered blank, not left as the placeholder
|
||||
assert "spotify-refresh = \n" in text
|
||||
# run-playlist.sh scrapes the playlist URL from this line to build the CSV
|
||||
assert "input = https://open.spotify.com/playlist/Z" in text
|
||||
# Spotify creds must NOT be rendered into confs: sldl never talks to
|
||||
# Spotify (run-playlist.sh feeds it a CSV); the creds live in _spotify.env
|
||||
assert "spotify-id" not in text
|
||||
assert "spotify-secret" not in text
|
||||
assert "spotify-refresh" not in text
|
||||
assert "CID" not in text
|
||||
assert "CSECRET" not in text
|
||||
# rendered config must be owner-only (holds the Soulseek password)
|
||||
assert stat.S_IMODE(os.stat(conf).st_mode) == 0o600
|
||||
|
||||
|
||||
def test_render_playlist_confs_includes_spotify_refresh_when_connected(db, config_dir):
|
||||
cs.set_credentials(db, "spotify", {"client_id": "CID", "client_secret": "CSECRET", "refresh_token": "RTOK"})
|
||||
cs.set_credentials(db, "soulseek", {"username": "seek", "password": "pw"})
|
||||
pl.create(db, name="rtest2", spotify_url="https://open.spotify.com/playlist/Z")
|
||||
def test_spotify_env_renders_creds_and_refresh_token(db, config_dir):
|
||||
# _spotify.env is the ONLY rendered home of Spotify creds; run-playlist.sh
|
||||
# sources it to fetch the playlist CSV, so values must be shell-quoted
|
||||
cs.set_credentials(
|
||||
db, "spotify", {"client_id": "CID", "client_secret": "CS'EC$(id)RET", "refresh_token": "RTOK"}
|
||||
)
|
||||
|
||||
conf = config_dir / "pipeline" / "rtest2.conf"
|
||||
text = conf.read_text()
|
||||
assert "spotify-refresh = RTOK" in text
|
||||
env = config_dir / "pipeline" / "_spotify.env"
|
||||
text = env.read_text()
|
||||
assert "SPOTIFY_CLIENT_ID=CID" in text
|
||||
# shlex.quote keeps a metacharacter-laden secret a single literal value
|
||||
assert "SPOTIFY_CLIENT_SECRET='CS'\"'\"'EC$(id)RET'" in text
|
||||
assert "SPOTIFY_REFRESH_TOKEN=RTOK" in text
|
||||
assert stat.S_IMODE(os.stat(env).st_mode) == 0o600
|
||||
|
||||
|
||||
def test_spotify_env_refresh_token_blank_until_connected(db, config_dir):
|
||||
cs.set_credentials(db, "spotify", {"client_id": "CID", "client_secret": "CSECRET"})
|
||||
|
||||
text = (config_dir / "pipeline" / "_spotify.env").read_text()
|
||||
# rendered blank (''), which sources cleanly and stays falsy in bash
|
||||
assert "SPOTIFY_REFRESH_TOKEN=''" in text
|
||||
|
||||
|
||||
def test_azuracast_renders_and_clears_base_url(db, config_dir):
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
"""get_playlist_tracks must parse BOTH /items response shapes: entries keyed
|
||||
"track" (grandfathered / extended-quota apps) and "item" (apps on Spotify's
|
||||
February 2026 renamed shape), and must fail loudly, not return [], when
|
||||
Spotify withholds a playlist's contents (no items field at all)."""
|
||||
import pytest
|
||||
|
||||
from app.services import spotify_client
|
||||
|
||||
URL = "https://open.spotify.com/playlist/XYZ123"
|
||||
|
||||
|
||||
class _Resp:
|
||||
def __init__(self, data):
|
||||
self._data = data
|
||||
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
def json(self):
|
||||
return self._data
|
||||
|
||||
|
||||
def _fake_api(monkeypatch, pages):
|
||||
it = iter(pages)
|
||||
monkeypatch.setattr(spotify_client, "_get_token", lambda db: "tok")
|
||||
monkeypatch.setattr(
|
||||
spotify_client.httpx,
|
||||
"get",
|
||||
lambda url, params=None, headers=None, timeout=None: _Resp(next(it)),
|
||||
)
|
||||
|
||||
|
||||
def _track(name, artist, isrc=None):
|
||||
return {
|
||||
"name": name,
|
||||
"artists": [{"name": artist}],
|
||||
"external_ids": {"isrc": isrc} if isrc else {},
|
||||
}
|
||||
|
||||
|
||||
def test_parses_legacy_track_shape(monkeypatch):
|
||||
_fake_api(monkeypatch, [{"items": [{"track": _track("Sandstorm", "Darude", "FIDAR9900011")}], "next": None}])
|
||||
tracks = spotify_client.get_playlist_tracks(None, URL)
|
||||
assert tracks == [{"artist": "Darude", "title": "Sandstorm", "isrc": "FIDAR9900011"}]
|
||||
|
||||
|
||||
def test_parses_renamed_item_shape(monkeypatch):
|
||||
# February 2026 shape: payload under "item", not "track"
|
||||
_fake_api(monkeypatch, [{"items": [{"item": _track("Sandstorm", "Darude")}], "next": None}])
|
||||
tracks = spotify_client.get_playlist_tracks(None, URL)
|
||||
assert tracks == [{"artist": "Darude", "title": "Sandstorm", "isrc": None}]
|
||||
|
||||
|
||||
def test_missing_items_field_raises_not_empty(monkeypatch):
|
||||
# metadata-only response (playlist not owned by the connected account on a
|
||||
# new app) must be an error the UI can show, never a silent empty list
|
||||
_fake_api(monkeypatch, [{"name": "DJ-USB", "public": True}])
|
||||
with pytest.raises(RuntimeError, match="without its contents"):
|
||||
spotify_client.get_playlist_tracks(None, URL)
|
||||
|
||||
|
||||
def test_skips_null_entries_and_paginates(monkeypatch):
|
||||
_fake_api(
|
||||
monkeypatch,
|
||||
[
|
||||
{"items": [{"track": None}, {"track": _track("A", "X")}], "next": "page2"},
|
||||
{"items": [{"item": _track("B", "Y")}], "next": None},
|
||||
],
|
||||
)
|
||||
tracks = spotify_client.get_playlist_tracks(None, URL)
|
||||
assert [t["title"] for t in tracks] == ["A", "B"]
|
||||
Reference in New Issue
Block a user