andrew/alembic
1
0
Fork 0
Code Pull Requests Actions Releases 18 Wiki Activity

0.6.2: Bypass sldl's Spotify client -- fetch the playlist ourselves, feed sldl a CSV

Browse Source 
sldl's vendored Spotify client still calls GET /playlists/{id}/tracks, which
Spotify removed in its February 2026 API changes. Grandfathered apps still get
a pass; apps created after the change get a hard 403 there no matter how they
authenticate (client-credentials or a correctly-scoped OAuth user token), so
sldl can never load a playlist for a new app regardless of what we hand it.

Instead of depending on sldl's Spotify client at all, run-playlist.sh now reads
the playlist itself via the still-working /items endpoint (new
spotify-playlist-csv.py, creds sourced from _spotify.env) and invokes sldl with
the CSV + --input-type csv, which override the conf's input lines while -c still
supplies Soulseek login, paths, and quality settings (verified live). The same
CSV format upgrade-mp3-to-flac.sh already feeds sldl. All artists are
comma-joined in the CSV so multi-artist tracks search no worse than before, and
a 403/404 on the fetch prints the account-visibility hint instead of a bare
traceback.

Since sldl no longer talks to Spotify, playlist .confs no longer carry
spotify-id/spotify-secret/spotify-refresh: _template.conf drops them,
render_playlist_confs stops injecting them, and a Spotify credential save no
longer re-renders confs (only _spotify.env). The retag step in run-playlist.sh
reuses the sourced _spotify.env creds instead of scraping the conf lines that
no longer exist. Confs are also re-rendered once at app startup so
already-deployed confs converge on upgrade (and shed the stale secret lines)
without waiting for a playlist or credential change. Playlist delete now also
removes the generated .csv.

Tests updated: conf rendering must NOT contain Spotify creds but must keep the
input URL line; new coverage for _spotify.env rendering (quoting, refresh-token
presence/blank, 0600).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
andrew
2026-07-15 08:48:30 -06:00
parent 9976c430aa
commit 3fbf580b88
8 changed files with 245 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pipeline/bin/run-playlist.sh
+50 -41
View File
@@ -25,6 +25,8 @@ PLAYLIST_NAME="${1:?Usage: $0 [--no-m3u] <playlist_name>}"
# ==== Paths ====
CONFIG_FILE=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/${PLAYLIST_NAME}.conf
SPOTIFY_ENV=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/_spotify.env
CSV_FILE=${ALEMBIC_CONFIG_DIR:-/config}/pipeline/${PLAYLIST_NAME}.csv
DROPBOX=${MUSIC_DATA_DIR:-/data/music}/sldl-dropbox/${PLAYLIST_NAME}
PLAYLISTS_DIR=${MUSIC_DATA_DIR:-/data/music}/playlists
QUARANTINE_DIR=${MUSIC_DATA_DIR:-/data/music}/Songs/untagged
@@ -54,6 +56,34 @@ if [[ ! -f "$CONFIG_FILE" ]]; then
exit 1
fi
# ==== Read the playlist from Spotify into a CSV sldl can search from ====
# sldl's own vendored Spotify client still calls GET /playlists/{id}/tracks,
# which Spotify removed in its February 2026 API changes. Grandfathered apps
# still get a pass there; apps created after that change get a hard 403
# regardless of auth method (client-credentials, or even a correctly-scoped,
# correctly-owned OAuth user token -- confirmed live, exit 134 unhandled
# APIException: Forbidden from Extractors.Spotify.GetPlaylist). sldl has no
# fallback to the still-working /items endpoint, so it can never read a
# playlist for a new app no matter what alembic hands it. Reading the
# playlist ourselves (via /items) and handing sldl a plain CSV instead
# sidesteps sldl's Spotify client entirely -- works the same for
# grandfathered and new apps alike.
SPOTIFY_URL=$(sed -n 's/^input *= *//p' "$CONFIG_FILE" | tr -d ' ')
[[ -f "$SPOTIFY_ENV" ]] && source "$SPOTIFY_ENV"
if [[ -z "$SPOTIFY_URL" || -z "${SPOTIFY_CLIENT_ID:-}" || -z "${SPOTIFY_CLIENT_SECRET:-}" ]]; then
log "ERROR: missing playlist URL in $CONFIG_FILE or Spotify credentials in $SPOTIFY_ENV"
exit 1
fi
log "Fetching playlist from Spotify: $SPOTIFY_URL"
if ! SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
SPOTIFY_REFRESH_TOKEN="${SPOTIFY_REFRESH_TOKEN:-}" \
python3 "${PIPELINE_DIR:-/app/pipeline}/lib/spotify-playlist-csv.py" "$SPOTIFY_URL" "$CSV_FILE" >> "$LOG" 2>&1; then
log "ERROR: failed to fetch playlist from Spotify — see $LOG"
exit 1
fi
# ==== Run sldl (vendored binary, subprocess of this container) ====
log "Running sldl for: $PLAYLIST_NAME"
@@ -62,37 +92,24 @@ log "Running sldl for: $PLAYLIST_NAME"
# download. A non-zero exit here is logged but not fatal.
#
# Hard timeout: without it a stuck sldl hangs FOREVER holding the shared lock,
# which silently skips every later-scheduled playlist for the night. (Seen
# 2026-06-18: a transient Spotify client-creds failure made sldl fall back to
# interactive OAuth — "manually open: https://accounts.spotify.com/..." — and
# it waited 7h on a browser callback that never comes. Fixed by always
# rendering `spotify-refresh` into the conf once a Spotify account is
# connected via /connect/spotify — sldl's own docs confirm supplying a
# refresh token skips the login-flow requirement entirely. This timeout stays
# as a backstop for any other cause of a stuck run.) timeout TERMs the run
# and KILLs after a grace period; there's no leftover container to clean up
# now that sldl is a plain subprocess instead of a docker-compose service.
# which silently skips every later-scheduled playlist for the night. timeout
# TERMs the run and KILLs after a grace period; there's no leftover container
# to clean up now that sldl is a plain subprocess instead of a docker-compose
# service.
#
# -c "$CONFIG_FILE" still supplies Soulseek login, download path, quality
# settings, port, etc. -- the CSV positional arg + --input-type csv override
# just the input source (confirmed: sldl ignores the conf's own `input =`/
# `input-type =` lines when both are given).
SLDL_TIMEOUT="${SLDL_TIMEOUT:-2700}" # 45 min; override via env
SLDL_EXIT=0
timeout --kill-after=30s "$SLDL_TIMEOUT" \
"$SLDL_BIN" -c "$CONFIG_FILE" >> "$LOG" 2>&1 \
"$SLDL_BIN" "$CSV_FILE" --input-type csv -c "$CONFIG_FILE" >> "$LOG" 2>&1 \
|| SLDL_EXIT=$?
if [[ "$SLDL_EXIT" -eq 124 || "$SLDL_EXIT" -eq 137 ]]; then
log "ERROR: sldl exceeded ${SLDL_TIMEOUT}s and was killed (likely a hang)"
elif [[ "$SLDL_EXIT" -eq 134 ]]; then
# sldl aborts (SIGABRT) on ANY unhandled .NET exception rather than failing
# cleanly. The one case we've actually seen in the wild: a valid, freshly-
# refreshed user token still gets 403 Forbidden loading the playlist,
# because Spotify scopes OAuth playlist reads to what the CONNECTED account
# can see (its own playlists, ones it collaborates on, or ones marked
# Public) -- unlike client-credentials, which could read any public
# playlist regardless of whose app it was. Give a pointed hint when that's
# the visible cause; otherwise just flag that sldl crashed outright.
if grep -q "APIException: Forbidden" "$LOG"; then
log "ERROR: sldl crashed (exit 134) -- Spotify returned 403 loading this playlist. The connected Spotify account can't see it: make sure it's owned by, or shared/followed by, whichever account is connected via /connect/spotify, or set it to Public on Spotify."
else
log "ERROR: sldl crashed (exit 134, unhandled exception) -- see the log above for the exception detail"
fi
log "ERROR: sldl crashed (exit 134, unhandled exception) -- see the log above for the exception detail"
fi
log "sldl finished with exit code $SLDL_EXIT"
@@ -125,27 +142,19 @@ log "Cleaning up .incomplete files in dropbox"
find "$DROPBOX" -name "*.incomplete" -type f -delete 2>>"$LOG" || true
# ==== Rewrite tags from Spotify (source of truth for matched tracks) ====
# Reads Spotify creds + playlist URL from the same conf sldl used. For each file
# in the dropbox that matches a Spotify playlist track, overwrites ARTIST
# Uses the playlist URL and _spotify.env credentials already loaded for the
# CSV fetch above (confs no longer carry Spotify creds). For each file in the
# dropbox that matches a Spotify playlist track, overwrites ARTIST
# (semicolon-joined), ALBUMARTIST (primary), ALBUM, TITLE, TRACKNUMBER,
# DISCNUMBER, DATE. GROUPING is preserved. Files that don't match are left for
# the fallback step below.
SPOTIFY_URL=$(sed -n 's/^input *= *//p' "$CONFIG_FILE" | tr -d ' ')
SPOTIFY_CLIENT_ID=$(sed -n 's/^spotify-id *= *//p' "$CONFIG_FILE" | tr -d ' ')
SPOTIFY_CLIENT_SECRET=$(sed -n 's/^spotify-secret *= *//p' "$CONFIG_FILE" | tr -d ' ')
SPOTIFY_REFRESH_TOKEN=$(sed -n 's/^spotify-refresh *= *//p' "$CONFIG_FILE" | tr -d ' ')
if [[ -n "$SPOTIFY_URL" && -n "$SPOTIFY_CLIENT_ID" && -n "$SPOTIFY_CLIENT_SECRET" ]]; then
log "Rewriting tags from Spotify for files in $DROPBOX"
if SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
SPOTIFY_REFRESH_TOKEN="$SPOTIFY_REFRESH_TOKEN" \
python3 ${PIPELINE_DIR:-/app/pipeline}/lib/spotify-retag.py "$SPOTIFY_URL" "$DROPBOX" >> "$LOG" 2>&1; then
log "Spotify retag complete"
else
log "WARNING: Spotify retag failed (exit $?) — continuing with sldl-supplied tags"
fi
log "Rewriting tags from Spotify for files in $DROPBOX"
if SPOTIFY_CLIENT_ID="$SPOTIFY_CLIENT_ID" SPOTIFY_CLIENT_SECRET="$SPOTIFY_CLIENT_SECRET" \
SPOTIFY_REFRESH_TOKEN="${SPOTIFY_REFRESH_TOKEN:-}" \
python3 ${PIPELINE_DIR:-/app/pipeline}/lib/spotify-retag.py "$SPOTIFY_URL" "$DROPBOX" >> "$LOG" 2>&1; then
log "Spotify retag complete"
else
log "Skipping Spotify retag — missing input/spotify-id/spotify-secret in $CONFIG_FILE"
log "WARNING: Spotify retag failed (exit $?) — continuing with sldl-supplied tags"
fi
# ==== Quarantine any files still missing essential tags ====