0.6.14: Fix Bandcamp sync silently failing to import new purchases
sync-bandcamp.sh set a hardcoded PATH at startup that left out /opt/venv/bin, which is where beet lives in this image. Every time the Bandcamp sync had a new purchase to import, its call into import-track.sh would run beet import with that broken PATH, fail with "command not found", and import-track.sh would just log the exit code and carry on, so sync-bandcamp.sh still reported success. The purchase sat on disk, never entered the beets library, and never showed up in Navidrome. This went unnoticed for weeks because most daily syncs have nothing new to import, so the broken code path rarely ran. Fixed the same copy-pasted PATH line in upgrade-mp3-to-flac.sh (which also calls beet directly) and notify-telegram.sh (harmless there, but fixed for consistency). Also moved the post-import duplicate cleanup (replace-with-better.sh and dedup-library.sh) out of sync-bandcamp.sh and into import-track.sh itself, so every import gets the same cleanup, not just Bandcamp purchases. A manual import or SMB drop that happens to match something already in the library no longer leaves a duplicate copy sitting there until someone runs the dedup review by hand.
This commit is contained in:
@@ -73,10 +73,10 @@ Optional, add these later if you want them:
|
|||||||
Pull the prebuilt image onto your Docker host:
|
Pull the prebuilt image onto your Docker host:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker pull git.kretzer.club/andrew/alembic:0.6.13
|
docker pull git.kretzer.club/andrew/alembic:0.6.14
|
||||||
```
|
```
|
||||||
|
|
||||||
That is the whole install. You do not need to download the source or build anything. The `0.6.13` is the version; you can pin to it so nothing changes under you, or use `latest` to always get the newest.
|
That is the whole install. You do not need to download the source or build anything. The `0.6.14` is the version; you can pin to it so nothing changes under you, or use `latest` to always get the newest.
|
||||||
|
|
||||||
(If you would rather build it yourself from source, you can, but you do not need to.)
|
(If you would rather build it yourself from source, you can, but you do not need to.)
|
||||||
|
|
||||||
@@ -136,7 +136,7 @@ Create a file called `docker-compose.yml` on your server (put it wherever you ke
|
|||||||
```yaml
|
```yaml
|
||||||
services:
|
services:
|
||||||
alembic:
|
alembic:
|
||||||
image: git.kretzer.club/andrew/alembic:0.6.13
|
image: git.kretzer.club/andrew/alembic:0.6.14
|
||||||
container_name: alembic
|
container_name: alembic
|
||||||
ports:
|
ports:
|
||||||
- "8420:8420"
|
- "8420:8420"
|
||||||
|
|||||||
@@ -242,6 +242,30 @@ BEETS_EXIT=0
|
|||||||
beet import -q -s "$DEST_DIR" >> "$LOG" 2>&1 || BEETS_EXIT=$?
|
beet import -q -s "$DEST_DIR" >> "$LOG" 2>&1 || BEETS_EXIT=$?
|
||||||
log "beets import finished with exit code $BEETS_EXIT"
|
log "beets import finished with exit code $BEETS_EXIT"
|
||||||
|
|
||||||
|
# ==== Safety net: clean up stragglers + exact duplicates ====
|
||||||
|
# With duplicate_action=keep, beets always imports rather than rejecting a
|
||||||
|
# real conflict (see beets/config.yaml) -- any file whose (albumartist,
|
||||||
|
# album, title) already exists in the library gets imported anyway as a
|
||||||
|
# `.1.ext` sibling. That's true whether this run came from the web UI, an
|
||||||
|
# SMB drop, or sync-bandcamp.sh, so the cleanup has to run here rather than
|
||||||
|
# per-caller (a 2026-07-23 incident: a Bandcamp sync's own call into this
|
||||||
|
# script failed and silently stranded ~150 already-owned files in import-me/
|
||||||
|
# for two weeks; a later unrelated manual import swept them back in and
|
||||||
|
# duplicated them, and nothing had deduped since).
|
||||||
|
log "Running replace-with-better safety pass on /downloads stragglers"
|
||||||
|
if ${PIPELINE_DIR:-/app/pipeline}/lib/replace-with-better.sh --apply >> "$LOG" 2>&1; then
|
||||||
|
log "replace-with-better OK"
|
||||||
|
else
|
||||||
|
log "WARN: replace-with-better.sh exited non-zero (exit $?)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "Running dedup pass (exact-match duplicates only; FLAC > MP3, then largest file)"
|
||||||
|
if ${PIPELINE_DIR:-/app/pipeline}/lib/dedup-library.sh --apply >> "$LOG" 2>&1; then
|
||||||
|
log "dedup OK"
|
||||||
|
else
|
||||||
|
log "WARN: dedup-library.sh exited non-zero (exit $?)"
|
||||||
|
fi
|
||||||
|
|
||||||
# ==== Regenerate M3U if playlist was specified ====
|
# ==== Regenerate M3U if playlist was specified ====
|
||||||
if [[ -n "$PLAYLIST_NAME" ]]; then
|
if [[ -n "$PLAYLIST_NAME" ]]; then
|
||||||
log "Regenerating M3U for $PLAYLIST_NAME"
|
log "Regenerating M3U for $PLAYLIST_NAME"
|
||||||
|
|||||||
@@ -20,7 +20,9 @@
|
|||||||
# Returns exit 0 on send-OK, non-zero otherwise.
|
# Returns exit 0 on send-OK, non-zero otherwise.
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
# /opt/venv/bin leads PATH for consistency with the other pipeline scripts,
|
||||||
|
# even though this one only shells out to curl.
|
||||||
|
PATH=/opt/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||||
|
|
||||||
PARSE_MODE=""
|
PARSE_MODE=""
|
||||||
if [[ "${1:-}" == "--html" ]]; then
|
if [[ "${1:-}" == "--html" ]]; then
|
||||||
|
|||||||
@@ -7,7 +7,15 @@
|
|||||||
# Run as root (cron). Logs to ${ALEMBIC_CONFIG_DIR:-/config}/logs/bandcamp-YYYYMMDD.log.
|
# Run as root (cron). Logs to ${ALEMBIC_CONFIG_DIR:-/config}/logs/bandcamp-YYYYMMDD.log.
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
# /opt/venv/bin must lead PATH -- that's where `beet` lives (the app's own
|
||||||
|
# subprocess env puts it there too, see pipeline_runner._subprocess_env()).
|
||||||
|
# This script used to run as a bare root cron job (pre-2026-07-08 cutover to
|
||||||
|
# the app scheduler) where a minimal hardened PATH made sense; omitting the
|
||||||
|
# venv here silently broke every downstream `beet import` call inside
|
||||||
|
# import-track.sh once this started running through the app instead --
|
||||||
|
# import-track.sh swallows that failure and still reports OK, so a purchase
|
||||||
|
# would sit unimported until someone noticed it missing from Navidrome.
|
||||||
|
PATH=/opt/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||||
export PATH
|
export PATH
|
||||||
|
|
||||||
CONFIG="${ALEMBIC_CONFIG_DIR:-/config}/pipeline/bandcamp/config.env"
|
CONFIG="${ALEMBIC_CONFIG_DIR:-/config}/pipeline/bandcamp/config.env"
|
||||||
@@ -87,7 +95,10 @@ shopt -u dotglob nullglob
|
|||||||
|
|
||||||
# Hand off to the standard manual-import pipeline. No playlist tag — Bandcamp
|
# Hand off to the standard manual-import pipeline. No playlist tag — Bandcamp
|
||||||
# purchases aren't part of any Spotify playlist. The import-track.sh guard,
|
# purchases aren't part of any Spotify playlist. The import-track.sh guard,
|
||||||
# albumartist fallback, beets import, and Navidrome scan all kick in normally.
|
# albumartist fallback, beets import, straggler/dedup safety net, and
|
||||||
|
# Navidrome scan all kick in normally (import-track.sh runs the
|
||||||
|
# replace-with-better + dedup-library safety net itself now, for every
|
||||||
|
# caller, not just this one).
|
||||||
log "Calling import-track.sh"
|
log "Calling import-track.sh"
|
||||||
if ${PIPELINE_DIR:-/app/pipeline}/bin/import-track.sh >> "$LOG" 2>&1; then
|
if ${PIPELINE_DIR:-/app/pipeline}/bin/import-track.sh >> "$LOG" 2>&1; then
|
||||||
log "import-track.sh OK"
|
log "import-track.sh OK"
|
||||||
@@ -95,28 +106,5 @@ else
|
|||||||
log "WARN: import-track.sh exited non-zero (exit $?)"
|
log "WARN: import-track.sh exited non-zero (exit $?)"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Safety net: if anything is stranded in ${MUSIC_DATA_DIR:-/data/music}/downloads (rare with
|
|
||||||
# duplicate_action=keep set in beets config, but possible for files beets
|
|
||||||
# couldn't process), find each one's library counterpart and replace if the
|
|
||||||
# new file is higher quality, OR import as new if there's no counterpart.
|
|
||||||
log "Running replace-with-better safety pass on /downloads stragglers"
|
|
||||||
if ${PIPELINE_DIR:-/app/pipeline}/lib/replace-with-better.sh --apply >> "$LOG" 2>&1; then
|
|
||||||
log "replace-with-better OK"
|
|
||||||
else
|
|
||||||
log "WARN: replace-with-better.sh exited non-zero (exit $?)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Now dedupe across the library. With duplicate_action=keep beets imported
|
|
||||||
# every Bandcamp file even when it conflicts with an existing track at the
|
|
||||||
# same path (it creates `.1.ext` siblings). dedup-library.sh's policy is
|
|
||||||
# "FLAC > MP3, then largest file" — Bandcamp version wins, soulseek version
|
|
||||||
# is removed from both the beets DB and disk.
|
|
||||||
log "Running dedup pass (Bandcamp FLAC wins over older Soulseek copies)"
|
|
||||||
if ${PIPELINE_DIR:-/app/pipeline}/lib/dedup-library.sh --apply >> "$LOG" 2>&1; then
|
|
||||||
log "dedup OK"
|
|
||||||
else
|
|
||||||
log "WARN: dedup-library.sh exited non-zero (exit $?)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
log "=== Bandcamp sync done ==="
|
log "=== Bandcamp sync done ==="
|
||||||
exit 0
|
exit 0
|
||||||
|
|||||||
@@ -20,7 +20,8 @@
|
|||||||
# upgrade-mp3-to-flac.sh --csv-only # just write the CSV; don't run sldl
|
# upgrade-mp3-to-flac.sh --csv-only # just write the CSV; don't run sldl
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
# /opt/venv/bin must lead PATH -- `beet` (used heavily below) lives there.
|
||||||
|
PATH=/opt/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||||
export PATH
|
export PATH
|
||||||
|
|
||||||
# These helpers are the same ones import-track.sh sources and runs under its
|
# These helpers are the same ones import-track.sh sources and runs under its
|
||||||
|
|||||||
Reference in New Issue
Block a user