andrew/alembic
1
0
Fork 0
Code Pull Requests Actions Releases 18 Wiki Activity

Fix stuck dedup candidate, enrich timeout/starvation, and qobuz status error

Browse Source 
1. enrich-buy-url.py re-queried every no-match track on every daily run (external,
   rate-limited lookups over the whole backlog), which blew past its 30-min
   timeout. Cache "tried, no match" with a BUY_URL_TRIED marker tag and skip
   those for 30 days (--force still re-checks). This is what made "Look up buy
   links" fail with TIMEOUT after 1800s.

2. Move maintenance:enrich_buy_url to run last (09:45) in the 9am block, after
   the fingerprint index (09:25) and status report (09:30). enrich starting at
   09:10 and holding the shared lock is what left "Rebuild fingerprint index"
   skipped_lock.

3. pipeline-status.sh: guard the qobuz/app_id read with a file-exists check.
   `< missing 2>/dev/null` still leaks the shell's redirection error (opened
   before 2>/dev/null applies), so the daily report logged
   "qobuz/app_id: No such file or directory" every run.

4. dedup confirm_and_apply: a candidate whose delete target no longer exists
   (already removed by an earlier dedup/upgrade/hand) was filtered out and the
   apply silently no-op'd, leaving it stuck in the queue with no way to delete
   or clear it (the 100 gecs case). Now mark such candidates applied so a Delete
   click clears them. Covered by tests/test_dedup_review.py.

Tests: 46 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
andrew
2026-07-11 14:54:52 -06:00
parent fdcba992fd
commit 04c36af6b3
5 changed files with 121 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pipeline/lib/pipeline-status.sh
+7 -2
View File
@@ -315,8 +315,13 @@ dedup_today_status() {
# silently drops Qobuz and falls through to iTunes.
if [[ -f ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/token ]]; then
q_token=$(tr -d '\r\n' < ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/token)
q_appid=$(tr -d '\r\n' < ${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/app_id 2>/dev/null)
q_appid=${q_appid:-798273057}
# Guard the app_id read with a file-exists check: `< missing 2>/dev/null`
# still leaks the shell's own "No such file" redirection error (the redirect
# is opened before 2>/dev/null takes effect). Most setups have a token but
# no separate app_id, so this fired on every run.
q_appid=798273057
q_appid_file="${ALEMBIC_CONFIG_DIR:-/config}/pipeline/qobuz/app_id"
[[ -f "$q_appid_file" ]] && q_appid=$(tr -d '\r\n' < "$q_appid_file")
q_code=$(curl -s -o /dev/null --connect-timeout 5 --max-time 12 -A "Mozilla/5.0" \
-H "X-App-Id: $q_appid" -H "X-User-Auth-Token: $q_token" -w '%{http_code}' \
"https://www.qobuz.com/api.json/0.2/track/search?query=test&limit=1&app_id=$q_appid" 2>/dev/null)