andrew
4e0cfb8463
pipeline_runner: subprocess execution wrapper replacing flock -n with a single asyncio.Lock (deliberately one global lock, not per-resource -- matches the old flock's all-jobs-share-one-lock behavior rather than over-engineering it). Records one job_runs row per invocation (started/finished/status/exit_code/summary/log_path/triggered_by). If the lock is already held, records status='skipped_lock' immediately instead of silently dropping the run. Found and fixed a real concurrency bug via testing: the obvious asyncio.wait_for(lock.acquire(), timeout=0) idiom for a non-blocking try-acquire is broken in asyncio -- the wrapping Task's first iteration and the timeout-0 callback race with no guaranteed ordering, so it timed out on literally every call, including the very first uncontended one. Fixed using lock.locked() + acquire(), relying on acquire()'s fast path never suspending when uncontended. scheduler_service: AsyncIOScheduler with the default in-memory job store (NOT SQLAlchemyJobStore -- it pickles job functions to persist them, and the _lib()/_beet() factory closures here aren't picklable; MemoryJobStore avoids this since register_all_jobs() rebuilds the whole schedule from code + DB on every startup anyway). All 19 maintenance jobs ported 1:1 from /etc/cron.d/sldl-maintenance with their exact cron expressions; dedup deliberately has no --apply (dry-run only on schedule, per the false-negative-biased dedup preference). Playlist jobs are synced live from the playlists table (sync_playlist_jobs), so add/remove-in-the-UI takes effect with no redeploy -- wired into playlist_service's create/update/ delete. Maintenance job enable/disable persists to the scheduled_jobs table so a paused job stays paused across a restart despite the in-memory job store. trigger_now() supports "Run now" by invoking a job's function immediately with triggered_by='manual', bypassing its schedule. Wired into main.py's lifespan (start on boot, shutdown on exit). Verified via integration tests: 36 jobs register correctly (17 seeded playlists + 19 maintenance) with correct cron triggers; live playlist create/delete correctly adds/removes its scheduler job; maintenance enable/disable persists to the DB and takes effect live; trigger_now correctly bypasses the schedule; pipeline_runner correctly handles success/failure/timeout/concurrent-lock-contention with real subprocesses; and the full app boots with the scheduler running and shuts down cleanly.
41 lines
971 B
Python
41 lines
971 B
Python
from contextlib import asynccontextmanager
|
|
|
|
from fastapi import FastAPI
|
|
from fastapi.staticfiles import StaticFiles
|
|
from starlette.middleware.sessions import SessionMiddleware
|
|
|
|
from app.db import enable_beets_db_wal, init_db
|
|
from app.routers import auth, dashboard
|
|
from app.services import scheduler_service
|
|
from app.settings import settings
|
|
|
|
|
|
@asynccontextmanager
|
|
async def lifespan(_app: FastAPI):
|
|
init_db()
|
|
enable_beets_db_wal()
|
|
|
|
scheduler = scheduler_service.create_scheduler()
|
|
scheduler_service.register_all_jobs(scheduler)
|
|
scheduler.start()
|
|
|
|
yield
|
|
|
|
scheduler.shutdown(wait=False)
|
|
|
|
|
|
def create_app() -> FastAPI:
|
|
app = FastAPI(title="alembic", lifespan=lifespan)
|
|
|
|
app.add_middleware(SessionMiddleware, secret_key=settings.session_secret)
|
|
|
|
app.mount("/static", StaticFiles(directory="app/static"), name="static")
|
|
|
|
app.include_router(auth.router)
|
|
app.include_router(dashboard.router)
|
|
|
|
return app
|
|
|
|
|
|
app = create_app()
|