[celery] Queue-based task prioritization (672c81bf)
This commit is contained in:
@ -117,6 +117,34 @@ Import gym sessions and routines from a Flexify backup.
|
||||
cover body-composition metrics on Task scrobbles and `bodyweight_kg` on
|
||||
Workout scrobbles.
|
||||
|
||||
** DONE [#A] Fix celery task prioritization especially for agent sessions :celery:tasks:agents:
|
||||
:PROPERTIES:
|
||||
:ID: 672c81bf-bba9-a963-e8ac-52246d976cea
|
||||
:END:
|
||||
|
||||
Split Celery work into four queues so time-sensitive tasks run promptly even
|
||||
when long batch jobs are queued:
|
||||
|
||||
- `priority` — interactive/user-facing tasks (agent session prompts, Mopidy
|
||||
queue/playlist adds, favorite toggles, reverse geocoding) get a dedicated
|
||||
high-concurrency worker (`--concurrency=4 --prefetch-multiplier=1`).
|
||||
- `charts` — per-scrobble chart updates (`update_charts_for_timestamp`) stay on
|
||||
their own queue, no longer blocked behind scheduled chart rebuilds (those
|
||||
moved to `background`).
|
||||
- `background` — heavy/scheduled jobs (db backup, trend computation, sentiment
|
||||
backfill, chart rebuilds, twitch VOD check, historical Last.fm dispatch) run
|
||||
on a dedicated low-concurrency worker (`--concurrency=1`).
|
||||
- `default` — everything else (imports, notifications, archivebox pushes,
|
||||
board-game expansion fetches). `CELERY_TASK_DEFAULT_QUEUE` is now set
|
||||
explicitly to `default`; previously un-routed tasks were published to
|
||||
Celery's default `celery` queue, which no worker consumed.
|
||||
|
||||
Workers run as three FreeBSD daemons (`vrobbler_celery` on `default,charts`,
|
||||
plus new `vrobbler_celery_priority` and `vrobbler_celery_background` rc.d
|
||||
scripts with distinct nodenames). Procfile, justfile, Makefile, Drone and
|
||||
Gitea workflows updated accordingly.
|
||||
|
||||
|
||||
* Version 64.5 [1/1]
|
||||
** DONE [#B] Clean up issues with org-mode notes :orgmode:notes:scrobbles:
|
||||
:PROPERTIES:
|
||||
|
||||
Reference in New Issue
Block a user