secstate/vrobbler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[celery] Rename default,charts worker to vrobbler_celery_charts

Browse Source
This commit is contained in:
Colin Powell
2026-08-13 19:01:27 -04:00
parent 74d0c67f02
commit 8af58f5f25
6 changed files with 54 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.drone.yml
View File

@ -37,7 +37,7 @@ steps:
- pip install git+https://code.lab.unbl.ink/secstate/vrobbler.git@main
- vrobbler migrate
- vrobbler collectstatic --noinput
- service vrobbler_celery restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && service vrobbler restart
- service vrobbler_celery_charts restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && service vrobbler restart
when:
ref:
- refs/tags/*

2
.gitea/workflows/ci.yml
View File

@ -137,7 +137,7 @@ jobs:
python3 -c "import vrobbler; print(f'vrobbler {vrobbler.__version__} installed OK')"
vrobbler migrate
vrobbler collectstatic --noinput
service vrobbler_celery restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && service vrobbler restart
service vrobbler_celery_charts restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && service vrobbler restart
- name: Notify deploy success (ntfy)
if: success()

27
CHANGELOG.org
View File

@ -1,6 +1,20 @@
#+title: CHANGELOG
* Version 65.4 [1/1]
* Version 65.4 [2/2]
** DONE [#B] Rename the default,charts celery worker to vrobbler_celery_charts :celery:deploy:
:PROPERTIES:
:ID: 92686951-8c87-4eae-8c48-1bed87567e93
:END:
*** Description
The FreeBSD rc.d worker that consumes the `default,charts` queues is renamed
from `vrobbler_celery` to `vrobbler_celery_charts`, matching the naming of its
companion workers (`vrobbler_celery_priority`, `vrobbler_celery_background`).
The gitea action, Drone pipeline, Makefile deploy target, and the rc.d script
itself are updated so every deploy restarts the three celery workers plus
celerybeat.
** DONE [#A] Fix database backup bug where running in celery task fails :bug:backups:celery:tasks:
:PROPERTIES:
:ID: 4117482f-4774-49b8-93b2-5b97adb194bd
@ -125,7 +139,8 @@ long batch jobs are queued up:
Also set `CELERY_TASK_DEFAULT_QUEUE = "default"` — previously un-routed tasks
were published to Celery's default `celery` queue, which no worker consumed.
New rc.d scripts `vrobbler_celery_priority` and `vrobbler_celery_background`
added alongside the existing `vrobbler_celery` (now `default,charts`).
added alongside the existing `vrobbler_celery` (renamed
`vrobbler_celery_charts`; now `default,charts`).
** DONE [#A] Before enriching anything, trust the POST data :feature:scrobbles:metadata:
:PROPERTIES:
@ -315,10 +330,10 @@ when long batch jobs are queued:
explicitly to `default`; previously un-routed tasks were published to
Celery's default `celery` queue, which no worker consumed.
Workers run as three FreeBSD daemons (`vrobbler_celery` on `default,charts`,
plus new `vrobbler_celery_priority` and `vrobbler_celery_background` rc.d
scripts with distinct nodenames). Procfile, justfile, Makefile, Drone and
Gitea workflows updated accordingly.
Workers run as three FreeBSD daemons (`vrobbler_celery_charts` on
`default,charts`, plus new `vrobbler_celery_priority` and
`vrobbler_celery_background` rc.d scripts with distinct nodenames). Procfile,
justfile, Makefile, Drone and Gitea workflows updated accordingly.
** DONE [#A] Before enriching anything, trust the POST data :feature:scrobbles:metadata:

2
Makefile
View File

@ -1,5 +1,5 @@
deploy:
ssh vrobbler.service "pip uninstall vrobbler && pip install git+https://code.lab.unbl.ink/secstate/vrobbler.git && service vrobbler restart && service vrobbler_celery restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && vrobbler migrate"
ssh vrobbler.service "pip uninstall vrobbler && pip install git+https://code.lab.unbl.ink/secstate/vrobbler.git && service vrobbler restart && service vrobbler_celery_charts restart && service vrobbler_celery_priority restart && service vrobbler_celery_background restart && service vrobbler_celerybeat restart && vrobbler migrate"
logs:
ssh life.unbl.ink tail -n 100 -f /var/log/vrobbler.json
test:

14
PROJECT.org
View File

@ -646,3 +646,17 @@ rc.subr default of `/sbin:/bin:/usr/sbin:/usr/bin`) that omits `/usr/local/bin`,
where `pg_dump` lives. The task now locates `pg_dump` with `shutil.which()`,
falling back to the common PostgreSQL install locations, instead of relying on
the inherited PATH.
** DONE [#B] Rename the default,charts celery worker to vrobbler_celery_charts :celery:deploy:
:PROPERTIES:
:ID: 92686951-8c87-4eae-8c48-1bed87567e93
:END:
*** Description
The FreeBSD rc.d worker that consumes the `default,charts` queues is renamed
from `vrobbler_celery` to `vrobbler_celery_charts`, matching the naming of its
companion workers (`vrobbler_celery_priority`, `vrobbler_celery_background`).
The gitea action, Drone pipeline, Makefile deploy target, and the rc.d script
itself are updated so every deploy restarts the three celery workers plus
celerybeat.

32
rc.d/vrobbler_celery → rc.d/vrobbler_celery_charts
View File

@ -2,22 +2,22 @@
#
# rc.d script for the vrobbler Celery worker.
#
# PROVIDE: vrobbler_celery
# PROVIDE: vrobbler_celery_charts
# REQUIRE: NETWORKING postgresql redis
# KEYWORD: shutdown
#
# Installation (on the FreeBSD jail):
#
# install -m 0555 rc.d/vrobbler_celery /usr/local/etc/rc.d/vrobbler_celery
# sysrc vrobbler_celery_enable="YES"
# service vrobbler_celery start
# install -m 0555 rc.d/vrobbler_celery_charts /usr/local/etc/rc.d/vrobbler_celery_charts
# sysrc vrobbler_celery_charts_enable="YES"
# service vrobbler_celery_charts start
#
# Configuration (via sysrc or /etc/rc.conf):
#
# vrobbler_celery_queues - queues to consume (default: default,charts)
# vrobbler_celery_bin - path to celery (default: /usr/local/bin/celery)
# vrobbler_celery_user - user to run as (default: root)
# vrobbler_celery_flags - extra daemon(8) flags
# vrobbler_celery_charts_queues - queues to consume (default: default,charts)
# vrobbler_celery_charts_bin - path to celery (default: /usr/local/bin/celery)
# vrobbler_celery_charts_user - user to run as (default: root)
# vrobbler_celery_charts_flags - extra daemon(8) flags
#
# NOTE: do not name the celery path knob "${name}_program". FreeBSD's
# rc.subr(8) overrides $command with ${name}_program whenever it is set,
@ -26,7 +26,7 @@
# The command is run under daemon(8) with a supervisor pidfile (-P) and
# restart-on-crash (-R), so the worker is automatically restarted if it dies,
# in the same way immortal used to. Sending SIGTERM to the supervisor (which
# is what `service vrobbler_celery stop` does) forwards the signal to celery,
# is what `service vrobbler_celery_charts stop` does) forwards the signal to celery,
# which shuts the worker down gracefully.
#
# The default,charts worker handles on-demand tasks plus per-scrobble chart
@ -36,20 +36,20 @@
#
. /etc/rc.subr
name="vrobbler_celery"
rcvar="vrobbler_celery_enable"
name="vrobbler_celery_charts"
rcvar="vrobbler_celery_charts_enable"
load_rc_config ${name}
: ${vrobbler_celery_enable:="NO"}
: ${vrobbler_celery_queues:="default,charts"}
: ${vrobbler_celery_bin:="/usr/local/bin/celery"}
: ${vrobbler_celery_autoscale:="2,6"}
: ${vrobbler_celery_charts_enable:="NO"}
: ${vrobbler_celery_charts_queues:="default,charts"}
: ${vrobbler_celery_charts_bin:="/usr/local/bin/celery"}
: ${vrobbler_celery_charts_autoscale:="2,6"}
: ${vrobbler_log_dir:="/var/log"}
pidfile="/var/run/${name}.pid"
command="/usr/sbin/daemon"
command_args="-f -P ${pidfile} -R 5 -t ${name} ${vrobbler_celery_bin} -A vrobbler worker -Q ${vrobbler_celery_queues} -n default@%h -l info --logfile ${vrobbler_log_dir}/${name}.log"
command_args="-f -P ${pidfile} -R 5 -t ${name} ${vrobbler_celery_charts_bin} -A vrobbler worker -Q ${vrobbler_celery_charts_queues} -n default@%h -l info --logfile ${vrobbler_log_dir}/${name}.log"
run_rc_command "$1"