Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d6f71e0761 | |||
| b00ebf49dd | |||
| 2385e9c7bd | |||
| d78529efe2 | |||
| f373e98e3d | |||
| 7559ce7824 | |||
| 2c481bd53a | |||
| 0deb3ee634 | |||
| 28a53d70eb | |||
| 98d4e8bacb | |||
| 8f97131b8d | |||
| 7c1f709f96 | |||
| 4b005e0e5b | |||
| d5dd63be0d | |||
| 5aa89b7e0a | |||
| cf444e8dd4 | |||
| a74a89c747 | |||
| 1695f7393e | |||
| 4468e68110 | |||
| da08eca4ab | |||
| 08752e30a4 | |||
| 619718c045 | |||
| cb23d5a5be | |||
| ec4c190e6c | |||
| 58126928c7 | |||
| c0d2881585 | |||
| 41a68291a4 | |||
| 0a411bedf4 | |||
| f2b67b38dc | |||
| 662ebe66b9 | |||
| 5e0dffdc7a | |||
| 2283a6c640 | |||
| 327ba94c63 |
@ -1,68 +0,0 @@
|
||||
name: build
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["**"]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
VROBBLER_DATABASE_URL: sqlite:///test.db
|
||||
VROBBLER_USDA_API_KEY: ${{ vars.VROBBLER_USDA_API_KEY }}
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Cache pip/poetry
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cache/pip
|
||||
~/.cache/pypoetry
|
||||
key: ${{ runner.os }}-py311-${{ hashFiles('**/poetry.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-py311-
|
||||
|
||||
- name: Install Poetry
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install poetry
|
||||
|
||||
- name: Install deps
|
||||
run: |
|
||||
cp vrobbler.conf.test vrobbler.conf
|
||||
poetry install --with test
|
||||
|
||||
- name: Pytest with coverage
|
||||
run: |
|
||||
poetry run pytest -n 5 --cov-report term:skip-covered --cov=vrobbler tests
|
||||
|
||||
- name: Notify success (ntfy)
|
||||
if: success()
|
||||
run: |
|
||||
curl -fsS \
|
||||
-H "Title: vrobbler CI success" \
|
||||
-H "Priority: low" \
|
||||
-H "Tags: success,vrobbler" \
|
||||
-H "Actions: view, Changes, ${{ gitea.server_url }}/${{ gitea.repository }}/commit/${{ gitea.sha }}; view, Build, ${{ gitea.server_url }}/${{ gitea.repository }}/actions/runs/${{ gitea.run_id }}" \
|
||||
-d "✅ Build succeeded: ${{ gitea.repository }} @ ${{ gitea.sha }}" \
|
||||
https://ntfy.unbl.ink/drone
|
||||
|
||||
- name: Notify failure (ntfy)
|
||||
if: failure()
|
||||
run: |
|
||||
curl -fsS \
|
||||
-H "Title: vrobbler CI failure" \
|
||||
-H "Priority: high" \
|
||||
-H "Tags: failure,vrobbler" \
|
||||
-H "Actions: view, Changes, ${{ gitea.server_url }}/${{ gitea.repository }}/commit/${{ gitea.sha }}; view, Build, ${{ gitea.server_url }}/${{ gitea.repository }}/actions/runs/${{ gitea.run_id }}" \
|
||||
-d "❌ Build failed: ${{ gitea.repository }} @ ${{ gitea.sha }}" \
|
||||
https://ntfy.unbl.ink/drone
|
||||
@ -1,8 +1,14 @@
|
||||
name: deploy
|
||||
name: ci
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["**"]
|
||||
tags: ["*"]
|
||||
pull_request:
|
||||
|
||||
concurrency:
|
||||
group: ${{ gitea.workflow }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
test:
|
||||
@ -68,6 +74,7 @@ jobs:
|
||||
|
||||
build-and-deploy:
|
||||
needs: [test]
|
||||
if: startsWith(gitea.ref, 'refs/tags/')
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
171
PROJECT.org
171
PROJECT.org
@ -88,7 +88,8 @@ fetching and simple saving.
|
||||
*** Metadata sources
|
||||
**** Scraper
|
||||
|
||||
* Backlog [0/22] :vrobbler:project:personal:
|
||||
* Backlog [0/24] :vrobbler:project:personal:
|
||||
** TODO [#C] After transition to linux add curl_cffi as webpage scrapper again :webpages:metadata:
|
||||
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :bug:music:scrobbles:
|
||||
:PROPERTIES:
|
||||
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
|
||||
@ -603,6 +604,174 @@ a helper method to create board game scrobbles given a json blob. It's
|
||||
independent of the email flow it was originally creatdd for
|
||||
|
||||
** TODO [#B] Is there way to create unique slugs for media instances :media_types:
|
||||
** TODO [#A] Update how board game scrobbles work :boardgames:
|
||||
|
||||
*** Description
|
||||
|
||||
When we scrobble a board game from a BGG URL, instead of going to the media
|
||||
detail page, we should go to the scrobble detail page, with the Edit Log form
|
||||
expanded by default.
|
||||
|
||||
The Edit log form should have from top to bottom:
|
||||
|
||||
- Board/Variant (one or many BoardGameVariant in a multi-select widget)
|
||||
- People (which should be similar to the Bird widget on BirdLocation and allow setting per user score, win true/false, rank, new true/false, seat_ordrer)
|
||||
- Expansion ids (which should a multi-select widget of expansions for this game)
|
||||
- Location (which should be a drop down of BoardGameLocations for this user)
|
||||
|
||||
* Version 59.4 [2/2]
|
||||
** DONE Fix bug in fetching expansions for board games :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 17995312-e76e-4a50-b591-0eab78cb59ab
|
||||
:END:
|
||||
|
||||
#+begin_src python
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/boardgames/management/commands/fetch_expansions.py", line 60, in handle
|
||||
fetch_and_link_expansions(game, expansions)
|
||||
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/boardgames/utils.py", line 51, in fetch_and_link_expansions
|
||||
expansion = BoardGame.find_or_create(str(exp_id))
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/boardgames/models.py", line 409, in find_or_create
|
||||
bgg_data = lookup_boardgame_from_bgg(lookup_id=lookup_id)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/boardgames/sources/bgg.py", line 15, in lookup_boardgame_from_bgg
|
||||
game = bgg.game(game_id=lookup_id)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/boardgamegeek/api.py", line 1045, in game
|
||||
raise BGGApiError(msg)
|
||||
boardgamegeek.exceptions.BGGApiError: invalid data for game id: 242117
|
||||
#+end_src
|
||||
|
||||
** DONE Board games should have genres extracted from family data :boardgames:metadata:
|
||||
:PROPERTIES:
|
||||
:ID: 7214b270-dccc-4b98-ac58-ff4f76c8cda9
|
||||
:END:
|
||||
|
||||
|
||||
* Version 59.3 [2/2]
|
||||
** DONE Exclude some board games from auto-expansion imports :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 51ffdf20-e732-4774-b781-c3501d26d46f
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
Some board games, especially trading card games, have silly amounts of expansions.
|
||||
We should have a setting SKIP_AUTO_EXPANSION_DOWNLOAD that is a list of BGG ids of
|
||||
games where expansions should not be download automatically. This exclusion should also auto-include
|
||||
any games with "Collectible Card Games" in it's family.
|
||||
|
||||
** DONE Should be able to add new variants to board games via the log data form :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 5ed0dd25-3026-3da8-dc5c-f2a75751af9a
|
||||
:END:
|
||||
|
||||
* Version 59.2 [1/1]
|
||||
** DONE Fix test failure in discgolf app :discgolf:tests:
|
||||
:PROPERTIES:
|
||||
:ID: 813ae357-0568-5a4c-1a35-172e95d02740
|
||||
:END:
|
||||
|
||||
* Version 59.1 [1/1]
|
||||
** DONE Fix bug when expansions have no image :boardgames:bug:
|
||||
:PROPERTIES:
|
||||
:ID: 9fee96c9-c6a0-32d9-b6f8-212c60fc3540
|
||||
:END:
|
||||
|
||||
* Version 59.0 [3/3]
|
||||
** DONE [#A] Add BoardGameVariant model :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 0ffb20d5-252f-b13d-473d-5529014602ff
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
Variants represent unique boards being used per scrobble or scenarios when
|
||||
playing a game. Scrobbles of a board game may have one or more
|
||||
boardgame_variant_ids assocaited with their log data, and a variant is created
|
||||
for one specific board game.
|
||||
|
||||
|
||||
** DONE [#A] Lookup all Expansions for a game when creating it :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 8a84b06d-555c-4701-9058-ff364c89c198
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
We don't want to blow up the BGG API, but if possible with not too
|
||||
many calls, when we scrobble a board game, in order to allow
|
||||
populating the "Expansions" multi select, we should fetch any
|
||||
expansions for the board game when creating it for the first time.
|
||||
|
||||
We should also create a managemnt script to update existing board games.
|
||||
|
||||
** DONE [#A] Board game expansion lookup should be async on URL scrobbles :boardgames:
|
||||
:PROPERTIES:
|
||||
:ID: 968d8dde-f906-cdf0-af4e-b87ce28ddbbb
|
||||
:END:
|
||||
|
||||
* Version 58.8 [1/1]
|
||||
** DONE [#B] Clean up trend templates :trends:templates:
|
||||
:PROPERTIES:
|
||||
:ID: 83237e2c-857b-47c9-c86c-32a5e3f1359d
|
||||
:END:
|
||||
|
||||
* Version 58.7 [2/2]
|
||||
** DONE [#B] Split up chart page between tables and maloja :charts:templates:
|
||||
:PROPERTIES:
|
||||
:ID: 103ab084-2016-cfa4-c677-3c5fdc54cce0
|
||||
:END:
|
||||
** DONE [#A] Fix CI so we don't double run deploys and builds :ci:
|
||||
:PROPERTIES:
|
||||
:ID: 1a93e7cb-b883-aae5-2bd5-fcdd6e16f8ab
|
||||
:END:
|
||||
|
||||
* Version 58.6 [1/1]
|
||||
** DONE [#B] Cleanup commands should check for broken images :metadata:cleanup:
|
||||
:PROPERTIES:
|
||||
:ID: bacce321-73c7-ae1f-bfa7-c3ee517b5441
|
||||
:END:
|
||||
|
||||
* Version 58.5 [1/1]
|
||||
** DONE [#A] The maloja style charts are messed up :templates:charts:
|
||||
:PROPERTIES:
|
||||
:ID: 987397a2-7e74-4eb1-87cc-4c8bbe1c7b23
|
||||
:END:
|
||||
|
||||
* Version 58.4 [2/2]
|
||||
** DONE [#B] Allow people all trends or individual trends :trends:profiles:
|
||||
:PROPERTIES:
|
||||
:ID: 1d081152-abd1-73c2-a625-903565a10c6c
|
||||
:END:
|
||||
** DONE [#A] Fix a bug in board game scorelog data :boardgames:logdata:
|
||||
:PROPERTIES:
|
||||
:ID: 014bab30-13bf-fae7-e678-4666a8d38ae4
|
||||
:END:
|
||||
|
||||
* Version 58.3 [1/1]
|
||||
** DONE [#A] Remove curl-cffi as it doesn't work on FreeBSD :webpages:deps:
|
||||
:PROPERTIES:
|
||||
:ID: 6bc1b0dd-e449-3d32-a176-46451e793e5d
|
||||
:END:
|
||||
|
||||
* Version 58.2 [2/2]
|
||||
** DONE [#B] Add more robust webpage scraping :webpages:metadata:
|
||||
:PROPERTIES:
|
||||
:ID: 84d9bfa5-75c0-0718-764e-379f7456602a
|
||||
:END:
|
||||
** DONE [#B] Time of Day Categories trend :trends:
|
||||
:PROPERTIES:
|
||||
:ID: 6598074f-2290-46db-967b-29f45d30be29
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
Added a "Time of Day Categories" trend that groups scrobbles for Books, Trails,
|
||||
Birding Locations, and Board Games into Early Bird (5-10:59am), Day Jay (11am-6:59pm),
|
||||
and Night Owl (7pm-4:59am) buckets. Shows both overall and per-media-type breakdowns.
|
||||
|
||||
|
||||
* Version 58.1 [1/1]
|
||||
** DONE [#B] Add auto genre tagging for papers :books:papers:metadata:
|
||||
|
||||
19
poetry.lock
generated
19
poetry.lock
generated
@ -967,6 +967,23 @@ prompt-toolkit = ">=3.0.36"
|
||||
[package.extras]
|
||||
testing = ["pytest (>=7.2.1)", "pytest-cov (>=4.0.0)", "tox (>=4.4.3)"]
|
||||
|
||||
[[package]]
|
||||
name = "cloudscraper"
|
||||
version = "1.2.71"
|
||||
description = "A Python module to bypass Cloudflare's anti-bot page."
|
||||
optional = false
|
||||
python-versions = "*"
|
||||
groups = ["main"]
|
||||
files = [
|
||||
{file = "cloudscraper-1.2.71-py2.py3-none-any.whl", hash = "sha256:76f50ca529ed2279e220837befdec892626f9511708e200d48d5bb76ded679b0"},
|
||||
{file = "cloudscraper-1.2.71.tar.gz", hash = "sha256:429c6e8aa6916d5bad5c8a5eac50f3ea53c9ac22616f6cb21b18dcc71517d0d3"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
pyparsing = ">=2.4.7"
|
||||
requests = ">=2.9.2"
|
||||
requests-toolbelt = ">=0.9.1"
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
@ -6812,4 +6829,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt
|
||||
[metadata]
|
||||
lock-version = "2.1"
|
||||
python-versions = ">=3.11,<3.15"
|
||||
content-hash = "3081efc33689e0cb1e2cc12709f2fbfd84a084ee4a083eebba80db2eb80fb0a6"
|
||||
content-hash = "beac677c269bb8618ca802e5f92f7558391d8d26f1b2150f3c8a6d3417848cb1"
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "vrobbler"
|
||||
version = "58.1"
|
||||
version = "59.4"
|
||||
description = ""
|
||||
authors = ["Colin Powell <colin@unbl.ink>"]
|
||||
|
||||
@ -11,6 +11,7 @@ django-extensions = "^3.1.5"
|
||||
python-dateutil = "^2.8.2"
|
||||
python-dotenv = ">=0.20.0,<2"
|
||||
python-json-logger = "^2.0.2"
|
||||
cloudscraper = "^1.2.71"
|
||||
colorlog = "^6.6.0"
|
||||
httpx = "<=0.27.2"
|
||||
djangorestframework = "^3.13.1"
|
||||
|
||||
@ -5,6 +5,7 @@ from boardgames.models import (
|
||||
BoardGameLocation,
|
||||
BoardGamePublisher,
|
||||
BoardGameDesigner,
|
||||
BoardGameVariant,
|
||||
)
|
||||
|
||||
from scrobbles.admin import ScrobbleInline
|
||||
@ -42,6 +43,19 @@ class BoardGameLocationAdmin(admin.ModelAdmin):
|
||||
ordering = ("-created",)
|
||||
|
||||
|
||||
@admin.register(BoardGameVariant)
|
||||
class BoardGameVariantAdmin(admin.ModelAdmin):
|
||||
date_hierarchy = "created"
|
||||
list_display = (
|
||||
"name",
|
||||
"board_game",
|
||||
"uuid",
|
||||
)
|
||||
raw_id_fields = ("board_game",)
|
||||
search_fields = ("name", "board_game__title")
|
||||
ordering = ("-created",)
|
||||
|
||||
|
||||
@admin.register(BoardGame)
|
||||
class BoardGameAdmin(admin.ModelAdmin):
|
||||
date_hierarchy = "created"
|
||||
|
||||
@ -20,6 +20,12 @@ class BoardGameLocationSerializer(serializers.HyperlinkedModelSerializer):
|
||||
fields = "__all__"
|
||||
|
||||
|
||||
class BoardGameVariantSerializer(serializers.HyperlinkedModelSerializer):
|
||||
class Meta:
|
||||
model = models.BoardGameVariant
|
||||
fields = "__all__"
|
||||
|
||||
|
||||
class BoardGameSerializer(serializers.HyperlinkedModelSerializer):
|
||||
class Meta:
|
||||
model = models.BoardGame
|
||||
|
||||
@ -22,6 +22,12 @@ class BoardGameLocationViewSet(viewsets.ModelViewSet):
|
||||
permission_classes = [permissions.IsAuthenticated]
|
||||
|
||||
|
||||
class BoardGameVariantViewSet(viewsets.ModelViewSet):
|
||||
queryset = models.BoardGameVariant.objects.all().order_by("-created")
|
||||
serializer_class = serializers.BoardGameVariantSerializer
|
||||
permission_classes = [permissions.IsAuthenticated]
|
||||
|
||||
|
||||
class BoardGameViewSet(viewsets.ModelViewSet):
|
||||
queryset = models.BoardGame.objects.all().order_by("-created")
|
||||
serializer_class = serializers.BoardGameSerializer
|
||||
|
||||
0
vrobbler/apps/boardgames/management/__init__.py
Normal file
0
vrobbler/apps/boardgames/management/__init__.py
Normal file
@ -0,0 +1,81 @@
|
||||
import logging
|
||||
import time
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = "Refresh board game metadata from BGG (categories→genres, families→tags)"
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument(
|
||||
"--commit",
|
||||
action="store_true",
|
||||
help="Persist changes to the database",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="Update all games even if they already have a published_date",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--batch-size",
|
||||
type=int,
|
||||
default=50,
|
||||
help="Number of games to process per batch (default: 50)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--sleep",
|
||||
type=float,
|
||||
default=1.0,
|
||||
help="Seconds to sleep between API calls (default: 1.0)",
|
||||
)
|
||||
|
||||
def handle(self, *args, **options):
|
||||
from boardgames.models import BoardGame
|
||||
|
||||
commit = options["commit"]
|
||||
force = options["force"]
|
||||
batch_size = options["batch_size"]
|
||||
sleep_secs = options["sleep"]
|
||||
|
||||
qs = BoardGame.objects.exclude(bggeek_id__isnull=True).exclude(bggeek_id="")
|
||||
total = qs.count()
|
||||
self.stdout.write(f"Found {total} board games with BGG IDs")
|
||||
|
||||
if not commit:
|
||||
self.stdout.write(
|
||||
"Dry run — no API calls will be made. Use --commit to run lookups."
|
||||
)
|
||||
return
|
||||
|
||||
enriched = 0
|
||||
skipped = 0
|
||||
|
||||
for batch_num, offset in enumerate(range(0, total, batch_size)):
|
||||
batch = qs[offset : offset + batch_size]
|
||||
for game in batch:
|
||||
try:
|
||||
game.fix_metadata(force_update=force)
|
||||
enriched += 1
|
||||
except Exception as e:
|
||||
self.stdout.write(
|
||||
self.style.WARNING(
|
||||
f" [SKIPPED] {game.title} (BGG {game.bggeek_id}): {e}"
|
||||
)
|
||||
)
|
||||
skipped += 1
|
||||
time.sleep(sleep_secs)
|
||||
|
||||
self.stdout.write(
|
||||
f" Batch {batch_num + 1}: {offset + len(batch)}/{total} — "
|
||||
f"enriched: {enriched}, skipped: {skipped}"
|
||||
)
|
||||
|
||||
self.stdout.write(
|
||||
f"\nResults:\n"
|
||||
f" Games enriched: {enriched}\n"
|
||||
f" Games skipped: {skipped}"
|
||||
)
|
||||
@ -0,0 +1,63 @@
|
||||
import logging
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
|
||||
from boardgames.utils import board_names_to_variants
|
||||
from scrobbles.models import Scrobble
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = "Convert existing board scrobble log 'board' keys to 'variant_ids'"
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument(
|
||||
"--commit",
|
||||
action="store_true",
|
||||
help="Persist changes to the database",
|
||||
)
|
||||
|
||||
def handle(self, *args, **options):
|
||||
commit = options.get("commit", False)
|
||||
|
||||
board_scrobbles = Scrobble.objects.filter(
|
||||
board_game__isnull=False,
|
||||
log__board__isnull=False,
|
||||
).exclude(log__board="")
|
||||
|
||||
total = board_scrobbles.count()
|
||||
self.stdout.write(f"Found {total} scrobbles with a 'board' key in log data")
|
||||
|
||||
if total == 0:
|
||||
return
|
||||
|
||||
updated = 0
|
||||
for scrobble in board_scrobbles.iterator(chunk_size=100):
|
||||
log = scrobble.log
|
||||
board_value = log.pop("board", None)
|
||||
if not board_value:
|
||||
continue
|
||||
|
||||
variant_ids = board_names_to_variants(
|
||||
scrobble.board_game, [board_value]
|
||||
)
|
||||
if variant_ids:
|
||||
log["variant_ids"] = variant_ids
|
||||
|
||||
if commit:
|
||||
Scrobble.objects.filter(pk=scrobble.pk).update(log=log)
|
||||
updated += 1
|
||||
else:
|
||||
updated += 1
|
||||
|
||||
if commit:
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(
|
||||
f"Updated {updated} scrobbles (changes committed)"
|
||||
)
|
||||
)
|
||||
else:
|
||||
self.stdout.write(
|
||||
f"Would update {updated} scrobbles (pass --commit to persist)"
|
||||
)
|
||||
@ -0,0 +1,80 @@
|
||||
import logging
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
|
||||
from boardgames.models import BoardGame
|
||||
from boardgames.sources.bgg import lookup_boardgame_from_bgg
|
||||
from boardgames.utils import fetch_and_link_expansions
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = "Fetch and link expansions for existing board games from BGG"
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument(
|
||||
"--commit",
|
||||
action="store_true",
|
||||
help="Persist changes to the database",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bggeek-id",
|
||||
type=str,
|
||||
help="Only process a single game by BGG ID",
|
||||
)
|
||||
|
||||
def handle(self, *args, **options):
|
||||
commit = options.get("commit", False)
|
||||
bggeek_id = options.get("bggeek_id")
|
||||
|
||||
games = (
|
||||
BoardGame.objects.exclude(bggeek_id__isnull=True)
|
||||
.exclude(bggeek_id="")
|
||||
.exclude(skip_expansions=True)
|
||||
)
|
||||
if bggeek_id:
|
||||
games = games.filter(bggeek_id=bggeek_id)
|
||||
|
||||
total = games.count()
|
||||
self.stdout.write(f"Found {total} board games with BGG IDs")
|
||||
|
||||
if total == 0:
|
||||
return
|
||||
|
||||
updated = 0
|
||||
for game in games.iterator(chunk_size=100):
|
||||
try:
|
||||
data = lookup_boardgame_from_bgg(lookup_id=str(game.bggeek_id))
|
||||
except Exception as e:
|
||||
self.stdout.write(
|
||||
self.style.WARNING(
|
||||
f"Failed to fetch BGG data for {game.title} ({game.bggeek_id}): {e}"
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
expansions = data.get("expansions", [])
|
||||
if not expansions:
|
||||
continue
|
||||
|
||||
if commit:
|
||||
fetch_and_link_expansions(game, expansions)
|
||||
updated += 1
|
||||
self.stdout.write(
|
||||
f" Linked {len(expansions)} expansions to {game.title}"
|
||||
)
|
||||
else:
|
||||
self.stdout.write(
|
||||
f" Would link {len(expansions)} expansions to {game.title}"
|
||||
)
|
||||
updated += 1
|
||||
|
||||
if commit:
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f"Updated {updated} games (changes committed)")
|
||||
)
|
||||
else:
|
||||
self.stdout.write(
|
||||
f"Would update {updated} games (pass --commit to persist)"
|
||||
)
|
||||
62
vrobbler/apps/boardgames/migrations/0016_boardgamevariant.py
Normal file
62
vrobbler/apps/boardgames/migrations/0016_boardgamevariant.py
Normal file
@ -0,0 +1,62 @@
|
||||
# Generated by Django 4.2.29 on 2026-07-02 22:23
|
||||
|
||||
from django.db import migrations, models
|
||||
import django.db.models.deletion
|
||||
import django_extensions.db.fields
|
||||
import uuid
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
("boardgames", "0015_alter_boardgame_genre"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.CreateModel(
|
||||
name="BoardGameVariant",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.BigAutoField(
|
||||
auto_created=True,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
verbose_name="ID",
|
||||
),
|
||||
),
|
||||
(
|
||||
"created",
|
||||
django_extensions.db.fields.CreationDateTimeField(
|
||||
auto_now_add=True, verbose_name="created"
|
||||
),
|
||||
),
|
||||
(
|
||||
"modified",
|
||||
django_extensions.db.fields.ModificationDateTimeField(
|
||||
auto_now=True, verbose_name="modified"
|
||||
),
|
||||
),
|
||||
("name", models.CharField(max_length=255)),
|
||||
("description", models.TextField(blank=True, null=True)),
|
||||
(
|
||||
"uuid",
|
||||
models.UUIDField(
|
||||
blank=True, default=uuid.uuid4, editable=False, null=True
|
||||
),
|
||||
),
|
||||
(
|
||||
"board_game",
|
||||
models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
related_name="variants",
|
||||
to="boardgames.boardgame",
|
||||
),
|
||||
),
|
||||
],
|
||||
options={
|
||||
"get_latest_by": "modified",
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
]
|
||||
@ -0,0 +1,18 @@
|
||||
# Generated by Django 4.2.29 on 2026-07-04 15:28
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
("boardgames", "0016_boardgamevariant"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="boardgame",
|
||||
name="skip_expansions",
|
||||
field=models.BooleanField(default=False),
|
||||
),
|
||||
]
|
||||
@ -71,11 +71,12 @@ class BoardGameLogData(BaseLogData, LongPlayLogData):
|
||||
difficulty: Optional[int] = None
|
||||
solo: Optional[bool] = None
|
||||
two_handed: Optional[bool] = None
|
||||
expansion_ids: Optional[int] = None
|
||||
expansion_ids: Optional[list[int]] = None
|
||||
moves: Optional[list] = None
|
||||
rated: Optional[str] = None
|
||||
speed: Optional[str] = None
|
||||
variant: Optional[str] = None
|
||||
variant_ids: Optional[list[int]] = None
|
||||
lichess_id: Optional[int] = None
|
||||
board: Optional[str] = None
|
||||
rounds: Optional[int] = None
|
||||
@ -92,6 +93,7 @@ class BoardGameLogData(BaseLogData, LongPlayLogData):
|
||||
|
||||
@classmethod
|
||||
def override_fields(cls) -> dict:
|
||||
from boardgames.widgets import VariantSelectWidget
|
||||
from scrobbles.forms import NotesDictField
|
||||
|
||||
fields = {}
|
||||
@ -106,10 +108,30 @@ class BoardGameLogData(BaseLogData, LongPlayLogData):
|
||||
required=False,
|
||||
widget=forms.Select(),
|
||||
),
|
||||
"variant_ids": forms.ModelMultipleChoiceField(
|
||||
queryset=BoardGameVariant.objects.all(),
|
||||
required=False,
|
||||
widget=VariantSelectWidget(attrs={"size": 5}),
|
||||
),
|
||||
"expansion_ids": forms.ModelMultipleChoiceField(
|
||||
queryset=BoardGame.objects.filter(
|
||||
expansion_for_boardgame__isnull=False
|
||||
),
|
||||
required=False,
|
||||
widget=forms.SelectMultiple(attrs={"size": 5}),
|
||||
),
|
||||
}
|
||||
fields.update(custom_fields)
|
||||
return fields
|
||||
|
||||
@cached_property
|
||||
def variants(self) -> list["BoardGameVariant"]:
|
||||
if not self.variant_ids:
|
||||
return []
|
||||
return list(
|
||||
BoardGameVariant.objects.filter(id__in=self.variant_ids)
|
||||
)
|
||||
|
||||
@cached_property
|
||||
def location(self):
|
||||
if not self.location_id:
|
||||
@ -120,7 +142,12 @@ class BoardGameLogData(BaseLogData, LongPlayLogData):
|
||||
def player_log(self) -> str:
|
||||
if self.players:
|
||||
return ", ".join(
|
||||
[BoardGameScoreLogData(**player).__str__() for player in self.players]
|
||||
[
|
||||
BoardGameScoreLogData(
|
||||
**{k: v for k, v in player.items() if k in BoardGameScoreLogData.__dataclass_fields__}
|
||||
).__str__()
|
||||
for player in self.players
|
||||
]
|
||||
)
|
||||
return ""
|
||||
|
||||
@ -130,13 +157,21 @@ class BoardGameLogData(BaseLogData, LongPlayLogData):
|
||||
if self.board:
|
||||
html_parts.append(f'<div class="boardgame-board">{self.board}</div>')
|
||||
|
||||
if self.variants:
|
||||
variant_names = ", ".join(v.name for v in self.variants)
|
||||
html_parts.append(
|
||||
f'<div class="boardgame-variants">Variants: {variant_names}</div>'
|
||||
)
|
||||
|
||||
if self.location:
|
||||
html_parts.append(f'<div class="boardgame-location">{self.location}</div>')
|
||||
|
||||
if self.players:
|
||||
players_html = []
|
||||
for player_data in self.players:
|
||||
player = BoardGameScoreLogData(**player_data)
|
||||
player = BoardGameScoreLogData(
|
||||
**{k: v for k, v in player_data.items() if k in BoardGameScoreLogData.__dataclass_fields__}
|
||||
)
|
||||
player_info = player.name
|
||||
if player.score:
|
||||
player_info += f" ({player.score})"
|
||||
@ -265,6 +300,7 @@ class BoardGame(ScrobblableMixin):
|
||||
expansion_for_boardgame = models.ForeignKey(
|
||||
"self", **BNULL, on_delete=models.DO_NOTHING
|
||||
)
|
||||
skip_expansions = models.BooleanField(default=False)
|
||||
|
||||
@property
|
||||
def subtitle(self) -> str:
|
||||
@ -299,18 +335,29 @@ class BoardGame(ScrobblableMixin):
|
||||
if not data:
|
||||
data = lookup_boardgame_from_bgg(str(self.bggeek_id))
|
||||
|
||||
cover_url = data.pop("cover_url")
|
||||
year = data.pop("year_published")
|
||||
publisher_name = data.pop("publisher_name")
|
||||
expansions = data.pop("expansions", [])
|
||||
cover_url = data.pop("cover_url", "")
|
||||
year_published = data.pop("year_published", None)
|
||||
if year_published is None:
|
||||
year_published = data.pop("published_year", None)
|
||||
publisher_name = data.pop("publisher_name", "")
|
||||
|
||||
if year:
|
||||
data["published_year"] = int(year)
|
||||
if year_published:
|
||||
data["published_year"] = int(year_published)
|
||||
|
||||
if not data["min_players"]:
|
||||
data.pop("min_players")
|
||||
if not data["min_players"]:
|
||||
data.pop("max_players")
|
||||
|
||||
# Pop extra BGG metadata that isn't a model field
|
||||
categories = data.pop("categories", [])
|
||||
families = data.pop("families", [])
|
||||
data.pop("mechanics", None)
|
||||
data.pop("designers", None)
|
||||
data.pop("publishers", None)
|
||||
data.pop("publisher", None)
|
||||
|
||||
# Fun trick for updating all fields at once
|
||||
BoardGame.objects.filter(pk=self.id).update(**data)
|
||||
self.refresh_from_db()
|
||||
@ -322,10 +369,20 @@ class BoardGame(ScrobblableMixin):
|
||||
) = BoardGamePublisher.objects.get_or_create(name=publisher_name)
|
||||
self.save()
|
||||
|
||||
for cat in categories:
|
||||
self.genre.add(cat.strip())
|
||||
for fam in families:
|
||||
self.tags.add(fam.strip())
|
||||
|
||||
# Go get cover image if the URL is present
|
||||
if cover_url and not self.cover:
|
||||
self.save_image_from_url(cover_url)
|
||||
|
||||
from boardgames.utils import fetch_and_link_expansions
|
||||
|
||||
if not self.skip_expansions:
|
||||
fetch_and_link_expansions(self, expansions)
|
||||
|
||||
def save_image_from_url(self, url):
|
||||
headers = {"User-Agent": "Vrobbler 0.11.12"}
|
||||
r = requests.get(url, headers=headers)
|
||||
@ -334,7 +391,12 @@ class BoardGame(ScrobblableMixin):
|
||||
self.cover.save(fname, ContentFile(r.content), save=True)
|
||||
|
||||
@classmethod
|
||||
def find_or_create(cls, lookup_id: str, data: dict[str, Any] = {}) -> "BoardGame":
|
||||
def find_or_create(
|
||||
cls,
|
||||
lookup_id: str,
|
||||
data: dict[str, Any] = {},
|
||||
defer_expansions: bool = False,
|
||||
) -> "BoardGame":
|
||||
"""Given a Lookup ID (either BGG or BGA ID), return a board game object"""
|
||||
game = cls.objects.filter(bggeek_id=lookup_id).first()
|
||||
if not game:
|
||||
@ -357,16 +419,19 @@ class BoardGame(ScrobblableMixin):
|
||||
else:
|
||||
bgg_data = lookup_boardgame_from_bgg(title=lookup_id)
|
||||
|
||||
expansions = bgg_data.pop("expansions", [])
|
||||
mechanics = bgg_data.pop("mechanics", [])
|
||||
designers = bgg_data.pop("designers", [])
|
||||
categories = bgg_data.pop("categories", [])
|
||||
families = bgg_data.pop("families", [])
|
||||
publishers = bgg_data.pop("publishers", [])
|
||||
publisher = bgg_data.pop("publisher", [])
|
||||
cover_url = bgg_data.pop("cover_url")
|
||||
cover_url = bgg_data.pop("cover_url") or ""
|
||||
|
||||
game = cls.objects.create(**bgg_data)
|
||||
|
||||
game.save_image_from_url(cover_url)
|
||||
if cover_url:
|
||||
game.save_image_from_url(cover_url)
|
||||
game.cooperative = data.get("cooperative", False)
|
||||
game.highest_wins = data.get("highestWins", True)
|
||||
game.no_points = data.get("noPoints", False)
|
||||
@ -375,6 +440,18 @@ class BoardGame(ScrobblableMixin):
|
||||
if publisher:
|
||||
publisher, _ = BoardGamePublisher.objects.get_or_create(name=publisher)
|
||||
game.publisher = publisher
|
||||
|
||||
skip_expansions = (
|
||||
game.bggeek_id is not None
|
||||
and str(game.bggeek_id).isdigit()
|
||||
and int(game.bggeek_id) in settings.SKIP_AUTO_EXPANSION_DOWNLOAD
|
||||
) or any(
|
||||
c == "Collectible Card Games" for c in categories
|
||||
)
|
||||
|
||||
if skip_expansions:
|
||||
game.skip_expansions = True
|
||||
|
||||
game.save()
|
||||
|
||||
if designers:
|
||||
@ -389,4 +466,33 @@ class BoardGame(ScrobblableMixin):
|
||||
publisher, _ = BoardGamePublisher.objects.get_or_create(name=name)
|
||||
game.publishers.add(publisher)
|
||||
|
||||
for cat in categories:
|
||||
game.genre.add(cat.strip())
|
||||
for fam in families:
|
||||
game.tags.add(fam.strip())
|
||||
|
||||
if expansions and not game.skip_expansions:
|
||||
if defer_expansions:
|
||||
from boardgames.tasks import fetch_board_game_expansions
|
||||
|
||||
fetch_board_game_expansions.delay(game.id, expansions)
|
||||
else:
|
||||
from boardgames.utils import fetch_and_link_expansions
|
||||
|
||||
fetch_and_link_expansions(game, expansions)
|
||||
|
||||
return game
|
||||
|
||||
|
||||
class BoardGameVariant(TimeStampedModel):
|
||||
name = models.CharField(max_length=255)
|
||||
board_game = models.ForeignKey(
|
||||
BoardGame,
|
||||
on_delete=models.CASCADE,
|
||||
related_name="variants",
|
||||
)
|
||||
description = models.TextField(**BNULL)
|
||||
uuid = models.UUIDField(default=uuid4, editable=False, **BNULL)
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.name} ({self.board_game.title})"
|
||||
|
||||
@ -32,9 +32,13 @@ def lookup_boardgame_from_bgg(
|
||||
)
|
||||
game_dict["mechanics"] = game.mechanics
|
||||
game_dict["categories"] = game.categories
|
||||
game_dict["families"] = game.families
|
||||
game_dict["designers"] = game.designers
|
||||
game_dict["publishers"] = game.publishers
|
||||
if game.publishers:
|
||||
game_dict["publisher"] = game.publishers[0]
|
||||
game_dict["expansions"] = [
|
||||
{"id": exp.id, "name": exp.name} for exp in game.expansions
|
||||
]
|
||||
|
||||
return game_dict
|
||||
|
||||
29
vrobbler/apps/boardgames/tasks.py
Normal file
29
vrobbler/apps/boardgames/tasks.py
Normal file
@ -0,0 +1,29 @@
|
||||
import logging
|
||||
|
||||
from celery import shared_task
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@shared_task
|
||||
def fetch_board_game_expansions(board_game_id, expansions_data):
|
||||
from boardgames.models import BoardGame
|
||||
from boardgames.utils import fetch_and_link_expansions
|
||||
|
||||
game = BoardGame.objects.filter(id=board_game_id).first()
|
||||
if not game:
|
||||
logger.warning(
|
||||
"Board game not found for expansion linking",
|
||||
extra={"board_game_id": board_game_id},
|
||||
)
|
||||
return
|
||||
|
||||
fetch_and_link_expansions(game, expansions_data)
|
||||
logger.info(
|
||||
"Linked expansions for board game",
|
||||
extra={
|
||||
"board_game_id": board_game_id,
|
||||
"title": game.title,
|
||||
"count": len(expansions_data),
|
||||
},
|
||||
)
|
||||
@ -0,0 +1,97 @@
|
||||
<select name="{{ widget.name }}"{% include "django/forms/widgets/attrs.html" %}>
|
||||
{% for group_name, group_choices, group_index in widget.optgroups %}
|
||||
{% for option in group_choices %}
|
||||
{% include option.template_name with widget=option %}
|
||||
{% endfor %}
|
||||
{% endfor %}
|
||||
</select>
|
||||
<button type="button" class="btn btn-sm btn-outline-secondary mt-1" data-bs-toggle="modal" data-bs-target="#addVariantModal">
|
||||
+ Add variant
|
||||
</button>
|
||||
|
||||
<div class="modal fade" id="addVariantModal" tabindex="-1" aria-hidden="true">
|
||||
<div class="modal-dialog modal-sm">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h5 class="modal-title">Add Variant</h5>
|
||||
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div class="mb-3">
|
||||
<label for="newVariantName" class="form-label">Name</label>
|
||||
<input type="text" class="form-control" id="newVariantName" placeholder="e.g. Map A">
|
||||
</div>
|
||||
<div class="mb-3">
|
||||
<label for="newVariantDescription" class="form-label">Description (optional)</label>
|
||||
<input type="text" class="form-control" id="newVariantDescription">
|
||||
</div>
|
||||
<p class="text-danger d-none" id="variantError"></p>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Cancel</button>
|
||||
<button type="button" class="btn btn-primary" id="saveVariantBtn">Add</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
(function() {
|
||||
var select = document.getElementById('{{ widget.attrs.id }}');
|
||||
if (!select) return;
|
||||
|
||||
var saveBtn = document.getElementById('saveVariantBtn');
|
||||
if (!saveBtn) return;
|
||||
|
||||
var modalEl = document.getElementById('addVariantModal');
|
||||
var nameInput = document.getElementById('newVariantName');
|
||||
var descInput = document.getElementById('newVariantDescription');
|
||||
var errorEl = document.getElementById('variantError');
|
||||
|
||||
saveBtn.addEventListener('click', function() {
|
||||
var name = nameInput.value.trim();
|
||||
if (!name) return;
|
||||
|
||||
var boardGameId = select.getAttribute('data-board-game-id');
|
||||
var ajaxUrl = select.getAttribute('data-ajax-url');
|
||||
var csrfToken = document.querySelector('[name=csrfmiddlewaretoken]');
|
||||
if (!csrfToken) return;
|
||||
|
||||
errorEl.classList.add('d-none');
|
||||
|
||||
fetch(ajaxUrl, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/x-www-form-urlencoded',
|
||||
'X-CSRFToken': csrfToken.value,
|
||||
},
|
||||
body: new URLSearchParams({
|
||||
name: name,
|
||||
description: descInput.value.trim(),
|
||||
board_game_id: boardGameId,
|
||||
}),
|
||||
})
|
||||
.then(function(r) { return r.json(); })
|
||||
.then(function(data) {
|
||||
if (data.error) {
|
||||
errorEl.textContent = data.error;
|
||||
errorEl.classList.remove('d-none');
|
||||
return;
|
||||
}
|
||||
var opt = document.createElement('option');
|
||||
opt.value = data.id;
|
||||
opt.textContent = data.name;
|
||||
opt.selected = true;
|
||||
select.appendChild(opt);
|
||||
var modal = bootstrap.Modal.getInstance(modalEl);
|
||||
if (modal) modal.hide();
|
||||
nameInput.value = '';
|
||||
descInput.value = '';
|
||||
})
|
||||
.catch(function() {
|
||||
errorEl.textContent = 'Failed to create variant';
|
||||
errorEl.classList.remove('d-none');
|
||||
});
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
0
vrobbler/apps/boardgames/tests/__init__.py
Normal file
0
vrobbler/apps/boardgames/tests/__init__.py
Normal file
226
vrobbler/apps/boardgames/tests/test_models.py
Normal file
226
vrobbler/apps/boardgames/tests/test_models.py
Normal file
@ -0,0 +1,226 @@
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from django.contrib.auth import get_user_model
|
||||
|
||||
from boardgames.models import BoardGame, BoardGameVariant
|
||||
|
||||
User = get_user_model()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_game_variant_creation():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
variant = BoardGameVariant.objects.create(
|
||||
name="Test Variant",
|
||||
board_game=game,
|
||||
description="A test variant",
|
||||
)
|
||||
assert variant.name == "Test Variant"
|
||||
assert variant.board_game == game
|
||||
assert variant.description == "A test variant"
|
||||
assert variant.uuid is not None
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_game_variant_str():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
variant = BoardGameVariant.objects.create(
|
||||
name="Test Variant",
|
||||
board_game=game,
|
||||
)
|
||||
assert str(variant) == "Test Variant (Test Game)"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_game_variant_optional_description():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
variant = BoardGameVariant.objects.create(
|
||||
name="Test Variant",
|
||||
board_game=game,
|
||||
)
|
||||
assert variant.description is None
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_game_variant_related_name():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
variant1 = BoardGameVariant.objects.create(
|
||||
name="Variant 1",
|
||||
board_game=game,
|
||||
)
|
||||
variant2 = BoardGameVariant.objects.create(
|
||||
name="Variant 2",
|
||||
board_game=game,
|
||||
)
|
||||
assert list(game.variants.all()) == [variant1, variant2]
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_game_variant_cascade_delete():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
variant = BoardGameVariant.objects.create(
|
||||
name="Test Variant",
|
||||
board_game=game,
|
||||
)
|
||||
game.delete()
|
||||
assert BoardGameVariant.objects.count() == 0
|
||||
|
||||
|
||||
def _mock_bgg_game(bggeek_id, title, expansions=None):
|
||||
"""Build a fake BGG game object shape used by lookup_boardgame_from_bgg."""
|
||||
class FakeGame:
|
||||
id = bggeek_id
|
||||
name = title
|
||||
description = f"Description of {title}"
|
||||
yearpublished = 2020
|
||||
image = "https://example.com/cover.jpg"
|
||||
minplayers = 1
|
||||
maxplayers = 4
|
||||
minage = 8
|
||||
rating_average = 7.5
|
||||
bgg_rank = 100
|
||||
playingtime = 60
|
||||
mechanics = []
|
||||
categories = []
|
||||
families = []
|
||||
designers = []
|
||||
publishers = []
|
||||
|
||||
@property
|
||||
def expansions(self):
|
||||
if expansions is None:
|
||||
return []
|
||||
return expansions
|
||||
|
||||
return FakeGame()
|
||||
|
||||
|
||||
def _mock_bgg_client(return_game):
|
||||
"""Return a callable that creates a fake BGGClient instance."""
|
||||
class FakeBGGClient:
|
||||
def __init__(self, access_token=None):
|
||||
pass
|
||||
|
||||
def game(self, game_id=None, name=None):
|
||||
return return_game
|
||||
|
||||
return FakeBGGClient()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.models.requests.get")
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_find_or_create_links_expansions(mock_bgg, mock_get):
|
||||
exp1 = type("Thing", (), {"id": 200, "name": "Expansion 1"})()
|
||||
exp2 = type("Thing", (), {"id": 201, "name": "Expansion 2"})()
|
||||
base_game = _mock_bgg_game("100", "Base Game", expansions=[exp1, exp2])
|
||||
exp1_game = _mock_bgg_game("200", "Expansion 1")
|
||||
exp2_game = _mock_bgg_game("201", "Expansion 2")
|
||||
mock_bgg.side_effect = [
|
||||
_mock_bgg_client(base_game),
|
||||
_mock_bgg_client(exp1_game),
|
||||
_mock_bgg_client(exp2_game),
|
||||
]
|
||||
mock_get.return_value.status_code = 200
|
||||
mock_get.return_value.content = b"fake_image_data"
|
||||
|
||||
game = BoardGame.find_or_create("100")
|
||||
assert game.title == "Base Game"
|
||||
|
||||
expansions = BoardGame.objects.filter(expansion_for_boardgame=game)
|
||||
assert expansions.count() == 2
|
||||
assert {e.title for e in expansions} == {"Expansion 1", "Expansion 2"}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_find_or_create_skips_expansions_for_existing_game(mock_bgg):
|
||||
game = BoardGame.objects.create(title="Base Game", bggeek_id="100")
|
||||
mock_bgg.assert_not_called()
|
||||
|
||||
result = BoardGame.find_or_create("100")
|
||||
assert result == game
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.models.requests.get")
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_fetch_and_link_expansions(mock_bgg, mock_get):
|
||||
game = BoardGame.objects.create(title="Base Game", bggeek_id="100")
|
||||
exp_game = _mock_bgg_game("200", "Expansion 1")
|
||||
mock_bgg.side_effect = [_mock_bgg_client(exp_game)]
|
||||
mock_get.return_value.status_code = 200
|
||||
mock_get.return_value.content = b"fake_image_data"
|
||||
|
||||
from boardgames.utils import fetch_and_link_expansions
|
||||
|
||||
fetch_and_link_expansions(game, [{"id": 200, "name": "Expansion 1"}])
|
||||
|
||||
expansion = BoardGame.objects.get(bggeek_id="200")
|
||||
assert expansion.expansion_for_boardgame == game
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.models.requests.get")
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_fix_metadata_links_expansions(mock_bgg, mock_get):
|
||||
game = BoardGame.objects.create(title="Base Game", bggeek_id="100")
|
||||
exp_thing = type("Thing", (), {"id": 200, "name": "Expansion 1"})()
|
||||
base_with_exp = _mock_bgg_game("100", "Base Game", expansions=[exp_thing])
|
||||
exp_game = _mock_bgg_game("200", "Expansion 1")
|
||||
# first call = fix_metadata -> lookup_boardgame_from_bgg(lookup_id='100')
|
||||
# second call = find_or_create inside fetch_and_link_expansions for expansion
|
||||
mock_bgg.side_effect = [
|
||||
_mock_bgg_client(base_with_exp),
|
||||
_mock_bgg_client(exp_game),
|
||||
]
|
||||
mock_get.return_value.status_code = 200
|
||||
mock_get.return_value.content = b"fake_image_data"
|
||||
|
||||
game.fix_metadata(force_update=True)
|
||||
|
||||
expansion = BoardGame.objects.get(bggeek_id="200")
|
||||
assert expansion.expansion_for_boardgame == game
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_management_command_fetch_expansions_dry_run(mock_bgg, capsys):
|
||||
from django.core.management import call_command
|
||||
|
||||
game = BoardGame.objects.create(title="Base Game", bggeek_id="100")
|
||||
exp_thing = type("Thing", (), {"id": 200, "name": "Expansion 1"})()
|
||||
base_with_exp = _mock_bgg_game("100", "Base Game", expansions=[exp_thing])
|
||||
mock_bgg.side_effect = [_mock_bgg_client(base_with_exp)]
|
||||
|
||||
call_command("fetch_expansions")
|
||||
captured = capsys.readouterr()
|
||||
assert "Would link 1 expansions" in captured.out
|
||||
|
||||
assert BoardGame.objects.filter(bggeek_id="200").count() == 0
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("boardgames.models.requests.get")
|
||||
@patch("boardgames.sources.bgg.BGGClient")
|
||||
def test_management_command_fetch_expansions_commit(mock_bgg, mock_get, capsys):
|
||||
from django.core.management import call_command
|
||||
|
||||
game = BoardGame.objects.create(title="Base Game", bggeek_id="100")
|
||||
exp_thing = type("Thing", (), {"id": 200, "name": "Expansion 1"})()
|
||||
exp_game = _mock_bgg_game("200", "Expansion 1")
|
||||
base_with_exp = _mock_bgg_game("100", "Base Game", expansions=[exp_thing])
|
||||
mock_bgg.side_effect = [
|
||||
_mock_bgg_client(base_with_exp),
|
||||
_mock_bgg_client(exp_game),
|
||||
]
|
||||
mock_get.return_value.status_code = 200
|
||||
mock_get.return_value.content = b"fake_image_data"
|
||||
|
||||
call_command("fetch_expansions", commit=True)
|
||||
captured = capsys.readouterr()
|
||||
assert "Updated 1 games" in captured.out
|
||||
|
||||
expansion = BoardGame.objects.get(bggeek_id="200")
|
||||
assert expansion.expansion_for_boardgame == game
|
||||
106
vrobbler/apps/boardgames/tests/test_utils.py
Normal file
106
vrobbler/apps/boardgames/tests/test_utils.py
Normal file
@ -0,0 +1,106 @@
|
||||
import pytest
|
||||
from django.contrib.auth import get_user_model
|
||||
|
||||
from boardgames.models import BoardGame, BoardGameVariant
|
||||
from boardgames.utils import board_names_to_variants
|
||||
from scrobbles.models import Scrobble
|
||||
|
||||
User = get_user_model()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_creates_variant():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
ids = board_names_to_variants(game, ["Map A"])
|
||||
assert len(ids) == 1
|
||||
variant = BoardGameVariant.objects.get(id=ids[0])
|
||||
assert variant.name == "Map A"
|
||||
assert variant.board_game == game
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_reuses_existing():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
existing = BoardGameVariant.objects.create(
|
||||
name="Map A", board_game=game
|
||||
)
|
||||
ids = board_names_to_variants(game, ["Map A"])
|
||||
assert len(ids) == 1
|
||||
assert ids[0] == existing.id
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_multiple_names():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
ids = board_names_to_variants(game, ["Map A", "Map B"])
|
||||
assert len(ids) == 2
|
||||
names = set(BoardGameVariant.objects.filter(id__in=ids).values_list("name", flat=True))
|
||||
assert names == {"Map A", "Map B"}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_splits_fullwidth_slash():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
ids = board_names_to_variants(game, ["Map A/Map B"])
|
||||
assert len(ids) == 2
|
||||
names = set(BoardGameVariant.objects.filter(id__in=ids).values_list("name", flat=True))
|
||||
assert names == {"Map A", "Map B"}
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_skips_empty_parts():
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
ids = board_names_to_variants(game, ["/Map A/"])
|
||||
assert len(ids) == 1
|
||||
assert BoardGameVariant.objects.get(id=ids[0]).name == "Map A"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_board_names_to_variants_different_games_independent():
|
||||
game1 = BoardGame.objects.create(title="Game 1")
|
||||
game2 = BoardGame.objects.create(title="Game 2")
|
||||
ids1 = board_names_to_variants(game1, ["Map A"])
|
||||
ids2 = board_names_to_variants(game2, ["Map A"])
|
||||
assert ids1 != ids2
|
||||
assert BoardGameVariant.objects.count() == 2
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_management_command_dry_run(capsys):
|
||||
from django.core.management import call_command
|
||||
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
user = User.objects.create(username="tester")
|
||||
scrobble = Scrobble.objects.create(
|
||||
user=user,
|
||||
board_game=game,
|
||||
log={"board": "Map A"},
|
||||
)
|
||||
|
||||
call_command("convert_board_to_variants")
|
||||
captured = capsys.readouterr()
|
||||
assert "Would update 1 scrobbles" in captured.out
|
||||
|
||||
scrobble.refresh_from_db()
|
||||
assert "board" in scrobble.log
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_management_command_commit():
|
||||
from django.core.management import call_command
|
||||
|
||||
game = BoardGame.objects.create(title="Test Game")
|
||||
user = User.objects.create(username="tester")
|
||||
scrobble = Scrobble.objects.create(
|
||||
user=user,
|
||||
board_game=game,
|
||||
log={"board": "Map A"},
|
||||
)
|
||||
|
||||
call_command("convert_board_to_variants", commit=True)
|
||||
|
||||
scrobble.refresh_from_db()
|
||||
assert "board" not in scrobble.log
|
||||
assert "variant_ids" in scrobble.log
|
||||
variant = BoardGameVariant.objects.get(board_game=game, name="Map A")
|
||||
assert variant.id in scrobble.log["variant_ids"]
|
||||
@ -20,4 +20,9 @@ urlpatterns = [
|
||||
views.BoardGamePublisherDetailView.as_view(),
|
||||
name="publisher_detail",
|
||||
),
|
||||
path(
|
||||
"variants/ajax-create/",
|
||||
views.ajax_create_variant,
|
||||
name="ajax-create-variant",
|
||||
),
|
||||
]
|
||||
|
||||
@ -0,0 +1,69 @@
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from boardgames.models import BoardGame, BoardGameVariant
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def board_names_to_variants(
|
||||
board_game: BoardGame, board_names: list[str]
|
||||
) -> list[int]:
|
||||
"""Given a board game and a list of board/scenario names, find or create
|
||||
BoardGameVariant records and return their IDs.
|
||||
|
||||
Splits each name on the full-width slash ``/`` so that a single field
|
||||
containing ``Map A/Map B`` produces two separate variants.
|
||||
"""
|
||||
variant_ids: list[int] = []
|
||||
for raw_name in board_names:
|
||||
for part in raw_name.split("/"):
|
||||
name = part.strip()
|
||||
if not name:
|
||||
continue
|
||||
variant, was_created = BoardGameVariant.objects.get_or_create(
|
||||
board_game=board_game,
|
||||
name=name,
|
||||
)
|
||||
logger.debug(
|
||||
"Resolved board variant",
|
||||
extra={
|
||||
"board_game_id": board_game.id,
|
||||
"variant_name": name,
|
||||
"variant_id": variant.id,
|
||||
"was_created": was_created,
|
||||
},
|
||||
)
|
||||
variant_ids.append(variant.id)
|
||||
return variant_ids
|
||||
|
||||
|
||||
def fetch_and_link_expansions(
|
||||
board_game: BoardGame, expansions_data: list[dict[str, Any]]
|
||||
) -> None:
|
||||
"""Given a board game and a list of expansion dicts (with 'id' and 'name'),
|
||||
find or create each expansion BoardGame and link it via expansion_for_boardgame.
|
||||
"""
|
||||
if board_game.skip_expansions:
|
||||
logger.info(
|
||||
"Skipping expansion fetch for board game with skip_expansions=True",
|
||||
extra={"board_game_id": board_game.id, "title": board_game.title},
|
||||
)
|
||||
return
|
||||
|
||||
for exp_data in expansions_data:
|
||||
exp_id = exp_data.get("id")
|
||||
if not exp_id:
|
||||
continue
|
||||
expansion = BoardGame.find_or_create(str(exp_id))
|
||||
if expansion and expansion.id != board_game.id:
|
||||
expansion.expansion_for_boardgame = board_game
|
||||
expansion.save(update_fields=["expansion_for_boardgame"])
|
||||
logger.info(
|
||||
"Linked expansion to board game",
|
||||
extra={
|
||||
"board_game_id": board_game.id,
|
||||
"expansion_id": expansion.id,
|
||||
"expansion_name": expansion.title,
|
||||
},
|
||||
)
|
||||
|
||||
@ -1,6 +1,9 @@
|
||||
import datetime
|
||||
|
||||
from django.http import JsonResponse
|
||||
from django.utils import timezone
|
||||
from django.views import generic
|
||||
from django.views.decorators.http import require_POST
|
||||
from boardgames.models import BoardGame, BoardGameDesigner, BoardGamePublisher
|
||||
from scrobbles.models import Scrobble
|
||||
from scrobbles.views import (
|
||||
@ -10,6 +13,38 @@ from scrobbles.views import (
|
||||
)
|
||||
|
||||
|
||||
@require_POST
|
||||
def ajax_create_variant(request):
|
||||
name = request.POST.get("name", "").strip()
|
||||
board_game_id = request.POST.get("board_game_id")
|
||||
description = request.POST.get("description", "").strip()
|
||||
|
||||
if not name or not board_game_id:
|
||||
return JsonResponse({"error": "Name and board game are required"}, status=400)
|
||||
|
||||
try:
|
||||
board_game_id = int(board_game_id)
|
||||
except (ValueError, TypeError):
|
||||
return JsonResponse({"error": "Invalid board game"}, status=400)
|
||||
|
||||
from boardgames.models import BoardGameVariant
|
||||
|
||||
variant = BoardGameVariant.objects.filter(
|
||||
name__iexact=name,
|
||||
board_game_id=board_game_id,
|
||||
).first()
|
||||
if variant:
|
||||
return JsonResponse({"id": variant.id, "name": variant.name})
|
||||
|
||||
variant = BoardGameVariant.objects.create(
|
||||
name=name,
|
||||
board_game_id=board_game_id,
|
||||
description=description or None,
|
||||
)
|
||||
|
||||
return JsonResponse({"id": variant.id, "name": variant.name})
|
||||
|
||||
|
||||
class BoardGameListView(ScrobbleableListView):
|
||||
model = BoardGame
|
||||
|
||||
|
||||
9
vrobbler/apps/boardgames/widgets.py
Normal file
9
vrobbler/apps/boardgames/widgets.py
Normal file
@ -0,0 +1,9 @@
|
||||
from django import forms
|
||||
|
||||
|
||||
class VariantSelectWidget(forms.SelectMultiple):
|
||||
template_name = "boardgames/widgets/variant_select.html"
|
||||
|
||||
def get_context(self, name, value, attrs):
|
||||
context = super().get_context(name, value, attrs)
|
||||
return context
|
||||
@ -18,8 +18,17 @@ MISSING_ALL = [
|
||||
"publish_year",
|
||||
]
|
||||
|
||||
def _cover_missing_or_broken(book) -> bool:
|
||||
if not bool(book.cover):
|
||||
return True
|
||||
try:
|
||||
return not book.cover.storage.exists(book.cover.name)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
|
||||
MISSING_GROUPS = {
|
||||
"cover": lambda b: not bool(b.cover),
|
||||
"cover": _cover_missing_or_broken,
|
||||
"summary": lambda b: not b.summary,
|
||||
"isbn": lambda b: not b.isbn_13 and not b.isbn_10,
|
||||
"pages": lambda b: b.pages is None,
|
||||
|
||||
@ -3,6 +3,7 @@ from charts.views import (
|
||||
BirdsChartView,
|
||||
ChartDetailView,
|
||||
ChartRecordView,
|
||||
MalojaChartsView,
|
||||
SpotifyTracksView,
|
||||
)
|
||||
from django.urls import path
|
||||
@ -11,6 +12,7 @@ app_name = "charts"
|
||||
|
||||
urlpatterns = [
|
||||
path("charts/", ChartRecordView.as_view(), name="charts-home"),
|
||||
path("charts/maloja/", MalojaChartsView.as_view(), name="maloja-charts"),
|
||||
path("charts/spotify/", SpotifyTracksView.as_view(), name="spotify-tracks"),
|
||||
path("charts/bandcamp/", BandcampTracksView.as_view(), name="bandcamp-tracks"),
|
||||
path("charts/birds/", BirdsChartView.as_view(), name="birds-chart"),
|
||||
|
||||
@ -114,175 +114,13 @@ class ChartRecordView(TemplateView):
|
||||
context["current_week"] = current_week
|
||||
context["current_day"] = current_day
|
||||
|
||||
context["chart_keys"] = {
|
||||
"today": "Today",
|
||||
"week": "This Week",
|
||||
"month": "This Month",
|
||||
"year": "This Year",
|
||||
"all": "All Time",
|
||||
}
|
||||
|
||||
context["maloja_charts"] = {
|
||||
"artist": {
|
||||
"today": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"artist",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
day=current_day,
|
||||
)
|
||||
),
|
||||
"week": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"artist",
|
||||
year=current_year,
|
||||
week=current_week,
|
||||
)
|
||||
),
|
||||
"month": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"artist",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
)
|
||||
),
|
||||
"year": list(
|
||||
self.get_charts_for_period(user, "artist", year=current_year)
|
||||
),
|
||||
"all": list(self.get_charts_for_period(user, "artist")),
|
||||
},
|
||||
"album": {
|
||||
"today": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"album",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
day=current_day,
|
||||
)
|
||||
),
|
||||
"week": list(
|
||||
self.get_charts_for_period(
|
||||
user, "album", year=current_year, week=current_week
|
||||
)
|
||||
),
|
||||
"month": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"album",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
)
|
||||
),
|
||||
"year": list(
|
||||
self.get_charts_for_period(user, "album", year=current_year)
|
||||
),
|
||||
"all": list(self.get_charts_for_period(user, "album")),
|
||||
},
|
||||
"tv_series": {
|
||||
"today": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"tv_series",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
day=current_day,
|
||||
)
|
||||
),
|
||||
"week": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"tv_series",
|
||||
year=current_year,
|
||||
week=current_week,
|
||||
)
|
||||
),
|
||||
"month": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"tv_series",
|
||||
year=current_year,
|
||||
month=current_month,
|
||||
)
|
||||
),
|
||||
"year": list(
|
||||
self.get_charts_for_period(user, "tv_series", year=current_year)
|
||||
),
|
||||
"all": list(self.get_charts_for_period(user, "tv_series")),
|
||||
},
|
||||
}
|
||||
|
||||
if not date_param:
|
||||
context["period"] = "current"
|
||||
context["year"] = current_year
|
||||
context["month"] = current_month
|
||||
context["month_name"] = calendar.month_name[current_month]
|
||||
context["week"] = current_week
|
||||
context["day"] = current_day
|
||||
|
||||
context["charts"] = {
|
||||
"artist": list(
|
||||
self.get_charts_for_period(
|
||||
user, "artist", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"album": list(
|
||||
self.get_charts_for_period(
|
||||
user, "album", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"track": list(
|
||||
self.get_charts_for_period(
|
||||
user, "track", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"tv_series": list(
|
||||
self.get_charts_for_period(
|
||||
user, "tv_series", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"video": list(
|
||||
self.get_charts_for_period(
|
||||
user, "video", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"board_game": list(
|
||||
self.get_charts_for_period(
|
||||
user, "board_game", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"book": list(
|
||||
self.get_charts_for_period(
|
||||
user, "book", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"food": list(
|
||||
self.get_charts_for_period(
|
||||
user, "food", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"podcast": list(
|
||||
self.get_charts_for_period(
|
||||
user, "podcast", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
"trail": list(
|
||||
self.get_charts_for_period(
|
||||
user, "trail", year=current_year, limit=20
|
||||
)
|
||||
),
|
||||
}
|
||||
else:
|
||||
# Resolve date parameters
|
||||
if date_param:
|
||||
parts = date_param.split("-")
|
||||
year = int(parts[0])
|
||||
|
||||
week = None
|
||||
month = None
|
||||
day = None
|
||||
|
||||
if len(parts) >= 2 and parts[1].startswith("W"):
|
||||
week = int(parts[1].lstrip("W"))
|
||||
elif len(parts) >= 2 and parts[1]:
|
||||
@ -290,20 +128,17 @@ class ChartRecordView(TemplateView):
|
||||
month = int(parts[1])
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
if len(parts) >= 3:
|
||||
if parts[2].startswith("W"):
|
||||
week = int(parts[2].lstrip("W"))
|
||||
elif not parts[2].startswith("W"):
|
||||
day = int(parts[2])
|
||||
|
||||
context["period"] = "historical"
|
||||
context["year"] = year
|
||||
context["month"] = month
|
||||
context["month_name"] = calendar.month_name[month] if month else None
|
||||
context["week"] = week
|
||||
context["day"] = day
|
||||
|
||||
period_str = str(year)
|
||||
if month:
|
||||
period_str = f"{calendar.month_name[month]} {period_str}"
|
||||
@ -312,109 +147,82 @@ class ChartRecordView(TemplateView):
|
||||
if day:
|
||||
period_str = f"{calendar.month_name[month]} {day}, {year}"
|
||||
context["period_str"] = period_str
|
||||
else:
|
||||
year = current_year
|
||||
month = current_month
|
||||
week = current_week
|
||||
day = current_day
|
||||
context["period"] = "current"
|
||||
context["year"] = current_year
|
||||
context["month"] = current_month
|
||||
context["month_name"] = calendar.month_name[current_month]
|
||||
context["week"] = current_week
|
||||
context["day"] = current_day
|
||||
|
||||
context["charts"] = {
|
||||
"artist": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"artist",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"album": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"album",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"track": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"track",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"tv_series": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"tv_series",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"video": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"video",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"board_game": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"board_game",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"book": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"book",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"food": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"food",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"podcast": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"podcast",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
"trail": list(
|
||||
self.get_charts_for_period(
|
||||
user,
|
||||
"trail",
|
||||
year=year,
|
||||
month=month,
|
||||
week=week,
|
||||
day=day,
|
||||
)
|
||||
),
|
||||
}
|
||||
# List-group tables default to week-level when no date param (matches active tab)
|
||||
if not date_param:
|
||||
list_year = current_year
|
||||
list_month = None
|
||||
list_week = current_week
|
||||
list_day = None
|
||||
else:
|
||||
list_year = year
|
||||
list_month = month
|
||||
list_week = week
|
||||
list_day = day
|
||||
|
||||
context["charts"] = {
|
||||
"artist": list(
|
||||
self.get_charts_for_period(
|
||||
user, "artist", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"album": list(
|
||||
self.get_charts_for_period(
|
||||
user, "album", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"track": list(
|
||||
self.get_charts_for_period(
|
||||
user, "track", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"tv_series": list(
|
||||
self.get_charts_for_period(
|
||||
user, "tv_series", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"video": list(
|
||||
self.get_charts_for_period(
|
||||
user, "video", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"board_game": list(
|
||||
self.get_charts_for_period(
|
||||
user, "board_game", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"book": list(
|
||||
self.get_charts_for_period(
|
||||
user, "book", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"food": list(
|
||||
self.get_charts_for_period(
|
||||
user, "food", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"podcast": list(
|
||||
self.get_charts_for_period(
|
||||
user, "podcast", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
"trail": list(
|
||||
self.get_charts_for_period(
|
||||
user, "trail", year=list_year, month=list_month, week=list_week, day=list_day, limit=20
|
||||
)
|
||||
),
|
||||
}
|
||||
|
||||
bird_data = self.get_bird_chart_data(
|
||||
user,
|
||||
@ -628,6 +436,57 @@ class ChartRecordView(TemplateView):
|
||||
}
|
||||
|
||||
|
||||
class MalojaChartsView(ChartRecordView):
|
||||
"""Three maloja-themed image grid widgets (artists, albums, TV series)
|
||||
with Today/Week/Month/Year/All tabs. Each tab computes its own period
|
||||
from the current date — no query param needed."""
|
||||
|
||||
template_name = "charts/maloja_charts.html"
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
context = super(ChartRecordView, self).get_context_data(**kwargs)
|
||||
user = self.request.user
|
||||
|
||||
if not user.is_authenticated:
|
||||
context["maloja_charts"] = {}
|
||||
context["chart_keys"] = {}
|
||||
return context
|
||||
|
||||
now = timezone.now()
|
||||
now = now_user_timezone(user.profile)
|
||||
today = now.date()
|
||||
|
||||
context["chart_keys"] = {
|
||||
"today": "Today",
|
||||
"week": "This Week",
|
||||
"month": "This Month",
|
||||
"year": "This Year",
|
||||
"all": "All Time",
|
||||
}
|
||||
|
||||
tab_params = {
|
||||
"today": {"year": today.year, "month": today.month, "day": today.day},
|
||||
"week": {"year": today.year, "week": today.isocalendar()[1]},
|
||||
"month": {"year": today.year, "month": today.month},
|
||||
"year": {"year": today.year},
|
||||
}
|
||||
|
||||
maloja_charts = {}
|
||||
for media_type in ("artist", "album", "tv_series"):
|
||||
tabs = {}
|
||||
for key in ("today", "week", "month", "year"):
|
||||
tabs[key] = list(
|
||||
self.get_charts_for_period(user, media_type, **tab_params[key])
|
||||
)
|
||||
tabs["all"] = list(
|
||||
self.get_charts_for_period(user, media_type)
|
||||
)
|
||||
maloja_charts[media_type] = tabs
|
||||
|
||||
context["maloja_charts"] = maloja_charts
|
||||
return context
|
||||
|
||||
|
||||
MEDIA_TYPE_LABELS = {
|
||||
"artist": ("🎤", "Top Artists"),
|
||||
"album": ("💿", "Top Albums"),
|
||||
|
||||
@ -12,7 +12,10 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _parse_udisc_datetime(raw: str) -> datetime:
|
||||
return parse_datetime(raw)
|
||||
dt = parse_datetime(raw)
|
||||
if timezone.is_naive(dt):
|
||||
return timezone.make_aware(dt)
|
||||
return dt
|
||||
|
||||
|
||||
def _resolve_player(name: str, user_id: int) -> Person:
|
||||
|
||||
@ -0,0 +1,202 @@
|
||||
import logging
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
from django.db import models, transaction
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = "Enrich artist and album metadata (covers, thumbnails) from MusicBrainz and TheAudioDB"
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="Overwrite existing cover image and metadata",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Show what would be done without making changes",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--artists",
|
||||
action="store_true",
|
||||
help="Only process artists",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--albums",
|
||||
action="store_true",
|
||||
help="Only process albums",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--needs-metadata",
|
||||
action="store_true",
|
||||
help="Only process items missing metadata or with broken images",
|
||||
)
|
||||
|
||||
def _has_broken_image(self, obj, field_name: str) -> bool:
|
||||
field = getattr(obj, field_name, None)
|
||||
if not field or not field.name:
|
||||
return False
|
||||
try:
|
||||
return not field.storage.exists(field.name)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
def handle(self, *args, **options):
|
||||
from music.models import Album, Artist
|
||||
|
||||
force = options["force"]
|
||||
dry_run = options["dry_run"]
|
||||
only_artists = options["artists"]
|
||||
only_albums = options["albums"]
|
||||
needs_metadata = options["needs_metadata"]
|
||||
|
||||
if not only_artists and not only_albums:
|
||||
only_artists = only_albums = True
|
||||
|
||||
updated_total = 0
|
||||
errors_total = 0
|
||||
|
||||
if only_artists:
|
||||
updated_total += self._process_artists(force, dry_run, needs_metadata)
|
||||
errors_total = 0 # reset per section
|
||||
|
||||
if only_albums:
|
||||
updated_total += self._process_albums(force, dry_run, needs_metadata)
|
||||
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f"\nDone! {updated_total} items processed")
|
||||
)
|
||||
|
||||
def _get_artists(self, needs_metadata: bool):
|
||||
from music.models import Artist
|
||||
|
||||
qs = Artist.objects.all()
|
||||
if needs_metadata:
|
||||
qs = qs.filter(
|
||||
models.Q(theaudiodb_id__isnull=True)
|
||||
| models.Q(theaudiodb_id="")
|
||||
| models.Q(thumbnail__isnull=True)
|
||||
| models.Q(thumbnail="")
|
||||
)
|
||||
|
||||
broken = []
|
||||
if needs_metadata:
|
||||
broken_qs = Artist.objects.exclude(
|
||||
models.Q(thumbnail__isnull=True) | models.Q(thumbnail=""),
|
||||
)
|
||||
for artist in broken_qs.iterator():
|
||||
if self._has_broken_image(artist, "thumbnail"):
|
||||
broken.append(artist)
|
||||
|
||||
return list(qs) + broken
|
||||
|
||||
def _get_albums(self, needs_metadata: bool):
|
||||
from music.models import Album
|
||||
|
||||
qs = Album.objects.all()
|
||||
if needs_metadata:
|
||||
qs = qs.filter(
|
||||
models.Q(cover_image__isnull=True)
|
||||
| models.Q(cover_image="")
|
||||
| models.Q(cover_image="default-image-replace-me")
|
||||
)
|
||||
|
||||
broken = []
|
||||
if needs_metadata:
|
||||
broken_qs = Album.objects.exclude(
|
||||
models.Q(cover_image__isnull=True) | models.Q(cover_image=""),
|
||||
)
|
||||
for album in broken_qs.iterator():
|
||||
if self._has_broken_image(album, "cover_image"):
|
||||
broken.append(album)
|
||||
|
||||
return list(qs) + broken
|
||||
|
||||
def _process_artists(self, force, dry_run, needs_metadata):
|
||||
from music.models import Artist
|
||||
|
||||
artists = self._get_artists(needs_metadata) if needs_metadata else list(Artist.objects.all())
|
||||
total = len(artists)
|
||||
self.stdout.write(f"Processing {total} artists")
|
||||
|
||||
if dry_run:
|
||||
for artist in artists:
|
||||
has_tadb = bool(artist.theaudiodb_id)
|
||||
has_thumb = bool(artist.thumbnail)
|
||||
thumb_broken = self._has_broken_image(artist, "thumbnail")
|
||||
status = f"theaudiodb_id={'✓' if has_tadb else '✗'}"
|
||||
if thumb_broken:
|
||||
status += ", thumbnail=BROKEN"
|
||||
elif has_thumb:
|
||||
status += ", thumbnail=✓"
|
||||
else:
|
||||
status += ", thumbnail=✗"
|
||||
self.stdout.write(f" [DRY RUN] Would fix {artist.name} ({status})")
|
||||
return 0
|
||||
|
||||
updated = 0
|
||||
errors = 0
|
||||
for artist in artists:
|
||||
try:
|
||||
with transaction.atomic():
|
||||
artist.fix_metadata(
|
||||
force_update=force or self._has_broken_image(artist, "thumbnail")
|
||||
)
|
||||
updated += 1
|
||||
self.stdout.write(f" [ARTIST {updated}/{total}] {artist.name}")
|
||||
except Exception as e:
|
||||
errors += 1
|
||||
self.stdout.write(
|
||||
self.style.ERROR(f" Error updating artist {artist.name}: {e}")
|
||||
)
|
||||
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f"\nArtists done! {updated} updated, {errors} errors")
|
||||
)
|
||||
return updated
|
||||
|
||||
def _process_albums(self, force, dry_run, needs_metadata):
|
||||
from music.models import Album
|
||||
|
||||
albums = self._get_albums(needs_metadata) if needs_metadata else list(Album.objects.all())
|
||||
total = len(albums)
|
||||
self.stdout.write(f"Processing {total} albums")
|
||||
|
||||
if dry_run:
|
||||
for album in albums:
|
||||
has_cover = bool(album.cover_image)
|
||||
cover_broken = self._has_broken_image(album, "cover_image")
|
||||
if cover_broken:
|
||||
status = "cover=BROKEN"
|
||||
elif has_cover:
|
||||
status = "cover=✓"
|
||||
else:
|
||||
status = "cover=✗"
|
||||
self.stdout.write(f" [DRY RUN] Would fix {album.name} ({status})")
|
||||
return 0
|
||||
|
||||
updated = 0
|
||||
errors = 0
|
||||
for album in albums:
|
||||
try:
|
||||
with transaction.atomic():
|
||||
if self._has_broken_image(album, "cover_image") or force:
|
||||
album.fetch_artwork(force=True)
|
||||
else:
|
||||
album.fix_metadata()
|
||||
updated += 1
|
||||
self.stdout.write(f" [ALBUM {updated}/{total}] {album.name}")
|
||||
except Exception as e:
|
||||
errors += 1
|
||||
self.stdout.write(
|
||||
self.style.ERROR(f" Error updating album {album.name}: {e}")
|
||||
)
|
||||
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f"\nAlbums done! {updated} updated, {errors} errors")
|
||||
)
|
||||
return updated
|
||||
@ -42,6 +42,7 @@ class UserProfileForm(forms.ModelForm):
|
||||
"home_scrobble_limit",
|
||||
"live_now_playing",
|
||||
"weigh_in_units",
|
||||
"trends_disabled",
|
||||
]
|
||||
widgets = {
|
||||
"lastfm_password": forms.PasswordInput(render_value=True),
|
||||
|
||||
@ -0,0 +1,27 @@
|
||||
# Generated by Django 4.2.29 on 2026-06-25 20:43
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
("profiles", "0039_userprofile_live_now_playing"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="userprofile",
|
||||
name="disabled_trends",
|
||||
field=models.JSONField(
|
||||
blank=True,
|
||||
default=list,
|
||||
help_text="List of trend slugs the user has disabled",
|
||||
),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="userprofile",
|
||||
name="trends_disabled",
|
||||
field=models.BooleanField(default=False),
|
||||
),
|
||||
]
|
||||
@ -106,6 +106,14 @@ class UserProfile(TimeStampedModel):
|
||||
default=WeighUnit.METRIC,
|
||||
)
|
||||
|
||||
trends_disabled = models.BooleanField(default=False)
|
||||
|
||||
disabled_trends = models.JSONField(
|
||||
default=list,
|
||||
blank=True,
|
||||
help_text="List of trend slugs the user has disabled",
|
||||
)
|
||||
|
||||
def __str__(self):
|
||||
return f"User profile for {self.user}"
|
||||
|
||||
|
||||
@ -8,6 +8,7 @@ import pytz
|
||||
import requests
|
||||
from beers.models import Beer
|
||||
from boardgames.models import BoardGame, BoardGameDesigner, BoardGameLocation
|
||||
from boardgames.utils import board_names_to_variants
|
||||
from books.constants import READCOMICSONLINE_URL
|
||||
from books.models import Book, BookLogData, BookPageLogData, Paper
|
||||
from books.utils import parse_readcomicsonline_uri
|
||||
@ -370,7 +371,7 @@ def manual_scrobble_board_game(
|
||||
source: str = "BGG",
|
||||
action: Optional[str] = None,
|
||||
) -> Scrobble | None:
|
||||
boardgame = BoardGame.find_or_create(bggeek_id)
|
||||
boardgame = BoardGame.find_or_create(bggeek_id, defer_expansions=True)
|
||||
|
||||
if not boardgame:
|
||||
logger.error(f"No board game found for ID {bggeek_id}")
|
||||
@ -495,7 +496,9 @@ def email_scrobble_board_game(
|
||||
if play_dict.get("rounds", False):
|
||||
log_data["rounds"] = play_dict.get("rounds")
|
||||
if play_dict.get("board", False):
|
||||
log_data["board"] = play_dict.get("board")
|
||||
log_data["variant_ids"] = board_names_to_variants(
|
||||
base_game, [play_dict.get("board")]
|
||||
)
|
||||
|
||||
log_data["players"] = []
|
||||
for score_dict in play_dict.get("playerScores", []):
|
||||
|
||||
@ -33,7 +33,7 @@ from django.http import (
|
||||
JsonResponse,
|
||||
)
|
||||
from django.shortcuts import get_object_or_404, redirect
|
||||
from django.urls import reverse_lazy
|
||||
from django.urls import reverse, reverse_lazy
|
||||
from django.utils import timezone
|
||||
from django.utils.dateformat import DateFormat
|
||||
from django.views.decorators.csrf import csrf_exempt
|
||||
@ -1235,6 +1235,28 @@ class ScrobbleDetailView(DetailView):
|
||||
def get_form_class(self):
|
||||
return self.object.media_obj.logdata_cls().form()
|
||||
|
||||
def _update_board_game_widgets(self, form):
|
||||
from boardgames.models import BoardGame
|
||||
|
||||
if not isinstance(self.object.media_obj, BoardGame):
|
||||
return
|
||||
|
||||
if "expansion_ids" in form.fields:
|
||||
expansions = BoardGame.objects.filter(
|
||||
expansion_for_boardgame=self.object.media_obj
|
||||
)
|
||||
form.fields["expansion_ids"].queryset = expansions
|
||||
|
||||
if "variant_ids" in form.fields:
|
||||
form.fields["variant_ids"].widget.attrs["data-board-game-id"] = (
|
||||
self.object.media_obj.id
|
||||
)
|
||||
form.fields["variant_ids"].widget.attrs["data-ajax-url"] = (
|
||||
self.request.build_absolute_uri(
|
||||
reverse("boardgames:ajax-create-variant")
|
||||
)
|
||||
)
|
||||
|
||||
def get_form(self):
|
||||
FormClass = self.get_form_class()
|
||||
|
||||
@ -1245,12 +1267,15 @@ class ScrobbleDetailView(DetailView):
|
||||
else:
|
||||
log["notes"] = self.object.logdata.notes_as_str(separator="\n")
|
||||
|
||||
return FormClass(initial=log)
|
||||
form = FormClass(initial=log)
|
||||
self._update_board_game_widgets(form)
|
||||
return form
|
||||
|
||||
def post(self, request, *args, **kwargs):
|
||||
self.object = self.get_object()
|
||||
FormClass = self.get_form_class()
|
||||
form = FormClass(request.POST)
|
||||
self._update_board_game_widgets(form)
|
||||
|
||||
if form.is_valid():
|
||||
data = form.cleaned_data.copy()
|
||||
@ -1263,6 +1288,12 @@ class ScrobbleDetailView(DetailView):
|
||||
if data.get("with_people_ids") is not None:
|
||||
data["with_people_ids"] = [p.id for p in data["with_people_ids"]]
|
||||
|
||||
if data.get("expansion_ids") is not None:
|
||||
data["expansion_ids"] = [e.id for e in data["expansion_ids"]]
|
||||
|
||||
if data.get("variant_ids") is not None:
|
||||
data["variant_ids"] = [v.id for v in data["variant_ids"]]
|
||||
|
||||
if data.get("mood_reason_ids") is not None:
|
||||
data["mood_reason_ids"] = [r.id for r in data["mood_reason_ids"]]
|
||||
|
||||
|
||||
@ -38,15 +38,36 @@ class Command(BaseCommand):
|
||||
overall_start = timezone.now()
|
||||
ok_count = 0
|
||||
fail_count = 0
|
||||
skipped_count = 0
|
||||
|
||||
for user in users:
|
||||
total_trends = len(TREND_REGISTRY)
|
||||
self.stdout.write(f" {user} ({user.id}): {total_trends} trends...")
|
||||
try:
|
||||
profile = user.profile
|
||||
if profile.trends_disabled:
|
||||
self.stdout.write(
|
||||
self.style.WARNING(
|
||||
f" {user} ({user.id}): trends disabled globally, skipping"
|
||||
)
|
||||
)
|
||||
skipped_count += len(TREND_REGISTRY)
|
||||
continue
|
||||
disabled_trends = set(profile.disabled_trends or [])
|
||||
except Exception:
|
||||
disabled_trends = set()
|
||||
|
||||
active_slugs = [
|
||||
s for s in TREND_REGISTRY if s not in disabled_trends
|
||||
]
|
||||
total_trends = len(active_slugs)
|
||||
self.stdout.write(
|
||||
f" {user} ({user.id}): {total_trends} trends ("
|
||||
f"{len(disabled_trends & set(TREND_REGISTRY.keys()))} disabled)..."
|
||||
)
|
||||
user_start = timezone.now()
|
||||
user_ok = 0
|
||||
user_fail = 0
|
||||
|
||||
for idx, (slug, _) in enumerate(TREND_REGISTRY.items(), start=1):
|
||||
for idx, slug in enumerate(active_slugs, start=1):
|
||||
periods = get_supported_periods(slug)
|
||||
self.stdout.write(f" [{idx}/{total_trends}] {slug}...\n")
|
||||
for period in periods:
|
||||
@ -76,7 +97,7 @@ class Command(BaseCommand):
|
||||
overall_elapsed = (timezone.now() - overall_start).total_seconds()
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(
|
||||
f"Done! {ok_count} OK, {fail_count} failed "
|
||||
f"Done! {ok_count} OK, {fail_count} failed, {skipped_count} skipped "
|
||||
f"({overall_elapsed:.1f}s across {total_users} user(s))"
|
||||
)
|
||||
)
|
||||
|
||||
@ -26,7 +26,17 @@ def compute_user_trends(user_id):
|
||||
logger.warning("User %s not found, skipping trends", user_id)
|
||||
return
|
||||
|
||||
total = len(TREND_REGISTRY)
|
||||
try:
|
||||
profile = user.profile
|
||||
if profile.trends_disabled:
|
||||
logger.info("User %s (%d) has trends disabled, skipping", user, user_id)
|
||||
return
|
||||
disabled = set(profile.disabled_trends or [])
|
||||
except Exception:
|
||||
disabled = set()
|
||||
|
||||
active_slugs = [s for s in TREND_REGISTRY if s not in disabled]
|
||||
total = len(active_slugs)
|
||||
logger.info(
|
||||
"Computing %d trends for user %s (%d)",
|
||||
total,
|
||||
@ -34,7 +44,7 @@ def compute_user_trends(user_id):
|
||||
user_id,
|
||||
)
|
||||
|
||||
for idx, (slug, _) in enumerate(TREND_REGISTRY.items(), start=1):
|
||||
for idx, slug in enumerate(active_slugs, start=1):
|
||||
compute_single_trend.delay(user_id, slug)
|
||||
|
||||
logger.info("Dispatched all %d trends for user %s (%d)", total, user, user_id)
|
||||
@ -52,6 +62,21 @@ def compute_single_trend(user_id, slug):
|
||||
logger.warning("Unknown trend slug '%s' for user %d", slug, user_id)
|
||||
return
|
||||
|
||||
try:
|
||||
profile = user.profile
|
||||
if profile.trends_disabled:
|
||||
logger.info(
|
||||
"User %d has trends disabled, skipping '%s'", user_id, slug
|
||||
)
|
||||
return
|
||||
if slug in (profile.disabled_trends or []):
|
||||
logger.info(
|
||||
"User %d has trend '%s' disabled, skipping", user_id, slug
|
||||
)
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
periods = get_supported_periods(slug)
|
||||
|
||||
for period in periods:
|
||||
|
||||
@ -1,45 +1,78 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data.distribution %}
|
||||
{{ data.distribution|json_script:"activity-distribution-data" }}
|
||||
<p class="text-muted mb-3">
|
||||
Total scrobbles{% if current_period_label %} ({{ current_period_label }}){% endif %}: <strong>{{ data.total_count }}</strong>
|
||||
</p>
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Media Type</th>
|
||||
<th class="text-end">Total</th>
|
||||
<th class="text-end">Completed</th>
|
||||
<th class="text-end">%</th>
|
||||
<th>Distribution</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% with max=data.distribution.0.count %}
|
||||
{% for entry in data.distribution %}
|
||||
<tr>
|
||||
<td>{{ entry.media_type }}</td>
|
||||
<td class="text-end">{{ entry.count }}</td>
|
||||
<td class="text-end">{{ entry.completed }}</td>
|
||||
<td class="text-end">{{ entry.pct }}%</td>
|
||||
<td style="width: 30%;">
|
||||
{% if max > 0 %}
|
||||
<div class="progress" style="height: 12px;">
|
||||
<div class="progress-bar" role="progressbar"
|
||||
style="width: {{ entry.pct }}%;"
|
||||
aria-valuenow="{{ entry.count }}"
|
||||
aria-valuemin="0" aria-valuemax="{{ max }}">
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
{% endwith %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<canvas id="activityDistChart" width="700" style="max-width:100%;"></canvas>
|
||||
<script>
|
||||
(function() {
|
||||
var el = document.getElementById('activity-distribution-data');
|
||||
if (!el) return;
|
||||
var data = JSON.parse(el.textContent);
|
||||
if (!data.length) return;
|
||||
var canvas = document.getElementById('activityDistChart');
|
||||
var ctx = canvas.getContext('2d');
|
||||
var W = canvas.width;
|
||||
var rowH = 34;
|
||||
var labelW = 140;
|
||||
var barLeft = labelW + 8;
|
||||
var barRight = W - 80;
|
||||
var barMaxW = barRight - barLeft;
|
||||
var padTop = 8;
|
||||
|
||||
var maxCount = data[0].count;
|
||||
|
||||
// Set canvas height based on data
|
||||
canvas.height = data.length * rowH + padTop;
|
||||
|
||||
function roundRect(x, y, w, h, r) {
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(x + r, y);
|
||||
ctx.lineTo(x + w - r, y);
|
||||
ctx.quadraticCurveTo(x + w, y, x + w, y + r);
|
||||
ctx.lineTo(x + w, y + h - r);
|
||||
ctx.quadraticCurveTo(x + w, y + h, x + w - r, y + h);
|
||||
ctx.lineTo(x + r, y + h);
|
||||
ctx.quadraticCurveTo(x, y + h, x, y + h - r);
|
||||
ctx.lineTo(x, y + r);
|
||||
ctx.quadraticCurveTo(x, y, x + r, y);
|
||||
ctx.closePath();
|
||||
}
|
||||
|
||||
data.forEach(function(entry, i) {
|
||||
var y = padTop + i * rowH;
|
||||
var pct = entry.count / maxCount;
|
||||
var barW = pct * barMaxW;
|
||||
|
||||
// Label
|
||||
ctx.fillStyle = '#374151';
|
||||
ctx.font = '13px sans-serif';
|
||||
ctx.textAlign = 'right';
|
||||
ctx.textBaseline = 'middle';
|
||||
ctx.fillText(entry.media_type, labelW - 6, y + rowH / 2);
|
||||
|
||||
// Bar background
|
||||
ctx.fillStyle = '#f3f4f6';
|
||||
roundRect(barLeft, y + 4, barMaxW, rowH - 8, 4);
|
||||
ctx.fill();
|
||||
|
||||
// Bar fill — green (top) to red (bottom)
|
||||
var hue = 120 * (1 - i / (data.length - 1 || 1));
|
||||
ctx.fillStyle = 'hsl(' + hue + ', 65%, 50%)';
|
||||
roundRect(barLeft, y + 4, Math.max(barW, 2), rowH - 8, 4);
|
||||
ctx.fill();
|
||||
|
||||
// Count + percentage label on the right
|
||||
ctx.fillStyle = '#374151';
|
||||
ctx.font = 'bold 13px sans-serif';
|
||||
ctx.textAlign = 'left';
|
||||
ctx.textBaseline = 'middle';
|
||||
ctx.fillText(entry.count + ' (' + entry.pct + '%)', barRight + 6, y + rowH / 2);
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% else %}
|
||||
<p class="text-muted">No activity data found.</p>
|
||||
{% endif %}
|
||||
|
||||
@ -1,43 +1,74 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data.moods %}
|
||||
{{ data.moods|json_script:"mood-distribution-data" }}
|
||||
<p class="text-muted mb-3">
|
||||
Total mood check-ins{% if current_period_label %} ({{ current_period_label }}){% endif %}: <strong>{{ data.total }}</strong>
|
||||
· Positive: <strong>{{ data.positive_count }}</strong>
|
||||
· Negative: <strong>{{ data.negative_count }}</strong>
|
||||
</p>
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mood</th>
|
||||
<th class="text-end">Count</th>
|
||||
<th>Distribution</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% with max=data.moods.0.count %}
|
||||
{% for entry in data.moods %}
|
||||
<tr>
|
||||
<td>{{ entry.mood }}</td>
|
||||
<td class="text-end">{{ entry.count }}</td>
|
||||
<td style="width: 40%;">
|
||||
{% if max > 0 %}
|
||||
<div class="progress" style="height: 12px;">
|
||||
<div class="progress-bar" role="progressbar"
|
||||
style="width: {% widthratio entry.count max 100 %}%;"
|
||||
aria-valuenow="{{ entry.count }}"
|
||||
aria-valuemin="0" aria-valuemax="{{ max }}">
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
{% endwith %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<canvas id="moodDistChart" width="700" style="max-width:100%;"></canvas>
|
||||
<script>
|
||||
(function() {
|
||||
var el = document.getElementById('mood-distribution-data');
|
||||
if (!el) return;
|
||||
var data = JSON.parse(el.textContent);
|
||||
if (!data.length) return;
|
||||
var canvas = document.getElementById('moodDistChart');
|
||||
var ctx = canvas.getContext('2d');
|
||||
var rowH = 34;
|
||||
var labelW = 140;
|
||||
var barLeft = labelW + 8;
|
||||
var barRight = canvas.width - 80;
|
||||
var barMaxW = barRight - barLeft;
|
||||
var padTop = 8;
|
||||
|
||||
canvas.height = data.length * rowH + padTop;
|
||||
|
||||
var maxCount = data[0].count;
|
||||
|
||||
function roundRect(x, y, w, h, r) {
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(x + r, y);
|
||||
ctx.lineTo(x + w - r, y);
|
||||
ctx.quadraticCurveTo(x + w, y, x + w, y + r);
|
||||
ctx.lineTo(x + w, y + h - r);
|
||||
ctx.quadraticCurveTo(x + w, y + h, x + w - r, y + h);
|
||||
ctx.lineTo(x + r, y + h);
|
||||
ctx.quadraticCurveTo(x, y + h, x, y + h - r);
|
||||
ctx.lineTo(x, y + r);
|
||||
ctx.quadraticCurveTo(x, y, x + r, y);
|
||||
ctx.closePath();
|
||||
}
|
||||
|
||||
data.forEach(function(entry, i) {
|
||||
var y = padTop + i * rowH;
|
||||
var pct = entry.count / maxCount;
|
||||
var barW = pct * barMaxW;
|
||||
|
||||
ctx.fillStyle = '#374151';
|
||||
ctx.font = '13px sans-serif';
|
||||
ctx.textAlign = 'right';
|
||||
ctx.textBaseline = 'middle';
|
||||
ctx.fillText(entry.mood, labelW - 6, y + rowH / 2);
|
||||
|
||||
ctx.fillStyle = '#f3f4f6';
|
||||
roundRect(barLeft, y + 4, barMaxW, rowH - 8, 4);
|
||||
ctx.fill();
|
||||
|
||||
var hue = 120 * (1 - i / (data.length - 1 || 1));
|
||||
ctx.fillStyle = 'hsl(' + hue + ', 65%, 50%)';
|
||||
roundRect(barLeft, y + 4, Math.max(barW, 2), rowH - 8, 4);
|
||||
ctx.fill();
|
||||
|
||||
ctx.fillStyle = '#374151';
|
||||
ctx.font = 'bold 13px sans-serif';
|
||||
ctx.textAlign = 'left';
|
||||
ctx.textBaseline = 'middle';
|
||||
ctx.fillText('' + entry.count, barRight + 6, y + rowH / 2);
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% else %}
|
||||
<p class="text-muted">No mood distribution data found.</p>
|
||||
{% endif %}
|
||||
|
||||
@ -1,37 +1,105 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data.trajectory %}
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Date</th>
|
||||
<th class="text-end">Avg Quality</th>
|
||||
<th class="text-end">Check-ins</th>
|
||||
<th>Mood Bar</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for entry in data.trajectory %}
|
||||
<tr>
|
||||
<td>{{ entry.date }}</td>
|
||||
<td class="text-end">{{ entry.avg_quality }}</td>
|
||||
<td class="text-end">{{ entry.count }}</td>
|
||||
<td style="width: 40%;">
|
||||
<div class="progress" style="height: 16px;">
|
||||
<div class="progress-bar {% if entry.avg_quality >= 5 %}bg-success{% elif entry.avg_quality >= 4 %}bg-info{% elif entry.avg_quality >= 3 %}bg-warning{% else %}bg-danger{% endif %}"
|
||||
role="progressbar"
|
||||
style="width: {% widthratio entry.avg_quality 7 100 %}%;"
|
||||
aria-valuenow="{{ entry.avg_quality }}"
|
||||
aria-valuemin="1" aria-valuemax="7">
|
||||
</div>
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
{{ data.trajectory|json_script:"mood-trajectory-data" }}
|
||||
<div class="d-flex flex-column align-items-center">
|
||||
<canvas id="moodTrajectoryChart" width="700" height="300" style="max-width:100%;"></canvas>
|
||||
<div class="d-flex gap-4 mt-2 small text-muted">
|
||||
<span>← Earlier</span>
|
||||
<span>Later →</span>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
(function() {
|
||||
var el = document.getElementById('mood-trajectory-data');
|
||||
if (!el) return;
|
||||
var data = JSON.parse(el.textContent);
|
||||
if (!data.length) return;
|
||||
var canvas = document.getElementById('moodTrajectoryChart');
|
||||
var ctx = canvas.getContext('2d');
|
||||
var W = canvas.width, H = canvas.height;
|
||||
var pad = { top: 20, right: 20, bottom: 30, left: 40 };
|
||||
var plotW = W - pad.left - pad.right;
|
||||
var plotH = H - pad.top - pad.bottom;
|
||||
|
||||
var yMin = 1, yMax = 7;
|
||||
var yRange = yMax - yMin;
|
||||
var maxCount = data.reduce(function(m, d) { return Math.max(m, d.count); }, 0);
|
||||
|
||||
function xPos(i) {
|
||||
return pad.left + (i / (data.length - 1 || 1)) * plotW;
|
||||
}
|
||||
function yPos(val) {
|
||||
return pad.top + (1 - (val - yMin) / yRange) * plotH;
|
||||
}
|
||||
|
||||
// Background grid
|
||||
ctx.strokeStyle = '#e5e7eb';
|
||||
ctx.lineWidth = 0.5;
|
||||
for (var q = 1; q <= 7; q++) {
|
||||
var y = yPos(q);
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(pad.left, y);
|
||||
ctx.lineTo(W - pad.right, y);
|
||||
ctx.stroke();
|
||||
ctx.fillStyle = '#9ca3af';
|
||||
ctx.font = '11px sans-serif';
|
||||
ctx.textAlign = 'right';
|
||||
ctx.fillText(q.toFixed(1), pad.left - 5, y + 4);
|
||||
}
|
||||
|
||||
// Reference line at neutral (4)
|
||||
var neutralY = yPos(4);
|
||||
ctx.strokeStyle = '#d1d5db';
|
||||
ctx.lineWidth = 1;
|
||||
ctx.setLineDash([4, 4]);
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(pad.left, neutralY);
|
||||
ctx.lineTo(W - pad.right, neutralY);
|
||||
ctx.stroke();
|
||||
ctx.setLineDash([]);
|
||||
ctx.fillStyle = '#9ca3af';
|
||||
ctx.font = '11px sans-serif';
|
||||
ctx.textAlign = 'left';
|
||||
ctx.fillText('neutral', W - pad.right + 4, neutralY + 4);
|
||||
|
||||
// Line chart
|
||||
ctx.beginPath();
|
||||
data.forEach(function(d, i) {
|
||||
var x = xPos(i);
|
||||
var y = yPos(d.avg_quality);
|
||||
if (i === 0) ctx.moveTo(x, y);
|
||||
else ctx.lineTo(x, y);
|
||||
});
|
||||
ctx.strokeStyle = '#6366f1';
|
||||
ctx.lineWidth = 2.5;
|
||||
ctx.lineJoin = 'round';
|
||||
ctx.stroke();
|
||||
|
||||
// Gradient fill below the line
|
||||
var gradient = ctx.createLinearGradient(0, pad.top, 0, H - pad.bottom);
|
||||
gradient.addColorStop(0, 'rgba(99, 102, 241, 0.25)');
|
||||
gradient.addColorStop(1, 'rgba(99, 102, 241, 0.02)');
|
||||
ctx.lineTo(xPos(data.length - 1), yPos(yMin));
|
||||
ctx.lineTo(xPos(0), yPos(yMin));
|
||||
ctx.closePath();
|
||||
ctx.fillStyle = gradient;
|
||||
ctx.fill();
|
||||
|
||||
// Dots
|
||||
data.forEach(function(d, i) {
|
||||
var x = xPos(i);
|
||||
var y = yPos(d.avg_quality);
|
||||
ctx.beginPath();
|
||||
ctx.arc(x, y, 3.5, 0, 2 * Math.PI);
|
||||
ctx.fillStyle = '#6366f1';
|
||||
ctx.fill();
|
||||
ctx.strokeStyle = '#fff';
|
||||
ctx.lineWidth = 1.5;
|
||||
ctx.stroke();
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% else %}
|
||||
<p class="text-muted">No mood check-in data found.</p>
|
||||
{% endif %}
|
||||
|
||||
@ -1,50 +1,90 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data.hours %}
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Hour</th>
|
||||
<th class="text-end">Scrobbles</th>
|
||||
<th>Distribution</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% with total=data.hours|dictsortreversed:"count"|first %}
|
||||
{% with max_count=total.count %}
|
||||
{% for entry in data.hours %}
|
||||
<tr>
|
||||
<td>
|
||||
{% if entry.hour == 0 %}
|
||||
12 AM
|
||||
{% elif entry.hour < 12 %}
|
||||
{{ entry.hour }} AM
|
||||
{% elif entry.hour == 12 %}
|
||||
12 PM
|
||||
{% else %}
|
||||
{{ entry.hour|add:"-12" }} PM
|
||||
{% endif %}
|
||||
</td>
|
||||
<td class="text-end">{{ entry.count }}</td>
|
||||
<td style="width: 40%;">
|
||||
{% if max_count > 0 %}
|
||||
<div class="progress" style="height: 12px;">
|
||||
<div class="progress-bar" role="progressbar"
|
||||
style="width: {{ entry.count|floatformat:0 }}%;"
|
||||
aria-valuenow="{{ entry.count }}"
|
||||
aria-valuemin="0" aria-valuemax="{{ max_count }}">
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
{% endwith %}
|
||||
{% endwith %}
|
||||
</tbody>
|
||||
</table>
|
||||
{{ data.hours|json_script:"peak-hours-data" }}
|
||||
<div class="d-flex flex-wrap align-items-start justify-content-center gap-4">
|
||||
<div>
|
||||
<canvas id="peakHoursChart" width="300" height="300" style="max-width:100%;"></canvas>
|
||||
</div>
|
||||
<div class="table-responsive" style="max-height:360px; overflow-y:auto;">
|
||||
<table class="table table-sm table-borderless mb-0" id="peakHoursLegend">
|
||||
<thead>
|
||||
<tr>
|
||||
<th style="width:14px; padding-right:0;"></th>
|
||||
<th>Hour</th>
|
||||
<th class="text-end">Scrobbles</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for entry in data.hours %}
|
||||
<tr>
|
||||
<td class="p-1 text-center">
|
||||
<span class="legend-swatch" data-idx="{{ forloop.counter0 }}" style="display:inline-block; width:12px; height:12px; border-radius:2px;"></span>
|
||||
</td>
|
||||
<td>
|
||||
{% if entry.hour == 0 %}
|
||||
12 AM
|
||||
{% elif entry.hour < 12 %}
|
||||
{{ entry.hour }} AM
|
||||
{% elif entry.hour == 12 %}
|
||||
12 PM
|
||||
{% else %}
|
||||
{{ entry.hour|add:"-12" }} PM
|
||||
{% endif %}
|
||||
</td>
|
||||
<td class="text-end">{{ entry.count }}</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
(function() {
|
||||
var dataEl = document.getElementById('peak-hours-data');
|
||||
if (!dataEl) return;
|
||||
var data = JSON.parse(dataEl.textContent);
|
||||
var total = data.reduce(function(s, h) { return s + h.count; }, 0) || 1;
|
||||
var canvas = document.getElementById('peakHoursChart');
|
||||
if (!canvas) return;
|
||||
var ctx = canvas.getContext('2d');
|
||||
var cx = canvas.width / 2;
|
||||
var cy = canvas.height / 2;
|
||||
var radius = Math.min(cx, cy) - 10;
|
||||
var startAngle = -Math.PI / 2;
|
||||
|
||||
var minCount = data.reduce(function(m, h) { return Math.min(m, h.count); }, Infinity);
|
||||
var maxCount = data.reduce(function(m, h) { return Math.max(m, h.count); }, 0);
|
||||
var range = maxCount - minCount || 1;
|
||||
|
||||
function countToHue(count) {
|
||||
var t = (count - minCount) / range;
|
||||
return 120 * t;
|
||||
}
|
||||
|
||||
data.forEach(function(entry, i) {
|
||||
var sliceAngle = (entry.count / total) * 2 * Math.PI;
|
||||
var hue = countToHue(entry.count);
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(cx, cy);
|
||||
ctx.arc(cx, cy, radius, startAngle, startAngle + sliceAngle);
|
||||
ctx.closePath();
|
||||
ctx.fillStyle = 'hsl(' + hue + ', 70%, 50%)';
|
||||
ctx.fill();
|
||||
ctx.strokeStyle = '#fff';
|
||||
ctx.lineWidth = 1.5;
|
||||
ctx.stroke();
|
||||
startAngle += sliceAngle;
|
||||
});
|
||||
|
||||
document.querySelectorAll('.legend-swatch').forEach(function(el) {
|
||||
var idx = parseInt(el.getAttribute('data-idx'), 10);
|
||||
var entry = data[idx];
|
||||
var hue = countToHue(entry.count);
|
||||
el.style.backgroundColor = 'hsl(' + hue + ', 70%, 50%)';
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% else %}
|
||||
<p class="text-muted">No activity data found.</p>
|
||||
{% endif %}
|
||||
|
||||
@ -0,0 +1,79 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data.total and data.total > 0 %}
|
||||
<div class="mb-4">
|
||||
{% for slug, info in data.categories.items %}
|
||||
{% if forloop.first %}
|
||||
<div class="card text-center border-0 bg-light mb-3">
|
||||
<div class="card-body py-4">
|
||||
{% if slug == "early_bird" %}
|
||||
<div class="display-1">🌅</div>
|
||||
{% elif slug == "day_jay" %}
|
||||
<div class="display-1">☀️</div>
|
||||
{% else %}
|
||||
<div class="display-1">🌙</div>
|
||||
{% endif %}
|
||||
<h3 class="mt-2">{{ info.label }}</h3>
|
||||
<div class="display-4 fw-bold">{{ info.pct }}%</div>
|
||||
<div class="text-muted">{{ info.count }} scrobbles</div>
|
||||
</div>
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="card text-center border-0 bg-light mb-2">
|
||||
<div class="card-body py-2 d-flex align-items-center justify-content-between px-4">
|
||||
<div class="d-flex align-items-center gap-3">
|
||||
{% if slug == "early_bird" %}
|
||||
<span style="font-size:1.5rem;">🌅</span>
|
||||
{% elif slug == "day_jay" %}
|
||||
<span style="font-size:1.5rem;">☀️</span>
|
||||
{% else %}
|
||||
<span style="font-size:1.5rem;">🌙</span>
|
||||
{% endif %}
|
||||
<h5 class="mb-0">{{ info.label }}</h5>
|
||||
</div>
|
||||
<div class="text-end">
|
||||
<span class="fs-4 fw-bold">{{ info.pct }}%</span>
|
||||
<br><small class="text-muted">{{ info.count }} scrobbles</small>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
<div class="text-center text-muted small mt-2">Total: {{ data.total }} scrobbles across Books, Trails, Birding Locations, and Board Games</div>
|
||||
</div>
|
||||
|
||||
<h5>By Media Type</h5>
|
||||
{% for mt, mt_data in data.by_media_type.items %}
|
||||
<h6 class="mt-3">{{ mt }}</h6>
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Category</th>
|
||||
<th class="text-end">Scrobbles</th>
|
||||
<th class="text-end">%</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for slug, info in mt_data.categories.items %}
|
||||
<tr>
|
||||
<td>{{ info.label }}</td>
|
||||
<td class="text-end">{{ info.count }}</td>
|
||||
<td class="text-end">{{ info.pct }}%</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
<tr class="table-secondary">
|
||||
<td><strong>Total</strong></td>
|
||||
<td class="text-end"><strong>{{ mt_data.total }}</strong></td>
|
||||
<td class="text-end"></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{% endfor %}
|
||||
|
||||
{% else %}
|
||||
<p class="text-muted">No data found for Books, Trails, Birding Locations, or Board Games in this period.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
@ -1,38 +1,27 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
{% if data %}
|
||||
<div class="table-responsive">
|
||||
<table class="table table-striped table-sm">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Media Type</th>
|
||||
<th class="text-end">Recent ({{ current_period_label }})</th>
|
||||
<th class="text-end">Previous ({{ current_period_label }})</th>
|
||||
<th class="text-end">Change</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for mt, info in data.items %}
|
||||
<tr>
|
||||
<td>{{ mt }}</td>
|
||||
<td class="text-end">{{ info.recent }}</td>
|
||||
<td class="text-end">{{ info.previous }}</td>
|
||||
<td class="text-end">
|
||||
{% if info.change_pct > 0 %}
|
||||
<span class="text-success">+{{ info.change_pct }}%</span>
|
||||
{% elif info.change_pct < 0 %}
|
||||
<span class="text-danger">{{ info.change_pct }}%</span>
|
||||
{% else %}
|
||||
<span class="text-muted">0%</span>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
<div class="row g-3">
|
||||
{% if data %}
|
||||
{% for mt, info in data.items %}
|
||||
<div class="col-6 col-md-4 col-lg-3 col-xl-2">
|
||||
<div class="card text-center h-100">
|
||||
<div class="card-body d-flex flex-column align-items-center justify-content-center" style="aspect-ratio: 1;">
|
||||
{% if info.change_pct > 0 %}
|
||||
<div class="display-3 lh-1 text-success">↑</div>
|
||||
<div class="fs-6 text-success mt-1">+{{ info.change_pct }}%</div>
|
||||
{% elif info.change_pct < 0 %}
|
||||
<div class="display-3 lh-1 text-danger">↓</div>
|
||||
<div class="fs-6 text-danger mt-1">{{ info.change_pct }}%</div>
|
||||
{% else %}
|
||||
<div class="display-3 lh-1 text-muted">→</div>
|
||||
<div class="fs-6 text-muted mt-1">0%</div>
|
||||
{% endif %}
|
||||
<div class="fw-bold mt-2">{{ mt }}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% else %}
|
||||
<p class="text-muted">No trending data found.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
<div class="col-12">
|
||||
<p class="text-muted">No trending data found.</p>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
@ -3,6 +3,10 @@
|
||||
{% block title %}{{ trend.title }}{% endblock %}
|
||||
|
||||
{% block lists %}
|
||||
{% if trend_not_found %}
|
||||
<div class="alert alert-warning">Trend not found.</div>
|
||||
|
||||
{% else %}
|
||||
<div class="row mb-3">
|
||||
<div class="col-12">
|
||||
<a href="{% url 'trends:trends-home' %}" class="btn btn-sm btn-outline-secondary mb-2">← All Trends</a>
|
||||
@ -35,13 +39,21 @@
|
||||
{% if computed_at %}
|
||||
<small class="text-muted">Last computed: {{ computed_at|date:"F j, Y H:i" }}</small>
|
||||
{% endif %}
|
||||
|
||||
<div class="mt-2">
|
||||
<form method="post" action="{% url 'trends:trend-toggle-disabled' trend.slug %}" class="d-inline">
|
||||
{% csrf_token %}
|
||||
{% if trend_disabled %}
|
||||
<button type="submit" class="btn btn-sm btn-outline-success">Enable this trend</button>
|
||||
{% else %}
|
||||
<button type="submit" class="btn btn-sm btn-outline-danger">Disable this trend</button>
|
||||
{% endif %}
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% if trend_not_found %}
|
||||
<div class="alert alert-warning">Trend not found.</div>
|
||||
|
||||
{% elif data is None %}
|
||||
{% if data is None %}
|
||||
<div class="alert alert-info">
|
||||
No data computed yet for this period. Trends are updated once daily, check back later.
|
||||
</div>
|
||||
@ -55,6 +67,9 @@
|
||||
{% elif trend.slug == "reading-pace-vs-activity" %}
|
||||
{% include "trends/_reading_pace.html" %}
|
||||
|
||||
{% elif trend.slug == "time-of-day-categories" %}
|
||||
{% include "trends/_time_of_day_categories.html" %}
|
||||
|
||||
{% elif trend.slug == "trending-up" %}
|
||||
{% include "trends/_trending_up.html" %}
|
||||
|
||||
@ -82,5 +97,6 @@
|
||||
{% elif trend.slug == "mood-weather" %}
|
||||
{% include "trends/_mood_weather.html" %}
|
||||
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
@ -25,11 +25,23 @@
|
||||
</a>
|
||||
</h5>
|
||||
<p class="card-text text-muted">{{ trend.description }}</p>
|
||||
{% if trend.computed_at %}
|
||||
<small class="text-muted">Last computed: {{ trend.computed_at|date:"M j, Y H:i" }}</small>
|
||||
{% else %}
|
||||
<span class="badge bg-warning text-dark">Pending</span>
|
||||
{% endif %}
|
||||
<div class="d-flex justify-content-between align-items-center">
|
||||
<div>
|
||||
{% if trend.computed_at %}
|
||||
<small class="text-muted">Last computed: {{ trend.computed_at|date:"M j, Y H:i" }}</small>
|
||||
{% else %}
|
||||
<span class="badge bg-warning text-dark">Pending</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
<form method="post" action="{% url 'trends:trend-toggle-disabled' trend.slug %}" class="m-0">
|
||||
{% csrf_token %}
|
||||
{% if trend.disabled %}
|
||||
<button type="submit" class="btn btn-sm btn-outline-success">Enable</button>
|
||||
{% else %}
|
||||
<button type="submit" class="btn btn-sm btn-outline-danger">Disable</button>
|
||||
{% endif %}
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -15,6 +15,7 @@ from trends.trends.mood import (
|
||||
compute_mood_weather,
|
||||
)
|
||||
from trends.trends.reading import compute_reading_pace_vs_activity
|
||||
from trends.trends.time_of_day import compute_time_of_day_categories
|
||||
from trends.trends.trending import compute_trending_up
|
||||
|
||||
TREND_REGISTRY = {}
|
||||
@ -44,5 +45,8 @@ compute_peak_hours = register("peak-hours")(compute_peak_hours)
|
||||
compute_reading_pace_vs_activity = register("reading-pace-vs-activity")(
|
||||
compute_reading_pace_vs_activity
|
||||
)
|
||||
compute_time_of_day_categories = register("time-of-day-categories")(
|
||||
compute_time_of_day_categories
|
||||
)
|
||||
compute_trending_up = register("trending-up")(compute_trending_up)
|
||||
compute_weekly_rhythm = register("weekly-rhythm")(compute_weekly_rhythm)
|
||||
|
||||
95
vrobbler/apps/trends/trends/time_of_day.py
Normal file
95
vrobbler/apps/trends/trends/time_of_day.py
Normal file
@ -0,0 +1,95 @@
|
||||
from collections import OrderedDict
|
||||
|
||||
from django.db.models import Count, Q
|
||||
from django.db.models.functions import Extract
|
||||
from scrobbles.models import Scrobble
|
||||
|
||||
|
||||
TARGET_MEDIA_TYPES = ["Book", "Trail", "BirdingLocation", "BoardGame"]
|
||||
|
||||
CATEGORIES = OrderedDict(
|
||||
[
|
||||
("early_bird", {"label": "Early Bird", "hours": {5, 6, 7, 8, 9, 10}}),
|
||||
("day_jay", {"label": "Day Jay", "hours": {11, 12, 13, 14, 15, 16, 17, 18}}),
|
||||
("night_owl", {"label": "Night Owl", "hours": {19, 20, 21, 22, 23, 0, 1, 2, 3, 4}}),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def _categorize_hour(hour):
|
||||
for slug, cat in CATEGORIES.items():
|
||||
if hour in cat["hours"]:
|
||||
return slug
|
||||
return None
|
||||
|
||||
|
||||
def compute_time_of_day_categories(user, period="last_30"):
|
||||
from trends.utils import get_date_range
|
||||
|
||||
start, end = get_date_range(period)
|
||||
filters = Q(user=user, media_type__in=TARGET_MEDIA_TYPES, timestamp__isnull=False)
|
||||
if start:
|
||||
filters &= Q(timestamp__gte=start)
|
||||
if end:
|
||||
filters &= Q(timestamp__lte=end)
|
||||
|
||||
qs = (
|
||||
Scrobble.objects.filter(filters)
|
||||
.annotate(hour=Extract("timestamp", "hour"))
|
||||
.values("media_type", "hour")
|
||||
.annotate(count=Count("id"))
|
||||
.order_by("media_type", "hour")
|
||||
)
|
||||
|
||||
raw = {}
|
||||
for row in qs:
|
||||
mt = row["media_type"]
|
||||
raw.setdefault(mt, {})[row["hour"]] = row["count"]
|
||||
|
||||
by_media_type = {}
|
||||
grand_totals = {"early_bird": 0, "day_jay": 0, "night_owl": 0}
|
||||
grand_total = 0
|
||||
|
||||
for mt in TARGET_MEDIA_TYPES:
|
||||
mt_data = raw.get(mt, {})
|
||||
cat_counts = {"early_bird": 0, "day_jay": 0, "night_owl": 0}
|
||||
mt_total = 0
|
||||
for hour, count in mt_data.items():
|
||||
slug = _categorize_hour(hour)
|
||||
if slug:
|
||||
cat_counts[slug] += count
|
||||
mt_total += count
|
||||
mt_categories = {}
|
||||
for slug in CATEGORIES:
|
||||
c = cat_counts[slug]
|
||||
mt_categories[slug] = {
|
||||
"count": c,
|
||||
"pct": round((c / mt_total * 100), 1) if mt_total else 0,
|
||||
"label": CATEGORIES[slug]["label"],
|
||||
}
|
||||
grand_totals[slug] += c
|
||||
by_media_type[mt] = {
|
||||
"total": mt_total,
|
||||
"categories": dict(
|
||||
sorted(mt_categories.items(), key=lambda x: x[1]["count"], reverse=True)
|
||||
),
|
||||
}
|
||||
grand_total += mt_total
|
||||
|
||||
categories = {}
|
||||
for slug in CATEGORIES:
|
||||
c = grand_totals[slug]
|
||||
categories[slug] = {
|
||||
"count": c,
|
||||
"pct": round((c / grand_total * 100), 1) if grand_total else 0,
|
||||
"label": CATEGORIES[slug]["label"],
|
||||
}
|
||||
categories = dict(
|
||||
sorted(categories.items(), key=lambda x: x[1]["count"], reverse=True)
|
||||
)
|
||||
|
||||
return {
|
||||
"categories": categories,
|
||||
"total": grand_total,
|
||||
"by_media_type": by_media_type,
|
||||
}
|
||||
@ -1,9 +1,18 @@
|
||||
from django.urls import path
|
||||
from trends.views import TrendDetailView, TrendListView
|
||||
from trends.views import (
|
||||
ToggleTrendDisabledView,
|
||||
TrendDetailView,
|
||||
TrendListView,
|
||||
)
|
||||
|
||||
app_name = "trends"
|
||||
|
||||
urlpatterns = [
|
||||
path("trends/", TrendListView.as_view(), name="trends-home"),
|
||||
path("trends/<slug:trend_slug>/", TrendDetailView.as_view(), name="trend-detail"),
|
||||
path(
|
||||
"trends/<slug:trend_slug>/toggle/",
|
||||
ToggleTrendDisabledView.as_view(),
|
||||
name="trend-toggle-disabled",
|
||||
),
|
||||
]
|
||||
|
||||
@ -3,6 +3,7 @@ from datetime import timedelta
|
||||
|
||||
from django.utils import timezone
|
||||
from trends.models import PERIOD_CHOICES, TrendResult
|
||||
from trends.trends import TREND_REGISTRY
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@ -25,6 +26,7 @@ TIME_BOUND_TRENDS = {
|
||||
"mood-weather",
|
||||
"peak-hours",
|
||||
"reading-pace-vs-activity",
|
||||
"time-of-day-categories",
|
||||
"trending-up",
|
||||
"weekly-rhythm",
|
||||
}
|
||||
@ -34,6 +36,13 @@ TREND_PERIOD_OVERRIDES = {
|
||||
}
|
||||
|
||||
|
||||
def get_disabled_trends(user):
|
||||
profile = user.profile
|
||||
if profile.trends_disabled:
|
||||
return set(TREND_REGISTRY.keys())
|
||||
return set(profile.disabled_trends or [])
|
||||
|
||||
|
||||
def get_supported_periods(trend_slug):
|
||||
if trend_slug in TREND_PERIOD_OVERRIDES:
|
||||
slugs = TREND_PERIOD_OVERRIDES[trend_slug]
|
||||
|
||||
@ -1,8 +1,11 @@
|
||||
from django.contrib import messages
|
||||
from django.contrib.auth.mixins import LoginRequiredMixin
|
||||
from django.views.generic import TemplateView
|
||||
from django.http import HttpResponseRedirect
|
||||
from django.urls import reverse
|
||||
from django.views.generic import TemplateView, View
|
||||
from trends.models import TrendResult
|
||||
from trends.trends import TREND_REGISTRY
|
||||
from trends.utils import get_period_nav, get_supported_periods
|
||||
from trends.utils import get_disabled_trends, get_period_nav, get_supported_periods
|
||||
|
||||
TREND_METADATA = {
|
||||
"activity-distribution": {
|
||||
@ -55,6 +58,11 @@ TREND_METADATA = {
|
||||
"description": "Compare how long you read per session with and without concurrent music.",
|
||||
"icon": "📊",
|
||||
},
|
||||
"time-of-day-categories": {
|
||||
"title": "Time of Day Categories",
|
||||
"description": "Are you an early bird, day jay, or night owl? Categorized by Books, Trails, Birding Locations, and Board Games.",
|
||||
"icon": "🦉",
|
||||
},
|
||||
"trending-up": {
|
||||
"title": "Trending Media Types",
|
||||
"description": "Which media types have you been consuming more or less of recently?",
|
||||
@ -73,6 +81,8 @@ class TrendListView(LoginRequiredMixin, TemplateView):
|
||||
|
||||
def get_context_data(self, **kwargs):
|
||||
ctx = super().get_context_data(**kwargs)
|
||||
disabled = get_disabled_trends(self.request.user)
|
||||
|
||||
results = TrendResult.objects.filter(
|
||||
user=self.request.user,
|
||||
).order_by("trend_slug", "-computed_at")
|
||||
@ -84,8 +94,11 @@ class TrendListView(LoginRequiredMixin, TemplateView):
|
||||
|
||||
trends = []
|
||||
for slug in TREND_REGISTRY:
|
||||
if slug in disabled:
|
||||
continue
|
||||
meta = TREND_METADATA.get(slug, {})
|
||||
result = latest_by_slug.get(slug)
|
||||
is_disabled = slug in (self.request.user.profile.disabled_trends or [])
|
||||
trends.append(
|
||||
{
|
||||
"slug": slug,
|
||||
@ -94,6 +107,7 @@ class TrendListView(LoginRequiredMixin, TemplateView):
|
||||
"icon": meta.get("icon", ""),
|
||||
"computed_at": result.computed_at if result else None,
|
||||
"has_data": result is not None,
|
||||
"disabled": is_disabled,
|
||||
}
|
||||
)
|
||||
ctx["trends"] = trends
|
||||
@ -143,4 +157,25 @@ class TrendDetailView(LoginRequiredMixin, TemplateView):
|
||||
ctx["computed_at"] = None
|
||||
ctx["data"] = None
|
||||
|
||||
disabled = get_disabled_trends(self.request.user)
|
||||
ctx["trend_disabled"] = slug in disabled
|
||||
ctx["globally_disabled"] = self.request.user.profile.trends_disabled
|
||||
|
||||
return ctx
|
||||
|
||||
|
||||
class ToggleTrendDisabledView(LoginRequiredMixin, View):
|
||||
def post(self, request, trend_slug):
|
||||
profile = request.user.profile
|
||||
disabled = set(profile.disabled_trends or [])
|
||||
if trend_slug in disabled:
|
||||
disabled.discard(trend_slug)
|
||||
messages.success(request, f"Trend re-enabled.")
|
||||
else:
|
||||
disabled.add(trend_slug)
|
||||
messages.success(request, f"Trend disabled.")
|
||||
profile.disabled_trends = list(disabled)
|
||||
profile.save(update_fields=["disabled_trends"])
|
||||
return HttpResponseRedirect(
|
||||
request.META.get("HTTP_REFERER", reverse("trends:trends-home"))
|
||||
)
|
||||
|
||||
@ -17,8 +17,8 @@ MISSING_ALL = [
|
||||
]
|
||||
|
||||
MISSING_GROUPS = {
|
||||
"cover": lambda g: not bool(g.cover),
|
||||
"screenshot": lambda g: not bool(g.screenshot),
|
||||
"cover": lambda g: _image_missing_or_broken(g, "cover"),
|
||||
"screenshot": lambda g: _image_missing_or_broken(g, "screenshot"),
|
||||
"summary": lambda g: not g.summary,
|
||||
"rating": lambda g: g.rating is None,
|
||||
"release_date": lambda g: g.release_date is None,
|
||||
@ -28,6 +28,16 @@ MISSING_GROUPS = {
|
||||
}
|
||||
|
||||
|
||||
def _image_missing_or_broken(game, field_name) -> bool:
|
||||
field = getattr(game, field_name)
|
||||
if not bool(field):
|
||||
return True
|
||||
try:
|
||||
return not field.storage.exists(field.name)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
|
||||
def _game_matches(game, flags):
|
||||
if not flags:
|
||||
return False
|
||||
@ -103,6 +113,7 @@ class Command(BaseCommand):
|
||||
|
||||
if all_missing:
|
||||
flags = MISSING_ALL
|
||||
fix_broken_images = True
|
||||
|
||||
if not flags and not game_id and not force and not fix_broken_images:
|
||||
self.stdout.write(
|
||||
|
||||
@ -30,6 +30,19 @@ class Command(BaseCommand):
|
||||
action="store_true",
|
||||
help="Only process channels with a twitch_id",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--needs-metadata",
|
||||
action="store_true",
|
||||
help="Only process channels missing youtube_id, twitch_id, cover image, or with broken cover image",
|
||||
)
|
||||
|
||||
def _has_broken_image(self, channel) -> bool:
|
||||
if not channel.cover_image or not channel.cover_image.name:
|
||||
return False
|
||||
try:
|
||||
return not channel.cover_image.storage.exists(channel.cover_image.name)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
def handle(self, *args, **options):
|
||||
from videos.models import Channel
|
||||
@ -38,6 +51,7 @@ class Command(BaseCommand):
|
||||
dry_run = options["dry_run"]
|
||||
youtube_only = options["youtube_only"]
|
||||
twitch_only = options["twitch_only"]
|
||||
needs_metadata = options["needs_metadata"]
|
||||
|
||||
qs = Channel.objects.all()
|
||||
|
||||
@ -45,29 +59,61 @@ class Command(BaseCommand):
|
||||
qs = qs.exclude(youtube_id__isnull=True).exclude(youtube_id="")
|
||||
elif twitch_only:
|
||||
qs = qs.exclude(twitch_id__isnull=True).exclude(twitch_id="")
|
||||
elif needs_metadata:
|
||||
no_id = models.Q(youtube_id__isnull=True) & models.Q(twitch_id__isnull=True)
|
||||
no_id |= models.Q(youtube_id="") & models.Q(twitch_id="")
|
||||
no_id |= models.Q(youtube_id__isnull=True) & models.Q(twitch_id="")
|
||||
no_id |= models.Q(youtube_id="") & models.Q(twitch_id__isnull=True)
|
||||
qs = qs.filter(
|
||||
no_id
|
||||
| models.Q(cover_image__isnull=True)
|
||||
| models.Q(cover_image="")
|
||||
)
|
||||
else:
|
||||
qs = qs.filter(
|
||||
models.Q(youtube_id__isnull=False) | models.Q(twitch_id__isnull=False)
|
||||
).exclude(youtube_id="", twitch_id="")
|
||||
|
||||
total = qs.count()
|
||||
self.stdout.write(f"Processing {total} channels")
|
||||
self.stdout.write(f"Processing {total} channels from DB filter")
|
||||
|
||||
broken_channels = []
|
||||
if needs_metadata:
|
||||
broken_qs = Channel.objects.filter(
|
||||
cover_image__isnull=False,
|
||||
).exclude(
|
||||
cover_image="",
|
||||
)
|
||||
if youtube_only:
|
||||
broken_qs = broken_qs.exclude(youtube_id__isnull=True).exclude(youtube_id="")
|
||||
elif twitch_only:
|
||||
broken_qs = broken_qs.exclude(twitch_id__isnull=True).exclude(twitch_id="")
|
||||
for channel in broken_qs.iterator():
|
||||
if self._has_broken_image(channel):
|
||||
broken_channels.append(channel)
|
||||
|
||||
all_channels = list(qs) + broken_channels
|
||||
total = len(all_channels)
|
||||
self.stdout.write(f"Total channels to process: {total}")
|
||||
|
||||
if dry_run:
|
||||
for channel in qs.iterator():
|
||||
for channel in all_channels:
|
||||
source = "youtube" if channel.youtube_id else "twitch"
|
||||
identifier = channel.youtube_id or channel.twitch_id
|
||||
status = f"({source}: {identifier})"
|
||||
if self._has_broken_image(channel):
|
||||
status += " [image BROKEN]"
|
||||
self.stdout.write(
|
||||
f" [DRY RUN] Would fix {channel.name} ({source}: {identifier})"
|
||||
f" [DRY RUN] Would fix {channel.name} {status}"
|
||||
)
|
||||
return
|
||||
|
||||
updated = 0
|
||||
errors = 0
|
||||
for channel in qs.iterator():
|
||||
for channel in all_channels:
|
||||
try:
|
||||
with transaction.atomic():
|
||||
channel.fix_metadata(force=force)
|
||||
channel.fix_metadata(force=force or self._has_broken_image(channel))
|
||||
updated += 1
|
||||
source = "youtube" if channel.youtube_id else "twitch"
|
||||
self.stdout.write(f" [{updated}/{total}] {channel.name} ({source})")
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
import logging
|
||||
import os
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
from django.db import models, transaction
|
||||
@ -28,9 +29,18 @@ class Command(BaseCommand):
|
||||
parser.add_argument(
|
||||
"--needs-metadata",
|
||||
action="store_true",
|
||||
help="Only process series missing imdb_id or cover image",
|
||||
help="Only process series missing imdb_id or with broken cover image",
|
||||
)
|
||||
|
||||
def _has_broken_image(self, series) -> bool:
|
||||
"""Check if a series has a cover_image set but the file is missing."""
|
||||
if not series.cover_image:
|
||||
return False
|
||||
try:
|
||||
return not os.path.exists(series.cover_image.path)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
def handle(self, *args, **options):
|
||||
from videos.models import Series
|
||||
|
||||
@ -51,25 +61,50 @@ class Command(BaseCommand):
|
||||
)
|
||||
|
||||
total = qs.count()
|
||||
self.stdout.write(f"Processing {total} series")
|
||||
self.stdout.write(f"Processing {total} series from DB filter")
|
||||
|
||||
# Also find series with broken cover images
|
||||
broken_image_series = []
|
||||
if needs_metadata:
|
||||
broken_qs = Series.objects.filter(
|
||||
cover_image__isnull=False,
|
||||
).exclude(
|
||||
models.Q(imdb_id__isnull=True)
|
||||
| models.Q(imdb_id="")
|
||||
| models.Q(cover_image__isnull=True)
|
||||
| models.Q(cover_image=""),
|
||||
)
|
||||
if imdb_id:
|
||||
broken_qs = broken_qs.filter(imdb_id=imdb_id)
|
||||
for series in broken_qs.iterator():
|
||||
if self._has_broken_image(series):
|
||||
broken_image_series.append(series)
|
||||
|
||||
all_series = list(qs) + broken_image_series
|
||||
total = len(all_series)
|
||||
self.stdout.write(f"Total series to process: {total}")
|
||||
|
||||
if dry_run:
|
||||
for series in qs.iterator():
|
||||
for series in all_series:
|
||||
has_imdb = bool(series.imdb_id)
|
||||
has_image = bool(series.cover_image)
|
||||
self.stdout.write(
|
||||
f" [DRY RUN] Would fix {series.name}"
|
||||
f" (imdb_id={'✓' if has_imdb else '✗'}"
|
||||
f", image={'✓' if has_image else '✗'})"
|
||||
)
|
||||
image_broken = self._has_broken_image(series)
|
||||
status = f"imdb_id={'✓' if has_imdb else '✗'}"
|
||||
if image_broken:
|
||||
status += ", image=BROKEN"
|
||||
elif has_image:
|
||||
status += ", image=✓"
|
||||
else:
|
||||
status += ", image=✗"
|
||||
self.stdout.write(f" [DRY RUN] Would fix {series.name} ({status})")
|
||||
return
|
||||
|
||||
updated = 0
|
||||
errors = 0
|
||||
for series in qs.iterator():
|
||||
for series in all_series:
|
||||
try:
|
||||
with transaction.atomic():
|
||||
series.fix_metadata(force_update=force)
|
||||
series.fix_metadata(force_update=force or self._has_broken_image(series))
|
||||
updated += 1
|
||||
self.stdout.write(f" [{updated}/{total}] {series.name}")
|
||||
except Exception as e:
|
||||
|
||||
@ -45,6 +45,37 @@ class Domain(TimeStampedModel):
|
||||
)
|
||||
|
||||
|
||||
def _fetch_url_raw(url: str) -> Optional[str]:
|
||||
"""Fetch raw HTML for a URL.
|
||||
|
||||
Tries two strategies in order:
|
||||
1. trafilatura (standard, works for most sites)
|
||||
2. cloudscraper (handles Cloudflare JS challenges)
|
||||
|
||||
cloudscraper is lazy-imported so deployments that cannot compile
|
||||
its dependencies (e.g. FreeBSD) still work.
|
||||
"""
|
||||
raw = trafilatura.fetch_url(url)
|
||||
if raw:
|
||||
return raw
|
||||
|
||||
logger.debug("trafilatura returned nothing for %s, trying cloudscraper", url)
|
||||
try:
|
||||
import cloudscraper
|
||||
except ImportError:
|
||||
logger.debug("cloudscraper not available")
|
||||
else:
|
||||
try:
|
||||
scraper = cloudscraper.create_scraper()
|
||||
resp = scraper.get(url, timeout=30)
|
||||
resp.raise_for_status()
|
||||
return resp.text
|
||||
except Exception as exc:
|
||||
logger.debug("cloudscraper failed for %s: %s", url, exc)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
class WebPage(ScrobblableMixin):
|
||||
COMPLETION_PERCENT = getattr(settings, "WEBSITE_COMPLETION_PERCENT", 100)
|
||||
|
||||
@ -80,7 +111,7 @@ class WebPage(ScrobblableMixin):
|
||||
|
||||
def _update_extract_from_web(self, raw_text: str = "", force=True):
|
||||
if not raw_text:
|
||||
raw_text = requests.get(self.url, headers=headers).text
|
||||
raw_text = _fetch_url_raw(self.url)
|
||||
if not self.extract or force:
|
||||
self.extract = trafilatura.extract(raw_text)
|
||||
self.save(update_fields=["extract"])
|
||||
@ -259,7 +290,7 @@ class WebPage(ScrobblableMixin):
|
||||
return False
|
||||
|
||||
def fetch_data_from_web(self, save=True, force=True):
|
||||
raw_text = trafilatura.fetch_url(self.url)
|
||||
raw_text = _fetch_url_raw(self.url)
|
||||
if not self.extract or force:
|
||||
self.extract = trafilatura.extract(
|
||||
raw_text,
|
||||
|
||||
@ -103,7 +103,13 @@ DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
|
||||
TIME_ZONE = os.getenv("VROBBLER_TIME_ZONE", "America/New_York")
|
||||
|
||||
ALLOWED_HOSTS = ["*"]
|
||||
CSRF_TRUSTED_ORIGINS = [os.getenv("VROBBLER_TRUSTED_ORIGINS", "http://localhost:8000")]
|
||||
CSRF_TRUSTED_ORIGINS = [
|
||||
os.getenv(
|
||||
"VROBBLER_TRUSTED_ORIGINS",
|
||||
"http://localhost:8000",
|
||||
),
|
||||
"https://dev-vrobbler.lab.unbl.ink",
|
||||
]
|
||||
X_FRAME_OPTIONS = "SAMEORIGIN"
|
||||
|
||||
REDIS_URL = os.getenv("VROBBLER_REDIS_URL", None)
|
||||
@ -399,6 +405,12 @@ else:
|
||||
|
||||
SCIHUB_DOMAIN = os.getenv("VROBBLER_SCIHUB_DOMAIN", "sci-hub.st")
|
||||
|
||||
SKIP_AUTO_EXPANSION_DOWNLOAD = [
|
||||
int(x.strip())
|
||||
for x in os.getenv("VROBBLER_SKIP_AUTO_EXPANSION_DOWNLOAD", "").split(",")
|
||||
if x.strip().isdigit()
|
||||
]
|
||||
|
||||
JSON_LOGGING = os.getenv("VROBBLER_JSON_LOGGING", "false").lower() in TRUTHY
|
||||
LOG_TYPE = "json" if JSON_LOGGING else "log"
|
||||
|
||||
|
||||
@ -46,6 +46,20 @@
|
||||
<p>
|
||||
<a href="{{object.start_url}}">Play again</a>
|
||||
</p>
|
||||
{% if object.genre.all %}
|
||||
<p>Genres:
|
||||
{% for tag in object.genre.all %}
|
||||
<a href="{% url 'boardgames:boardgame_list' %}?genre={{ tag.name|urlencode }}" class="badge bg-secondary text-decoration-none">{{ tag.name }}</a>
|
||||
{% endfor %}
|
||||
</p>
|
||||
{% endif %}
|
||||
{% if object.tags.all %}
|
||||
<p>Tags:
|
||||
{% for tag in object.tags.all %}
|
||||
<a href="{% url 'boardgames:boardgame_list' %}?tag={{ tag.name|urlencode }}" class="badge bg-secondary text-decoration-none">{{ tag.name }}</a>
|
||||
{% endfor %}
|
||||
</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if charts %}
|
||||
<div class="row">
|
||||
|
||||
@ -120,9 +120,67 @@
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% include "scrobbles/_top_charts.html" %}
|
||||
<div class="row mb-3">
|
||||
<div class="col-12">
|
||||
<a href="{% url 'charts:maloja-charts' %}" class="btn btn-sm btn-outline-secondary">🎨 Maloja Widgets</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
{% if charts.artist %}
|
||||
<div class="col-md-6 col-lg-4 chart-section">
|
||||
<h3>🎤 Top Artists</h3>
|
||||
<ul class="list-group">
|
||||
{% for chart in charts.artist|slice:":20" %}
|
||||
<li class="list-group-item d-flex justify-content-between align-items-center">
|
||||
<span class="me-2"><strong>#{{chart.rank}}</strong></span>
|
||||
<a href="{{chart.artist.get_absolute_url}}">{{chart.artist.name}}</a>
|
||||
<span class="badge bg-primary rounded-pill">{{chart.count}}</span>
|
||||
</li>
|
||||
{% endfor %}
|
||||
<li class="list-group-item">
|
||||
<a href="{% url 'charts:chart-detail' 'artist' %}">View all »</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if charts.album %}
|
||||
<div class="col-md-6 col-lg-4 chart-section">
|
||||
<h3>💿 Top Albums</h3>
|
||||
<ul class="list-group">
|
||||
{% for chart in charts.album|slice:":20" %}
|
||||
<li class="list-group-item d-flex justify-content-between align-items-center">
|
||||
<span class="me-2"><strong>#{{chart.rank}}</strong></span>
|
||||
<a href="{{chart.album.get_absolute_url}}">{{chart.album.name}}</a>
|
||||
<span class="badge bg-primary rounded-pill">{{chart.count}}</span>
|
||||
</li>
|
||||
{% endfor %}
|
||||
<li class="list-group-item">
|
||||
<a href="{% url 'charts:chart-detail' 'album' %}">View all »</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if charts.tv_series %}
|
||||
<div class="col-md-6 col-lg-4 chart-section">
|
||||
<h3>📺 Top TV Series</h3>
|
||||
<ul class="list-group">
|
||||
{% for chart in charts.tv_series|slice:":20" %}
|
||||
<li class="list-group-item d-flex justify-content-between align-items-center">
|
||||
<span class="me-2"><strong>#{{chart.rank}}</strong></span>
|
||||
<a href="{{chart.tv_series.get_absolute_url}}">{{chart.tv_series.name}}</a>
|
||||
<span class="badge bg-primary rounded-pill">{{chart.count}}</span>
|
||||
</li>
|
||||
{% endfor %}
|
||||
<li class="list-group-item">
|
||||
<a href="{% url 'charts:chart-detail' 'tv_series' %}">View all »</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<div class="row">
|
||||
{% if charts.track %}
|
||||
<div class="col-md-6 col-lg-4 chart-section">
|
||||
<h3>🎵 Top Tracks</h3>
|
||||
|
||||
45
vrobbler/templates/charts/maloja_charts.html
Normal file
45
vrobbler/templates/charts/maloja_charts.html
Normal file
@ -0,0 +1,45 @@
|
||||
{% extends "base_list.html" %}
|
||||
{% load static %}
|
||||
|
||||
{% block title %}Maloja Widgets{% endblock %}
|
||||
|
||||
{% block head_extra %}
|
||||
<style>
|
||||
.container { margin-bottom: 100px; }
|
||||
h2 { padding-top: 20px; }
|
||||
.nav-tabs { cursor: pointer; }
|
||||
.image-wrapper { contain: content; }
|
||||
.image-wrapper :hover { background: rgba(0,0,0,0.3); }
|
||||
.caption {
|
||||
position: fixed; top: 5px; left: 5px;
|
||||
padding: 3px; font-size: 90%;
|
||||
color: white; background: rgba(0,0,0,0.4);
|
||||
}
|
||||
.caption-medium {
|
||||
position: fixed; top: 5px; left: 5px;
|
||||
padding: 3px; font-size: 75%;
|
||||
color: white; background: rgba(0,0,0,0.4);
|
||||
}
|
||||
.caption-small {
|
||||
position: fixed; top: 5px; left: 5px;
|
||||
padding: 3px; font-size: 60%;
|
||||
color: white; background: rgba(0,0,0,0.4);
|
||||
}
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block lists %}
|
||||
|
||||
{% block grid_view_button %}{% endblock %}
|
||||
|
||||
<div class="row mb-3">
|
||||
<div class="col-12">
|
||||
<a href="{% url 'charts:charts-home' %}" class="btn btn-sm btn-outline-secondary">← Full Charts</a>
|
||||
<a href="{% url 'charts:spotify-tracks' %}" class="btn btn-sm btn-outline-secondary">🎵 Spotify Tracks</a>
|
||||
<a href="{% url 'charts:bandcamp-tracks' %}" class="btn btn-sm btn-outline-secondary">🎵 Bandcamp Tracks</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% include "scrobbles/_top_charts.html" %}
|
||||
|
||||
{% endblock %}
|
||||
@ -130,6 +130,11 @@
|
||||
</ul>
|
||||
</div>
|
||||
{% endif %}
|
||||
{% elif field.name == "trends_disabled" %}
|
||||
<p class="checkbox-row">
|
||||
{{ field }}
|
||||
{{ field.label_tag }}
|
||||
</p>
|
||||
{% elif field.name == "home_scrobble_limit" %}
|
||||
<p>
|
||||
{{ field.label_tag }}
|
||||
|
||||
@ -49,11 +49,12 @@
|
||||
</div>
|
||||
<div style="float:left; width:300px;">
|
||||
<div style="display:flex; flex-wrap: wrap;">
|
||||
{% for i in "67891011121314" %}
|
||||
{% with artists|get_item:forloop.counter|add:5 as artist %}
|
||||
{% for i in "123456789" %}
|
||||
{% with forloop.counter|add:4 as idx %}
|
||||
{% with artists|get_item:idx as artist %}
|
||||
{% if artist %}
|
||||
<div class="image-wrapper" style="width:33%">
|
||||
<div class="caption-small">#{{forloop.counter|add:6}} {{artist.artist.name}}</div>
|
||||
<div class="caption-small">#{{forloop.counter|add:5}} {{artist.artist.name}}</div>
|
||||
{% if artist.artist.thumbnail %}
|
||||
<a href="{{artist.artist.get_absolute_url}}"><img src="{{artist.artist.thumbnail_medium.url}}" width="100px"></a>
|
||||
{% else %}
|
||||
@ -62,6 +63,7 @@
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endwith %}
|
||||
{% endwith %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
@ -95,7 +97,7 @@
|
||||
<div style="display:block">
|
||||
<div style="float:left;">
|
||||
<div class="image-wrapper" style="display:flex; flex-wrap: wrap; margin:0">
|
||||
<div class="caption">#1 {{albums.0.album.title}}</div>
|
||||
<div class="caption">#1 {{albums.0.album.name}}</div>
|
||||
{% if albums.0.album.cover_image %}
|
||||
<a href="{{albums.0.album.get_absolute_url}}"><img src="{{albums.0.album.cover_image_medium.url}}" width="300px"></a>
|
||||
{% else %}
|
||||
@ -109,7 +111,7 @@
|
||||
{% with albums|get_item:forloop.counter as album %}
|
||||
{% if album %}
|
||||
<div class="image-wrapper" style="width:50%">
|
||||
<div class="caption-medium">#{{forloop.counter|add:1}} {{album.album.title}}</div>
|
||||
<div class="caption-medium">#{{forloop.counter|add:1}} {{album.album.name}}</div>
|
||||
{% if album.album.cover_image %}
|
||||
<a href="{{album.album.get_absolute_url}}"><img src="{{album.album.cover_image_medium.url}}" width="150px"></a>
|
||||
{% else %}
|
||||
@ -123,11 +125,12 @@
|
||||
</div>
|
||||
<div style="float:left; width:300px;">
|
||||
<div style="display:flex; flex-wrap: wrap;">
|
||||
{% for i in "67891011121314" %}
|
||||
{% with albums|get_item:forloop.counter|add:5 as album %}
|
||||
{% for i in "123456789" %}
|
||||
{% with forloop.counter|add:4 as idx %}
|
||||
{% with albums|get_item:idx as album %}
|
||||
{% if album %}
|
||||
<div class="image-wrapper" style="width:33%">
|
||||
<div class="caption-small">#{{forloop.counter|add:6}} {{album.album.title}}</div>
|
||||
<div class="caption-small">#{{forloop.counter|add:5}} {{album.album.name}}</div>
|
||||
{% if album.album.cover_image %}
|
||||
<a href="{{album.album.get_absolute_url}}"><img src="{{album.album.cover_image_medium.url}}" width="100px"></a>
|
||||
{% else %}
|
||||
@ -136,6 +139,7 @@
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endwith %}
|
||||
{% endwith %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
@ -197,11 +201,12 @@
|
||||
</div>
|
||||
<div style="float:left; width:300px;">
|
||||
<div style="display:flex; flex-wrap: wrap;">
|
||||
{% for i in "67891011121314" %}
|
||||
{% with shows|get_item:forloop.counter|add:5 as show %}
|
||||
{% for i in "123456789" %}
|
||||
{% with forloop.counter|add:4 as idx %}
|
||||
{% with shows|get_item:idx as show %}
|
||||
{% if show %}
|
||||
<div class="image-wrapper" style="width:33%">
|
||||
<div class="caption-small">#{{forloop.counter|add:6}} {{show.tv_series.name}}</div>
|
||||
<div class="caption-small">#{{forloop.counter|add:5}} {{show.tv_series.name}}</div>
|
||||
{% if show.tv_series.cover_image %}
|
||||
<a href="{{show.tv_series.get_absolute_url}}"><img src="{{show.tv_series.cover_small.url}}" width="100px" height="100px" style="object-fit: cover"></a>
|
||||
{% else %}
|
||||
@ -210,6 +215,7 @@
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endwith %}
|
||||
{% endwith %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -76,9 +76,11 @@
|
||||
<div class="btn-group me-2">
|
||||
<a href="{% url 'scrobbles:scrobble-list' %}" class="btn btn-sm btn-outline-secondary">Live</a>
|
||||
</div>
|
||||
{% if not user.profile.trends_disabled %}
|
||||
<div class="btn-group me-2">
|
||||
<a href="{% url 'trends:trends-home' %}" class="btn btn-sm btn-outline-secondary">Trends</a>
|
||||
</div>
|
||||
{% endif %}
|
||||
<div class="btn-group me-2">
|
||||
{% if user.profile.lastfm_username and not user.profile.lastfm_auto_import %}
|
||||
<form action="{% url 'scrobbles:lastfm-import' %}" method="get">
|
||||
|
||||
@ -16,6 +16,7 @@ from vrobbler.apps.boardgames.api.views import (
|
||||
BoardGameDesignerViewSet,
|
||||
BoardGamePublisherViewSet,
|
||||
BoardGameLocationViewSet,
|
||||
BoardGameVariantViewSet,
|
||||
)
|
||||
|
||||
from vrobbler.apps.books import urls as book_urls
|
||||
@ -146,6 +147,7 @@ router.register(r"boardgames", BoardGameViewSet)
|
||||
router.register(r"boardgame-designers", BoardGameDesignerViewSet)
|
||||
router.register(r"boardgame-publishers", BoardGamePublisherViewSet)
|
||||
router.register(r"boardgame-locations", BoardGameLocationViewSet)
|
||||
router.register(r"boardgame-variants", BoardGameVariantViewSet)
|
||||
router.register(r"podcast-producers", ProducerViewSet)
|
||||
router.register(r"podcast-episodes", PodcastEpisodeViewSet)
|
||||
router.register(r"podcasts", PodcastViewSet)
|
||||
|
||||
Reference in New Issue
Block a user