Compare commits

...

12 Commits
54.4 ... 55.2

Author SHA1 Message Date
68ff230f13 [release] Bump to version 55.2
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 2m0s
deploy / build-and-deploy (push) Successful in 30s
- Fix bug in scrobble id in calendar view
- Video game cleanup script should clear out broken images
2026-06-18 15:27:29 -04:00
57a952a6d1 [templates] Fix bug in calendar view
Some checks failed
build / test (push) Has been cancelled
2026-06-18 15:27:09 -04:00
718fcf7392 [videogames] Fix broken images in cleanup
All checks were successful
build / test (push) Successful in 2m0s
2026-06-18 15:24:47 -04:00
52adcf83c7 [release] Bump to version 55.1
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 30s
- Clean up metadata scrapping for video games
2026-06-18 15:08:46 -04:00
0061623f7e [videogames] Fix metadata scrapping for video games
Some checks failed
build / test (push) Has been cancelled
2026-06-18 15:08:23 -04:00
ec73e5151e [release] Bump to version 55.0
All checks were successful
build / test (push) Successful in 1m58s
deploy / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 37s
- Use pk ID for scrobble detail view, not uuid
- Display videogame screenshots on scrobble detail if they exist
- Add autotagging to webpages based on domain, title
2026-06-18 12:15:16 -04:00
2c90dd38b5 [project] Add todos 2026-06-18 12:14:58 -04:00
c6b1e42d7a [scrobbles] Use IDs not UUIDs in URLs
All checks were successful
build / test (push) Successful in 1m59s
2026-06-18 11:25:57 -04:00
fcf86d5b3f [scrobbles] Add screenshots to templates
All checks were successful
build / test (push) Successful in 2m6s
2026-06-18 10:54:28 -04:00
6fde9ec8d2 [webpages] Add autotagging to webpages
All checks were successful
build / test (push) Successful in 1m59s
2026-06-18 10:43:21 -04:00
0f1882b21f [release] Bump to version 54.5
All checks were successful
build / test (push) Successful in 2m7s
deploy / build-and-deploy (push) Has been skipped
deploy / test (push) Successful in 1m58s
- Fix bug in generating mood trends
2026-06-17 21:46:21 -04:00
e819a2db0d [trends] Fix bug in mood trend generation 2026-06-17 21:45:45 -04:00
35 changed files with 1011 additions and 125 deletions

View File

@ -88,7 +88,7 @@ fetching and simple saving.
*** Metadata sources
**** Scraper
* Backlog [0/20] :vrobbler:project:personal:
* Backlog [0/21] :vrobbler:project:personal:
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :bug:music:scrobbles:
:PROPERTIES:
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
@ -590,6 +590,51 @@ We should rename `email_scrobble_board_game` to reflect the fact that it's just
a helper method to create board game scrobbles given a json blob. It's
independent of the email flow it was originally creatdd for
** TODO [#B] Is there way to create unique slugs for media instances :media_types:
* Version 55.2 [2/2]
** DONE [#A] Fix bug in scrobble id in calendar view :templates:
:PROPERTIES:
:ID: 8cb34852-b18f-e794-cd9b-fb1ecad70a0d
:END:
** DONE [#A] Video game cleanup script should clear out broken images :metadata:videogames:
:PROPERTIES:
:ID: ca1f1ea9-0f79-082c-5ff7-867671faff4b
:END:
* Version 55.1 [1/1]
** DONE [#A] Clean up metadata scrapping for video games :metadata:videogames:
:PROPERTIES:
:ID: fbc421b5-21a3-4aed-9062-c59192ead065
:END:
* Version 55.0 [3/3]
** DONE [#B] Use pk ID for scrobble detail view, not uuid :scrobbles:
:PROPERTIES:
:ID: 9cc3b285-e478-041e-394b-3d550aefbe1d
:END:
** DONE [#B] Display videogame screenshots on scrobble detail if they exist :videogames:templates:
:PROPERTIES:
:ID: 0406d082-20f6-0d12-76e2-f281c4801468
:END:
** DONE [#B] Add autotagging to webpages based on domain, title :webpages:metadata:
:PROPERTIES:
:ID: f658435b-f7a0-42e6-b9f6-226678a77a55
:END:
*** Description
For easier filtering, like we do with tasks, we should auto tag WebPage instances
based on the domain name split part by periods (so news.ycombinator.com tags: news, ycombinator, com)
And also based on the nouns in the title.
* Version 54.5 [1/1]
** DONE Fix bug in generating mood trends :trends:
:PROPERTIES:
:ID: 8e75abfa-8e70-d85b-00a4-a4813bbce879
:END:
* Version 54.4 [2/2]
** DONE [#A] Remove all-time trends :trends:
:PROPERTIES:

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "vrobbler"
version = "54.4"
version = "55.2"
description = ""
authors = ["Colin Powell <colin@unbl.ink>"]

View File

@ -519,7 +519,7 @@ def test_scrobble_detail_view_with_notes_as_flat_list(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
assert "First note" in response.content.decode()
@ -545,7 +545,7 @@ def test_scrobble_detail_view_with_notes_as_dict_timestamps(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()
@ -574,7 +574,7 @@ def test_scrobble_detail_view_with_notes_and_labels(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()
@ -597,7 +597,7 @@ def test_scrobble_detail_view_post_updates_log(client):
"description": "Original description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
client.force_login(user)
response = client.post(
@ -896,7 +896,7 @@ def test_change_visibility_owner_can_change(client):
)
client.force_login(user)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 302
@ -919,7 +919,7 @@ def test_change_visibility_non_owner_gets_404(client):
)
client.force_login(other)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 404
@ -937,7 +937,7 @@ def test_change_visibility_anonymous_redirects_to_login(client):
task=task, media_type="Task", user=user, visibility="private",
timestamp=timezone.now(),
)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 302
assert "/login/" in response.url
@ -956,7 +956,7 @@ def test_regenerate_share_token_invalidates_old_sqid(client):
old_sqid = encode_scrobble_share(scrobble.id, scrobble.share_token_version)
client.force_login(user)
url = reverse("scrobbles:regenerate-share-token", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:regenerate-share-token", kwargs={"pk": scrobble.id})
response = client.post(url)
assert response.status_code == 302
@ -985,7 +985,7 @@ def test_share_analytics_owner_can_view(client):
)
client.force_login(user)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
@ -1005,7 +1005,7 @@ def test_share_analytics_non_owner_gets_404(client):
)
client.force_login(other)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 404
@ -1027,7 +1027,7 @@ def test_share_analytics_shows_view_logs(client):
client.get(share_url)
client.force_login(user)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()

View File

@ -225,7 +225,7 @@ class Book(LongPlayScrobblableMixin):
@property
def resume_start_url(self):
return reverse("scrobbles:start", kwargs={"uuid": self.uuid}) + "?resume=1"
return reverse("scrobbles:start", kwargs={"media_uuid": self.uuid}) + "?resume=1"
@classmethod
def get_from_comicvine(

View File

@ -114,7 +114,7 @@ class ScrobblableMixin(TimeStampedModel):
@property
def start_url(self):
return reverse("scrobbles:start", kwargs={"uuid": self.uuid})
return reverse("scrobbles:start", kwargs={"media_uuid": self.uuid})
@property
def strings(self) -> ScrobblableConstants:
@ -162,7 +162,7 @@ class LongPlayScrobblableMixin(ScrobblableMixin):
return False
def get_longplay_finish_url(self):
return reverse("scrobbles:longplay-finish", kwargs={"uuid": self.uuid})
return reverse("scrobbles:longplay-finish", kwargs={"media_uuid": self.uuid})
def first_long_play_scrobble_for_user(self, user) -> Optional["Scrobble"]:
last = self.last_long_play_scrobble_for_user(user)

View File

@ -798,6 +798,12 @@ class Scrobble(TimeStampedModel):
format="JPEG",
options={"quality": 75},
)
screenshot_large = ImageSpecField(
source="screenshot",
processors=[ResizeToFit(800, 800)],
format="JPEG",
options={"quality": 85},
)
long_play_seconds = models.BigIntegerField(**BNULL)
long_play_complete = models.BooleanField(**BNULL)
long_play_last_scrobble = models.ForeignKey(
@ -900,7 +906,7 @@ class Scrobble(TimeStampedModel):
@property
def finish_url(self) -> str:
return reverse("scrobbles:finish", kwargs={"uuid": self.uuid})
return reverse("scrobbles:finish", kwargs={"pk": self.pk})
def save(self, *args, **kwargs):
class_name = self.media_obj.__class__.__name__
@ -942,10 +948,7 @@ class Scrobble(TimeStampedModel):
return super(Scrobble, self).save(*args, **kwargs)
def get_absolute_url(self):
if not self.uuid:
self.uuid = uuid4()
self.save(update_fields=["uuid"])
return reverse("scrobbles:detail", kwargs={"uuid": self.uuid})
return reverse("scrobbles:detail", kwargs={"pk": self.pk})
def get_share_url(self):
if self.visibility == Visibility.PRIVATE:

View File

@ -44,7 +44,7 @@ urlpatterns = [
name="lookup-manual-scrobble",
),
path(
"long-play-finish/<slug:uuid>/",
"long-play-finish/<slug:media_uuid>/",
views.scrobble_longplay_finish,
name="longplay-finish",
),
@ -160,38 +160,38 @@ urlpatterns = [
name="shared-detail",
),
path(
"scrobbles/<slug:uuid>/",
"scrobbles/<int:pk>/",
views.ScrobbleDetailView.as_view(),
name="detail",
),
path(
"scrobbles/<slug:uuid>/regenerate-share-token/",
"scrobbles/<int:pk>/regenerate-share-token/",
views.RegenerateShareTokenView.as_view(),
name="regenerate-share-token",
),
path(
"scrobbles/<slug:uuid>/change-visibility/",
"scrobbles/<int:pk>/change-visibility/",
views.ChangeVisibilityView.as_view(),
name="change-visibility",
),
path(
"scrobbles/<slug:uuid>/share-analytics/",
"scrobbles/<int:pk>/share-analytics/",
views.ScrobbleShareAnalyticsView.as_view(),
name="share-analytics",
),
path(
"scrobbles/<slug:uuid>/add-to-mopidy-queue/",
"scrobbles/<int:pk>/add-to-mopidy-queue/",
views.add_to_mopidy_queue,
name="add-to-mopidy-queue",
),
path(
"scrobbles/<slug:uuid>/add-to-mopidy-monthly-playlist/",
"scrobbles/<int:pk>/add-to-mopidy-monthly-playlist/",
views.add_to_mopidy_monthly_playlist,
name="add-to-mopidy-monthly-playlist",
),
path("scrobbles/<slug:uuid>/start/", views.scrobble_start, name="start"),
path("scrobbles/<slug:uuid>/finish/", views.scrobble_finish, name="finish"),
path("scrobbles/<slug:uuid>/cancel/", views.scrobble_cancel, name="cancel"),
path("scrobbles/<slug:media_uuid>/start/", views.scrobble_start, name="start"),
path("scrobbles/<int:pk>/finish/", views.scrobble_finish, name="finish"),
path("scrobbles/<int:pk>/cancel/", views.scrobble_cancel, name="cancel"),
path(
"favorite/<str:media_type>/<int:object_id>/toggle/",
views.toggle_favorite,

View File

@ -1,4 +1,5 @@
import hashlib
import html
import logging
import requests
import re
@ -795,6 +796,7 @@ def tokenize_title_to_tags(title: str) -> list[str]:
if not title:
return []
title = html.unescape(title)
cleaned = re.sub(r"[\(\)\[\]\{\}]", "", title)
cleaned = re.sub(r"[^\w\s]", "", cleaned)

View File

@ -839,10 +839,10 @@ def import_audioscrobbler_file(request):
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_start(request, uuid):
def scrobble_start(request, media_uuid):
logger.info(
"[scrobble_start] called",
extra={"request": request, "uuid": uuid},
extra={"request": request, "media_uuid": media_uuid},
)
user = request.user
success_url = request.META.get("HTTP_REFERER")
@ -853,14 +853,14 @@ def scrobble_start(request, uuid):
media_obj = None
for app, model in PLAY_AGAIN_MEDIA.items():
media_model = apps.get_model(app_label=app, model_name=model)
media_obj = media_model.objects.filter(uuid=uuid).first()
media_obj = media_model.objects.filter(uuid=media_uuid).first()
if media_obj:
break
if not media_obj:
logger.info(
"[scrobble_start] media object not found",
extra={"uuid": uuid, "user_id": user.id},
extra={"media_uuid": media_uuid, "user_id": user.id},
)
raise Exception("No media object provided to scrobble")
@ -897,7 +897,7 @@ def scrobble_start(request, uuid):
)
else:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
if (
@ -915,7 +915,7 @@ def scrobble_start(request, uuid):
@api_view(["GET"])
def scrobble_longplay_finish(request, uuid):
def scrobble_longplay_finish(request, media_uuid):
user = request.user
success_url = request.META.get("HTTP_REFERER")
@ -923,7 +923,7 @@ def scrobble_longplay_finish(request, uuid):
return HttpResponseRedirect(success_url)
# Try scrobble UUID first
scrobble = Scrobble.objects.filter(uuid=uuid, user=user).first()
scrobble = Scrobble.objects.filter(uuid=media_uuid, user=user).first()
if scrobble:
if scrobble.long_play_complete == True:
scrobble.long_play_complete = None
@ -947,13 +947,13 @@ def scrobble_longplay_finish(request, uuid):
media_obj = None
for app, model in LONG_PLAY_MEDIA.items():
media_model = apps.get_model(app_label=app, model_name=model)
media_obj = media_model.objects.filter(uuid=uuid).first()
media_obj = media_model.objects.filter(uuid=media_uuid).first()
if media_obj:
break
if not media_obj:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
return HttpResponseRedirect(success_url)
@ -976,14 +976,14 @@ def scrobble_longplay_finish(request, uuid):
)
else:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
return HttpResponseRedirect(success_url)
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_finish(request, uuid):
def scrobble_finish(request, pk):
user = request.user
success_url = request.META.get("HTTP_REFERER")
if not success_url:
@ -992,7 +992,7 @@ def scrobble_finish(request, uuid):
if not user.is_authenticated:
return HttpResponseRedirect(success_url)
scrobble = Scrobble.objects.filter(user=user, uuid=uuid).first()
scrobble = Scrobble.objects.filter(user=user, pk=pk).first()
if scrobble:
scrobble.stop(force_finish=True)
messages.add_message(
@ -1007,14 +1007,14 @@ def scrobble_finish(request, uuid):
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_cancel(request, uuid):
def scrobble_cancel(request, pk):
user = request.user
success_url = reverse_lazy("vrobbler-home")
if not user.is_authenticated:
return HttpResponseRedirect(success_url)
scrobble = Scrobble.objects.filter(user=user, uuid=uuid).first()
scrobble = Scrobble.objects.filter(user=user, pk=pk).first()
if scrobble:
scrobble.cancel()
messages.add_message(
@ -1028,11 +1028,11 @@ def scrobble_cancel(request, uuid):
@require_POST
def add_to_mopidy_queue(request, uuid):
def add_to_mopidy_queue(request, pk):
if not request.user.is_authenticated:
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
mopidy_url = request.user.profile.mopidy_api_url
if not mopidy_url:
@ -1041,22 +1041,22 @@ def add_to_mopidy_queue(request, uuid):
messages.ERROR,
"Mopidy API URL not configured in your profile settings.",
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
from scrobbles.tasks import add_scrobble_to_mopidy_queue as task
task.delay(scrobble.id)
msg = f'Adding "{scrobble.media_obj}" to Mopidy queue.'
messages.add_message(request, messages.SUCCESS, msg)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
@require_POST
def add_to_mopidy_monthly_playlist(request, uuid):
def add_to_mopidy_monthly_playlist(request, pk):
if not request.user.is_authenticated:
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
profile = request.user.profile
pattern = profile.monthly_mopidy_playlist_pattern
@ -1066,7 +1066,7 @@ def add_to_mopidy_monthly_playlist(request, uuid):
messages.ERROR,
"Monthly playlist pattern or Mopidy API URL not configured in your profile.",
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
now = now_user_timezone(profile)
playlist_name = DateFormat(now).format(pattern)
@ -1079,7 +1079,7 @@ def add_to_mopidy_monthly_playlist(request, uuid):
messages.SUCCESS,
f'Adding "{scrobble.media_obj}" to monthly playlist "{playlist_name}".',
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
@require_POST
@ -1184,8 +1184,6 @@ class ScrobbleStatusView(LoginRequiredMixin, TemplateView):
class ScrobbleDetailView(DetailView):
model = Scrobble
slug_field = "uuid"
slug_url_kwarg = "uuid"
paginate_by = 100
def get_object(self, queryset=None):
@ -1385,15 +1383,15 @@ class ScrobbleExploreView(ListView):
class RegenerateShareTokenView(LoginRequiredMixin, View):
def post(self, request, uuid):
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
def post(self, request, pk):
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
scrobble.regenerate_share_token()
return redirect(scrobble.get_absolute_url())
class ChangeVisibilityView(LoginRequiredMixin, View):
def post(self, request, uuid):
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
def post(self, request, pk):
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
visibility = request.POST.get("visibility")
if visibility not in (Visibility.PUBLIC, Visibility.SHARED, Visibility.PRIVATE):
return redirect(scrobble.get_absolute_url())
@ -1404,8 +1402,6 @@ class ChangeVisibilityView(LoginRequiredMixin, View):
class ScrobbleShareAnalyticsView(LoginRequiredMixin, DetailView):
model = Scrobble
slug_field = "uuid"
slug_url_kwarg = "uuid"
template_name = "scrobbles/scrobble_share_analytics.html"
def get_queryset(self):
@ -1721,6 +1717,7 @@ class ScrobbleCalendarView(LoginRequiredMixin, TemplateView):
for scrobble in day_map[day_num]:
day_scrobbles.append(
{
"id": scrobble.pk,
"uuid": scrobble.uuid,
"emoji": self.MEDIA_EMOJI.get(scrobble.media_type, "📌"),
"title": (

View File

@ -1,13 +1,10 @@
from collections import Counter, defaultdict
from datetime import timedelta
from django.db.models import Count, Q
from django.db.models.functions import Extract
from django.utils import timezone
from django.db.models import Q
from scrobbles.models import Scrobble
def _mood_scrobbles(user, period="all_time"):
def _mood_scrobbles(user, period="last_30"):
from trends.utils import get_date_range
start, end = get_date_range(period)
@ -19,17 +16,25 @@ def _mood_scrobbles(user, period="all_time"):
return Scrobble.objects.filter(filters).select_related("mood")
def _parse_quality(raw):
try:
return int(raw)
except (TypeError, ValueError):
return None
def _avg_quality(values):
if not values:
nums = [v for v in values if v is not None]
if not nums:
return 0.0
return round(sum(values) / len(values), 2)
return round(sum(nums) / len(nums), 2)
def compute_mood_trajectory(user, period="all_time"):
def compute_mood_trajectory(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period).order_by("timestamp")
by_date = defaultdict(list)
for s in scrobbles:
quality = s.log.get("mood_quality")
quality = _parse_quality(s.log.get("mood_quality"))
if quality is not None:
day_key = s.timestamp.strftime("%Y-%m-%d")
by_date[day_key].append(quality)
@ -48,13 +53,13 @@ def compute_mood_trajectory(user, period="all_time"):
return {"trajectory": trajectory}
def compute_mood_by_time(user, period="all_time"):
def compute_mood_by_time(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
by_hour = defaultdict(list)
by_day = defaultdict(list)
for s in scrobbles:
quality = s.log.get("mood_quality")
quality = _parse_quality(s.log.get("mood_quality"))
if quality is not None and s.timestamp:
by_hour[s.timestamp.hour].append(quality)
by_day[s.timestamp.isoweekday()].append(quality)
@ -94,7 +99,7 @@ def compute_mood_by_time(user, period="all_time"):
return {"hours": hours, "days": days}
def compute_mood_distribution(user, period="all_time"):
def compute_mood_distribution(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
mood_counts = Counter()
type_counts = Counter()
@ -120,7 +125,7 @@ def compute_mood_distribution(user, period="all_time"):
}
def compute_mood_streaks(user, period="all_time"):
def compute_mood_streaks(user, period="last_30"):
scrobbles = list(
_mood_scrobbles(user, period).order_by("timestamp")
)
@ -169,13 +174,13 @@ def compute_mood_streaks(user, period="all_time"):
return {"streaks": streaks[:10], "current_streak": current_streak}
def compute_mood_weather(user, period="all_time"):
def compute_mood_weather(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
by_condition = defaultdict(list)
by_temp_range = defaultdict(list)
for s in scrobbles:
quality = s.log.get("mood_quality")
quality = _parse_quality(s.log.get("mood_quality"))
if quality is None:
continue
desc = s.log.get("weather_description")
@ -183,7 +188,11 @@ def compute_mood_weather(user, period="all_time"):
if desc:
by_condition[desc].append(quality)
if temp is not None:
bucket = f"{(int(temp) // 10) * 10}-{(int(temp) // 10) * 10 + 9}F"
try:
temp_f = float(temp)
except (TypeError, ValueError):
continue
bucket = f"{(int(temp_f) // 10) * 10}-{(int(temp_f) // 10) * 10 + 9}F"
by_temp_range[bucket].append(quality)
conditions = [

View File

@ -10,26 +10,27 @@ def hrs_to_secs(hrs: float) -> int:
return int(hrs * 60 * 60)
def lookup_game_from_hltb(name_or_id: str) -> Optional[dict]:
def lookup_game_from_hltb(name_or_id: str, search_by_title: bool = False) -> Optional[dict]:
"""Lookup game on HowLongToBeat.com via HLtB ID or a name string and return
the data in a dictonary mapped to our internal game fields
"""
hltb_game = {}
try:
hltb_id = int(name_or_id)
except ValueError:
hltb_id = None
if not search_by_title:
try:
hltb_id = int(name_or_id)
except ValueError:
hltb_id = None
if hltb_id:
hltb_game = HowLongToBeat().search_from_id(hltb_id)
logger.info(f"Found game on HLtB for ID {hltb_id}")
if hltb_id:
hltb_game = HowLongToBeat().search_from_id(hltb_id)
logger.info(f"Found game on HLtB for ID {hltb_id}")
if not hltb_game:
results = HowLongToBeat().search(name_or_id)
if not results:
logger.warn(f"Lookup of game on HLtB failed for ID {name_or_id}")
logger.warn(f"Lookup of game on HLtB failed via search {name_or_id!r}")
return
hltb_game = results[0]

View File

@ -19,6 +19,7 @@ GAMES_URL = "https://api.igdb.com/v4/games"
ALT_NAMES_URL = "https://api.igdb.com/v4/alternative_names"
SCREENSHOT_URL = "https://api.igdb.com/v4/screenshots"
COVER_URL = "https://api.igdb.com/v4/covers"
PLATFORMS_URL = "https://api.igdb.com/v4/platforms"
IGDB_CLIENT_ID = getattr(settings, "IGDB_CLIENT_ID")
IGDB_CLIENT_SECRET = getattr(settings, "IGDB_CLIENT_SECRET")
@ -35,6 +36,20 @@ def get_igdb_token() -> str:
return results.get("access_token")
def lookup_platform_names(platform_ids: list, headers: dict) -> list:
"""Resolve IGDB platform IDs to platform names"""
if not platform_ids:
return []
ids_str = ",".join(str(pid) for pid in platform_ids)
body = f"fields name; where id = ({ids_str});"
resp = requests.post(PLATFORMS_URL, data=body, headers=headers)
if resp.status_code != 200:
logger.warn(f"Failed to resolve platform IDs {platform_ids}")
return []
results = json.loads(resp.content)
return [p["name"] for p in results if "name" in p]
def lookup_game_id_from_gdb(name: str) -> str:
headers = {
@ -62,9 +77,10 @@ def lookup_game_id_from_gdb(name: str) -> str:
"details": results.get("details"),
},
)
# Sort our result by IDs so we always get the lowest ID, which is likely to be the least esoteric game
results = sorted(results, key=lambda k: k.get("game", 250000))
return results[0].get("game", "")
# Sort results by release date (oldest first) to prefer the original game
results = [r for r in results if r.get("game")]
results = sorted(results, key=lambda k: k.get("published_at") or 9999999999)
return results[0].get("game", "") if results else ""
def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
@ -118,6 +134,16 @@ def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
for genre in game.get("genres"):
genres.append(genre["name"])
platforms = []
if "release_dates" in game.keys():
platform_ids = set()
for rd in game["release_dates"]:
pid = rd.get("platform")
if pid is not None:
platform_ids.add(pid)
if platform_ids:
platforms = lookup_platform_names(list(platform_ids), headers)
game_dict = {
"igdb_id": game.get("id"),
"title": game.get("name"),
@ -129,6 +155,7 @@ def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
"release_date": release_date,
"summary": game.get("summary"),
"genres": genres,
"platforms": platforms,
}
return game_dict

View File

@ -0,0 +1,529 @@
import logging
import time
from django.core.management.base import BaseCommand
logger = logging.getLogger(__name__)
MISSING_ALL = [
"cover",
"screenshot",
"summary",
"rating",
"release_date",
"release_year",
"igdb_id",
"hltb_id",
]
MISSING_GROUPS = {
"cover": lambda g: not bool(g.cover),
"screenshot": lambda g: not bool(g.screenshot),
"summary": lambda g: not g.summary,
"rating": lambda g: g.rating is None,
"release_date": lambda g: g.release_date is None,
"release_year": lambda g: g.release_year is None,
"igdb_id": lambda g: g.igdb_id is None,
"hltb_id": lambda g: g.hltb_id is None,
}
def _game_matches(game, flags):
if not flags:
return False
for flag in flags:
fn = MISSING_GROUPS.get(flag)
if fn and fn(game):
return True
return False
class Command(BaseCommand):
help = "Backfill missing metadata on video games from IGDB and HowLongToBeat"
def add_arguments(self, parser):
parser.add_argument(
"--commit",
action="store_true",
help="Commit changes to the database",
)
parser.add_argument(
"--batch-size",
type=int,
default=100,
help="Number of games to process per batch (default: 100)",
)
parser.add_argument(
"--sleep",
type=float,
default=0.5,
help="Seconds to sleep between API calls (default: 0.5)",
)
parser.add_argument(
"--force",
action="store_true",
help="Re-fetch metadata even if data already exists",
)
parser.add_argument(
"--game-id",
type=int,
help="Only process a specific game by ID",
)
parser.add_argument(
"--fix-broken-images",
action="store_true",
help="Check and refetch broken/deleted game images (cover, screenshot, hltb_cover)",
)
for flag in MISSING_ALL:
parser.add_argument(
f"--missing-{flag}",
dest="missing_flags",
action="append_const",
const=flag,
help=f"Process games missing {flag}",
)
parser.add_argument(
"--all",
action="store_true",
dest="all_missing",
help="Process games missing any metadata field",
)
def handle(self, *args, **options):
from videogames.models import VideoGame
commit = options["commit"]
batch_size = options["batch_size"]
sleep_secs = options["sleep"]
force = options["force"]
game_id = options["game_id"]
fix_broken_images = options.get("fix_broken_images", False)
flags = options.get("missing_flags") or []
all_missing = options["all_missing"]
if all_missing:
flags = MISSING_ALL
if not flags and not game_id and not force and not fix_broken_images:
self.stdout.write(
"No filters specified. Use --all, --missing-*, --game-id, --force, or --fix-broken-images."
)
return
if game_id:
qs = VideoGame.objects.filter(id=game_id)
else:
qs = VideoGame.objects.all()
if flags:
qs = [g for g in qs.iterator() if _game_matches(g, flags)]
else:
qs = list(qs)
total = len(qs)
self.stdout.write(f"Found {total} games to process")
if not commit:
self.stdout.write(
"Dry run — no API calls will be made. Use --commit to run lookups."
)
return
title_mismatches = []
enriched = 0
skipped = 0
stats = {
"cover_fixed": 0,
"screenshot_fixed": 0,
"summary_fixed": 0,
"rating_fixed": 0,
"release_date_fixed": 0,
"release_year_fixed": 0,
"igdb_id_found": 0,
"hltb_id_found": 0,
"images_fixed": 0,
}
enriched_any = bool(flags or game_id or force)
if enriched_any:
for batch_num, offset in enumerate(range(0, len(qs), batch_size)):
batch = qs[offset : offset + batch_size]
for game in batch:
result = self._enrich_game(game, sleep_secs, force)
self._check_retroarch_name(game, title_mismatches)
if result:
enriched += 1
for key in stats:
if result.get(key):
stats[key] += 1
else:
skipped += 1
self.stdout.write(
f" Batch {batch_num + 1}: {offset + len(batch)}/{total}"
f"enriched: {enriched}, skipped: {skipped}"
)
if fix_broken_images:
broken_stats = self._fix_broken_images(qs, sleep_secs)
stats["images_fixed"] = broken_stats["images_fixed"]
self.stdout.write(
f"\nResults (commit={commit}):\n"
f" Games enriched: {enriched}\n"
f" Games skipped: {skipped}\n"
f" Covers fixed: {stats['cover_fixed']}\n"
f" Screenshots fixed: {stats['screenshot_fixed']}\n"
f" Summaries fixed: {stats['summary_fixed']}\n"
f" Ratings fixed: {stats['rating_fixed']}\n"
f" Release dates fixed: {stats['release_date_fixed']}\n"
f" Release years fixed: {stats['release_year_fixed']}\n"
f" IGDB IDs found: {stats['igdb_id_found']}\n"
f" HLtB IDs found: {stats['hltb_id_found']}"
)
if fix_broken_images:
self.stdout.write(f" Broken images fixed: {stats['images_fixed']}")
if title_mismatches:
self.stdout.write("\nTitle vs retroarch_name mismatches (not auto-fixed):")
for retroarch_name, title, game_id in title_mismatches:
self.stdout.write(
f" Game #{game_id}: retroarch_name={retroarch_name!r} vs title={title!r}"
)
def _clean_retroarch_name(self, name):
if not name:
return ""
name = name.strip()
if "(" in name:
name = name.split("(")[0].strip()
return name
def _check_retroarch_name(self, game, mismatches):
if not game.retroarch_name:
return
cleaned = self._clean_retroarch_name(game.retroarch_name)
if cleaned.lower() != game.title.lower():
mismatches.append((game.retroarch_name, game.title, game.id))
if "retroarch-mismatch" not in game.tags.names():
game.tags.add("retroarch-mismatch")
self.stdout.write(
f" [TAG] {game} — tagged as retroarch-mismatch"
)
def _enrich_game(self, game, sleep_secs, force):
from videogames.igdb import lookup_game_id_from_gdb, lookup_game_from_igdb
from videogames.howlongtobeat import lookup_game_from_hltb
search_name = self._clean_retroarch_name(game.retroarch_name) or game.title
changed = {}
if not game.hltb_id:
hltb_data = None
if search_name:
hltb_data = lookup_game_from_hltb(search_name, search_by_title=True)
time.sleep(sleep_secs)
if not hltb_data and game.title and game.title != search_name:
hltb_data = lookup_game_from_hltb(game.title, search_by_title=True)
time.sleep(sleep_secs)
if hltb_data:
result = self._apply_hltb_data(game, hltb_data, force)
if result:
changed.update(result)
igdb_data = None
if not game.igdb_id and search_name:
igdb_id = lookup_game_id_from_gdb(search_name)
time.sleep(sleep_secs)
if igdb_id:
igdb_data = lookup_game_from_igdb(str(igdb_id))
time.sleep(sleep_secs)
elif game.igdb_id:
igdb_data = lookup_game_from_igdb(str(game.igdb_id))
time.sleep(sleep_secs)
if igdb_data:
igdb_title = igdb_data.get("title", "")
igdb_title_clean = self._clean_retroarch_name(igdb_title)
if igdb_title_clean.lower() == search_name.lower():
if game.igdb_id is None and igdb_data.get("igdb_id"):
game.igdb_id = int(igdb_data["igdb_id"])
game.save(update_fields=["igdb_id"])
changed["igdb_id_found"] = True
self.stdout.write(f" [IGDB_ID] {game} — found IGDB ID {game.igdb_id}")
result = self._apply_igdb_data(game, igdb_data, force)
if result:
changed.update(result)
else:
self.stdout.write(
f" [IGDB] {game} — title mismatch (IGDB: {igdb_title!r} vs expected: {search_name!r}), re-searching…"
)
resolved = False
for candidate in (search_name, game.title if game.title != search_name else None):
if not candidate:
continue
new_id = lookup_game_id_from_gdb(candidate)
time.sleep(sleep_secs)
if not new_id:
continue
new_data = lookup_game_from_igdb(str(new_id))
time.sleep(sleep_secs)
if not new_data:
continue
new_title = new_data.get("title", "")
new_title_clean = self._clean_retroarch_name(new_title)
if new_title_clean.lower() == candidate.lower():
game.igdb_id = int(new_id)
if new_title and new_title != game.title:
game.title = new_title
changed["title_updated"] = True
self.stdout.write(f" [TITLE] {game} — updated title to {new_title!r} from IGDB")
game.save(update_fields=["igdb_id"] + (["title"] if changed.get("title_updated") else []))
changed["igdb_id_found"] = True
self.stdout.write(f" [IGDB_ID] {game} — re-found IGDB ID {game.igdb_id}")
if "igdb-mismatch" in game.tags.names():
game.tags.remove("igdb-mismatch")
self.stdout.write(f" [TAG] {game} — removed igdb-mismatch tag")
result = self._apply_igdb_data(game, new_data, force)
if result:
changed.update(result)
resolved = True
break
if not resolved and "igdb-mismatch" not in game.tags.names():
game.tags.add("igdb-mismatch")
self.stdout.write(f" [TAG] {game} — tagged igdb-mismatch")
# If retroarch-mismatch tag exists but no longer applies, remove it
if "retroarch-mismatch" in game.tags.names():
cleaned = self._clean_retroarch_name(game.retroarch_name or "")
if cleaned.lower() == game.title.lower():
game.tags.remove("retroarch-mismatch")
self.stdout.write(f" [TAG] {game} — removed retroarch-mismatch tag (title now matches)")
return changed if changed else None
def _apply_igdb_data(self, game, data, force):
from django.core.files.base import ContentFile
import requests
changed = {
"cover_fixed": False,
"screenshot_fixed": False,
"summary_fixed": False,
"rating_fixed": False,
"release_date_fixed": False,
}
update_fields = []
if data.get("alternative_name") and not game.alternative_name:
game.alternative_name = data["alternative_name"]
update_fields.append("alternative_name")
if data.get("summary") and (not game.summary or force):
game.summary = data["summary"]
update_fields.append("summary")
changed["summary_fixed"] = True
if data.get("rating") is not None and (game.rating is None or force):
game.rating = data["rating"]
update_fields.append("rating")
changed["rating_fixed"] = True
if data.get("rating_count") is not None and (game.rating_count is None or force):
game.rating_count = data["rating_count"]
update_fields.append("rating_count")
if data.get("release_date") and (game.release_date is None or force):
game.release_date = data["release_date"]
update_fields.append("release_date")
changed["release_date_fixed"] = True
if update_fields:
game.save(update_fields=update_fields)
self.stdout.write(f" [IGDB] {game}{', '.join(update_fields)}")
cover_url = data.get("cover_url")
if cover_url:
r = requests.get(cover_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.cover.save(fname, ContentFile(r.content), save=True)
changed["cover_fixed"] = True
self.stdout.write(f" [COVER] {game} — cover saved from IGDB")
screenshot_url = data.get("screenshot_url")
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
changed["screenshot_fixed"] = True
self.stdout.write(f" [SCREENSHOT] {game} — screenshot saved from IGDB")
genres = data.get("genres", [])
if genres:
existing = set(game.genre.names())
new_genres = [g for g in genres if g not in existing]
if new_genres:
game.genre.add(*new_genres)
self.stdout.write(f" [GENRES] {game} — added {len(new_genres)} genres")
platforms = data.get("platforms", [])
if platforms:
existing = set(game.platforms.values_list("name", flat=True))
new_platforms = [p for p in platforms if p not in existing]
if new_platforms:
from videogames.models import VideoGamePlatform
for name in new_platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
self.stdout.write(f" [PLATFORMS] {game} — added {len(new_platforms)} platforms")
if "igdb-enriched" not in game.tags.names():
game.tags.add("igdb-enriched")
self.stdout.write(f" [TAG] {game} — tagged igdb-enriched")
return changed if any(changed.values()) else None
def _apply_hltb_data(self, game, data, force):
from django.core.files.base import ContentFile
import requests
changed = {
"hltb_id_found": False,
"release_year_fixed": False,
}
update_fields = []
hltb_title = data.get("title", "")
if hltb_title and hltb_title != game.title:
game.title = hltb_title
update_fields.append("title")
self.stdout.write(f" [TITLE] {game} — updated title to {hltb_title!r}")
if data.get("hltb_id") and (game.hltb_id is None or force):
game.hltb_id = data["hltb_id"]
update_fields.append("hltb_id")
changed["hltb_id_found"] = True
self.stdout.write(f" [HLTB_ID] {game} — found HLtB ID {data['hltb_id']}")
if data.get("release_year") and (game.release_year is None or force):
game.release_year = data["release_year"]
update_fields.append("release_year")
changed["release_year_fixed"] = True
if data.get("main_story_time") and (game.main_story_time is None or force):
game.main_story_time = data["main_story_time"]
update_fields.append("main_story_time")
if data.get("main_extra_time") and (game.main_extra_time is None or force):
game.main_extra_time = data["main_extra_time"]
update_fields.append("main_extra_time")
if data.get("completionist_time") and (game.completionist_time is None or force):
game.completionist_time = data["completionist_time"]
update_fields.append("completionist_time")
if data.get("hltb_score") is not None and (game.hltb_score is None or force):
game.hltb_score = data["hltb_score"]
update_fields.append("hltb_score")
if update_fields:
game.save(update_fields=update_fields)
self.stdout.write(f" [HLTB] {game}{', '.join(update_fields)}")
cover_url = data.get("cover_url")
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
if r.status_code == 200:
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
self.stdout.write(f" [HLTB_COVER] {game} — cover saved from HLtB")
platforms = data.get("platforms", [])
if platforms:
existing = set(game.platforms.values_list("name", flat=True))
new_platforms = [p for p in platforms if p not in existing]
if new_platforms:
from videogames.models import VideoGamePlatform
for name in new_platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
self.stdout.write(f" [PLATFORMS] {game} — added {len(new_platforms)} platforms")
if "hltb-enriched" not in game.tags.names():
game.tags.add("hltb-enriched")
self.stdout.write(f" [TAG] {game} — tagged hltb-enriched")
return changed if any(changed.values()) else None
def _fix_broken_images(self, games, sleep_secs):
from django.core.files.base import ContentFile
import requests
from videogames.igdb import lookup_game_from_igdb
from videogames.howlongtobeat import lookup_game_from_hltb
stats = {"cover_fixed": 0, "screenshot_fixed": 0, "images_fixed": 0}
for game in games:
for field_name, source in [
("cover", "igdb"),
("screenshot", "igdb"),
("hltb_cover", "hltb"),
]:
field = getattr(game, field_name)
if not field.name:
continue
if field.storage.exists(field.name):
continue
self.stdout.write(
f" [IMAGE] {game}{field_name} is broken (file missing), refetching…"
)
if source == "igdb" and game.igdb_id:
data = lookup_game_from_igdb(str(game.igdb_id))
time.sleep(sleep_secs)
if not data:
continue
url = data.get("cover_url" if field_name == "cover" else "screenshot_url")
if not url:
continue
r = requests.get(url)
if r.status_code != 200:
continue
fname = f"{game.title}_{game.uuid}.jpg"
getattr(game, field_name).save(fname, ContentFile(r.content), save=True)
stats["images_fixed"] += 1
self.stdout.write(f" [IMAGE] {game}{field_name} refetched from IGDB")
elif source == "hltb" and game.hltb_id:
data = lookup_game_from_hltb(str(game.hltb_id))
time.sleep(sleep_secs)
if not data:
continue
url = data.get("cover_url")
if not url:
continue
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(url, headers=headers)
if r.status_code != 200:
continue
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
stats["images_fixed"] += 1
self.stdout.write(f" [IMAGE] {game} — hltb_cover refetched from HLtB")
return stats

View File

@ -215,12 +215,16 @@ class VideoGame(LongPlayScrobblableMixin):
def fix_metadata(self, force_update: bool = False):
from videogames.utils import (
get_or_create_videogame,
load_game_data_from_hltb,
load_game_data_from_igdb,
)
if self.hltb_id and force_update:
get_or_create_videogame(str(self.hltb_id), force_update)
if not self.hltb_id:
load_game_data_from_hltb(self.id)
if not self.igdb_id:
# This almost never works without intervention
self.igdb_id = lookup_game_id_from_gdb(self.title)

View File

@ -153,6 +153,13 @@ def import_retroarch_lrtl_files(playlog_path: str, user_id: int) -> List[dict]:
continue
logger.info(f"Queued scrobble for game {found_game.id}")
log_data = {"emulated": True}
if last_scrobble and last_scrobble.log:
prev = last_scrobble.log
if prev.get("emulator"):
log_data["emulator"] = prev["emulator"]
new_scrobbles.append(
Scrobble(
video_game_id=found_game.id,
@ -168,6 +175,7 @@ def import_retroarch_lrtl_files(playlog_path: str, user_id: int) -> List[dict]:
user_id=user_id,
source="Retroarch",
media_type=Scrobble.MediaType.VIDEO_GAME,
log=log_data,
)
)
created_scrobbles = Scrobble.objects.bulk_create(new_scrobbles)

View File

@ -7,8 +7,6 @@ from videogames.howlongtobeat import lookup_game_from_hltb
from videogames.igdb import lookup_game_from_igdb
from videogames.models import VideoGame, VideoGamePlatform
from vrobbler.apps.videogames.exceptions import GameNotFound
logger = logging.getLogger(__name__)
@ -16,22 +14,33 @@ def get_or_create_videogame(
name_or_id: str,
force_update: bool = False,
) -> Optional[VideoGame]:
"""Look up game by name or ID from HowLongToBeat"""
"""Look up game by name or ID from HowLongToBeat, then enrich with IGDB"""
game_dict = lookup_game_from_hltb(name_or_id)
hltb_data = lookup_game_from_hltb(name_or_id)
if not game_dict:
game_dict = lookup_game_from_igdb(name_or_id)
if hltb_data:
game = _create_update_from_dict(hltb_data, force_update)
else:
igdb_data = lookup_game_from_igdb(name_or_id)
if igdb_data:
game = _create_update_from_dict(igdb_data, force_update)
else:
return None
if not game_dict:
return
if game:
game.fix_metadata()
return game
def _create_update_from_dict(
game_dict: dict, force_update: bool = False
) -> Optional[VideoGame]:
# Create missing platforms and prep for loading after create
platform_ids = []
if "platforms" in game_dict.keys():
platforms = game_dict.get("platforms", [])
if platforms:
for platform in game_dict.get("platforms", []):
for platform in platforms:
p, _created = VideoGamePlatform.objects.get_or_create(name=platform)
platform_ids.append(p.id)
game_dict.pop("platforms")
@ -48,7 +57,7 @@ def get_or_create_videogame(
title = game_dict.get("title")
if not title:
raise GameNotFound(name_or_id)
return None
hltb_id = game_dict.get("hltb_id")
igdb_id = game_dict.get("igdb_id")
@ -69,21 +78,19 @@ def get_or_create_videogame(
VideoGame.objects.filter(pk=game.id).update(**game_dict)
game.refresh_from_db()
# Associate plaforms
if platform_ids:
game.platforms.add(*platform_ids)
if genres:
game.genre.add(*genres)
if not game.screenshot and screenshot_url:
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
# Go get cover image if the URL is present
if cover_url and not game.hltb_cover:
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
logger.debug(r.status_code)
@ -91,12 +98,89 @@ def get_or_create_videogame(
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
logger.debug("Loaded cover image from HLtB")
game.fix_metadata()
tag = "hltb-enriched" if hltb_id else "igdb-enriched"
if tag not in game.tags.names():
game.tags.add(tag)
logger.info(f"Game {game} tagged {tag}")
return game
def load_game_data_from_igdb(game_id: int, igdb_id: str = "") -> Optional[VideoGame]:
def load_game_data_from_hltb(
game_id: int, expected_title: str = ""
) -> Optional[VideoGame]:
"""Look up HLtB data for an existing game and apply it"""
game = VideoGame.objects.filter(id=game_id).first()
if not game:
logger.warn(f"Video game with ID {game_id} not found")
return
logger.info(f"Looking up HLtB data for {game}")
hltb_data = lookup_game_from_hltb(game.title, search_by_title=True)
if not hltb_data:
logger.warn(f"No HLtB data found for {game}")
return
update_fields = []
hltb_title = hltb_data.get("title", "")
if hltb_title and hltb_title != game.title:
game.title = hltb_title
update_fields.append("title")
logger.info(f"Game {game.id} title updated to {hltb_title!r}")
if hltb_data.get("hltb_id") and (game.hltb_id is None):
game.hltb_id = hltb_data["hltb_id"]
update_fields.append("hltb_id")
if hltb_data.get("release_year") and (game.release_year is None):
game.release_year = hltb_data["release_year"]
update_fields.append("release_year")
if hltb_data.get("main_story_time") and (game.main_story_time is None):
game.main_story_time = hltb_data["main_story_time"]
update_fields.append("main_story_time")
if hltb_data.get("main_extra_time") and (game.main_extra_time is None):
game.main_extra_time = hltb_data["main_extra_time"]
update_fields.append("main_extra_time")
if hltb_data.get("completionist_time") and (game.completionist_time is None):
game.completionist_time = hltb_data["completionist_time"]
update_fields.append("completionist_time")
if hltb_data.get("hltb_score") is not None and (game.hltb_score is None):
game.hltb_score = hltb_data["hltb_score"]
update_fields.append("hltb_score")
if update_fields:
game.save(update_fields=update_fields)
platforms = hltb_data.get("platforms", [])
if platforms:
for name in platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
cover_url = hltb_data.get("cover_url")
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
if r.status_code == 200:
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
if "hltb-enriched" not in game.tags.names():
game.tags.add("hltb-enriched")
logger.info(f"Game {game} tagged hltb-enriched")
return game
def load_game_data_from_igdb(
game_id: int, igdb_id: str = "", expected_title: str = ""
) -> Optional[VideoGame]:
"""Look up game, if it doesn't exist, lookup data from igdb"""
game = VideoGame.objects.filter(id=game_id).first()
if not game:
@ -116,25 +200,68 @@ def load_game_data_from_igdb(game_id: int, igdb_id: str = "") -> Optional[VideoG
logger.warn(f"No game data found on IGDB for ID {igdb_id}")
return
igdb_title = game_dict.get("title", "")
igdb_title_clean = igdb_title.split(" (")[0].strip() if " (" in igdb_title else igdb_title
expected = expected_title or game.title
if igdb_title_clean.lower() != expected.lower():
logger.info(
f"IGDB title {igdb_title!r} doesn't match expected {expected!r} for {game} — re-searching…"
)
from videogames.igdb import lookup_game_id_from_gdb
new_id = lookup_game_id_from_gdb(expected)
if new_id:
new_data = lookup_game_from_igdb(str(new_id))
if new_data:
new_title = new_data.get("title", "")
new_title_clean = new_title.split(" (")[0].strip() if " (" in new_title else new_title
if new_title_clean.lower() == expected.lower():
game_dict = new_data
igdb_id = int(new_id)
if game.igdb_id != igdb_id:
game.igdb_id = igdb_id
game.save(update_fields=["igdb_id"])
logger.info(f"Game {game} IGDB ID updated to {igdb_id}")
igdb_title = game_dict.get("title", "")
igdb_title_clean = igdb_title.split(" (")[0].strip() if " (" in igdb_title else igdb_title
if igdb_title_clean.lower() != expected.lower():
if "igdb-mismatch" not in game.tags.names():
game.tags.add("igdb-mismatch")
logger.info(
f"Game {game} tagged igdb-mismatch (IGDB: {igdb_title!r} vs expected: {expected!r})"
)
return
screenshot_url = game_dict.pop("screenshot_url")
cover_url = game_dict.pop("cover_url")
genres = game_dict.pop("genres")
platforms = game_dict.pop("platforms", [])
VideoGame.objects.filter(pk=game.id).update(**game_dict)
game.refresh_from_db()
game.genre.add(*genres)
if not game.screenshot and screenshot_url:
if platforms:
for name in platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
if not game.cover and cover_url:
if cover_url:
r = requests.get(cover_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.cover.save(fname, ContentFile(r.content), save=True)
if "igdb-enriched" not in game.tags.names():
game.tags.add("igdb-enriched")
logger.info(f"Game {game} tagged igdb-enriched")
return game

View File

@ -3,3 +3,6 @@ from django.apps import AppConfig
class WebpagesConfig(AppConfig):
name = "webpages"
def ready(self):
import webpages.signals # noqa

View File

@ -0,0 +1,68 @@
import re
from django.core.management.base import BaseCommand
from scrobbles.utils import tokenize_title_to_tags
from webpages.models import WebPage
def _clean(s: str) -> str:
return re.sub(r"[^\x20-\x7e]", "", s)
class Command(BaseCommand):
help = "Backfill auto tags on webpages from domain and title"
def add_arguments(self, parser):
parser.add_argument(
"--commit",
action="store_true",
help="Actually add tags",
)
def handle(self, *args, **options):
commit = options["commit"]
webpages = WebPage.objects.all()
total = webpages.count()
updated_count = 0
skipped_count = 0
for i, webpage in enumerate(webpages.iterator(), start=1):
new_tags = set()
if webpage.domain:
parts = webpage.domain.root.split(".")
for part in parts:
part = part.strip().lower()
if part and part != "www":
new_tags.add(part)
if webpage.title:
title_tags = tokenize_title_to_tags(webpage.title)
new_tags.update(title_tags)
existing_tags = {
t.name for t in webpage.tags.all()
}
tags_to_add = new_tags - existing_tags
if tags_to_add:
updated_count += 1
if commit:
for tag in tags_to_add:
webpage.tags.add(tag)
self.stdout.write(
f"[{i}/{total}] Added tags to {_clean(str(webpage))}: "
f"{sorted(tags_to_add)}"
)
else:
self.stdout.write(
f"[{i}/{total}] [DRY RUN] Would add tags to "
f"{_clean(str(webpage))}: {sorted(tags_to_add)}"
)
else:
skipped_count += 1
self.stdout.write(f"\nDone. {updated_count} webpages to update, "
f"{skipped_count} already up to date.")

View File

@ -0,0 +1,29 @@
import logging
from django.db.models.signals import post_save
from django.dispatch import receiver
from scrobbles.utils import tokenize_title_to_tags
from webpages.models import WebPage
logger = logging.getLogger(__name__)
@receiver(post_save, sender=WebPage)
def add_auto_tags_to_webpage(sender, instance, **kwargs):
existing_tags = {t.name for t in instance.tags.all()}
if instance.domain:
domain_parts = instance.domain.root.split(".")
for part in domain_parts:
part = part.strip().lower()
if part and part != "www" and part not in existing_tags:
instance.tags.add(part)
existing_tags.add(part)
if instance.title:
title_tags = tokenize_title_to_tags(instance.title)
for tag in title_tags:
if tag not in existing_tags:
instance.tags.add(tag)
existing_tags.add(tag)

View File

@ -16,8 +16,23 @@ from webpages.models import WebPage
class WebPageListView(ScrobbleableListView):
model = WebPage
def get_queryset(self):
queryset = super().get_queryset()
tag_name = self.request.GET.get("tag")
if tag_name:
from django.contrib.contenttypes.models import ContentType
from taggit.models import TaggedItem
ct = ContentType.objects.get_for_model(WebPage)
webpages = TaggedItem.objects.filter(
content_type=ct,
tag__name__iexact=tag_name,
).values_list("object_id", flat=True)
queryset = queryset.filter(pk__in=webpages)
return queryset
def get_context_data(self, **kwargs):
context_data = super().get_context_data(**kwargs)
context_data["active_tag"] = self.request.GET.get("tag", "")
user = self.request.user
now = timezone.now()
start_day_of_week = now - datetime.timedelta(days=now.weekday())

View File

@ -322,8 +322,8 @@
<span class="progress-bar-fill" style="width: {{scrobble.percent_played}}%;"></span>
</div>
<p class="action-buttons">
<a href="{% url "scrobbles:cancel" scrobble.uuid %}">Cancel</a>
<a class="right" href="{% url "scrobbles:finish" scrobble.uuid %}">Finish</a>
<a href="{% url "scrobbles:cancel" scrobble.id %}">Cancel</a>
<a class="right" href="{% url "scrobbles:finish" scrobble.id %}">Finish</a>
</p>
{% if not forloop.last %}<hr/>{% endif %}
</div>

View File

@ -65,7 +65,7 @@
{% for scrobble in scrobbles.all|dictsortreversed:"timestamp" %}
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now reading{% else %}{{scrobble.session_pages_read}}{% endif %}</td>
<td>{% for author in scrobble.book.authors.all %}<a href="{{author.get_absolute_url}}">{{author}}</a>{% if not forloop.last %}, {% endif %}{% endfor %}</td>
</tr>

View File

@ -48,7 +48,7 @@
{% for scrobble in scrobbles.all|dictsortreversed:"timestamp" %}
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now reading{% else %}{{scrobble.session_pages_read}}{% endif %}</td>
</tr>
{% endfor %}

View File

@ -1,7 +1,7 @@
{% load humanize %}
{% load naturalduration %}
<tr {% if scrobble.in_progress %}class="in-progress"{% endif %}>
<td>{% if scrobble.in_progress %}<a href="{{scrobble.get_absolute_url}}">{{scrobble.media_obj.strings.verb}} now</a> | <a class="right" href="{% url "scrobbles:finish" scrobble.uuid %}">Finish</a>{% else %}<a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp|naturaltime}}</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}<a href="{{scrobble.get_absolute_url}}">{{scrobble.media_obj.strings.verb}} now</a> | <a class="right" href="{% url "scrobbles:finish" scrobble.id %}">Finish</a>{% else %}<a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp|naturaltime}}</a>{% endif %}</td>
<td>
{% if scrobble.media_type in "Task" %}
<p><em><a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj.title|truncatechars_html:45}} - {% if scrobble.logdata %}{% if scrobble.logdata.title %}{{scrobble.logdata.title}}{% endif %}{% endif %}</a></em></p>

View File

@ -205,7 +205,7 @@ header.navbar { display: none !important; }
style="background:{% if not cd.is_today %}{{ cd.color }}{% endif %};">
<div class="day-number"><a href="{% url 'vrobbler-home' %}?date={{ year }}-{{ month|stringformat:'02d' }}-{{ cd.day|stringformat:'02d' }}" style="color:inherit;text-decoration:none;">{{ cd.day }}</a>{% if cd.total_count > 0 %} <span style="font-weight:400;color:#999;font-size:0.75rem;">{{ cd.total_count }}</span>{% endif %}</div>
{% for s in cd.scrobbles %}
<a href="{% url 'scrobbles:detail' uuid=s.uuid %}"
<a href="{% url 'scrobbles:detail' pk=s.id %}"
class="event-card"
title="{{ s.title }} — {{ s.media_type }}">{{ s.emoji }} {{ s.title }}</a>
{% endfor %}

View File

@ -88,7 +88,7 @@
{% endif %}">
{{ object.get_visibility_display }}
</span>
<form method="post" action="{% url 'scrobbles:change-visibility' object.uuid %}" class="d-inline-flex align-items-center gap-1">
<form method="post" action="{% url 'scrobbles:change-visibility' object.id %}" class="d-inline-flex align-items-center gap-1">
{% csrf_token %}
<select name="visibility" class="form-select form-select-sm" style="width:auto;" onchange="this.form.submit()">
<option value="private" {% if object.visibility == 'private' %}selected{% endif %}>Private</option>
@ -100,25 +100,25 @@
<span class="small text-muted">Share link:</span>
<code class="small" id="share-link">{{ request.scheme }}://{{ request.get_host }}{{ object.get_share_url }}</code>
<button class="btn btn-sm btn-outline-secondary" onclick="navigator.clipboard.writeText(document.getElementById('share-link').textContent.trim())">Copy</button>
<form method="post" action="{% url 'scrobbles:regenerate-share-token' object.uuid %}" class="d-inline">
<form method="post" action="{% url 'scrobbles:regenerate-share-token' object.id %}" class="d-inline">
{% csrf_token %}
<button type="submit" class="btn btn-sm btn-outline-warning">Regenerate</button>
</form>
{% if object.share_view_count %}
<span class="text-muted small ms-2">{{ object.share_view_count }} view{{ object.share_view_count|pluralize }}</span>
{% endif %}
<a href="{% url 'scrobbles:share-analytics' object.uuid %}" class="btn btn-sm btn-outline-info ms-2">Analytics</a>
<a href="{% url 'scrobbles:share-analytics' object.id %}" class="btn btn-sm btn-outline-info ms-2">Analytics</a>
{% endif %}
</div>
{% endif %}
{% if object.media_type == "Track" and has_mopidy_uri and user.profile.mopidy_api_url %}
<form method="post" action="{% url 'scrobbles:add-to-mopidy-queue' object.uuid %}" class="mb-1">
<form method="post" action="{% url 'scrobbles:add-to-mopidy-queue' object.id %}" class="mb-1">
{% csrf_token %}
<button type="submit" class="btn btn-sm btn-outline-secondary">add to mopidy queue</button>
</form>
{% if user.profile.monthly_mopidy_playlist_pattern %}
<form method="post" action="{% url 'scrobbles:add-to-mopidy-monthly-playlist' object.uuid %}" class="mb-1">
<form method="post" action="{% url 'scrobbles:add-to-mopidy-monthly-playlist' object.id %}" class="mb-1">
{% csrf_token %}
<button type="submit" class="btn btn-sm btn-outline-secondary">add to monthly playlist</button>
</form>
@ -185,6 +185,12 @@
</div>
{% endif %}
{% if object.screenshot %}
<div class="mb-3">
<img src="{{ object.screenshot_large.url }}" class="img-fluid rounded" style="max-height: 600px;" alt="Screenshot" />
</div>
{% endif %}
<p>
Tags:
{% if object.tags.all %}
@ -282,7 +288,7 @@
<tbody>
{% for scrobble in related_scrobbles %}
<tr{% if scrobble.id == object.id %} class="table-active fw-bold"{% endif %}>
<td>{% if scrobble.id == object.id %}{{ scrobble.timestamp|date:"M d, Y" }}{% else %}<a href="{% url 'scrobbles:detail' scrobble.uuid %}">{{ scrobble.timestamp|date:"M d, Y" }}</a>{% endif %}</td>
<td>{% if scrobble.id == object.id %}{{ scrobble.timestamp|date:"M d, Y" }}{% else %}<a href="{% url 'scrobbles:detail' scrobble.id %}">{{ scrobble.timestamp|date:"M d, Y" }}</a>{% endif %}</td>
<td>
{% if scrobble.media_type == "Task" and scrobble.logdata.title %}{{ scrobble.media_obj.title }}: {{ scrobble.logdata.title }}{% else %}{{ scrobble.media_obj.title|default:scrobble.media_obj }}{% endif %}
</td>

View File

@ -103,7 +103,7 @@
{% for scrobble in scrobbles %}
<div class="result-item">
<div class="result-title">
<a href="{% url 'scrobbles:detail' scrobble.uuid %}">
<a href="{% url 'scrobbles:detail' scrobble.id %}">
{{ scrobble|truncatechars:100 }}
</a>
</div>

View File

@ -117,6 +117,12 @@
</div>
{% endif %}
{% if object.screenshot %}
<div class="mb-3">
<img src="{{ object.screenshot_large.url }}" class="img-fluid rounded" style="max-height: 600px;" alt="Screenshot" />
</div>
{% endif %}
<p>
Tags:
{% if object.tags.all %}

View File

@ -81,7 +81,7 @@
<td><a href="{{scrobble.get_media_source_url}}">{{scrobble.logdata.title}}</a></td>
<td>{{scrobble.logdata.notes_as_str}}</td>
<td>{{scrobble.source}}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">No</a>{% endif %}</td>
</tr>
{% endfor %}
</tbody>

View File

@ -87,7 +87,7 @@
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">Not yet</a>{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' media_uuid=scrobble.uuid %}">Not yet</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now playing{% else %}{{scrobble.playback_position_seconds|natural_duration}}{% endif %}</td>
<td>{% for platform in scrobble.video_game.platforms.all %}<a href="{{platform.get_absolute_url}}">{{platform}}</a>{% if not forloop.last %}, {% endif %}{% endfor %}</td>
<td>{% if scrobble.videogame_save_data %}<a href="{{scrobble.videogame_save_data.url}}">Save data</a>{% else %}Not yet{% endif %}</td>

View File

@ -204,6 +204,13 @@
<p>Source: <a href="{{object.url}}">{{object.domain}}</a></p>
{% if object.date %}<p>Published: <em>{{object.date}}</em></p>{% endif %}
<p>Time to read: {{object.estimated_time_to_read_in_minutes}} minutes</p>
{% if object.tags.all %}
<p>Tags:
{% for tag in object.tags.all %}
<a href="{% url 'webpages:webpage_list' %}?tag={{ tag.name|urlencode }}">{{ tag.name }}</a>{% if not forloop.last %}, {% endif %}
{% endfor %}
</p>
{% endif %}
</div>
{% if object.extract %}
<div class="col">