Compare commits

..

9 Commits
47.2 ... 48.1

Author SHA1 Message Date
568772a0e6 [release] Bump to version 48.1
All checks were successful
deploy / test (push) Successful in 2m0s
build / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 35s
- Generate a report of tracks with mistmatched metadata
- Date parsing failing in eBird imports
2026-06-08 11:18:08 -04:00
91c3376256 [music] Add mgmt command to see mismatched metadata
Some checks failed
build / test (push) Has been cancelled
2026-06-08 11:17:36 -04:00
58639c6fc1 [importers] Add error_log to surface import errors
All checks were successful
build / test (push) Successful in 2m2s
2026-06-08 10:58:02 -04:00
228441ddc5 [importers] Fix parsing of dates in ebird files 2026-06-08 10:52:50 -04:00
6341075f07 [agent] Add some notes about how we track tasks 2026-06-08 10:52:32 -04:00
a135b9f5f2 [project] Update format for PROJECT file
All checks were successful
build / test (push) Successful in 1m58s
2026-06-07 11:10:30 -04:00
9088412d1e [release] Bump to version 48.0
All checks were successful
build / test (push) Successful in 2m1s
deploy / test (push) Successful in 2m2s
deploy / build-and-deploy (push) Successful in 33s
- Show team or player images on sport detail and scrobble detail
- Add fix_metadta method to Video instances
2026-06-07 11:06:58 -04:00
c7339fbe31 [templates] Clean up how str and subtitles work 2026-06-07 11:06:30 -04:00
4ce3dc03c5 [videos] Add fix_metadata for videos 2026-06-07 10:13:55 -04:00
22 changed files with 685 additions and 140 deletions

View File

@ -14,3 +14,11 @@ ro class method should call the utility function.
Be sure to check pyproject.toml for project defaults. Specifically for black and Be sure to check pyproject.toml for project defaults. Specifically for black and
isort expectations. isort expectations.
All tasks live in the PROJECT.org file and include an org ID that is a uuid to make them unique.
In local development, environment variables for various sensitive values live in a .envrc file
The .envrc file can be loaded into a shell environment to allow access to most third party services
Care should be taken when using .envrc that we do not spam services we use in production with requests

View File

@ -17,6 +17,7 @@ tasks, Todoist tasks, web pages I've read and trails I've hiked has turned out
to be sometimes cathartic and sometimes functional as I try to remember when I to be sometimes cathartic and sometimes functional as I try to remember when I
did a thing. did a thing.
* Features * Features
** Beer ** Beer
*** Triggers *** Triggers
@ -86,7 +87,8 @@ fetching and simple saving.
**** Bookmarklet **** Bookmarklet
*** Metadata sources *** Metadata sources
**** Scraper **** Scraper
* Backlog [0/12] :vrobbler:project:personal:
* Backlog [0/14] :vrobbler:project:personal:
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles: ** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
:PROPERTIES: :PROPERTIES:
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb :ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
@ -498,6 +500,116 @@ needed import celery task. This is how the WebDAV celery task currently works.
This would also be an opporunity to clean up the code around WebDAV imports This would also be an opporunity to clean up the code around WebDAV imports
and make them more re-usable for other import services. and make them more re-usable for other import services.
** TODO [#A] Add an exception list of artists as a constant that are exempted from splitting :music:artists:metadata:
:PROPERTIES:
:ID: fd86a11a-73ec-470d-b5e3-2d90ba9137c8
:END:
*** Description
Certain artists like "Simon & Garfunkel" are actually one artist. While we don't want to mess with splitting up
tracks into featured artists, we should have a "LITERAL_ARTIST_TITLES" constant that can have exceptions like
this put into it and then we stop trying to pull the artist apart when we run into it.
** TODO [#A] Before enriching anything, trust the POST data :feature:scrobbles:metadata:
:PROPERTIES:
:ID: db6b05f8-09f4-49f5-9838-fbacc9fe9cd0
:END:
*** Description
Both Jellyfin and Mopidy provide a decent amount of metadata when they POST to our webhooks.
In most cases, we should be able to trust this data to created music tracks or videos rather
than going to third-party services to enrich. Thus, for tracks and videos we should search in
the local database for imdb_id or musicbrainz_id for the specific content and, if found, not
enrich further.
If not found, tracks and videos from mopidy and jellyfin should be created as completely as
possible using only the POST data from the webhooks, tagged the scrobble with "webhook-metadata-only"
and start the scrobble. A separate celery task should be kicked off to enrich the track or video
async with the POST data stored in the log["raw_data"] and used by the celery enrichment task
to go try to enrich the media instance. Should this enrichment fail, tag the scrobble as "enrichment-failed"
log a warning and move on.
* Version 48.1 [2/2]
** DONE [#A] Generate a report of tracks with mistmatched metadata :music:tracks:metadata:
:PROPERTIES:
:ID: 684b8cd2-a3c1-4995-ba9e-7abdb02c37f2
:END:
*** Description
We should have a management command that outputs a CSV file of track IDs where
the log["raw_data"]["Artist"] (for Jellyfin) or log["raw_data"]["artist"]
(mopidy) value does not match the Track.artists names.
And we should see the same thing for albums (log["raw_data"]["Album"] or
log["raw_data]["album"]).
It should output the fields "track_id", "track_artist_name", "track_album_name",
"raw_artist", "raw_album", "source"
Where source is either Jellyfin or Mopidy based on the keys.
Put the file /tmp/metadata-report.csv by default and overwrite exsiting reports.
The command should also accept a file-path to overide this default.
** DONE [#A] Date parsing failing in eBird imports :birds:ebird:importers:
:PROPERTIES:
:ID: 72e0254f-39d8-4843-9857-623e0362d77e
:END:
*** Description
On line 45 in the apps/birds/importer.py file, the import is thorowing this
error:
#+begin_src python
ValueError: time data 'Jun 7, 2026, 5:15 PM' does not match format '%B %d, %Y %I:%M %p'
#+end_src
Historically other files starting on May 24 worked, so I suspect this is a
problem of the date formatter expecting Long month names and zero-padded days
and the only sample we had was a three-letter month (May) and days with two
digites(24 through the 31)
We should also add a "error_log" to the importers so that errors tha occur are
surfaced, even when 0 successful files were processed. And we should make sure
all importers do this as well.
* Version 48.0 [2/2]
** DONE [#B] Show team or player images on sport detail and scrobble detail :sports:templates:
:PROPERTIES:
:ID: 68c17383-ee6e-4b5f-b3f5-1b637a0a3ea8
:END:
*** Description
On the sport event detail page, we should show the images of the teams or
players invovled.
Also, those images for the sport event should be shown on the scrobble detail
page for sport event scrobble details.
** DONE [#B] Add fix_metadta method to Video instances :videos:metadata:
:PROPERTIES:
:ID: 9df5404d-1b60-4eee-b7cf-1f7e6dfade65
:END:
*** Description
Turns out we don't have a fix_metadata method for videos. We should add that using
the basic logic from find_or_create on the Video model.
* Version 47.2 [1/1] * Version 47.2 [1/1]
** DONE [#B] Add OMDB source as backup when TMDB returns nothing :videos:metadata:imdb: ** DONE [#B] Add OMDB source as backup when TMDB returns nothing :videos:metadata:imdb:
:PROPERTIES: :PROPERTIES:
@ -520,6 +632,7 @@ up unenriched.
:ID: 4d50ca2e-f45b-dde8-e3c9-cd84f353b349 :ID: 4d50ca2e-f45b-dde8-e3c9-cd84f353b349
:END: :END:
* Version 47.0 [1/1] * Version 47.0 [1/1]
** DONE [#B] Change sports scrobbling a bit :feature:sports:scrobbles: ** DONE [#B] Change sports scrobbling a bit :feature:sports:scrobbles:
:PROPERTIES: :PROPERTIES:
@ -935,6 +1048,7 @@ Format should be: "%Y-%m-%dT%H:%M:%S.%fZ"
:ID: 68a011b2-bb6f-3ba8-2312-5947c41db9ac :ID: 68a011b2-bb6f-3ba8-2312-5947c41db9ac
:END: :END:
* Version 39.0 [3/3] * Version 39.0 [3/3]
** DONE [#B] Clean up org-mode tasks metadata :bug:tasks:metadata: ** DONE [#B] Clean up org-mode tasks metadata :bug:tasks:metadata:
:PROPERTIES: :PROPERTIES:
@ -989,6 +1103,7 @@ If the task is completed, don't touch it.
:ID: 63dc633c-4382-e6a5-e663-b01871ce86ce :ID: 63dc633c-4382-e6a5-e663-b01871ce86ce
:END: :END:
* Version 38.0 [38/38] * Version 38.0 [38/38]
** DONE [#A] Fix release flow to be easier to trigger :pyproject:release:tooling: ** DONE [#A] Fix release flow to be easier to trigger :pyproject:release:tooling:
:PROPERTIES: :PROPERTIES:
@ -1380,6 +1495,7 @@ urllib.error.HTTPError: HTTP Error 500: Internal Server Error
This may have already been resolved ... need to just confirm it. This may have already been resolved ... need to just confirm it.
* Version 37.0 [4/4] * Version 37.0 [4/4]
** DONE [#A] Tasks from org-mode should properly update notes and leave them out of the body :vrobbler:bug:tasks: ** DONE [#A] Tasks from org-mode should properly update notes and leave them out of the body :vrobbler:bug:tasks:
:PROPERTIES: :PROPERTIES:
@ -1398,11 +1514,13 @@ https://life.lab.unbl.ink/scrobble/e39779c8-62a5-46a6-bdef-fb7662810dc6/start/
:PROPERTIES: :PROPERTIES:
:ID: e3e49a9a-67d2-8ad8-1114-6f05effee9b7 :ID: e3e49a9a-67d2-8ad8-1114-6f05effee9b7
:END: :END:
* Version 36.0 [1/1] * Version 36.0 [1/1]
** DONE [#A] Refactor how videos are scrobbled :vrobbler:vidoes:feature:personal:project: ** DONE [#A] Refactor how videos are scrobbled :vrobbler:vidoes:feature:personal:project:
:PROPERTIES: :PROPERTIES:
:ID: 6034a11d-5376-994d-9a4b-e1640e258cfa :ID: 6034a11d-5376-994d-9a4b-e1640e258cfa
:END: :END:
* Version 35.0 [3/3] * Version 35.0 [3/3]
** DONE [#B] Add youtube link in place of IMDB on video detail page :vrobbler:feature:videos:personal:project: ** DONE [#B] Add youtube link in place of IMDB on video detail page :vrobbler:feature:videos:personal:project:
:PROPERTIES: :PROPERTIES:
@ -1416,6 +1534,7 @@ https://life.lab.unbl.ink/scrobble/e39779c8-62a5-46a6-bdef-fb7662810dc6/start/
:PROPERTIES: :PROPERTIES:
:ID: d1ba1ca1-509b-13a9-1307-b2dc94a2eafe :ID: d1ba1ca1-509b-13a9-1307-b2dc94a2eafe
:END: :END:
* Version 34.0 [4/4] * Version 34.0 [4/4]
** DONE [#A] Use bgg-api for BoardGameGeek lookups :vrobbler:feature:boardgames:personal:project: ** DONE [#A] Use bgg-api for BoardGameGeek lookups :vrobbler:feature:boardgames:personal:project:
:PROPERTIES: :PROPERTIES:
@ -1475,6 +1594,7 @@ TypeError: can only concatenate str (not "NoneType") to str
A good lesson in using constants for things. A good lesson in using constants for things.
* Version 33.0 [3/3] * Version 33.0 [3/3]
** DONE [#A] Fix bug where scrobble is_stale only uses seconds not total_seconds :vrobbler:bug:scrobbles:personal:project: ** DONE [#A] Fix bug where scrobble is_stale only uses seconds not total_seconds :vrobbler:bug:scrobbles:personal:project:
:PROPERTIES: :PROPERTIES:
@ -1489,6 +1609,7 @@ TypeError: can only concatenate str (not "NoneType") to str
:ID: 4955cc34-0882-50db-92f7-f36a95bf57a4 :ID: 4955cc34-0882-50db-92f7-f36a95bf57a4
:END: :END:
<2025-10-28 Tue> <2025-10-28 Tue>
* Version 32.0 [2/2] * Version 32.0 [2/2]
** DONE [#B] Save path to reading source on book scrobbles and show it on the detail page :vrobbler:feature:books:personal:project: ** DONE [#B] Save path to reading source on book scrobbles and show it on the detail page :vrobbler:feature:books:personal:project:
:PROPERTIES: :PROPERTIES:
@ -1498,6 +1619,7 @@ TypeError: can only concatenate str (not "NoneType") to str
:PROPERTIES: :PROPERTIES:
:ID: 9fe09567-11a3-7083-53c7-07458a9591d0 :ID: 9fe09567-11a3-7083-53c7-07458a9591d0
:END: :END:
* Version 31.0 [3/3] * Version 31.0 [3/3]
** DONE [#A] Stop comic book webpage scrobbles from overwriting old scrobbles :vrobbler:personal:bug:books:scrobbling: ** DONE [#A] Stop comic book webpage scrobbles from overwriting old scrobbles :vrobbler:personal:bug:books:scrobbling:
:PROPERTIES: :PROPERTIES:
@ -1511,6 +1633,7 @@ TypeError: can only concatenate str (not "NoneType") to str
:PROPERTIES: :PROPERTIES:
:ID: 9a870c05-6d20-0803-d35d-c03fbe1d0ee1 :ID: 9a870c05-6d20-0803-d35d-c03fbe1d0ee1
:END: :END:
* Version 30.0 [3/3] * Version 30.0 [3/3]
** DONE [#A] Fix readcomicsonline browsing to update pages :vrobbler:books:feature:comicbook:personal:project:scrobbling: ** DONE [#A] Fix readcomicsonline browsing to update pages :vrobbler:books:feature:comicbook:personal:project:scrobbling:
:PROPERTIES: :PROPERTIES:
@ -1525,10 +1648,13 @@ TypeError: can only concatenate str (not "NoneType") to str
:PROPERTIES: :PROPERTIES:
:ID: d22cec3f-117f-f203-33a5-efbefa8a5cee :ID: d22cec3f-117f-f203-33a5-efbefa8a5cee
:END: :END:
* Version 29.0 [1/1] * Version 29.0 [1/1]
** DONE HOTFIX podcast lookups, final ** DONE HOTFIX podcast lookups, final
* Version 28.0 [1/1] * Version 28.0 [1/1]
** DONE HOTFIX podcast lookups ** DONE HOTFIX podcast lookups
* Version 27.0 [3/3] * Version 27.0 [3/3]
** DONE [#A] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal: ** DONE [#A] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal:
:PROPERTIES: :PROPERTIES:
@ -1569,6 +1695,7 @@ it's annoying.
actually be much more reliable than the current state of the podcast lookup actually be much more reliable than the current state of the podcast lookup
which depends on the file to be name properly. which depends on the file to be name properly.
* Version 26.0 [3/3] * Version 26.0 [3/3]
** DONE Clean up templates for scrobble details :vrobbler:personal:bug:templates: ** DONE Clean up templates for scrobble details :vrobbler:personal:bug:templates:
:PROPERTIES: :PROPERTIES:
@ -1582,6 +1709,7 @@ it's annoying.
:PROPERTIES: :PROPERTIES:
:ID: c03a38ce-b337-f4fa-adba-aee08d4329f5 :ID: c03a38ce-b337-f4fa-adba-aee08d4329f5
:END: :END:
* Version 25.0 [3/3] * Version 25.0 [3/3]
** DONE Add basic food templates and fix urls :food:vrobbler:personal:project:bug:urls: ** DONE Add basic food templates and fix urls :food:vrobbler:personal:project:bug:urls:
:PROPERTIES: :PROPERTIES:
@ -1595,6 +1723,7 @@ it's annoying.
:PROPERTIES: :PROPERTIES:
:ID: 7debfbaf-cdd8-f49b-57ff-804bfe7c9236 :ID: 7debfbaf-cdd8-f49b-57ff-804bfe7c9236
:END: :END:
* Version 24.0 [2/2] * Version 24.0 [2/2]
** DONE Clean up logdata for various media :personal:feature:project:vrobbler:logdata: ** DONE Clean up logdata for various media :personal:feature:project:vrobbler:logdata:
:PROPERTIES: :PROPERTIES:
@ -1605,6 +1734,7 @@ it's annoying.
:ID: 1a1c0aa6-0313-c8be-1676-5d6adddef0a4 :ID: 1a1c0aa6-0313-c8be-1676-5d6adddef0a4
:END: :END:
* Version 23.0 [3/3] * Version 23.0 [3/3]
** DONE Add dynamic forms for LogData classes :personal:feature:vrobbler:project:forms:logdata: ** DONE Add dynamic forms for LogData classes :personal:feature:vrobbler:project:forms:logdata:
:PROPERTIES: :PROPERTIES:
@ -1615,16 +1745,19 @@ it's annoying.
:PROPERTIES: :PROPERTIES:
:ID: 99f6bd77-dc8f-6ed1-0321-32a52c944264 :ID: 99f6bd77-dc8f-6ed1-0321-32a52c944264
:END: :END:
* Version 19.0 [1/1] * Version 19.0 [1/1]
** DONE Add periodic check for mood :vrobbler:feature:moods:personal:project: ** DONE Add periodic check for mood :vrobbler:feature:moods:personal:project:
:PROPERTIES: :PROPERTIES:
:ID: 55404488-c69f-0dd5-838e-1d1e15c873eb :ID: 55404488-c69f-0dd5-838e-1d1e15c873eb
:END: :END:
* Version 18.7 [1/1] * Version 18.7 [1/1]
** DONE Use the timezone history log to fix old Scrobbles that fall into those timezone blocks :vrobbler:chore:scrobbles:project:personal: ** DONE Use the timezone history log to fix old Scrobbles that fall into those timezone blocks :vrobbler:chore:scrobbles:project:personal:
:PROPERTIES: :PROPERTIES:
:ID: 9d055ac1-584b-20c8-7ad9-9ce36b329dc7 :ID: 9d055ac1-584b-20c8-7ad9-9ce36b329dc7
:END: :END:
* Version 18.4 [2/2] * Version 18.4 [2/2]
** DONE Track timezone changes for profiles :vrobbler:feature:profiles:personal:project: ** DONE Track timezone changes for profiles :vrobbler:feature:profiles:personal:project:
:PROPERTIES: :PROPERTIES:
@ -1638,11 +1771,13 @@ it's annoying.
- Note taken on [2025-07-20 Sun 16:21] - Note taken on [2025-07-20 Sun 16:21]
This thing is kicking my butt. As it stands it works, but the scrobbles are not assigned to the tracks properly. This thing is kicking my butt. As it stands it works, but the scrobbles are not assigned to the tracks properly.
* Version 18.3 [1/1] * Version 18.3 [1/1]
** DONE Add timezone awarness to IMAP importer :personal:project:vrobbler:feature:importer:imap:timezones: ** DONE Add timezone awarness to IMAP importer :personal:project:vrobbler:feature:importer:imap:timezones:
:PROPERTIES: :PROPERTIES:
:ID: 05837b7c-96aa-6190-3678-e2ae7c7cac75 :ID: 05837b7c-96aa-6190-3678-e2ae7c7cac75
:END: :END:
* Version 18 [4/4] * Version 18 [4/4]
** DONE Condense tracks of the same title by the same artist with multiple albums :vrobbler:feature:music:project:personal: ** DONE Condense tracks of the same title by the same artist with multiple albums :vrobbler:feature:music:project:personal:
:PROPERTIES: :PROPERTIES:
@ -1764,6 +1899,7 @@ it's annoying.
:PROPERTIES: :PROPERTIES:
:ID: 23f485e3-988c-6198-c79d-91fdf92f001c :ID: 23f485e3-988c-6198-c79d-91fdf92f001c
:END: :END:
* Version 17.0 [6/6] * Version 17.0 [6/6]
** DONE [#A] Fix bug in new task label lookup for Emacs/Org-mode :vrobbler:bug:tasks: ** DONE [#A] Fix bug in new task label lookup for Emacs/Org-mode :vrobbler:bug:tasks:
:PROPERTIES: :PROPERTIES:
@ -1827,6 +1963,7 @@ Not sure if the problem is in my Emacs hook sending or Vrobbler itself.
:PROPERTIES: :PROPERTIES:
:ID: df58f8d0-fa4a-2037-c7d7-e5388c239042 :ID: df58f8d0-fa4a-2037-c7d7-e5388c239042
:END: :END:
* Version 0.16.0 [19/19] * Version 0.16.0 [19/19]
** DONE [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music: ** DONE [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music:
:PROPERTIES: :PROPERTIES:
@ -1932,6 +2069,7 @@ out using that.
** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug: ** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug:
This was fixed a while ago, but there's a new manifested bug. Going to create a This was fixed a while ago, but there's a new manifested bug. Going to create a
separate bug tracking ticket for that. separate bug tracking ticket for that.
* Version 0.11.4 [9/9] * Version 0.11.4 [9/9]
** DONE Add rudimentary video game scrobbling :improvement:content:videogames: ** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
CLOSED: [2023-03-07 Tue 11:11] CLOSED: [2023-03-07 Tue 11:11]

View File

@ -1,6 +1,6 @@
[tool.poetry] [tool.poetry]
name = "vrobbler" name = "vrobbler"
version = "47.2" version = "48.1"
description = "" description = ""
authors = ["Colin Powell <colin@unbl.ink>"] authors = ["Colin Powell <colin@unbl.ink>"]

View File

@ -1,3 +1,4 @@
import tempfile
from datetime import timedelta from datetime import timedelta
import pytest import pytest
@ -128,6 +129,23 @@ class TestBirdingCSVImportModel:
assert imp.import_type == "Birding CSV" assert imp.import_type == "Birding CSV"
assert "Birding" in str(imp) assert "Birding" in str(imp)
def test_record_error(self, db, user):
imp = BirdingCSVImport.objects.create(user=user)
assert imp.error_log is None
imp.record_error("test error")
imp.refresh_from_db()
assert imp.error_log is not None
assert "test error" in imp.error_log
def test_record_error_appends(self, db, user):
imp = BirdingCSVImport.objects.create(user=user)
imp.record_error("first error")
imp.record_error("second error")
imp.refresh_from_db()
assert imp.error_log.count("\n") == 1
assert "first error" in imp.error_log
assert "second error" in imp.error_log
@pytest.mark.django_db(transaction=True) @pytest.mark.django_db(transaction=True)
def test_process_via_model(self, user, birding_csv_file): def test_process_via_model(self, user, birding_csv_file):
imp = BirdingCSVImport.objects.create(user=user) imp = BirdingCSVImport.objects.create(user=user)
@ -137,3 +155,35 @@ class TestBirdingCSVImportModel:
imp.refresh_from_db() imp.refresh_from_db()
assert imp.process_count == 1 assert imp.process_count == 1
assert imp.processed_finished is not None assert imp.processed_finished is not None
def test_record_error_on_bad_csv(self, user, db):
content = """Species,Count,Location,Observation Type,Observation Date,Start Time,Duration,Distance,Area,Party Size,Complete Checklist,# of species,Details
Canada Goose,6,Test Park,Stationary,"Bad Date",4:15 PM,9 minute(s),,,4,true,4 species,
"""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8"
) as f:
f.write(content)
file_path = f.name
errors = []
scrobbles = import_birding_csv(file_path, user.id, record_error=errors.append)
assert len(scrobbles) == 0
assert len(errors) == 1
assert "Could not parse date/time" in errors[0]
def test_record_error_on_bad_location(self, user, db):
content = """Species,Count,Location,Observation Type,Observation Date,Start Time,Duration,Distance,Area,Party Size,Complete Checklist,# of species,Details
Canada Goose,6,,Stationary,"May 10, 2026",4:15 PM,9 minute(s),,,4,true,4 species,
"""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8"
) as f:
f.write(content)
file_path = f.name
errors = []
scrobbles = import_birding_csv(file_path, user.id, record_error=errors.append)
assert len(scrobbles) == 0
assert len(errors) == 1
assert "Skipping rows with no location" in errors[0]

View File

@ -26,5 +26,5 @@ class BirdingLocationAdmin(admin.ModelAdmin):
@admin.register(BirdingCSVImport) @admin.register(BirdingCSVImport)
class BirdingCSVImportAdmin(admin.ModelAdmin): class BirdingCSVImportAdmin(admin.ModelAdmin):
date_hierarchy = "created" date_hierarchy = "created"
list_display = ("uuid", "process_count", "processed_finished", "processing_started") list_display = ("uuid", "process_count", "processed_finished", "processing_started", "error_log")
ordering = ("-created",) ordering = ("-created",)

View File

@ -2,7 +2,9 @@ import csv
import logging import logging
import re import re
from collections import defaultdict from collections import defaultdict
from datetime import datetime, timedelta from datetime import timedelta
from dateutil import parser
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
@ -35,11 +37,12 @@ def parse_coords(location_str):
def parse_timestamp(date_str, time_str): def parse_timestamp(date_str, time_str):
try: try:
dt = datetime.strptime(f"{date_str} {time_str}", "%B %d, %Y %I:%M %p") dt_str = f"{date_str} {time_str}".strip()
dt = parser.parse(dt_str)
return dt return dt
except (ValueError, TypeError): except (ValueError, TypeError):
try: try:
dt = datetime.strptime(date_str, "%B %d, %Y") dt = parser.parse(date_str)
return dt return dt
except (ValueError, TypeError): except (ValueError, TypeError):
logger.warning(f"Could not parse date/time: {date_str} {time_str}") logger.warning(f"Could not parse date/time: {date_str} {time_str}")
@ -61,7 +64,7 @@ def parse_int(value):
return None return None
def import_birding_csv(file_path, user_id): def import_birding_csv(file_path, user_id, record_error=None):
user = User.objects.get(id=user_id) user = User.objects.get(id=user_id)
new_scrobbles = [] new_scrobbles = []
@ -80,11 +83,17 @@ def import_birding_csv(file_path, user_id):
for (location_str, date_str, time_str), sighting_rows in groups.items(): for (location_str, date_str, time_str), sighting_rows in groups.items():
if not location_str: if not location_str:
logger.warning("Skipping rows with no location") msg = "Skipping rows with no location"
logger.warning(msg)
if record_error:
record_error(msg)
continue continue
timestamp = parse_timestamp(date_str, time_str) timestamp = parse_timestamp(date_str, time_str)
if not timestamp: if not timestamp:
msg = f"Could not parse date/time: {date_str} {time_str}"
if record_error:
record_error(msg)
continue continue
timestamp = user.profile.get_timestamp_with_tz(timestamp) timestamp = user.profile.get_timestamp_with_tz(timestamp)

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.29 on 2026-06-08 14:57
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("birds", "0002_birdingcsvimport"),
]
operations = [
migrations.AddField(
model_name="birdingcsvimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
]

View File

@ -61,9 +61,7 @@ class BirdSightingLogData(BaseLogData, WithPeopleLogData):
@cached_property @cached_property
def bird_list(self) -> str: def bird_list(self) -> str:
if self.birds: if self.birds:
return ", ".join( return ", ".join([BirdSightingEntry(**b).__str__() for b in self.birds])
[BirdSightingEntry(**b).__str__() for b in self.birds]
)
return "" return ""
def as_html(self) -> str: def as_html(self) -> str:
@ -80,9 +78,7 @@ class BirdSightingLogData(BaseLogData, WithPeopleLogData):
) )
if self.area: if self.area:
html_parts.append( html_parts.append(f'<div class="birding-area">Area: {self.area}</div>')
f'<div class="birding-area">Area: {self.area}</div>'
)
if self.party_size: if self.party_size:
html_parts.append( html_parts.append(
@ -105,9 +101,7 @@ class BirdSightingLogData(BaseLogData, WithPeopleLogData):
) )
if self.guide: if self.guide:
html_parts.append( html_parts.append(f'<div class="birding-guide">Guide: {self.guide}</div>')
f'<div class="birding-guide">Guide: {self.guide}</div>'
)
if self.duration_minutes: if self.duration_minutes:
html_parts.append( html_parts.append(
@ -183,9 +177,7 @@ class Bird(TimeStampedModel):
class BirdingLocation(ScrobblableMixin): class BirdingLocation(ScrobblableMixin):
description = models.TextField(**BNULL) description = models.TextField(**BNULL)
geo_location = models.ForeignKey( geo_location = models.ForeignKey(GeoLocation, **BNULL, on_delete=models.DO_NOTHING)
GeoLocation, **BNULL, on_delete=models.DO_NOTHING
)
ebird_hotspot_id = models.CharField(max_length=255, **BNULL) ebird_hotspot_id = models.CharField(max_length=255, **BNULL)
def get_absolute_url(self): def get_absolute_url(self):
@ -193,7 +185,7 @@ class BirdingLocation(ScrobblableMixin):
@property @property
def subtitle(self): def subtitle(self):
return "" return self.geo_location
@property @property
def strings(self) -> ScrobblableConstants: def strings(self) -> ScrobblableConstants:
@ -224,6 +216,7 @@ class BirdingCSVImport(TimeStampedModel):
processed_finished = models.DateTimeField(**BNULL) processed_finished = models.DateTimeField(**BNULL)
process_log = models.TextField(**BNULL) process_log = models.TextField(**BNULL)
process_count = models.IntegerField(**BNULL) process_count = models.IntegerField(**BNULL)
error_log = models.TextField(**BNULL)
csv_file = models.FileField(upload_to="birding-csv-uploads/", **BNULL) csv_file = models.FileField(upload_to="birding-csv-uploads/", **BNULL)
class Meta: class Meta:
@ -269,9 +262,7 @@ class BirdingCSVImport(TimeStampedModel):
self.save(update_fields=["process_log", "process_count"]) self.save(update_fields=["process_log", "process_count"])
return return
for count, scrobble in enumerate(scrobbles): for count, scrobble in enumerate(scrobbles):
scrobble_str = ( scrobble_str = f"{scrobble.id}\t{scrobble.timestamp}\t{scrobble.media_obj}"
f"{scrobble.id}\t{scrobble.timestamp}\t{scrobble.media_obj}"
)
log_line = f"{scrobble_str}" log_line = f"{scrobble_str}"
if count > 0: if count > 0:
log_line = "\n" + log_line log_line = "\n" + log_line
@ -279,6 +270,14 @@ class BirdingCSVImport(TimeStampedModel):
self.process_count = len(scrobbles) self.process_count = len(scrobbles)
self.save(update_fields=["process_log", "process_count"]) self.save(update_fields=["process_log", "process_count"])
def record_error(self, error_message):
log_line = f"{timezone.now().isoformat()}: {error_message}"
if self.error_log:
self.error_log += "\n" + log_line
else:
self.error_log = log_line
self.save(update_fields=["error_log"])
def scrobbles(self): def scrobbles(self):
from scrobbles.models import Scrobble from scrobbles.models import Scrobble
@ -297,6 +296,13 @@ class BirdingCSVImport(TimeStampedModel):
from birds.importer import import_birding_csv from birds.importer import import_birding_csv
self.mark_started() self.mark_started()
scrobbles = import_birding_csv(self.upload_file_path, self.user_id) try:
self.record_log(scrobbles) scrobbles = import_birding_csv(
self.mark_finished() self.upload_file_path, self.user_id, record_error=self.record_error
)
self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()

View File

@ -266,8 +266,9 @@ class BoardGame(ScrobblableMixin):
"self", **BNULL, on_delete=models.DO_NOTHING "self", **BNULL, on_delete=models.DO_NOTHING
) )
def __str__(self): @property
return self.title def subtitle(self) -> str:
return self.publisher
def get_absolute_url(self): def get_absolute_url(self):
return reverse("boardgames:boardgame_detail", kwargs={"slug": self.uuid}) return reverse("boardgames:boardgame_detail", kwargs={"slug": self.uuid})

View File

@ -173,11 +173,7 @@ class Book(LongPlayScrobblableMixin):
genre = TaggableManager(through=ObjectWithGenres, blank=True, verbose_name="Genre") genre = TaggableManager(through=ObjectWithGenres, blank=True, verbose_name="Genre")
def __str__(self) -> str: def __str__(self) -> str:
if self.issue_number and "Issue" not in str(self.title): return f"{self.title} - {self.subtitle}"
return f"{self.title} - Issue {self.issue_number}"
if self.volume_number and "Volume" not in str(self.title):
return f"{self.title} - Volume {self.volume_number}"
return f"{self.title}"
def save(self, *args, **kwargs): def save(self, *args, **kwargs):
if self.pages: if self.pages:
@ -188,7 +184,12 @@ class Book(LongPlayScrobblableMixin):
@property @property
def subtitle(self): def subtitle(self):
return f" by {self.author}" subtitle = self.author
if self.issue_number and "Issue" not in str(self.title):
subtitle += " - Issue {self.issue_number}"
if self.volume_number and "Volume" not in str(self.title):
subtitle += " - Volume {self.volume_number}"
return subtitle
@property @property
def strings(self) -> ScrobblableConstants: def strings(self) -> ScrobblableConstants:

View File

@ -0,0 +1,127 @@
import csv
import logging
from django.core.management.base import BaseCommand
logger = logging.getLogger(__name__)
def _get_source(raw_data):
if "Artist" in raw_data:
return "Jellyfin"
if "artist" in raw_data:
return "Mopidy"
return None
def _get_raw_values(raw_data, source):
if source == "Jellyfin":
return raw_data.get("Artist", ""), raw_data.get("Album", "")
return raw_data.get("artist", ""), raw_data.get("album", "")
def _normalize(name):
return name.strip().casefold()
def _artist_mismatch(raw_artist, track_artist_names):
if not raw_artist or not track_artist_names:
return False
track_names = [_normalize(n) for n in track_artist_names.split(" / ")]
raw = _normalize(raw_artist)
if raw in track_names:
return False
if raw == _normalize(track_artist_names):
return False
return True
def _album_mismatch(raw_album, track_album_name):
if not raw_album or not track_album_name:
return False
return _normalize(raw_album) != _normalize(track_album_name)
class Command(BaseCommand):
help = (
"Outputs a CSV of track IDs where raw metadata from scrobble logs "
"does not match the track's stored artists or album"
)
def add_arguments(self, parser):
parser.add_argument(
"--file-path",
type=str,
default="/tmp/metadata-report.csv",
help="Output CSV file path (default: /tmp/metadata-report.csv)",
)
def handle(self, *args, **options):
from scrobbles.models import Scrobble
file_path = options["file_path"]
qs = (
Scrobble.objects.filter(media_type=Scrobble.MediaType.TRACK)
.exclude(log__isnull=True)
.exclude(log={})
.select_related("track__album")
.prefetch_related("track__artists")
.iterator()
)
rows = []
for scrobble in qs:
track = scrobble.track
if not track:
continue
raw_data = scrobble.log.get("raw_data")
if not raw_data:
continue
source = _get_source(raw_data)
if not source:
continue
raw_artist, raw_album = _get_raw_values(raw_data, source)
if not raw_artist and not raw_album:
continue
track_artist_names = " / ".join(
track.artists.all().values_list("name", flat=True)
)
track_album_name = track.album.name if track.album else ""
if _artist_mismatch(raw_artist, track_artist_names) or _album_mismatch(
raw_album, track_album_name
):
rows.append(
{
"track_id": track.id,
"track_artist_name": track_artist_names,
"track_album_name": track_album_name,
"raw_artist": raw_artist,
"raw_album": raw_album,
"source": source,
}
)
fieldnames = [
"track_id",
"track_artist_name",
"track_album_name",
"raw_artist",
"raw_album",
"source",
]
with open(file_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(rows)
self.stdout.write(
self.style.SUCCESS(
f"Wrote {len(rows)} mismatched track(s) to {file_path}"
)
)

View File

@ -20,6 +20,7 @@ class ScrobbleInline(admin.TabularInline):
extra = 0 extra = 0
raw_id_fields = ( raw_id_fields = (
"video", "video",
"channel",
"podcast_episode", "podcast_episode",
"track", "track",
"video_game", "video_game",
@ -30,6 +31,7 @@ class ScrobbleInline(admin.TabularInline):
"board_game", "board_game",
"geo_location", "geo_location",
"task", "task",
"puzzle",
"mood", "mood",
"brick_set", "brick_set",
"trail", "trail",
@ -54,52 +56,44 @@ class ImportBaseAdmin(admin.ModelAdmin):
"process_count", "process_count",
"processed_finished", "processed_finished",
"processing_started", "processing_started",
"error_log",
) )
ordering = ("-created",) ordering = ("-created",)
@admin.register(AudioScrobblerTSVImport) @admin.register(AudioScrobblerTSVImport)
class AudioScrobblerTSVImportAdmin(ImportBaseAdmin): class AudioScrobblerTSVImportAdmin(ImportBaseAdmin): ...
...
@admin.register(LastFmImport) @admin.register(LastFmImport)
class LastFmImportAdmin(ImportBaseAdmin): class LastFmImportAdmin(ImportBaseAdmin): ...
...
@admin.register(KoReaderImport) @admin.register(KoReaderImport)
class KoReaderImportAdmin(ImportBaseAdmin): class KoReaderImportAdmin(ImportBaseAdmin): ...
...
@admin.register(RetroarchImport) @admin.register(RetroarchImport)
class RetroarchImportAdmin(ImportBaseAdmin): class RetroarchImportAdmin(ImportBaseAdmin): ...
...
class RetroarchImportAdmin(ImportBaseAdmin): class RetroarchImportAdmin(ImportBaseAdmin): ...
...
@admin.register(BGStatsImport) @admin.register(BGStatsImport)
class BGStatsImportAdmin(ImportBaseAdmin): class BGStatsImportAdmin(ImportBaseAdmin): ...
...
@admin.register(EBirdCSVImport) @admin.register(EBirdCSVImport)
class EBirdCSVImportAdmin(ImportBaseAdmin): class EBirdCSVImportAdmin(ImportBaseAdmin): ...
...
@admin.register(ScaleCSVImport) @admin.register(ScaleCSVImport)
class ScaleCSVImportAdmin(ImportBaseAdmin): class ScaleCSVImportAdmin(ImportBaseAdmin): ...
...
@admin.register(TrailGPXImport) @admin.register(TrailGPXImport)
class TrailGPXImportAdmin(ImportBaseAdmin): class TrailGPXImportAdmin(ImportBaseAdmin): ...
...
@admin.register(Genre) @admin.register(Genre)

View File

@ -0,0 +1,53 @@
# Generated by Django 4.2.29 on 2026-06-08 14:57
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("scrobbles", "0089_favoritemedia"),
]
operations = [
migrations.AddField(
model_name="audioscrobblertsvimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="bgstatsimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="ebirdcsvimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="koreaderimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="lastfmimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="retroarchimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="scalecsvimport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
migrations.AddField(
model_name="trailgpximport",
name="error_log",
field=models.TextField(blank=True, null=True),
),
]

View File

@ -65,6 +65,9 @@ class ScrobblableMixin(TimeStampedModel):
class Meta: class Meta:
abstract = True abstract = True
def __str__(self) -> str:
return f"{self.title} - {self.subtitle}"
@property @property
def run_time_seconds(self) -> int: def run_time_seconds(self) -> int:
run_time = 900 run_time = 900

View File

@ -74,6 +74,7 @@ class BaseFileImportMixin(TimeStampedModel):
processed_finished = models.DateTimeField(**BNULL) processed_finished = models.DateTimeField(**BNULL)
process_log = models.TextField(**BNULL) process_log = models.TextField(**BNULL)
process_count = models.IntegerField(**BNULL) process_count = models.IntegerField(**BNULL)
error_log = models.TextField(**BNULL)
class Meta: class Meta:
abstract = True abstract = True
@ -158,6 +159,14 @@ class BaseFileImportMixin(TimeStampedModel):
self.process_count = len(scrobbles) self.process_count = len(scrobbles)
self.save(update_fields=["process_log", "process_count"]) self.save(update_fields=["process_log", "process_count"])
def record_error(self, error_message):
log_line = f"{timezone.now().isoformat()}: {error_message}"
if self.error_log:
self.error_log += "\n" + log_line
else:
self.error_log = log_line
self.save(update_fields=["error_log"])
@property @property
def upload_file_path(self): def upload_file_path(self):
raise NotImplementedError raise NotImplementedError
@ -213,9 +222,14 @@ class KoReaderImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
scrobbles = process_koreader_sqlite_file(self.upload_file_path, self.user.id) try:
self.record_log(scrobbles) scrobbles = process_koreader_sqlite_file(self.upload_file_path, self.user.id)
self.mark_finished() self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class AudioScrobblerTSVImport(BaseFileImportMixin): class AudioScrobblerTSVImport(BaseFileImportMixin):
@ -255,10 +269,14 @@ class AudioScrobblerTSVImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
try:
scrobbles = import_audioscrobbler_tsv_file(self.upload_file_path, self.user.id) scrobbles = import_audioscrobbler_tsv_file(self.upload_file_path, self.user.id)
self.record_log(scrobbles) self.record_log(scrobbles)
self.mark_finished() except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class ScaleCSVImport(BaseFileImportMixin): class ScaleCSVImport(BaseFileImportMixin):
@ -297,9 +315,14 @@ class ScaleCSVImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
scrobbles = import_scale_csv(self.upload_file_path, self.user.id) try:
self.record_log(scrobbles) scrobbles = import_scale_csv(self.upload_file_path, self.user.id)
self.mark_finished() self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class TrailGPXImport(BaseFileImportMixin): class TrailGPXImport(BaseFileImportMixin):
@ -337,11 +360,16 @@ class TrailGPXImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
scrobbles = import_trail_gpx( try:
self.upload_file_path, self.user.id, self.original_filename scrobbles = import_trail_gpx(
) self.upload_file_path, self.user.id, self.original_filename
self.record_log(scrobbles) )
self.mark_finished() self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class LastFmImport(BaseFileImportMixin): class LastFmImport(BaseFileImportMixin):
@ -391,11 +419,14 @@ class LastFmImport(BaseFileImportMixin):
last_processed = last_import.processed_finished last_processed = last_import.processed_finished
self.mark_started() self.mark_started()
try:
scrobbles = lastfm.import_from_lastfm(last_processed, time_to=time_to) scrobbles = lastfm.import_from_lastfm(last_processed, time_to=time_to)
self.record_log(scrobbles)
self.record_log(scrobbles) except Exception as e:
self.mark_finished() self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class RetroarchImport(BaseFileImportMixin): class RetroarchImport(BaseFileImportMixin):
@ -426,43 +457,48 @@ class RetroarchImport(BaseFileImportMixin):
logger.info(f"You told me to force import from Retroarch") logger.info(f"You told me to force import from Retroarch")
self.mark_started() self.mark_started()
scrobbles = None
try:
if self.lrtl_file:
import os
import tempfile
import zipfile
if self.lrtl_file: tmpdir = tempfile.mkdtemp()
import os try:
import tempfile zip_path = os.path.join(tmpdir, "archive.zip")
import zipfile with open(zip_path, "wb") as f:
f.write(self.lrtl_file.read())
with zipfile.ZipFile(zip_path, "r") as zf:
zf.extractall(tmpdir)
os.unlink(zip_path)
scrobbles = retroarch.import_retroarch_lrtl_files(
tmpdir + "/",
self.user.id,
)
finally:
import shutil
shutil.rmtree(tmpdir, ignore_errors=True)
else:
if not self.user.profile.retroarch_path:
logger.info(
"Trying to import Retroarch logs, but user has no retroarch_path configured"
)
self.mark_finished()
return
tmpdir = tempfile.mkdtemp()
try:
zip_path = os.path.join(tmpdir, "archive.zip")
with open(zip_path, "wb") as f:
f.write(self.lrtl_file.read())
with zipfile.ZipFile(zip_path, "r") as zf:
zf.extractall(tmpdir)
os.unlink(zip_path)
scrobbles = retroarch.import_retroarch_lrtl_files( scrobbles = retroarch.import_retroarch_lrtl_files(
tmpdir + "/", self.user.profile.retroarch_path,
self.user.id, self.user.id,
) )
finally:
import shutil
shutil.rmtree(tmpdir, ignore_errors=True) self.record_log(scrobbles)
else: except Exception as e:
if not self.user.profile.retroarch_path: self.record_error(f"Import failed: {e}")
logger.info( logger.exception(f"Import failed for {self}")
"Tying to import Retroarch logs, but user has no retroarch_path configured" finally:
) self.mark_finished()
self.mark_finished()
return
scrobbles = retroarch.import_retroarch_lrtl_files(
self.user.profile.retroarch_path,
self.user.id,
)
self.record_log(scrobbles)
self.mark_finished()
class BGStatsImport(BaseFileImportMixin): class BGStatsImport(BaseFileImportMixin):
@ -499,17 +535,21 @@ class BGStatsImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
try:
import json
import json from scrobbles.scrobblers import email_scrobble_board_game
from scrobbles.scrobblers import email_scrobble_board_game with open(self.upload_file_path, "r", encoding="utf-8") as f:
parsed_json = json.load(f)
scrobbles = email_scrobble_board_game(parsed_json, self.user_id)
with open(self.upload_file_path, "r", encoding="utf-8") as f: self.record_log(scrobbles)
parsed_json = json.load(f) except Exception as e:
scrobbles = email_scrobble_board_game(parsed_json, self.user_id) self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
self.record_log(scrobbles) finally:
self.mark_finished() self.mark_finished()
class EBirdCSVImport(BaseFileImportMixin): class EBirdCSVImport(BaseFileImportMixin):
@ -554,9 +594,16 @@ class EBirdCSVImport(BaseFileImportMixin):
return return
self.mark_started() self.mark_started()
scrobbles = import_birding_csv(self.upload_file_path, self.user_id) try:
self.record_log(scrobbles) scrobbles = import_birding_csv(
self.mark_finished() self.upload_file_path, self.user_id, record_error=self.record_error
)
self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
class ScrobbleQuerySet(models.QuerySet): class ScrobbleQuerySet(models.QuerySet):

View File

@ -184,7 +184,7 @@ class ScrobbleableDetailView(ChartContextMixin, DetailView):
def get_context_data(self, **kwargs): def get_context_data(self, **kwargs):
context_data = super().get_context_data(**kwargs) context_data = super().get_context_data(**kwargs)
scrobbles = [] scrobbles = []
if not self.request.user.is_anonymous: if not self.request.user.is_anonymous and hasattr(self.object, "scrobble_set"):
scrobbles = self.object.scrobble_set.filter( scrobbles = self.object.scrobble_set.filter(
user=self.request.user user=self.request.user
).order_by("-timestamp") ).order_by("-timestamp")
@ -439,8 +439,7 @@ class ScrobbleListView(LoginRequiredMixin, ListView):
full_qs = getattr(self, "_full_queryset", None) full_qs = getattr(self, "_full_queryset", None)
if full_qs is not None and getattr(self, "tag_list", []): if full_qs is not None and getattr(self, "tag_list", []):
total = ( total = (
full_qs.aggregate(total=Sum("playback_position_seconds"))["total"] full_qs.aggregate(total=Sum("playback_position_seconds"))["total"] or 0
or 0
) )
ctx["total_time_seconds"] = total ctx["total_time_seconds"] = total
return ctx return ctx
@ -1245,8 +1244,7 @@ class ScrobbleDetailView(DetailView):
track=media_obj, user=self.object.user track=media_obj, user=self.object.user
).order_by("-timestamp")[:20] ).order_by("-timestamp")[:20]
context["has_mopidy_uri"] = any( context["has_mopidy_uri"] = any(
(s.log or {}).get("raw_data", {}).get("mopidy_uri") (s.log or {}).get("raw_data", {}).get("mopidy_uri") for s in scrobbles
for s in scrobbles
) )
else: else:
context["has_mopidy_uri"] = False context["has_mopidy_uri"] = False

View File

@ -123,17 +123,17 @@ class SportEvent(ScrobblableMixin):
super(SportEvent, self).save(*args, **kwargs) super(SportEvent, self).save(*args, **kwargs)
def __str__(self) -> str: def __str__(self) -> str:
league = self.league return f"{self.title} - {self.subtitle}"
if self.league and self.league.abbreviation_str:
league = self.league.abbreviation_str
return f"{self.title} - {league}"
def get_absolute_url(self): def get_absolute_url(self):
return reverse("sports:event_detail", kwargs={"slug": self.uuid}) return reverse("sports:event_detail", kwargs={"slug": self.uuid})
@property @property
def subtitle(self) -> str: def subtitle(self) -> str:
return self.comp_str league = self.league
if self.league and self.league.abbreviation_str:
league = self.league.abbreviation_str
return f"{league} {self.get_event_type_display()}"
@property @property
def comp_str(self) -> str: def comp_str(self) -> str:

View File

@ -163,12 +163,9 @@ class VideoGame(LongPlayScrobblableMixin):
platforms = models.ManyToManyField(VideoGamePlatform) platforms = models.ManyToManyField(VideoGamePlatform)
retroarch_name = models.CharField(max_length=255, **BNULL) retroarch_name = models.CharField(max_length=255, **BNULL)
def __str__(self):
return self.title
@property @property
def subtitle(self): def subtitle(self):
return f" On {self.platforms.first()}" return f"{self.platforms.first()}"
@property @property
def strings(self) -> ScrobblableConstants: def strings(self) -> ScrobblableConstants:

View File

@ -225,6 +225,8 @@ class Series(TimeStampedModel):
def is_episode_playing(self, user_id: int) -> bool: def is_episode_playing(self, user_id: int) -> bool:
last_scrobble = self.scrobbles_for_user(user_id, include_playing=True).first() last_scrobble = self.scrobbles_for_user(user_id, include_playing=True).first()
if not last_scrobble:
return False
return not last_scrobble.played_to_completion return not last_scrobble.played_to_completion
def fix_metadata(self, force_update=False): def fix_metadata(self, force_update=False):
@ -266,9 +268,7 @@ class Series(TimeStampedModel):
metadata = lookup_video_from_omdb(imdb_id) metadata = lookup_video_from_omdb(imdb_id)
if not metadata or not metadata.title: if not metadata or not metadata.title:
logger.warning( logger.warning(f"No metadata found for series {imdb_id} from TMDB or OMDB")
f"No metadata found for series {imdb_id} from TMDB or OMDB"
)
return series return series
vdict, _, cover, genres = metadata.as_dict_with_cover_and_genres() vdict, _, cover, genres = metadata.as_dict_with_cover_and_genres()
@ -420,6 +420,84 @@ class Video(ScrobblableMixin):
fname = f"{self.title}_{self.uuid}.jpg" fname = f"{self.title}_{self.uuid}.jpg"
self.cover_image.save(fname, ContentFile(r.content), save=True) self.cover_image.save(fname, ContentFile(r.content), save=True)
def fix_metadata(self, force_update: bool = False) -> None:
if self.video_type == self.VideoType.YOUTUBE and self.youtube_id:
vdict, _, cover, genres = lookup_video_from_youtube(
self.youtube_id
).as_dict_with_cover_and_genres()
for k, v in vdict.items():
setattr(self, k, v)
self.save()
if cover:
self.save_image_from_url(cover)
if genres:
self.genre.add(*genres)
return
if self.video_type == self.VideoType.TWITCH and self.twitch_id:
from videos.sources.twitch import lookup_video_from_twitch
metadata = lookup_video_from_twitch(self.twitch_id)
self.title = metadata.title or f"Twitch VOD {self.twitch_id}"
self.overview = metadata.overview
self.base_run_time_seconds = metadata.base_run_time_seconds
if metadata.upload_date:
self.upload_date = metadata.upload_date
if metadata.year:
self.year = metadata.year
self.video_type = Video.VideoType.TWITCH
if metadata.channel_id:
from videos.models import Channel
self.channel = Channel.objects.filter(
id=metadata.channel_id
).first()
self.save()
if metadata.cover_url:
self.save_image_from_url(metadata.cover_url)
return
if self.imdb_id:
try:
metadata = lookup_video_from_tmdb(self.imdb_id)
except Exception as e:
logger.warning(f"TMDB lookup failed for {self.imdb_id}: {e}")
metadata = None
if not metadata or not metadata.title:
metadata = lookup_video_from_omdb(self.imdb_id)
if not metadata or not metadata.title:
logger.warning(f"No metadata found for {self} from TMDB or OMDB")
return
vdict, series_id, cover, genres = (
metadata.as_dict_with_cover_and_genres()
)
for k, v in vdict.items():
setattr(self, k, v)
if series_id:
self.tv_series = Series.find_or_create(imdb_id=series_id)
self.save()
if cover:
self.save_image_from_url(cover)
if genres:
self.genre.add(*genres)
return
logger.warning(
f"No metadata source available for {self} (type={self.video_type})"
)
@classmethod @classmethod
def get_from_youtube_id(cls, youtube_id: str, overwrite: bool = False) -> "Video": def get_from_youtube_id(cls, youtube_id: str, overwrite: bool = False) -> "Video":
video, created = cls.objects.get_or_create(youtube_id=youtube_id) video, created = cls.objects.get_or_create(youtube_id=youtube_id)
@ -456,9 +534,7 @@ class Video(ScrobblableMixin):
metadata = lookup_video_from_omdb(imdb_id) metadata = lookup_video_from_omdb(imdb_id)
if not metadata or not metadata.title: if not metadata or not metadata.title:
logger.warning( logger.warning(f"No metadata found for {imdb_id} from TMDB or OMDB")
f"No metadata found for {imdb_id} from TMDB or OMDB"
)
return video return video
vdict, series_id, cover, genres = metadata.as_dict_with_cover_and_genres() vdict, series_id, cover, genres = metadata.as_dict_with_cover_and_genres()

View File

@ -32,7 +32,11 @@ class SeriesDetailView(LoginRequiredMixin, ChartContextMixin, generic.DetailView
context_data = super().get_context_data(**kwargs) context_data = super().get_context_data(**kwargs)
context_data["scrobbles"] = self.object.scrobbles_for_user(user_id) context_data["scrobbles"] = self.object.scrobbles_for_user(user_id)
next_episode_id = self.object.last_scrobbled_episode(user_id).next_imdb_id or "" next_episode_id = ""
if self.object.last_scrobbled_episode(user_id):
next_episode_id = (
self.object.last_scrobbled_episode(user_id).next_imdb_id or ""
)
if self.object.is_episode_playing(user_id): if self.object.is_episode_playing(user_id):
next_episode_id = "" next_episode_id = ""
if next_episode_id: if next_episode_id:

View File

@ -50,7 +50,11 @@
<h1 class="d-flex align-items-center gap-2"> <h1 class="d-flex align-items-center gap-2">
{% if object.media_type == "Video" %}🎬{% elif object.media_type == "Track" %}🎵{% elif object.media_type == "PodcastEpisode" %}🎙️{% elif object.media_type == "SportEvent" %}⚽{% elif object.media_type == "Book" %}📚{% elif object.media_type == "Paper" %}📄{% elif object.media_type == "VideoGame" %}🎮{% elif object.media_type == "BoardGame" %}🎲{% elif object.media_type == "GeoLocation" %}📍{% elif object.media_type == "Trail" %}🥾{% elif object.media_type == "Beer" %}🍺{% elif object.media_type == "Puzzle" %}🧩{% elif object.media_type == "Food" %}🍔{% elif object.media_type == "Task" %}✅{% elif object.media_type == "WebPage" %}🌐{% elif object.media_type == "LifeEvent" %}🎉{% elif object.media_type == "Mood" %}😊{% elif object.media_type == "BrickSet" %}🧱{% elif object.media_type == "Channel" %}📺{% endif %} {% if object.media_type == "Video" %}🎬{% elif object.media_type == "Track" %}🎵{% elif object.media_type == "PodcastEpisode" %}🎙️{% elif object.media_type == "SportEvent" %}⚽{% elif object.media_type == "Book" %}📚{% elif object.media_type == "Paper" %}📄{% elif object.media_type == "VideoGame" %}🎮{% elif object.media_type == "BoardGame" %}🎲{% elif object.media_type == "GeoLocation" %}📍{% elif object.media_type == "Trail" %}🥾{% elif object.media_type == "Beer" %}🍺{% elif object.media_type == "Puzzle" %}🧩{% elif object.media_type == "Food" %}🍔{% elif object.media_type == "Task" %}✅{% elif object.media_type == "WebPage" %}🌐{% elif object.media_type == "LifeEvent" %}🎉{% elif object.media_type == "Mood" %}😊{% elif object.media_type == "BrickSet" %}🧱{% elif object.media_type == "Channel" %}📺{% endif %}
{% if object.media_obj.get_absolute_url %}<a href="{{ object.media_obj.get_absolute_url }}">{% endif %}{{ object.media_obj }}{% if object.media_obj.get_absolute_url %}</a>{% endif %} {% if object.media_obj.get_absolute_url %}
<a href="{{ object.media_obj.get_absolute_url }}">{% endif %}
{{ object.media_obj.title }}
{% if object.media_obj.get_absolute_url %}</a>
{% endif %}
{% if user.is_authenticated and object.media_obj %} {% if user.is_authenticated and object.media_obj %}
<button id="favorite-btn" <button id="favorite-btn"
data-url="{% url 'scrobbles:toggle-favorite' object.media_type object.media_obj.id %}" data-url="{% url 'scrobbles:toggle-favorite' object.media_type object.media_obj.id %}"
@ -62,7 +66,15 @@
</button> </button>
{% endif %} {% endif %}
</h1> </h1>
<h2>{{ object.media_obj.subtitle }}</h2>
<p>
{% if object.media_type == "SportEvent" %}
{% for team in object.media_obj.teams.all %}
<img src="{{team.logo.url}}" width=150 />
{% endfor %}
{% endif %}
{% if object.media_type == "Task" and object.logdata.title %} {% if object.media_type == "Task" and object.logdata.title %}
</p>
<h2>{{ object.logdata.title }}</h2> <h2>{{ object.logdata.title }}</h2>
{% endif %} {% endif %}
<h3 class="text-muted">{{ object.local_timestamp }}</h3> <h3 class="text-muted">{{ object.local_timestamp }}</h3>

View File

@ -1,13 +1,16 @@
{% extends "base_detail.html" %} {% extends "base_detail.html" %}
{% block title %}{{object}}{% endblock %} {% block title %}{{object.title}}{% endblock %}
{% block details %} {% block details %}
<h2>{{object.subtitle}}</h2> <h2>{{object.league}} {{object.get_event_type_display}}</h2>
<div class="row"> <div class="row">
<h2>{{object.tv_series}}</h2>
<div class="col-md"> <div class="col-md">
{% for team in object.teams.all %}
<img src="{{team.logo.url}}" width=150 />
{% endfor %}
<hr />
<h3>Last scrobbles</h3> <h3>Last scrobbles</h3>
<div class="table-responsive"> <div class="table-responsive">
<table class="table table-striped table-sm"> <table class="table table-striped table-sm">