Compare commits

..

24 Commits
51.2 ... 53.0

Author SHA1 Message Date
d2705758c6 [release] Bump to version 53.0
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 1m59s
deploy / build-and-deploy (push) Successful in 52s
- Add a /trends/ page that shows trends based on scrobble data
- Notify users when Last.fm import completes
- Cleaner =GeoLocationLogData= deserialization
- Webpage scrobbles should diff existing webpages content
- Make ArchiveBox push asynchronous
2026-06-16 11:13:07 -04:00
f4368c31f3 [project] Update todos
Some checks failed
build / test (push) Has been cancelled
2026-06-16 11:12:50 -04:00
57f273b0cc [trends] Initial trends app
Some checks failed
build / test (push) Has been cancelled
2026-06-16 11:12:13 -04:00
ac82292200 [importers] Add last.fm import notifications 2026-06-16 10:26:08 -04:00
6a8432c08f [locations] Handle log data cleaner
All checks were successful
build / test (push) Successful in 1m54s
2026-06-16 10:12:58 -04:00
5a2c41155c [webpages] Add historical extract stashing
All checks were successful
build / test (push) Successful in 1m50s
2026-06-16 09:52:56 -04:00
83a046111b [webpages] Async pushing to archivebox
All checks were successful
build / test (push) Successful in 1m52s
2026-06-16 09:27:40 -04:00
ab10758f40 [release] Bump to version 52.2
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 1m56s
deploy / build-and-deploy (push) Successful in 50s
- Fix bug in recomputing long play seconds taking forever
2026-06-15 17:27:38 -04:00
88f16f0aaa [longplay] Fix recompute script
All checks were successful
build / test (push) Successful in 2m0s
2026-06-15 17:20:34 -04:00
c1744fab37 [release] Bump to version 52.1
All checks were successful
build / test (push) Successful in 1m56s
deploy / test (push) Successful in 2m1s
deploy / build-and-deploy (push) Successful in 29s
- Show time per scrobble in long play lists and total time playing
2026-06-15 15:27:10 -04:00
042a3eb737 [templates] Add aggregate data
Some checks failed
build / test (push) Has been cancelled
2026-06-15 15:26:46 -04:00
01d25e1b55 [release] Bump to version 52.0
All checks were successful
build / test (push) Successful in 1m55s
deploy / test (push) Successful in 2m1s
deploy / build-and-deploy (push) Successful in 31s
- Allow marking media as long play complete from detail page
- Fix how long play scrobbles are tracked
- Paginate or limite scrobbles on media admin pages
- Clean up books admin
- Clean up favorites admin
2026-06-15 14:56:27 -04:00
c0be131e3d [longplay] Add ability to undo finishes
Some checks failed
build / test (push) Has been cancelled
2026-06-15 14:55:58 -04:00
7d3f615ed7 [longplay] Make sure they're marked status is correct
All checks were successful
build / test (push) Successful in 2m2s
2026-06-15 14:37:20 -04:00
c2138b3ac6 [longplay] Add finish long play button
All checks were successful
build / test (push) Successful in 1m53s
2026-06-15 14:15:34 -04:00
947713d44a [longplay] Fix how we store long plays
All checks were successful
build / test (push) Successful in 2m3s
2026-06-15 14:12:21 -04:00
12b76837a3 [project] Update todos
All checks were successful
build / test (push) Successful in 2m11s
2026-06-15 13:43:58 -04:00
102494ede7 [admin] Use raw ids where possible and simplify scrobble inlines
All checks were successful
build / test (push) Successful in 1m55s
2026-06-15 13:25:10 -04:00
96bda8d4ad [data] Add example data 2026-06-15 12:38:21 -04:00
46956d06d8 [books] Clean up admin a little 2026-06-15 12:37:32 -04:00
8a28d0675b [release] Bump to version 51.4
All checks were successful
build / test (push) Successful in 1m56s
deploy / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 38s
- Clean up metadata comicbook enrichment
2026-06-15 12:20:18 -04:00
5f6e75b14e [books] Fix comic book metadata importing
Some checks failed
build / test (push) Has been cancelled
2026-06-15 12:19:54 -04:00
a96a42cdbf [release] Bump to version 51.3
All checks were successful
build / test (push) Successful in 2m12s
deploy / test (push) Successful in 2m6s
deploy / build-and-deploy (push) Successful in 1m5s
- Improve speed of index and chart pages
2026-06-12 13:35:01 -04:00
c7f5d7d384 [charts] Add index to speed things up 2026-06-12 13:34:41 -04:00
73 changed files with 2410 additions and 172 deletions

View File

@ -15,6 +15,8 @@ ro class method should call the utility function.
Be sure to check pyproject.toml for project defaults. Specifically for black and
isort expectations.
Imports in python files should always be top level if possible.
All tasks live in the PROJECT.org file and include an org ID that is a uuid to make them unique.
In local development, environment variables for various sensitive values live in a .envrc file

View File

@ -88,8 +88,8 @@ fetching and simple saving.
*** Metadata sources
**** Scraper
* Backlog [0/14] :vrobbler:project:personal:
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
* Backlog [0/19] :vrobbler:project:personal:
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :bug:music:scrobbles:
:PROPERTIES:
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
:END:
@ -387,11 +387,11 @@ fetching and simple saving.
}
}
#+end_src
** TODO [#C] User should be able to enable auto trail tracking via amail reader with Garmin LiveTrack URLs :vrobbler:trails:project:feature:personal:
** TODO [#C] Allow auto trail tracking via email with Garmin LiveTrack URLs :trails:feature:
:PROPERTIES:
:ID: 133bcf71-078f-4efa-a029-1eae4b4d146d
:END:
** TODO [#C] Fix exporting so it works reliably :exporting:project:feature:
** TODO [#C] Fix exporting so it works reliably :exporting:feature:
*** Description
@ -405,8 +405,7 @@ placed in the media directory:
And this should all be done in a celery task that is just kicked off by the
"Export" button on the frontend
** TODO [#B] Add AllTrails as a source for Trail data :vrobbler:trails:feature:personal:project:
** TODO [#B] Add AllTrails as a source for Trail data :trails:feature:
:PROPERTIES:
:ID: 39313362-cdfe-46e7-bbd4-9139a65c0b3c
:END:
@ -416,7 +415,7 @@ Pretty clear, I would love to make trails more useful. Historically I wasn't
hiking a lot, which made the source for this a bit silly. But it's clear that
AllTrails is the best source, though having TrailForks is nice to.
** TODO [#B] Add `garmin_activity_id` to the TrailLogData class :trails:feature:personal:project:
** TODO [#B] Add `garmin_activity_id` to the TrailLogData class :trails:feature:
:PROPERTIES:
:ID: 5a4fb0f8-0555-40ec-b06f-93c26bd686f4
:END:
@ -440,7 +439,7 @@ added.
They should also probably support markdown formatting and that should be
displayed in the template.
** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :books:feature:export:
https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
** TODO [#B] Make IMAP and WebDAV configurable :webdav:feature:imap:importers:
:PROPERTIES:
@ -465,7 +464,6 @@ needed import celery task. This is how the WebDAV celery task currently works.
This would also be an opporunity to clean up the code around WebDAV imports
and make them more re-usable for other import services.
** TODO [#A] Add an exception list of artists as a constant that are exempted from splitting :music:artists:metadata:
:PROPERTIES:
:ID: fd86a11a-73ec-470d-b5e3-2d90ba9137c8
@ -477,7 +475,6 @@ Certain artists like "Simon & Garfunkel" are actually one artist. While we don't
tracks into featured artists, we should have a "LITERAL_ARTIST_TITLES" constant that can have exceptions like
this put into it and then we stop trying to pull the artist apart when we run into it.
** TODO [#A] Before enriching anything, trust the POST data :feature:scrobbles:metadata:
:PROPERTIES:
:ID: db6b05f8-09f4-49f5-9838-fbacc9fe9cd0
@ -509,15 +506,251 @@ log a warning and move on.
We should have a global view `/favorites/` that shows the logged in users's
favorited media objects.
** TODO [#B] Scrape ComicBookRoundUp ratings for comic book metadata :vrobbler:books:feature:comicbook:personal:project:
- Note taken on [2025-09-25 Thu 10:51]
As an example https://comicbookroundup.com/comic-books/reviews/humanoids-publishing/the-history-of-science-fiction
** TODO [#B] Find page numbers for comic books from ComicVine :feature:books:
:PROPERTIES:
:ID: 79f867c3-1288-4143-b6bf-2a452983ee9f
:END:
** TODO [#C] Implement loguru into project :feature:loguru:logging:
:PROPERTIES:
:ID: efcd0c0a-db81-4518-9c23-5505d59e8ef5
:END:
*** Description
Would be great to formalize how we log so we can search for errors and such more
easily. And our exposure to PII is really low at this point in the project,
so we can probably use backtrace=True and diagnose=True to help us root cause
bugs faster.
** TODO [#B] Scrape ComicBookRoundUp ratings for comic book metadata :books:feature:comicbook:
:PROPERTIES:
:ID: b3cc57ca-3d2c-468d-ab7c-c47f1120309b
:END:
*** Description
As an example https://comicbookroundup.com/comic-books/reviews/humanoids-publishing/the-history-of-science-fiction
** TODO [#C] Make podcast date format configurable in settings :podcasts:configuration:
:PROPERTIES:
:ID: b01a94f8-328f-41ed-a62e-8b99c755b82d
:END:
*** Description
=PODCAST_DATE_FORMAT= is hardcoded to ="YYYY-MM-DD"=. Should be in Django settings or environment variables for deploy-specific configuration.
File: ~vrobbler/apps/podcasts/utils.py~ (line 13)
** TODO [#C] Extract zombie scrobble query into custom manager :refactoring:manager:
:PROPERTIES:
:ID: 79c874e1-ca6f-4bce-9259-e3eebdca8a41
:END:
*** Description
The zombie scrobble cleanup query lives in a utility function. Should be a
custom model manager method (e.g. =Scrobble.objects.zombies()=).
File: ~vrobbler/apps/scrobbles/utils.py~ (line 204)
** TODO [#C] Allow profile to set start of week :profiles:configuration:
:PROPERTIES:
:ID: 0449279a-9550-430e-be0c-816df7273080
:END:
*** Description
=start_of_week()= and =end_of_week()= use Monday as default. Should be a user
profile setting for different cultural week start conventions.
File: ~vrobbler/apps/profiles/utils.py~ (lines 39, 44)
** TODO [#C] Add constants for data dictionary keys (multiple files) :refactoring:constants:
:PROPERTIES:
:ID: d4415f9b-620a-4be7-925d-fa71c02ba1d1
:END:
*** Description
Multiple files use magic string literals for dict keys. Should be extracted to
named constants for maintainability.
- Files:
- ~vrobbler/apps/locations/models.py~ (line 63) -- ="lat"=, ="lon"= etc.
- ~vrobbler/apps/webpages/models.py~ (line 290) -- ="url"=
- ~vrobbler/apps/scrobbles/importers/tsv.py~ (line 55) -- ="S"= completion status
* Version 53.0 [5/5]
** DONE [#B] Add a /trends/ page that shows trends based on scrobble data :feature:trends:scrobbles:
:PROPERTIES:
:ID: 03e9fe30-2bc6-4062-bb24-e95b98daf05b
:END:
*** Description
This project is a bit invovled. But we should add a top level URL `trends` that shows
various trends as defined either in a static settings file, or dynamically via a database table.
Trends could be things like doing multiple things at the same time, like while driving, what
did we listen to this week, or while running, what were listening to this week?
Or more complicated trends like, how time per page changes based on the book I was reading, or if I was doing something else (music or sport event) while reading.
** DONE [#B] Notify users when Last.fm import completes :importers:notifications:
:PROPERTIES:
:ID: 92846b36-54c5-4b78-9c57-bdc401045fbe
:END:
*** Description
After a bulk import from Last.fm, users receive no confirmation. Should add a notification (in-app, email, or similar).
File: ~vrobbler/apps/scrobbles/importers/lastfm.py~ (line 96)
** DONE [#C] Cleaner =GeoLocationLogData= deserialization :models:refactoring:
:PROPERTIES:
:ID: 85465dbf-69b3-48cb-9df0-cd076c4470ab
:END:
*** Description
Currently special-cases =GeoLocationLogData= by reaching into a nested ="movement_detection"= key. Should be handled at the LogData dataclass level.
File: ~vrobbler/apps/scrobbles/models.py~ (line 977)
** DONE [#B] Webpage scrobbles should diff existing webpages content :webpages:metadata:
:PROPERTIES:
:ID: 25576197-258f-48d6-bfe9-e4172a0a1898
:END:
*** Description
Webpages change content between scrobbles. The current model stores the webpage content once, the
first time it's scrobbled. When a page has been seen before, we should move the existing content
to a new model HistoricalWebPage with the following fields:
webpage_id -> FK to WebPage
date -> date from existing WebPage content
domain -> same as existing WebPage content
extract -> copy of existing WebPage content
Once the HistoricalWebPage instance is successfully created, the new extract data
should be saved into the WebPage instance.
** DONE [#B] Make ArchiveBox push asynchronous :archivebox:async:
:PROPERTIES:
:ID: 17c116a7-5952-db37-e56c-2987c2fc456b
:END:
*** Description
=push_to_archivebox()= runs synchronously during the request. Should be moved to a
Celery task or similar background worker.
File: ~vrobbler/apps/webpages/models.py~ (line 133)
* Version 52.2 [1/1]
** DONE [#A] Fix bug in recomputing long play seconds taking forever :bug:longplay:commands:
:PROPERTIES:
:ID: 0a813cf9-17fb-dbd7-b5a7-7410d9bd4d8c
:END:
* Version 52.1 [1/1]
** DONE [#C] Show time per scrobble in long play lists and total time playing :templates:longplay:scrobbles:
:PROPERTIES:
:ID: b3d16230-8ec5-46db-b166-59e98d0ee06c
:END:
*** Description
Long play time should be show in the table of scrobbles on a media detail page.
The total time spent in a long play that's either no completed yet or completed
should be displayed as well. If completed, the date finished should be shown as
well.
* Version 52.0 [5/5]
** DONE [#B] Allow marking media as long play complete from detail page :templates:scrobbles:longplay:
:PROPERTIES:
:ID: 2c314768-be97-9b10-d13c-9cfd0f38a64e
:END:
** DONE [#A] Fix how long play scrobbles are tracked :scrobbles:longplay:serial:
:PROPERTIES:
:ID: 908b0493-cabf-40c1-825f-cd59a8ad0f7a
:END:
*** Description
Currently we have this idea of "long_play" scrobbles but there's a lot missing
to tie it together.
What we'd prefer is that when a new scrobble is added for a media_type that
`is_long_play` the most recent scrobble finished is added as the
`last_serial_scrobble` to the log data. But all the other long play stuff exsits
as data model fields. We should add `long_play_last_scrobble` as a FK to this
scrobble when creating a new longplay scrobble.
Additionally, `long_play_seconds` we should have a recompute management command
to walk backward from `long_play_last_scrobble` until a `long_play_complete`
scrobble is found (exclusive) and save the time.
We should also ony use `long_play_complete` field on the scrobble ... some
logdatas have a similar field, but we should make sure that we always use the
model field to determine if a long play is finished.
This should include a command to clean up long play data to consolidate around
the `long_play_complete` field.
** DONE [#B] Paginate or limite scrobbles on media admin pages :admin:scrobbles:media:
:PROPERTIES:
:ID: f02e487b-d7ed-4834-838a-303560f2ad3b
:END:
** DONE [#B] Clean up books admin :admin:books:bug:
:PROPERTIES:
:ID: 7539bee6-0a52-26f6-ebc6-5554ac49a716
:END:
** DONE [#B] Clean up favorites admin :admin:favorites:scrobbles:
:PROPERTIES:
:ID: f2be0c69-1bf8-b5a3-5269-9c8ea873361d
:END:
*** Description
Some FK lookups in admin should be raw_id_fields.
* Version 51.4 [1/1]
** DONE [#A] Clean up metadata comicbook enrichment :bug:comics:books:metadata:
:PROPERTIES:
:ID: cd875450-7117-78ca-8be4-9c8b73037dba
:END:
*** Description
Still getting wonky results with some comicbooks. Would be nice to be able to
tag a Book as a comicbook, and also gather volume information. I also noticed
that some books that are found in OL never get their comicvine_id populated. We
should make sure we always have comicvine_ids if available.
* Version 51.3 [1/1]
** DONE [#A] Improve speed of index and chart pages :bug:scrobbles:perf:
:PROPERTIES:
:ID: 031a23f8-7c02-4926-9884-6654ceca16c2
:END:
*** Description
Over the last few releases, the home page and charts pages have gotten really
slow.
We should look into what's causing the slowness and maybe do more agressive
query optimization or caching.
* Version 51.2 [2/2]
** DONE [#A] Fix bug where last page of book gets separate scrobble :bug:books:importers:koreader:
:PROPERTIES:

96
data/play-example.json Normal file
View File

@ -0,0 +1,96 @@
{
"about": "This is a Play file that can be read by Board Game Stats. If you see this text, try to use a share, export or open-in function to open it with Board Game Stats.",
"players": [
{
"uuid": "31f8b92e-11d8-4162-88b1-fd9c79eea249",
"id": 2,
"name": "Colin Powell",
"isAnonymous": false,
"modificationDate": "2025-10-18 08:32:40",
"metaData": "{\"isNpc\":0,\"playerAvatar\":{\"image\":\"AnnikaHeller_Capybara.webp\",\"shape\":[1,1,1,1],\"color\":[0.6,0.20000000298023224,0.9803921580314636]}}"
},
{
"uuid": "dd2d1881-84ab-474c-a6b2-3045d034dc40",
"id": 3,
"name": "Silas Sewell",
"isAnonymous": false,
"modificationDate": "2026-01-18 12:27:12",
"metaData": "{\"isNpc\":0,\"playerAvatar\":{\"image\":\"RoryMuldoon_07.webp\",\"shape\":[1,1,1,1],\"color\":[0,0,0.1835034190722739]}}"
}
],
"locations": [
{
"uuid": "14f7389c-767f-4725-9b35-906c407b293c",
"id": 3,
"name": "Timberwyck Farm",
"modificationDate": "2025-07-01 18:03:38"
}
],
"games": [
{
"uuid": "9e431cdd-b325-4061-a875-d415d46342c0",
"id": 1046,
"name": "Sweet Takes",
"modificationDate": "2026-04-11 16:25:35",
"cooperative": false,
"highestWins": true,
"noPoints": false,
"usesTeams": false,
"urlThumb": "https://cf.geekdo-images.com/l4HILZn5iLbceQeDph4G5A__small/img/gmGqlmwe9fniqYpniGwhg5RUgVQ=/fit-in/200x150/filters:strip_icc()/pic8784202.jpg",
"urlImage": "https://cf.geekdo-images.com/l4HILZn5iLbceQeDph4G5A__original/img/b4IU8WIEWRpacbXp0FHG9HfFRpw=/0x0/filters:format(jpeg)/pic8784202.jpg",
"bggName": "Sweet Takes",
"bggYear": 2023,
"bggId": 407581,
"designers": "Hisashi Hayashi",
"isBaseGame": 1,
"isExpansion": 0,
"rating": 67,
"minPlayerCount": 2,
"maxPlayerCount": 5,
"minPlayTime": 15,
"maxPlayTime": 15,
"minAge": 8
}
],
"plays": [
{
"uuid": "7b2fd583-e8f2-40fe-9565-90178390b87e",
"modificationDate": "2026-04-16 20:18:03",
"entryDate": "2026-04-16 20:13:33",
"playDate": "2026-04-16 20:13:33",
"usesTeams": false,
"durationMin": 4,
"ignored": false,
"manualWinner": false,
"rounds": 0,
"locationRefId": 3,
"gameRefId": 1046,
"board": "",
"scoringSetting": 1,
"metaData": "{\"playerRefId\":2,\"playGameBggVersion\":\"{\\\"versionId\\\":0,\\\"versionName\\\":\\\"\\\",\\\"imageUrl\\\":\\\"https:\\\\\\/\\\\\\/cf.geekdo-images.com\\\\\\/l4HILZn5iLbceQeDph4G5A__small\\\\\\/img\\\\\\/gmGqlmwe9fniqYpniGwhg5RUgVQ=\\\\\\/fit-in\\\\\\/200x150\\\\\\/filters:strip_icc()\\\\\\/pic8784202.jpg\\\",\\\"thumbUrl\\\":\\\"https:\\\\\\/\\\\\\/cf.geekdo-images.com\\\\\\/l4HILZn5iLbceQeDph4G5A__small\\\\\\/img\\\\\\/gmGqlmwe9fniqYpniGwhg5RUgVQ=\\\\\\/fit-in\\\\\\/200x150\\\\\\/filters:strip_icc()\\\\\\/pic8784202.jpg\\\",\\\"yearPublished\\\":0}\",\"playUsedGameCopy\":2}",
"playerScores": [
{
"score": "27",
"winner": false,
"newPlayer": false,
"startPlayer": false,
"playerRefId": 2,
"role": "",
"rank": 0,
"seatOrder": 0
},
{
"score": "36",
"winner": true,
"newPlayer": true,
"startPlayer": false,
"playerRefId": 3,
"rank": 0,
"seatOrder": 0
}
],
"expansionPlays": []
}
],
"userInfo": { "meRefId": 2 }
}

BIN
data/statistics.sqlite3 Normal file

Binary file not shown.

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "vrobbler"
version = "51.2"
version = "53.0"
description = ""
authors = ["Colin Powell <colin@unbl.ink>"]

View File

@ -27,6 +27,7 @@ class BeerAdmin(admin.ModelAdmin):
"uuid",
"title",
)
raw_id_fields = ("styles", "producer")
ordering = ("-created",)
search_fields = ("title",)
inlines = [

View File

@ -27,4 +27,5 @@ class BirdingLocationAdmin(admin.ModelAdmin):
class BirdingCSVImportAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("uuid", "process_count", "processed_finished", "processing_started", "error_log")
raw_id_fields = ("user",)
ordering = ("-created",)

View File

@ -38,6 +38,7 @@ class BoardGameLocationAdmin(admin.ModelAdmin):
"uuid",
"geo_location",
)
raw_id_fields = ("geo_location",)
ordering = ("-created",)
@ -49,6 +50,7 @@ class BoardGameAdmin(admin.ModelAdmin):
"title",
"published_year",
)
raw_id_fields = ("publisher", "publishers", "designers", "expansion_for_boardgame")
search_fields = ("title",)
ordering = ("-created",)
inlines = [

View File

@ -27,6 +27,7 @@ class BookAdmin(admin.ModelAdmin):
"first_publish_year",
"pages",
)
raw_id_fields = ("authors",)
search_fields = ("name",)
ordering = ("-created",)
inlines = [
@ -34,11 +35,11 @@ class BookAdmin(admin.ModelAdmin):
]
def issue_or_volume(self, obj):
return obj.issue_number or obj.volume_number
return obj.subtitle
@admin.register(Paper)
class BookAdmin(admin.ModelAdmin):
class PaperAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = (
"title",
@ -47,6 +48,7 @@ class BookAdmin(admin.ModelAdmin):
"first_publish_year",
"pages",
)
raw_id_fields = ("authors",)
search_fields = ("name",)
ordering = ("-created",)
inlines = [

View File

@ -344,13 +344,16 @@ def build_scrobbles_from_book_map(book_map: dict, user: "User") -> list["Scrobbl
def fix_long_play_stats_for_scrobbles(scrobbles: list) -> None:
"""Given a list of scrobbles, update pages read, long play seconds and check
for media completion"""
for media completion.
Uses the long_play_last_scrobble FK chain to accumulate time.
Consider using the recompute_long_play_seconds management command instead.
"""
for scrobble in scrobbles:
# But if there's a next scrobble, set pages read to their starting page
if scrobble.previous and not scrobble.previous.long_play_complete:
if scrobble.long_play_last_scrobble and not scrobble.long_play_last_scrobble.long_play_complete:
scrobble.long_play_seconds = scrobble.playback_position_seconds + (
scrobble.previous.long_play_seconds or 0
scrobble.long_play_last_scrobble.long_play_seconds or 0
)
else:
scrobble.long_play_seconds = scrobble.playback_position_seconds

View File

@ -160,7 +160,10 @@ class Command(BaseCommand):
)
def _enrich_book(self, book, sleep_secs):
from books.sources.comicvine import lookup_comic_from_comicvine
from books.sources.comicvine import (
lookup_comic_from_comicvine,
lookup_issue_by_comicvine_id,
)
from books.sources.google import lookup_book_from_google
from books.sources.openlibrary import lookup_book_from_openlibrary as lookup_book_from_ol
@ -168,13 +171,13 @@ class Command(BaseCommand):
author_name = book.author.name if book.author else None
book_dict = {}
is_comic = bool(book.readcomics_url) or (
book.issue_number is not None or book.volume_number is not None
)
if is_comic and READCOMICSONLINE_URL in (book.readcomics_url or ""):
cv_data = None
if book.comicvine_id:
cv_data = lookup_issue_by_comicvine_id(str(book.comicvine_id))
if not cv_data:
cv_data = lookup_comic_from_comicvine(title)
if cv_data:
book_dict.update(cv_data)
if cv_data:
book_dict.update(cv_data)
ol_data = lookup_book_from_ol(title, author=author_name)
time.sleep(sleep_secs)
@ -261,6 +264,14 @@ class Command(BaseCommand):
book.volume_number = data["volume_number"]
update_fields.append("volume_number")
if data.get("volume") and not book.volume:
book.volume = data["volume"]
update_fields.append("volume")
if data.get("volume_comicvine_id") and not book.volume_comicvine_id:
book.volume_comicvine_id = data["volume_comicvine_id"]
update_fields.append("volume_comicvine_id")
if update_fields:
book.save(update_fields=update_fields)
self.stdout.write(f" [ENRICHED] {book}{', '.join(update_fields)}")
@ -279,4 +290,12 @@ class Command(BaseCommand):
book.genre.add(*new_genres)
self.stdout.write(f" [GENRES] {book} — added {len(new_genres)} genres")
tags = data.pop("tags", [])
if tags:
existing_tags = set(book.tags.names())
new_tags = [t for t in tags if t not in existing_tags]
if new_tags:
book.tags.add(*new_tags)
self.stdout.write(f" [TAGS] {book} — added {', '.join(new_tags)}")
return changed if any(changed.values()) else None

View File

@ -0,0 +1,23 @@
# Generated by Django 4.2.29 on 2026-06-15 16:09
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("books", "0036_alter_book_genre_alter_paper_genre"),
]
operations = [
migrations.AddField(
model_name="book",
name="volume",
field=models.CharField(blank=True, max_length=255, null=True),
),
migrations.AddField(
model_name="book",
name="volume_comicvine_id",
field=models.CharField(blank=True, max_length=255, null=True),
),
]

View File

@ -20,6 +20,7 @@ from books.openlibrary import (
from books.sources.comicvine import (
ComicVineClient,
lookup_comic_from_comicvine,
lookup_issue_by_comicvine_id,
)
from books.sources.google import lookup_book_from_google
from books.sources.openlibrary import (
@ -150,6 +151,8 @@ class Book(LongPlayScrobblableMixin):
first_sentence = models.TextField(**BNULL)
# ComicVine
comicvine_id = models.CharField(max_length=255, **BNULL)
volume = models.CharField(max_length=255, **BNULL)
volume_comicvine_id = models.CharField(max_length=255, **BNULL)
readcomics_url = models.CharField(max_length=255, **BNULL)
next_readcomics_url = models.CharField(max_length=255, **BNULL)
issue_number = models.IntegerField(**BNULL)
@ -236,13 +239,17 @@ class Book(LongPlayScrobblableMixin):
if not book_dict:
return book
tags = book_dict.pop("tags", [])
genres = book_dict.pop("genres", book_dict.pop("generes", []))
for k, v in book_dict.items():
setattr(book, k, v)
book.save()
genres = book_dict.get("genres", [])
if genres:
book.genre.add(*genres)
if tags:
book.tags.add(*tags)
return book
@classmethod
@ -276,8 +283,10 @@ class Book(LongPlayScrobblableMixin):
book_dict = None
source_tag = None
tried_comicvine = False
if READCOMICSONLINE_URL in url:
book_dict = lookup_comic_from_comicvine(title)
tried_comicvine = True
if book_dict:
source_tag = MediaSourceTag.COMICVINE
book_dict["readcomics_url"] = get_comic_issue_url(url)
@ -302,6 +311,16 @@ class Book(LongPlayScrobblableMixin):
if ol_data and ol_data.get("cover_url"):
book_dict["cover_url"] = ol_data["cover_url"]
# Always try ComicVine as a fallback — it may recognize books that
# OL/Google don't flag as comics
if not tried_comicvine:
cv_data = lookup_comic_from_comicvine(title)
if cv_data:
for k, v in cv_data.items():
if v:
book_dict.setdefault(k, v)
source_tag = MediaSourceTag.COMICVINE
if not book_dict:
logger.warning(
"No book found in any source, using data as is",
@ -312,6 +331,7 @@ class Book(LongPlayScrobblableMixin):
authors = book_dict.pop("authors", [])
cover_url = book_dict.pop("cover_url", "")
genres = book_dict.pop("genres", book_dict.pop("generes", []))
tags = book_dict.pop("tags", [])
if authors:
for author_str in authors:
@ -331,6 +351,8 @@ class Book(LongPlayScrobblableMixin):
book.save_image_from_url(cover_url)
if genres:
book.genre.add(*genres)
if tags:
book.tags.add(*tags)
book.authors.add(*author_list)
if source_tag:
book.tags.add(source_tag.value)
@ -368,8 +390,14 @@ class Book(LongPlayScrobblableMixin):
data = lookup_comic_from_locg(str(self.title))
if not data and COMICVINE_API_KEY:
logger.warn(f"Checking ComicVine for {self.title}")
data = lookup_comic_from_comicvine(str(self.title))
if self.comicvine_id:
logger.warn(
f"Checking ComicVine by ID for {self.title}"
)
data = lookup_issue_by_comicvine_id(str(self.comicvine_id))
if not data:
logger.warn(f"Checking ComicVine for {self.title}")
data = lookup_comic_from_comicvine(str(self.title))
if not data:
logger.warn(f"Book not found in any sources: {self.title}")
@ -407,10 +435,10 @@ class Book(LongPlayScrobblableMixin):
)
data.pop("pages")
# Pop this, so we can look it up later
# Pop these so they don't get passed to update()
cover_url = data.pop("cover_url", "")
subject_key_list = data.pop("subject_key_list", [])
tags = data.pop("tags", [])
# Fun trick for updating all fields at once
Book.objects.filter(pk=self.id).update(**data)
@ -418,6 +446,8 @@ class Book(LongPlayScrobblableMixin):
if subject_key_list:
self.genre.add(*subject_key_list)
if tags:
self.tags.add(*tags)
if cover_url:
r = requests.get(cover_url)

View File

@ -17,8 +17,10 @@ class ComicVineClient(object):
account on https://comicvine.gamespot.com/ in order to obtain an API key.
"""
# All API requests made by this client will be made to this URL.
# All API requests made by this client will be made to these URLs.
API_URL = "https://comicvine.gamespot.com/api/search/"
ISSUE_API_URL = "https://comicvine.gamespot.com/api/issue/4000-{issue_id}/"
VOLUME_API_URL = "https://comicvine.gamespot.com/api/volume/4050-{volume_id}/"
# A valid User-Agent header must be set in order for our API requests to
# be accepted, otherwise our request will be rejected with a
@ -197,6 +199,74 @@ class ComicVineClient(object):
raise exception(message)
def get_issue(self, issue_id: str) -> dict:
"""
Fetch a single issue by its ComicVine ID directly from the issue detail
endpoint, which returns richer data than the search endpoint.
:param issue_id: The ComicVine numeric ID for the issue (e.g. "538480")
:type issue_id: str
:return: The full JSON response for the issue, or empty dict on failure.
:rtype: dict
"""
params = {
"api_key": self.api_key,
"format": "json",
}
url = self.ISSUE_API_URL.format(issue_id=issue_id)
response = requests.get(url, headers=self.HEADERS, params=params)
if not response.ok:
self._handle_http_error(response)
json_data = response.json()
if json_data.get("status_code") != 1:
error_msg = json_data.get("error", "Unknown ComicVine API error")
logger.error(
"ComicVine API returned status_code %s: %s",
json_data.get("status_code"),
error_msg,
)
return {}
return json_data.get("results", {})
def get_volume(self, volume_id: str) -> dict:
"""
Fetch a single volume by its ComicVine ID from the volume detail
endpoint. Used to get publisher info and other volume-level metadata.
:param volume_id: The ComicVine numeric ID for the volume (e.g. "91273")
:type volume_id: str
:return: The full JSON response for the volume, or empty dict on failure.
:rtype: dict
"""
params = {
"api_key": self.api_key,
"format": "json",
}
url = self.VOLUME_API_URL.format(volume_id=volume_id)
response = requests.get(url, headers=self.HEADERS, params=params)
if not response.ok:
self._handle_http_error(response)
json_data = response.json()
if json_data.get("status_code") != 1:
error_msg = json_data.get("error", "Unknown ComicVine API error")
logger.error(
"ComicVine API returned status_code %s: %s",
json_data.get("status_code"),
error_msg,
)
return {}
return json_data.get("results", {})
def lookup_comic_from_comicvine(title: str) -> dict:
original_title = title
@ -238,26 +308,113 @@ def lookup_comic_from_comicvine(title: str) -> dict:
if not found_result:
found_result = results[0]
title = found_result.get("name")
data_dict = _build_data_dict_from_issue(found_result, original_title)
_enrich_with_volume_data(client, data_dict)
return data_dict
if found_result.get("volume"):
title = found_result.get("volume").get("name")
def lookup_issue_by_comicvine_id(comicvine_id: str) -> dict:
"""
Look up an issue directly by its ComicVine ID using the issue detail
endpoint. Returns richer data than the search-based lookup.
:param comicvine_id: The ComicVine numeric ID for the issue (e.g. "538480")
:type comicvine_id: str
:return: A dict of extracted book metadata, or empty dict on failure.
:rtype: dict
"""
if not comicvine_id:
return {}
api_key = getattr(settings, "COMICVINE_API_KEY", "")
if not api_key:
logger.warning("No ComicVine API key configured, not looking anything up")
return {}
client = ComicVineClient(api_key=api_key)
issue_data = client.get_issue(comicvine_id)
if not issue_data:
logger.warning("No issue found on ComicVine for ID %s", comicvine_id)
return {}
data_dict = _build_data_dict_from_issue(issue_data, issue_data.get("name", ""))
_enrich_with_volume_data(client, data_dict)
return data_dict
def _build_data_dict_from_issue(issue_data: dict, original_title: str = "") -> dict:
"""
Build a book metadata dict from a ComicVine issue resource (either from
search results or issue detail endpoint). Both return the same shape of
issue data.
:param issue_data: The issue resource dict from ComicVine.
:param original_title: The original search term, if any.
:return: A dict of extracted book metadata.
:rtype: dict
"""
title = issue_data.get("name")
if issue_data.get("volume"):
title = issue_data.get("volume").get("name")
cover_url = None
if found_result.get("image"):
cover_url = found_result["image"].get("original_url")
if issue_data.get("image"):
cover_url = issue_data["image"].get("original_url")
volume_name = None
volume_cv_id = None
publisher_name = None
volume_data = issue_data.get("volume")
if volume_data:
volume_name = volume_data.get("name")
volume_cv_id = volume_data.get("id")
publisher_data = volume_data.get("publisher")
if publisher_data:
publisher_name = publisher_data.get("name")
data_dict = {
"title": title,
"original_title": original_title,
"issue_number": found_result.get("issue_number"),
"volume_number": found_result.get("volume_number"),
"issue_number": issue_data.get("issue_number"),
"volume_number": issue_data.get("volume_number"),
"volume": volume_name,
"volume_comicvine_id": volume_cv_id,
"publisher": publisher_name,
"cover_url": cover_url,
"comicvine_id": found_result.get("id"),
"comicvine_data": found_result,
"summary": found_result.get("description"),
"publish_date": found_result.get("cover_date"),
"first_publish_year": (found_result.get("cover_date") or "")[:4],
"comicvine_id": issue_data.get("id"),
"summary": issue_data.get("description"),
"publish_date": issue_data.get("cover_date"),
"first_publish_year": (issue_data.get("cover_date") or "")[:4],
"tags": ["comicbook"],
}
return data_dict
def _enrich_with_volume_data(client: ComicVineClient, data_dict: dict) -> None:
"""
Follow-up a successful issue lookup by fetching the volume detail and
filling in publisher and other volume-level metadata that the issue
endpoint doesn't provide.
:param client: An initialised ComicVineClient instance.
:param data_dict: The data dict from an issue lookup (mutated in place).
"""
volume_cv_id = data_dict.get("volume_comicvine_id")
if not volume_cv_id:
return
volume_data = client.get_volume(str(volume_cv_id))
if not volume_data:
return
publisher_data = volume_data.get("publisher")
if publisher_data:
publisher_name = publisher_data.get("name")
if publisher_name and not data_dict.get("publisher"):
data_dict["publisher"] = publisher_name
if not data_dict.get("volume"):
data_dict["volume"] = volume_data.get("name")

View File

@ -0,0 +1,146 @@
# Generated by Django 4.2.29 on 2026-06-12 16:09
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("charts", "0001_initial"),
]
operations = [
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "album", "rank"],
name="charts_char_user_id_1adcde_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "track", "rank"],
name="charts_char_user_id_d18aab_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "video", "rank"],
name="charts_char_user_id_de9f0a_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "board_game", "rank"],
name="charts_char_user_id_d5d58f_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "book", "rank"],
name="charts_char_user_id_e877cf_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "food", "rank"],
name="charts_char_user_id_a0ad71_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "podcast", "rank"],
name="charts_char_user_id_846b80_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "trail", "rank"],
name="charts_char_user_id_54feba_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "album", "rank"],
name="charts_char_user_id_a3dc49_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "track", "rank"],
name="charts_char_user_id_4b01ab_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "video", "rank"],
name="charts_char_user_id_2ac9d2_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "board_game", "rank"],
name="charts_char_user_id_ba968a_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "book", "rank"],
name="charts_char_user_id_e66751_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "food", "rank"],
name="charts_char_user_id_d23f06_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "podcast", "rank"],
name="charts_char_user_id_be8122_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "week", "trail", "rank"],
name="charts_char_user_id_b94ea9_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "day", "artist", "rank"],
name="charts_char_user_id_406e0e_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "day", "album", "rank"],
name="charts_char_user_id_322b0d_idx",
),
),
migrations.AddIndex(
model_name="chartrecord",
index=models.Index(
fields=["user", "year", "month", "day", "tv_series", "rank"],
name="charts_char_user_id_aa44b7_idx",
),
),
]

View File

@ -60,10 +60,29 @@ class ChartRecord(TimeStampedModel):
models.Index(fields=["user", "year", "geo_location", "rank"]),
models.Index(fields=["user", "year", "food", "rank"]),
models.Index(fields=["user", "year", "book", "rank"]),
models.Index(fields=["user", "year", "week", "artist", "rank"]),
models.Index(fields=["user", "year", "week", "tv_series", "rank"]),
models.Index(fields=["user", "year", "month", "artist", "rank"]),
models.Index(fields=["user", "year", "month", "album", "rank"]),
models.Index(fields=["user", "year", "month", "track", "rank"]),
models.Index(fields=["user", "year", "month", "tv_series", "rank"]),
models.Index(fields=["user", "year", "month", "video", "rank"]),
models.Index(fields=["user", "year", "month", "board_game", "rank"]),
models.Index(fields=["user", "year", "month", "book", "rank"]),
models.Index(fields=["user", "year", "month", "food", "rank"]),
models.Index(fields=["user", "year", "month", "podcast", "rank"]),
models.Index(fields=["user", "year", "month", "trail", "rank"]),
models.Index(fields=["user", "year", "week", "artist", "rank"]),
models.Index(fields=["user", "year", "week", "album", "rank"]),
models.Index(fields=["user", "year", "week", "track", "rank"]),
models.Index(fields=["user", "year", "week", "tv_series", "rank"]),
models.Index(fields=["user", "year", "week", "video", "rank"]),
models.Index(fields=["user", "year", "week", "board_game", "rank"]),
models.Index(fields=["user", "year", "week", "book", "rank"]),
models.Index(fields=["user", "year", "week", "food", "rank"]),
models.Index(fields=["user", "year", "week", "podcast", "rank"]),
models.Index(fields=["user", "year", "week", "trail", "rank"]),
models.Index(fields=["user", "year", "month", "day", "artist", "rank"]),
models.Index(fields=["user", "year", "month", "day", "album", "rank"]),
models.Index(fields=["user", "year", "month", "day", "tv_series", "rank"]),
]
@property

View File

@ -438,12 +438,14 @@ class ChartRecordView(TemplateView):
return context
def get_available_years(self, user):
return list(
ChartRecord.objects.filter(user=user)
.values_list("year", flat=True)
.distinct()
.order_by("-year")
)
if not hasattr(self, "_available_years"):
self._available_years = list(
ChartRecord.objects.filter(user=user)
.values_list("year", flat=True)
.distinct()
.order_by("-year")
)
return self._available_years
def get_period_type(self):
date_param = self.request.GET.get("date")

View File

@ -21,6 +21,7 @@ class FoodAdmin(admin.ModelAdmin):
"uuid",
"title",
)
raw_id_fields = ("category",)
ordering = ("-created",)
search_fields = ("title",)
inlines = [

View File

@ -28,6 +28,14 @@ class GeoLocationLogData(BaseLogData, WithPeopleLogData):
activity: str = ""
detected_at: str = ""
@classmethod
def from_log_dict(cls, log_dict: dict) -> dict:
instance_data = log_dict.get("movement_detection", {}).copy()
for field_name in ["description", "notes", "with_people_ids"]:
if field_name in log_dict:
instance_data[field_name] = log_dict[field_name]
return instance_data
class GeoLocation(ScrobblableMixin):
COMPLETION_PERCENT = getattr(settings, "LOCATION_COMPLETION_PERCENT", 100)

View File

@ -6,6 +6,7 @@ from people.models import Person, PersonScrobble
class PersonAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "bgg_username", "bgstats_id")
raw_id_fields = ("user", "created_by")
ordering = ("-created",)
search_fields = ("name",)

View File

@ -18,6 +18,7 @@ class PodcastAdmin(admin.ModelAdmin):
"producer",
"active",
)
raw_id_fields = ("producer",)
ordering = ("name",)
@ -28,6 +29,7 @@ class PodcastEpisodeAdmin(admin.ModelAdmin):
"title",
"podcast",
)
raw_id_fields = ("podcast",)
list_filter = ("podcast",)
ordering = ("-created",)
inlines = [

View File

@ -6,6 +6,7 @@ from profiles.models import UserProfile
@admin.register(UserProfile)
class UserProfileAdmin(admin.ModelAdmin):
date_hierarchy = "created"
raw_id_fields = ("user",)
ordering = ("-created",)
readonly_fields = ("timezone_change_log",)
exclude = (

View File

@ -21,6 +21,7 @@ class PuzzleAdmin(admin.ModelAdmin):
"uuid",
"title",
)
raw_id_fields = ("manufacturer",)
ordering = ("-created",)
search_fields = ("title",)
inlines = [

View File

@ -19,35 +19,11 @@ from scrobbles.mixins import Genre
class ScrobbleInline(admin.TabularInline):
model = Scrobble
extra = 0
raw_id_fields = (
"video",
"channel",
"podcast_episode",
"track",
"video_game",
"book",
"paper",
"sport_event",
"food",
"board_game",
"geo_location",
"task",
"puzzle",
"mood",
"brick_set",
"trail",
"beer",
"web_page",
"life_event",
"birding_location",
"user",
)
exclude = (
"scrobble_log",
"timezone",
"videogame_save_data",
"screenshot",
)
per_page = 15
ordering = ("-timestamp",)
show_change_link = True
fields = ("timestamp", "media_type", "source", "in_progress")
readonly_fields = fields
class ImportBaseAdmin(admin.ModelAdmin):
@ -121,7 +97,9 @@ class ScrobbleAdmin(admin.ModelAdmin):
"user",
)
raw_id_fields = (
"user",
"video",
"channel",
"podcast_episode",
"track",
"sport_event",
@ -140,6 +118,7 @@ class ScrobbleAdmin(admin.ModelAdmin):
"web_page",
"life_event",
"birding_location",
"long_play_last_scrobble",
)
list_filter = (
"is_paused",
@ -152,6 +131,7 @@ class ScrobbleAdmin(admin.ModelAdmin):
"user",
)
ordering = ("-timestamp",)
readonly_fields = ("share_token_version", "share_view_count")
def media_name(self, obj):
return obj.media_obj
@ -178,14 +158,19 @@ class FavoriteMediaAdmin(admin.ModelAdmin):
list_filter = ("media_type", "sent_to_mopidy", "user")
date_hierarchy = "created"
raw_id_fields = (
"user",
"video",
"channel",
"track",
"podcast_episode",
"sport_event",
"book",
"paper",
"video_game",
"board_game",
"geo_location",
"puzzle",
"food",
"task",
"mood",
"brick_set",

View File

@ -50,6 +50,18 @@ class BaseLogData(JSONDataclass):
def override_fields(cls) -> dict:
return {}
@classmethod
def from_log_dict(cls, log_dict: dict) -> dict:
"""Extract LogData keyword arguments from a stored log dict.
Override in subclasses to handle custom nesting/structure.
"""
return {
k: v
for k, v in log_dict.items()
if k in cls.__dataclass_fields__
}
def notes_as_str(self, separator: str = " | ") -> str:
import html
import re
@ -207,7 +219,7 @@ class BaseLogData(JSONDataclass):
@dataclass
class LongPlayLogData(JSONDataclass):
long_play_complete: bool = False
pass
@dataclass

View File

@ -93,7 +93,6 @@ class LastFM:
new_scrobbles.append(new_scrobble)
created = Scrobble.objects.bulk_create(new_scrobbles)
# TODO Add a notification for users that their import is complete
logger.info(
f"Last.fm import fnished",
extra={

View File

@ -0,0 +1,174 @@
from django.core.management.base import BaseCommand
from django.db import connection
from scrobbles.constants import LONG_PLAY_MEDIA
from scrobbles.models import Scrobble
BATCH_SIZE = 1000
class Command(BaseCommand):
help = (
"Backfill long_play_last_scrobble FK chains, then recompute "
"long_play_seconds by walking forward through scrobbles in "
"timestamp order with a running accumulator."
)
def add_arguments(self, parser):
parser.add_argument(
"--dry-run",
action="store_true",
help="Show what would be changed without making changes",
)
parser.add_argument(
"--media-type",
type=str,
help="Only process a specific media type (e.g., Book, VideoGame)",
)
def handle(self, *args, **options):
dry_run = options["dry_run"]
media_type = options.get("media_type")
media_types = list(LONG_PLAY_MEDIA.values())
if media_type:
if media_type not in media_types:
self.stdout.write(
self.style.ERROR(
f"Invalid media type '{media_type}'. "
f"Valid: {', '.join(media_types)}"
)
)
return
media_types = [media_type]
# Step 1: backfill long_play_last_scrobble
self.stdout.write("Step 1: Backfilling long_play_last_scrobble chains...")
total_backfilled = 0
for mt in media_types:
n = self._backfill_chain(mt, dry_run)
total_backfilled += n
self.stdout.write(f" {mt}: {n} scrobbles linked")
if dry_run:
self.stdout.write(
self.style.WARNING(
f"Would backfill {total_backfilled} scrobbles total. "
"Run without --dry-run to apply."
)
)
else:
self.stdout.write(
self.style.SUCCESS(f"Backfilled {total_backfilled} scrobbles")
)
# Step 2: recompute long_play_seconds
self.stdout.write(
"\nStep 2: Recomputing long_play_seconds in timestamp order..."
)
total_updated = 0
for mt in media_types:
n = self._recompute_for_media_type(mt, dry_run)
total_updated += n
self.stdout.write(f" {mt}: {n} scrobbles updated")
if dry_run:
self.stdout.write(
self.style.WARNING(
f"Dry run: would update {total_updated} scrobbles total. "
"Use without --dry-run to apply."
)
)
else:
self.stdout.write(
self.style.SUCCESS(f"Updated {total_updated} scrobbles")
)
def _recompute_for_media_type(self, media_type: str, dry_run: bool) -> int:
"""Process scrobbles for a single media type in timestamp order with a
running accumulator, avoiding O(n2) FK chain walks."""
fk = _media_type_to_fk(media_type)
fk_id = f"{fk}_id"
scrobbles = Scrobble.objects.filter(
media_type=media_type,
**{f"{fk}__isnull": False},
playback_position_seconds__isnull=False,
).order_by(fk_id, "user_id", "timestamp")
total = scrobbles.count()
if not total:
return 0
updated = 0
batch = []
last_key = None
running_total = 0
for scrobble in scrobbles.iterator():
key = (getattr(scrobble, fk_id), scrobble.user_id)
if key != last_key:
running_total = 0
last_key = key
running_total += scrobble.playback_position_seconds or 0
if scrobble.long_play_seconds != running_total:
updated += 1
if not dry_run:
scrobble.long_play_seconds = running_total
batch.append(scrobble)
if len(batch) >= BATCH_SIZE:
Scrobble.objects.bulk_update(batch, ["long_play_seconds"])
batch = []
if scrobble.long_play_complete:
running_total = 0
if batch:
Scrobble.objects.bulk_update(batch, ["long_play_seconds"])
return updated
def _backfill_chain(self, media_type: str, dry_run: bool) -> int:
"""Set long_play_last_scrobble on each scrobble to the previous
scrobble for the same media+user using a single UPDATE with a
correlated subquery."""
fk = _media_type_to_fk(media_type)
table = Scrobble._meta.db_table
if dry_run:
with connection.cursor() as cursor:
cursor.execute(
f"SELECT COUNT(*) FROM {table} "
f"WHERE long_play_last_scrobble_id IS NULL "
f"AND {fk}_id IS NOT NULL"
)
return cursor.fetchone()[0]
with connection.cursor() as cursor:
cursor.execute(
f"UPDATE {table} "
f"SET long_play_last_scrobble_id = ("
f" SELECT id FROM {table} AS prev "
f" WHERE prev.{fk}_id = {table}.{fk}_id "
f" AND prev.user_id = {table}.user_id "
f" AND prev.timestamp < {table}.timestamp "
f" ORDER BY prev.timestamp DESC LIMIT 1"
f") "
f"WHERE long_play_last_scrobble_id IS NULL "
f"AND {fk}_id IS NOT NULL"
)
return cursor.rowcount
def _media_type_to_fk(media_type):
mapping = {
"VideoGame": "video_game",
"Book": "book",
"BrickSet": "brick_set",
"Task": "task",
}
return mapping.get(media_type)

View File

@ -0,0 +1,19 @@
# Generated by Django 4.2.29 on 2026-06-12 16:09
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("scrobbles", "0096_convert_book_page_data_to_dict"),
]
operations = [
migrations.AddIndex(
model_name="scrobble",
index=models.Index(
fields=["user", "-timestamp"], name="scrobbles_s_user_id_d367a7_idx"
),
),
]

View File

@ -0,0 +1,25 @@
# Generated by Django 4.2.29 on 2026-06-15 17:48
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
("scrobbles", "0097_scrobble_scrobbles_s_user_id_d367a7_idx"),
]
operations = [
migrations.AddField(
model_name="scrobble",
name="long_play_last_scrobble",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="next_long_play_scrobbles",
to="scrobbles.scrobble",
),
),
]

View File

@ -165,23 +165,18 @@ class LongPlayScrobblableMixin(ScrobblableMixin):
return reverse("scrobbles:longplay-finish", kwargs={"uuid": self.uuid})
def first_long_play_scrobble_for_user(self, user) -> Optional["Scrobble"]:
return (
get_scrobbles_for_media(self, user)
.filter(
log__long_play_complete=False,
log__serial_scrobble_id__isnull=True,
)
.order_by("timestamp")
.first()
)
last = self.last_long_play_scrobble_for_user(user)
if not last:
return None
current = last
while current.long_play_last_scrobble and not current.long_play_last_scrobble.long_play_complete:
current = current.long_play_last_scrobble
return current
def last_long_play_scrobble_for_user(self, user) -> Optional["Scrobble"]:
return (
get_scrobbles_for_media(self, user)
.filter(
log__long_play_complete=False,
log__serial_scrobble_id__isnull=False,
)
.order_by("timestamp")
.last()
.filter(long_play_complete=False)
.order_by("-timestamp")
.first()
)

View File

@ -51,7 +51,10 @@ from scrobbles.constants import (
MEDIA_END_PADDING_SECONDS,
)
from scrobbles.importers.lastfm import LastFM
from scrobbles.notifications import ScrobbleNtfyNotification
from scrobbles.notifications import (
LastFmImportNtfyNotification,
ScrobbleNtfyNotification,
)
from scrobbles.utils import get_file_md5_hash, media_class_to_foreign_key
from sports.models import SportEvent
from taggit.managers import TaggableManager
@ -428,6 +431,8 @@ class LastFmImport(BaseFileImportMixin):
try:
scrobbles = lastfm.import_from_lastfm(last_processed, time_to=time_to)
self.record_log(scrobbles)
if scrobbles:
LastFmImportNtfyNotification(self, len(scrobbles)).send()
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
@ -612,6 +617,30 @@ class EBirdCSVImport(BaseFileImportMixin):
self.mark_finished()
TYPE_FK_PREFETCHES: dict[str, tuple[str, ...]] = {
"Video": ("video",),
"Track": ("track", "track__artist_fk"),
"PodcastEpisode": ("podcast_episode", "podcast_episode__podcast"),
"SportEvent": ("sport_event",),
"Book": ("book",),
"Paper": ("paper",),
"VideoGame": ("video_game",),
"BoardGame": ("board_game",),
"GeoLocation": ("geo_location",),
"Trail": ("trail",),
"Beer": ("beer",),
"Puzzle": ("puzzle",),
"Food": ("food",),
"Task": ("task",),
"WebPage": ("web_page",),
"LifeEvent": ("life_event",),
"Mood": ("mood",),
"BrickSet": ("brick_set",),
"Channel": ("channel",),
"BirdingLocation": ("birding_location",),
}
class ScrobbleQuerySet(models.QuerySet):
def with_related(self):
return self.select_related("user").prefetch_related(
@ -638,6 +667,13 @@ class ScrobbleQuerySet(models.QuerySet):
"birding_location",
)
def with_related_for_types(self, media_types: list[str]):
prefetches = []
for t in media_types:
if t in TYPE_FK_PREFETCHES:
prefetches.extend(TYPE_FK_PREFETCHES[t])
return self.select_related("user").prefetch_related(*prefetches)
class ShareViewLog(TimeStampedModel):
scrobble = models.ForeignKey(
@ -764,6 +800,12 @@ class Scrobble(TimeStampedModel):
)
long_play_seconds = models.BigIntegerField(**BNULL)
long_play_complete = models.BooleanField(**BNULL)
long_play_last_scrobble = models.ForeignKey(
"self",
**BNULL,
on_delete=models.SET_NULL,
related_name="next_long_play_scrobbles",
)
class Meta:
indexes = [
@ -778,6 +820,7 @@ class Scrobble(TimeStampedModel):
"is_paused",
]
),
models.Index(fields=["user", "-timestamp"]),
]
@classmethod
@ -810,11 +853,14 @@ class Scrobble(TimeStampedModel):
@classmethod
def as_dict_by_type(cls, scrobble_qs: models.QuerySet) -> dict:
scrobbles_by_type = defaultdict(list)
scrobbles = (
scrobble_qs.with_related()
if hasattr(scrobble_qs, "with_related")
else scrobble_qs
)
if hasattr(scrobble_qs, "with_related"):
media_types_present = list(
scrobble_qs.values_list("media_type", flat=True).distinct()
)
scrobbles = scrobble_qs.with_related_for_types(media_types_present)
else:
scrobbles = scrobble_qs
for scrobble in scrobbles:
scrobbles_by_type[scrobble.media_type].append(scrobble)
@ -829,7 +875,7 @@ class Scrobble(TimeStampedModel):
# Remove any locations without titles
if "GeoLocation" in scrobbles_by_type.keys():
for loc_scrobble in scrobbles_by_type["GeoLocation"]:
for loc_scrobble in list(scrobbles_by_type["GeoLocation"]):
if not loc_scrobble.media_obj.title:
scrobbles_by_type["GeoLocation"].remove(loc_scrobble)
scrobbles_by_type["GeoLocation_count"] -= 1
@ -911,27 +957,6 @@ class Scrobble(TimeStampedModel):
self.share_token_version += 1
self.save(update_fields=["share_token_version"])
def push_to_archivebox(self):
pushable_media = hasattr(self.media_obj, "push_to_archivebox") and callable(
self.media_obj.push_to_archivebox
)
if pushable_media and self.user.profile.archivebox_url:
try:
self.media_obj.push_to_archivebox(
url=self.user.profile.archivebox_url,
username=self.user.profile.archivebox_username,
password=self.user.profile.archivebox_password,
)
except Exception:
logger.info(
"Failed to push URL to archivebox",
extra={
"archivebox_url": self.user.profile.archivebox_url,
"archivebox_username": self.user.profile.archivebox_username,
},
)
@property
def logdata(self) -> Optional[logdata.BaseLogData]:
if self.media_obj:
@ -953,24 +978,8 @@ class Scrobble(TimeStampedModel):
if not log_dict:
log_dict = {}
# Special handling for GeoLocationLogData - data is nested under 'movement_detection'
# TODO there's a better way to fix this this at the LogData level
if logdata_cls.__name__ == "GeoLocationLogData":
instance_data = log_dict.get("movement_detection", {}).copy()
# Add top-level fields that GeoLocationLogData expects from BaseLogData/WithPeopleLogData
for field_name in ["description", "notes", "with_people_ids"]:
if field_name in log_dict:
instance_data[field_name] = log_dict[field_name]
try:
return logdata_cls(**instance_data)
except Exception as e:
logger.warning("Log data could not be loaded", e)
return logdata_cls()
# Strip log-only keys (stored in JSONField but not part of LogData dataclass)
logdata_kwargs = {
k: v for k, v in log_dict.items() if k in logdata_cls().__dataclass_fields__
}
# Use LogData's from_log_dict to handle any custom nesting/structure
logdata_kwargs = logdata_cls.from_log_dict(log_dict)
try:
return logdata_cls(**logdata_kwargs)
@ -1147,8 +1156,8 @@ class Scrobble(TimeStampedModel):
if self.is_long_play:
long_play_secs = 0
if self.previous and not self.previous.long_play_complete:
long_play_secs = self.previous.long_play_seconds or 0
if self.long_play_last_scrobble and not self.long_play_last_scrobble.long_play_complete:
long_play_secs = self.long_play_last_scrobble.long_play_seconds or 0
percent = int(((playback_seconds + long_play_secs) / run_time_secs) * 100)
return percent
@ -1463,7 +1472,25 @@ class Scrobble(TimeStampedModel):
if media.calories:
scrobble_data["log"] = FoodLogData(calories=media.calories).asdict
if mtype not in LONG_PLAY_MEDIA.values():
scrobble_data.pop("long_play_complete", None)
scrobble = cls.create(scrobble_data)
if mtype in LONG_PLAY_MEDIA.values():
last_finished = (
cls.objects.filter(
models.Q(**{key: media}),
user_id=user_id,
timestamp__lt=scrobble.timestamp,
)
.order_by("-timestamp")
.first()
)
if last_finished:
scrobble.long_play_last_scrobble = last_finished
scrobble.save(update_fields=["long_play_last_scrobble"])
return scrobble
@classmethod
@ -1722,8 +1749,8 @@ class Scrobble(TimeStampedModel):
# Set our playback seconds, and calc long play seconds
self.playback_position_seconds = seconds_elapsed
if self.previous:
past_seconds = self.previous.long_play_seconds or 0
if self.long_play_last_scrobble:
past_seconds = self.long_play_last_scrobble.long_play_seconds or 0
self.long_play_seconds = past_seconds + seconds_elapsed

View File

@ -79,6 +79,27 @@ class ScrobbleNtfyNotification(ScrobbleNotification):
)
class LastFmImportNtfyNotification(BasicNtfyNotification):
def __init__(self, lfm_import, scrobble_count):
super().__init__(lfm_import.user.profile)
self.ntfy_str = f"Imported {scrobble_count} scrobble(s) from Last.fm"
self.click_url = lfm_import.get_absolute_url()
self.title = "Last.fm Import Complete"
def send(self):
if self.profile and self.profile.ntfy_enabled and self.profile.ntfy_url:
requests.post(
self.profile.ntfy_url,
data=self.ntfy_str.encode(encoding="utf-8"),
headers={
"Title": self.title,
"Priority": "default",
"Tags": "musical_note",
"Click": self.click_url,
},
)
class MoodNtfyNotification(BasicNtfyNotification):
def __init__(self, profile, **kwargs):
super().__init__(profile)

View File

@ -32,6 +32,7 @@ from scrobbles.constants import (
)
from scrobbles.models import Scrobble
from scrobbles.notifications import ScrobbleNtfyNotification
from scrobbles.tasks import push_scrobble_to_archivebox
from scrobbles.utils import (
convert_to_seconds,
extract_domain,
@ -1028,8 +1029,7 @@ def manual_scrobble_webpage(
if action == "stop":
scrobble.stop(force_finish=True)
else:
# possibly async this?
scrobble.push_to_archivebox()
push_scrobble_to_archivebox.delay(scrobble.id)
return scrobble

View File

@ -252,6 +252,25 @@ def update_charts_for_timestamp(user_id, year, month, day, week):
logger.error(f"[charts] Failed to update charts: {e}")
@shared_task
def push_scrobble_to_archivebox(scrobble_id):
from scrobbles.models import Scrobble
scrobble = Scrobble.objects.filter(id=scrobble_id).first()
if not scrobble:
logger.warning(
"Scrobble %s not found for archivebox push", scrobble_id
)
return
webpage = scrobble.web_page
if not webpage:
logger.warning(
"Scrobble %s has no web_page for archivebox push", scrobble_id
)
return
webpage.push_to_archivebox(scrobble.user)
# ── Crontab replacements ──────────────────────────────────────────────────────

View File

@ -206,6 +206,19 @@ class ScrobbleableDetailView(ChartContextMixin, DetailView):
context_data["scrobbles"] = page_obj.object_list
context_data["is_paginated"] = paginator.num_pages > 1
media = self.object
if hasattr(media, "is_long_play_media") and media.is_long_play_media():
qs = media.scrobble_set.filter(user=self.request.user)
completed = qs.filter(long_play_complete=True).order_by("-timestamp").first()
if completed and completed.long_play_seconds:
context_data["long_play_total_seconds"] = completed.long_play_seconds
context_data["long_play_finished_date"] = completed.timestamp
else:
latest_finished = qs.filter(played_to_completion=True).order_by("-timestamp").first()
if latest_finished and latest_finished.long_play_seconds:
context_data["long_play_total_seconds"] = latest_finished.long_play_seconds
context_data["long_play_finished_date"] = None
return context_data
@ -909,6 +922,28 @@ def scrobble_longplay_finish(request, uuid):
if not user.is_authenticated:
return HttpResponseRedirect(success_url)
# Try scrobble UUID first
scrobble = Scrobble.objects.filter(uuid=uuid, user=user).first()
if scrobble:
if scrobble.long_play_complete == True:
scrobble.long_play_complete = None
scrobble.save(update_fields=["long_play_complete"])
messages.add_message(
request,
messages.INFO,
f"Long play of {scrobble.media_obj} marked as not complete.",
)
else:
scrobble.long_play_complete = True
scrobble.save(update_fields=["long_play_complete"])
messages.add_message(
request,
messages.SUCCESS,
f"Long play of {scrobble.media_obj} finished.",
)
return HttpResponseRedirect(success_url)
# Fall back to media UUID (existing behavior)
media_obj = None
for app, model in LONG_PLAY_MEDIA.items():
media_model = apps.get_model(app_label=app, model_name=model)
@ -917,10 +952,21 @@ def scrobble_longplay_finish(request, uuid):
break
if not media_obj:
return
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
)
return HttpResponseRedirect(success_url)
last_scrobble = media_obj.last_long_play_scrobble_for_user(user)
if last_scrobble and last_scrobble.long_play_complete == False:
if last_scrobble and last_scrobble.long_play_complete == True:
last_scrobble.long_play_complete = None
last_scrobble.save(update_fields=["long_play_complete"])
messages.add_message(
request,
messages.INFO,
f"Long play of {media_obj} marked as not complete.",
)
elif last_scrobble:
last_scrobble.long_play_complete = True
last_scrobble.save(update_fields=["long_play_complete"])
messages.add_message(
@ -1264,6 +1310,24 @@ class ScrobbleDetailView(DetailView):
else:
context["has_mopidy_uri"] = False
if self.object.is_long_play and fk_field:
all_scrobbles = Scrobble.objects.filter(
user=user, **{fk_field: media_obj}
)
completed = all_scrobbles.filter(
long_play_complete=True
).order_by("-timestamp").first()
if completed and completed.long_play_seconds:
context["long_play_total_seconds"] = completed.long_play_seconds
context["long_play_finished_date"] = completed.timestamp
else:
latest_finished = all_scrobbles.filter(
played_to_completion=True
).order_by("-timestamp").first()
if latest_finished and latest_finished.long_play_seconds:
context["long_play_total_seconds"] = latest_finished.long_play_seconds
context["long_play_finished_date"] = None
return context

View File

@ -23,6 +23,7 @@ class SportAdmin(admin.ModelAdmin):
class LeagueAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "abbreviation_str")
raw_id_fields = ("sport",)
ordering = ("name",)
@ -30,6 +31,7 @@ class LeagueAdmin(admin.ModelAdmin):
class PlayerAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "league", "team")
raw_id_fields = ("league", "team")
ordering = ("name",)
@ -37,6 +39,7 @@ class PlayerAdmin(admin.ModelAdmin):
class SeasonAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "league")
raw_id_fields = ("league",)
ordering = ("name",)
@ -44,6 +47,7 @@ class SeasonAdmin(admin.ModelAdmin):
class RoundAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "season")
raw_id_fields = ("season",)
ordering = ("name",)
@ -51,6 +55,7 @@ class RoundAdmin(admin.ModelAdmin):
class TeamAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("name", "league")
raw_id_fields = ("league",)
ordering = ("name",)
@ -63,6 +68,7 @@ class SportEventAdmin(admin.ModelAdmin):
"event_type",
"comp_str",
)
raw_id_fields = ("league", "teams", "players", "round")
list_filter = ("league", "event_type")
ordering = ("-created",)
inlines = [

View File

View File

@ -0,0 +1,10 @@
from django.contrib import admin
from trends.models import TrendResult
@admin.register(TrendResult)
class TrendResultAdmin(admin.ModelAdmin):
list_display = ("user", "trend_slug", "computed_at", "created")
list_filter = ("user", "trend_slug")
ordering = ("-computed_at",)

View File

@ -0,0 +1,6 @@
from django.apps import AppConfig
class TrendsConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "trends"

View File

@ -0,0 +1,40 @@
from django.contrib.auth import get_user_model
from django.core.management.base import BaseCommand
from trends.tasks import compute_user_trends
User = get_user_model()
class Command(BaseCommand):
help = "Compute trends for all users (or a specific user)"
def add_arguments(self, parser):
parser.add_argument(
"--user-id",
type=int,
help="Compute trends for a specific user only",
)
def handle(self, *args, **options):
if options["user_id"]:
user = User.objects.filter(id=options["user_id"]).first()
if not user:
self.stderr.write(
self.style.ERROR(f"User with id {options['user_id']} not found")
)
return
users = [user]
self.stdout.write(f"Computing trends for user: {user}")
else:
users = User.objects.filter(is_active=True)
self.stdout.write(f"Computing trends for {users.count()} users...")
for user in users:
try:
compute_user_trends(user.id)
self.stdout.write(self.style.SUCCESS(f" OK {user}"))
except Exception as e:
self.stderr.write(self.style.ERROR(f" FAILED {user}: {e}"))
self.stdout.write(self.style.SUCCESS("Done!"))

View File

@ -0,0 +1,57 @@
# Generated by Django 4.2.29 on 2026-06-16 14:52
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
import django_extensions.db.fields
class Migration(migrations.Migration):
initial = True
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name="TrendResult",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
django_extensions.db.fields.CreationDateTimeField(
auto_now_add=True, verbose_name="created"
),
),
(
"modified",
django_extensions.db.fields.ModificationDateTimeField(
auto_now=True, verbose_name="modified"
),
),
("trend_slug", models.CharField(db_index=True, max_length=100)),
("computed_at", models.DateTimeField(auto_now_add=True)),
("data", models.JSONField(default=dict)),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"unique_together": {("user", "trend_slug")},
},
),
]

View File

@ -0,0 +1,18 @@
from django.contrib.auth import get_user_model
from django.db import models
from django_extensions.db.models import TimeStampedModel
User = get_user_model()
class TrendResult(TimeStampedModel):
user = models.ForeignKey(User, on_delete=models.CASCADE)
trend_slug = models.CharField(max_length=100, db_index=True)
computed_at = models.DateTimeField(auto_now_add=True)
data = models.JSONField(default=dict)
class Meta:
unique_together = ["user", "trend_slug"]
def __str__(self):
return f"{self.user} - {self.trend_slug} ({self.computed_at})"

View File

@ -0,0 +1,39 @@
import logging
from celery import shared_task
from django.contrib.auth import get_user_model
from django.utils import timezone
from trends.models import TrendResult
from trends.trends import TREND_REGISTRY
logger = logging.getLogger(__name__)
User = get_user_model()
@shared_task
def compute_all_trends():
for user in User.objects.filter(is_active=True):
compute_user_trends.delay(user.id)
@shared_task
def compute_user_trends(user_id):
try:
user = User.objects.get(id=user_id)
except User.DoesNotExist:
logger.warning("User %s not found, skipping trends", user_id)
return
for slug, fn in TREND_REGISTRY.items():
try:
data = fn(user)
TrendResult.objects.update_or_create(
user=user,
trend_slug=slug,
defaults={"data": data, "computed_at": timezone.now()},
)
except Exception:
logger.exception(
"Failed to compute trend '%s' for user %s", slug, user_id
)

View File

@ -0,0 +1,89 @@
<div class="row">
{% if data.trails %}
<div class="col-md-6 mb-3">
<h4>🥾 While on Trails</h4>
{% for trail in data.trails %}
<div class="card mb-2">
<div class="card-body py-2">
<h6 class="card-title mb-1">
{% if trail.uuid %}
<a href="{% url 'trails:trail_detail' trail.uuid %}">{{ trail.name }}</a>
{% else %}
{{ trail.name }}
{% endif %}
<small class="text-muted">({{ trail.total_sessions }} sessions)</small>
</h6>
{% if trail.tracks %}
<table class="table table-sm table-borderless mb-0">
<thead>
<tr>
<th>Track</th>
<th>Artist</th>
<th class="text-end">Plays</th>
</tr>
</thead>
<tbody>
{% for t in trail.tracks %}
<tr>
<td>{% if t.track_uuid %}<a href="{% url 'music:track_detail' t.track_uuid %}">{{ t.track_name }}</a>{% else %}{{ t.track_name }}{% endif %}</td>
<td>{{ t.artist_name }}</td>
<td class="text-end">{{ t.count }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% endif %}
</div>
</div>
{% empty %}
<p class="text-muted">No concurrent listening data for trails.</p>
{% endfor %}
</div>
{% endif %}
{% if data.locations %}
<div class="col-md-6 mb-3">
<h4>📍 While at Locations</h4>
{% for loc in data.locations %}
<div class="card mb-2">
<div class="card-body py-2">
<h6 class="card-title mb-1">
{% if loc.uuid %}
<a href="{% url 'locations:geolocation_detail' loc.uuid %}">{{ loc.name }}</a>
{% else %}
{{ loc.name }}
{% endif %}
<small class="text-muted">({{ loc.total_sessions }} sessions)</small>
</h6>
{% if loc.tracks %}
<table class="table table-sm table-borderless mb-0">
<thead>
<tr>
<th>Track</th>
<th>Artist</th>
<th class="text-end">Plays</th>
</tr>
</thead>
<tbody>
{% for t in loc.tracks %}
<tr>
<td>{% if t.track_uuid %}<a href="{% url 'music:track_detail' t.track_uuid %}">{{ t.track_name }}</a>{% else %}{{ t.track_name }}{% endif %}</td>
<td>{{ t.artist_name }}</td>
<td class="text-end">{{ t.count }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% endif %}
</div>
</div>
{% empty %}
<p class="text-muted">No concurrent listening data for locations.</p>
{% endfor %}
</div>
{% endif %}
</div>
{% if not data.trails and not data.locations %}
<p class="text-muted">No concurrent listening data found.</p>
{% endif %}

View File

@ -0,0 +1,42 @@
<div class="row">
{% if data.books %}
{% for book in data.books %}
<div class="col-md-6 mb-3">
<div class="card">
<div class="card-body py-2">
<h6 class="card-title mb-1">
{% if book.book_uuid %}
<a href="{% url 'books:book_detail' book.book_uuid %}">{{ book.book_title }}</a>
{% else %}
{{ book.book_title }}
{% endif %}
<small class="text-muted">({{ book.total_sessions }} listening sessions)</small>
</h6>
{% if book.tracks %}
<table class="table table-sm table-borderless mb-0">
<thead>
<tr>
<th>Track</th>
<th>Artist</th>
<th class="text-end">Plays</th>
</tr>
</thead>
<tbody>
{% for t in book.tracks %}
<tr>
<td>{% if t.track_uuid %}<a href="{% url 'music:track_detail' t.track_uuid %}">{{ t.track_name }}</a>{% else %}{{ t.track_name }}{% endif %}</td>
<td>{{ t.artist_name }}</td>
<td class="text-end">{{ t.count }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% endif %}
</div>
</div>
</div>
{% endfor %}
{% else %}
<p class="text-muted">No concurrent reading data found.</p>
{% endif %}
</div>

View File

@ -0,0 +1,52 @@
<div class="row">
<div class="col-md-6 mb-3">
<div class="card">
<div class="card-body">
<h5 class="card-title">🎵 Reading with Music</h5>
{% if data.with_music %}
<table class="table table-sm mb-0">
<tr>
<th>Avg session duration</th>
<td>{{ data.with_music.avg_seconds }} seconds</td>
</tr>
<tr>
<th>Total reading time</th>
<td>{{ data.with_music.total_seconds }} seconds</td>
</tr>
<tr>
<th>Reading sessions</th>
<td>{{ data.with_music.sessions_count }}</td>
</tr>
</table>
{% else %}
<p class="text-muted mb-0">No data.</p>
{% endif %}
</div>
</div>
</div>
<div class="col-md-6 mb-3">
<div class="card">
<div class="card-body">
<h5 class="card-title">🔇 Reading without Music</h5>
{% if data.without_music %}
<table class="table table-sm mb-0">
<tr>
<th>Avg session duration</th>
<td>{{ data.without_music.avg_seconds }} seconds</td>
</tr>
<tr>
<th>Total reading time</th>
<td>{{ data.without_music.total_seconds }} seconds</td>
</tr>
<tr>
<th>Reading sessions</th>
<td>{{ data.without_music.sessions_count }}</td>
</tr>
</table>
{% else %}
<p class="text-muted mb-0">No data.</p>
{% endif %}
</div>
</div>
</div>
</div>

View File

@ -0,0 +1,38 @@
<div class="row">
<div class="col-12">
{% if data %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Media Type</th>
<th class="text-end">Recent (30 days)</th>
<th class="text-end">Previous (30 days)</th>
<th class="text-end">Change</th>
</tr>
</thead>
<tbody>
{% for mt, info in data.items %}
<tr>
<td>{{ mt }}</td>
<td class="text-end">{{ info.recent }}</td>
<td class="text-end">{{ info.previous }}</td>
<td class="text-end">
{% if info.change_pct > 0 %}
<span class="text-success">+{{ info.change_pct }}%</span>
{% elif info.change_pct < 0 %}
<span class="text-danger">{{ info.change_pct }}%</span>
{% else %}
<span class="text-muted">0%</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No trending data found.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,38 @@
{% extends "base_list.html" %}
{% block title %}{{ trend.title }}{% endblock %}
{% block lists %}
<div class="row mb-3">
<div class="col-12">
<a href="{% url 'trends:trends-home' %}" class="btn btn-sm btn-outline-secondary mb-2">&larr; All Trends</a>
<h2>{{ trend.icon }} {{ trend.title }}</h2>
<p class="text-muted">{{ trend.description }}</p>
{% if computed_at %}
<small class="text-muted">Last computed: {{ computed_at|date:"F j, Y H:i" }}</small>
{% endif %}
</div>
</div>
{% if trend_not_found %}
<div class="alert alert-warning">Trend not found.</div>
{% elif data is None %}
<div class="alert alert-info">
No data computed yet. Trends are updated once daily, check back later.
</div>
{% elif trend.slug == "concurrent-listening" %}
{% include "trends/_concurrent_listening.html" %}
{% elif trend.slug == "concurrent-reading" %}
{% include "trends/_concurrent_reading.html" %}
{% elif trend.slug == "reading-pace-vs-activity" %}
{% include "trends/_reading_pace.html" %}
{% elif trend.slug == "trending-up" %}
{% include "trends/_trending_up.html" %}
{% endif %}
{% endblock %}

View File

@ -0,0 +1,39 @@
{% extends "base_list.html" %}
{% block title %}Trends{% endblock %}
{% block lists %}
<div class="row">
{% if not user.is_authenticated %}
<div class="col-12">
<div class="alert alert-info">Log in to see your trends.</div>
</div>
{% elif not trends %}
<div class="col-12">
<div class="alert alert-info">
No trends computed yet. Trends are computed once daily, check back later.
</div>
</div>
{% else %}
{% for trend in trends %}
<div class="col-md-6 mb-3">
<div class="card h-100">
<div class="card-body">
<h5 class="card-title">
<a href="{% url 'trends:trend-detail' trend.slug %}" class="stretched-link text-decoration-none">
{{ trend.icon }} {{ trend.title }}
</a>
</h5>
<p class="card-text text-muted">{{ trend.description }}</p>
{% if trend.computed_at %}
<small class="text-muted">Last computed: {{ trend.computed_at|date:"M j, Y H:i" }}</small>
{% else %}
<span class="badge bg-warning text-dark">Pending</span>
{% endif %}
</div>
</div>
</div>
{% endfor %}
{% endif %}
</div>
{% endblock %}

View File

@ -0,0 +1,25 @@
from trends.trends.concurrent import compute_concurrent_listening, compute_concurrent_reading
from trends.trends.reading import compute_reading_pace_vs_activity
from trends.trends.trending import compute_trending_up
TREND_REGISTRY = {}
def register(slug):
def decorator(fn):
TREND_REGISTRY[slug] = fn
return fn
return decorator
compute_concurrent_listening = register("concurrent-listening")(
compute_concurrent_listening
)
compute_concurrent_reading = register("concurrent-reading")(
compute_concurrent_reading
)
compute_reading_pace_vs_activity = register("reading-pace-vs-activity")(
compute_reading_pace_vs_activity
)
compute_trending_up = register("trending-up")(
compute_trending_up
)

View File

@ -0,0 +1,222 @@
import datetime
from collections import defaultdict
from scrobbles.models import Scrobble
def _range_for(scrobble):
start = scrobble.timestamp
end = scrobble.stop_timestamp
if end is None:
try:
end = start + datetime.timedelta(hours=12)
except AttributeError:
end = start
return start, end
def _find_concurrent(anchor_scrobbles, paired_scrobbles):
"""Find paired scrobbles that overlap in time with anchor scrobbles.
Returns a dict mapping each anchor scrobble PK to a list of
paired scrobble PKs that overlap with it.
"""
anchor_ranges = {
s.pk: _range_for(s) for s in anchor_scrobbles
}
paired_ranges = {
s.pk: _range_for(s) for s in paired_scrobbles
}
anchor_to_paired = defaultdict(list)
for a_pk, (a_start, a_end) in anchor_ranges.items():
for p_pk, (p_start, p_end) in paired_ranges.items():
if a_start <= p_end and p_start <= a_end:
anchor_to_paired[a_pk].append(p_pk)
return anchor_to_paired
def _get_media_name(scrobble):
"""Return the name of the media object associated with a scrobble."""
for attr in [
"trail", "geo_location", "book", "track",
]:
obj = getattr(scrobble, attr, None)
if obj is not None:
return str(obj)
return "Unknown"
def compute_concurrent_listening(user):
"""Find what music was listened to while on trails or at locations.
Returns a dict with two keys: 'trails' and 'locations', each containing
a list of entries with the trail/location name and the tracks listened to.
"""
media_types_to_exclude_from_anchor = ("Track", "Book", "Video", "PodcastEpisode",
"VideoGame", "BoardGame", "Puzzle", "Food",
"Beer", "Task", "WebPage", "LifeEvent",
"Mood", "BrickSet", "Channel", "BirdingLocation",
"Paper", "SportEvent")
anchor_scrobbles = list(
Scrobble.objects.filter(
user=user,
timestamp__isnull=False,
played_to_completion=True,
)
.exclude(media_type__in=media_types_to_exclude_from_anchor)
.select_related("trail", "geo_location")
.order_by("-timestamp")
)
paired_scrobbles = list(
Scrobble.objects.filter(
user=user,
media_type="Track",
timestamp__isnull=False,
stop_timestamp__isnull=False,
played_to_completion=True,
)
.select_related("track")
.order_by("-timestamp")
)
if not anchor_scrobbles or not paired_scrobbles:
return {"trails": [], "locations": []}
anchor_to_paired = _find_concurrent(anchor_scrobbles, paired_scrobbles)
paired_by_pk = {s.pk: s for s in paired_scrobbles}
trails = []
locations = []
for anchor in anchor_scrobbles:
paired_pks = anchor_to_paired.get(anchor.pk, [])
if not paired_pks:
continue
tracks_by_name = defaultdict(int)
track_details = {}
for p_pk in paired_pks:
ps = paired_by_pk[p_pk]
track = ps.track
if track is None:
continue
name = str(track)
tracks_by_name[name] += 1
if name not in track_details:
track_details[name] = {
"track_name": name,
"track_uuid": str(track.uuid) if track.uuid else "",
"artist_name": str(track.artist) if track.artist else "",
}
anchor_name = _get_media_name(anchor)
entry = {
"name": anchor_name,
"uuid": "",
"total_sessions": len(paired_pks),
"tracks": sorted(
[
{**track_details[name], "count": count}
for name, count in tracks_by_name.items()
],
key=lambda x: x["count"],
reverse=True,
)[:20],
}
if anchor.media_type == "Trail":
entry["uuid"] = str(anchor.trail.uuid) if anchor.trail and anchor.trail.uuid else ""
trails.append(entry)
else:
entry["uuid"] = str(anchor.geo_location.uuid) if anchor.geo_location and anchor.geo_location.uuid else ""
locations.append(entry)
return {
"trails": sorted(trails, key=lambda x: x["total_sessions"], reverse=True)[:20],
"locations": sorted(locations, key=lambda x: x["total_sessions"], reverse=True)[:20],
}
def compute_concurrent_reading(user):
"""Find what music was listened to while reading books.
Returns a dict with key 'books' containing a list of entries with the
book title and the tracks listened to while reading.
"""
anchor_scrobbles = list(
Scrobble.objects.filter(
user=user,
media_type="Book",
timestamp__isnull=False,
stop_timestamp__isnull=False,
played_to_completion=True,
)
.select_related("book")
.order_by("-timestamp")
)
paired_scrobbles = list(
Scrobble.objects.filter(
user=user,
media_type="Track",
timestamp__isnull=False,
stop_timestamp__isnull=False,
played_to_completion=True,
)
.select_related("track")
.order_by("-timestamp")
)
if not anchor_scrobbles or not paired_scrobbles:
return {"books": []}
anchor_to_paired = _find_concurrent(anchor_scrobbles, paired_scrobbles)
paired_by_pk = {s.pk: s for s in paired_scrobbles}
books = []
for anchor in anchor_scrobbles:
paired_pks = anchor_to_paired.get(anchor.pk, [])
if not paired_pks:
continue
tracks_by_name = defaultdict(int)
track_details = {}
for p_pk in paired_pks:
ps = paired_by_pk[p_pk]
track = ps.track
if track is None:
continue
name = str(track)
tracks_by_name[name] += 1
if name not in track_details:
track_details[name] = {
"track_name": name,
"track_uuid": str(track.uuid) if track.uuid else "",
"artist_name": str(track.artist) if track.artist else "",
}
book = anchor.book
books.append({
"book_title": str(book) if book else "Unknown",
"book_uuid": str(book.uuid) if book and book.uuid else "",
"total_sessions": len(paired_pks),
"tracks": sorted(
[
{**track_details[name], "count": count}
for name, count in tracks_by_name.items()
],
key=lambda x: x["count"],
reverse=True,
)[:20],
})
return {
"books": sorted(books, key=lambda x: x["total_sessions"], reverse=True)[:20],
}

View File

@ -0,0 +1,86 @@
import datetime
from collections import defaultdict
from scrobbles.models import Scrobble
def compute_reading_pace_vs_activity(user):
"""Compare reading pace (seconds per session) when music is playing vs. not.
For each Book scrobble with a playback_position_seconds value, checks
whether there is an overlapping Track scrobble and groups the data.
Returns average session duration for both groups.
"""
book_scrobbles = list(
Scrobble.objects.filter(
user=user,
media_type="Book",
timestamp__isnull=False,
playback_position_seconds__isnull=False,
played_to_completion=True,
)
.select_related("book")
.order_by("-timestamp")
)
if not book_scrobbles:
return {"with_music": None, "without_music": None}
track_scrobbles = list(
Scrobble.objects.filter(
user=user,
media_type="Track",
timestamp__isnull=False,
played_to_completion=True,
)
.order_by("-timestamp")
)
track_ranges = []
for ts in track_scrobbles:
p_start = ts.timestamp
p_end = ts.stop_timestamp
if p_end is None:
try:
p_end = p_start + datetime.timedelta(hours=12)
except AttributeError:
p_end = p_start
track_ranges.append((p_start, p_end))
with_music_durations = []
without_music_durations = []
for bs in book_scrobbles:
b_start = bs.timestamp
b_end = bs.stop_timestamp
if b_end is None:
try:
b_end = b_start + datetime.timedelta(hours=12)
except AttributeError:
b_end = b_start
has_overlap = False
for p_start, p_end in track_ranges:
if b_start <= p_end and p_start <= b_end:
has_overlap = True
break
duration = bs.playback_position_seconds
if has_overlap:
with_music_durations.append(duration)
else:
without_music_durations.append(duration)
def _stats(durations):
if not durations:
return None
return {
"avg_seconds": int(sum(durations) / len(durations)),
"sessions_count": len(durations),
"total_seconds": sum(durations),
}
return {
"with_music": _stats(with_music_durations),
"without_music": _stats(without_music_durations),
}

View File

@ -0,0 +1,64 @@
from collections import defaultdict
from django.db.models import Count
from django.utils import timezone
from scrobbles.models import Scrobble
def compute_trending_up(user, days=30):
"""Compare scrobble counts per media type between two periods.
Compares the most recent N days against the N days before that,
returning the count for each period and the percentage change.
Returns a dict keyed by media_type with count and change info.
"""
now = timezone.now()
recent_start = now - timezone.timedelta(days=days)
previous_start = recent_start - timezone.timedelta(days=days)
recent_counts = defaultdict(int)
for row in (
Scrobble.objects.filter(
user=user,
timestamp__gte=recent_start,
timestamp__lte=now,
played_to_completion=True,
)
.values("media_type")
.annotate(count=Count("id"))
):
recent_counts[row["media_type"]] = row["count"]
previous_counts = defaultdict(int)
for row in (
Scrobble.objects.filter(
user=user,
timestamp__gte=previous_start,
timestamp__lt=recent_start,
played_to_completion=True,
)
.values("media_type")
.annotate(count=Count("id"))
):
previous_counts[row["media_type"]] = row["count"]
all_types = set(list(recent_counts.keys()) + list(previous_counts.keys()))
changes = {}
for mt in sorted(all_types):
rc = recent_counts.get(mt, 0)
pc = previous_counts.get(mt, 0)
if pc > 0:
change_pct = round(((rc - pc) / pc) * 100, 1)
elif rc > 0:
change_pct = 100.0
else:
change_pct = 0.0
changes[mt] = {
"recent": rc,
"previous": pc,
"change_pct": change_pct,
}
return changes

View File

@ -0,0 +1,10 @@
from django.urls import path
from trends.views import TrendDetailView, TrendListView
app_name = "trends"
urlpatterns = [
path("trends/", TrendListView.as_view(), name="trends-home"),
path("trends/<slug:trend_slug>/", TrendDetailView.as_view(), name="trend-detail"),
]

View File

@ -0,0 +1,89 @@
from django.contrib.auth.mixins import LoginRequiredMixin
from django.views.generic import TemplateView
from trends.models import TrendResult
from trends.trends import TREND_REGISTRY
TREND_METADATA = {
"concurrent-listening": {
"title": "Concurrent Listening",
"description": "What music were you listening to while on trails or at locations?",
"icon": "🎧",
},
"concurrent-reading": {
"title": "Concurrent Reading",
"description": "What music did you listen to while reading books?",
"icon": "📖",
},
"reading-pace-vs-activity": {
"title": "Reading Pace vs Music",
"description": "Compare how long you read per session with and without concurrent music.",
"icon": "📊",
},
"trending-up": {
"title": "Trending Media Types",
"description": "Which media types have you been consuming more or less of recently?",
"icon": "📈",
},
}
class TrendListView(LoginRequiredMixin, TemplateView):
template_name = "trends/trend_list.html"
def get_context_data(self, **kwargs):
ctx = super().get_context_data(**kwargs)
results = {
r.trend_slug: r
for r in TrendResult.objects.filter(
user=self.request.user
)
}
trends = []
for slug in TREND_REGISTRY:
meta = TREND_METADATA.get(slug, {})
result = results.get(slug)
trends.append({
"slug": slug,
"title": meta.get("title", slug),
"description": meta.get("description", ""),
"icon": meta.get("icon", ""),
"computed_at": result.computed_at if result else None,
"has_data": result is not None,
})
ctx["trends"] = trends
return ctx
class TrendDetailView(LoginRequiredMixin, TemplateView):
template_name = "trends/trend_detail.html"
def get_context_data(self, **kwargs):
ctx = super().get_context_data(**kwargs)
slug = kwargs["trend_slug"]
if slug not in TREND_REGISTRY:
ctx["trend_not_found"] = True
return ctx
meta = TREND_METADATA.get(slug, {})
ctx["trend"] = {
"slug": slug,
"title": meta.get("title", slug),
"description": meta.get("description", ""),
"icon": meta.get("icon", ""),
}
result = TrendResult.objects.filter(
user=self.request.user,
trend_slug=slug,
).first()
if result:
ctx["computed_at"] = result.computed_at
ctx["data"] = result.data
else:
ctx["computed_at"] = None
ctx["data"] = None
return ctx

View File

@ -26,6 +26,7 @@ class GameAdmin(admin.ModelAdmin):
"main_story_time",
"release_year",
)
raw_id_fields = ("platforms",)
search_fields = (
"title",
"alternative_name",

View File

@ -1,6 +1,6 @@
from django.contrib import admin
from webpages.models import Domain, WebPage
from webpages.models import Domain, HistoricalWebPage, WebPage
from scrobbles.admin import ScrobbleInline
@ -20,6 +20,12 @@ class DomainAdmin(admin.ModelAdmin):
inlines = [WebPageInline]
class HistoricalWebPageInline(admin.TabularInline):
model = HistoricalWebPage
extra = 0
readonly_fields = ("date", "domain", "extract", "created")
@admin.register(WebPage)
class WebPageAdmin(admin.ModelAdmin):
date_hierarchy = "created"
@ -33,4 +39,20 @@ class WebPageAdmin(admin.ModelAdmin):
search_fields = ("title",)
inlines = [
ScrobbleInline,
HistoricalWebPageInline,
]
@admin.register(HistoricalWebPage)
class HistoricalWebPageAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = (
"uuid",
"webpage",
"date",
"domain",
"created",
)
raw_id_fields = ("webpage", "domain")
ordering = ("-created",)
search_fields = ("webpage__title",)

View File

@ -0,0 +1,71 @@
# Generated by Django 4.2.29 on 2026-06-16 13:39
from django.db import migrations, models
import django.db.models.deletion
import django_extensions.db.fields
import uuid
class Migration(migrations.Migration):
dependencies = [
("webpages", "0009_alter_webpage_genre"),
]
operations = [
migrations.CreateModel(
name="HistoricalWebPage",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
django_extensions.db.fields.CreationDateTimeField(
auto_now_add=True, verbose_name="created"
),
),
(
"modified",
django_extensions.db.fields.ModificationDateTimeField(
auto_now=True, verbose_name="modified"
),
),
(
"uuid",
models.UUIDField(
blank=True, default=uuid.uuid4, editable=False, null=True
),
),
("date", models.DateField(blank=True, null=True)),
("extract", models.TextField(blank=True, null=True)),
(
"domain",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.DO_NOTHING,
to="webpages.domain",
),
),
(
"webpage",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="historical_webpages",
to="webpages.webpage",
),
),
],
options={
"get_latest_by": "modified",
"abstract": False,
},
),
]

View File

@ -17,6 +17,7 @@ from htmldate import find_date
from imagekit.models import ImageSpecField
from imagekit.processors import ResizeToFit
from scrobbles.mixins import ScrobblableConstants, ScrobblableMixin
from scrobbles.tasks import push_scrobble_to_archivebox
from taggit.managers import TaggableManager
logger = logging.getLogger(__name__)
@ -130,8 +131,7 @@ class WebPage(ScrobblableMixin):
},
)
scrobble = Scrobble.create_or_update(self, user_id, scrobble_data)
# TODO Possibly make this async?
scrobble.push_to_archivebox()
push_scrobble_to_archivebox.delay(scrobble.id)
return scrobble
def scrobbles(self, user):
@ -183,7 +183,13 @@ class WebPage(ScrobblableMixin):
if save:
self.save(update_fields=["date"])
def push_to_archivebox(self, url: str, username: str, password: str):
def push_to_archivebox(self, user):
profile = user.profile
url = profile.archivebox_url
if not url:
return
username = profile.archivebox_username
password = profile.archivebox_password
login_url = requests.compat.urljoin(url, "admin/login/")
session = requests.Session()
response = session.get(login_url)
@ -297,4 +303,41 @@ class WebPage(ScrobblableMixin):
if not webpage:
webpage = cls(url=data_dict.get("url"))
webpage.fetch_data_from_web(save=True)
else:
webpage._archive_and_refetch()
return webpage
def _archive_and_refetch(self):
"""Archive current content to HistoricalWebPage and re-fetch from web."""
if self.extract or self.date or self.domain:
HistoricalWebPage.objects.create(
webpage=self,
date=self.date,
domain=self.domain,
extract=self.extract,
)
self.extract = None
self.date = None
self.domain = None
self.title = None
self.base_run_time_seconds = None
self.image = None
self.fetch_data_from_web(save=True, force=True)
class HistoricalWebPage(TimeStampedModel):
uuid = models.UUIDField(default=uuid4, editable=False, **BNULL)
webpage = models.ForeignKey(
WebPage, on_delete=models.CASCADE, related_name="historical_webpages"
)
date = models.DateField(**BNULL)
domain = models.ForeignKey(Domain, on_delete=models.DO_NOTHING, **BNULL)
extract = models.TextField(**BNULL)
def __str__(self) -> str:
if self.webpage.title:
return "{} ({}) - {}".format(
self.webpage.title, self.webpage.domain, self.created
)
return "{} - {}".format(self.webpage.url, self.created)

View File

@ -134,6 +134,10 @@ CELERY_BEAT_SCHEDULE = {
"task": "scrobbles.tasks.rebuild_yearly_charts",
"schedule": crontab(hour=0, minute=30, day_of_month=1, month_of_year=1),
},
"compute-daily-trends": {
"task": "trends.tasks.compute_all_trends",
"schedule": crontab(hour=0, minute=10),
},
# ── Crontab replacements ─────────────────────────────────────────────
"database-backup": {
"task": "scrobbles.tasks.backup_database",
@ -192,6 +196,7 @@ INSTALLED_APPS = [
"scrobbles",
"people",
"charts",
"trends",
"videos",
"music",
"podcasts",

View File

@ -6,23 +6,29 @@
<tr>
<th scope="col">Latest</th>
<th scope="col">Title</th>
<th scope="col">Time</th>
<th scope="col">Scrobbles</th>
<th scope="col">Complete</th>
<th scope="col">Start</th>
<th scope="col">Finish</th>
</tr>
</thead>
<tbody>
{% for obj in object_list %}
{% if obj.title %}
{% with last=obj.scrobble_set.last %}
<tr>
<td><a href="{{obj.scrobble_set.last.get_absolute_url}}">{{obj.scrobble_set.last.local_timestamp}}
<td><a href="{{last.get_absolute_url}}">{{last.local_timestamp}}
<td><a href="{{obj.get_absolute_url}}">{{obj}}</a></td>
{% if request.user.is_authenticated %}
<td>{% if last.long_play_seconds %}{{ last.long_play_seconds|natural_duration }}{% elif last.elapsed_time %}{{ last.elapsed_time|natural_duration }}{% endif %}</td>
<td>{{obj.scrobble_count}}</td>
<td>{% if obj.scrobble_set.last.logdata.long_play_complete == True %}Yes{% endif %}</td>
<td>{% if obj.scrobble_set.last.long_play_complete == True %}Yes{% else %}No{% endif %}</td>
<td><a type="button" class="btn btn-sm btn-primary" href="{{obj.start_url}}">Scrobble</a></td>
<td><a type="button" class="btn btn-sm btn-warning" href="{{obj.get_longplay_finish_url}}">Finish</a></td>
{% endif %}
</tr>
{% endwith %}
{% endif %}
{% endfor %}
</tbody>

View File

@ -33,7 +33,11 @@
<p><a href="{{object.next_readcomics_url}}">Read next issue</a></p>
{% endif %}
<p>{{scrobbles.count}} scrobbles</p>
<p>
{{scrobbles.count}} scrobbles
{% if long_play_total_seconds %} | Total time: {{ long_play_total_seconds|natural_duration }}{% endif %}
{% if long_play_finished_date %} | Finished: {{ long_play_finished_date|date:"M d, Y" }}{% endif %}
</p>
<p><a href="{{object.resume_start_url}}">Resume reading</a></p>
</div>
@ -61,7 +65,7 @@
{% for scrobble in scrobbles.all|dictsortreversed:"timestamp" %}
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.logdata.long_play_complete == True %}Yes{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now reading{% else %}{{scrobble.session_pages_read}}{% endif %}</td>
<td>{% for author in scrobble.book.authors.all %}<a href="{{author.get_absolute_url}}">{{author}}</a>{% if not forloop.last %}, {% endif %}{% endfor %}</td>
</tr>

View File

@ -26,7 +26,11 @@
</div>
</div>
<div class="row">
<p>{{scrobbles.count}} scrobbles</p>
<p>
{{scrobbles.count}} scrobbles
{% if long_play_total_seconds %} | Total time: {{ long_play_total_seconds|natural_duration }}{% endif %}
{% if long_play_finished_date %} | Finished: {{ long_play_finished_date|date:"M d, Y" }}{% endif %}
</p>
</div>
<div class="row">
<div class="col-md">
@ -44,7 +48,7 @@
{% for scrobble in scrobbles.all|dictsortreversed:"timestamp" %}
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.logdata.long_play_complete == True %}Yes{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now reading{% else %}{{scrobble.session_pages_read}}{% endif %}</td>
</tr>
{% endfor %}

View File

@ -264,7 +264,13 @@
<h2 class="mt-4">All scrobbles of this {{ object.media_type|lower }}</h2>
{% if related_scrobbles %}
<p class="text-muted">{{ related_scrobbles.paginator.count }} scrobble{{ related_scrobbles.paginator.count|pluralize }}</p>
<p class="text-muted">
{{ related_scrobbles.paginator.count }} scrobble{{ related_scrobbles.paginator.count|pluralize }}
{% if object.is_long_play and long_play_total_seconds %}
| Total time: {{ long_play_total_seconds|natural_duration }}
{% if long_play_finished_date %} | Finished: {{ long_play_finished_date|date:"M d, Y" }}{% endif %}
{% endif %}
</p>
<table class="table table-striped table-sm">
<thead>
<tr>
@ -281,7 +287,11 @@
{% if scrobble.media_type == "Task" and scrobble.logdata.title %}{{ scrobble.media_obj.title }}: {{ scrobble.logdata.title }}{% else %}{{ scrobble.media_obj.title|default:scrobble.media_obj }}{% endif %}
</td>
<td>
{% if scrobble.playback_position_seconds %}{{ scrobble.playback_position_seconds|natural_duration }}{% endif %}
{% if scrobble.is_long_play and scrobble.long_play_seconds %}
{{ scrobble.playback_position_seconds|natural_duration }} ({{ scrobble.long_play_seconds|natural_duration }} total)
{% elif scrobble.playback_position_seconds %}
{{ scrobble.playback_position_seconds|natural_duration }}
{% endif %}
</td>
</tr>
{% endfor %}

View File

@ -50,7 +50,11 @@
{% endif %}
<div class="row">
<p>{{scrobbles.count}} scrobbles</p>
<p>
{{scrobbles.count}} scrobbles
{% if long_play_total_seconds %} | Total time: {{ long_play_total_seconds|natural_duration }}{% endif %}
{% if long_play_finished_date %} | Finished: {{ long_play_finished_date|date:"M d, Y" }}{% endif %}
</p>
<p>
<a href="{{object.start_url}}">Play again</a>
</p>
@ -67,6 +71,7 @@
<th scope="col">Title</th>
<th scope="col">Notes</th>
<th scope="col">Source</th>
<th scope="col">Completed</th>
</tr>
</thead>
<tbody>
@ -76,6 +81,7 @@
<td><a href="{{scrobble.get_media_source_url}}">{{scrobble.logdata.title}}</a></td>
<td>{{scrobble.logdata.notes_as_str}}</td>
<td>{{scrobble.source}}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">No</a>{% endif %}</td>
</tr>
{% endfor %}
</tbody>

View File

@ -26,7 +26,7 @@
{% block lists %}
<div class="row">
<div class="col-md">
<div class="table-responsive">{% include "_scrobblable_list.html" %}</div>
<div class="table-responsive">{% include "_longplay_scrobblable_list.html" %}</div>
</div>
</div>
{% endblock %}

View File

@ -59,7 +59,11 @@
</div>
</div>
<div class="row">
<p>{{scrobbles.count}} scrobbles</p>
<p>
{{scrobbles.count}} scrobbles
{% if long_play_total_seconds %} | Total time: {{ long_play_total_seconds|natural_duration }}{% endif %}
{% if long_play_finished_date %} | Finished: {{ long_play_finished_date|date:"M d, Y" }}{% endif %}
</p>
<p>
<a href="">Play again</a>
</p>
@ -83,7 +87,7 @@
<tr>
<td><a href="{{scrobble.get_absolute_url}}">{{scrobble.local_timestamp}}</a></td>
<td>{% if scrobble.long_play_complete == True %}Yes{% else %}Not yet{% endif %}</td>
<td>{% if scrobble.long_play_complete == True %}Yes <small><a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">(not complete?)</a></small>{% else %}<a href="{% url 'scrobbles:longplay-finish' uuid=scrobble.uuid %}">Not yet</a>{% endif %}</td>
<td>{% if scrobble.in_progress %}Now playing{% else %}{{scrobble.playback_position_seconds|natural_duration}}{% endif %}</td>
<td>{% for platform in scrobble.video_game.platforms.all %}<a href="{{platform.get_absolute_url}}">{{platform}}</a>{% if not forloop.last %}, {% endif %}{% endfor %}</td>
<td>{% if scrobble.videogame_save_data %}<a href="{{scrobble.videogame_save_data.url}}">Save data</a>{% else %}Not yet{% endif %}</td>

View File

@ -16,7 +16,7 @@
<div class="col-md">
<div class="table-responsive">
{% include "_scrobblable_list.html" %}
{% include "_longplay_scrobblable_list.html" %}
</div>
</div>
</div>

View File

@ -105,6 +105,7 @@ from vrobbler.apps.webpages.api.views import DomainViewSet, WebPageViewSet
from vrobbler.apps.people import urls as people_urls
from vrobbler.apps.charts import urls as charts_urls
from vrobbler.apps.trends import urls as trends_urls
# from vrobbler.apps.modern_ui import urls as modern_ui_urls
@ -182,6 +183,7 @@ urlpatterns = [
path("", include(profiles_urls, namespace="profiles")),
path("", include(people_urls, namespace="people")),
path("", include(charts_urls, namespace="charts")),
path("", include(trends_urls, namespace="trends")),
path("", scrobbles_views.RecentScrobbleList.as_view(), name="vrobbler-home"),
]
if settings.DEBUG: