Compare commits

...

54 Commits
54.2 ... 58.2

Author SHA1 Message Date
5e0dffdc7a [release] Bump to version 58.2
Some checks failed
build / test (push) Successful in 2m5s
deploy / test (push) Successful in 2m3s
deploy / build-and-deploy (push) Failing after 25s
- Add more robust webpage scraping
- Time of Day Categories trend
2026-06-23 23:04:48 -04:00
2283a6c640 [webpages] Add more robust scraping 2026-06-23 23:04:30 -04:00
327ba94c63 [trends] Add new time of day trend
All checks were successful
build / test (push) Successful in 2m4s
2026-06-23 22:21:18 -04:00
ee59cde882 [release] Bump to version 58.1
All checks were successful
build / test (push) Successful in 2m7s
deploy / test (push) Successful in 2m8s
deploy / build-and-deploy (push) Successful in 1m5s
- Add auto genre tagging for papers
2026-06-23 16:19:06 -04:00
c7b4656679 [papers] Add genre tagging 2026-06-23 16:18:50 -04:00
04f9e00c9c [release] Bump to version 58.0
All checks were successful
deploy / test (push) Successful in 2m7s
build / test (push) Successful in 2m3s
deploy / build-and-deploy (push) Successful in 1m2s
- Add scrobbling of Papers via webpages with doi.org links in them
2026-06-23 14:28:40 -04:00
c2dabd1dac [papers] Fix scrobbling of academic papers 2026-06-23 14:28:16 -04:00
7a0cb8b9d0 [release] Bump to version 57.1
Some checks failed
build / test (push) Successful in 2m3s
deploy / test (push) Successful in 2m5s
deploy / build-and-deploy (push) Failing after 2m18s
- Write poetry lock file
2026-06-21 23:06:14 -04:00
1c2c570c4b [deps] Lock poetry 2026-06-21 23:05:58 -04:00
0671ab432f [release] Bump to version 57.0
Some checks failed
build / test (push) Failing after 35s
deploy / test (push) Failing after 37s
deploy / build-and-deploy (push) Has been skipped
- Scrobble button on some media list pages dont work
- Use HTMx to update the Now Playing widget
- Add a live page that updates the scrobble list via JS polling
- Turns out we cant cache the now playing widget
- What would it look like to add an MCP server to expose scrobbles and media items?
2026-06-21 23:04:23 -04:00
893867419a [templates] Shorten up naturalduration rep 2026-06-21 23:04:01 -04:00
d9dfec81aa [scrobbles] Fix bug where media list scrobble btns didnt work 2026-06-21 23:00:28 -04:00
948fbc19bf [templates] Add HTMX support to Now Playing 2026-06-21 23:00:09 -04:00
7d708ad8a6 [templates] Add polling live page for all scrobbles
Some checks failed
build / test (push) Failing after 35s
2026-06-21 22:39:46 -04:00
e0505cb82c [templates] Fix caching issue with Now Playing 2026-06-21 22:36:58 -04:00
ab6459e4b0 [mcp] Add a basic mcp service
Some checks failed
build / test (push) Failing after 31s
2026-06-21 01:26:16 -04:00
c001248d1b [release] Bump to version 56.4
All checks were successful
build / test (push) Successful in 2m0s
deploy / test (push) Successful in 1m59s
deploy / build-and-deploy (push) Successful in 49s
- Add ability to do reverse address lookup on lat-long pairs
- Add address fields to GeoLocation
- Add better detail template for Disc Golf Courses
2026-06-21 01:04:57 -04:00
f1c777d4ef [discgolf] Add trail maps and addresses
Some checks failed
build / test (push) Has been cancelled
2026-06-21 01:02:43 -04:00
931488c288 [release] Bump to version 56.3
Some checks failed
build / test (push) Has been cancelled
deploy / test (push) Successful in 1m58s
deploy / build-and-deploy (push) Successful in 32s
- Fix bug in importer script from discgolf being added
2026-06-20 01:12:33 -04:00
ab897fd848 [importers] Just fix a smol bug 2026-06-20 01:12:14 -04:00
4f189b4d66 [release] Bump to version 56.2
Some checks failed
build / test (push) Has been cancelled
deploy / build-and-deploy (push) Has been cancelled
deploy / test (push) Has been cancelled
- Fix bug in creating people when importing course plays
2026-06-20 01:10:53 -04:00
1487504318 [discgolf] Fix bug in creating people 2026-06-20 01:10:34 -04:00
0655363a0d [release] Bump to version 56.1
All checks were successful
build / test (push) Successful in 2m4s
deploy / test (push) Successful in 2m0s
deploy / build-and-deploy (push) Successful in 49s
- Add tests to discgolf app
2026-06-20 00:55:35 -04:00
dccc80c615 [discgolf] Fix tests and naming scheme
Some checks failed
build / test (push) Has been cancelled
2026-06-20 00:55:16 -04:00
4f91d5b40b [tests] Fix broken birding tests 2026-06-20 00:42:46 -04:00
cb01781615 [release] Bump to version 56.0
Some checks failed
build / test (push) Failing after 1m33s
deploy / test (push) Failing after 1m35s
deploy / build-and-deploy (push) Has been skipped
- Add DiscGolf as a scrobbleable media
2026-06-20 00:37:42 -04:00
1f5fada8b1 [discgolf] Add new scrobble type
Some checks failed
build / test (push) Has been cancelled
2026-06-20 00:37:18 -04:00
31888a85cb [release] Bump to version 55.6
All checks were successful
build / test (push) Successful in 1m56s
deploy / test (push) Successful in 1m54s
deploy / build-and-deploy (push) Successful in 30s
- Figure out why historical Lastfm imports don't work
2026-06-19 14:12:44 -04:00
22d8b0787e [importers] Allow starting full import for new user
Some checks failed
build / test (push) Has been cancelled
2026-06-19 14:12:09 -04:00
8cc559752b [release] Bump to version 55.5
All checks were successful
build / test (push) Successful in 2m5s
deploy / test (push) Successful in 3m12s
deploy / build-and-deploy (push) Successful in 31s
- Fix bug in lastfm import for new users
2026-06-19 14:06:10 -04:00
db3f9696fa [importers] Fix smol bug in lastfm importer
Some checks failed
build / test (push) Has been cancelled
2026-06-19 14:05:46 -04:00
407d570c82 [release] Bump to version 55.4
All checks were successful
build / test (push) Successful in 2m12s
deploy / test (push) Successful in 2m4s
deploy / build-and-deploy (push) Successful in 1m22s
- Tighten up the speed of startup and first request
2026-06-19 13:41:23 -04:00
033239260f [perf] Add caching and lock protections
All checks were successful
build / test (push) Successful in 1m57s
2026-06-19 13:37:30 -04:00
9f854dc735 [release] Bump to version 55.3
All checks were successful
build / test (push) Successful in 2m6s
deploy / test (push) Successful in 2m3s
deploy / build-and-deploy (push) Successful in 39s
- =alt_names= feature for artists (commented out / dead code)
- Put chart rebuilds in a lower priority task queue
- Check for existing book scrobble and update page count
2026-06-19 01:14:39 -04:00
f29272a853 [music] Clean up dead code
Some checks failed
build / test (push) Has been cancelled
2026-06-19 01:13:28 -04:00
4e56d9420a [settings] Put chart rebuilds in their own queue 2026-06-19 01:12:50 -04:00
852a257159 [scrobbles] Clean up a TODO already done
All checks were successful
build / test (push) Successful in 1m59s
2026-06-18 16:59:03 -04:00
68ff230f13 [release] Bump to version 55.2
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 2m0s
deploy / build-and-deploy (push) Successful in 30s
- Fix bug in scrobble id in calendar view
- Video game cleanup script should clear out broken images
2026-06-18 15:27:29 -04:00
57a952a6d1 [templates] Fix bug in calendar view
Some checks failed
build / test (push) Has been cancelled
2026-06-18 15:27:09 -04:00
718fcf7392 [videogames] Fix broken images in cleanup
All checks were successful
build / test (push) Successful in 2m0s
2026-06-18 15:24:47 -04:00
52adcf83c7 [release] Bump to version 55.1
All checks were successful
build / test (push) Successful in 1m57s
deploy / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 30s
- Clean up metadata scrapping for video games
2026-06-18 15:08:46 -04:00
0061623f7e [videogames] Fix metadata scrapping for video games
Some checks failed
build / test (push) Has been cancelled
2026-06-18 15:08:23 -04:00
ec73e5151e [release] Bump to version 55.0
All checks were successful
build / test (push) Successful in 1m58s
deploy / test (push) Successful in 1m57s
deploy / build-and-deploy (push) Successful in 37s
- Use pk ID for scrobble detail view, not uuid
- Display videogame screenshots on scrobble detail if they exist
- Add autotagging to webpages based on domain, title
2026-06-18 12:15:16 -04:00
2c90dd38b5 [project] Add todos 2026-06-18 12:14:58 -04:00
c6b1e42d7a [scrobbles] Use IDs not UUIDs in URLs
All checks were successful
build / test (push) Successful in 1m59s
2026-06-18 11:25:57 -04:00
fcf86d5b3f [scrobbles] Add screenshots to templates
All checks were successful
build / test (push) Successful in 2m6s
2026-06-18 10:54:28 -04:00
6fde9ec8d2 [webpages] Add autotagging to webpages
All checks were successful
build / test (push) Successful in 1m59s
2026-06-18 10:43:21 -04:00
0f1882b21f [release] Bump to version 54.5
All checks were successful
build / test (push) Successful in 2m7s
deploy / build-and-deploy (push) Has been skipped
deploy / test (push) Successful in 1m58s
- Fix bug in generating mood trends
2026-06-17 21:46:21 -04:00
e819a2db0d [trends] Fix bug in mood trend generation 2026-06-17 21:45:45 -04:00
e03cf6c9b1 [release] Bump to version 54.4
All checks were successful
build / test (push) Successful in 2m0s
deploy / test (push) Successful in 1m58s
deploy / build-and-deploy (push) Successful in 29s
- Remove all-time trends
- Add a trend around moods
2026-06-17 17:09:30 -04:00
471e70ff7f [trends] Remove all time trends
Some checks failed
build / test (push) Has been cancelled
2026-06-17 17:09:11 -04:00
255e335d7a [trends] Add some mood related trends 2026-06-17 17:07:16 -04:00
c8cf80b513 [release] Bump to version 54.3
All checks were successful
deploy / test (push) Successful in 2m1s
build / test (push) Successful in 1m52s
deploy / build-and-deploy (push) Successful in 29s
- Fix bug in series metadata cleanup script
2026-06-17 12:05:50 -04:00
b4180afbed [videos] Fix metadata for series script
Some checks failed
build / test (push) Has been cancelled
2026-06-17 12:05:22 -04:00
121 changed files with 6372 additions and 478 deletions

View File

@ -88,7 +88,7 @@ fetching and simple saving.
*** Metadata sources
**** Scraper
* Backlog [0/20] :vrobbler:project:personal:
* Backlog [0/22] :vrobbler:project:personal:
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :bug:music:scrobbles:
:PROPERTIES:
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
@ -579,6 +579,18 @@ named constants for maintainability.
- ~vrobbler/apps/scrobbles/importers/tsv.py~ (line 55) -- ="S"= completion status
** TODO [#A] Deduplicate BGG plays before posting :boardgames:bgg:duplication:
:PROPERTIES:
:ID: e9b842bf-0049-42e7-a060-f3ebd0067d2f
:END:
*** Description
No check for existing BGG plays before posting, which can create duplicates.
Should look up past plays by =bggeek_id= first.
File: ~vrobbler/apps/boardgames/bgg.py~ (line 117)
** TODO [#C] Clean up naming of =bgsplay= parsing :importers:refactoring:
:PROPERTIES:
:ID: c751dbbc-464a-4e63-9fe3-e034303f7b54
@ -590,6 +602,254 @@ We should rename `email_scrobble_board_game` to reflect the fact that it's just
a helper method to create board game scrobbles given a json blob. It's
independent of the email flow it was originally creatdd for
** TODO [#B] Is there way to create unique slugs for media instances :media_types:
* Version 58.2 [2/2]
** DONE [#B] Add more robust webpage scraping :webpages:metadata:
:PROPERTIES:
:ID: 84d9bfa5-75c0-0718-764e-379f7456602a
:END:
** DONE [#B] Time of Day Categories trend :trends:
:PROPERTIES:
:ID: 6598074f-2290-46db-967b-29f45d30be29
:END:
*** Description
Added a "Time of Day Categories" trend that groups scrobbles for Books, Trails,
Birding Locations, and Board Games into Early Bird (5-10:59am), Day Jay (11am-6:59pm),
and Night Owl (7pm-4:59am) buckets. Shows both overall and per-media-type breakdowns.
* Version 58.1 [1/1]
** DONE [#B] Add auto genre tagging for papers :books:papers:metadata:
:PROPERTIES:
:ID: e6b5c3a5-7fc6-b530-96c2-b5962a716db6
:END:
* Version 58.0 [1/1]
** DONE [#B] Add scrobbling of Papers via webpages with doi.org links in them :feature:papers:
:PROPERTIES:
:ID: d30bb8aa-eefd-002c-38d5-3f2fcef345f2
:END:
* Version 57.1 [1/1]
** DONE [#A] Write poetry lock file :bug:deps:
:PROPERTIES:
:ID: 0f5a6f4b-a486-ba7e-bbce-f7581274398c
:END:
* Version 57.0 [5/5]
** DONE [#A] Scrobble button on some media list pages dont work :bug:scrobbles:
:PROPERTIES:
:ID: a3a5c707-2e3d-a6b1-0f7f-4c6f7433aa1f
:END:
** DONE [#B] Use HTMx to update the Now Playing widget :feature:templates:
:PROPERTIES:
:ID: 5f5631fc-9ee1-d5a5-d0f8-94fea6fbbfa4
:END:
** DONE [#B] Add a live page that updates the scrobble list via JS polling :feature:templates:
:PROPERTIES:
:ID: 58790d76-dc6e-8aa5-2dc0-e64fe786fbf1
:END:
** DONE [#A] Turns out we cant cache the now playing widget :bug:templates:
:PROPERTIES:
:ID: 9ce669ea-c000-cdfe-a634-ad5cdaeae81c
:END:
** DONE [#C] What would it look like to add an MCP server to expose scrobbles and media items? :mcpserver:feature:
:PROPERTIES:
:ID: c5fca159-c7e0-5795-7c05-bbc48f539650
:END:
* Version 56.4 [3/3]
** DONE [#B] Add ability to do reverse address lookup on lat-long pairs :geolocations:feature:
:PROPERTIES:
:ID: 86c071ff-7638-41ba-6b65-1382df1cb5aa
:END:
** DONE [#B] Add address fields to GeoLocation :addresses:geolocation:
:PROPERTIES:
:ID: a55ae508-07ab-ccdd-e453-846bd3fca6fb
:END:
** DONE [#B] Add better detail template for Disc Golf Courses :discgolf:templates:
:PROPERTIES:
:ID: 12cee67c-f723-0fa3-848a-cbc6e4d65fc3
:END:
* Version 56.3 [1/1]
** DONE [#A] Fix bug in importer script from discgolf being added :bug:
:PROPERTIES:
:ID: c3733f96-18f1-eef8-f5d9-edaf97e35623
:END:
* Version 56.2 [1/1]
** DONE [#A] Fix bug in creating people when importing course plays :discgolf:bug:
:PROPERTIES:
:ID: 255e9886-098b-39ae-1077-25e43223660e
:END:
* Version 56.1 [1/1]
** DONE [#A] Add tests to discgolf app :discgolf:tests:
:PROPERTIES:
:ID: 28e8344e-c3cf-19af-ce1c-cb821d4fcb5f
:END:
* Version 56.0 [1/1]
** DONE [#B] Add DiscGolf as a scrobbleable media :discgolf:
:PROPERTIES:
:ID: 8cdde5d3-0ae5-7d5a-99d2-200c86afae03
:END:
*** Description
I have a csv file fro the uDisc disc golf scoring app that looks like:
Singles round. Note second row is the par for the course
#+begin_src csv
PlayerName,CourseName,LayoutName,StartDate,EndDate,Total,+/-,RoundRating,Hole1,Hole2,Hole3,Hole4,Hole5,Hole6,Hole7,Hole8,Hole9
Par,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,27,,,3,3,3,3,3,3,3,3,3
Colin Powell,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,30,3,,2,3,4,4,3,4,3,3,4
Asa Sewell,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,44,17,,5,4,4,8,5,5,4,4,5
Emma Sweet,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,41,14,,5,4,5,6,3,4,3,5,6
Jane Sewell,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,44,17,,4,5,5,5,5,5,4,6,5
Nabby Sewell,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,59,32,,6,6,7,7,6,7,6,6,8
Silas Sewell,DR Front 9,Custom Layout,2026-06-19 1535-0400,2026-06-19 1725-0400,41,14,,5,5,4,5,3,5,4,4,6`
#+end_src
Teams of two or more persons. Note second row is the par for the course
#+begin_src csv
PlayerName,CourseName,LayoutName,StartDate,EndDate,Total,+/-,RoundRating,Hole1,Hole2,Hole3,Hole4,Hole5,Hole6,Hole7,Hole8,Hole9
Par,Peninsula Links,Main,2026-06-19 2322-0400,2026-06-19 2323-0400,27,,,3,3,3,3,3,3,3,3,3
Colin Powell + Asa Sewell,Peninsula Links,Main,2026-06-19 2322-0400,2026-06-19 2323-0400,29,2,,3,4,2,3,2,3,5,3,4
Emma Sweet + Jane Sewell,Peninsula Links,Main,2026-06-19 2322-0400,2026-06-19 2323-0400,28,1,,4,3,4,2,3,4,3,3,2
#+end_src
We should add a new app called discgolf that has the following data models:
- DiscGolfRound - scrobblable media + course_id, round_type (Singles, Teams)
- DiscGolfCourse - name, layoutname, number_of_holes
And the logdata for a DiscGolfOuting scrobble should have:
+ {person: {hole_number: score}, total: int}
+ {team: {name: "", people: [person, person], hole_number: score}, total: int}
+ weather
+ fun_factor (miserable, not great, so-so, good, excellent, party time)
* Version 55.6 [1/1]
** DONE [#A] Figure out why historical Lastfm imports don't work :importers:lastfm:music:
:PROPERTIES:
:ID: 71b18c1b-de96-6d93-20fa-de2ec0df1288
:END:
* Version 55.5 [1/1]
** DONE [#B] Fix bug in lastfm import for new users :importers:lastfm:music:
:PROPERTIES:
:ID: d034966d-0c7f-e512-4cf8-7329c9026b6f
:END:
* Version 55.4 [1/1]
** DONE [#A] Tighten up the speed of startup and first request :perf:
:PROPERTIES:
:ID: 9ee8834c-6be2-d04b-df6d-56375504083f
:END:
* Version 55.3 [3/3]
** DONE [#C] =alt_names= feature for artists (commented out / dead code) :music:dead-code:
:PROPERTIES:
:ID: e22060a2-5f7a-4f33-9056-309ecd27159c
:END:
*** Description
File: ~vrobbler/apps/music/models.py~ (line 236)
An entire block of code for tracking alternate artist names is commented
out. The TODO questions whether it even works. Review: either implement
properly or remove the dead code.
** DONE [#A] Put chart rebuilds in a lower priority task queue :charts:tasks:
:PROPERTIES:
:ID: 43c90de0-fc1c-1139-dac7-9b7c82006b2e
:END:
** DONE [#A] Check for existing book scrobble and update page count :books:scrobbling:
:PROPERTIES:
:ID: 1a0609bc-6b16-4da4-96c1-59588229e4b4
:END:
*** Description
File: ~vrobbler/apps/scrobbles/scrobblers.py~ (line 330)
When scrobbling a book (comic), the code doesn't check for prior scrobbles to
update reading progress. Needed for proper page-count tracking.
* Version 55.2 [2/2]
** DONE [#A] Fix bug in scrobble id in calendar view :templates:
:PROPERTIES:
:ID: 8cb34852-b18f-e794-cd9b-fb1ecad70a0d
:END:
** DONE [#A] Video game cleanup script should clear out broken images :metadata:videogames:
:PROPERTIES:
:ID: ca1f1ea9-0f79-082c-5ff7-867671faff4b
:END:
* Version 55.1 [1/1]
** DONE [#A] Clean up metadata scrapping for video games :metadata:videogames:
:PROPERTIES:
:ID: fbc421b5-21a3-4aed-9062-c59192ead065
:END:
* Version 55.0 [3/3]
** DONE [#B] Use pk ID for scrobble detail view, not uuid :scrobbles:
:PROPERTIES:
:ID: 9cc3b285-e478-041e-394b-3d550aefbe1d
:END:
** DONE [#B] Display videogame screenshots on scrobble detail if they exist :videogames:templates:
:PROPERTIES:
:ID: 0406d082-20f6-0d12-76e2-f281c4801468
:END:
** DONE [#B] Add autotagging to webpages based on domain, title :webpages:metadata:
:PROPERTIES:
:ID: f658435b-f7a0-42e6-b9f6-226678a77a55
:END:
*** Description
For easier filtering, like we do with tasks, we should auto tag WebPage instances
based on the domain name split part by periods (so news.ycombinator.com tags: news, ycombinator, com)
And also based on the nouns in the title.
* Version 54.5 [1/1]
** DONE Fix bug in generating mood trends :trends:
:PROPERTIES:
:ID: 8e75abfa-8e70-d85b-00a4-a4813bbce879
:END:
* Version 54.4 [2/2]
** DONE [#A] Remove all-time trends :trends:
:PROPERTIES:
:ID: 53b231d1-7677-8cd3-1d88-dae110aba1e6
:END:
*** Description
All time trends take forever to calculate and don't provide too much data
** DONE [#B] Add a trend around moods :moods:trends:
:PROPERTIES:
:ID: fba3f4ae-8f97-ee0b-e762-31630884518a
:END:
* Version 54.3 [1/1]
** DONE [#B] Fix bug in series metadata cleanup script :videos:metadta:
:PROPERTIES:
:ID: 85448702-907c-5d63-f5af-7795661d7c46
:END:
* Version 54.2 [4/4]
** DONE [#B] Add script to clean up TV series metadata :videos:metadata:
:PROPERTIES:

View File

@ -1,2 +1,2 @@
web: python manage.py runserver 0.0.0.0:8014
worker: celery -A vrobbler worker -l DEBUG
worker: celery -A vrobbler worker -Q default,charts -l DEBUG

835
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "vrobbler"
version = "54.2"
version = "58.2"
description = ""
authors = ["Colin Powell <colin@unbl.ink>"]
@ -9,8 +9,10 @@ python = ">=3.11,<3.15"
Django = "^4.0.3"
django-extensions = "^3.1.5"
python-dateutil = "^2.8.2"
python-dotenv = "^0.20.0"
python-dotenv = ">=0.20.0,<2"
python-json-logger = "^2.0.2"
cloudscraper = "^1.2.71"
curl-cffi = "^0.15.0"
colorlog = "^6.6.0"
httpx = "<=0.27.2"
djangorestframework = "^3.13.1"
@ -43,6 +45,7 @@ ipython = "^8.14.0"
pendulum = "^3"
trafilatura = "^1.6.3"
django-imagekit = "^5.0.0"
django-mcp-server = "^0.5.7"
thefuzz = "^0.22.1"
dataclass-wizard = "^0.35.0"
webdavclient3 = "^3.14.6"
@ -65,6 +68,7 @@ lxml = ">=5.5.0"
vaderSentiment = "^3.3.2"
sqids = "^0.5.2"
python-amazon-paapi = "^6.3.0"
yake = "^0.7.3"
[tool.poetry.group.test]
optional = true

View File

View File

@ -0,0 +1,63 @@
import tempfile
import pytest
from django.contrib.auth import get_user_model
User = get_user_model()
@pytest.fixture
def user(db):
return User.objects.create(email="golfer@example.com")
@pytest.fixture
def udisc_singles_csv_content():
return """PlayerName,CourseName,LayoutName,StartDate,Hole1,Hole2,Hole3,Total
Par,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",3,3,3,9
Alice,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",4,2,3,9
Bob,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",3,4,5,12
"""
@pytest.fixture
def udisc_singles_csv_file(udisc_singles_csv_content):
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8-sig"
) as f:
f.write(udisc_singles_csv_content)
return f.name
@pytest.fixture
def udisc_teams_csv_content():
return """PlayerName,CourseName,LayoutName,StartDate,Hole1,Hole2,Hole3,Total
Par,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",3,3,3,9
Alice+Bob,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",4,2,3,9
Charlie+Diana,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",3,4,5,12
"""
@pytest.fixture
def udisc_teams_csv_file(udisc_teams_csv_content):
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8-sig"
) as f:
f.write(udisc_teams_csv_content)
return f.name
@pytest.fixture
def udisc_csv_no_par_content():
return """PlayerName,CourseName,LayoutName,StartDate,Hole1,Hole2,Hole3,Total
Alice,Maple Hill,Mountains,"Jun 15, 2026 10:00 AM",4,2,3,9
"""
@pytest.fixture
def udisc_csv_no_par_file(udisc_csv_no_par_content):
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8-sig"
) as f:
f.write(udisc_csv_no_par_content)
return f.name

View File

@ -0,0 +1,102 @@
from discgolf.models import DiscGolfCourse, DiscGolfLogData
from scrobbles.dataclasses import BaseLogData
class TestDiscGolfCourseModel:
def test_create_course(self, db):
course = DiscGolfCourse.objects.create(
title="Maple Hill",
layout_name="Mountains",
number_of_holes=18,
par_total=54,
par_per_hole={"hole_1": 3, "hole_2": 3},
)
assert course.uuid is not None
assert str(course) == "Maple Hill (Mountains)"
assert course.subtitle == "Mountains"
def test_subtitle_fallback(self, db):
course = DiscGolfCourse.objects.create(title="Maple Hill")
assert course.subtitle == ""
def test_logdata_cls(self, db):
course = DiscGolfCourse.objects.create(title="Maple Hill")
assert course.logdata_cls is DiscGolfLogData
assert issubclass(course.logdata_cls, BaseLogData)
def test_strings(self, db):
course = DiscGolfCourse.objects.create(title="Maple Hill")
assert course.strings.verb == "Playing"
assert course.strings.tags == "golf"
def test_primary_image_url(self, db):
course = DiscGolfCourse.objects.create(title="Maple Hill")
assert course.primary_image_url == ""
def test_get_absolute_url(self, db):
course = DiscGolfCourse.objects.create(title="Maple Hill")
url = course.get_absolute_url()
assert str(course.uuid) in url
assert url.startswith("/disc-golf/")
def test_find_or_create_new(self, db):
course = DiscGolfCourse.find_or_create(
"New Course", layout_name="Default"
)
assert course.title == "New Course"
assert course.layout_name == "Default"
def test_find_or_create_existing(self, db):
created = DiscGolfCourse.objects.create(
title="Existing", layout_name="Alpha"
)
found = DiscGolfCourse.find_or_create("Existing", layout_name="Beta")
assert found.id == created.id
assert found.layout_name == "Alpha"
def test_scrobbles_method(self, db, user):
from datetime import datetime
import pytz
from scrobbles.models import Scrobble
course = DiscGolfCourse.objects.create(title="Maple Hill")
dt1 = datetime(2026, 6, 15, 14, 0, 0, tzinfo=pytz.UTC)
dt2 = datetime(2026, 6, 14, 14, 0, 0, tzinfo=pytz.UTC)
s1 = Scrobble.objects.create(
user=user,
disc_golf_course=course,
media_type=Scrobble.MediaType.DISC_GOLF,
timestamp=dt1,
)
s2 = Scrobble.objects.create(
user=user,
disc_golf_course=course,
media_type=Scrobble.MediaType.DISC_GOLF,
timestamp=dt2,
)
qs = course.scrobbles(user.id)
assert list(qs) == [s1, s2]
class TestDiscGolfLogData:
def test_basic_logdata(self):
data = DiscGolfLogData()
assert data.scores is None
assert data.weather is None
assert data.fun_factor is None
assert data.course_name is None
def test_logdata_with_scores(self):
data = DiscGolfLogData(
scores={"Alice": {"person_id": 1, "total": 9}},
weather="Sunny",
fun_factor="High",
course_name="Maple Hill",
par=9,
round_type="Singles",
)
assert data.scores["Alice"]["total"] == 9
assert data.weather == "Sunny"
assert data.round_type == "Singles"

View File

@ -0,0 +1,150 @@
from unittest.mock import patch
from discgolf.models import DiscGolfCourse
from discgolf.utils import _parse_udisc_datetime, import_udisc_csv
from people.models import Person
from scrobbles.models import Scrobble
class TestParserHelpers:
def test_parse_udisc_datetime(self):
dt = _parse_udisc_datetime("Jun 15, 2026 10:00 AM")
assert dt is not None
assert dt.year == 2026
assert dt.month == 6
assert dt.day == 15
assert dt.hour == 10
assert dt.minute == 0
def test_parse_udisc_datetime_date_only(self):
dt = _parse_udisc_datetime("Jun 15, 2026")
assert dt is not None
assert dt.year == 2026
class TestImportUdiscCSV:
def test_import_singles_creates_course(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
course = DiscGolfCourse.objects.filter(title="Maple Hill").first()
assert course is not None
assert course.layout_name == "Mountains"
assert course.number_of_holes == 3
assert course.par_total == 9
assert course.par_per_hole == {"hole_1": 3, "hole_2": 3, "hole_3": 3}
def test_import_singles_creates_scrobble(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
assert Scrobble.objects.filter(source="uDisc").count() == 1
def test_import_singles_logdata(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
scrobble = Scrobble.objects.filter(source="uDisc").first()
log = scrobble.log
assert log["course_name"] == "Maple Hill"
assert log["par"] == 9
assert log["round_type"] == "Singles"
assert "Alice" in log["scores"]
assert "Bob" in log["scores"]
assert log["scores"]["Alice"]["total"] == 9
assert log["scores"]["Bob"]["total"] == 12
def test_import_singles_creates_people(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
assert Person.objects.filter(name="Alice").exists()
assert Person.objects.filter(name="Bob").exists()
def test_import_teams_creates_scrobble(self, user, udisc_teams_csv_file):
import_udisc_csv(udisc_teams_csv_file, user.id)
assert Scrobble.objects.filter(source="uDisc").count() == 1
def test_import_teams_logdata(self, user, udisc_teams_csv_file):
import_udisc_csv(udisc_teams_csv_file, user.id)
scrobble = Scrobble.objects.filter(source="uDisc").first()
assert scrobble.log["round_type"] == "Teams"
alice_bob = scrobble.log["scores"]["Alice+Bob"]
assert "person_ids" in alice_bob
assert len(alice_bob["person_ids"]) == 2
def test_import_creates_team_people(self, user, udisc_teams_csv_file):
import_udisc_csv(udisc_teams_csv_file, user.id)
assert Person.objects.filter(name="Alice").exists()
assert Person.objects.filter(name="Bob").exists()
assert Person.objects.filter(name="Charlie").exists()
assert Person.objects.filter(name="Diana").exists()
def test_import_teams_par_per_hole(self, user, udisc_teams_csv_file):
import_udisc_csv(udisc_teams_csv_file, user.id)
course = DiscGolfCourse.objects.get(title="Maple Hill")
assert course.par_per_hole == {"hole_1": 3, "hole_2": 3, "hole_3": 3}
def test_import_no_par_returns_empty(self, user, udisc_csv_no_par_file):
result = import_udisc_csv(udisc_csv_no_par_file, user.id)
assert result == []
def test_import_empty_csv(self, user, db):
import tempfile
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8-sig"
) as f:
f.write("PlayerName,CourseName,LayoutName,StartDate,Hole1,Total\n")
path = f.name
result = import_udisc_csv(path, user.id)
assert result == []
def test_import_idempotent(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
import_udisc_csv(udisc_singles_csv_file, user.id)
assert DiscGolfCourse.objects.filter(title="Maple Hill").count() == 1
assert Scrobble.objects.filter(source="uDisc").count() == 2
def test_import_course_defaults_only_on_create(
self, user, udisc_singles_csv_file
):
import_udisc_csv(udisc_singles_csv_file, user.id)
course = DiscGolfCourse.objects.get(title="Maple Hill")
assert course.layout_name == "Mountains"
course.layout_name = "Updated"
course.save()
import_udisc_csv(udisc_singles_csv_file, user.id)
course.refresh_from_db()
assert course.layout_name == "Updated"
@patch("discgolf.utils.ScrobbleNtfyNotification")
def test_import_sends_notification(self, mock_notification_class, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
mock_notification_class.assert_called_once()
mock_notification_class.return_value.send.assert_called_once()
def test_import_hole_scores_per_player(self, user, udisc_singles_csv_file):
import_udisc_csv(udisc_singles_csv_file, user.id)
scrobble = Scrobble.objects.filter(source="uDisc").first()
alice = scrobble.log["scores"]["Alice"]
assert alice["hole_1"] == 4
assert alice["hole_2"] == 2
assert alice["hole_3"] == 3
bob = scrobble.log["scores"]["Bob"]
assert bob["hole_1"] == 3
assert bob["hole_2"] == 4
assert bob["hole_3"] == 5
def test_import_record_error_on_bad_data(self, user, db):
import tempfile
content = """PlayerName,CourseName,LayoutName,StartDate,Hole1,Hole2,Hole3,Total
Par,,Mountains,"Jun 15, 2026 10:00 AM",3,3,3,9
"""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".csv", delete=False, encoding="utf-8-sig"
) as f:
f.write(content)
path = f.name
errors = []
result = import_udisc_csv(path, user.id, record_error=errors.append)
assert len(result) == 1
course = DiscGolfCourse.objects.first()
assert course.title == ""

View File

@ -0,0 +1,58 @@
from datetime import datetime
import pytz
from django.contrib.auth import get_user_model
from django.test import Client
from django.urls import reverse
from discgolf.models import DiscGolfCourse
from scrobbles.models import Scrobble
User = get_user_model()
class TestDiscGolfCourseViews:
def _make_scrobble(self, user, course):
dt = datetime(2026, 6, 15, 14, 0, 0, tzinfo=pytz.UTC)
return Scrobble.objects.create(
user=user,
disc_golf_course=course,
media_type=Scrobble.MediaType.DISC_GOLF,
timestamp=dt,
)
def test_course_list_anonymous(self, db, user):
course = DiscGolfCourse.objects.create(title="Maple Hill")
self._make_scrobble(user, course)
client = Client()
response = client.get(reverse("discgolf:course_list"))
assert response.status_code == 200
def test_course_list_shows_course(self, db, user):
course = DiscGolfCourse.objects.create(title="Maple Hill")
self._make_scrobble(user, course)
client = Client()
response = client.get(reverse("discgolf:course_list"))
assert response.status_code == 200
assert "Maple Hill" in response.content.decode()
def test_course_detail_anonymous(self, db, user):
course = DiscGolfCourse.objects.create(title="Maple Hill")
self._make_scrobble(user, course)
client = Client()
response = client.get(
reverse("discgolf:course_detail", kwargs={"slug": course.uuid})
)
assert response.status_code == 200
def test_course_detail_shows_course(self, db, user):
course = DiscGolfCourse.objects.create(
title="Maple Hill", layout_name="Mountains"
)
self._make_scrobble(user, course)
client = Client()
response = client.get(
reverse("discgolf:course_detail", kwargs={"slug": course.uuid})
)
assert response.status_code == 200
assert "Maple Hill" in response.content.decode()

View File

@ -519,7 +519,7 @@ def test_scrobble_detail_view_with_notes_as_flat_list(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
assert "First note" in response.content.decode()
@ -545,7 +545,7 @@ def test_scrobble_detail_view_with_notes_as_dict_timestamps(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()
@ -574,7 +574,7 @@ def test_scrobble_detail_view_with_notes_and_labels(client):
"description": "Test description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()
@ -597,7 +597,7 @@ def test_scrobble_detail_view_post_updates_log(client):
"description": "Original description",
},
)
url = reverse("scrobbles:detail", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
client.force_login(user)
response = client.post(
@ -896,7 +896,7 @@ def test_change_visibility_owner_can_change(client):
)
client.force_login(user)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 302
@ -919,7 +919,7 @@ def test_change_visibility_non_owner_gets_404(client):
)
client.force_login(other)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 404
@ -937,7 +937,7 @@ def test_change_visibility_anonymous_redirects_to_login(client):
task=task, media_type="Task", user=user, visibility="private",
timestamp=timezone.now(),
)
url = reverse("scrobbles:change-visibility", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:change-visibility", kwargs={"pk": scrobble.id})
response = client.post(url, {"visibility": "shared"})
assert response.status_code == 302
assert "/login/" in response.url
@ -956,7 +956,7 @@ def test_regenerate_share_token_invalidates_old_sqid(client):
old_sqid = encode_scrobble_share(scrobble.id, scrobble.share_token_version)
client.force_login(user)
url = reverse("scrobbles:regenerate-share-token", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:regenerate-share-token", kwargs={"pk": scrobble.id})
response = client.post(url)
assert response.status_code == 302
@ -985,7 +985,7 @@ def test_share_analytics_owner_can_view(client):
)
client.force_login(user)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
@ -1005,7 +1005,7 @@ def test_share_analytics_non_owner_gets_404(client):
)
client.force_login(other)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 404
@ -1027,7 +1027,7 @@ def test_share_analytics_shows_view_logs(client):
client.get(share_url)
client.force_login(user)
url = reverse("scrobbles:share-analytics", kwargs={"uuid": scrobble.uuid})
url = reverse("scrobbles:share-analytics", kwargs={"pk": scrobble.id})
response = client.get(url)
assert response.status_code == 200
content = response.content.decode()

View File

@ -4,9 +4,15 @@ from unittest.mock import MagicMock, patch
import pytest
from vrobbler import context_processors
from vrobbler.context_processors import version_info
@pytest.fixture(autouse=True)
def reset_git_cache():
context_processors._GIT_COMMIT = None
@pytest.fixture
def mock_request():
return MagicMock()

View File

@ -1,8 +1,19 @@
from books.models import Author, Book, Paper
from books.models import Author, Book, Journal, Paper
from django.contrib import admin
from scrobbles.admin import ScrobbleInline
@admin.register(Journal)
class JournalAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = (
"title",
"website_url",
)
search_fields = ("title",)
ordering = ("-created",)
@admin.register(Author)
class AuthorAdmin(admin.ModelAdmin):
date_hierarchy = "created"

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.29 on 2026-06-23 14:45
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("books", "0037_book_volume_book_volume_comicvine_id"),
]
operations = [
migrations.AddField(
model_name="paper",
name="pdf_file",
field=models.FileField(blank=True, null=True, upload_to="papers/pdf/"),
),
]

View File

@ -0,0 +1,92 @@
from django.db import migrations, models
import django.db.models.deletion
import django_extensions.db.fields
import uuid
def migrate_journal_data(apps, schema_editor):
Paper = apps.get_model("books", "Paper")
Journal = apps.get_model("books", "Journal")
for paper in Paper.objects.all():
old_journal = getattr(paper, "journal", None)
if old_journal:
journal, _ = Journal.objects.get_or_create(title=str(old_journal))
paper._journal_tmp = journal
paper.save(update_fields=["_journal_tmp"])
def reverse_migrate_journal_data(apps, schema_editor):
Paper = apps.get_model("books", "Paper")
for paper in Paper.objects.all():
if paper._journal_tmp:
paper.journal = paper._journal_tmp.title
paper.save(update_fields=["journal"])
class Migration(migrations.Migration):
dependencies = [
("books", "0038_paper_pdf_file"),
]
operations = [
migrations.CreateModel(
name="Journal",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
django_extensions.db.fields.CreationDateTimeField(
auto_now_add=True, verbose_name="created"
),
),
(
"modified",
django_extensions.db.fields.ModificationDateTimeField(
auto_now=True, verbose_name="modified"
),
),
(
"uuid",
models.UUIDField(
blank=True, default=uuid.uuid4, editable=False, null=True
),
),
("title", models.CharField(max_length=255)),
("description", models.TextField(blank=True, null=True)),
("website_url", models.URLField(blank=True, max_length=500, null=True)),
],
options={
"get_latest_by": "modified",
"abstract": False,
},
),
migrations.AddField(
model_name="paper",
name="_journal_tmp",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.DO_NOTHING,
to="books.journal",
),
),
migrations.RunPython(migrate_journal_data, reverse_migrate_journal_data),
migrations.RemoveField(
model_name="paper",
name="journal",
),
migrations.RenameField(
model_name="paper",
old_name="_journal_tmp",
new_name="journal",
),
]

View File

@ -27,7 +27,11 @@ from books.sources.amazon import lookup_book_from_amazon
from books.sources.openlibrary import (
lookup_book_from_openlibrary as lookup_book_from_ol,
)
from books.sources.semantic import lookup_paper_from_semantic
from books.sources.semantic import (
lookup_paper_from_semantic,
lookup_paper_from_semantic_by_doi,
)
from books.sources.scihub import SciHubService
from books.utils import get_comic_issue_url
from django.conf import settings
from django.contrib.auth import get_user_model
@ -82,6 +86,16 @@ class BookLogData(BaseLogData, LongPlayLogData):
return int(total_duration / len(self.page_data))
class Journal(TimeStampedModel):
uuid = models.UUIDField(default=uuid4, editable=False, **BNULL)
title = models.CharField(max_length=255)
description = models.TextField(**BNULL)
website_url = models.URLField(max_length=500, **BNULL)
def __str__(self):
return self.title
class Author(TimeStampedModel):
name = models.CharField(max_length=255)
uuid = models.UUIDField(default=uuid4, editable=False, **BNULL)
@ -225,7 +239,7 @@ class Book(LongPlayScrobblableMixin):
@property
def resume_start_url(self):
return reverse("scrobbles:start", kwargs={"uuid": self.uuid}) + "?resume=1"
return reverse("scrobbles:start", kwargs={"media_uuid": self.uuid}) + "?resume=1"
@classmethod
def get_from_comicvine(
@ -540,6 +554,21 @@ class Book(LongPlayScrobblableMixin):
return progress
@dataclass
class PaperLogData(BaseLogData):
@classmethod
def override_fields(cls) -> dict:
from scrobbles.forms import NotesDictField
fields = {}
for base in cls.mro()[1:]:
if hasattr(base, "override_fields"):
base_fields = base.override_fields()
fields.update(base_fields)
fields["notes"] = NotesDictField(required=False)
return fields
class Paper(LongPlayScrobblableMixin):
"""Keeps track of Academic Papers"""
@ -559,14 +588,29 @@ class Paper(LongPlayScrobblableMixin):
language = models.CharField(max_length=4, **BNULL)
first_publish_year = models.IntegerField(**BNULL)
publish_date = models.DateField(**BNULL)
journal = models.CharField(max_length=255, **BNULL)
journal = models.ForeignKey(Journal, on_delete=models.DO_NOTHING, **BNULL)
journal_volume = models.CharField(max_length=50, **BNULL)
abstract = models.TextField(**BNULL)
tldr = models.CharField(max_length=255, **BNULL)
openaccess_pdf_url = models.CharField(max_length=255, **BNULL)
pdf_file = models.FileField(upload_to="papers/pdf/", **BNULL)
genre = TaggableManager(through=ObjectWithGenres, blank=True, verbose_name="Genre")
@property
def logdata_cls(self):
return PaperLogData
@property
def scihub_url(self):
if not self.doi_id:
return None
domain = getattr(settings, "SCIHUB_DOMAIN", "sci-hub.st")
return f"https://{domain}/{self.doi_id}"
def get_absolute_url(self):
return reverse("books:paper_detail", kwargs={"slug": self.uuid})
@classmethod
def get_from_semantic(cls, title: str, overwrite: bool = False) -> "Paper":
paper, created = cls.objects.get_or_create(title=title)
@ -577,7 +621,7 @@ class Paper(LongPlayScrobblableMixin):
if created or overwrite:
author_list = []
author_dicts = paper_dict.pop("author_dicts")
author_dicts = paper_dict.pop("author_dicts", None)
if author_dicts:
for author_dict in author_dicts:
if author_dict.get("authorId"):
@ -588,8 +632,11 @@ class Paper(LongPlayScrobblableMixin):
if a_created:
author.name = author_dict.get("name")
author.save()
# TODO enrich author?
...
journal_name = paper_dict.pop("journal_name", None)
if journal_name:
journal, _ = Journal.objects.get_or_create(title=journal_name)
paper.journal = journal
for k, v in paper_dict.items():
setattr(paper, k, v)
@ -601,3 +648,78 @@ class Paper(LongPlayScrobblableMixin):
if genres:
paper.genre.add(*genres)
return paper
@classmethod
def find_or_create_by_doi(cls, doi_url: str) -> "Paper":
doi = doi_url.replace("https://doi.org/", "").split("?")[0].rstrip("/")
paper = cls.objects.filter(doi_id=doi).first()
if paper:
return paper
paper = cls(doi_id=doi, title=f"Paper {doi}")
paper.save()
from books.sources.crossref import lookup_paper_from_crossref
paper_dict = lookup_paper_from_semantic_by_doi(doi)
if not paper_dict or not paper_dict.get("abstract"):
paper_dict = lookup_paper_from_crossref(doi)
if paper_dict:
author_list = []
author_dicts = paper_dict.pop("author_dicts", None)
if author_dicts:
for author_dict in author_dicts:
author_id = author_dict.get("authorId")
if author_id:
author, a_created = Author.objects.get_or_create(
semantic_id=author_id
)
author_list.append(author)
if a_created:
author.name = author_dict.get("name")
author.save()
else:
author_name = author_dict.get("name")
if author_name:
author, a_created = Author.objects.get_or_create(
name=author_name
)
author_list.append(author)
journal_name = paper_dict.pop("journal_name", None)
if journal_name:
journal, _ = Journal.objects.get_or_create(title=journal_name)
paper.journal = journal
for k, v in paper_dict.items():
if v is not None:
setattr(paper, k, v)
paper.save()
if author_list:
paper.authors.add(*author_list)
genres = paper_dict.pop("genres", [])
if genres:
paper.genre.add(*genres)
if not paper.pdf_file:
service = SciHubService()
if paper.openaccess_pdf_url:
pdf_content = service.fetch_from_url(paper.openaccess_pdf_url)
if pdf_content:
filename = f"{doi.replace('/', '_')}.pdf"
paper.pdf_file.save(filename, ContentFile(pdf_content))
if not paper.pdf_file:
try:
pdf_content = service.fetch_pdf(doi)
if pdf_content:
filename = f"{doi.replace('/', '_')}.pdf"
paper.pdf_file.save(filename, ContentFile(pdf_content))
except Exception as e:
logger.error(
"[paper] sci-hub PDF download failed",
extra={"doi": doi, "error": str(e)},
)
return paper

View File

@ -0,0 +1,149 @@
import json
import logging
import re
import requests
import yake
CROSSREF_WORK_URL = "https://api.crossref.org/works/{}"
logger = logging.getLogger(__name__)
_STOPWORDS = {
"this", "that", "these", "those", "the", "a", "an", "in", "on", "at",
"to", "for", "of", "and", "or", "is", "are", "was", "were", "be",
"been", "being", "have", "has", "had", "do", "does", "did", "will",
"would", "can", "could", "may", "might", "shall", "should", "not",
"no", "nor", "with", "from", "by", "as", "at", "but", "if", "because",
"while", "although", "however", "we", "our", "their", "its", "it",
"they", "them", "also", "more", "most", "new", "such", "into",
"across", "between", "through", "about", "after", "before", "during",
"within", "without", "other", "many", "some", "each", "every", "both",
"few", "own", "via",
}
_DROP_PHRASES = {
"paper", "study", "studies", "research", "introduction", "conclusion",
"conclusions", "background", "methods", "results", "findings",
"analysis", "approach", "approaches", "framework", "theory",
"theories", "concept", "concepts", "model", "models", "process",
"processes", "role", "roles", "factor", "factors", "effect",
"effects", "impact", "implication", "implications", "actor", "actors",
"article", "chapter", "section", "discussion", "review", "overview",
"summary", "methodology", "special issue", "implications",
"limitations", "findings", "purpose", "objective", "objectives",
"design", "setting", "participants", "sample", "data",
"contemporary", "little", "empirical", "theoretical",
"organizations", "dissent",
}
def _strip_jats(text: str) -> str:
if not text:
return ""
text = re.sub(r"</?jats:[^>]*>", "", text)
text = re.sub(r"^\s*Abstract\s*", "", text)
return text.strip()
def _extract_genres_from_abstract(abstract: str, max_keywords: int = 8) -> list[str]:
if not abstract or len(abstract) < 50:
return []
kw_extractor = yake.KeywordExtractor(lan="en", n=2, top=max_keywords)
keywords = kw_extractor.extract_keywords(abstract)
genres = []
seen = set()
for kw, score in keywords:
kw_lower = kw.lower().strip()
if kw_lower in seen or kw_lower in _DROP_PHRASES:
continue
words = [w for w in kw_lower.split() if w not in _STOPWORDS]
cleaned = " ".join(words)
if not cleaned or len(cleaned) < 3 or cleaned in seen:
continue
if cleaned in _DROP_PHRASES:
continue
seen.add(cleaned)
genres.append(cleaned)
return genres
def lookup_paper_from_crossref(doi: str) -> dict:
url = CROSSREF_WORK_URL.format(doi)
headers = {"User-Agent": "Vrobbler/1.0 (mailto:hello@example.com)"}
response = requests.get(url, headers=headers)
if response.status_code != 200:
logger.warning(
"Bad response from Crossref",
extra={"doi": doi, "status": response.status_code},
)
return {"doi_id": doi}
try:
data = response.json()
except json.JSONDecodeError:
return {"doi_id": doi}
msg = data.get("message", {})
if not msg:
return {"doi_id": doi}
paper_dict = {"doi_id": doi}
titles = msg.get("title", [])
if titles:
paper_dict["title"] = titles[0]
abstract = msg.get("abstract", "")
if abstract:
stripped = _strip_jats(abstract)
paper_dict["abstract"] = stripped
genres = _extract_genres_from_abstract(stripped)
if genres:
paper_dict["genres"] = genres
author_dicts = []
for author in msg.get("author", []):
given = author.get("given", "")
family = author.get("family", "")
name = f"{given} {family}".strip()
if not name:
continue
entry = {"name": name}
orcid = author.get("ORCID", "")
if orcid:
orcid_id = orcid.replace("https://orcid.org/", "")
entry["authorId"] = orcid_id
author_dicts.append(entry)
if author_dicts:
paper_dict["author_dicts"] = author_dicts
container = msg.get("container-title", [])
if container:
paper_dict["journal_name"] = container[0]
volume = msg.get("volume")
if volume:
paper_dict["journal_volume"] = volume
page = msg.get("page")
if page:
try:
parts = page.split("-")
if len(parts) == 2:
paper_dict["pages"] = int(parts[1]) - int(parts[0])
except (ValueError, IndexError):
pass
for date_field in ("published-print", "published-online", "created"):
date_data = msg.get(date_field)
if date_data and date_data.get("date-parts"):
parts = date_data["date-parts"][0]
if len(parts) >= 1:
paper_dict["first_publish_year"] = int(parts[0])
if len(parts) >= 3:
paper_dict["publish_date"] = f"{parts[0]:04d}-{parts[1]:02d}-{parts[2]:02d}"
break
return paper_dict

View File

@ -0,0 +1,142 @@
import logging
from typing import Optional
from urllib.parse import urljoin
import requests
from bs4 import BeautifulSoup
from django.conf import settings
logger = logging.getLogger(__name__)
SCIHUB_DOMAINS = [
"sci-hub.ru",
"sci-hub.ee",
"sci-hub.st",
"sci-hub.do",
]
class SciHubService:
def __init__(self):
self.session = requests.Session()
self.session.headers.update(
{
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
)
def fetch_from_url(self, url: str) -> Optional[bytes]:
try:
resp = self.session.get(url, timeout=60)
if resp.status_code != 200:
logger.warning(
"[pdf] URL download failed",
extra={"status": resp.status_code, "url": url},
)
return None
if not self._looks_like_pdf(resp):
return None
return resp.content
except requests.RequestException as e:
logger.error(
"[pdf] URL download request failed",
extra={"url": url, "error": str(e)},
)
return None
def fetch_pdf(self, doi: str) -> Optional[bytes]:
configured_domain = getattr(settings, "SCIHUB_DOMAIN", None)
domains_to_try = (
[configured_domain] + SCIHUB_DOMAINS
if configured_domain and configured_domain not in SCIHUB_DOMAINS
else SCIHUB_DOMAINS
)
for domain in domains_to_try:
url = f"https://{domain}/{doi}"
logger.info(
"[scihub] trying domain",
extra={"domain": domain, "doi": doi},
)
try:
response = self.session.get(url, timeout=30)
if response.status_code != 200:
continue
pdf_url = self._extract_pdf_url(response.text, url)
if not pdf_url:
continue
pdf_response = self.session.get(pdf_url, timeout=60)
if pdf_response.status_code != 200:
continue
if not self._looks_like_pdf(pdf_response):
continue
logger.info(
"[scihub] PDF downloaded successfully",
extra={
"domain": domain,
"doi": doi,
"size": len(pdf_response.content),
},
)
return pdf_response.content
except requests.RequestException as e:
logger.debug(
"[scihub] domain failed",
extra={"domain": domain, "doi": doi, "error": str(e)},
)
continue
logger.warning(
"[scihub] all domains failed",
extra={"doi": doi, "tried": domains_to_try},
)
return None
def _looks_like_pdf(self, response: requests.Response) -> bool:
content_type = response.headers.get("Content-Type", "")
if "application/pdf" in content_type:
return True
if content_type.startswith("application/octet"):
return True
if response.url.endswith(".pdf"):
return True
return False
def _extract_pdf_url(self, html: str, page_url: str) -> Optional[str]:
soup = BeautifulSoup(html, "html.parser")
iframe = soup.find("iframe", {"id": "pdf"})
if iframe and iframe.get("src"):
src = iframe["src"]
if src.startswith("http"):
return src
return urljoin(page_url, src)
embed = soup.find("embed", {"type": "application/pdf"})
if embed and embed.get("src"):
src = embed["src"]
if src.startswith("http"):
return src
return urljoin(page_url, src)
download_div = soup.find("div", {"id": "download"})
if download_div:
link = download_div.find("a")
if link and link.get("href"):
href = link["href"]
if href.startswith("http"):
return href
return urljoin(page_url, href)
for link in soup.find_all("a", href=True):
href = link["href"]
if ".pdf" in href:
if href.startswith("http"):
return href
return urljoin(page_url, href)
return None

View File

@ -9,6 +9,7 @@ PAPER_SEARCH_URL = (
"https://api.semanticscholar.org/graph/v1/paper/search/match?query={}"
)
PAPER_DETAIL_URL = "https://api.semanticscholar.org/graph/v1/paper/{}?fields=title,authors,url,year,abstract,externalIds,citationCount,referenceCount,journal,fieldsOfStudy,publicationDate,openAccessPdf"
PAPER_DOI_URL = "https://api.semanticscholar.org/graph/v1/paper/DOI:{}?fields=title,authors,url,year,abstract,externalIds,citationCount,referenceCount,journal,fieldsOfStudy,publicationDate,openAccessPdf"
logger = logging.getLogger(__name__)
@ -39,6 +40,18 @@ def lookup_paper_from_semantic(title: str) -> dict:
if not result:
return paper_dict
paper_dict.update(_parse_semantic_result(result))
paper_dict.setdefault("title", title)
if paper_dict.get("publish_date"):
paper_dict["publish_date"] = datetime.strptime(
paper_dict["publish_date"], "%Y-%m-%d"
)
return paper_dict
def _parse_semantic_result(result: dict) -> dict:
paper_dict = {}
page_str = result.get("journal", {}).get("pages")
if page_str:
try:
@ -55,12 +68,10 @@ def lookup_paper_from_semantic(title: str) -> dict:
paper_dict["corpus_id"] = result.get("externalIds", {}).get("CorpusId")
paper_dict["semantic_title"] = result.get("title")
paper_dict["first_publish_year"] = result.get("year")
paper_dict["publish_date"] = datetime.strptime(
result.get("publicationDate", "1950-01-01"), "%Y-%m-%d"
)
paper_dict["publish_date"] = result.get("publicationDate")
paper_dict["abstract"] = result.get("abstract")
paper_dict["tldr"] = result.get("bib", {}).get("abstract")
paper_dict["journal"] = result.get("journal", {}).get("name")
paper_dict["journal_name"] = result.get("journal", {}).get("name")
paper_dict["journal_volume"] = result.get("journal", {}).get("volume")
paper_dict["openaccess_pdf_url"] = result.get("openAccessPdf", {}).get("url")
paper_dict["base_run_time_seconds"] = paper_dict.get("pages", 10) * getattr(
@ -68,5 +79,19 @@ def lookup_paper_from_semantic(title: str) -> dict:
)
paper_dict["author_dicts"] = result.get("authors")
paper_dict["genres"] = result.get("fieldsOfStudy")
return paper_dict
def lookup_paper_from_semantic_by_doi(doi: str) -> dict:
response = get_api_result(PAPER_DOI_URL.format(doi))
if not response:
return {"doi_id": doi}
result = json.loads(response.content)
if not result:
return {"doi_id": doi}
paper_dict = _parse_semantic_result(result)
if not paper_dict.get("title"):
paper_dict["title"] = result.get("title", f"Paper {doi}")
return paper_dict

View File

@ -16,4 +16,15 @@ urlpatterns = [
views.AuthorDetailView.as_view(),
name="author_detail",
),
path("papers/", views.PaperListView.as_view(), name="paper_list"),
path(
"papers/<slug:slug>/",
views.PaperDetailView.as_view(),
name="paper_detail",
),
path(
"papers/<slug:slug>/upload_pdf/",
views.PaperUploadPdfView.as_view(),
name="paper_upload_pdf",
),
]

View File

@ -1,5 +1,8 @@
from django.http import HttpResponseRedirect
from django.urls import reverse
from django.views import View
from django.views import generic
from books.models import Book, Author
from books.models import Book, Author, Paper
from scrobbles.views import ScrobbleableListView, ScrobbleableDetailView
@ -15,3 +18,24 @@ class BookDetailView(ScrobbleableDetailView):
class AuthorDetailView(generic.DetailView):
model = Author
slug_field = "uuid"
class PaperListView(ScrobbleableListView):
model = Paper
class PaperDetailView(ScrobbleableDetailView):
model = Paper
class PaperUploadPdfView(View):
def post(self, request, slug):
paper = Paper.objects.filter(uuid=slug).first()
if not paper or not request.user.is_authenticated:
return HttpResponseRedirect(reverse("books:paper_detail", args=[slug]))
pdf_file = request.FILES.get("pdf_file")
if pdf_file:
paper.pdf_file.save(pdf_file.name, pdf_file)
return HttpResponseRedirect(reverse("books:paper_detail", args=[slug]))

View File

@ -0,0 +1,109 @@
# Generated by Django 4.2.29 on 2026-06-19 16:25
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("charts", "0002_chartrecord_charts_char_user_id_1adcde_idx_and_more"),
]
operations = [
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("artist__isnull", False)),
fields=("user", "year", "month", "week", "day", "artist"),
name="unique_chart_artist_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("album__isnull", False)),
fields=("user", "year", "month", "week", "day", "album"),
name="unique_chart_album_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("track__isnull", False)),
fields=("user", "year", "month", "week", "day", "track"),
name="unique_chart_track_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("tv_series__isnull", False)),
fields=("user", "year", "month", "week", "day", "tv_series"),
name="unique_chart_tv_series_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("video__isnull", False)),
fields=("user", "year", "month", "week", "day", "video"),
name="unique_chart_video_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("podcast__isnull", False)),
fields=("user", "year", "month", "week", "day", "podcast"),
name="unique_chart_podcast_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("podcast_episode__isnull", False)),
fields=("user", "year", "month", "week", "day", "podcast_episode"),
name="unique_chart_podcast_episode_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("board_game__isnull", False)),
fields=("user", "year", "month", "week", "day", "board_game"),
name="unique_chart_board_game_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("trail__isnull", False)),
fields=("user", "year", "month", "week", "day", "trail"),
name="unique_chart_trail_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("geo_location__isnull", False)),
fields=("user", "year", "month", "week", "day", "geo_location"),
name="unique_chart_geo_location_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("food__isnull", False)),
fields=("user", "year", "month", "week", "day", "food"),
name="unique_chart_food_period",
),
),
migrations.AddConstraint(
model_name="chartrecord",
constraint=models.UniqueConstraint(
condition=models.Q(("book__isnull", False)),
fields=("user", "year", "month", "week", "day", "book"),
name="unique_chart_book_period",
),
),
]

View File

@ -2,6 +2,7 @@ import calendar
from django.contrib.auth import get_user_model
from django.db import models
from django.db.models import Q
from django.urls import reverse
from django_extensions.db.models import TimeStampedModel
@ -84,6 +85,68 @@ class ChartRecord(TimeStampedModel):
models.Index(fields=["user", "year", "month", "day", "album", "rank"]),
models.Index(fields=["user", "year", "month", "day", "tv_series", "rank"]),
]
constraints = [
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "artist"],
condition=Q(artist__isnull=False),
name="unique_chart_artist_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "album"],
condition=Q(album__isnull=False),
name="unique_chart_album_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "track"],
condition=Q(track__isnull=False),
name="unique_chart_track_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "tv_series"],
condition=Q(tv_series__isnull=False),
name="unique_chart_tv_series_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "video"],
condition=Q(video__isnull=False),
name="unique_chart_video_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "podcast"],
condition=Q(podcast__isnull=False),
name="unique_chart_podcast_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "podcast_episode"],
condition=Q(podcast_episode__isnull=False),
name="unique_chart_podcast_episode_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "board_game"],
condition=Q(board_game__isnull=False),
name="unique_chart_board_game_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "trail"],
condition=Q(trail__isnull=False),
name="unique_chart_trail_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "geo_location"],
condition=Q(geo_location__isnull=False),
name="unique_chart_geo_location_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "food"],
condition=Q(food__isnull=False),
name="unique_chart_food_period",
),
models.UniqueConstraint(
fields=["user", "year", "month", "week", "day", "book"],
condition=Q(book__isnull=False),
name="unique_chart_book_period",
),
]
@property
def media_obj(self):

View File

@ -6,6 +6,7 @@ from typing import Optional
import pytz
from django.apps import apps
from django.conf import settings
from django.db import transaction
from django.db.models import Count, Q
from django.utils import timezone
@ -186,60 +187,64 @@ def build_charts(
ranks = {count: rank for rank, count in enumerate(unique_counts, start=1)}
media_field = f"{media_type}_id"
records_to_create = []
records_to_update = []
existing = ChartRecord.objects.filter(
period_filter, user=user, **{media_field + "__isnull": False}
)
existing_by_media_id = {getattr(r, media_field): r for r in existing}
found_media_ids = set()
with transaction.atomic():
records_to_create = []
records_to_update = []
for result in results:
media_id = result[config["values"]]
if media_id is None:
continue
found_media_ids.add(media_id)
chart_record_data = {
"user_id": user.id,
"year": year,
"month": month,
"week": week,
"day": day,
"rank": ranks[result["scrobble_count"]],
"count": result["scrobble_count"],
}
chart_record_data[media_field] = media_id
if media_id in existing_by_media_id:
existing_record = existing_by_media_id[media_id]
existing_record.rank = chart_record_data["rank"]
existing_record.count = chart_record_data["count"]
records_to_update.append(existing_record)
else:
records_to_create.append(ChartRecord(**chart_record_data))
ids_to_delete = [
r.id for r in existing if getattr(r, media_field) not in found_media_ids
]
if ids_to_delete:
ChartRecord.objects.filter(id__in=ids_to_delete).delete()
if records_to_update:
ChartRecord.objects.bulk_update(
records_to_update, ["rank", "count"], batch_size=500
existing = ChartRecord.objects.select_for_update().filter(
period_filter, user=user, **{media_field + "__isnull": False}
)
existing_by_media_id = {getattr(r, media_field): r for r in existing}
found_media_ids = set()
if records_to_create:
ChartRecord.objects.bulk_create(records_to_create, batch_size=500)
for result in results:
media_id = result[config["values"]]
if media_id is None:
continue
logger.info(
f"Built {len(records_to_create)} new, {len(records_to_update)} updated "
f"chart records for {media_type}, period "
f"{year}-{month or ''}-{week or ''}-{day or ''} for {user}"
)
found_media_ids.add(media_id)
chart_record_data = {
"user_id": user.id,
"year": year,
"month": month,
"week": week,
"day": day,
"rank": ranks[result["scrobble_count"]],
"count": result["scrobble_count"],
}
chart_record_data[media_field] = media_id
if media_id in existing_by_media_id:
existing_record = existing_by_media_id[media_id]
existing_record.rank = chart_record_data["rank"]
existing_record.count = chart_record_data["count"]
records_to_update.append(existing_record)
else:
records_to_create.append(ChartRecord(**chart_record_data))
ids_to_delete = [
r.id
for r in existing
if getattr(r, media_field) not in found_media_ids
]
if ids_to_delete:
ChartRecord.objects.filter(id__in=ids_to_delete).delete()
if records_to_update:
ChartRecord.objects.bulk_update(
records_to_update, ["rank", "count"], batch_size=500
)
if records_to_create:
ChartRecord.objects.bulk_create(records_to_create, batch_size=500)
logger.info(
f"Built {len(records_to_create)} new, {len(records_to_update)} updated "
f"chart records for {media_type}, period "
f"{year}-{month or ''}-{week or ''}-{day or ''} for {user}"
)
def build_yesterdays_charts(user, media_types: Optional[list] = None) -> None:

View File

View File

@ -0,0 +1,14 @@
from discgolf.models import DiscGolfCourse
from django.contrib import admin
from scrobbles.admin import ScrobbleInline
@admin.register(DiscGolfCourse)
class DiscGolfCourseAdmin(admin.ModelAdmin):
date_hierarchy = "created"
list_display = ("title", "layout_name", "number_of_holes", "par_total", "pdga_slug", "udisc_id")
raw_id_fields = ("trail",)
search_fields = ("title", "layout_name")
inlines = [
ScrobbleInline,
]

View File

View File

@ -0,0 +1,14 @@
from discgolf import models
from rest_framework import serializers
class DiscGolfCourseSerializer(serializers.HyperlinkedModelSerializer):
pdga_link = serializers.ReadOnlyField()
udisc_link = serializers.ReadOnlyField()
class Meta:
model = models.DiscGolfCourse
fields = "__all__"

View File

@ -0,0 +1,13 @@
from rest_framework import permissions, viewsets
from discgolf.api import serializers
from discgolf import models
class DiscGolfCourseViewSet(viewsets.ModelViewSet):
queryset = models.DiscGolfCourse.objects.all().order_by("-created")
serializer_class = serializers.DiscGolfCourseSerializer
permission_classes = [permissions.IsAuthenticated]

View File

@ -0,0 +1,6 @@
from django.apps import AppConfig
class DiscgolfConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "discgolf"

View File

@ -0,0 +1,83 @@
# Generated by Django 4.2.29 on 2026-06-20 04:19
from django.db import migrations, models
import django_extensions.db.fields
import taggit.managers
import uuid
class Migration(migrations.Migration):
initial = True
dependencies = [
("taggit", "0004_alter_taggeditem_content_type_alter_taggeditem_tag"),
("scrobbles", "0098_scrobble_long_play_last_scrobble"),
]
operations = [
migrations.CreateModel(
name="DiscGolfCourse",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
django_extensions.db.fields.CreationDateTimeField(
auto_now_add=True, verbose_name="created"
),
),
(
"modified",
django_extensions.db.fields.ModificationDateTimeField(
auto_now=True, verbose_name="modified"
),
),
(
"uuid",
models.UUIDField(
blank=True, default=uuid.uuid4, editable=False, null=True
),
),
("title", models.CharField(blank=True, max_length=255, null=True)),
("base_run_time_seconds", models.IntegerField(blank=True, null=True)),
("description", models.TextField(blank=True, null=True)),
(
"layout_name",
models.CharField(blank=True, max_length=255, null=True),
),
("number_of_holes", models.IntegerField(blank=True, null=True)),
("par_total", models.IntegerField(blank=True, null=True)),
(
"genre",
taggit.managers.TaggableManager(
blank=True,
help_text="A comma-separated list of tags.",
through="scrobbles.ObjectWithGenres",
to="scrobbles.Genre",
verbose_name="Genre",
),
),
(
"tags",
taggit.managers.TaggableManager(
blank=True,
help_text="A comma-separated list of tags.",
through="taggit.TaggedItem",
to="taggit.Tag",
verbose_name="Tags",
),
),
],
options={
"abstract": False,
},
),
]

View File

@ -0,0 +1,25 @@
# Generated by Django 4.2.29 on 2026-06-20 04:29
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
("trails", "0009_trail_route_waypoint"),
("discgolf", "0001_initial"),
]
operations = [
migrations.AddField(
model_name="discgolfcourse",
name="trail",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.DO_NOTHING,
to="trails.trail",
),
),
]

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.29 on 2026-06-20 04:35
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("discgolf", "0002_discgolfcourse_trail"),
]
operations = [
migrations.AddField(
model_name="discgolfcourse",
name="par_per_hole",
field=models.JSONField(blank=True, null=True),
),
]

View File

@ -0,0 +1,23 @@
# Generated by Django 4.2.29 on 2026-06-21 04:16
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("discgolf", "0003_discgolfcourse_par_per_hole"),
]
operations = [
migrations.AddField(
model_name="discgolfcourse",
name="pdga_slug",
field=models.CharField(blank=True, max_length=255, null=True),
),
migrations.AddField(
model_name="discgolfcourse",
name="udisc_id",
field=models.CharField(blank=True, max_length=255, null=True),
),
]

View File

@ -0,0 +1,90 @@
from dataclasses import dataclass
from typing import Optional, TypedDict
from django.apps import apps
from django.db import models
from django.urls import reverse
from scrobbles.dataclasses import BaseLogData, WithPeopleLogData
from scrobbles.mixins import ScrobblableConstants, ScrobblableMixin
BNULL = {"blank": True, "null": True}
class DiscGolfSingleScores(TypedDict, total=False):
person_id: int
total: int
class DiscGolfTeamScores(TypedDict, total=False):
person_ids: list[int]
total: int
@dataclass
class DiscGolfLogData(BaseLogData, WithPeopleLogData):
scores: Optional[dict[str, DiscGolfSingleScores | DiscGolfTeamScores]] = None
weather: Optional[str] = None
fun_factor: Optional[str] = None
course_name: Optional[str] = None
par: Optional[int] = None
round_type: Optional[str] = None
class DiscGolfCourse(ScrobblableMixin):
description = models.TextField(**BNULL)
layout_name = models.CharField(max_length=255, **BNULL)
number_of_holes = models.IntegerField(**BNULL)
par_total = models.IntegerField(**BNULL)
par_per_hole = models.JSONField(**BNULL)
trail = models.ForeignKey(
"trails.Trail", on_delete=models.DO_NOTHING, **BNULL
)
pdga_slug = models.CharField(max_length=255, **BNULL)
udisc_id = models.CharField(max_length=255, **BNULL)
@property
def pdga_link(self) -> str:
if self.pdga_slug:
return f"https://www.pdga.com/course-directory/course/{self.pdga_slug}/"
return ""
@property
def udisc_link(self) -> str:
if self.udisc_id:
return f"https://udisc.com/courses/{self.udisc_id}/"
return ""
def get_absolute_url(self) -> str:
return reverse("discgolf:course_detail", kwargs={"slug": self.uuid})
def __str__(self):
return f"{self.title} ({self.layout_name or 'Default'})"
@property
def logdata_cls(self):
return DiscGolfLogData
@property
def subtitle(self):
return self.layout_name or ""
@property
def strings(self) -> ScrobblableConstants:
return ScrobblableConstants(verb="Playing", tags="golf")
@property
def primary_image_url(self) -> str:
return ""
@classmethod
def find_or_create(cls, name: str, **defaults) -> "DiscGolfCourse":
course = cls.objects.filter(title=name).first()
if not course:
course = cls.objects.create(title=name, **defaults)
return course
def scrobbles(self, user_id):
Scrobble = apps.get_model("scrobbles", "Scrobble")
return Scrobble.objects.filter(user_id=user_id, disc_golf_course=self).order_by(
"-timestamp"
)

View File

@ -0,0 +1,14 @@
from django.urls import path
from discgolf import views
app_name = "discgolf"
urlpatterns = [
path("disc-golf/", views.DiscGolfCourseListView.as_view(), name="course_list"),
path(
"disc-golf/<slug:slug>/",
views.DiscGolfCourseDetailView.as_view(),
name="course_detail",
),
]

View File

@ -0,0 +1,129 @@
import csv
import logging
from datetime import datetime
from dateutil.parser import parse as parse_datetime
from django.utils import timezone
from people.models import Person
from scrobbles.models import Scrobble
from scrobbles.notifications import ScrobbleNtfyNotification
logger = logging.getLogger(__name__)
def _parse_udisc_datetime(raw: str) -> datetime:
return parse_datetime(raw)
def _resolve_player(name: str, user_id: int) -> Person:
name = name.strip()
existing = Person.objects.filter(name=name, created_by_id=user_id).first()
if existing:
return existing
return Person.objects.create(name=name, created_by_id=user_id)
def import_udisc_csv(
file_path: str, user_id: int, record_error=None
) -> list[Scrobble]:
from discgolf.models import DiscGolfCourse
new_scrobbles = []
with open(file_path, newline="", encoding="utf-8-sig") as f:
reader = csv.DictReader(f)
rows = list(reader)
if not rows:
return []
par_row = None
player_rows = []
for row in rows:
name = row.get("PlayerName", "").strip()
if name.lower() == "par":
par_row = row
else:
player_rows.append(row)
if not par_row:
return []
course_name = par_row.get("CourseName", "").strip()
layout_name = par_row.get("LayoutName", "").strip()
start_date_raw = par_row.get("StartDate", "").strip()
start_dt = _parse_udisc_datetime(start_date_raw) if start_date_raw else timezone.now()
number_of_holes = sum(1 for k in par_row if k.startswith("Hole") and k[4:].isdigit())
par_total_str = par_row.get("Total", "").strip()
par_total = int(par_total_str) if par_total_str.isdigit() else None
par_per_hole = {}
for k, v in par_row.items():
if k.startswith("Hole") and k[4:].isdigit() and v:
hole_num = int(k[4:])
try:
par_per_hole[f"hole_{hole_num}"] = int(v)
except (ValueError, TypeError):
pass
course, _ = DiscGolfCourse.objects.get_or_create(
title=course_name,
defaults={
"layout_name": layout_name,
"number_of_holes": number_of_holes,
"par_total": par_total,
"par_per_hole": par_per_hole or None,
},
)
is_teams = "+" in player_rows[0].get("PlayerName", "") if player_rows else False
round_type = "Teams" if is_teams else "Singles"
scores = {}
for row in player_rows:
player_name = row.get("PlayerName", "").strip()
hole_scores = {}
for k, v in row.items():
if k.startswith("Hole") and k[4:].isdigit() and v:
hole_num = int(k[4:])
try:
hole_scores[f"hole_{hole_num}"] = int(v)
except (ValueError, TypeError):
pass
total_str = row.get("Total", "").strip()
total = int(total_str) if total_str.isdigit() else None
if total is not None:
hole_scores["total"] = total
if is_teams:
people = player_name.split("+")
person_ids = [_resolve_player(p.strip(), user_id).id for p in people]
hole_scores["person_ids"] = person_ids
else:
person = _resolve_player(player_name, user_id)
hole_scores["person_id"] = person.id
scores[player_name] = hole_scores
log = {
"scores": scores,
"course_name": course_name,
"par": par_total,
"round_type": round_type,
}
scrobble_dict = {
"user_id": user_id,
"timestamp": start_dt,
"source": "uDisc",
"playback_position_seconds": 0,
"log": log,
}
scrobble = Scrobble.create_or_update(course, user_id, scrobble_dict)
if scrobble:
new_scrobbles.append(scrobble)
ScrobbleNtfyNotification(scrobble).send()
return new_scrobbles

View File

@ -0,0 +1,32 @@
from django.apps import apps
from discgolf.models import DiscGolfCourse
from scrobbles.views import (
ScrobbleableListView,
ScrobbleableDetailView,
ChartContextMixin,
)
class DiscGolfCourseListView(ScrobbleableListView):
model = DiscGolfCourse
class DiscGolfCourseDetailView(ScrobbleableDetailView, ChartContextMixin):
model = DiscGolfCourse
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
Scrobble = apps.get_model("scrobbles", "Scrobble")
context["trail_gpx_url"] = None
latest = (
Scrobble.objects.filter(
trail=self.object.trail, gpx_file__isnull=False
)
.order_by("-timestamp")
.first()
)
if latest and latest.gpx_file:
context["trail_gpx_url"] = latest.gpx_file.url
return context

View File

@ -1,4 +1,7 @@
import time
from django.contrib import admin
from django.http import HttpRequest
from locations.models import GeoLocation
@ -14,9 +17,29 @@ class GeoLocationAdmin(admin.ModelAdmin):
"lon",
"title",
"altitude",
"city",
"state_province",
"country",
)
ordering = ("-created",)
search_fields = ("title",)
actions = ["reverse_geocode_selected"]
inlines = [
ScrobbleInline,
]
@admin.action(description="Reverse geocode selected locations")
def reverse_geocode_selected(self, request: HttpRequest, queryset):
updated = 0
errors = 0
for i, location in enumerate(queryset.iterator()):
if location.reverse_geocode():
updated += 1
else:
errors += 1
if i < queryset.count() - 1:
time.sleep(1.1)
msg = f"Reverse geocoded {updated} locations"
if errors:
msg += f", {errors} failed"
self.message_user(request, msg)

View File

@ -0,0 +1,38 @@
# Generated by Django 4.2.29 on 2026-06-21 04:26
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("locations", "0010_clean_start"),
]
operations = [
migrations.AddField(
model_name="geolocation",
name="city",
field=models.CharField(blank=True, max_length=255, null=True),
),
migrations.AddField(
model_name="geolocation",
name="country",
field=models.CharField(blank=True, max_length=100, null=True),
),
migrations.AddField(
model_name="geolocation",
name="postal_code",
field=models.CharField(blank=True, max_length=20, null=True),
),
migrations.AddField(
model_name="geolocation",
name="state_province",
field=models.CharField(blank=True, max_length=255, null=True),
),
migrations.AddField(
model_name="geolocation",
name="street",
field=models.TextField(blank=True, null=True),
),
]

View File

@ -45,6 +45,11 @@ class GeoLocation(ScrobblableMixin):
truncated_lat = models.FloatField(**BNULL)
truncated_lon = models.FloatField(**BNULL)
altitude = models.FloatField(**BNULL)
street = models.TextField(**BNULL)
city = models.CharField(max_length=255, **BNULL)
state_province = models.CharField(max_length=255, **BNULL)
postal_code = models.CharField(max_length=20, **BNULL)
country = models.CharField(max_length=100, **BNULL)
class Meta:
unique_together = [["lat", "lon", "altitude"]]
@ -55,6 +60,11 @@ class GeoLocation(ScrobblableMixin):
return f"{self.lat} x {self.lon}"
@property
def display_address(self) -> str:
parts = [self.street, self.city, self.state_province, self.postal_code, self.country]
return ", ".join(p for p in parts if p)
def get_absolute_url(self):
return reverse("locations:geolocation_detail", kwargs={"slug": self.uuid})
@ -121,6 +131,17 @@ class GeoLocation(ScrobblableMixin):
return fetch_current_weather(self.lat, self.lon)
def reverse_geocode(self) -> bool:
from locations.utils import reverse_geocode
result = reverse_geocode(self.lat, self.lon)
if result is None:
return False
for field, value in result.items():
setattr(self, field, value)
self.save(update_fields=list(result.keys()))
return True
def loc_diff(self, old_lat_lon: tuple) -> tuple:
return (
abs(Decimal(old_lat_lon[0]) - Decimal(self.lat)),

View File

@ -201,6 +201,50 @@ def detect_movement(
return result
NOMINATIM_URL = "https://nominatim.openstreetmap.org/reverse"
USER_AGENT = "Vrobbler/1.0 (https://github.com/secstate/vrobbler)"
def reverse_geocode(lat: float, lon: float) -> Optional[dict]:
"""Reverse geocode lat/lon to an address using Nominatim.
Returns a dict with address fields, or None on failure.
Nominatim usage policy: max 1 request per second.
"""
params = {
"lat": lat,
"lon": lon,
"format": "json",
}
headers = {"User-Agent": USER_AGENT}
try:
resp = requests.get(NOMINATIM_URL, params=params, headers=headers, timeout=10)
resp.raise_for_status()
except requests.RequestException as e:
logger.warning("Failed to reverse geocode %s,%s: %s", lat, lon, e)
return None
data = resp.json()
if "error" in data:
logger.warning("Nominatim error for %s,%s: %s", lat, lon, data["error"])
return None
address = data.get("address", {})
return {
"street": address.get("road")
or address.get("pedestrian")
or address.get("footway"),
"city": address.get("city")
or address.get("town")
or address.get("village")
or address.get("hamlet"),
"state_province": address.get("state"),
"postal_code": address.get("postcode"),
"country": address.get("country"),
}
NWS_URL = "https://forecast.weather.gov/MapClick.php"

View File

@ -1,8 +1,22 @@
from django.core.cache import cache
from music.models import Artist, Album
CACHE_TTL = 300
def music_lists(request):
artist_list = cache.get("music_lists_artist_list")
if artist_list is None:
artist_list = list(Artist.objects.all().only("id", "name"))
cache.set("music_lists_artist_list", artist_list, CACHE_TTL)
album_list = cache.get("music_lists_album_list")
if album_list is None:
album_list = list(Album.objects.all().only("id", "name"))
cache.set("music_lists_album_list", album_list, CACHE_TTL)
return {
"artist_list": Artist.objects.all(),
"album_list": Album.objects.all(),
"artist_list": artist_list,
"album_list": album_list,
}

View File

@ -236,19 +236,6 @@ class Artist(TimeStampedModel):
)
artist.fix_metadata()
# TODO: See if this alt_names stuff actually works or causes hard to debug problems
# If we did find our artist, but the found name is slightly differnt, record that
# if artist and alt_name:
# if not artist.alt_names:
# artist.alt_names = alt_name
# else:
# artist.alt_names += f"\\{alt_name}"
# logger.info(
# f"Add alt_name {alt_name} to artist {artist}",
# extra={"alt_name": alt_name, "artist_id": artist.id},
# )
# artist.save(update_fields=["alt_names"])
return artist

View File

@ -40,6 +40,7 @@ class UserProfileForm(forms.ModelForm):
"enable_public_widgets",
"widget_custom_css",
"home_scrobble_limit",
"live_now_playing",
"weigh_in_units",
]
widgets = {

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.29 on 2026-06-22 02:44
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("profiles", "0038_userprofile_media_type_visibility"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="live_now_playing",
field=models.BooleanField(default=False),
),
]

View File

@ -98,6 +98,8 @@ class UserProfile(TimeStampedModel):
home_scrobble_limit = models.IntegerField(default=20)
live_now_playing = models.BooleanField(default=False)
weigh_in_units = models.CharField(
max_length=16,
choices=WeighUnit.choices,

View File

@ -12,6 +12,7 @@ from scrobbles.models import (
Scrobble,
ShareViewLog,
TrailGPXImport,
UDiscCSVImport,
)
from scrobbles.mixins import Genre
@ -73,6 +74,10 @@ class ScaleCSVImportAdmin(ImportBaseAdmin): ...
class TrailGPXImportAdmin(ImportBaseAdmin): ...
@admin.register(UDiscCSVImport)
class UDiscCSVImportAdmin(ImportBaseAdmin): ...
@admin.register(Genre)
class GenreAdmin(admin.ModelAdmin):
list_display = (
@ -118,6 +123,7 @@ class ScrobbleAdmin(admin.ModelAdmin):
"web_page",
"life_event",
"birding_location",
"disc_golf_course",
"long_play_last_scrobble",
)
list_filter = (
@ -179,4 +185,5 @@ class FavoriteMediaAdmin(admin.ModelAdmin):
"web_page",
"life_event",
"birding_location",
"disc_golf_course",
)

View File

@ -14,6 +14,7 @@ LONG_PLAY_MEDIA = {
"books": "Book",
"bricksets": "BrickSet",
"tasks": "Task",
"papers": "Paper",
}
# Media types that should just be finished if they go over time
@ -25,17 +26,26 @@ AUTO_FINISH_MEDIA = {
}
PLAY_AGAIN_MEDIA = {
"videogames": "VideoGame",
"videos": "Video",
"music": "Track",
"podcasts": "PodcastEpisode",
"sports": "SportEvent",
"books": "Book",
"videogames": "VideoGame",
"boardgames": "BoardGame",
"moods": "Mood",
"bricksets": "BrickSet",
"locations": "GeoLocation",
"trails": "Trail",
"beers": "Beer",
"puzzles": "Puzzle",
"foods": "Food",
"locations": "GeoLocation",
"videos": "Video",
"tasks": "Task",
"webpages": "WebPage",
"lifeevents": "LifeEvent",
"moods": "Mood",
"bricksets": "BrickSet",
"channels": "Channel",
"birds": "BirdingLocation",
"discgolf": "DiscGolfCourse",
}
MEDIA_END_PADDING_SECONDS = {
@ -52,6 +62,7 @@ SCROBBLE_CONTENT_URLS = {
"-b": ["https://www.amazon.com/"],
"-t": ["https://app.todoist.com/app/task/{id}"],
"-p": ["https://www.ipdb.plus/IPDb/puzzle.php?id="],
"-pp": ["https://doi.org/"],
"-l": ["https://brickset.com/sets/"],
"-c": ["https://readcomicsonline.ru"],
"-h": ["https://www.twitch.tv/"],
@ -73,6 +84,8 @@ MANUAL_SCROBBLE_FNS = {
"-c": "manual_scrobble_book",
"-f": "manual_scrobble_food",
"-h": "manual_scrobble_twitch_channel",
"-dg": "manual_scrobble_discgolf",
"-pp": "manual_scrobble_paper",
}

View File

@ -27,15 +27,19 @@ def month_color(request):
def now_playing(request):
user = request.user
now = timezone.now()
if not user.is_authenticated:
return {}
return {
"now_playing_list": Scrobble.objects.filter(
in_progress=True,
is_paused=False,
user=user,
).exclude(
media_type__in=EXCLUDE_FROM_NOW_PLAYING,
)
"now_playing_list": list(
Scrobble.objects.filter(
in_progress=True,
is_paused=False,
user=user,
)
.exclude(
media_type__in=EXCLUDE_FROM_NOW_PLAYING,
)
.select_related("track", "video", "podcast_episode")
),
}

View File

@ -19,6 +19,7 @@ DEFAULT_RETROARCH_PATH = "var/retroarch/"
DEFAULT_BGSTATS_PATH = "var/bgstats/"
DEFAULT_EBIRD_PATH = "var/ebird/"
DEFAULT_SCALE_PATH = "var/scale/"
DEFAULT_UDISC_PATH = "var/udisc/"
def import_from_webdav_for_all_users(
@ -48,6 +49,7 @@ def import_from_webdav_for_all_users(
bgstats_count = 0
ebird_count = 0
scale_count = 0
udisc_count = 0
for user_id in webdav_enabled_user_ids:
client = get_webdav_client(user_id)
@ -78,15 +80,20 @@ def import_from_webdav_for_all_users(
)
logger.info("Scanning WebDAV scale for user %s", user_id)
scale_count += scan_webdav_for_scale(client, user_id)
logger.info("Scanning WebDAV udisc for user %s", user_id)
udisc_count += scan_webdav_for_udisc(
client, user_id, include_processed=include_processed
)
logger.info(
"Started %d KOReader, %d Trail GPX, %d Retroarch, %d BGStats, %d eBird, %d Scale WebDAV imports",
"Started %d KOReader, %d Trail GPX, %d Retroarch, %d BGStats, %d eBird, %d Scale, %d uDisc WebDAV imports",
ko_count,
gpx_count,
retro_count,
bgstats_count,
ebird_count,
scale_count,
udisc_count,
extra={
"koreader": ko_count,
"trail_gpx": gpx_count,
@ -94,9 +101,10 @@ def import_from_webdav_for_all_users(
"bgstats": bgstats_count,
"ebird": ebird_count,
"scale": scale_count,
"udisc": udisc_count,
},
)
return ko_count, gpx_count, retro_count, bgstats_count, ebird_count, scale_count
return ko_count, gpx_count, retro_count, bgstats_count, ebird_count, scale_count, udisc_count
def scan_webdav_for_koreader(
@ -811,3 +819,115 @@ def scan_webdav_for_scale(webdav_client, user_id):
os.unlink(tmp.name)
return new_imports
def scan_webdav_for_udisc(webdav_client, user_id, include_processed=False):
"""Download .csv files from WebDAV var/udisc/ and queue imports for new files.
After importing, files are moved to var/udisc/processed/ so they are
not re-imported on subsequent scans unless *include_processed* is True.
"""
from scrobbles.models import UDiscCSVImport
from scrobbles.tasks import process_udisc_csv_import
udisc_path = DEFAULT_UDISC_PATH
try:
webdav_client.info(udisc_path)
except:
logger.info("No var/udisc/ directory on webdav", extra={"user_id": user_id})
return 0
try:
files = webdav_client.list(udisc_path)
except Exception as e:
logger.warning(
"Could not list var/udisc/",
extra={"user_id": user_id, "error": str(e)},
)
return 0
processed_dir = f"{udisc_path}processed/"
try:
webdav_client.mkdir(processed_dir, recursive=True)
except Exception:
pass
new_imports = 0
already_imported = set(
UDiscCSVImport.objects.filter(user_id=user_id).values_list(
"original_filename", flat=True
)
)
for fname in files:
fname = os.path.basename(fname)
if not fname.lower().endswith(".csv"):
continue
if fname == "processed":
continue
if fname in already_imported:
logger.debug(f"Skipping already-imported {fname}")
continue
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=fname)
try:
webdav_client.download_sync(
remote_path=f"{udisc_path}{fname}", local_path=tmp.name
)
imp = UDiscCSVImport.objects.create(
user_id=user_id,
original_filename=fname,
)
with open(tmp.name, "rb") as f:
imp.csv_file.save(fname, f, save=True)
stem, ext = os.path.splitext(fname)
ts = datetime.now().strftime("%Y%m%dT%H%M%S")
webdav_client.move(
f"{udisc_path}{fname}",
f"{processed_dir}{stem}_{ts}{ext}",
)
process_udisc_csv_import.delay(imp.id)
new_imports += 1
except Exception as e:
logger.error(f"Failed to import uDisc CSV file {fname}: {e}")
finally:
os.unlink(tmp.name)
if include_processed:
try:
processed_files = webdav_client.list(processed_dir)
except Exception as e:
logger.warning(
"Could not list var/udisc/processed/",
extra={"user_id": user_id, "error": str(e)},
)
return new_imports
for fname in processed_files:
fname = os.path.basename(fname)
if not fname.lower().endswith(".csv"):
continue
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=fname)
try:
webdav_client.download_sync(
remote_path=f"{processed_dir}{fname}", local_path=tmp.name
)
imp = UDiscCSVImport.objects.create(
user_id=user_id,
original_filename=fname,
)
with open(tmp.name, "rb") as f:
imp.csv_file.save(fname, f, save=True)
process_udisc_csv_import.delay(imp.id)
new_imports += 1
except Exception as e:
logger.error(
f"Failed to import processed uDisc CSV file {fname}: {e}"
)
finally:
os.unlink(tmp.name)
return new_imports

View File

@ -0,0 +1,518 @@
from mcp_server import MCPToolset
from scrobbles.models import Scrobble
from scrobbles.constants import LONG_PLAY_MEDIA
class ScrobbleToolset(MCPToolset):
def list_recent_scrobbles(
self,
days: int = 7,
media_type: str | None = None,
limit: int = 50,
) -> list[dict]:
"""List scrobbles from the last N days, optionally filtered by media type.
Valid media_type values: Video, Track, PodcastEpisode, SportEvent, Book,
Paper, VideoGame, BoardGame, GeoLocation, Trail, Beer, Puzzle, Food, Task,
WebPage, LifeEvent, Mood, BrickSet, Channel, BirdingLocation, DiscGolfCourse
"""
qs = (
Scrobble.objects.filter(user=self.request.user)
.select_related(
"video", "track", "book", "video_game", "board_game",
"beer", "puzzle", "food", "trail", "task", "web_page",
"life_event", "mood", "brick_set", "podcast_episode",
"sport_event", "geo_location", "birding_location",
"disc_golf_course", "channel",
)
.order_by("-timestamp")
)
from django.utils import timezone
import datetime
qs = qs.filter(timestamp__gte=timezone.now() - datetime.timedelta(days=days))
if media_type:
qs = qs.filter(media_type=media_type)
qs = qs[:limit]
return [_scrobble_to_dict(s) for s in qs]
def get_scrobble(self, uuid: str) -> dict | None:
"""Get a single scrobble by its UUID."""
try:
s = Scrobble.objects.filter(user=self.request.user).get(uuid=uuid)
except Scrobble.DoesNotExist:
return None
return _scrobble_to_dict(s)
def search_scrobbles(
self, query: str, media_type: str | None = None, limit: int = 20
) -> list[dict]:
"""Search scrobbles by text in their log data or related media titles."""
from django.db.models import Q
qs = Scrobble.objects.filter(user=self.request.user).order_by("-timestamp")
if media_type:
qs = qs.filter(media_type=media_type)
qs = qs.filter(
Q(log__icontains=query)
| Q(video__title__icontains=query)
| Q(track__title__icontains=query)
| Q(book__title__icontains=query)
| Q(video_game__title__icontains=query)
| Q(board_game__title__icontains=query)
| Q(beer__title__icontains=query)
| Q(food__title__icontains=query)
| Q(trail__title__icontains=query)
| Q(task__title__icontains=query)
| Q(web_page__title__icontains=query)
| Q(life_event__title__icontains=query)
| Q(puzzle__title__icontains=query)
| Q(brick_set__title__icontains=query)
| Q(podcast_episode__title__icontains=query)
)[:limit]
return [_scrobble_to_dict(s) for s in qs]
def get_scrobbles_by_date(
self, date: str, media_type: str | None = None
) -> list[dict]:
"""Get scrobbles for a specific date (YYYY-MM-DD format)."""
import datetime
try:
dt = datetime.datetime.strptime(date, "%Y-%m-%d").date()
except ValueError:
return []
qs = Scrobble.objects.filter(
user=self.request.user,
timestamp__date=dt,
).order_by("-timestamp")
if media_type:
qs = qs.filter(media_type=media_type)
return [_scrobble_to_dict(s) for s in qs]
def get_in_progress_scrobbles(
self, media_type: str | None = None
) -> list[dict]:
"""Get scrobbles currently in progress (started but not finished).
These are long-play items like books, video games, brick sets, or tasks."""
qs = Scrobble.objects.filter(
user=self.request.user,
in_progress=True,
).order_by("-timestamp")
if media_type:
qs = qs.filter(media_type=media_type)
return [_scrobble_to_dict(s) for s in qs]
def get_long_play_scrobbles(
self, status: str = "in_progress", media_type: str | None = None
) -> list[dict]:
"""Get long-play scrobbles (books, video games, brick sets, tasks).
Status can be 'in_progress' or 'completed'."""
types = list(LONG_PLAY_MEDIA.values())
qs = Scrobble.objects.filter(
user=self.request.user,
media_type__in=types,
).order_by("-timestamp")
if media_type:
qs = qs.filter(media_type=media_type)
if status == "in_progress":
qs = qs.filter(in_progress=True)
elif status == "completed":
qs = qs.filter(in_progress=False)
return [_scrobble_to_dict(s) for s in qs]
class MediaToolset(MCPToolset):
def get_book(self, uuid: str) -> dict | None:
"""Get a book by UUID."""
from books.models import Book
try:
b = Book.objects.get(uuid=uuid)
except Book.DoesNotExist:
return None
return _media_to_dict(b, fields=["title", "pages", "language",
"first_publish_year", "isbn_13",
"publisher", "summary"])
def list_books(self, author: str | None = None, limit: int = 20) -> list[dict]:
"""List books, optionally filtered by author name."""
from books.models import Book
qs = Book.objects.all().order_by("title")
if author:
qs = qs.filter(authors__name__icontains=author)
return [_media_to_dict(b, fields=["title", "pages", "language",
"first_publish_year", "isbn_13",
"publisher"]) for b in qs[:limit]]
def get_track(self, uuid: str) -> dict | None:
"""Get a music track by UUID."""
from music.models import Track
try:
t = Track.objects.select_related("artist_fk").get(uuid=uuid)
except Track.DoesNotExist:
return None
return _media_to_dict(t, fields=["title", "base_run_time_seconds",
"artist_fk__name", "genre"])
def list_tracks(self, artist: str | None = None, limit: int = 20) -> list[dict]:
"""List music tracks, optionally filtered by artist name."""
from music.models import Track
qs = Track.objects.select_related("artist_fk").all().order_by("title")
if artist:
qs = qs.filter(artist_fk__name__icontains=artist)
return [_media_to_dict(t, fields=["title", "base_run_time_seconds",
"artist_fk__name", "genre"])
for t in qs[:limit]]
def get_video(self, uuid: str) -> dict | None:
"""Get a video by UUID."""
from videos.models import Video
try:
v = Video.objects.select_related("tv_series", "channel").get(uuid=uuid)
except Video.DoesNotExist:
return None
return _media_to_dict(v, fields=["title", "year", "overview",
"imdb_id", "imdb_rating",
"tv_series__name", "channel__title",
"season_number", "episode_number"])
def list_videos(self, series: str | None = None, limit: int = 20) -> list[dict]:
"""List videos, optionally filtered by series name."""
from videos.models import Video
qs = Video.objects.select_related("tv_series", "channel").all().order_by("title")
if series:
qs = qs.filter(tv_series__name__icontains=series)
return [_media_to_dict(v, fields=["title", "year", "overview",
"tv_series__name", "channel__title",
"season_number", "episode_number"])
for v in qs[:limit]]
def get_board_game(self, uuid: str) -> dict | None:
"""Get a board game by UUID."""
from boardgames.models import BoardGame
try:
bg = BoardGame.objects.get(uuid=uuid)
except BoardGame.DoesNotExist:
return None
return _media_to_dict(bg, fields=["title", "genre"])
def list_board_games(self, limit: int = 20) -> list[dict]:
"""List board games."""
from boardgames.models import BoardGame
qs = BoardGame.objects.all().order_by("title")[:limit]
return [_media_to_dict(bg, fields=["title", "genre"]) for bg in qs]
def get_podcast_episode(self, uuid: str) -> dict | None:
"""Get a podcast episode by UUID."""
from podcasts.models import PodcastEpisode
try:
pe = PodcastEpisode.objects.select_related("podcast", "producer").get(
uuid=uuid
)
except PodcastEpisode.DoesNotExist:
return None
return _media_to_dict(pe, fields=["title", "podcast__title",
"producer__name", "base_run_time_seconds"])
def get_beer(self, uuid: str) -> dict | None:
"""Get a beer by UUID."""
from beers.models import Beer
try:
b = Beer.objects.select_related("style", "producer").get(uuid=uuid)
except Beer.DoesNotExist:
return None
return _media_to_dict(b, fields=["title", "style__name",
"producer__name", "abv"])
def get_brick_set(self, uuid: str) -> dict | None:
"""Get a brick set (LEGO) by UUID."""
from bricksets.models import BrickSet
try:
bs = BrickSet.objects.get(uuid=uuid)
except BrickSet.DoesNotExist:
return None
return _media_to_dict(bs, fields=["title", "piece_count", "set_number"])
def get_video_game(self, uuid: str) -> dict | None:
"""Get a video game by UUID."""
from videogames.models import VideoGame
try:
vg = VideoGame.objects.get(uuid=uuid)
except VideoGame.DoesNotExist:
return None
return _media_to_dict(vg, fields=["title", "genre",
"base_run_time_seconds"])
def get_puzzle(self, uuid: str) -> dict | None:
"""Get a puzzle by UUID."""
from puzzles.models import Puzzle
try:
p = Puzzle.objects.select_related("manufacturer").get(uuid=uuid)
except Puzzle.DoesNotExist:
return None
return _media_to_dict(p, fields=["title", "piece_count",
"manufacturer__name"])
def get_web_page(self, uuid: str) -> dict | None:
"""Get a web page by UUID."""
from webpages.models import WebPage
try:
wp = WebPage.objects.select_related("domain").get(uuid=uuid)
except WebPage.DoesNotExist:
return None
return _media_to_dict(wp, fields=["title", "url", "domain__name"])
def get_task(self, uuid: str) -> dict | None:
"""Get a task by UUID."""
from tasks.models import Task
try:
t = Task.objects.get(uuid=uuid)
except Task.DoesNotExist:
return None
return _media_to_dict(t, fields=["title", "completed"])
def get_trail(self, uuid: str) -> dict | None:
"""Get a trail by UUID."""
from trails.models import Trail
try:
t = Trail.objects.get(uuid=uuid)
except Trail.DoesNotExist:
return None
return _media_to_dict(t, fields=["title", "genre", "base_run_time_seconds"])
def get_geo_location(self, uuid: str) -> dict | None:
"""Get a geo location by UUID."""
from locations.models import GeoLocation
try:
gl = GeoLocation.objects.get(uuid=uuid)
except GeoLocation.DoesNotExist:
return None
return _media_to_dict(gl, fields=["title", "latitude", "longitude"])
def get_life_event(self, uuid: str) -> dict | None:
"""Get a life event by UUID."""
from lifeevents.models import LifeEvent
try:
le = LifeEvent.objects.get(uuid=uuid)
except LifeEvent.DoesNotExist:
return None
return _media_to_dict(le, fields=["title", "event_date", "genre"])
def get_mood(self, uuid: str) -> dict | None:
"""Get a mood entry by UUID."""
from moods.models import Mood
try:
m = Mood.objects.get(uuid=uuid)
except Mood.DoesNotExist:
return None
return _media_to_dict(m, fields=["title", "mood_type", "mood_quality"])
def get_food(self, uuid: str) -> dict | None:
"""Get a food entry by UUID."""
from foods.models import Food
try:
f = Food.objects.select_related("category").get(uuid=uuid)
except Food.DoesNotExist:
return None
return _media_to_dict(f, fields=["title", "category__name"])
def get_bird_sighting(self, uuid: str) -> dict | None:
"""Get a bird sighting by UUID."""
from birds.models import BirdSighting
try:
bs = BirdSighting.objects.select_related("bird").get(uuid=uuid)
except BirdSighting.DoesNotExist:
return None
return _media_to_dict(bs, fields=["title", "bird__common_name",
"bird__scientific_name", "location"])
def get_disc_golf_course(self, uuid: str) -> dict | None:
"""Get a disc golf course by UUID."""
from discgolf.models import DiscGolfCourse
try:
dg = DiscGolfCourse.objects.get(uuid=uuid)
except DiscGolfCourse.DoesNotExist:
return None
return _media_to_dict(dg, fields=["title", "holes", "location"])
class StatsToolset(MCPToolset):
def get_scrobble_counts(self, days: int = 30) -> list[dict]:
"""Get scrobble counts grouped by media type for the last N days."""
from django.utils import timezone
import datetime
from django.db.models import Count
cutoff = timezone.now() - datetime.timedelta(days=days)
qs = (
Scrobble.objects.filter(user=self.request.user, timestamp__gte=cutoff)
.values("media_type")
.annotate(count=Count("id"))
.order_by("-count")
)
return list(qs)
def get_top_media(
self, media_type: str, days: int = 30, limit: int = 10
) -> list[dict]:
"""Get the most-scrobbled items of a given media type in the last N days.
Valid media_type values: Video, Track, Book, BoardGame, Beer, etc."""
from django.utils import timezone
import datetime
from django.db.models import Count
cutoff = timezone.now() - datetime.timedelta(days=days)
rel_field = _media_type_to_rel_field(media_type)
if not rel_field:
return []
qs = (
Scrobble.objects.filter(
user=self.request.user,
media_type=media_type,
timestamp__gte=cutoff,
)
.values(rel_field)
.annotate(count=Count("id"))
.order_by("-count")
)[:limit]
results = []
for row in qs:
obj_id = row[rel_field]
if obj_id is None:
continue
results.append({"id": obj_id, "count": row["count"]})
return results
def _media_type_to_rel_field(media_type: str) -> str | None:
mapping = {
"Video": "video",
"Track": "track",
"PodcastEpisode": "podcast_episode",
"SportEvent": "sport_event",
"Book": "book",
"Paper": "paper",
"VideoGame": "video_game",
"BoardGame": "board_game",
"GeoLocation": "geo_location",
"Trail": "trail",
"Beer": "beer",
"Puzzle": "puzzle",
"Food": "food",
"Task": "task",
"WebPage": "web_page",
"LifeEvent": "life_event",
"Mood": "mood",
"BrickSet": "brick_set",
"Channel": "channel",
"BirdingLocation": "birding_location",
"DiscGolfCourse": "disc_golf_course",
}
return mapping.get(media_type)
def _scrobble_to_dict(s: Scrobble) -> dict:
result = {
"uuid": str(s.uuid),
"media_type": s.media_type,
"timestamp": s.timestamp.isoformat() if s.timestamp else None,
"stop_timestamp": s.stop_timestamp.isoformat() if s.stop_timestamp else None,
"in_progress": s.in_progress,
"played_to_completion": s.played_to_completion,
"source": s.source,
"visibility": s.visibility,
"timezone": s.timezone,
}
if s.log:
result["log"] = s.log
rel = _scrobble_related_to_dict(s)
if rel:
result["media"] = rel
return result
def _scrobble_related_to_dict(s: Scrobble) -> dict | None:
if s.video:
return _media_to_dict(s.video, fields=["title", "year", "imdb_id",
"imdb_rating"])
if s.track:
return _media_to_dict(s.track, fields=["title",
"base_run_time_seconds"])
if s.book:
return _media_to_dict(s.book, fields=["title", "pages"])
if s.video_game:
return _media_to_dict(s.video_game, fields=["title",
"base_run_time_seconds"])
if s.board_game:
return _media_to_dict(s.board_game, fields=["title"])
if s.beer:
return _media_to_dict(s.beer, fields=["title"])
if s.puzzle:
return _media_to_dict(s.puzzle, fields=["title", "piece_count"])
if s.food:
return _media_to_dict(s.food, fields=["title"])
if s.trail:
return _media_to_dict(s.trail, fields=["title",
"base_run_time_seconds"])
if s.task:
return _media_to_dict(s.task, fields=["title"])
if s.web_page:
return _media_to_dict(s.web_page, fields=["title", "url"])
if s.life_event:
return _media_to_dict(s.life_event, fields=["title", "event_date"])
if s.mood:
return _media_to_dict(s.mood, fields=["title"])
if s.brick_set:
return _media_to_dict(s.brick_set, fields=["title", "set_number"])
if s.podcast_episode:
return _media_to_dict(s.podcast_episode, fields=["title"])
if s.sport_event:
return {"title": str(s.sport_event)}
if s.geo_location:
return _media_to_dict(s.geo_location, fields=["title", "latitude",
"longitude"])
if s.birding_location:
return _media_to_dict(s.birding_location, fields=["title"])
if s.disc_golf_course:
return _media_to_dict(s.disc_golf_course, fields=["title", "holes"])
if s.channel:
return _media_to_dict(s.channel, fields=["title"])
return None
def _media_to_dict(obj, fields: list[str] | None = None) -> dict:
if obj is None:
return {}
result = {}
if hasattr(obj, "uuid"):
result["uuid"] = str(obj.uuid)
if hasattr(obj, "title"):
result["title"] = obj.title
if fields is None:
return result
resolved = _resolve_fields(obj, fields)
for k, v in resolved.items():
if k not in result:
result[k] = v
return result
def _resolve_fields(obj, fields: list[str]) -> dict:
result = {}
for field in fields:
parts = field.split("__")
val = obj
try:
for part in parts:
val = getattr(val, part)
except AttributeError:
continue
if val is not None:
if hasattr(val, "all"):
val = [str(v) for v in val.all()]
result[field] = val
return result

View File

@ -0,0 +1,156 @@
# Generated by Django 4.2.29 on 2026-06-20 04:19
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
import django_extensions.db.fields
import scrobbles.models
import uuid
class Migration(migrations.Migration):
dependencies = [
("discgolf", "0001_initial"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("scrobbles", "0098_scrobble_long_play_last_scrobble"),
]
operations = [
migrations.AddField(
model_name="favoritemedia",
name="disc_golf",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
to="discgolf.discgolfcourse",
),
),
migrations.AddField(
model_name="scrobble",
name="disc_golf",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.DO_NOTHING,
to="discgolf.discgolfcourse",
),
),
migrations.AlterField(
model_name="favoritemedia",
name="media_type",
field=models.CharField(
choices=[
("Video", "Video"),
("Track", "Track"),
("PodcastEpisode", "Podcast episode"),
("SportEvent", "Sport event"),
("Book", "Book"),
("Paper", "Paper"),
("VideoGame", "Video game"),
("BoardGame", "Board game"),
("GeoLocation", "GeoLocation"),
("Trail", "Trail"),
("Beer", "Beer"),
("Puzzle", "Puzzle"),
("Food", "Food"),
("Task", "Task"),
("WebPage", "Web Page"),
("LifeEvent", "Life event"),
("Mood", "Mood"),
("BrickSet", "Brick set"),
("Channel", "Channel"),
("DiscGolf", "Disc golf"),
],
max_length=20,
),
),
migrations.AlterField(
model_name="scrobble",
name="media_type",
field=models.CharField(
choices=[
("Video", "Video"),
("Track", "Track"),
("PodcastEpisode", "Podcast episode"),
("SportEvent", "Sport event"),
("Book", "Book"),
("Paper", "Paper"),
("VideoGame", "Video game"),
("BoardGame", "Board game"),
("GeoLocation", "GeoLocation"),
("Trail", "Trail"),
("Beer", "Beer"),
("Puzzle", "Puzzle"),
("Food", "Food"),
("Task", "Task"),
("WebPage", "Web Page"),
("LifeEvent", "Life event"),
("Mood", "Mood"),
("BrickSet", "Brick set"),
("Channel", "Channel"),
("DiscGolf", "Disc golf"),
],
default="Video",
max_length=20,
),
),
migrations.CreateModel(
name="UDiscCSVImport",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
django_extensions.db.fields.CreationDateTimeField(
auto_now_add=True, verbose_name="created"
),
),
(
"modified",
django_extensions.db.fields.ModificationDateTimeField(
auto_now=True, verbose_name="modified"
),
),
("uuid", models.UUIDField(default=uuid.uuid4, editable=False)),
("processing_started", models.DateTimeField(blank=True, null=True)),
("processed_finished", models.DateTimeField(blank=True, null=True)),
("process_log", models.TextField(blank=True, null=True)),
("process_count", models.IntegerField(blank=True, null=True)),
("error_log", models.TextField(blank=True, null=True)),
(
"csv_file",
models.FileField(
blank=True,
null=True,
upload_to=scrobbles.models.UDiscCSVImport.get_path,
),
),
(
"original_filename",
models.CharField(blank=True, max_length=255, null=True),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.DO_NOTHING,
related_name="scrobbles_udisccsvimport_set",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"verbose_name": "uDisc CSV Import",
},
),
]

View File

@ -0,0 +1,84 @@
# Generated by Django 4.2.29 on 2026-06-20 04:53
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("scrobbles", "0099_favoritemedia_disc_golf_scrobble_disc_golf_and_more"),
]
operations = [
migrations.RenameField(
model_name="favoritemedia",
old_name="disc_golf",
new_name="disc_golf_course",
),
migrations.RenameField(
model_name="scrobble",
old_name="disc_golf",
new_name="disc_golf_course",
),
migrations.AlterField(
model_name="favoritemedia",
name="media_type",
field=models.CharField(
choices=[
("Video", "Video"),
("Track", "Track"),
("PodcastEpisode", "Podcast episode"),
("SportEvent", "Sport event"),
("Book", "Book"),
("Paper", "Paper"),
("VideoGame", "Video game"),
("BoardGame", "Board game"),
("GeoLocation", "GeoLocation"),
("Trail", "Trail"),
("Beer", "Beer"),
("Puzzle", "Puzzle"),
("Food", "Food"),
("Task", "Task"),
("WebPage", "Web Page"),
("LifeEvent", "Life event"),
("Mood", "Mood"),
("BrickSet", "Brick set"),
("Channel", "Channel"),
("BirdingLocation", "Birding location"),
("DiscGolfCourse", "Disc golf"),
],
max_length=20,
),
),
migrations.AlterField(
model_name="scrobble",
name="media_type",
field=models.CharField(
choices=[
("Video", "Video"),
("Track", "Track"),
("PodcastEpisode", "Podcast episode"),
("SportEvent", "Sport event"),
("Book", "Book"),
("Paper", "Paper"),
("VideoGame", "Video game"),
("BoardGame", "Board game"),
("GeoLocation", "GeoLocation"),
("Trail", "Trail"),
("Beer", "Beer"),
("Puzzle", "Puzzle"),
("Food", "Food"),
("Task", "Task"),
("WebPage", "Web Page"),
("LifeEvent", "Life event"),
("Mood", "Mood"),
("BrickSet", "Brick set"),
("Channel", "Channel"),
("BirdingLocation", "Birding location"),
("DiscGolfCourse", "Disc golf"),
],
default="Video",
max_length=20,
),
),
]

View File

@ -114,7 +114,7 @@ class ScrobblableMixin(TimeStampedModel):
@property
def start_url(self):
return reverse("scrobbles:start", kwargs={"uuid": self.uuid})
return reverse("scrobbles:start", kwargs={"media_uuid": self.uuid})
@property
def strings(self) -> ScrobblableConstants:
@ -162,7 +162,7 @@ class LongPlayScrobblableMixin(ScrobblableMixin):
return False
def get_longplay_finish_url(self):
return reverse("scrobbles:longplay-finish", kwargs={"uuid": self.uuid})
return reverse("scrobbles:longplay-finish", kwargs={"media_uuid": self.uuid})
def first_long_play_scrobble_for_user(self, user) -> Optional["Scrobble"]:
last = self.last_long_play_scrobble_for_user(user)

View File

@ -11,6 +11,7 @@ import pendulum
import pytz
from beers.models import Beer
from birds.models import BirdingLocation
from discgolf.models import DiscGolfCourse
from boardgames.models import BoardGame
from books.koreader import process_koreader_sqlite_file
from books.models import Book, BookLogData, BookPageLogData, Paper
@ -617,6 +618,60 @@ class EBirdCSVImport(BaseFileImportMixin):
self.mark_finished()
class UDiscCSVImport(BaseFileImportMixin):
class Meta:
verbose_name = "uDisc CSV Import"
user = models.ForeignKey(
User,
on_delete=models.DO_NOTHING,
**BNULL,
related_name="scrobbles_udisccsvimport_set",
)
@property
def import_type(self) -> str:
return "uDisc"
def get_absolute_url(self):
return reverse("scrobbles:udisc-csv-import-detail", kwargs={"slug": self.uuid})
def get_path(instance, filename):
extension = filename.split(".")[-1]
uuid = instance.uuid
return f"udisc-csv-uploads/{uuid}.{extension}"
@property
def upload_file_path(self):
if getattr(settings, "USE_S3_STORAGE"):
path = self.csv_file.url
else:
path = self.csv_file.path
return path
csv_file = models.FileField(upload_to=get_path, **BNULL)
original_filename = models.CharField(max_length=255, **BNULL)
def process(self, force=False):
from discgolf.utils import import_udisc_csv
if self.processed_finished and not force:
logger.info(f"{self} already processed on {self.processed_finished}")
return
self.mark_started()
try:
scrobbles = import_udisc_csv(
self.upload_file_path, self.user_id, record_error=self.record_error
)
self.record_log(scrobbles)
except Exception as e:
self.record_error(f"Import failed: {e}")
logger.exception(f"Import failed for {self}")
finally:
self.mark_finished()
TYPE_FK_PREFETCHES: dict[str, tuple[str, ...]] = {
"Video": ("video",),
"Track": ("track", "track__artist_fk"),
@ -638,6 +693,7 @@ TYPE_FK_PREFETCHES: dict[str, tuple[str, ...]] = {
"BrickSet": ("brick_set",),
"Channel": ("channel",),
"BirdingLocation": ("birding_location",),
"DiscGolfCourse": ("disc_golf_course",),
}
@ -665,6 +721,7 @@ class ScrobbleQuerySet(models.QuerySet):
"mood",
"brick_set",
"birding_location",
"disc_golf_course",
)
def with_related_for_types(self, media_types: list[str]):
@ -715,6 +772,7 @@ class Scrobble(TimeStampedModel):
BRICKSET = "BrickSet", "Brick set"
CHANNEL = "Channel", "Channel"
BIRDING_LOCATION = "BirdingLocation", "Birding location"
DISC_GOLF = "DiscGolfCourse", "Disc golf"
@classmethod
def list(cls):
@ -745,6 +803,9 @@ class Scrobble(TimeStampedModel):
birding_location = models.ForeignKey(
BirdingLocation, on_delete=models.DO_NOTHING, **BNULL
)
disc_golf_course = models.ForeignKey(
DiscGolfCourse, on_delete=models.DO_NOTHING, **BNULL
)
media_type = models.CharField(
max_length=20, choices=MediaType.choices, default=MediaType.VIDEO
)
@ -798,6 +859,12 @@ class Scrobble(TimeStampedModel):
format="JPEG",
options={"quality": 75},
)
screenshot_large = ImageSpecField(
source="screenshot",
processors=[ResizeToFit(800, 800)],
format="JPEG",
options={"quality": 85},
)
long_play_seconds = models.BigIntegerField(**BNULL)
long_play_complete = models.BooleanField(**BNULL)
long_play_last_scrobble = models.ForeignKey(
@ -900,7 +967,7 @@ class Scrobble(TimeStampedModel):
@property
def finish_url(self) -> str:
return reverse("scrobbles:finish", kwargs={"uuid": self.uuid})
return reverse("scrobbles:finish", kwargs={"pk": self.pk})
def save(self, *args, **kwargs):
class_name = self.media_obj.__class__.__name__
@ -942,10 +1009,7 @@ class Scrobble(TimeStampedModel):
return super(Scrobble, self).save(*args, **kwargs)
def get_absolute_url(self):
if not self.uuid:
self.uuid = uuid4()
self.save(update_fields=["uuid"])
return reverse("scrobbles:detail", kwargs={"uuid": self.uuid})
return reverse("scrobbles:detail", kwargs={"pk": self.pk})
def get_share_url(self):
if self.visibility == Visibility.PRIVATE:
@ -1313,6 +1377,10 @@ class Scrobble(TimeStampedModel):
media_obj = self.channel
if self.birding_location:
media_obj = self.birding_location
if self.paper:
media_obj = self.paper
if self.disc_golf_course:
media_obj = self.disc_golf_course
return media_obj
def __str__(self):
@ -1871,6 +1939,9 @@ class FavoriteMedia(TimeStampedModel):
birding_location = models.ForeignKey(
BirdingLocation, on_delete=models.CASCADE, **BNULL
)
disc_golf_course = models.ForeignKey(
DiscGolfCourse, on_delete=models.CASCADE, **BNULL
)
media_type = models.CharField(max_length=20, choices=Scrobble.MediaType.choices)
sent_to_mopidy = models.BooleanField(default=False)
@ -1921,6 +1992,8 @@ class FavoriteMedia(TimeStampedModel):
media_obj = self.channel
if self.birding_location:
media_obj = self.birding_location
if self.disc_golf_course:
media_obj = self.disc_golf_course
return media_obj
@classmethod
@ -1950,6 +2023,7 @@ class FavoriteMedia(TimeStampedModel):
"Mood": "mood",
"BrickSet": "brick_set",
"BirdingLocation": "birding_location",
"DiscGolfCourse": "disc_golf_course",
}
fk = fk_map.get(media_type)

View File

@ -9,10 +9,11 @@ import requests
from beers.models import Beer
from boardgames.models import BoardGame, BoardGameDesigner, BoardGameLocation
from books.constants import READCOMICSONLINE_URL
from books.models import Book, BookLogData, BookPageLogData
from books.models import Book, BookLogData, BookPageLogData, Paper
from books.utils import parse_readcomicsonline_uri
from bricksets.models import BrickSet
from dateutil.parser import parse
from discgolf.models import DiscGolfCourse
from django.utils import timezone
from foods.models import Food
from foods.sources.rscraper import RecipeScraperService
@ -330,8 +331,6 @@ def manual_scrobble_book(
source = READCOMICSONLINE_URL.replace("https://", "")
# TODO: Check for scrobble of this book already and if so, update the page count
book = Book.find_or_create(title, url=url, enrich=True)
scrobble_dict = {
@ -642,6 +641,8 @@ def manual_scrobble_from_url(
item_id = "tt" + str(item_id)
elif content_key == "-h" and "twitch.tv" in url:
item_id = url
elif content_key == "-pp" and "doi.org" in url:
item_id = url
scrobble_fn = MANUAL_SCROBBLE_FNS[content_key]
return eval(scrobble_fn)(item_id, user_id, source=source, action=action)
@ -996,6 +997,38 @@ def manual_scrobble_task(
return scrobble
def manual_scrobble_paper(
doi_url: str,
user_id: int,
source: str = "Bookmarklet",
action: Optional[str] = None,
):
paper = Paper.find_or_create_by_doi(doi_url)
scrobble_dict = {
"user_id": user_id,
"timestamp": timezone.now(),
"playback_position_seconds": 0,
"source": source,
}
logger.info(
"[vrobbler-scrobble] paper scrobble request received",
extra={
"paper_id": paper.id,
"user_id": user_id,
"scrobble_dict": scrobble_dict,
"media_type": Scrobble.MediaType.PAPER,
},
)
scrobble = Scrobble.create_or_update(paper, user_id, scrobble_dict)
if action == "stop":
scrobble.stop(force_finish=True)
return scrobble
def manual_scrobble_webpage(
url: str,
user_id: int,
@ -1329,3 +1362,32 @@ def manual_scrobble_food(
)
return Scrobble.create_or_update(food, user_id, scrobble_dict)
def manual_scrobble_discgolf(
item_id: str,
user_id: int,
source: str = "Vrobbler",
action: Optional[str] = None,
):
from discgolf.models import DiscGolfCourse
course, _ = DiscGolfCourse.objects.get_or_create(title=item_id.strip())
scrobble_dict = {
"user_id": user_id,
"timestamp": timezone.now(),
"playback_position_seconds": 0,
"source": source,
}
logger.info(
"[scrobblers] manual disc golf scrobble request received",
extra={
"course_id": course.id,
"user_id": user_id,
"scrobble_dict": scrobble_dict,
},
)
return Scrobble.create_or_update(course, user_id, scrobble_dict)

View File

@ -1,5 +1,6 @@
import logging
from django.core.cache import cache
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
from django.utils import timezone
@ -9,6 +10,7 @@ from scrobbles.tasks import (
add_favorite_to_mopidy_playlist,
CHARTABLE_MEDIA_TYPES,
remove_favorite_from_mopidy_playlist,
reverse_geocode_geolocation,
SCROBBLES_WITHOUT_CHARTS,
update_charts_for_timestamp,
)
@ -52,6 +54,11 @@ def _update_charts_for_timestamp(user, ts):
if ts is None:
return
lock_key = f"chart_update_{user.id}"
if not cache.add(lock_key, "locked", timeout=30):
logger.info(f"Chart update already queued for user {user.id}, skipping")
return
if timezone.is_naive(ts):
ts = timezone.make_aware(ts)
@ -80,6 +87,31 @@ def add_tags_from_task_title(sender, instance, **kwargs):
instance.tags.add(tag)
@receiver(post_save, sender=Scrobble)
def reverse_geocode_on_scrobble_creation(sender, instance, created, **kwargs):
if not created:
return
if not instance.geo_location_id:
logger.info(
"Skipping reverse geocode: scrobble %s has no geo_location",
instance.id,
)
return
if instance.geo_location.postal_code:
logger.info(
"Skipping reverse geocode: geo_location %s already has postal_code %s",
instance.geo_location_id,
instance.geo_location.postal_code,
)
return
logger.info(
"Enqueuing reverse geocode for geo_location %s",
instance.geo_location_id,
)
reverse_geocode_geolocation.delay(instance.geo_location_id)
@receiver(post_save, sender=FavoriteMedia)
def add_to_mopidy_playlist_on_favorite(sender, instance, created, **kwargs):
if not created:

View File

@ -12,6 +12,7 @@ from charts.utils import (
from django.apps import apps
from django.conf import settings
from django.contrib.auth import get_user_model
from django.core.cache import cache
from django.db import models
from django.utils import timezone
@ -170,6 +171,16 @@ def process_ebird_csv_import(import_id):
birding_import.process()
@shared_task
def process_udisc_csv_import(import_id):
UDiscCSVImport = apps.get_model("scrobbles", "UDiscCSVImport")
udisc_import = UDiscCSVImport.objects.filter(id=import_id).first()
if not udisc_import:
logger.warn(f"UDiscCSVImport not found with id {import_id}")
return
udisc_import.process()
@shared_task
def process_scale_csv_import(import_id):
ScaleCSVImport = apps.get_model("scrobbles", "ScaleCSVImport")
@ -241,6 +252,11 @@ def update_charts_for_timestamp(user_id, year, month, day, week):
logger.error(f"User with id {user_id} not found")
return
lock_key = f"chart_update_running_{user_id}"
if not cache.add(lock_key, "locked", timeout=300):
logger.info(f"Chart update already running for user {user_id}, skipping")
return
try:
build_daily_charts(user, year, month, day, CHARTABLE_MEDIA_TYPES)
build_weekly_charts(user, year, week, CHARTABLE_MEDIA_TYPES)
@ -250,6 +266,8 @@ def update_charts_for_timestamp(user_id, year, month, day, week):
logger.info(f"[charts] Updated charts for {user} on {date_str}")
except Exception as e:
logger.error(f"[charts] Failed to update charts: {e}")
finally:
cache.delete(lock_key)
@shared_task
@ -708,3 +726,41 @@ def add_scrobble_to_mopidy_monthly_playlist(scrobble_id):
break
add_track_to_mopidy_monthly_playlist(scrobble)
@shared_task
def reverse_geocode_geolocation(geo_location_id):
from locations.models import GeoLocation
location = GeoLocation.objects.filter(id=geo_location_id).first()
if not location:
logger.info(
"Skipping reverse geocode: geo_location %s not found",
geo_location_id,
)
return
if location.postal_code:
logger.info(
"Skipping reverse geocode: geo_location %s already has postal_code %s",
geo_location_id,
location.postal_code,
)
return
logger.info(
"Reverse geocoding geo_location %s (%s, %s)",
geo_location_id,
location.lat,
location.lon,
)
if location.reverse_geocode():
logger.info(
"Reverse geocode succeeded for geo_location %s: %s",
geo_location_id,
location.display_address,
)
else:
logger.warning(
"Reverse geocode failed for geo_location %s",
geo_location_id,
)

View File

@ -5,6 +5,7 @@ from tasks.webhooks import EmacsWebhookView, TodoistWebhookView
app_name = "scrobbles"
urlpatterns = [
path("now-playing/", views.NowPlayingPartialView.as_view(), name="now-playing-partial"),
path("calendar/", views.ScrobbleCalendarView.as_view(), name="calendar"),
path("search/", views.ScrobbleSearchView.as_view(), name="search"),
path("status/", views.ScrobbleStatusView.as_view(), name="status"),
@ -44,7 +45,7 @@ urlpatterns = [
name="lookup-manual-scrobble",
),
path(
"long-play-finish/<slug:uuid>/",
"long-play-finish/<slug:media_uuid>/",
views.scrobble_longplay_finish,
name="longplay-finish",
),
@ -147,6 +148,11 @@ urlpatterns = [
views.ScrobbleBirdingCSVImportDetailView.as_view(),
name="ebird-csv-import-detail",
),
path(
"imports/udisc-csv/<slug:slug>/",
views.ScrobbleUDiscCSVImportDetailView.as_view(),
name="udisc-csv-import-detail",
),
path(
"long-plays/",
views.ScrobbleLongPlaysView.as_view(),
@ -160,38 +166,38 @@ urlpatterns = [
name="shared-detail",
),
path(
"scrobbles/<slug:uuid>/",
"scrobbles/<int:pk>/",
views.ScrobbleDetailView.as_view(),
name="detail",
),
path(
"scrobbles/<slug:uuid>/regenerate-share-token/",
"scrobbles/<int:pk>/regenerate-share-token/",
views.RegenerateShareTokenView.as_view(),
name="regenerate-share-token",
),
path(
"scrobbles/<slug:uuid>/change-visibility/",
"scrobbles/<int:pk>/change-visibility/",
views.ChangeVisibilityView.as_view(),
name="change-visibility",
),
path(
"scrobbles/<slug:uuid>/share-analytics/",
"scrobbles/<int:pk>/share-analytics/",
views.ScrobbleShareAnalyticsView.as_view(),
name="share-analytics",
),
path(
"scrobbles/<slug:uuid>/add-to-mopidy-queue/",
"scrobbles/<int:pk>/add-to-mopidy-queue/",
views.add_to_mopidy_queue,
name="add-to-mopidy-queue",
),
path(
"scrobbles/<slug:uuid>/add-to-mopidy-monthly-playlist/",
"scrobbles/<int:pk>/add-to-mopidy-monthly-playlist/",
views.add_to_mopidy_monthly_playlist,
name="add-to-mopidy-monthly-playlist",
),
path("scrobbles/<slug:uuid>/start/", views.scrobble_start, name="start"),
path("scrobbles/<slug:uuid>/finish/", views.scrobble_finish, name="finish"),
path("scrobbles/<slug:uuid>/cancel/", views.scrobble_cancel, name="cancel"),
path("scrobbles/<slug:media_uuid>/start/", views.scrobble_start, name="start"),
path("scrobbles/<int:pk>/finish/", views.scrobble_finish, name="finish"),
path("scrobbles/<int:pk>/cancel/", views.scrobble_cancel, name="cancel"),
path(
"favorite/<str:media_type>/<int:object_id>/toggle/",
views.toggle_favorite,

View File

@ -1,4 +1,5 @@
import hashlib
import html
import logging
import requests
import re
@ -153,10 +154,11 @@ def import_lastfm_for_all_users(restart=False):
last_processed = lfm_import.processed_finished
else:
logger.info(
f"Not resuming failed LastFM import {lfm_import.id} for user {user_id}, use restart=True to restart"
"No existing LastFM import, we should start a monthly parsing of lastFm for this user going back to 2002"
"No existing LastFM import for user %s, "
"starting a full parse",
user_id,
)
continue
last_processed = None
lfm_client = LastFM(user=get_user_model().objects.filter(id=user_id).first())
@ -795,6 +797,7 @@ def tokenize_title_to_tags(title: str) -> list[str]:
if not title:
return []
title = html.unescape(title)
cleaned = re.sub(r"[\(\)\[\]\{\}]", "", title)
cleaned = re.sub(r"[^\w\s]", "", cleaned)

View File

@ -87,6 +87,7 @@ from scrobbles.models import (
ScrobbleQuerySet,
ShareViewLog,
TrailGPXImport,
UDiscCSVImport,
)
from scrobbles.scrobblers import *
from scrobbles.tasks import (
@ -360,11 +361,35 @@ class RecentScrobbleList(ListView):
return Scrobble.objects.all().order_by("-timestamp")
class NowPlayingPartialView(LoginRequiredMixin, TemplateView):
template_name = "scrobbles/_now_playing.html"
def get_context_data(self, **kwargs):
ctx = super().get_context_data(**kwargs)
from scrobbles.constants import EXCLUDE_FROM_NOW_PLAYING
ctx["now_playing_list"] = list(
Scrobble.objects.filter(
in_progress=True,
is_paused=False,
user=self.request.user,
)
.exclude(media_type__in=EXCLUDE_FROM_NOW_PLAYING)
.select_related("track", "video", "podcast_episode")
)
return ctx
class ScrobbleListView(LoginRequiredMixin, ListView):
model = Scrobble
paginate_by = 100
template_name = "scrobbles/scrobble_all_list.html"
def get_template_names(self):
if self.request.headers.get("HX-Request"):
return ["scrobbles/_scrobble_all_content.html"]
return ["scrobbles/scrobble_all_list.html"]
def get_queryset(self):
qs = Scrobble.objects.filter(user=self.request.user).order_by("-timestamp")
tags_param = self.request.GET.get("tags", "")
@ -535,6 +560,8 @@ class BaseScrobbleImportDetailView(DetailView):
title = "Scale CSV Import"
if self.model == TrailGPXImport:
title = "Trail GPX Import"
if self.model == UDiscCSVImport:
title = "uDisc CSV Import"
context_data["title"] = title
return context_data
@ -571,6 +598,10 @@ class ScrobbleBirdingCSVImportDetailView(BaseScrobbleImportDetailView):
model = EBirdCSVImport
class ScrobbleUDiscCSVImportDetailView(BaseScrobbleImportDetailView):
model = UDiscCSVImport
class ManualScrobbleView(FormView):
form_class = ScrobbleForm
template_name = "scrobbles/manual_form.html"
@ -579,7 +610,12 @@ class ManualScrobbleView(FormView):
item_str = form.cleaned_data.get("item_id")
logger.debug(f"Looking for scrobblable media with input {item_str}")
key, item_id = item_str[:2], item_str[3:]
if len(item_str) > 2 and item_str[:3] in MANUAL_SCROBBLE_FNS:
key = item_str[:3]
item_id = item_str[4:]
else:
key = item_str[:2]
item_id = item_str[3:]
scrobble_fn = MANUAL_SCROBBLE_FNS[key]
scrobble = eval(scrobble_fn)(item_id, self.request.user.id)
@ -839,10 +875,10 @@ def import_audioscrobbler_file(request):
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_start(request, uuid):
def scrobble_start(request, media_uuid):
logger.info(
"[scrobble_start] called",
extra={"request": request, "uuid": uuid},
extra={"request": request, "media_uuid": media_uuid},
)
user = request.user
success_url = request.META.get("HTTP_REFERER")
@ -853,14 +889,14 @@ def scrobble_start(request, uuid):
media_obj = None
for app, model in PLAY_AGAIN_MEDIA.items():
media_model = apps.get_model(app_label=app, model_name=model)
media_obj = media_model.objects.filter(uuid=uuid).first()
media_obj = media_model.objects.filter(uuid=media_uuid).first()
if media_obj:
break
if not media_obj:
logger.info(
"[scrobble_start] media object not found",
extra={"uuid": uuid, "user_id": user.id},
extra={"media_uuid": media_uuid, "user_id": user.id},
)
raise Exception("No media object provided to scrobble")
@ -887,7 +923,7 @@ def scrobble_start(request, uuid):
if last_scrobble and last_scrobble.logdata:
next_page = last_scrobble.logdata.page_end + 1
log_data = {"page_start": next_page}
media_obj.scrobble_for_user(user_id, log=log_data)
scrobble = media_obj.scrobble_for_user(user_id, log=log_data)
if scrobble:
messages.add_message(
@ -897,7 +933,7 @@ def scrobble_start(request, uuid):
)
else:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
if (
@ -915,7 +951,7 @@ def scrobble_start(request, uuid):
@api_view(["GET"])
def scrobble_longplay_finish(request, uuid):
def scrobble_longplay_finish(request, media_uuid):
user = request.user
success_url = request.META.get("HTTP_REFERER")
@ -923,7 +959,7 @@ def scrobble_longplay_finish(request, uuid):
return HttpResponseRedirect(success_url)
# Try scrobble UUID first
scrobble = Scrobble.objects.filter(uuid=uuid, user=user).first()
scrobble = Scrobble.objects.filter(uuid=media_uuid, user=user).first()
if scrobble:
if scrobble.long_play_complete == True:
scrobble.long_play_complete = None
@ -947,13 +983,13 @@ def scrobble_longplay_finish(request, uuid):
media_obj = None
for app, model in LONG_PLAY_MEDIA.items():
media_model = apps.get_model(app_label=app, model_name=model)
media_obj = media_model.objects.filter(uuid=uuid).first()
media_obj = media_model.objects.filter(uuid=media_uuid).first()
if media_obj:
break
if not media_obj:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
return HttpResponseRedirect(success_url)
@ -976,14 +1012,14 @@ def scrobble_longplay_finish(request, uuid):
)
else:
messages.add_message(
request, messages.ERROR, f"Media with uuid {uuid} not found."
request, messages.ERROR, f"Media with uuid {media_uuid} not found."
)
return HttpResponseRedirect(success_url)
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_finish(request, uuid):
def scrobble_finish(request, pk):
user = request.user
success_url = request.META.get("HTTP_REFERER")
if not success_url:
@ -992,7 +1028,7 @@ def scrobble_finish(request, uuid):
if not user.is_authenticated:
return HttpResponseRedirect(success_url)
scrobble = Scrobble.objects.filter(user=user, uuid=uuid).first()
scrobble = Scrobble.objects.filter(user=user, pk=pk).first()
if scrobble:
scrobble.stop(force_finish=True)
messages.add_message(
@ -1007,14 +1043,14 @@ def scrobble_finish(request, uuid):
@api_view(["GET"])
@permission_classes([IsAuthenticated])
def scrobble_cancel(request, uuid):
def scrobble_cancel(request, pk):
user = request.user
success_url = reverse_lazy("vrobbler-home")
if not user.is_authenticated:
return HttpResponseRedirect(success_url)
scrobble = Scrobble.objects.filter(user=user, uuid=uuid).first()
scrobble = Scrobble.objects.filter(user=user, pk=pk).first()
if scrobble:
scrobble.cancel()
messages.add_message(
@ -1028,11 +1064,11 @@ def scrobble_cancel(request, uuid):
@require_POST
def add_to_mopidy_queue(request, uuid):
def add_to_mopidy_queue(request, pk):
if not request.user.is_authenticated:
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
mopidy_url = request.user.profile.mopidy_api_url
if not mopidy_url:
@ -1041,22 +1077,22 @@ def add_to_mopidy_queue(request, uuid):
messages.ERROR,
"Mopidy API URL not configured in your profile settings.",
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
from scrobbles.tasks import add_scrobble_to_mopidy_queue as task
task.delay(scrobble.id)
msg = f'Adding "{scrobble.media_obj}" to Mopidy queue.'
messages.add_message(request, messages.SUCCESS, msg)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
@require_POST
def add_to_mopidy_monthly_playlist(request, uuid):
def add_to_mopidy_monthly_playlist(request, pk):
if not request.user.is_authenticated:
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
profile = request.user.profile
pattern = profile.monthly_mopidy_playlist_pattern
@ -1066,7 +1102,7 @@ def add_to_mopidy_monthly_playlist(request, uuid):
messages.ERROR,
"Monthly playlist pattern or Mopidy API URL not configured in your profile.",
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
now = now_user_timezone(profile)
playlist_name = DateFormat(now).format(pattern)
@ -1079,7 +1115,7 @@ def add_to_mopidy_monthly_playlist(request, uuid):
messages.SUCCESS,
f'Adding "{scrobble.media_obj}" to monthly playlist "{playlist_name}".',
)
return redirect("scrobbles:detail", uuid=uuid)
return redirect("scrobbles:detail", pk=pk)
@require_POST
@ -1108,6 +1144,7 @@ def toggle_favorite(request, media_type, object_id):
"Mood": ("moods", "Mood"),
"BrickSet": ("bricksets", "BrickSet"),
"BirdingLocation": ("birds", "BirdingLocation"),
"DiscGolfCourse": ("discgolf", "DiscGolfCourse"),
}
app_label, model_name = app_model_map.get(media_type, (None, None))
@ -1184,8 +1221,6 @@ class ScrobbleStatusView(LoginRequiredMixin, TemplateView):
class ScrobbleDetailView(DetailView):
model = Scrobble
slug_field = "uuid"
slug_url_kwarg = "uuid"
paginate_by = 100
def get_object(self, queryset=None):
@ -1385,15 +1420,15 @@ class ScrobbleExploreView(ListView):
class RegenerateShareTokenView(LoginRequiredMixin, View):
def post(self, request, uuid):
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
def post(self, request, pk):
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
scrobble.regenerate_share_token()
return redirect(scrobble.get_absolute_url())
class ChangeVisibilityView(LoginRequiredMixin, View):
def post(self, request, uuid):
scrobble = get_object_or_404(Scrobble, uuid=uuid, user=request.user)
def post(self, request, pk):
scrobble = get_object_or_404(Scrobble, pk=pk, user=request.user)
visibility = request.POST.get("visibility")
if visibility not in (Visibility.PUBLIC, Visibility.SHARED, Visibility.PRIVATE):
return redirect(scrobble.get_absolute_url())
@ -1404,8 +1439,6 @@ class ChangeVisibilityView(LoginRequiredMixin, View):
class ScrobbleShareAnalyticsView(LoginRequiredMixin, DetailView):
model = Scrobble
slug_field = "uuid"
slug_url_kwarg = "uuid"
template_name = "scrobbles/scrobble_share_analytics.html"
def get_queryset(self):
@ -1721,6 +1754,7 @@ class ScrobbleCalendarView(LoginRequiredMixin, TemplateView):
for scrobble in day_map[day_num]:
day_scrobbles.append(
{
"id": scrobble.pk,
"uuid": scrobble.uuid,
"emoji": self.MEDIA_EMOJI.get(scrobble.media_type, "📌"),
"title": (

View File

@ -0,0 +1,29 @@
# Generated by Django 4.2.29 on 2026-06-22 02:44
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
(
"trends",
"0002_alter_trendresult_unique_together_trendresult_period_and_more",
),
]
operations = [
migrations.AlterField(
model_name="trendresult",
name="period",
field=models.CharField(
choices=[
("last_30", "Last 30 days"),
("last_90", "Last 90 days"),
("last_year", "Last year"),
],
default="last_30",
max_length=20,
),
),
]

View File

@ -8,7 +8,6 @@ PERIOD_CHOICES = [
("last_30", "Last 30 days"),
("last_90", "Last 90 days"),
("last_year", "Last year"),
("all_time", "All time"),
]
@ -18,7 +17,7 @@ class TrendResult(TimeStampedModel):
period = models.CharField(
max_length=20,
choices=PERIOD_CHOICES,
default="all_time",
default="last_30",
)
computed_at = models.DateTimeField(auto_now_add=True)
data = models.JSONField(default=dict)

View File

@ -0,0 +1,78 @@
<div class="row">
<div class="col-md-6">
<h5 class="mb-3">By Hour of Day</h5>
{% if data.hours %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Hour</th>
<th class="text-end">Avg Quality</th>
<th class="text-end">Check-ins</th>
</tr>
</thead>
<tbody>
{% for entry in data.hours %}
{% if entry.count > 0 %}
<tr>
<td>
{% if entry.hour == 0 %}
12 AM
{% elif entry.hour < 12 %}
{{ entry.hour }} AM
{% elif entry.hour == 12 %}
12 PM
{% else %}
{{ entry.hour|add:"-12" }} PM
{% endif %}
</td>
<td class="text-end">
<span class="{% if entry.avg_quality >= 5 %}text-success{% elif entry.avg_quality >= 4 %}text-info{% elif entry.avg_quality >= 3 %}text-warning{% else %}text-danger{% endif %}">
{{ entry.avg_quality }}
</span>
</td>
<td class="text-end">{{ entry.count }}</td>
</tr>
{% endif %}
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No hourly data.</p>
{% endif %}
</div>
<div class="col-md-6">
<h5 class="mb-3">By Day of Week</h5>
{% if data.days %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Day</th>
<th class="text-end">Avg Quality</th>
<th class="text-end">Check-ins</th>
</tr>
</thead>
<tbody>
{% for entry in data.days %}
{% if entry.count > 0 %}
<tr>
<td>{{ entry.day_name }}</td>
<td class="text-end">
<span class="{% if entry.avg_quality >= 5 %}text-success{% elif entry.avg_quality >= 4 %}text-info{% elif entry.avg_quality >= 3 %}text-warning{% else %}text-danger{% endif %}">
{{ entry.avg_quality }}
</span>
</td>
<td class="text-end">{{ entry.count }}</td>
</tr>
{% endif %}
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No daily data.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,45 @@
<div class="row">
<div class="col-12">
{% if data.moods %}
<p class="text-muted mb-3">
Total mood check-ins{% if current_period_label %} ({{ current_period_label }}){% endif %}: <strong>{{ data.total }}</strong>
&middot; Positive: <strong>{{ data.positive_count }}</strong>
&middot; Negative: <strong>{{ data.negative_count }}</strong>
</p>
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Mood</th>
<th class="text-end">Count</th>
<th>Distribution</th>
</tr>
</thead>
<tbody>
{% with max=data.moods.0.count %}
{% for entry in data.moods %}
<tr>
<td>{{ entry.mood }}</td>
<td class="text-end">{{ entry.count }}</td>
<td style="width: 40%;">
{% if max > 0 %}
<div class="progress" style="height: 12px;">
<div class="progress-bar" role="progressbar"
style="width: {% widthratio entry.count max 100 %}%;"
aria-valuenow="{{ entry.count }}"
aria-valuemin="0" aria-valuemax="{{ max }}">
</div>
</div>
{% endif %}
</td>
</tr>
{% endfor %}
{% endwith %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No mood distribution data found.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,47 @@
<div class="row">
<div class="col-12">
{% if data.current_streak %}
<div class="alert alert-info">
<strong>Current streak:</strong>
{{ data.current_streak.length }} consecutive
<span class="{% if data.current_streak.mood_type == 'positive' %}text-success{% else %}text-danger{% endif %}">
{{ data.current_streak.mood_type }}
</span>
check-ins since {{ data.current_streak.start_date }}.
</div>
{% endif %}
{% if data.streaks %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>#</th>
<th>Mood Type</th>
<th class="text-end">Length</th>
<th>Start</th>
<th>End</th>
</tr>
</thead>
<tbody>
{% for streak in data.streaks %}
<tr>
<td>{{ forloop.counter }}</td>
<td>
<span class="{% if streak.mood_type == 'positive' %}text-success{% elif streak.mood_type == 'negative' %}text-danger{% endif %}">
{{ streak.mood_type|title }}
</span>
</td>
<td class="text-end">{{ streak.length }}</td>
<td>{{ streak.start_date }}</td>
<td>{{ streak.end_date }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No streak data found.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,39 @@
<div class="row">
<div class="col-12">
{% if data.trajectory %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Date</th>
<th class="text-end">Avg Quality</th>
<th class="text-end">Check-ins</th>
<th>Mood Bar</th>
</tr>
</thead>
<tbody>
{% for entry in data.trajectory %}
<tr>
<td>{{ entry.date }}</td>
<td class="text-end">{{ entry.avg_quality }}</td>
<td class="text-end">{{ entry.count }}</td>
<td style="width: 40%;">
<div class="progress" style="height: 16px;">
<div class="progress-bar {% if entry.avg_quality >= 5 %}bg-success{% elif entry.avg_quality >= 4 %}bg-info{% elif entry.avg_quality >= 3 %}bg-warning{% else %}bg-danger{% endif %}"
role="progressbar"
style="width: {% widthratio entry.avg_quality 7 100 %}%;"
aria-valuenow="{{ entry.avg_quality }}"
aria-valuemin="1" aria-valuemax="7">
</div>
</div>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No mood check-in data found.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,64 @@
<div class="row">
<div class="col-md-6">
<h5 class="mb-3">By Weather Condition</h5>
{% if data.conditions %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Condition</th>
<th class="text-end">Avg Quality</th>
<th class="text-end">Check-ins</th>
</tr>
</thead>
<tbody>
{% for entry in data.conditions %}
<tr>
<td>{{ entry.condition }}</td>
<td class="text-end">
<span class="{% if entry.avg_quality >= 5 %}text-success{% elif entry.avg_quality >= 4 %}text-info{% elif entry.avg_quality >= 3 %}text-warning{% else %}text-danger{% endif %}">
{{ entry.avg_quality }}
</span>
</td>
<td class="text-end">{{ entry.count }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No weather-linked mood data found.</p>
{% endif %}
</div>
<div class="col-md-6">
<h5 class="mb-3">By Temperature Range</h5>
{% if data.temp_ranges %}
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Temp Range</th>
<th class="text-end">Avg Quality</th>
<th class="text-end">Check-ins</th>
</tr>
</thead>
<tbody>
{% for entry in data.temp_ranges %}
<tr>
<td>{{ entry.range }}</td>
<td class="text-end">
<span class="{% if entry.avg_quality >= 5 %}text-success{% elif entry.avg_quality >= 4 %}text-info{% elif entry.avg_quality >= 3 %}text-warning{% else %}text-danger{% endif %}">
{{ entry.avg_quality }}
</span>
</td>
<td class="text-end">{{ entry.count }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<p class="text-muted">No temperature-linked mood data found.</p>
{% endif %}
</div>
</div>

View File

@ -0,0 +1,65 @@
<div class="row">
<div class="col-12">
{% if data.total and data.total > 0 %}
<h5>Overall</h5>
<div class="table-responsive mb-4">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Category</th>
<th class="text-end">Scrobbles</th>
<th class="text-end">%</th>
</tr>
</thead>
<tbody>
{% for slug, info in data.categories.items %}
<tr>
<td>{{ info.label }}</td>
<td class="text-end">{{ info.count }}</td>
<td class="text-end">{{ info.pct }}%</td>
</tr>
{% endfor %}
<tr class="table-secondary">
<td><strong>Total</strong></td>
<td class="text-end"><strong>{{ data.total }}</strong></td>
<td class="text-end"></td>
</tr>
</tbody>
</table>
</div>
<h5>By Media Type</h5>
{% for mt, mt_data in data.by_media_type.items %}
<h6 class="mt-3">{{ mt }}</h6>
<div class="table-responsive">
<table class="table table-striped table-sm">
<thead>
<tr>
<th>Category</th>
<th class="text-end">Scrobbles</th>
<th class="text-end">%</th>
</tr>
</thead>
<tbody>
{% for slug, info in mt_data.categories.items %}
<tr>
<td>{{ info.label }}</td>
<td class="text-end">{{ info.count }}</td>
<td class="text-end">{{ info.pct }}%</td>
</tr>
{% endfor %}
<tr class="table-secondary">
<td><strong>Total</strong></td>
<td class="text-end"><strong>{{ mt_data.total }}</strong></td>
<td class="text-end"></td>
</tr>
</tbody>
</table>
</div>
{% endfor %}
{% else %}
<p class="text-muted">No data found for Books, Trails, Birding Locations, or Board Games in this period.</p>
{% endif %}
</div>
</div>

View File

@ -55,6 +55,9 @@
{% elif trend.slug == "reading-pace-vs-activity" %}
{% include "trends/_reading_pace.html" %}
{% elif trend.slug == "time-of-day-categories" %}
{% include "trends/_time_of_day_categories.html" %}
{% elif trend.slug == "trending-up" %}
{% include "trends/_trending_up.html" %}
@ -67,5 +70,20 @@
{% elif trend.slug == "activity-distribution" %}
{% include "trends/_activity_distribution.html" %}
{% elif trend.slug == "mood-trajectory" %}
{% include "trends/_mood_trajectory.html" %}
{% elif trend.slug == "mood-by-time" %}
{% include "trends/_mood_by_time.html" %}
{% elif trend.slug == "mood-distribution" %}
{% include "trends/_mood_distribution.html" %}
{% elif trend.slug == "mood-streaks" %}
{% include "trends/_mood_streaks.html" %}
{% elif trend.slug == "mood-weather" %}
{% include "trends/_mood_weather.html" %}
{% endif %}
{% endblock %}

View File

@ -7,7 +7,15 @@ from trends.trends.concurrent import (
compute_concurrent_listening,
compute_concurrent_reading,
)
from trends.trends.mood import (
compute_mood_by_time,
compute_mood_distribution,
compute_mood_streaks,
compute_mood_trajectory,
compute_mood_weather,
)
from trends.trends.reading import compute_reading_pace_vs_activity
from trends.trends.time_of_day import compute_time_of_day_categories
from trends.trends.trending import compute_trending_up
TREND_REGISTRY = {}
@ -28,9 +36,17 @@ compute_activity_distribution = register("activity-distribution")(
# compute_concurrent_listening
# )
compute_concurrent_reading = register("concurrent-reading")(compute_concurrent_reading)
compute_mood_by_time = register("mood-by-time")(compute_mood_by_time)
compute_mood_distribution = register("mood-distribution")(compute_mood_distribution)
compute_mood_streaks = register("mood-streaks")(compute_mood_streaks)
compute_mood_trajectory = register("mood-trajectory")(compute_mood_trajectory)
compute_mood_weather = register("mood-weather")(compute_mood_weather)
compute_peak_hours = register("peak-hours")(compute_peak_hours)
compute_reading_pace_vs_activity = register("reading-pace-vs-activity")(
compute_reading_pace_vs_activity
)
compute_time_of_day_categories = register("time-of-day-categories")(
compute_time_of_day_categories
)
compute_trending_up = register("trending-up")(compute_trending_up)
compute_weekly_rhythm = register("weekly-rhythm")(compute_weekly_rhythm)

View File

@ -0,0 +1,208 @@
from collections import Counter, defaultdict
from django.db.models import Q
from scrobbles.models import Scrobble
def _mood_scrobbles(user, period="last_30"):
from trends.utils import get_date_range
start, end = get_date_range(period)
filters = Q(user=user, media_type=Scrobble.MediaType.MOOD)
if start:
filters &= Q(timestamp__gte=start)
if end:
filters &= Q(timestamp__lte=end)
return Scrobble.objects.filter(filters).select_related("mood")
def _parse_quality(raw):
try:
return int(raw)
except (TypeError, ValueError):
return None
def _avg_quality(values):
nums = [v for v in values if v is not None]
if not nums:
return 0.0
return round(sum(nums) / len(nums), 2)
def compute_mood_trajectory(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period).order_by("timestamp")
by_date = defaultdict(list)
for s in scrobbles:
quality = _parse_quality(s.log.get("mood_quality"))
if quality is not None:
day_key = s.timestamp.strftime("%Y-%m-%d")
by_date[day_key].append(quality)
trajectory = []
for date_key in sorted(by_date):
values = by_date[date_key]
trajectory.append(
{
"date": date_key,
"avg_quality": _avg_quality(values),
"count": len(values),
}
)
return {"trajectory": trajectory}
def compute_mood_by_time(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
by_hour = defaultdict(list)
by_day = defaultdict(list)
for s in scrobbles:
quality = _parse_quality(s.log.get("mood_quality"))
if quality is not None and s.timestamp:
by_hour[s.timestamp.hour].append(quality)
by_day[s.timestamp.isoweekday()].append(quality)
hours = []
for h in range(24):
vals = by_hour.get(h, [])
hours.append(
{
"hour": h,
"avg_quality": _avg_quality(vals),
"count": len(vals),
}
)
DAY_NAMES = {
1: "Monday",
2: "Tuesday",
3: "Wednesday",
4: "Thursday",
5: "Friday",
6: "Saturday",
7: "Sunday",
}
days = []
for d in range(1, 8):
vals = by_day.get(d, [])
days.append(
{
"day_index": d,
"day_name": DAY_NAMES[d],
"avg_quality": _avg_quality(vals),
"count": len(vals),
}
)
return {"hours": hours, "days": days}
def compute_mood_distribution(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
mood_counts = Counter()
type_counts = Counter()
for s in scrobbles:
if s.mood and s.mood.title:
mood_counts[s.mood.title] += 1
mood_type = s.log.get("mood_type")
if mood_type:
type_counts[mood_type] += 1
moods = [
{"mood": mood, "count": count}
for mood, count in mood_counts.most_common()
]
total = sum(mood_counts.values())
return {
"moods": moods,
"total": total,
"positive_count": type_counts.get("positive", 0),
"negative_count": type_counts.get("negative", 0),
}
def compute_mood_streaks(user, period="last_30"):
scrobbles = list(
_mood_scrobbles(user, period).order_by("timestamp")
)
if not scrobbles:
return {"streaks": [], "current_streak": None}
streaks = []
current_start = scrobbles[0].timestamp.date()
current_type = scrobbles[0].log.get("mood_type") or "unknown"
current_length = 1
for s in scrobbles[1:]:
mood_type = s.log.get("mood_type") or "unknown"
if mood_type == current_type:
current_length += 1
else:
streaks.append(
{
"start_date": current_start.isoformat(),
"end_date": scrobbles[scrobbles.index(s) - 1].timestamp.date().isoformat(),
"mood_type": current_type,
"length": current_length,
}
)
current_start = s.timestamp.date()
current_type = mood_type
current_length = 1
streaks.append(
{
"start_date": current_start.isoformat(),
"end_date": scrobbles[-1].timestamp.date().isoformat(),
"mood_type": current_type,
"length": current_length,
}
)
streaks.sort(key=lambda x: x["length"], reverse=True)
current_streak = {
"mood_type": current_type,
"length": current_length,
"start_date": current_start.isoformat(),
}
return {"streaks": streaks[:10], "current_streak": current_streak}
def compute_mood_weather(user, period="last_30"):
scrobbles = _mood_scrobbles(user, period)
by_condition = defaultdict(list)
by_temp_range = defaultdict(list)
for s in scrobbles:
quality = _parse_quality(s.log.get("mood_quality"))
if quality is None:
continue
desc = s.log.get("weather_description")
temp = s.log.get("weather_temp")
if desc:
by_condition[desc].append(quality)
if temp is not None:
try:
temp_f = float(temp)
except (TypeError, ValueError):
continue
bucket = f"{(int(temp_f) // 10) * 10}-{(int(temp_f) // 10) * 10 + 9}F"
by_temp_range[bucket].append(quality)
conditions = [
{"condition": cond, "avg_quality": _avg_quality(vals), "count": len(vals)}
for cond, vals in sorted(by_condition.items(), key=lambda x: len(x[1]), reverse=True)
]
temp_ranges = [
{"range": rng, "avg_quality": _avg_quality(vals), "count": len(vals)}
for rng, vals in sorted(by_temp_range.items())
]
return {"conditions": conditions, "temp_ranges": temp_ranges}

View File

@ -0,0 +1,89 @@
from collections import OrderedDict
from django.db.models import Count, Q
from django.db.models.functions import Extract
from scrobbles.models import Scrobble
TARGET_MEDIA_TYPES = ["Book", "Trail", "BirdingLocation", "BoardGame"]
CATEGORIES = OrderedDict(
[
("early_bird", {"label": "Early Bird", "hours": {5, 6, 7, 8, 9, 10}}),
("day_jay", {"label": "Day Jay", "hours": {11, 12, 13, 14, 15, 16, 17, 18}}),
("night_owl", {"label": "Night Owl", "hours": {19, 20, 21, 22, 23, 0, 1, 2, 3, 4}}),
]
)
def _categorize_hour(hour):
for slug, cat in CATEGORIES.items():
if hour in cat["hours"]:
return slug
return None
def compute_time_of_day_categories(user, period="last_30"):
from trends.utils import get_date_range
start, end = get_date_range(period)
filters = Q(user=user, media_type__in=TARGET_MEDIA_TYPES, timestamp__isnull=False)
if start:
filters &= Q(timestamp__gte=start)
if end:
filters &= Q(timestamp__lte=end)
qs = (
Scrobble.objects.filter(filters)
.annotate(hour=Extract("timestamp", "hour"))
.values("media_type", "hour")
.annotate(count=Count("id"))
.order_by("media_type", "hour")
)
raw = {}
for row in qs:
mt = row["media_type"]
raw.setdefault(mt, {})[row["hour"]] = row["count"]
by_media_type = {}
grand_totals = {"early_bird": 0, "day_jay": 0, "night_owl": 0}
grand_total = 0
for mt in TARGET_MEDIA_TYPES:
mt_data = raw.get(mt, {})
cat_counts = {"early_bird": 0, "day_jay": 0, "night_owl": 0}
mt_total = 0
for hour, count in mt_data.items():
slug = _categorize_hour(hour)
if slug:
cat_counts[slug] += count
mt_total += count
by_media_type[mt] = {
"total": mt_total,
"categories": {},
}
for slug in CATEGORIES:
c = cat_counts[slug]
by_media_type[mt]["categories"][slug] = {
"count": c,
"pct": round((c / mt_total * 100), 1) if mt_total else 0,
"label": CATEGORIES[slug]["label"],
}
grand_totals[slug] += c
grand_total += mt_total
categories = {}
for slug in CATEGORIES:
c = grand_totals[slug]
categories[slug] = {
"count": c,
"pct": round((c / grand_total * 100), 1) if grand_total else 0,
"label": CATEGORIES[slug]["label"],
}
return {
"categories": categories,
"total": grand_total,
"by_media_type": by_media_type,
}

View File

@ -10,7 +10,6 @@ PERIOD_DAYS = {
"last_30": 30,
"last_90": 90,
"last_year": 365,
"all_time": None,
}
PERIOD_LABELS = dict(PERIOD_CHOICES)
@ -19,8 +18,16 @@ TIME_BOUND_TRENDS = {
"activity-distribution",
"concurrent-reading",
"concurrent-listening",
"mood-by-time",
"mood-distribution",
"mood-streaks",
"mood-trajectory",
"mood-weather",
"peak-hours",
"reading-pace-vs-activity",
"time-of-day-categories",
"trending-up",
"weekly-rhythm",
}
TREND_PERIOD_OVERRIDES = {
@ -32,9 +39,7 @@ def get_supported_periods(trend_slug):
if trend_slug in TREND_PERIOD_OVERRIDES:
slugs = TREND_PERIOD_OVERRIDES[trend_slug]
return {s: PERIOD_LABELS[s] for s in slugs}
if trend_slug in TIME_BOUND_TRENDS:
return dict(PERIOD_LABELS)
return {"all_time": PERIOD_LABELS["all_time"]}
return dict(PERIOD_LABELS)
def get_period_days(period):
@ -61,7 +66,7 @@ def get_period_nav(current_period, trend_slug):
return prev_period, next_period
def compute_and_save_trend(user, slug, period="all_time"):
def compute_and_save_trend(user, slug, period="last_30"):
"""Compute a single trend for a given period and persist the result.
Returns elapsed seconds on success, raises on failure.

View File

@ -20,6 +20,31 @@ TREND_METADATA = {
"description": "What music did you listen to while reading books?",
"icon": "📖",
},
"mood-trajectory": {
"title": "Mood Trajectory",
"description": "How your mood quality has changed over time.",
"icon": "📈",
},
"mood-by-time": {
"title": "Mood by Time",
"description": "How your mood varies by hour of day and day of week.",
"icon": "🕐",
},
"mood-distribution": {
"title": "Mood Distribution",
"description": "Which moods you feel most often.",
"icon": "🎭",
},
"mood-streaks": {
"title": "Mood Streaks",
"description": "Your longest runs of positive and negative moods.",
"icon": "🔥",
},
"mood-weather": {
"title": "Mood & Weather",
"description": "How weather conditions correlate with your mood.",
"icon": "🌤",
},
"peak-hours": {
"title": "Peak Activity Hours",
"description": "What time of day are you most active?",
@ -30,6 +55,11 @@ TREND_METADATA = {
"description": "Compare how long you read per session with and without concurrent music.",
"icon": "📊",
},
"time-of-day-categories": {
"title": "Time of Day Categories",
"description": "Are you an early bird, day jay, or night owl? Categorized by Books, Trails, Birding Locations, and Board Games.",
"icon": "🦉",
},
"trending-up": {
"title": "Trending Media Types",
"description": "Which media types have you been consuming more or less of recently?",
@ -86,7 +116,7 @@ class TrendDetailView(LoginRequiredMixin, TemplateView):
ctx["trend_not_found"] = True
return ctx
period = self.request.GET.get("period", "all_time")
period = self.request.GET.get("period", "last_30")
meta = TREND_METADATA.get(slug, {})
ctx["trend"] = {

View File

@ -10,26 +10,27 @@ def hrs_to_secs(hrs: float) -> int:
return int(hrs * 60 * 60)
def lookup_game_from_hltb(name_or_id: str) -> Optional[dict]:
def lookup_game_from_hltb(name_or_id: str, search_by_title: bool = False) -> Optional[dict]:
"""Lookup game on HowLongToBeat.com via HLtB ID or a name string and return
the data in a dictonary mapped to our internal game fields
"""
hltb_game = {}
try:
hltb_id = int(name_or_id)
except ValueError:
hltb_id = None
if not search_by_title:
try:
hltb_id = int(name_or_id)
except ValueError:
hltb_id = None
if hltb_id:
hltb_game = HowLongToBeat().search_from_id(hltb_id)
logger.info(f"Found game on HLtB for ID {hltb_id}")
if hltb_id:
hltb_game = HowLongToBeat().search_from_id(hltb_id)
logger.info(f"Found game on HLtB for ID {hltb_id}")
if not hltb_game:
results = HowLongToBeat().search(name_or_id)
if not results:
logger.warn(f"Lookup of game on HLtB failed for ID {name_or_id}")
logger.warn(f"Lookup of game on HLtB failed via search {name_or_id!r}")
return
hltb_game = results[0]

View File

@ -19,6 +19,7 @@ GAMES_URL = "https://api.igdb.com/v4/games"
ALT_NAMES_URL = "https://api.igdb.com/v4/alternative_names"
SCREENSHOT_URL = "https://api.igdb.com/v4/screenshots"
COVER_URL = "https://api.igdb.com/v4/covers"
PLATFORMS_URL = "https://api.igdb.com/v4/platforms"
IGDB_CLIENT_ID = getattr(settings, "IGDB_CLIENT_ID")
IGDB_CLIENT_SECRET = getattr(settings, "IGDB_CLIENT_SECRET")
@ -35,6 +36,20 @@ def get_igdb_token() -> str:
return results.get("access_token")
def lookup_platform_names(platform_ids: list, headers: dict) -> list:
"""Resolve IGDB platform IDs to platform names"""
if not platform_ids:
return []
ids_str = ",".join(str(pid) for pid in platform_ids)
body = f"fields name; where id = ({ids_str});"
resp = requests.post(PLATFORMS_URL, data=body, headers=headers)
if resp.status_code != 200:
logger.warn(f"Failed to resolve platform IDs {platform_ids}")
return []
results = json.loads(resp.content)
return [p["name"] for p in results if "name" in p]
def lookup_game_id_from_gdb(name: str) -> str:
headers = {
@ -62,9 +77,10 @@ def lookup_game_id_from_gdb(name: str) -> str:
"details": results.get("details"),
},
)
# Sort our result by IDs so we always get the lowest ID, which is likely to be the least esoteric game
results = sorted(results, key=lambda k: k.get("game", 250000))
return results[0].get("game", "")
# Sort results by release date (oldest first) to prefer the original game
results = [r for r in results if r.get("game")]
results = sorted(results, key=lambda k: k.get("published_at") or 9999999999)
return results[0].get("game", "") if results else ""
def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
@ -118,6 +134,16 @@ def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
for genre in game.get("genres"):
genres.append(genre["name"])
platforms = []
if "release_dates" in game.keys():
platform_ids = set()
for rd in game["release_dates"]:
pid = rd.get("platform")
if pid is not None:
platform_ids.add(pid)
if platform_ids:
platforms = lookup_platform_names(list(platform_ids), headers)
game_dict = {
"igdb_id": game.get("id"),
"title": game.get("name"),
@ -129,6 +155,7 @@ def lookup_game_from_igdb(name_or_igdb_id: str) -> Dict:
"release_date": release_date,
"summary": game.get("summary"),
"genres": genres,
"platforms": platforms,
}
return game_dict

View File

@ -0,0 +1,529 @@
import logging
import time
from django.core.management.base import BaseCommand
logger = logging.getLogger(__name__)
MISSING_ALL = [
"cover",
"screenshot",
"summary",
"rating",
"release_date",
"release_year",
"igdb_id",
"hltb_id",
]
MISSING_GROUPS = {
"cover": lambda g: not bool(g.cover),
"screenshot": lambda g: not bool(g.screenshot),
"summary": lambda g: not g.summary,
"rating": lambda g: g.rating is None,
"release_date": lambda g: g.release_date is None,
"release_year": lambda g: g.release_year is None,
"igdb_id": lambda g: g.igdb_id is None,
"hltb_id": lambda g: g.hltb_id is None,
}
def _game_matches(game, flags):
if not flags:
return False
for flag in flags:
fn = MISSING_GROUPS.get(flag)
if fn and fn(game):
return True
return False
class Command(BaseCommand):
help = "Backfill missing metadata on video games from IGDB and HowLongToBeat"
def add_arguments(self, parser):
parser.add_argument(
"--commit",
action="store_true",
help="Commit changes to the database",
)
parser.add_argument(
"--batch-size",
type=int,
default=100,
help="Number of games to process per batch (default: 100)",
)
parser.add_argument(
"--sleep",
type=float,
default=0.5,
help="Seconds to sleep between API calls (default: 0.5)",
)
parser.add_argument(
"--force",
action="store_true",
help="Re-fetch metadata even if data already exists",
)
parser.add_argument(
"--game-id",
type=int,
help="Only process a specific game by ID",
)
parser.add_argument(
"--fix-broken-images",
action="store_true",
help="Check and refetch broken/deleted game images (cover, screenshot, hltb_cover)",
)
for flag in MISSING_ALL:
parser.add_argument(
f"--missing-{flag}",
dest="missing_flags",
action="append_const",
const=flag,
help=f"Process games missing {flag}",
)
parser.add_argument(
"--all",
action="store_true",
dest="all_missing",
help="Process games missing any metadata field",
)
def handle(self, *args, **options):
from videogames.models import VideoGame
commit = options["commit"]
batch_size = options["batch_size"]
sleep_secs = options["sleep"]
force = options["force"]
game_id = options["game_id"]
fix_broken_images = options.get("fix_broken_images", False)
flags = options.get("missing_flags") or []
all_missing = options["all_missing"]
if all_missing:
flags = MISSING_ALL
if not flags and not game_id and not force and not fix_broken_images:
self.stdout.write(
"No filters specified. Use --all, --missing-*, --game-id, --force, or --fix-broken-images."
)
return
if game_id:
qs = VideoGame.objects.filter(id=game_id)
else:
qs = VideoGame.objects.all()
if flags:
qs = [g for g in qs.iterator() if _game_matches(g, flags)]
else:
qs = list(qs)
total = len(qs)
self.stdout.write(f"Found {total} games to process")
if not commit:
self.stdout.write(
"Dry run — no API calls will be made. Use --commit to run lookups."
)
return
title_mismatches = []
enriched = 0
skipped = 0
stats = {
"cover_fixed": 0,
"screenshot_fixed": 0,
"summary_fixed": 0,
"rating_fixed": 0,
"release_date_fixed": 0,
"release_year_fixed": 0,
"igdb_id_found": 0,
"hltb_id_found": 0,
"images_fixed": 0,
}
enriched_any = bool(flags or game_id or force)
if enriched_any:
for batch_num, offset in enumerate(range(0, len(qs), batch_size)):
batch = qs[offset : offset + batch_size]
for game in batch:
result = self._enrich_game(game, sleep_secs, force)
self._check_retroarch_name(game, title_mismatches)
if result:
enriched += 1
for key in stats:
if result.get(key):
stats[key] += 1
else:
skipped += 1
self.stdout.write(
f" Batch {batch_num + 1}: {offset + len(batch)}/{total}"
f"enriched: {enriched}, skipped: {skipped}"
)
if fix_broken_images:
broken_stats = self._fix_broken_images(qs, sleep_secs)
stats["images_fixed"] = broken_stats["images_fixed"]
self.stdout.write(
f"\nResults (commit={commit}):\n"
f" Games enriched: {enriched}\n"
f" Games skipped: {skipped}\n"
f" Covers fixed: {stats['cover_fixed']}\n"
f" Screenshots fixed: {stats['screenshot_fixed']}\n"
f" Summaries fixed: {stats['summary_fixed']}\n"
f" Ratings fixed: {stats['rating_fixed']}\n"
f" Release dates fixed: {stats['release_date_fixed']}\n"
f" Release years fixed: {stats['release_year_fixed']}\n"
f" IGDB IDs found: {stats['igdb_id_found']}\n"
f" HLtB IDs found: {stats['hltb_id_found']}"
)
if fix_broken_images:
self.stdout.write(f" Broken images fixed: {stats['images_fixed']}")
if title_mismatches:
self.stdout.write("\nTitle vs retroarch_name mismatches (not auto-fixed):")
for retroarch_name, title, game_id in title_mismatches:
self.stdout.write(
f" Game #{game_id}: retroarch_name={retroarch_name!r} vs title={title!r}"
)
def _clean_retroarch_name(self, name):
if not name:
return ""
name = name.strip()
if "(" in name:
name = name.split("(")[0].strip()
return name
def _check_retroarch_name(self, game, mismatches):
if not game.retroarch_name:
return
cleaned = self._clean_retroarch_name(game.retroarch_name)
if cleaned.lower() != game.title.lower():
mismatches.append((game.retroarch_name, game.title, game.id))
if "retroarch-mismatch" not in game.tags.names():
game.tags.add("retroarch-mismatch")
self.stdout.write(
f" [TAG] {game} — tagged as retroarch-mismatch"
)
def _enrich_game(self, game, sleep_secs, force):
from videogames.igdb import lookup_game_id_from_gdb, lookup_game_from_igdb
from videogames.howlongtobeat import lookup_game_from_hltb
search_name = self._clean_retroarch_name(game.retroarch_name) or game.title
changed = {}
if not game.hltb_id:
hltb_data = None
if search_name:
hltb_data = lookup_game_from_hltb(search_name, search_by_title=True)
time.sleep(sleep_secs)
if not hltb_data and game.title and game.title != search_name:
hltb_data = lookup_game_from_hltb(game.title, search_by_title=True)
time.sleep(sleep_secs)
if hltb_data:
result = self._apply_hltb_data(game, hltb_data, force)
if result:
changed.update(result)
igdb_data = None
if not game.igdb_id and search_name:
igdb_id = lookup_game_id_from_gdb(search_name)
time.sleep(sleep_secs)
if igdb_id:
igdb_data = lookup_game_from_igdb(str(igdb_id))
time.sleep(sleep_secs)
elif game.igdb_id:
igdb_data = lookup_game_from_igdb(str(game.igdb_id))
time.sleep(sleep_secs)
if igdb_data:
igdb_title = igdb_data.get("title", "")
igdb_title_clean = self._clean_retroarch_name(igdb_title)
if igdb_title_clean.lower() == search_name.lower():
if game.igdb_id is None and igdb_data.get("igdb_id"):
game.igdb_id = int(igdb_data["igdb_id"])
game.save(update_fields=["igdb_id"])
changed["igdb_id_found"] = True
self.stdout.write(f" [IGDB_ID] {game} — found IGDB ID {game.igdb_id}")
result = self._apply_igdb_data(game, igdb_data, force)
if result:
changed.update(result)
else:
self.stdout.write(
f" [IGDB] {game} — title mismatch (IGDB: {igdb_title!r} vs expected: {search_name!r}), re-searching…"
)
resolved = False
for candidate in (search_name, game.title if game.title != search_name else None):
if not candidate:
continue
new_id = lookup_game_id_from_gdb(candidate)
time.sleep(sleep_secs)
if not new_id:
continue
new_data = lookup_game_from_igdb(str(new_id))
time.sleep(sleep_secs)
if not new_data:
continue
new_title = new_data.get("title", "")
new_title_clean = self._clean_retroarch_name(new_title)
if new_title_clean.lower() == candidate.lower():
game.igdb_id = int(new_id)
if new_title and new_title != game.title:
game.title = new_title
changed["title_updated"] = True
self.stdout.write(f" [TITLE] {game} — updated title to {new_title!r} from IGDB")
game.save(update_fields=["igdb_id"] + (["title"] if changed.get("title_updated") else []))
changed["igdb_id_found"] = True
self.stdout.write(f" [IGDB_ID] {game} — re-found IGDB ID {game.igdb_id}")
if "igdb-mismatch" in game.tags.names():
game.tags.remove("igdb-mismatch")
self.stdout.write(f" [TAG] {game} — removed igdb-mismatch tag")
result = self._apply_igdb_data(game, new_data, force)
if result:
changed.update(result)
resolved = True
break
if not resolved and "igdb-mismatch" not in game.tags.names():
game.tags.add("igdb-mismatch")
self.stdout.write(f" [TAG] {game} — tagged igdb-mismatch")
# If retroarch-mismatch tag exists but no longer applies, remove it
if "retroarch-mismatch" in game.tags.names():
cleaned = self._clean_retroarch_name(game.retroarch_name or "")
if cleaned.lower() == game.title.lower():
game.tags.remove("retroarch-mismatch")
self.stdout.write(f" [TAG] {game} — removed retroarch-mismatch tag (title now matches)")
return changed if changed else None
def _apply_igdb_data(self, game, data, force):
from django.core.files.base import ContentFile
import requests
changed = {
"cover_fixed": False,
"screenshot_fixed": False,
"summary_fixed": False,
"rating_fixed": False,
"release_date_fixed": False,
}
update_fields = []
if data.get("alternative_name") and not game.alternative_name:
game.alternative_name = data["alternative_name"]
update_fields.append("alternative_name")
if data.get("summary") and (not game.summary or force):
game.summary = data["summary"]
update_fields.append("summary")
changed["summary_fixed"] = True
if data.get("rating") is not None and (game.rating is None or force):
game.rating = data["rating"]
update_fields.append("rating")
changed["rating_fixed"] = True
if data.get("rating_count") is not None and (game.rating_count is None or force):
game.rating_count = data["rating_count"]
update_fields.append("rating_count")
if data.get("release_date") and (game.release_date is None or force):
game.release_date = data["release_date"]
update_fields.append("release_date")
changed["release_date_fixed"] = True
if update_fields:
game.save(update_fields=update_fields)
self.stdout.write(f" [IGDB] {game}{', '.join(update_fields)}")
cover_url = data.get("cover_url")
if cover_url:
r = requests.get(cover_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.cover.save(fname, ContentFile(r.content), save=True)
changed["cover_fixed"] = True
self.stdout.write(f" [COVER] {game} — cover saved from IGDB")
screenshot_url = data.get("screenshot_url")
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
changed["screenshot_fixed"] = True
self.stdout.write(f" [SCREENSHOT] {game} — screenshot saved from IGDB")
genres = data.get("genres", [])
if genres:
existing = set(game.genre.names())
new_genres = [g for g in genres if g not in existing]
if new_genres:
game.genre.add(*new_genres)
self.stdout.write(f" [GENRES] {game} — added {len(new_genres)} genres")
platforms = data.get("platforms", [])
if platforms:
existing = set(game.platforms.values_list("name", flat=True))
new_platforms = [p for p in platforms if p not in existing]
if new_platforms:
from videogames.models import VideoGamePlatform
for name in new_platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
self.stdout.write(f" [PLATFORMS] {game} — added {len(new_platforms)} platforms")
if "igdb-enriched" not in game.tags.names():
game.tags.add("igdb-enriched")
self.stdout.write(f" [TAG] {game} — tagged igdb-enriched")
return changed if any(changed.values()) else None
def _apply_hltb_data(self, game, data, force):
from django.core.files.base import ContentFile
import requests
changed = {
"hltb_id_found": False,
"release_year_fixed": False,
}
update_fields = []
hltb_title = data.get("title", "")
if hltb_title and hltb_title != game.title:
game.title = hltb_title
update_fields.append("title")
self.stdout.write(f" [TITLE] {game} — updated title to {hltb_title!r}")
if data.get("hltb_id") and (game.hltb_id is None or force):
game.hltb_id = data["hltb_id"]
update_fields.append("hltb_id")
changed["hltb_id_found"] = True
self.stdout.write(f" [HLTB_ID] {game} — found HLtB ID {data['hltb_id']}")
if data.get("release_year") and (game.release_year is None or force):
game.release_year = data["release_year"]
update_fields.append("release_year")
changed["release_year_fixed"] = True
if data.get("main_story_time") and (game.main_story_time is None or force):
game.main_story_time = data["main_story_time"]
update_fields.append("main_story_time")
if data.get("main_extra_time") and (game.main_extra_time is None or force):
game.main_extra_time = data["main_extra_time"]
update_fields.append("main_extra_time")
if data.get("completionist_time") and (game.completionist_time is None or force):
game.completionist_time = data["completionist_time"]
update_fields.append("completionist_time")
if data.get("hltb_score") is not None and (game.hltb_score is None or force):
game.hltb_score = data["hltb_score"]
update_fields.append("hltb_score")
if update_fields:
game.save(update_fields=update_fields)
self.stdout.write(f" [HLTB] {game}{', '.join(update_fields)}")
cover_url = data.get("cover_url")
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
if r.status_code == 200:
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
self.stdout.write(f" [HLTB_COVER] {game} — cover saved from HLtB")
platforms = data.get("platforms", [])
if platforms:
existing = set(game.platforms.values_list("name", flat=True))
new_platforms = [p for p in platforms if p not in existing]
if new_platforms:
from videogames.models import VideoGamePlatform
for name in new_platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
self.stdout.write(f" [PLATFORMS] {game} — added {len(new_platforms)} platforms")
if "hltb-enriched" not in game.tags.names():
game.tags.add("hltb-enriched")
self.stdout.write(f" [TAG] {game} — tagged hltb-enriched")
return changed if any(changed.values()) else None
def _fix_broken_images(self, games, sleep_secs):
from django.core.files.base import ContentFile
import requests
from videogames.igdb import lookup_game_from_igdb
from videogames.howlongtobeat import lookup_game_from_hltb
stats = {"cover_fixed": 0, "screenshot_fixed": 0, "images_fixed": 0}
for game in games:
for field_name, source in [
("cover", "igdb"),
("screenshot", "igdb"),
("hltb_cover", "hltb"),
]:
field = getattr(game, field_name)
if not field.name:
continue
if field.storage.exists(field.name):
continue
self.stdout.write(
f" [IMAGE] {game}{field_name} is broken (file missing), refetching…"
)
if source == "igdb" and game.igdb_id:
data = lookup_game_from_igdb(str(game.igdb_id))
time.sleep(sleep_secs)
if not data:
continue
url = data.get("cover_url" if field_name == "cover" else "screenshot_url")
if not url:
continue
r = requests.get(url)
if r.status_code != 200:
continue
fname = f"{game.title}_{game.uuid}.jpg"
getattr(game, field_name).save(fname, ContentFile(r.content), save=True)
stats["images_fixed"] += 1
self.stdout.write(f" [IMAGE] {game}{field_name} refetched from IGDB")
elif source == "hltb" and game.hltb_id:
data = lookup_game_from_hltb(str(game.hltb_id))
time.sleep(sleep_secs)
if not data:
continue
url = data.get("cover_url")
if not url:
continue
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(url, headers=headers)
if r.status_code != 200:
continue
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
stats["images_fixed"] += 1
self.stdout.write(f" [IMAGE] {game} — hltb_cover refetched from HLtB")
return stats

View File

@ -215,12 +215,16 @@ class VideoGame(LongPlayScrobblableMixin):
def fix_metadata(self, force_update: bool = False):
from videogames.utils import (
get_or_create_videogame,
load_game_data_from_hltb,
load_game_data_from_igdb,
)
if self.hltb_id and force_update:
get_or_create_videogame(str(self.hltb_id), force_update)
if not self.hltb_id:
load_game_data_from_hltb(self.id)
if not self.igdb_id:
# This almost never works without intervention
self.igdb_id = lookup_game_id_from_gdb(self.title)

View File

@ -153,6 +153,13 @@ def import_retroarch_lrtl_files(playlog_path: str, user_id: int) -> List[dict]:
continue
logger.info(f"Queued scrobble for game {found_game.id}")
log_data = {"emulated": True}
if last_scrobble and last_scrobble.log:
prev = last_scrobble.log
if prev.get("emulator"):
log_data["emulator"] = prev["emulator"]
new_scrobbles.append(
Scrobble(
video_game_id=found_game.id,
@ -168,6 +175,7 @@ def import_retroarch_lrtl_files(playlog_path: str, user_id: int) -> List[dict]:
user_id=user_id,
source="Retroarch",
media_type=Scrobble.MediaType.VIDEO_GAME,
log=log_data,
)
)
created_scrobbles = Scrobble.objects.bulk_create(new_scrobbles)

View File

@ -15,13 +15,11 @@ def natural_duration(value):
seconds = remainder % 60
parts = []
if days:
parts.append(f"{days} day{'s' if days != 1 else ''}")
parts.append(f"{days} d")
if hours:
parts.append(f"{hours} hour{'s' if hours != 1 else ''}")
parts.append(f"{hours} hr")
if minutes:
parts.append(f"{minutes} minute{'s' if minutes != 1 else ''}")
parts.append(f"{minutes} min")
if seconds or not parts:
parts.append(f"{seconds} second{'s' if seconds != 1 else ''}")
if len(parts) == 1:
return parts[0]
return ", ".join(parts[:-1]) + " and " + parts[-1]
parts.append(f"{seconds} sec")
return ", ".join(parts)

View File

@ -7,8 +7,6 @@ from videogames.howlongtobeat import lookup_game_from_hltb
from videogames.igdb import lookup_game_from_igdb
from videogames.models import VideoGame, VideoGamePlatform
from vrobbler.apps.videogames.exceptions import GameNotFound
logger = logging.getLogger(__name__)
@ -16,22 +14,33 @@ def get_or_create_videogame(
name_or_id: str,
force_update: bool = False,
) -> Optional[VideoGame]:
"""Look up game by name or ID from HowLongToBeat"""
"""Look up game by name or ID from HowLongToBeat, then enrich with IGDB"""
game_dict = lookup_game_from_hltb(name_or_id)
hltb_data = lookup_game_from_hltb(name_or_id)
if not game_dict:
game_dict = lookup_game_from_igdb(name_or_id)
if hltb_data:
game = _create_update_from_dict(hltb_data, force_update)
else:
igdb_data = lookup_game_from_igdb(name_or_id)
if igdb_data:
game = _create_update_from_dict(igdb_data, force_update)
else:
return None
if not game_dict:
return
if game:
game.fix_metadata()
return game
def _create_update_from_dict(
game_dict: dict, force_update: bool = False
) -> Optional[VideoGame]:
# Create missing platforms and prep for loading after create
platform_ids = []
if "platforms" in game_dict.keys():
platforms = game_dict.get("platforms", [])
if platforms:
for platform in game_dict.get("platforms", []):
for platform in platforms:
p, _created = VideoGamePlatform.objects.get_or_create(name=platform)
platform_ids.append(p.id)
game_dict.pop("platforms")
@ -48,7 +57,7 @@ def get_or_create_videogame(
title = game_dict.get("title")
if not title:
raise GameNotFound(name_or_id)
return None
hltb_id = game_dict.get("hltb_id")
igdb_id = game_dict.get("igdb_id")
@ -69,21 +78,19 @@ def get_or_create_videogame(
VideoGame.objects.filter(pk=game.id).update(**game_dict)
game.refresh_from_db()
# Associate plaforms
if platform_ids:
game.platforms.add(*platform_ids)
if genres:
game.genre.add(*genres)
if not game.screenshot and screenshot_url:
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
# Go get cover image if the URL is present
if cover_url and not game.hltb_cover:
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
logger.debug(r.status_code)
@ -91,12 +98,89 @@ def get_or_create_videogame(
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
logger.debug("Loaded cover image from HLtB")
game.fix_metadata()
tag = "hltb-enriched" if hltb_id else "igdb-enriched"
if tag not in game.tags.names():
game.tags.add(tag)
logger.info(f"Game {game} tagged {tag}")
return game
def load_game_data_from_igdb(game_id: int, igdb_id: str = "") -> Optional[VideoGame]:
def load_game_data_from_hltb(
game_id: int, expected_title: str = ""
) -> Optional[VideoGame]:
"""Look up HLtB data for an existing game and apply it"""
game = VideoGame.objects.filter(id=game_id).first()
if not game:
logger.warn(f"Video game with ID {game_id} not found")
return
logger.info(f"Looking up HLtB data for {game}")
hltb_data = lookup_game_from_hltb(game.title, search_by_title=True)
if not hltb_data:
logger.warn(f"No HLtB data found for {game}")
return
update_fields = []
hltb_title = hltb_data.get("title", "")
if hltb_title and hltb_title != game.title:
game.title = hltb_title
update_fields.append("title")
logger.info(f"Game {game.id} title updated to {hltb_title!r}")
if hltb_data.get("hltb_id") and (game.hltb_id is None):
game.hltb_id = hltb_data["hltb_id"]
update_fields.append("hltb_id")
if hltb_data.get("release_year") and (game.release_year is None):
game.release_year = hltb_data["release_year"]
update_fields.append("release_year")
if hltb_data.get("main_story_time") and (game.main_story_time is None):
game.main_story_time = hltb_data["main_story_time"]
update_fields.append("main_story_time")
if hltb_data.get("main_extra_time") and (game.main_extra_time is None):
game.main_extra_time = hltb_data["main_extra_time"]
update_fields.append("main_extra_time")
if hltb_data.get("completionist_time") and (game.completionist_time is None):
game.completionist_time = hltb_data["completionist_time"]
update_fields.append("completionist_time")
if hltb_data.get("hltb_score") is not None and (game.hltb_score is None):
game.hltb_score = hltb_data["hltb_score"]
update_fields.append("hltb_score")
if update_fields:
game.save(update_fields=update_fields)
platforms = hltb_data.get("platforms", [])
if platforms:
for name in platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
cover_url = hltb_data.get("cover_url")
if cover_url:
headers = {"User-Agent": "Vrobbler 0.11.12"}
r = requests.get(cover_url, headers=headers)
if r.status_code == 200:
fname = f"{game.title}_cover_{game.uuid}.jpg"
game.hltb_cover.save(fname, ContentFile(r.content), save=True)
if "hltb-enriched" not in game.tags.names():
game.tags.add("hltb-enriched")
logger.info(f"Game {game} tagged hltb-enriched")
return game
def load_game_data_from_igdb(
game_id: int, igdb_id: str = "", expected_title: str = ""
) -> Optional[VideoGame]:
"""Look up game, if it doesn't exist, lookup data from igdb"""
game = VideoGame.objects.filter(id=game_id).first()
if not game:
@ -116,25 +200,68 @@ def load_game_data_from_igdb(game_id: int, igdb_id: str = "") -> Optional[VideoG
logger.warn(f"No game data found on IGDB for ID {igdb_id}")
return
igdb_title = game_dict.get("title", "")
igdb_title_clean = igdb_title.split(" (")[0].strip() if " (" in igdb_title else igdb_title
expected = expected_title or game.title
if igdb_title_clean.lower() != expected.lower():
logger.info(
f"IGDB title {igdb_title!r} doesn't match expected {expected!r} for {game} — re-searching…"
)
from videogames.igdb import lookup_game_id_from_gdb
new_id = lookup_game_id_from_gdb(expected)
if new_id:
new_data = lookup_game_from_igdb(str(new_id))
if new_data:
new_title = new_data.get("title", "")
new_title_clean = new_title.split(" (")[0].strip() if " (" in new_title else new_title
if new_title_clean.lower() == expected.lower():
game_dict = new_data
igdb_id = int(new_id)
if game.igdb_id != igdb_id:
game.igdb_id = igdb_id
game.save(update_fields=["igdb_id"])
logger.info(f"Game {game} IGDB ID updated to {igdb_id}")
igdb_title = game_dict.get("title", "")
igdb_title_clean = igdb_title.split(" (")[0].strip() if " (" in igdb_title else igdb_title
if igdb_title_clean.lower() != expected.lower():
if "igdb-mismatch" not in game.tags.names():
game.tags.add("igdb-mismatch")
logger.info(
f"Game {game} tagged igdb-mismatch (IGDB: {igdb_title!r} vs expected: {expected!r})"
)
return
screenshot_url = game_dict.pop("screenshot_url")
cover_url = game_dict.pop("cover_url")
genres = game_dict.pop("genres")
platforms = game_dict.pop("platforms", [])
VideoGame.objects.filter(pk=game.id).update(**game_dict)
game.refresh_from_db()
game.genre.add(*genres)
if not game.screenshot and screenshot_url:
if platforms:
for name in platforms:
p, _ = VideoGamePlatform.objects.get_or_create(name=name)
game.platforms.add(p)
if screenshot_url:
r = requests.get(screenshot_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.screenshot.save(fname, ContentFile(r.content), save=True)
if not game.cover and cover_url:
if cover_url:
r = requests.get(cover_url)
if r.status_code == 200:
fname = f"{game.title}_{game.uuid}.jpg"
game.cover.save(fname, ContentFile(r.content), save=True)
if "igdb-enriched" not in game.tags.names():
game.tags.add("igdb-enriched")
logger.info(f"Game {game} tagged igdb-enriched")
return game

View File

@ -1,8 +1,24 @@
from django.core.cache import cache
from videos.models import Video, Series
CACHE_TTL = 300
def video_lists(request):
movie_list = cache.get("video_lists_movie_list")
if movie_list is None:
movie_list = list(
Video.objects.filter(video_type=Video.VideoType.MOVIE).only("id", "title")
)
cache.set("video_lists_movie_list", movie_list, CACHE_TTL)
series_list = cache.get("video_lists_series_list")
if series_list is None:
series_list = list(Series.objects.all().only("id", "name"))
cache.set("video_lists_series_list", series_list, CACHE_TTL)
return {
"movie_list": Video.objects.filter(video_type=Video.VideoType.MOVIE),
"series_list": Series.objects.all(),
"movie_list": movie_list,
"series_list": series_list,
}

View File

@ -1,7 +1,7 @@
import logging
from django.core.management.base import BaseCommand
from django.db import transaction
from django.db import models, transaction
logger = logging.getLogger(__name__)
@ -25,6 +25,11 @@ class Command(BaseCommand):
type=str,
help="Only process series with this imdb_id",
)
parser.add_argument(
"--needs-metadata",
action="store_true",
help="Only process series missing imdb_id or cover image",
)
def handle(self, *args, **options):
from videos.models import Series
@ -32,18 +37,30 @@ class Command(BaseCommand):
force = options["force"]
dry_run = options["dry_run"]
imdb_id = options["imdb_id"]
needs_metadata = options["needs_metadata"]
qs = Series.objects.all()
if imdb_id:
qs = qs.filter(imdb_id=imdb_id)
if needs_metadata:
qs = qs.filter(
models.Q(imdb_id__isnull=True)
| models.Q(imdb_id="")
| models.Q(cover_image__isnull=True)
| models.Q(cover_image="")
)
total = qs.count()
self.stdout.write(f"Processing {total} series")
if dry_run:
for series in qs.iterator():
has_imdb = bool(series.imdb_id)
has_image = bool(series.cover_image)
self.stdout.write(
f" [DRY RUN] Would fix {series.name} (imdb_id={series.imdb_id})"
f" [DRY RUN] Would fix {series.name}"
f" (imdb_id={'' if has_imdb else ''}"
f", image={'' if has_image else ''})"
)
return

View File

@ -321,13 +321,27 @@ class Series(TimeStampedModel):
return not last_scrobble.played_to_completion
def fix_metadata(self, force_update=False):
name_or_id = self.name
if self.imdb_id:
name_or_id = self.imdb_id
video_metadata: VideoMetadata = lookup_video_from_tmdb(name_or_id)
from tmdbv3api import TV
if not video_metadata.title:
logger.warning(f"No imdb data for {self}")
if not self.imdb_id:
tv = TV()
results = tv.search(self.name)
if results:
show_id = results[0].id
external_ids = tv.external_ids(show_id)
if external_ids and external_ids.imdb_id:
self.imdb_id = external_ids.imdb_id
self.save(update_fields=["imdb_id"])
else:
logger.warning(f"No IMDB ID found on TMDB for {self}")
return
else:
logger.warning(f"No results on TMDB for {self.name}")
return
video_metadata = lookup_video_from_tmdb(self.imdb_id)
if not video_metadata or not video_metadata.title:
logger.warning(f"No metadata for {self}")
return
if video_metadata.cover_url and (not self.cover_image or force_update):
@ -336,8 +350,8 @@ class Series(TimeStampedModel):
fname = f"{self.name}_{self.uuid}.jpg"
self.cover_image.save(fname, ContentFile(r.content), save=True)
self.plot = video_metadata.plot
self.imdb_rating = video_metadata.imdb_rating
self.plot = video_metadata.plot or ""
self.imdb_rating = getattr(video_metadata, "imdb_rating", None)
self.save()
if video_metadata.genres:

View File

@ -1,4 +1,5 @@
import logging
import os
import pendulum
from django.conf import settings
@ -8,6 +9,8 @@ from videos.metadata import VideoMetadata, VideoType
TMDB_KEY = getattr(settings, "TMDB_API_KEY", "")
os.environ.setdefault("TMDB_API_KEY", TMDB_KEY)
tmdb = TMDb(key=TMDB_KEY, language="en-US", region="US")
TMDB_IMAGE_URL = "https://image.tmdb.org/t/p/original"
@ -43,6 +46,28 @@ def lookup_video_from_tmdb(name_or_id: str, kind: str = "movie") -> VideoMetadat
) # TODO: enrich this with TMDB url
video_metadata.year = pendulum.parse(media.release_date).year
video_metadata.genres = [g.get("name", "") for g in media.genres]
video_metadata.tmdb_id = media.id
video_metadata.base_run_time_seconds = media.runtime * 60
video_metadata.plot = media.overview
video_metadata.overview = media.overview
video_metadata.tmdb_rating = media.vote_average
if len(tmdb_result.tv_results) > 0:
media = TV().details(tmdb_result.tv_results[0].id)
video_metadata.video_type = VideoType.TV_EPISODE.value
video_metadata.title = media.name
video_metadata.cover_url = (
TMDB_IMAGE_URL + media.poster_path
)
video_metadata.year = pendulum.parse(media.first_air_date).year if media.first_air_date else None
video_metadata.genres = [g.get("name", "") for g in media.genres]
video_metadata.tmdb_id = media.id
video_metadata.base_run_time_seconds = (
media.episode_run_time[0] * 60 if media.episode_run_time else 1800
)
video_metadata.plot = media.overview
video_metadata.overview = media.overview
video_metadata.tmdb_rating = media.vote_average
if len(tmdb_result.tv_episode_results) > 0:
video_metadata.video_type = VideoType.TV_EPISODE.value
@ -63,15 +88,12 @@ def lookup_video_from_tmdb(name_or_id: str, kind: str = "movie") -> VideoMetadat
series.save()
video_metadata.tv_series_id = series.id
video_metadata.tmdb_id = media.id
video_metadata.plot = media.overview
video_metadata.overview = media.overview
if not media:
logger.warning("Video not found on TMDB", extra={"imdb_id": imdb_id})
return video_metadata
video_metadata.tmdb_id = media.id
video_metadata.base_run_time_seconds = media.runtime * 60
video_metadata.plot = media.overview
video_metadata.overview = media.overview
video_metadata.tmdb_rating = media.vote_average
# video_metadata.next_imdb_id = imdb_result.get("next episode", None)
return video_metadata

View File

@ -3,3 +3,6 @@ from django.apps import AppConfig
class WebpagesConfig(AppConfig):
name = "webpages"
def ready(self):
import webpages.signals # noqa

View File

@ -0,0 +1,68 @@
import re
from django.core.management.base import BaseCommand
from scrobbles.utils import tokenize_title_to_tags
from webpages.models import WebPage
def _clean(s: str) -> str:
return re.sub(r"[^\x20-\x7e]", "", s)
class Command(BaseCommand):
help = "Backfill auto tags on webpages from domain and title"
def add_arguments(self, parser):
parser.add_argument(
"--commit",
action="store_true",
help="Actually add tags",
)
def handle(self, *args, **options):
commit = options["commit"]
webpages = WebPage.objects.all()
total = webpages.count()
updated_count = 0
skipped_count = 0
for i, webpage in enumerate(webpages.iterator(), start=1):
new_tags = set()
if webpage.domain:
parts = webpage.domain.root.split(".")
for part in parts:
part = part.strip().lower()
if part and part != "www":
new_tags.add(part)
if webpage.title:
title_tags = tokenize_title_to_tags(webpage.title)
new_tags.update(title_tags)
existing_tags = {
t.name for t in webpage.tags.all()
}
tags_to_add = new_tags - existing_tags
if tags_to_add:
updated_count += 1
if commit:
for tag in tags_to_add:
webpage.tags.add(tag)
self.stdout.write(
f"[{i}/{total}] Added tags to {_clean(str(webpage))}: "
f"{sorted(tags_to_add)}"
)
else:
self.stdout.write(
f"[{i}/{total}] [DRY RUN] Would add tags to "
f"{_clean(str(webpage))}: {sorted(tags_to_add)}"
)
else:
skipped_count += 1
self.stdout.write(f"\nDone. {updated_count} webpages to update, "
f"{skipped_count} already up to date.")

View File

@ -2,6 +2,8 @@ import logging
from typing import Dict, Optional
from uuid import uuid4
import cloudscraper
from curl_cffi import requests as curl_requests
import pendulum
import requests
import trafilatura
@ -45,6 +47,37 @@ class Domain(TimeStampedModel):
)
def _fetch_url_raw(url: str) -> Optional[str]:
"""Fetch raw HTML for a URL.
Tries three strategies in order:
1. trafilatura (standard, works for most sites)
2. cloudscraper (handles basic Cloudflare JS challenges)
3. curl_cffi (impersonates browser TLS fingerprint for aggressive
Cloudflare / bot detection)
"""
raw = trafilatura.fetch_url(url)
if raw:
return raw
logger.debug("trafilatura returned nothing for %s, trying cloudscraper", url)
try:
scraper = cloudscraper.create_scraper()
resp = scraper.get(url, timeout=30)
resp.raise_for_status()
return resp.text
except Exception as exc:
logger.debug("cloudscraper failed for %s: %s, trying curl_cffi", url, exc)
try:
resp = curl_requests.get(
url, impersonate="chrome124", timeout=30
)
resp.raise_for_status()
return resp.text
except Exception as e:
logger.warning("curl_cffi also failed for %s: %s", url, e)
return None
class WebPage(ScrobblableMixin):
COMPLETION_PERCENT = getattr(settings, "WEBSITE_COMPLETION_PERCENT", 100)
@ -80,7 +113,7 @@ class WebPage(ScrobblableMixin):
def _update_extract_from_web(self, raw_text: str = "", force=True):
if not raw_text:
raw_text = requests.get(self.url, headers=headers).text
raw_text = _fetch_url_raw(self.url)
if not self.extract or force:
self.extract = trafilatura.extract(raw_text)
self.save(update_fields=["extract"])
@ -259,7 +292,7 @@ class WebPage(ScrobblableMixin):
return False
def fetch_data_from_web(self, save=True, force=True):
raw_text = trafilatura.fetch_url(self.url)
raw_text = _fetch_url_raw(self.url)
if not self.extract or force:
self.extract = trafilatura.extract(
raw_text,

View File

@ -0,0 +1,29 @@
import logging
from django.db.models.signals import post_save
from django.dispatch import receiver
from scrobbles.utils import tokenize_title_to_tags
from webpages.models import WebPage
logger = logging.getLogger(__name__)
@receiver(post_save, sender=WebPage)
def add_auto_tags_to_webpage(sender, instance, **kwargs):
existing_tags = {t.name for t in instance.tags.all()}
if instance.domain:
domain_parts = instance.domain.root.split(".")
for part in domain_parts:
part = part.strip().lower()
if part and part != "www" and part not in existing_tags:
instance.tags.add(part)
existing_tags.add(part)
if instance.title:
title_tags = tokenize_title_to_tags(instance.title)
for tag in title_tags:
if tag not in existing_tags:
instance.tags.add(tag)
existing_tags.add(tag)

View File

@ -16,8 +16,23 @@ from webpages.models import WebPage
class WebPageListView(ScrobbleableListView):
model = WebPage
def get_queryset(self):
queryset = super().get_queryset()
tag_name = self.request.GET.get("tag")
if tag_name:
from django.contrib.contenttypes.models import ContentType
from taggit.models import TaggedItem
ct = ContentType.objects.get_for_model(WebPage)
webpages = TaggedItem.objects.filter(
content_type=ct,
tag__name__iexact=tag_name,
).values_list("object_id", flat=True)
queryset = queryset.filter(pk__in=webpages)
return queryset
def get_context_data(self, **kwargs):
context_data = super().get_context_data(**kwargs)
context_data["active_tag"] = self.request.GET.get("tag", "")
user = self.request.user
now = timezone.now()
start_day_of_week = now - datetime.timedelta(days=now.weekday())

View File

@ -4,46 +4,55 @@ from importlib.metadata import version as get_version
from pathlib import Path
_GIT_COMMIT = None
def version_info(request):
global _GIT_COMMIT
try:
app_version = get_version("vrobbler")
except Exception:
app_version = "unknown"
commit = os.environ.get("VROBBLER_COMMIT")
if not commit:
# Try to import from _commit.py module first
if commit:
return {"app_version": app_version, "git_commit": commit}
if _GIT_COMMIT is not None:
return {"app_version": app_version, "git_commit": _GIT_COMMIT}
try:
from vrobbler._commit import commit as _commit
except ImportError:
pass
else:
if _commit and _commit != "unknown":
_GIT_COMMIT = _commit
return {"app_version": app_version, "git_commit": _GIT_COMMIT}
commit_file = Path("/var/lib/vrobbler/commit.txt")
if commit_file.exists():
try:
from vrobbler._commit import commit as _commit
except ImportError:
commit = commit_file.read_text().strip()
if commit:
_GIT_COMMIT = commit
return {"app_version": app_version, "git_commit": _GIT_COMMIT}
except OSError:
pass
else:
if _commit and _commit != "unknown":
return {"app_version": app_version, "git_commit": _commit}
# Try to read from commit file (written during deploy)
commit_file = Path("/var/lib/vrobbler/commit.txt")
if commit_file.exists():
try:
commit = commit_file.read_text().strip()
if commit:
return {"app_version": app_version, "git_commit": commit}
except OSError:
pass
# Fall back to git command
PROJECT_ROOT = Path(__file__).resolve().parent.parent
try:
commit = (
subprocess.check_output(
["git", "rev-parse", "--short", "HEAD"],
cwd=PROJECT_ROOT,
stderr=subprocess.DEVNULL,
)
.decode("utf-8")
.strip()
PROJECT_ROOT = Path(__file__).resolve().parent.parent
try:
_GIT_COMMIT = (
subprocess.check_output(
["git", "rev-parse", "--short", "HEAD"],
cwd=PROJECT_ROOT,
stderr=subprocess.DEVNULL,
)
except (subprocess.SubprocessError, FileNotFoundError):
commit = "unknown"
.decode("utf-8")
.strip()
)
except (subprocess.SubprocessError, FileNotFoundError):
_GIT_COMMIT = "unknown"
return {"app_version": app_version, "git_commit": commit}
return {"app_version": app_version, "git_commit": _GIT_COMMIT}

View File

@ -122,6 +122,15 @@ CELERY_ACCEPT_CONTENT = ["json"]
CELERY_RESULT_EXTENDED = True
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_TASK_CREATE_MISSING_QUEUES = True
CELERY_TASK_ROUTES = {
"scrobbles.tasks.update_charts_for_timestamp": {"queue": "charts"},
"scrobbles.tasks.create_yesterdays_charts": {"queue": "charts"},
"scrobbles.tasks.rebuild_weekly_charts": {"queue": "charts"},
"scrobbles.tasks.rebuild_monthly_charts": {"queue": "charts"},
"scrobbles.tasks.rebuild_yearly_charts": {"queue": "charts"},
}
CELERY_BEAT_SCHEDULE = {
"build-yesterdays-charts": {
"task": "scrobbles.tasks.create_yesterdays_charts",
@ -219,6 +228,7 @@ INSTALLED_APPS = [
"foods",
"lifeevents",
"moods",
"discgolf",
"birds",
"mathfilters",
"rest_framework",
@ -226,6 +236,7 @@ INSTALLED_APPS = [
"allauth.account",
"allauth.socialaccount",
"django_celery_results",
"mcp_server",
]
SITE_ID = 1
@ -319,6 +330,13 @@ REST_FRAMEWORK = {
"PAGE_SIZE": 200,
}
DJANGO_MCP_AUTHENTICATION_CLASSES = [
"oauth2_provider.contrib.rest_framework.OAuth2Authentication",
"rest_framework.authentication.TokenAuthentication",
"rest_framework.authentication.SessionAuthentication",
]
DJANGO_MCP_GET_SERVER_INSTRUCTIONS_TOOL = True
LOGIN_REDIRECT_URL = "/"
AUTH_PASSWORD_VALIDATORS = [
@ -352,11 +370,11 @@ USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.1/howto/static-files/
#
from storages.backends import s3boto3
USE_S3_STORAGE = os.getenv("VROBBLER_USE_S3", "False").lower() in TRUTHY
if USE_S3_STORAGE:
from storages.backends import s3boto3
AWS_S3_ENDPOINT_URL = os.getenv("AWS_S3_ENDPOINT_URL", "")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME", "")
AWS_S3_ACCESS_KEY_ID = os.getenv("AWS_S3_ACCESS_KEY_ID")
@ -379,6 +397,8 @@ else:
MEDIA_URL = os.getenv("VROBBLER_MEDIA_URL", "/media/")
SCIHUB_DOMAIN = os.getenv("VROBBLER_SCIHUB_DOMAIN", "sci-hub.st")
JSON_LOGGING = os.getenv("VROBBLER_JSON_LOGGING", "false").lower() in TRUTHY
LOG_TYPE = "json" if JSON_LOGGING else "log"

Some files were not shown because too many files have changed in this diff Show More