Compare commits

...

38 Commits
0.16.4 ... 17.0

Author SHA1 Message Date
bffbf47c2f [release] 17.0 2025-06-24 10:56:14 -04:00
f4e81da533 [project] Completing task for label bug 2025-06-24 10:55:01 -04:00
4b7f5459be [project] Add new task for TV series bug 2025-06-24 10:53:57 -04:00
c68b0e9d7e [tasks] Fix bug in emacs labels 2025-06-24 10:52:47 -04:00
32ec65116b [tasks] Move to single context for task titles 2025-06-22 21:26:20 -04:00
da8d26fcd9 [trails] Add alltrails and gaiagps ids to model 2025-06-22 20:26:37 -04:00
d33954e494 [project] Tidy up todos 2025-06-17 10:53:26 -04:00
1b306d6493 [project] Move todos.org file to PROJECT.org 2025-06-17 10:50:30 -04:00
c881143e1b [todos] Update backlog with tasks from Todoist 2025-06-17 10:18:48 -04:00
141700fcb3 [music] Fixes RYM slug including commas 2025-06-16 09:38:19 -04:00
7357b5bfec [todos] Update todo formatting 2025-06-16 09:35:30 -04:00
99cabd0007 [todos] Finish another task! 2025-06-13 12:35:24 -04:00
cf77e12cc3 [tasks] Add description of task to the template 2025-06-13 12:34:55 -04:00
3f2cbbb34a [videos] Fix TMDB lookup for movies 2025-06-13 12:10:43 -04:00
650ecf12c6 Merge branch 'main' into develop 2025-06-13 11:55:04 -04:00
d7cc009d07 [ci] Make deploys on happen on tags 2025-06-13 11:52:14 -04:00
a872cf3611 [templates] Use TMDB rating ID if available 2025-06-13 11:47:42 -04:00
1f9713312b [videos] Fix lookup for mevie posters 2025-06-13 11:47:04 -04:00
159e555d7c [templates] Remove global counts from home page 2025-06-13 11:33:21 -04:00
981f4f9c9a [videos] Quick fix 2025-06-13 11:23:37 -04:00
ddd5ce1392 Merge branch 'develop' 2025-06-13 11:21:07 -04:00
7a75b31b56 [todos] Update task for metadata fixing 2025-06-13 11:20:34 -04:00
24ac545f55 [videos] Switch to TMDB for scraping videos 2025-06-13 11:19:15 -04:00
d5da8ae701 [todos] Update priority task 2025-06-13 09:43:08 -04:00
53a04d064d [videos] At least stop IMDB from breaking 2025-06-13 09:15:48 -04:00
c0871a3b9e [scrobbles] Fix failing tests 2025-06-12 09:57:32 -04:00
d917dd8b2c [tasks] Fix source checking for emacs/orgmode tasks when updating 2025-06-12 09:54:16 -04:00
6fc8084f2d [scrobbles] Fix log of media type 2025-06-11 12:01:02 -04:00
41d6fe8ff6 [todos] Add new task 2025-06-10 11:27:37 -04:00
73838312cd [tasks] Clean up emacs webhook flow 2025-06-10 11:06:47 -04:00
eb2dd4c839 [todos] Clean up todos and add new milestone 17 2025-06-10 10:05:40 -04:00
1a0c4f69f0 Merge branch 'develop' 2025-06-09 11:26:25 -04:00
356e579558 [brickset] Little fix to scrobble of bricksets 2025-06-09 10:40:12 -04:00
e980e3c5c9 [brickset] Fix manual scrobbling and admin 2025-06-09 10:36:13 -04:00
8773542099 [tasks] Fix source being orgmode not emacs 2025-06-08 03:05:43 -04:00
70378c9968 [tasks] Fix updating and stopping tasks from Emacs 2025-06-08 02:40:07 -04:00
c871087496 [tasks] First try at adding an emacs webhook 2025-06-08 01:09:03 -04:00
059d7780a0 [templates] Add counts and durations to new tables 2025-06-07 20:02:21 -04:00
33 changed files with 958 additions and 305 deletions

View File

@ -39,8 +39,8 @@ steps:
- vrobbler collectstatic --noinput
- immortalctl restart celery && immortalctl restart vrobbler
when:
branch:
- main
ref:
- refs/tags/*
- name: build success notification
image: parrazam/drone-ntfy:0.3-linux-amd64
when:

View File

@ -1,196 +1,64 @@
#+title: TODOs
A fun way to keep track of things in the project to fix or improve.
* Backlog [6/23]
** TODO [#A] Fix small bug in views for TV series where next episode is now None :vrobbler:bug:personal:videos:
* Version 0.16.0
** TODO [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music:
Effectively, any track that comes in without a MusicBrainz ID does some funky
lookup where it doesn't find a track without an MB id and the track title /
artist combination and creates a new track everytime. This has to be cleaend up
by condensing the duplicated tracks into the original proper track.
But it opens a bigger quesiton about how much MB id should the drive the app
lookup. If it can't be depended on to exist from all sources, it really can't be
canonical. Instead, the combination of track title / artist is really the best
we can do. Last.fm also has this problem, where it doesn't know about albums and
definitely does not know or care about MB ids.
** DONE Add a user profile page with ability to change settings :profiles:improvement:
- Note taken on [2025-04-04 Fri 10:51]
[[orgit-rev:~/src/code.unbl.ink/secstate/vrobbler/::93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44][~/src/code.unbl.ink/secstate/vrobbler/ (magit-rev 93c16d8)]]
https://code.unbl.ink/secstate/vrobbler/commit/93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44
** DONE What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
- Note taken on [2025-04-04 Fri 10:46]
Nothing. Over the last few months I built out a youtube model in videos and
use a bookmarklet scrobbling pattern. Now web-scrobbler is just disabled for
Youtube.
May want to revisit this at some point and only scrobble tracks from Youtube,
because many people use YT for music listening.
** DONE [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
CLOSED: [2023-04-06 Thu 14:09]
** DONE Add a "stop_timestamp" so we don't rely on content length :improvement:scrobbling:
CLOSED: [2023-04-02 Sun 23:58]
Essentially, we currently have the timestamp as when the content began
scrobbling and then calculate the finish time from the length of the content.
This works pretty well because we know how long most things are.
But in some cases, sports events or long podcasts, we may start mid-way through
an event or finish halfway through but still want to mark it as done. In these
cases, knowing the finish time could be useful, especially when interfacing with
other scrobblers which may have different definitions of when a scrobble
finishes or started.
** DONE Fix bug with Various Artist albums being labeled with first artist as album artist :scrobbling:bug:music:
CLOSED: [2023-03-27 Mon 20:18]
:LOGBOOK:
CLOCK: [2023-03-26 Sun 22:01]--[2023-03-27 Mon 01:07] => 3:06
:END:
** DONE Fix bug with weekly aggregator being blank on Sundays :aggregators:music:bug:
CLOSED: [2023-03-26 Sun 13:52]
** DONE Fix KoReader scrobbling to use pages rather than time of last read :scrobbling:books:improvement:
CLOSED: [2023-03-26 Sun 13:51]
:LOGBOOK:
CLOCK: [2023-03-26 Sun 13:11]--[2023-03-26 Sun 13:51] => 0:40
:END:
** DONE [#A] Add django-storage to store files on S3 :settings:improvement:
CLOSED: [2023-03-24 Fri 14:46]
:LOGBOOK:
CLOCK: [2023-03-24 Fri 10:47]--[2023-03-24 Fri 14:46] => 3:59
CLOCK: [2023-03-24 Fri 10:36]--[2023-03-24 Fri 10:40] => 0:04
:END:
** DONE Fix vrobbler settings not using booleans :settings:bug:
CLOSED: [2023-03-24 Fri 10:45]
:LOGBOOK:
CLOCK: [2023-03-24 Fri 10:40]--[2023-03-24 Fri 10:46] => 0:06
:END:
** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
CLOSED: [2023-03-24 Fri 00:31]
The live view will be blank every Monday, no reason to tie it to a day of the
week. It should be "the last 7 days"
** DONE [#B] Implement a detail view for TV shows :improvement:views:
CLOSED: [2023-03-22 Wed 17:05]
** DONE [#B] Implement a detail view for Movies :improvement:views:
CLOSED: [2023-03-22 Wed 17:05]
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
CLOSED: [2023-03-22 Wed 17:04]
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
CLOSED: [2023-03-22 Wed 17:01]
** DONE Add live chart view like Maloja :improvement:views:
CLOSED: [2023-03-07 Tue 11:13]
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
CLOSED: [2023-03-22 Wed 17:06]
An example:
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
This is actually going to be moot because we can import from LastFM, and
web-scrobbler integrates well with LastFM. The only thing to think through here
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
from Youtube (all videos get pushed, sigh).
** DONE Add Amazon scraper to look up books when OL fails :books:improvement:
This turned out to be a non-starter ... Amazon is aggressive at disallowing
scraping quality. And all the OSS tools out there are stuck in an arms race
trying to keep them from breaking.
That said, Google Books actually has a decent API (for now), and I've built this
out using that.
** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug:
This was fixed a while ago, but there's a new manifested bug. Going to create a
separate bug tracking ticket for that.
* Version 0.11.4
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
CLOSED: [2023-03-07 Tue 11:11]
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
CLOSED: [2023-03-07 Tue 11:11]
** DONE [#A] Fix fetching artwork without release group :bug:
CLOSED: [2023-01-29 Sun 14:27]
When we get artwork from Musicbrianz, and it's not found, we should check for
release groups as well. This will stop issues with missing artwork because of
obscure MB release matches.
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
CLOSED: [2023-01-30 Mon 18:31]
:LOGBOOK:
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
:END:
If we play music from Jellyfin and the track reaches 90% completion, the
scrobbling goes crazy and starts creating new scrobbles with every update.
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
as complete for the following conditions:
- Play stopped and percent played beyond 90%
- Play completely finished
But if we keep listening beyond 90, we should basically ignore updates (or just
update the existing scrobble)
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
CLOSED: [2023-02-03 Fri 16:52]
An example of the format:
#+begin_src csv
,
#AUDIOSCROBBLER/1.1
#TZ/UNKNOWN
#CLIENT/Rockbox sansaclipplus $Revision$
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
311 311 Jackolanterns Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
,
#+begin_src python
ERROR django.request:241 log_response Internal Server Error: /series/c24100d1-da45-4abe-86bf-27cfce9b1f89/
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
return self.dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/mixins.py", line 73, in dispatch
return super().dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
return handler(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/views/generic/detail.py", line 109, in get
context = self.get_context_data(object=self.object)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/videos/views.py", line 33, in get_context_data
context_data["next_episode_id"] = "tt" + next_episode_id
~~~~~^~~~~~~~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str
#+end_src
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
CLOSED: [2023-02-17 Fri 00:10]
This would allow a few nice flows. One, you'd be able to record the play of an
entire album by just dropping the muscibrainz_id in. This could be helpful for
offline listening. It would also mean bad metadata from mopidy would not break
scrobbling.
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
CLOSED: [2023-02-17 Fri 00:11]
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
CLOSED: [2023-03-07 Tue 11:09]
** TODO [#A] Send periodic check notifications for board games and beer that ask if you're still scrobbling :vrobbler:personal:project:beers:boardgames:notifications:feature:
** TODO [#A] Fix raw text webpage title not truncating to 254 chars :vrobbler:personal:bug:webpages:
** TODO [#A] Fix koreader scrobble imports to use DST properly :vrobbler:personal:bug:books:imports:
The page data has the canonical date something was read in it, but it seems to be an hour off. I traced this back to being off during DST, so we just need the importer to be aware of whether a user is using DST or not and roll back an hour for part of the year.
Given a UUID from musicbrainz, we should be able to scrobble an album or
individual track.
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
CLOSED: [2023-03-07 Tue 11:10]
:LOGBOOK:
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
:END:
Maloja does this cool thing where artists and tracks get recorded as the top
track of a given week, month or year. They get gold, silver or bronze stars for
their place in the time period.
I could see this being implemented as a separate Chart table which gets
populated at the end of a time period and has a start and end date that defines
a period, along with a one, two, three instance.
Of course, it could also be a data model without a table, where it runs some fun
calculations, stores it's values in Redis as a long-term lookup table and just
has to re-populate when the server restarts.
* Backlog
Also, we'd need to adjust any old scrobbles that took place with DST off to roll them back by an hour.
** TODO [#A] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
** TODO [#B] Add AllTrails as a source for Trail data :vrobbler:trails:feature:personal:project:
Pretty clear, I would love to make trails more useful. Historically I wasn't
hiking a lot, which made the source for this a bit silly. But it's clear that
AllTrails is the best source, though having TrailForks is nice to.
** TODO [#B] Add `garmin_activity_id` to the TrailMetadataLog class :vrobbler:trails:feature:personal:project:
Would be nice to have some loose connection to the actual event in my Garmin profile.
** TODO [#B] Explore a way to add metadata editing to scrobbles after saving :vrobbler:spike:scrobbling:personal:project:
Could be as simple as a JSON form on the scrobble detail page (do I have have one of those yet?).
** TODO [#B] Explore a good way to show notes and descriptions from scrobbles to users :personal:project:scrobbling:vrobbler:spike:
** TODO [#B] Add webdav syncing to retroarch imports :vrobbler:videogames:webdav:feature:project:personal:
** TODO [#B] Fix task app to only use one tag for the context a task was done in and allow configurable contexts by user profile :personal:vrobbler:feature:tasks:project:
** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
** TODO [#C] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal:
Rather than pick up an existing Podcast using the podcast title in the mopidy
file name, Vrobbler creates a new podcast with no enriched data. Not a big deal
for my use as the volume of podcasts I listen to makes manual fixes easy. But
it's annoying.
** TODO [#C] Move to using more robust mopidy-webhooks pacakge form pypi :utility:improvement:
:PROPERTIES:
:ID: ab31fdc3-359c-1b1d-6b9d-546b476021ba
:END:
*** Example payloads from mopidy-webhooks
**** Podcast playback ended
#+begin_src json
@ -481,4 +349,260 @@ has to re-populate when the server restarts.
}
}
#+end_src
** TODO Fix bug in podcast scrobbling where a second scrobble is created after completion :scrobbling:podcasts:bug:
** TODO [#C] User should be able to enable auto trail tracking via amail reader with Garmin LiveTrack URLs :vrobbler:trails:project:feature:personal:
** TODO [#C] Allow users to see tasks on calendar view :vrobbler:personal:project:templates:feature:
https://codepen.io/oliviale/pen/QYqybo
** TODO [#C] Come pu with a possible flow using WebDAV and super-productivity for tasks :personal:feature:project:vrobbler:tasks:
* Version 17.0
** DONE [#A] Fix bug in new task label lookup for Emacs/Org-mode :vrobbler:bug:tasks:
:PROPERTIES:
:ID: 683fb109-dfc4-85e4-80f0-ea618434f61e
:END:
** DONE [#C] Replace commas in the bandcamp URL for artists with nothing :vrobbler:music:bug:personal:
:PROPERTIES:
:ID: 9b30d67b-91f0-a480-dfaa-5d9dc090e76c
:END:
- Note taken on [2025-06-16 Mon 09:36]
This firt appeared with Black Country, New Road, where the RYM slug generator
leaves commas in and ends up sending you to a 404. I suspect this wont be the
first tweak we'll need to this, as the RYM link creator is really just
guessing based on the artist name at the path.
** DONE [#A] Investigate new source of video metadata :personal:project:video:imdb:
:PROPERTIES:
:ID: df2b486c-1170-5199-c312-9bc87760d962
:END:
Cinemagoer broke and I probably should find a more reilable source of video data.
- Note taken on [2025-06-13 Fri 11:19]
TMDB is much more reliable, but does require an API key. That's all setup now,
so hopefully this breaking IMDB crap is over.
** DONE [#A] IMDB video lookups are failing :personal:bug:video:imdb:
:PROPERTIES:
:ID: 38f1081f-37b4-f4f2-79aa-c1e87eca4b69
:END:
<2025-06-13 Fri>
- Note taken on [2025-06-13 Fri 08:24]
Looks like Cinemagoer is broken: https://github.com/cinemagoer/cinemagoer/issues/537
** DONE [#A] Emacs is not syncing notes :personal:scrobbling:emacs:bug:
:PROPERTIES:
:ID: c79cd491-b30f-0945-d84b-b8cac7562791
:END:
<2025-06-12 Thu 9:30>
Not sure if the problem is in my Emacs hook sending or Vrobbler itself.
- Note taken on [2025-06-12 Thu 09:47]
Adding a quick note to check on it
- Note taken on [2025-06-12 Thu 09:50]
Ah ha. All the messing about with the source field meant that I was looking
for `emacs` as a source but the hook was initially setting sources to
`orgmode` I think I prefer `orgmode` as the source, so updating it thusly.
Fixed in `490d60cbbb1f8bf90b5fc47d8685b15bdc1d485b`
** DONE [#A] Show the description of a task in the string rep for a scrobble of a Task :personal:project:scrobbling:vrobbler:feature:
:PROPERTIES:
:ID: df58f8d0-fa4a-2037-c7d7-e5388c239042
:END:
* Version 0.16.0
** DONE [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music:
:PROPERTIES:
:ID: 670e8634-49b5-dce9-1684-14f2ffb797f1
:END:
Effectively, any track that comes in without a MusicBrainz ID does some funky
lookup where it doesn't find a track without an MB id and the track title /
artist combination and creates a new track every time. This has to be cleaned up
by condensing the duplicated tracks into the original proper track.
But it opens a bigger question about how much MB id should the drive the app
lookup. If it can't be depended on to exist from all sources, it really can't be
canonical. Instead, the combination of track title / artist is really the best
we can do. Last.fm also has this problem, where it doesn't know about albums and
definitely does not know or care about MB ids.
** DONE Add a user profile page with ability to change settings :profiles:improvement:
- Note taken on [2025-04-04 Fri 10:51]
[[orgit-rev:~/src/code.unbl.ink/secstate/vrobbler/::93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44][~/src/code.unbl.ink/secstate/vrobbler/ (magit-rev 93c16d8)]]
https://code.unbl.ink/secstate/vrobbler/commit/93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44
** DONE What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
- Note taken on [2025-04-04 Fri 10:46]
Nothing. Over the last few months I built out a youtube model in videos and
use a bookmarklet scrobbling pattern. Now web-scrobbler is just disabled for
Youtube.
May want to revisit this at some point and only scrobble tracks from Youtube,
because many people use YT for music listening.
** DONE [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
CLOSED: [2023-04-06 Thu 14:09]
** DONE Add a "stop_timestamp" so we don't rely on content length :improvement:scrobbling:
CLOSED: [2023-04-02 Sun 23:58]
Essentially, we currently have the timestamp as when the content began
scrobbling and then calculate the finish time from the length of the content.
This works pretty well because we know how long most things are.
But in some cases, sports events or long podcasts, we may start mid-way through
an event or finish halfway through but still want to mark it as done. In these
cases, knowing the finish time could be useful, especially when interfacing with
other scrobblers which may have different definitions of when a scrobble
finishes or started.
** DONE Fix bug with Various Artist albums being labeled with first artist as album artist :scrobbling:bug:music:
CLOSED: [2023-03-27 Mon 20:18]
:LOGBOOK:
CLOCK: [2023-03-26 Sun 22:01]--[2023-03-27 Mon 01:07] => 3:06
:END:
** DONE Fix bug with weekly aggregator being blank on Sundays :aggregators:music:bug:
CLOSED: [2023-03-26 Sun 13:52]
** DONE Fix KoReader scrobbling to use pages rather than time of last read :scrobbling:books:improvement:
CLOSED: [2023-03-26 Sun 13:51]
:LOGBOOK:
CLOCK: [2023-03-26 Sun 13:11]--[2023-03-26 Sun 13:51] => 0:40
:END:
** DONE [#A] Add django-storage to store files on S3 :settings:improvement:
CLOSED: [2023-03-24 Fri 14:46]
:LOGBOOK:
CLOCK: [2023-03-24 Fri 10:47]--[2023-03-24 Fri 14:46] => 3:59
CLOCK: [2023-03-24 Fri 10:36]--[2023-03-24 Fri 10:40] => 0:04
:END:
** DONE Fix vrobbler settings not using booleans :settings:bug:
CLOSED: [2023-03-24 Fri 10:45]
:LOGBOOK:
CLOCK: [2023-03-24 Fri 10:40]--[2023-03-24 Fri 10:46] => 0:06
:END:
** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
CLOSED: [2023-03-24 Fri 00:31]
The live view will be blank every Monday, no reason to tie it to a day of the
week. It should be "the last 7 days"
** DONE [#B] Implement a detail view for TV shows :improvement:views:
CLOSED: [2023-03-22 Wed 17:05]
** DONE [#B] Implement a detail view for Movies :improvement:views:
CLOSED: [2023-03-22 Wed 17:05]
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
CLOSED: [2023-03-22 Wed 17:04]
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
CLOSED: [2023-03-22 Wed 17:01]
** DONE Add live chart view like Maloja :improvement:views:
CLOSED: [2023-03-07 Tue 11:13]
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
CLOSED: [2023-03-22 Wed 17:06]
An example:
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
This is actually going to be moot because we can import from LastFM, and
web-scrobbler integrates well with LastFM. The only thing to think through here
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
from Youtube (all videos get pushed, sigh).
** DONE Add Amazon scraper to look up books when OL fails :books:improvement:
This turned out to be a non-starter ... Amazon is aggressive at disallowing
scraping quality. And all the OSS tools out there are stuck in an arms race
trying to keep them from breaking.
That said, Google Books actually has a decent API (for now), and I've built this
out using that.
** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug:
This was fixed a while ago, but there's a new manifested bug. Going to create a
separate bug tracking ticket for that.
* Version 0.11.4
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
CLOSED: [2023-03-07 Tue 11:11]
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
CLOSED: [2023-03-07 Tue 11:11]
** DONE [#A] Fix fetching artwork without release group :bug:
CLOSED: [2023-01-29 Sun 14:27]
When we get artwork from Musicbrianz, and it's not found, we should check for
release groups as well. This will stop issues with missing artwork because of
obscure MB release matches.
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
CLOSED: [2023-01-30 Mon 18:31]
:LOGBOOK:
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
:END:
If we play music from Jellyfin and the track reaches 90% completion, the
scrobbling goes crazy and starts creating new scrobbles with every update.
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
as complete for the following conditions:
- Play stopped and percent played beyond 90%
- Play completely finished
But if we keep listening beyond 90, we should basically ignore updates (or just
update the existing scrobble)
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
CLOSED: [2023-02-03 Fri 16:52]
An example of the format:
#+begin_src csv
,
#AUDIOSCROBBLER/1.1
#TZ/UNKNOWN
#CLIENT/Rockbox sansaclipplus $Revision$
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
311 311 Jackolanterns Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
,
#+end_src
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
CLOSED: [2023-02-17 Fri 00:10]
This would allow a few nice flows. One, you'd be able to record the play of an
entire album by just dropping the muscibrainz_id in. This could be helpful for
offline listening. It would also mean bad metadata from mopidy would not break
scrobbling.
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
CLOSED: [2023-02-17 Fri 00:11]
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
CLOSED: [2023-03-07 Tue 11:09]
Given a UUID from musicbrainz, we should be able to scrobble an album or
individual track.
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
CLOSED: [2023-03-07 Tue 11:10]
:LOGBOOK:
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
:END:
Maloja does this cool thing where artists and tracks get recorded as the top
track of a given week, month or year. They get gold, silver or bronze stars for
their place in the time period.
I could see this being implemented as a separate Chart table which gets
populated at the end of a time period and has a start and end date that defines
a period, along with a one, two, three instance.
Of course, it could also be a data model without a table, where it runs some fun
calculations, stores it's values in Redis as a long-term lookup table and just
has to re-populate when the server restarts.

49
poetry.lock generated
View File

@ -1065,6 +1065,21 @@ ssh = ["bcrypt (>=3.1.5)"]
test = ["certifi", "cryptography-vectors (==43.0.3)", "pretend", "pytest (>=6.2.0)", "pytest-benchmark", "pytest-cov", "pytest-xdist"]
test-randomorder = ["pytest-randomly"]
[[package]]
name = "dacite"
version = "1.9.2"
description = "Simple creation of data classes from dictionaries."
optional = false
python-versions = ">=3.7"
groups = ["main"]
files = [
{file = "dacite-1.9.2-py3-none-any.whl", hash = "sha256:053f7c3f5128ca2e9aceb66892b1a3c8936d02c686e707bee96e19deef4bc4a0"},
{file = "dacite-1.9.2.tar.gz", hash = "sha256:6ccc3b299727c7aa17582f0021f6ae14d5de47c7227932c47fec4cdfefd26f09"},
]
[package.extras]
dev = ["black", "coveralls", "mypy", "pre-commit", "pylint", "pytest (>=5)", "pytest-benchmark", "pytest-cov"]
[[package]]
name = "dataclass-wizard"
version = "0.22.0"
@ -4610,6 +4625,23 @@ files = [
[package.dependencies]
rapidfuzz = ">=3.0.0,<4.0.0"
[[package]]
name = "themoviedb"
version = "1.0.2"
description = "A modern and easy to use API wrapper for The Movie Database (TMDb) API v3 written in Python"
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "themoviedb-1.0.2-py3-none-any.whl", hash = "sha256:badf85e91010c7085509f40270bf2a40ea30ee5ef3ed6fb3ec332c5e50adb576"},
{file = "themoviedb-1.0.2.tar.gz", hash = "sha256:7835615142a44e7ca25e48645a3a3c5e06b382e8c518c38c3537effa9a2596ce"},
]
[package.dependencies]
aiohttp = ">=3.8.4"
dacite = ">=1.8.0"
requests = ">=2.31.0"
[[package]]
name = "time-machine"
version = "2.16.0"
@ -4710,6 +4742,21 @@ files = [
{file = "tld-0.13.tar.gz", hash = "sha256:93dde5e1c04bdf1844976eae440706379d21f4ab235b73c05d7483e074fb5629"},
]
[[package]]
name = "tmdbv3api"
version = "1.9.0"
description = "A lightweight Python library for The Movie Database (TMDb) API."
optional = false
python-versions = "*"
groups = ["main"]
files = [
{file = "tmdbv3api-1.9.0-py3-none-any.whl", hash = "sha256:2bcd8c6e8902397860715a71045f200ecc3ee06804ecf786cb4c1e09b2deeba8"},
{file = "tmdbv3api-1.9.0.tar.gz", hash = "sha256:504c5da6b99c4516ff160a01576112d097f209c0534f943c15c4b56cbd92c33b"},
]
[package.dependencies]
requests = "*"
[[package]]
name = "toml"
version = "0.10.2"
@ -5452,4 +5499,4 @@ cffi = ["cffi (>=1.11)"]
[metadata]
lock-version = "2.1"
python-versions = ">=3.9,<3.12"
content-hash = "cdd7f577fe3a4c5c8cc960e0070d93b7ddbb2a7968fab63d72bb039afaa05bbe"
content-hash = "3a483aefea0a3afebf187b17b7df72a158788024ca8121b512b39567fb5ec8ca"

View File

@ -54,6 +54,8 @@ meta-yt = "^0.1.9"
berserk = "^0.13.2"
poetry-bumpversion = "^0.3.3"
orgparse = "^0.4.20250520"
tmdbv3api = "^1.9.0"
themoviedb = "^1.0.2"
[tool.poetry.group.test]
optional = true

View File

@ -16,7 +16,9 @@ class BrickSet(LongPlayScrobblableMixin):
number = models.CharField(max_length=10, **BNULL)
release_year = models.IntegerField(**BNULL)
piece_count = models.IntegerField(**BNULL)
brickset_rating = models.DecimalField(max_digits=3, decimal_places=1, **BNULL)
brickset_rating = models.DecimalField(
max_digits=3, decimal_places=1, **BNULL
)
lego_item_number = models.CharField(max_length=10, **BNULL)
box_image = models.ImageField(upload_to="brickset/boxes/", **BNULL)
box_image_small = ImageSpecField(
@ -48,13 +50,23 @@ class BrickSet(LongPlayScrobblableMixin):
def get_absolute_url(self):
return reverse("bricksets:brickset_detail", kwargs={"slug": self.uuid})
def __str__(self) -> str:
name = str(self.title)
if not self.title:
name = str(self.number)
return name
@property
def logdata_cls(self):
return BrickSetLogData
@classmethod
def find_or_create(cls, title: str) -> "BrickSet":
return cls.objects.filter(title=title).first()
def find_or_create(cls, brickset_id: str) -> "BrickSet":
brickset = cls.objects.filter(number=brickset_id).first()
if not brickset:
# TODO: enrich this from the website
brickset = cls.objects.create(number=brickset_id)
return brickset
@property
def primary_image_url(self) -> str:

View File

@ -16,6 +16,7 @@ class AlbumAdmin(admin.ModelAdmin):
"theaudiodb_mood",
"musicbrainz_id",
)
raw_id_fields = ("album_artist",)
list_filter = (
"theaudiodb_score",
"theaudiodb_genre",
@ -53,6 +54,10 @@ class TrackAdmin(admin.ModelAdmin):
"artist",
"musicbrainz_id",
)
raw_id_fields = (
"album",
"artist",
)
list_filter = ("album", "artist")
search_fields = ("title",)
ordering = ("-created",)

View File

@ -161,7 +161,7 @@ class Artist(TimeStampedModel):
@property
def rym_link(self):
artist_slug = self.name.lower().replace(" ", "-")
artist_slug = self.name.lower().replace(" ", "-").replace(",", "")
return f"https://rateyourmusic.com/artist/{artist_slug}/"
@property

View File

@ -27,7 +27,7 @@ class ScrobbleInline(admin.TabularInline):
"geo_location",
"task",
"mood",
"brickset",
"brick_set",
"trail",
"beer",
"web_page",
@ -122,9 +122,12 @@ class ScrobbleAdmin(admin.ModelAdmin):
"video_game",
"board_game",
"geo_location",
"puzzle",
"paper",
"food",
"task",
"mood",
"brickset",
"brick_set",
"trail",
"beer",
"web_page",

View File

@ -27,14 +27,14 @@ MEDIA_END_PADDING_SECONDS = {
TODOIST_TASK_URL = "https://app.todoist.com/app/task/{id}"
SCROBBLE_CONTENT_URLS = {
"-i": "https://www.imdb.com/title/",
"-s": "https://www.thesportsdb.com/event/",
"-g": "https://boardgamegeek.com/boardgame/",
"-u": "https://untappd.com/",
"-b": "https://www.amazon.com/",
"-t": "https://app.todoist.com/app/task/{id}",
"-i": "https://www.youtube.com/watch?v=",
"-p": "https://www.ipdb.plus/IPDb/puzzle.php?id=",
"-i": ["https://www.imdb.com/title/", "https://www.youtube.com/watch?v="],
"-s": ["https://www.thesportsdb.com/event/"],
"-g": ["https://boardgamegeek.com/boardgame/"],
"-u": ["https://untappd.com/"],
"-b": ["https://www.amazon.com/"],
"-t": ["https://app.todoist.com/app/task/{id}"],
"-p": ["https://www.ipdb.plus/IPDb/puzzle.php?id="],
"-l": ["https://brickset.com/sets/"],
}
EXCLUDE_FROM_NOW_PLAYING = ("GeoLocation",)
@ -49,6 +49,7 @@ MANUAL_SCROBBLE_FNS = {
"-w": "manual_scrobble_webpage",
"-t": "manual_scrobble_task",
"-p": "manual_scrobble_puzzle",
"-l": "manual_scrobble_brickset",
}

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.19 on 2025-06-09 14:33
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("scrobbles", "0069_scrobble_puzzle_alter_scrobble_media_type"),
]
operations = [
migrations.RenameField(
model_name="scrobble",
old_name="brickset",
new_name="brick_set",
),
]

View File

@ -92,7 +92,6 @@ class ScrobblableMixin(TimeStampedModel):
"media_type": self.__class__,
"user_id": user_id,
"scrobble_data": scrobble_data,
"media_type": Scrobble.MediaType.WEBPAGE,
},
)
return Scrobble.create_or_update(self, user_id, scrobble_data)

View File

@ -549,7 +549,7 @@ class Scrobble(TimeStampedModel):
LifeEvent, on_delete=models.DO_NOTHING, **BNULL
)
mood = models.ForeignKey(Mood, on_delete=models.DO_NOTHING, **BNULL)
brickset = models.ForeignKey(
brick_set = models.ForeignKey(
BrickSet, on_delete=models.DO_NOTHING, **BNULL
)
media_type = models.CharField(
@ -635,6 +635,14 @@ class Scrobble(TimeStampedModel):
for scrobble in scrobble_qs:
scrobbles_by_type[scrobble.media_type].append(scrobble)
if not scrobbles_by_type.get(scrobble.media_type + "_count"):
scrobbles_by_type[scrobble.media_type + "_count"] = 0
scrobbles_by_type[scrobble.media_type + "_count"] += 1
if not scrobbles_by_type.get(scrobble.media_type + "_time"):
scrobbles_by_type[scrobble.media_type + "_time"] = 0
scrobbles_by_type[scrobble.media_type + "_time"] += int(
(scrobble.elapsed_time)
)
return scrobbles_by_type
@ -1011,8 +1019,8 @@ class Scrobble(TimeStampedModel):
media_obj = self.life_event
if self.mood:
media_obj = self.mood
if self.brickset:
media_obj = self.brickset
if self.brick_set:
media_obj = self.brick_set
if self.trail:
media_obj = self.trail
if self.beer:

View File

@ -8,6 +8,7 @@ import pytz
from beers.models import Beer
from boardgames.models import BoardGame
from books.models import Book
from bricksets.models import BrickSet
from dateutil.parser import parse
from django.utils import timezone
from locations.constants import LOCATION_PROVIDERS
@ -23,7 +24,7 @@ from scrobbles.constants import (
SCROBBLE_CONTENT_URLS,
)
from scrobbles.models import Scrobble
from scrobbles.utils import convert_to_seconds
from scrobbles.utils import convert_to_seconds, extract_domain
from sports.models import SportEvent
from sports.thesportsdb import lookup_event_from_thesportsdb
from tasks.models import Task
@ -33,8 +34,7 @@ from videos.models import Video
from webpages.models import WebPage
from vrobbler.apps.tasks.constants import (
TODOIST_TITLE_PREFIX_LABELS,
TODOIST_TITLE_SUFFIX_LABELS,
TODOIST_TITLE_LABELS
)
logger = logging.getLogger(__name__)
@ -322,14 +322,12 @@ def manual_scrobble_from_url(
we know about the content type, and routes it to the appropriate media
scrobbler. Otherwise, return nothing."""
content_key = ""
try:
domain = url.split("//")[-1].split("/")[0]
except IndexError:
domain = None
domain = extract_domain(url)
for key, content_url in SCROBBLE_CONTENT_URLS.items():
if domain in content_url:
content_key = key
for key, content_urls in SCROBBLE_CONTENT_URLS.items():
for content_url in content_urls:
if domain in content_url:
content_key = key
item_id = None
if not content_key:
@ -343,8 +341,10 @@ def manual_scrobble_from_url(
except IndexError:
pass
if content_key == "-i":
if content_key == "-i" and "v=" in url:
item_id = url.split("v=")[1].split("&")[0]
elif content_key == "-i" and "title/tt" in url:
item_id = "tt" + str(item_id)
scrobble_fn = MANUAL_SCROBBLE_FNS[content_key]
return eval(scrobble_fn)(item_id, user_id, action=action)
@ -420,10 +420,8 @@ def todoist_scrobble_task(
# user.profile.task_context_tags which will result in context-based tag titles
# We'd also have to migrate existing tasks to the new context based ones (maybe)
for label in todoist_task["todoist_label_list"]:
if label in TODOIST_TITLE_PREFIX_LABELS:
prefix = label
if label in TODOIST_TITLE_SUFFIX_LABELS:
suffix = label
if label in TODOIST_TITLE_LABELS:
title = label
if not prefix and suffix:
logger.warning(
@ -431,8 +429,6 @@ def todoist_scrobble_task(
extra={"todoist_scrobble_task": todoist_task},
)
title = " ".join([prefix.capitalize(), suffix.capitalize()])
task = Task.find_or_create(title)
timestamp = pendulum.parse(todoist_task.get("updated_at", timezone.now()))
@ -500,6 +496,149 @@ def todoist_scrobble_task(
return scrobble
def emacs_scrobble_update_task(
emacs_id: str, emacs_notes: dict, user_id: int
) -> Optional[Scrobble]:
scrobble = Scrobble.objects.filter(
in_progress=True,
user_id=user_id,
log__source_id=emacs_id,
log__source="orgmode",
).first()
if not scrobble:
logger.info(
"[emacs_scrobble_update_task] no task found",
extra={
"emacs_notes": emacs_notes,
"user_id": user_id,
"media_type": Scrobble.MediaType.TASK,
},
)
return
notes_updated = False
for note in emacs_notes:
existing_note_ts = [
n.get("timestamp") for n in scrobble.log.get("notes", [])
]
if not scrobble.log.get('notes"'):
scrobble.log["notes"] = []
if note.get("timestamp") not in existing_note_ts:
scrobble.log["notes"].append(
{note.get("timestamp"): note.get("content")}
)
notes_updated = True
if notes_updated:
scrobble.save(update_fields=["log"])
logger.info(
"[emacs_scrobble_update_task] emacs note added",
extra={
"emacs_note": emacs_notes,
"user_id": user_id,
"media_type": Scrobble.MediaType.TASK,
},
)
return scrobble
def emacs_scrobble_task(
task_data: dict,
user_id: int,
started: bool = False,
stopped: bool = False,
) -> Scrobble | None:
prefix = ""
suffix = ""
source_id = task_data.get("source_id")
for label in task_data.get("labels", []):
if label in TODOIST_TITLE_LABELS:
title = label
if not prefix and not suffix:
logger.warning(
"Missing a prefix and suffix tag for task",
extra={"emacs_scrobble_task": task_data},
)
title = " ".join([prefix.capitalize(), suffix.capitalize()])
task = Task.find_or_create(title)
timestamp = pendulum.parse(task_data.get("updated_at", timezone.now()))
in_progress_scrobble = Scrobble.objects.filter(
user_id=user_id,
in_progress=True,
log__source_id=source_id,
log__source="orgmode",
task=task,
).last()
if not in_progress_scrobble and stopped:
logger.info(
"[emacs_scrobble_task] cannot stop already stopped task",
extra={
"emacs_id": source_id,
},
)
return
if in_progress_scrobble and started:
logger.info(
"[emacs_scrobble_task] cannot start already started task",
extra={
"emacs_id": source_id,
},
)
return in_progress_scrobble
# Finish an in-progress scrobble
if in_progress_scrobble and stopped:
logger.info(
"[emacs_scrobble_task] finishing",
extra={
"emacs_id": source_id,
},
)
in_progress_scrobble.stop(timestamp=timestamp, force_finish=True)
return in_progress_scrobble
if in_progress_scrobble:
return in_progress_scrobble
notes = task_data.pop("notes")
if notes:
task_data["notes"] = []
for note in notes:
task_data["notes"].append(
{note.get("timestamp"): note.get("content")}
)
scrobble_dict = {
"user_id": user_id,
"timestamp": timestamp,
"playback_position_seconds": 0,
"source": "Org-mode",
"log": task_data,
}
logger.info(
"[emacs_scrobble_task] creating",
extra={
"task_id": task.id,
"user_id": user_id,
"scrobble_dict": scrobble_dict,
"media_type": Scrobble.MediaType.TASK,
},
)
scrobble = Scrobble.create_or_update(task, user_id, scrobble_dict)
return scrobble
def manual_scrobble_task(url: str, user_id: int, action: Optional[str] = None):
source_id = re.findall(r"\d+", url)[0]
@ -708,3 +847,34 @@ def manual_scrobble_puzzle(
# TODO Kick out a process to enrich the media here, and in every scrobble event
return Scrobble.create_or_update(puzzle, user_id, scrobble_dict)
def manual_scrobble_brickset(
brickset_id: str, user_id: int, action: Optional[str] = None
):
brickset = BrickSet.find_or_create(brickset_id)
if not brickset:
logger.error(f"No brickset found for Brickset ID {brickset_id}")
return
scrobble_dict = {
"user_id": user_id,
"timestamp": timezone.now(),
"playback_position_seconds": 0,
"source": "Vrobbler",
"log": {"serial_scrobble_id": ""},
}
logger.info(
"[vrobbler-scrobble] brickset scrobble request received",
extra={
"brickset_id": brickset.id,
"user_id": user_id,
"scrobble_dict": scrobble_dict,
"media_type": Scrobble.MediaType.BRICKSET,
},
)
# TODO Kick out a process to enrich the media here, and in every scrobble event
# TODO Need to check for past scrobbles and auto populate serial scrobble id if possible
return Scrobble.create_or_update(brickset, user_id, scrobble_dict)

View File

@ -1,6 +1,6 @@
from django.urls import path
from scrobbles import views
from tasks.webhooks import todoist_webhook
from tasks.webhooks import todoist_webhook, emacs_webhook
app_name = "scrobbles"
@ -36,11 +36,6 @@ urlpatterns = [
views.web_scrobbler_webhook,
name="web-scrobbler-webhook",
),
path(
"webhook/emacs/",
views.emacs_webhook,
name="emacs-webhook",
),
path(
"webhook/gps/",
views.gps_webhook,
@ -57,6 +52,13 @@ urlpatterns = [
name="mopidy-webhook",
),
path("webhook/todoist/", todoist_webhook, name="todoist-webhook"),
path("webhook/emacs/", emacs_webhook, name="emacs_webhook"),
path("export/", views.export, name="export"),
path(
"imports/",
views.ScrobbleImportListView.as_view(),
name="import-detail",
),
path("export/", views.export, name="export"),
path(
"imports/",

View File

@ -3,6 +3,7 @@ import logging
import re
from datetime import datetime, timedelta, tzinfo
from sqlite3 import IntegrityError
from urllib.parse import urlparse
import pytz
from django.apps import apps
@ -17,9 +18,10 @@ from scrobbles.tasks import (
process_lastfm_import,
process_retroarch_import,
)
from vrobbler.apps.scrobbles.notifications import NtfyNotification
from webdav.client import get_webdav_client
from vrobbler.apps.scrobbles.notifications import NtfyNotification
logger = logging.getLogger(__name__)
User = get_user_model()
@ -67,7 +69,8 @@ def get_scrobbles_for_media(media_obj, user: User) -> models.QuerySet:
return Scrobble.objects.filter(media_query, user=user)
def get_recently_played_board_games(user: User) -> dict: ...
def get_recently_played_board_games(user: User) -> dict:
...
def get_long_plays_in_progress(user: User) -> dict:
@ -187,8 +190,8 @@ def delete_zombie_scrobbles(dry_run=True):
def import_from_webdav_for_all_users(restart=False):
"""Grab a list of all users with WebDAV enabled and kickoff imports for them"""
from scrobbles.models import KoReaderImport
from books.koreader import fetch_file_from_webdav
from scrobbles.models import KoReaderImport
# LastFmImport = apps.get_model("scrobbles", "LastFMImport")
webdav_enabled_user_ids = UserProfile.objects.filter(
@ -271,6 +274,7 @@ def get_file_md5_hash(file_path: str) -> str:
file_hash.update(chunk)
return file_hash.hexdigest()
def deduplicate_tracks(commit=False) -> int:
from music.models import Track
@ -293,7 +297,11 @@ def deduplicate_tracks(commit=False) -> int:
try:
other.delete()
except IntegrityError as e:
print("could not delete ", other.id, f": IntegrityError {e}")
print(
"could not delete ",
other.id,
f": IntegrityError {e}",
)
return len(dups)
@ -318,3 +326,13 @@ def send_stop_notifications_for_in_progress_scrobbles() -> int:
notifications_sent += 1
return notifications_sent
def extract_domain(url):
parsed_url = urlparse(url)
domain = (
parsed_url.netloc.split(".")[-2]
+ "."
+ parsed_url.netloc.split(".")[-1]
)
return domain

View File

@ -201,7 +201,7 @@ class RecentScrobbleList(ListView):
processed_finished__isnull=True,
user=self.request.user,
)
data["counts"] = scrobble_counts(user)
data["counts"] = [] #scrobble_counts(user)
else:
data["weekly_data"] = week_of_scrobbles()
data["counts"] = scrobble_counts()
@ -524,35 +524,6 @@ def gps_webhook(request):
return Response({"scrobble_id": scrobble.id}, status=status.HTTP_200_OK)
@csrf_exempt
@api_view(["POST"])
def emacs_webhook(request):
try:
data_dict = json.loads(request.data)
except TypeError:
data_dict = request.data
logger.info(
"[emacs_webhook] called",
extra={
"post_data": data_dict,
"user_id": 1,
},
)
# TODO Fix this so we have to authenticate!
user_id = 1
if request.user.id:
user_id = request.user.id
# scrobble = gpslogger_scrobble_location(data_dict, user_id)
# if not scrobble:
# return Response({}, status=status.HTTP_200_OK)
return Response({"post_data": post_data}, status=status.HTTP_200_OK)
@csrf_exempt
@permission_classes([IsAuthenticated])
@api_view(["POST"])

View File

@ -1,10 +1,12 @@
TODOIST_TITLE_PREFIX_LABELS = ["work", "home", "personal"]
TODOIST_TITLE_SUFFIX_LABELS = [
TODOIST_TITLE_LABELS = [
"bug",
"project",
"feature",
"errand",
"chore",
"admin",
"computer",
"farm",
"baking",
"meeting",
"habit",
"research",

View File

@ -50,7 +50,8 @@ class Task(LongPlayScrobblableMixin):
url = ""
scrobble = self.scrobbles(user_id).first()
if scrobble:
url = TODOIST_TASK_URL.format(id=scrobble.logdata.todoist_id)
if scrobble.log.get("source") == "todoist":
url = TODOIST_TASK_URL.format(id=scrobble.logdata.todist_id)
return url
def subtitle_for_user(self, user_id):

View File

@ -1,16 +1,18 @@
import json
import logging
import pendulum
from django.views.decorators.csrf import csrf_exempt
from profiles.models import UserProfile
from rest_framework import status
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from scrobbles.scrobblers import (
emacs_scrobble_task,
emacs_scrobble_update_task,
todoist_scrobble_task,
todoist_scrobble_update_task,
)
from profiles.models import UserProfile
logger = logging.getLogger(__name__)
@ -29,7 +31,7 @@ def todoist_webhook(request):
todoist_type, todoist_event = post_data.get("event_name").split(":")
event_data = post_data.get("event_data", {})
is_item_type = todoist_type == "item"
is_note_type = todoist_type == "note"
is_note_type = todoist_tyllll = "note"
new_labels = event_data.get("labels", [])
old_labels = (
post_data.get("event_data_extra", {})
@ -118,3 +120,54 @@ def todoist_webhook(request):
extra={"scrobble_id": scrobble.id},
)
return Response({"scrobble_id": scrobble.id}, status=status.HTTP_200_OK)
@csrf_exempt
@permission_classes([IsAuthenticated])
@api_view(["POST"])
def emacs_webhook(request):
try:
post_data = json.loads(request.data)
except TypeError:
post_data = request.data
logger.info(
"[emacs_webhook] called",
extra={"post_data": post_data},
)
task_in_progress = post_data.get("state") == "STRT"
task_stopped = post_data.get("state") == "DONE"
post_data["source_id"] = post_data.pop("emacs_id")
# TODO: Figure out why token auth is not working
user_id = request.user.id
if not user_id:
user_id = 1
scrobble = None
if post_data.get("source_id"):
scrobble = emacs_scrobble_task(
post_data, user_id, started=task_in_progress, stopped=task_stopped
)
if not scrobble:
logger.info(
"[emacs_webhook] finished with no note or task found",
extra={"scrobble_id": None},
)
return Response(
{"error": "No scrobble found to be updated"},
status=status.HTTP_304_NOT_MODIFIED,
)
if task_in_progress and post_data.get("notes"):
emacs_scrobble_update_task(
post_data.get("source_id"),
post_data.get("notes"),
user_id,
)
logger.info(
"[emacs_webhook] finished",
extra={"scrobble_id": scrobble.id},
)
return Response({"scrobble_id": scrobble.id}, status=status.HTTP_200_OK)

View File

@ -0,0 +1,23 @@
# Generated by Django 4.2.19 on 2025-06-16 14:20
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("trails", "0004_alter_trail_run_time_seconds"),
]
operations = [
migrations.AddField(
model_name="trail",
name="alltrails_id",
field=models.CharField(blank=True, max_length=255, null=True),
),
migrations.AddField(
model_name="trail",
name="gaiagps_id",
field=models.CharField(blank=True, max_length=255, null=True),
),
]

View File

@ -9,9 +9,7 @@ from locations.models import GeoLocation
BNULL = {"blank": True, "null": True}
class Trail(ScrobblableMixin):
class PrincipalType(models.TextChoices):
WOODS = "WOODS"
ROAD = "ROAD"
@ -19,7 +17,7 @@ class Trail(ScrobblableMixin):
MOUNTAIN = "MOUNTAIN"
class ActivityType(models.TextChoices):
WALK= "WALK"
WALK = "WALK"
HIKE = "HIKE"
RUN = "RUN"
BIKE = "BIKE"
@ -39,8 +37,14 @@ class Trail(ScrobblableMixin):
)
strava_id = models.CharField(max_length=255, **BNULL)
trailforks_id = models.CharField(max_length=255, **BNULL)
principal_type = models.CharField(max_length=10, choices=PrincipalType.choices, **BNULL)
default_activity_type = models.CharField(max_length=10, choices=ActivityType.choices, **BNULL)
alltrails_id = models.CharField(max_length=255, **BNULL)
gaiagps_id = models.CharField(max_length=255, **BNULL)
principal_type = models.CharField(
max_length=10, choices=PrincipalType.choices, **BNULL
)
default_activity_type = models.CharField(
max_length=10, choices=ActivityType.choices, **BNULL
)
def get_absolute_url(self):
return reverse("trails:trail_detail", kwargs={"slug": self.uuid})

View File

@ -0,0 +1,61 @@
from typing import Optional
import requests
from bs4 import BeautifulSoup
def get_description_from_soup(soup) -> Optional[int]:
rating = None
try:
potential_desc = soup.find("div", id_="descriptionContainer")
potential_rating = soup.find("div", class_="allmusic-rating")
if potential_rating:
rating = int(strip_and_clean(potential_rating.get_text()))
except ValueError:
pass
return rating
def scrape_gaia_trail(url):
headers = {"User-Agent": "Mozilla/5.0"}
r = requests.get(url, headers=headers)
r.raise_for_status()
soup = BeautifulSoup(r.text, "html.parser")
# Title
title = soup.select_one("h1").get_text(strip=True)
# Stats are inside divs with classes like "flex" and label-value pairs
stats = {}
stat_blocks = soup.select("div.flex.flex-col.items-start.text-sm span")
for i in range(0, len(stat_blocks) - 1, 2):
label = stat_blocks[i].get_text(strip=True)
value = stat_blocks[i + 1].get_text(strip=True)
stats[label] = value
# Description (under 'Overview' header)
overview_heading = soup.find(
lambda tag: tag.name == "h2" and "Overview" in tag.text
)
if overview_heading:
desc_parts = []
for sib in overview_heading.next_siblings:
if sib.name and sib.name.startswith("h"):
break
if sib.name == "p":
desc_parts.append(sib.get_text(strip=True))
description = "\n\n".join(desc_parts).strip()
else:
description = None
return {
"url": url,
"title": title,
"stats": stats,
"description": description,
}
# Example usage
url = "https://www.gaiagps.com/hike/318136/"
trail_data = scrape_gaia_trail(url)
print(trail_data)

View File

@ -25,6 +25,7 @@ class VideoMetadata:
60 # Silly default, but things break if this is 0 or null
)
imdb_id: Optional[str]
tmdb_id: Optional[str]
youtube_id: Optional[str]
# IMDB specific
@ -35,6 +36,7 @@ class VideoMetadata:
tv_series_id: Optional[int]
plot: Optional[str]
imdb_rating: Optional[str]
tmdb_rating: Optional[str]
cover_url: Optional[str]
overview: Optional[str]
@ -57,6 +59,8 @@ class VideoMetadata:
def as_dict_with_cover_and_genres(self) -> tuple:
video_dict = vars(self)
cover = video_dict.pop("cover_url")
genres = video_dict.pop("genres")
cover = None
if "cover_url" in video_dict.keys():
cover = video_dict.pop("cover_url", "")
genres = video_dict.pop("genres", [])
return video_dict, cover, genres

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.19 on 2025-06-13 15:04
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('videos', '0022_alter_video_run_time_seconds'),
]
operations = [
migrations.AddField(
model_name='video',
name='tmdb_rating',
field=models.FloatField(blank=True, null=True),
),
]

View File

@ -2,7 +2,6 @@ import logging
from typing import Optional
from uuid import uuid4
import pendulum
import requests
from django.conf import settings
from django.core.files.base import ContentFile
@ -21,8 +20,8 @@ from scrobbles.mixins import (
from taggit.managers import TaggableManager
from videos.metadata import VideoMetadata
from videos.sources.imdb import lookup_video_from_imdb
from vrobbler.apps.videos.sources.youtube import lookup_video_from_youtube
from videos.sources.tmdb import lookup_video_from_tmdb
from videos.sources.youtube import lookup_video_from_youtube
YOUTUBE_VIDEO_URL = "https://www.youtube.com/watch?v="
YOUTUBE_CHANNEL_URL = "https://www.youtube.com/channel/"
@ -208,6 +207,7 @@ class Video(ScrobblableMixin):
next_imdb_id = models.CharField(max_length=20, **BNULL)
imdb_id = models.CharField(max_length=20, **BNULL)
imdb_rating = models.FloatField(**BNULL)
tmdb_rating = models.FloatField(**BNULL)
cover_image = models.ImageField(upload_to="videos/video/", **BNULL)
cover_image_small = ImageSpecField(
source="cover_image",
@ -309,11 +309,13 @@ class Video(ScrobblableMixin):
def get_from_imdb_id(
cls, imdb_id: str, overwrite: bool = False
) -> "Video":
if "tt" in imdb_id:
imdb_id = imdb_id[2:]
video, created = cls.objects.get_or_create(imdb_id=imdb_id)
if not created and not overwrite:
return video
vdict, cover, genres = lookup_video_from_imdb(
vdict, cover, genres = lookup_video_from_tmdb(
imdb_id
).as_dict_with_cover_and_genres()
if created or overwrite:

View File

@ -72,26 +72,30 @@ def lookup_video_from_imdb(
video_metadata.video_type = VideoType.MOVIE.value
series_name = None
if imdb_result.get("kind") == "episode":
series_name = imdb_result.get("episode of", None).data.get(
"title", None
)
series, _ = Series.objects.get_or_create(name=series_name)
video_metadata.video_type = VideoType.TV_EPISODE.value
video_metadata.tv_series_id = series.id
try:
series_name = imdb_result.get("episode of", None).data.get(
"title", None
)
except IndexError:
series_name = None
if series_name:
series, _ = Series.objects.get_or_create(name=series_name)
video_metadata.video_type = VideoType.TV_EPISODE.value
video_metadata.tv_series_id = series.id
if imdb_result.get("runtimes"):
video_metadata.run_time_seconds = (
int(imdb_result.get("runtimes")[0]) * 60
)
video_metadata.imdb_id = name_or_id
video_metadata.title = imdb_result.get("title", "")
video_metadata.imdb_id = imdb_result.get("imdbID")
video_metadata.episode_number = imdb_result.get("episode", None)
video_metadata.season_number = imdb_result.get("season", None)
video_metadata.next_imdb_id = imdb_result.get("next episode", None)
video_metadata.year = imdb_result.get("year", None)
video_metadata.plot = imdb_result.get("plot outline")
video_metadata.imdb_rating = imdb_result.get("rating")
video_metadata.genres = imdb_result.get("genres")
video_metadata.plot = imdb_result.get("plot outline", "")
video_metadata.imdb_rating = imdb_result.get("rating", None)
video_metadata.genres = imdb_result.get("genres", [])
return video_metadata

View File

@ -0,0 +1,81 @@
import logging
import pendulum
from django.conf import settings
from themoviedb import TMDb
from tmdbv3api import TV, Movie, TMDb as TMDb_direct
from videos.metadata import VideoMetadata, VideoType
TMDB_KEY = getattr(settings, "TMDB_API_KEY", "")
tmdb_direct = TMDb_direct()
tmdb_direct.api_key = TMDB_KEY
tmdb = TMDb(key=TMDB_KEY, language="en-US", region="US")
TMDB_IMAGE_URL = "https://image.tmdb.org/t/p/original"
logger = logging.getLogger(__name__)
def lookup_video_from_tmdb(
name_or_id: str, kind: str = "movie"
) -> VideoMetadata:
from videos.models import Series
imdb_id = name_or_id
if name_or_id.startswith("tt"):
imdb_id = name_or_id[2:]
video_metadata = VideoMetadata(imdb_id=imdb_id)
tmdb_result = tmdb.find().by_imdb("tt" + imdb_id)
if not tmdb_result:
logger.info(
"[lookup_video_from_tmdb] no video found on tmdb",
extra={"name_or_id": name_or_id},
)
return None
video_metadata = VideoMetadata(imdb_id=imdb_id)
media = None
show = None
if len(tmdb_result.movie_results) > 0:
media = Movie().details(tmdb_result.movie_results[0].id)
video_metadata.video_type = VideoType.MOVIE.value
video_metadata.title = media.title
video_metadata.cover_url = TMDB_IMAGE_URL + media.poster_path # TODO: enrich this with TMDB url
video_metadata.year = pendulum.parse(media.release_date).year
video_metadata.genres = [g.get("name", "") for g in media.genres]
if len(tmdb_result.tv_episode_results) > 0:
video_metadata.video_type = VideoType.TV_EPISODE.value
media = tmdb_result.tv_episode_results[0]
video_metadata.title = media.name
video_metadata.cover_url = TMDB_IMAGE_URL + media.still_path # TODO: enrich this with TMDB url
video_metadata.episode_number = media.episode_number
video_metadata.season_number = media.season_number
video_metadata.year = media.air_date.year
series_imdb_id = TV().external_ids(media.show_id).imdb_id[2:]
series, created = Series.objects.get_or_create(imdb_id=series_imdb_id)
if created:
show_data = TV().details(tv_id=media.show_id)
series.name = show_data.name
series.save()
video_metadata.tv_series_id = series.id
if not media:
logger.warning("Video not found on TMDB", extra={"imdb_id":imdb_id})
return video_metadata
video_metadata.tmdb_id = media.id
video_metadata.run_time_seconds = media.runtime * 60
video_metadata.plot = media.overview
video_metadata.overview = media.overview
video_metadata.tmdb_rating = media.vote_average
#video_metadata.next_imdb_id = imdb_result.get("next episode", None)
return video_metadata

View File

@ -9,7 +9,7 @@
<div class="col-md">
{% if Track %}
<h2>Music</h2>
{% with scrobbles=Track %}
{% with scrobbles=Track count=Track_count time=Track_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
@ -18,63 +18,70 @@
{% if Task %}
<h2>Latest tasks</h2>
{% with scrobbles=Task %}
{% with scrobbles=Task count=Task_count time=Task_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if Video %}
<h2>Videos</h2>
{% with scrobbles=Video %}
{% with scrobbles=Video count=Video_count time=Video_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if WebPage %}
<h4>Web pages</h4>
{% with scrobbles=WebPage %}
{% with scrobbles=WebPage count=WebPage_count time=WebPage_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if SportEvent %}
<h2>Sports</h2>
{% with scrobbles=SportEvent %}
{% with scrobbles=SportEvent count=SportEvent_count time=SportEvent_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if PodcastEpisode %}
<h2>Latest podcasts</h2>
{% with scrobbles=PodcastEpisode %}
{% with scrobbles=PodcastEpisode count=PodcastEpisode_count time=PodcastEpisode_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if VideoGame %}
<h4>Video games</h4>
{% with scrobbles=VideoGame %}
{% with scrobbles=VideoGame count=VideoGame_count time=VideoGame_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if BoardGame %}
<h4>Board games</h4>
{% with scrobbles=BoardGame %}
{% with scrobbles=BoardGame count=BoardGame_count time=BoardGame_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if Beer %}
<h4>Beers</h4>
{% with scrobbles=Beer %}
{% with scrobbles=Beer count=Beer_count time=Beer_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if BrickSet %}
<h4>Brick sets</h4>
{% with scrobbles=BrickSet count=BrickSet_count time=BrickSet_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}
{% if Book %}
<h4>Books</h4>
{% with scrobbles=Book %}
{% with scrobbles=Book count=Book_count time=Book_time %}
{% include "scrobbles/_scrobble_table.html" %}
{% endwith %}
{% endif %}

View File

@ -1,7 +1,13 @@
{% load humanize %}
{% load naturalduration %}
<tr>
<tr {% if scrobble.in_progress %}class="in-progress"{% endif %}>
<td>{% if scrobble.in_progress %}{{scrobble.media_obj.strings.verb}} now | <a class="right" href="{% url "scrobbles:finish" scrobble.uuid %}">Finish</a>{% else %}{{scrobble.timestamp|naturaltime}}{% endif %}</td>
<td><a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj|truncatechars_html:50}}</a></td>
<td>
{% if scrobble.media_type == "Task" %}
<p><em><a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj.title|truncatechars_html:45}} - {% if scrobble.logdata %}{% if scrobble.logdata.description %}{{scrobble.logdata.description}}{% endif %}{% endif %}</a></em></p>
{% else %}
<a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj|truncatechars_html:45}}</a>
{% endif %}
</td>
<td>{{scrobble.elapsed_time|natural_duration}}</td>
</tr>

View File

@ -4,6 +4,7 @@
<div class="tab-pane fade show" id="latest-beers" role="tabpanel"
aria-labelledby="latest-beers-tab">
<div class="table-responsive">
{{count}} scrobble {% if time %}| {{time|natural_duration}}{% endif %}
<table class="table table-striped table-sm">
<thead>
<tr>

View File

@ -43,6 +43,10 @@
background:rgba(0,0,0,0.4);
}
.in-progress {
background: #CCFFFF;
}
.in-progress a { color: black }
</style>
{% endblock %}
{% block content %}

View File

@ -62,6 +62,7 @@
<td>{{scrobble.timestamp}}</td>
<td><a href="{{scrobble.get_media_source_url}}">{{scrobble.logdata.description}}</a></td>
<td>{{scrobble.source}}</td>
<td>{{scrobble.log.notes}}</td>
</tr>
{% endfor %}
</tbody>

View File

@ -57,6 +57,7 @@ dd {
<div class="row header">
<div class="cover image-wrapper">
{% if object.imdb_rating %}<div class="caption">{{object.imdb_rating}}</div>{% endif %}
{% if object.tmdb_rating %}<div class="caption">{{object.tmdb_rating}}</div>{% endif %}
<img src="{% if object.cover_image %}{{object.cover_image.url}}{% else %}{% static 'images/no-video-cover.jpg' %}{% endif %}" width="400px" />
<div class="caption-footer">{{object.year}}{% if object.tv_series %} | <b>S</b>{{object.season_number}} <b>E</b>{{object.episode_number}}{% endif %}</div>
</div>