Compare commits
31 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bffbf47c2f | |||
| f4e81da533 | |||
| 4b7f5459be | |||
| c68b0e9d7e | |||
| 32ec65116b | |||
| da8d26fcd9 | |||
| d33954e494 | |||
| 1b306d6493 | |||
| c881143e1b | |||
| 141700fcb3 | |||
| 7357b5bfec | |||
| 99cabd0007 | |||
| cf77e12cc3 | |||
| 3f2cbbb34a | |||
| 650ecf12c6 | |||
| d7cc009d07 | |||
| a872cf3611 | |||
| 1f9713312b | |||
| 159e555d7c | |||
| 981f4f9c9a | |||
| ddd5ce1392 | |||
| 7a75b31b56 | |||
| 24ac545f55 | |||
| d5da8ae701 | |||
| 53a04d064d | |||
| c0871a3b9e | |||
| d917dd8b2c | |||
| 6fc8084f2d | |||
| 41d6fe8ff6 | |||
| 73838312cd | |||
| eb2dd4c839 |
@ -39,8 +39,8 @@ steps:
|
||||
- vrobbler collectstatic --noinput
|
||||
- immortalctl restart celery && immortalctl restart vrobbler
|
||||
when:
|
||||
branch:
|
||||
- main
|
||||
ref:
|
||||
- refs/tags/*
|
||||
- name: build success notification
|
||||
image: parrazam/drone-ntfy:0.3-linux-amd64
|
||||
when:
|
||||
|
||||
@ -1,196 +1,64 @@
|
||||
#+title: TODOs
|
||||
|
||||
A fun way to keep track of things in the project to fix or improve.
|
||||
* Backlog [6/23]
|
||||
** TODO [#A] Fix small bug in views for TV series where next episode is now None :vrobbler:bug:personal:videos:
|
||||
|
||||
* Version 0.16.0
|
||||
** TODO [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music:
|
||||
Effectively, any track that comes in without a MusicBrainz ID does some funky
|
||||
lookup where it doesn't find a track without an MB id and the track title /
|
||||
artist combination and creates a new track everytime. This has to be cleaend up
|
||||
by condensing the duplicated tracks into the original proper track.
|
||||
|
||||
But it opens a bigger quesiton about how much MB id should the drive the app
|
||||
lookup. If it can't be depended on to exist from all sources, it really can't be
|
||||
canonical. Instead, the combination of track title / artist is really the best
|
||||
we can do. Last.fm also has this problem, where it doesn't know about albums and
|
||||
definitely does not know or care about MB ids.
|
||||
|
||||
** DONE Add a user profile page with ability to change settings :profiles:improvement:
|
||||
- Note taken on [2025-04-04 Fri 10:51]
|
||||
[[orgit-rev:~/src/code.unbl.ink/secstate/vrobbler/::93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44][~/src/code.unbl.ink/secstate/vrobbler/ (magit-rev 93c16d8)]]
|
||||
|
||||
https://code.unbl.ink/secstate/vrobbler/commit/93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44
|
||||
|
||||
** DONE What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
|
||||
- Note taken on [2025-04-04 Fri 10:46]
|
||||
|
||||
Nothing. Over the last few months I built out a youtube model in videos and
|
||||
use a bookmarklet scrobbling pattern. Now web-scrobbler is just disabled for
|
||||
Youtube.
|
||||
|
||||
May want to revisit this at some point and only scrobble tracks from Youtube,
|
||||
because many people use YT for music listening.
|
||||
|
||||
** DONE [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
|
||||
CLOSED: [2023-04-06 Thu 14:09]
|
||||
** DONE Add a "stop_timestamp" so we don't rely on content length :improvement:scrobbling:
|
||||
CLOSED: [2023-04-02 Sun 23:58]
|
||||
|
||||
Essentially, we currently have the timestamp as when the content began
|
||||
scrobbling and then calculate the finish time from the length of the content.
|
||||
This works pretty well because we know how long most things are.
|
||||
|
||||
But in some cases, sports events or long podcasts, we may start mid-way through
|
||||
an event or finish halfway through but still want to mark it as done. In these
|
||||
cases, knowing the finish time could be useful, especially when interfacing with
|
||||
other scrobblers which may have different definitions of when a scrobble
|
||||
finishes or started.
|
||||
** DONE Fix bug with Various Artist albums being labeled with first artist as album artist :scrobbling:bug:music:
|
||||
CLOSED: [2023-03-27 Mon 20:18]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-26 Sun 22:01]--[2023-03-27 Mon 01:07] => 3:06
|
||||
:END:
|
||||
** DONE Fix bug with weekly aggregator being blank on Sundays :aggregators:music:bug:
|
||||
CLOSED: [2023-03-26 Sun 13:52]
|
||||
** DONE Fix KoReader scrobbling to use pages rather than time of last read :scrobbling:books:improvement:
|
||||
CLOSED: [2023-03-26 Sun 13:51]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-26 Sun 13:11]--[2023-03-26 Sun 13:51] => 0:40
|
||||
:END:
|
||||
** DONE [#A] Add django-storage to store files on S3 :settings:improvement:
|
||||
CLOSED: [2023-03-24 Fri 14:46]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-24 Fri 10:47]--[2023-03-24 Fri 14:46] => 3:59
|
||||
CLOCK: [2023-03-24 Fri 10:36]--[2023-03-24 Fri 10:40] => 0:04
|
||||
:END:
|
||||
** DONE Fix vrobbler settings not using booleans :settings:bug:
|
||||
CLOSED: [2023-03-24 Fri 10:45]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-24 Fri 10:40]--[2023-03-24 Fri 10:46] => 0:06
|
||||
:END:
|
||||
** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
|
||||
CLOSED: [2023-03-24 Fri 00:31]
|
||||
The live view will be blank every Monday, no reason to tie it to a day of the
|
||||
week. It should be "the last 7 days"
|
||||
** DONE [#B] Implement a detail view for TV shows :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE [#B] Implement a detail view for Movies :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:04]
|
||||
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
|
||||
CLOSED: [2023-03-22 Wed 17:01]
|
||||
** DONE Add live chart view like Maloja :improvement:views:
|
||||
CLOSED: [2023-03-07 Tue 11:13]
|
||||
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:06]
|
||||
|
||||
An example:
|
||||
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
|
||||
|
||||
This is actually going to be moot because we can import from LastFM, and
|
||||
web-scrobbler integrates well with LastFM. The only thing to think through here
|
||||
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
|
||||
from Youtube (all videos get pushed, sigh).
|
||||
|
||||
** DONE Add Amazon scraper to look up books when OL fails :books:improvement:
|
||||
This turned out to be a non-starter ... Amazon is aggressive at disallowing
|
||||
scraping quality. And all the OSS tools out there are stuck in an arms race
|
||||
trying to keep them from breaking.
|
||||
|
||||
That said, Google Books actually has a decent API (for now), and I've built this
|
||||
out using that.
|
||||
|
||||
** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug:
|
||||
This was fixed a while ago, but there's a new manifested bug. Going to create a
|
||||
separate bug tracking ticket for that.
|
||||
* Version 0.11.4
|
||||
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
|
||||
** DONE [#A] Fix fetching artwork without release group :bug:
|
||||
CLOSED: [2023-01-29 Sun 14:27]
|
||||
|
||||
When we get artwork from Musicbrianz, and it's not found, we should check for
|
||||
release groups as well. This will stop issues with missing artwork because of
|
||||
obscure MB release matches.
|
||||
|
||||
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
|
||||
CLOSED: [2023-01-30 Mon 18:31]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
|
||||
:END:
|
||||
|
||||
If we play music from Jellyfin and the track reaches 90% completion, the
|
||||
scrobbling goes crazy and starts creating new scrobbles with every update.
|
||||
|
||||
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
|
||||
as complete for the following conditions:
|
||||
|
||||
- Play stopped and percent played beyond 90%
|
||||
- Play completely finished
|
||||
|
||||
But if we keep listening beyond 90, we should basically ignore updates (or just
|
||||
update the existing scrobble)
|
||||
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
|
||||
CLOSED: [2023-02-03 Fri 16:52]
|
||||
|
||||
An example of the format:
|
||||
#+begin_src csv
|
||||
,
|
||||
#AUDIOSCROBBLER/1.1
|
||||
#TZ/UNKNOWN
|
||||
#CLIENT/Rockbox sansaclipplus $Revision$
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Jackolantern’s Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
|
||||
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
|
||||
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
|
||||
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
|
||||
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
|
||||
,
|
||||
#+begin_src python
|
||||
ERROR django.request:241 log_response Internal Server Error: /series/c24100d1-da45-4abe-86bf-27cfce9b1f89/
|
||||
Traceback (most recent call last):
|
||||
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
|
||||
response = get_response(request)
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
|
||||
response = wrapped_callback(request, *callback_args, **callback_kwargs)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
|
||||
return self.dispatch(request, *args, **kwargs)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/mixins.py", line 73, in dispatch
|
||||
return super().dispatch(request, *args, **kwargs)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
|
||||
return handler(request, *args, **kwargs)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/django/views/generic/detail.py", line 109, in get
|
||||
context = self.get_context_data(object=self.object)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/videos/views.py", line 33, in get_context_data
|
||||
context_data["next_episode_id"] = "tt" + next_episode_id
|
||||
~~~~~^~~~~~~~~~~~~~~~~
|
||||
TypeError: can only concatenate str (not "NoneType") to str
|
||||
#+end_src
|
||||
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:10]
|
||||
|
||||
This would allow a few nice flows. One, you'd be able to record the play of an
|
||||
entire album by just dropping the muscibrainz_id in. This could be helpful for
|
||||
offline listening. It would also mean bad metadata from mopidy would not break
|
||||
scrobbling.
|
||||
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:11]
|
||||
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:09]
|
||||
** TODO [#A] Send periodic check notifications for board games and beer that ask if you're still scrobbling :vrobbler:personal:project:beers:boardgames:notifications:feature:
|
||||
** TODO [#A] Fix raw text webpage title not truncating to 254 chars :vrobbler:personal:bug:webpages:
|
||||
** TODO [#A] Fix koreader scrobble imports to use DST properly :vrobbler:personal:bug:books:imports:
|
||||
The page data has the canonical date something was read in it, but it seems to be an hour off. I traced this back to being off during DST, so we just need the importer to be aware of whether a user is using DST or not and roll back an hour for part of the year.
|
||||
|
||||
Given a UUID from musicbrainz, we should be able to scrobble an album or
|
||||
individual track.
|
||||
|
||||
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:10]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
|
||||
:END:
|
||||
|
||||
Maloja does this cool thing where artists and tracks get recorded as the top
|
||||
track of a given week, month or year. They get gold, silver or bronze stars for
|
||||
their place in the time period.
|
||||
|
||||
I could see this being implemented as a separate Chart table which gets
|
||||
populated at the end of a time period and has a start and end date that defines
|
||||
a period, along with a one, two, three instance.
|
||||
|
||||
Of course, it could also be a data model without a table, where it runs some fun
|
||||
calculations, stores it's values in Redis as a long-term lookup table and just
|
||||
has to re-populate when the server restarts.
|
||||
* Backlog
|
||||
Also, we'd need to adjust any old scrobbles that took place with DST off to roll them back by an hour.
|
||||
** TODO [#A] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
|
||||
** TODO [#B] Add AllTrails as a source for Trail data :vrobbler:trails:feature:personal:project:
|
||||
Pretty clear, I would love to make trails more useful. Historically I wasn't
|
||||
hiking a lot, which made the source for this a bit silly. But it's clear that
|
||||
AllTrails is the best source, though having TrailForks is nice to.
|
||||
** TODO [#B] Add `garmin_activity_id` to the TrailMetadataLog class :vrobbler:trails:feature:personal:project:
|
||||
Would be nice to have some loose connection to the actual event in my Garmin profile.
|
||||
** TODO [#B] Explore a way to add metadata editing to scrobbles after saving :vrobbler:spike:scrobbling:personal:project:
|
||||
Could be as simple as a JSON form on the scrobble detail page (do I have have one of those yet?).
|
||||
** TODO [#B] Explore a good way to show notes and descriptions from scrobbles to users :personal:project:scrobbling:vrobbler:spike:
|
||||
** TODO [#B] Add webdav syncing to retroarch imports :vrobbler:videogames:webdav:feature:project:personal:
|
||||
** TODO [#B] Fix task app to only use one tag for the context a task was done in and allow configurable contexts by user profile :personal:vrobbler:feature:tasks:project:
|
||||
** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
|
||||
https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
|
||||
** TODO [#C] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal:
|
||||
Rather than pick up an existing Podcast using the podcast title in the mopidy
|
||||
file name, Vrobbler creates a new podcast with no enriched data. Not a big deal
|
||||
for my use as the volume of podcasts I listen to makes manual fixes easy. But
|
||||
it's annoying.
|
||||
** TODO [#C] Move to using more robust mopidy-webhooks pacakge form pypi :utility:improvement:
|
||||
:PROPERTIES:
|
||||
:ID: ab31fdc3-359c-1b1d-6b9d-546b476021ba
|
||||
:END:
|
||||
*** Example payloads from mopidy-webhooks
|
||||
**** Podcast playback ended
|
||||
#+begin_src json
|
||||
@ -481,4 +349,260 @@ has to re-populate when the server restarts.
|
||||
}
|
||||
}
|
||||
#+end_src
|
||||
** TODO Fix bug in podcast scrobbling where a second scrobble is created after completion :scrobbling:podcasts:bug:
|
||||
** TODO [#C] User should be able to enable auto trail tracking via amail reader with Garmin LiveTrack URLs :vrobbler:trails:project:feature:personal:
|
||||
** TODO [#C] Allow users to see tasks on calendar view :vrobbler:personal:project:templates:feature:
|
||||
https://codepen.io/oliviale/pen/QYqybo
|
||||
** TODO [#C] Come pu with a possible flow using WebDAV and super-productivity for tasks :personal:feature:project:vrobbler:tasks:
|
||||
* Version 17.0
|
||||
** DONE [#A] Fix bug in new task label lookup for Emacs/Org-mode :vrobbler:bug:tasks:
|
||||
:PROPERTIES:
|
||||
:ID: 683fb109-dfc4-85e4-80f0-ea618434f61e
|
||||
:END:
|
||||
** DONE [#C] Replace commas in the bandcamp URL for artists with nothing :vrobbler:music:bug:personal:
|
||||
:PROPERTIES:
|
||||
:ID: 9b30d67b-91f0-a480-dfaa-5d9dc090e76c
|
||||
:END:
|
||||
|
||||
- Note taken on [2025-06-16 Mon 09:36]
|
||||
|
||||
This firt appeared with Black Country, New Road, where the RYM slug generator
|
||||
leaves commas in and ends up sending you to a 404. I suspect this wont be the
|
||||
first tweak we'll need to this, as the RYM link creator is really just
|
||||
guessing based on the artist name at the path.
|
||||
|
||||
** DONE [#A] Investigate new source of video metadata :personal:project:video:imdb:
|
||||
:PROPERTIES:
|
||||
:ID: df2b486c-1170-5199-c312-9bc87760d962
|
||||
:END:
|
||||
|
||||
Cinemagoer broke and I probably should find a more reilable source of video data.
|
||||
|
||||
- Note taken on [2025-06-13 Fri 11:19]
|
||||
|
||||
TMDB is much more reliable, but does require an API key. That's all setup now,
|
||||
so hopefully this breaking IMDB crap is over.
|
||||
|
||||
** DONE [#A] IMDB video lookups are failing :personal:bug:video:imdb:
|
||||
:PROPERTIES:
|
||||
:ID: 38f1081f-37b4-f4f2-79aa-c1e87eca4b69
|
||||
:END:
|
||||
<2025-06-13 Fri>
|
||||
|
||||
- Note taken on [2025-06-13 Fri 08:24]
|
||||
|
||||
Looks like Cinemagoer is broken: https://github.com/cinemagoer/cinemagoer/issues/537
|
||||
|
||||
** DONE [#A] Emacs is not syncing notes :personal:scrobbling:emacs:bug:
|
||||
:PROPERTIES:
|
||||
:ID: c79cd491-b30f-0945-d84b-b8cac7562791
|
||||
:END:
|
||||
<2025-06-12 Thu 9:30>
|
||||
|
||||
Not sure if the problem is in my Emacs hook sending or Vrobbler itself.
|
||||
|
||||
- Note taken on [2025-06-12 Thu 09:47]
|
||||
|
||||
Adding a quick note to check on it
|
||||
|
||||
- Note taken on [2025-06-12 Thu 09:50]
|
||||
|
||||
Ah ha. All the messing about with the source field meant that I was looking
|
||||
for `emacs` as a source but the hook was initially setting sources to
|
||||
`orgmode` I think I prefer `orgmode` as the source, so updating it thusly.
|
||||
|
||||
Fixed in `490d60cbbb1f8bf90b5fc47d8685b15bdc1d485b`
|
||||
|
||||
** DONE [#A] Show the description of a task in the string rep for a scrobble of a Task :personal:project:scrobbling:vrobbler:feature:
|
||||
:PROPERTIES:
|
||||
:ID: df58f8d0-fa4a-2037-c7d7-e5388c239042
|
||||
:END:
|
||||
* Version 0.16.0
|
||||
** DONE [#A] Jellyfin, bandcamp tracks from Mopidy create duplicate music tracks :bug:scrobbling:music:
|
||||
:PROPERTIES:
|
||||
:ID: 670e8634-49b5-dce9-1684-14f2ffb797f1
|
||||
:END:
|
||||
Effectively, any track that comes in without a MusicBrainz ID does some funky
|
||||
lookup where it doesn't find a track without an MB id and the track title /
|
||||
artist combination and creates a new track every time. This has to be cleaned up
|
||||
by condensing the duplicated tracks into the original proper track.
|
||||
|
||||
But it opens a bigger question about how much MB id should the drive the app
|
||||
lookup. If it can't be depended on to exist from all sources, it really can't be
|
||||
canonical. Instead, the combination of track title / artist is really the best
|
||||
we can do. Last.fm also has this problem, where it doesn't know about albums and
|
||||
definitely does not know or care about MB ids.
|
||||
|
||||
** DONE Add a user profile page with ability to change settings :profiles:improvement:
|
||||
- Note taken on [2025-04-04 Fri 10:51]
|
||||
[[orgit-rev:~/src/code.unbl.ink/secstate/vrobbler/::93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44][~/src/code.unbl.ink/secstate/vrobbler/ (magit-rev 93c16d8)]]
|
||||
|
||||
https://code.unbl.ink/secstate/vrobbler/commit/93c16d80ecff4cd1663cf9ec40fbe6d8f58c3e44
|
||||
|
||||
** DONE What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
|
||||
- Note taken on [2025-04-04 Fri 10:46]
|
||||
|
||||
Nothing. Over the last few months I built out a youtube model in videos and
|
||||
use a bookmarklet scrobbling pattern. Now web-scrobbler is just disabled for
|
||||
Youtube.
|
||||
|
||||
May want to revisit this at some point and only scrobble tracks from Youtube,
|
||||
because many people use YT for music listening.
|
||||
|
||||
** DONE [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
|
||||
CLOSED: [2023-04-06 Thu 14:09]
|
||||
** DONE Add a "stop_timestamp" so we don't rely on content length :improvement:scrobbling:
|
||||
CLOSED: [2023-04-02 Sun 23:58]
|
||||
|
||||
Essentially, we currently have the timestamp as when the content began
|
||||
scrobbling and then calculate the finish time from the length of the content.
|
||||
This works pretty well because we know how long most things are.
|
||||
|
||||
But in some cases, sports events or long podcasts, we may start mid-way through
|
||||
an event or finish halfway through but still want to mark it as done. In these
|
||||
cases, knowing the finish time could be useful, especially when interfacing with
|
||||
other scrobblers which may have different definitions of when a scrobble
|
||||
finishes or started.
|
||||
** DONE Fix bug with Various Artist albums being labeled with first artist as album artist :scrobbling:bug:music:
|
||||
CLOSED: [2023-03-27 Mon 20:18]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-26 Sun 22:01]--[2023-03-27 Mon 01:07] => 3:06
|
||||
:END:
|
||||
** DONE Fix bug with weekly aggregator being blank on Sundays :aggregators:music:bug:
|
||||
CLOSED: [2023-03-26 Sun 13:52]
|
||||
** DONE Fix KoReader scrobbling to use pages rather than time of last read :scrobbling:books:improvement:
|
||||
CLOSED: [2023-03-26 Sun 13:51]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-26 Sun 13:11]--[2023-03-26 Sun 13:51] => 0:40
|
||||
:END:
|
||||
** DONE [#A] Add django-storage to store files on S3 :settings:improvement:
|
||||
CLOSED: [2023-03-24 Fri 14:46]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-24 Fri 10:47]--[2023-03-24 Fri 14:46] => 3:59
|
||||
CLOCK: [2023-03-24 Fri 10:36]--[2023-03-24 Fri 10:40] => 0:04
|
||||
:END:
|
||||
** DONE Fix vrobbler settings not using booleans :settings:bug:
|
||||
CLOSED: [2023-03-24 Fri 10:45]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-03-24 Fri 10:40]--[2023-03-24 Fri 10:46] => 0:06
|
||||
:END:
|
||||
** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
|
||||
CLOSED: [2023-03-24 Fri 00:31]
|
||||
The live view will be blank every Monday, no reason to tie it to a day of the
|
||||
week. It should be "the last 7 days"
|
||||
** DONE [#B] Implement a detail view for TV shows :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE [#B] Implement a detail view for Movies :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:04]
|
||||
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
|
||||
CLOSED: [2023-03-22 Wed 17:01]
|
||||
** DONE Add live chart view like Maloja :improvement:views:
|
||||
CLOSED: [2023-03-07 Tue 11:13]
|
||||
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:06]
|
||||
|
||||
An example:
|
||||
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
|
||||
|
||||
This is actually going to be moot because we can import from LastFM, and
|
||||
web-scrobbler integrates well with LastFM. The only thing to think through here
|
||||
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
|
||||
from Youtube (all videos get pushed, sigh).
|
||||
|
||||
** DONE Add Amazon scraper to look up books when OL fails :books:improvement:
|
||||
This turned out to be a non-starter ... Amazon is aggressive at disallowing
|
||||
scraping quality. And all the OSS tools out there are stuck in an arms race
|
||||
trying to keep them from breaking.
|
||||
|
||||
That said, Google Books actually has a decent API (for now), and I've built this
|
||||
out using that.
|
||||
|
||||
** DONE Fix bug in Jellyfin scrobbles that spam more scrobbles after completion :scrobbling:videos:bug:
|
||||
This was fixed a while ago, but there's a new manifested bug. Going to create a
|
||||
separate bug tracking ticket for that.
|
||||
* Version 0.11.4
|
||||
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
|
||||
** DONE [#A] Fix fetching artwork without release group :bug:
|
||||
CLOSED: [2023-01-29 Sun 14:27]
|
||||
|
||||
When we get artwork from Musicbrianz, and it's not found, we should check for
|
||||
release groups as well. This will stop issues with missing artwork because of
|
||||
obscure MB release matches.
|
||||
|
||||
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
|
||||
CLOSED: [2023-01-30 Mon 18:31]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
|
||||
:END:
|
||||
|
||||
If we play music from Jellyfin and the track reaches 90% completion, the
|
||||
scrobbling goes crazy and starts creating new scrobbles with every update.
|
||||
|
||||
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
|
||||
as complete for the following conditions:
|
||||
|
||||
- Play stopped and percent played beyond 90%
|
||||
- Play completely finished
|
||||
|
||||
But if we keep listening beyond 90, we should basically ignore updates (or just
|
||||
update the existing scrobble)
|
||||
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
|
||||
CLOSED: [2023-02-03 Fri 16:52]
|
||||
|
||||
An example of the format:
|
||||
#+begin_src csv
|
||||
,
|
||||
#AUDIOSCROBBLER/1.1
|
||||
#TZ/UNKNOWN
|
||||
#CLIENT/Rockbox sansaclipplus $Revision$
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Jackolantern’s Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
|
||||
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
|
||||
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
|
||||
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
|
||||
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
|
||||
,
|
||||
#+end_src
|
||||
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:10]
|
||||
|
||||
This would allow a few nice flows. One, you'd be able to record the play of an
|
||||
entire album by just dropping the muscibrainz_id in. This could be helpful for
|
||||
offline listening. It would also mean bad metadata from mopidy would not break
|
||||
scrobbling.
|
||||
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:11]
|
||||
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:09]
|
||||
|
||||
Given a UUID from musicbrainz, we should be able to scrobble an album or
|
||||
individual track.
|
||||
|
||||
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:10]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
|
||||
:END:
|
||||
|
||||
Maloja does this cool thing where artists and tracks get recorded as the top
|
||||
track of a given week, month or year. They get gold, silver or bronze stars for
|
||||
their place in the time period.
|
||||
|
||||
I could see this being implemented as a separate Chart table which gets
|
||||
populated at the end of a time period and has a start and end date that defines
|
||||
a period, along with a one, two, three instance.
|
||||
|
||||
Of course, it could also be a data model without a table, where it runs some fun
|
||||
calculations, stores it's values in Redis as a long-term lookup table and just
|
||||
has to re-populate when the server restarts.
|
||||
49
poetry.lock
generated
49
poetry.lock
generated
@ -1065,6 +1065,21 @@ ssh = ["bcrypt (>=3.1.5)"]
|
||||
test = ["certifi", "cryptography-vectors (==43.0.3)", "pretend", "pytest (>=6.2.0)", "pytest-benchmark", "pytest-cov", "pytest-xdist"]
|
||||
test-randomorder = ["pytest-randomly"]
|
||||
|
||||
[[package]]
|
||||
name = "dacite"
|
||||
version = "1.9.2"
|
||||
description = "Simple creation of data classes from dictionaries."
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
groups = ["main"]
|
||||
files = [
|
||||
{file = "dacite-1.9.2-py3-none-any.whl", hash = "sha256:053f7c3f5128ca2e9aceb66892b1a3c8936d02c686e707bee96e19deef4bc4a0"},
|
||||
{file = "dacite-1.9.2.tar.gz", hash = "sha256:6ccc3b299727c7aa17582f0021f6ae14d5de47c7227932c47fec4cdfefd26f09"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
dev = ["black", "coveralls", "mypy", "pre-commit", "pylint", "pytest (>=5)", "pytest-benchmark", "pytest-cov"]
|
||||
|
||||
[[package]]
|
||||
name = "dataclass-wizard"
|
||||
version = "0.22.0"
|
||||
@ -4610,6 +4625,23 @@ files = [
|
||||
[package.dependencies]
|
||||
rapidfuzz = ">=3.0.0,<4.0.0"
|
||||
|
||||
[[package]]
|
||||
name = "themoviedb"
|
||||
version = "1.0.2"
|
||||
description = "A modern and easy to use API wrapper for The Movie Database (TMDb) API v3 written in Python"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
groups = ["main"]
|
||||
files = [
|
||||
{file = "themoviedb-1.0.2-py3-none-any.whl", hash = "sha256:badf85e91010c7085509f40270bf2a40ea30ee5ef3ed6fb3ec332c5e50adb576"},
|
||||
{file = "themoviedb-1.0.2.tar.gz", hash = "sha256:7835615142a44e7ca25e48645a3a3c5e06b382e8c518c38c3537effa9a2596ce"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
aiohttp = ">=3.8.4"
|
||||
dacite = ">=1.8.0"
|
||||
requests = ">=2.31.0"
|
||||
|
||||
[[package]]
|
||||
name = "time-machine"
|
||||
version = "2.16.0"
|
||||
@ -4710,6 +4742,21 @@ files = [
|
||||
{file = "tld-0.13.tar.gz", hash = "sha256:93dde5e1c04bdf1844976eae440706379d21f4ab235b73c05d7483e074fb5629"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tmdbv3api"
|
||||
version = "1.9.0"
|
||||
description = "A lightweight Python library for The Movie Database (TMDb) API."
|
||||
optional = false
|
||||
python-versions = "*"
|
||||
groups = ["main"]
|
||||
files = [
|
||||
{file = "tmdbv3api-1.9.0-py3-none-any.whl", hash = "sha256:2bcd8c6e8902397860715a71045f200ecc3ee06804ecf786cb4c1e09b2deeba8"},
|
||||
{file = "tmdbv3api-1.9.0.tar.gz", hash = "sha256:504c5da6b99c4516ff160a01576112d097f209c0534f943c15c4b56cbd92c33b"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
requests = "*"
|
||||
|
||||
[[package]]
|
||||
name = "toml"
|
||||
version = "0.10.2"
|
||||
@ -5452,4 +5499,4 @@ cffi = ["cffi (>=1.11)"]
|
||||
[metadata]
|
||||
lock-version = "2.1"
|
||||
python-versions = ">=3.9,<3.12"
|
||||
content-hash = "cdd7f577fe3a4c5c8cc960e0070d93b7ddbb2a7968fab63d72bb039afaa05bbe"
|
||||
content-hash = "3a483aefea0a3afebf187b17b7df72a158788024ca8121b512b39567fb5ec8ca"
|
||||
|
||||
@ -54,6 +54,8 @@ meta-yt = "^0.1.9"
|
||||
berserk = "^0.13.2"
|
||||
poetry-bumpversion = "^0.3.3"
|
||||
orgparse = "^0.4.20250520"
|
||||
tmdbv3api = "^1.9.0"
|
||||
themoviedb = "^1.0.2"
|
||||
|
||||
[tool.poetry.group.test]
|
||||
optional = true
|
||||
|
||||
@ -161,7 +161,7 @@ class Artist(TimeStampedModel):
|
||||
|
||||
@property
|
||||
def rym_link(self):
|
||||
artist_slug = self.name.lower().replace(" ", "-")
|
||||
artist_slug = self.name.lower().replace(" ", "-").replace(",", "")
|
||||
return f"https://rateyourmusic.com/artist/{artist_slug}/"
|
||||
|
||||
@property
|
||||
|
||||
@ -92,7 +92,6 @@ class ScrobblableMixin(TimeStampedModel):
|
||||
"media_type": self.__class__,
|
||||
"user_id": user_id,
|
||||
"scrobble_data": scrobble_data,
|
||||
"media_type": Scrobble.MediaType.WEBPAGE,
|
||||
},
|
||||
)
|
||||
return Scrobble.create_or_update(self, user_id, scrobble_data)
|
||||
|
||||
@ -34,8 +34,7 @@ from videos.models import Video
|
||||
from webpages.models import WebPage
|
||||
|
||||
from vrobbler.apps.tasks.constants import (
|
||||
TODOIST_TITLE_PREFIX_LABELS,
|
||||
TODOIST_TITLE_SUFFIX_LABELS,
|
||||
TODOIST_TITLE_LABELS
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -421,10 +420,8 @@ def todoist_scrobble_task(
|
||||
# user.profile.task_context_tags which will result in context-based tag titles
|
||||
# We'd also have to migrate existing tasks to the new context based ones (maybe)
|
||||
for label in todoist_task["todoist_label_list"]:
|
||||
if label in TODOIST_TITLE_PREFIX_LABELS:
|
||||
prefix = label
|
||||
if label in TODOIST_TITLE_SUFFIX_LABELS:
|
||||
suffix = label
|
||||
if label in TODOIST_TITLE_LABELS:
|
||||
title = label
|
||||
|
||||
if not prefix and suffix:
|
||||
logger.warning(
|
||||
@ -432,8 +429,6 @@ def todoist_scrobble_task(
|
||||
extra={"todoist_scrobble_task": todoist_task},
|
||||
)
|
||||
|
||||
title = " ".join([prefix.capitalize(), suffix.capitalize()])
|
||||
|
||||
task = Task.find_or_create(title)
|
||||
|
||||
timestamp = pendulum.parse(todoist_task.get("updated_at", timezone.now()))
|
||||
@ -508,7 +503,7 @@ def emacs_scrobble_update_task(
|
||||
in_progress=True,
|
||||
user_id=user_id,
|
||||
log__source_id=emacs_id,
|
||||
log__source="emacs",
|
||||
log__source="orgmode",
|
||||
).first()
|
||||
|
||||
if not scrobble:
|
||||
@ -561,11 +556,9 @@ def emacs_scrobble_task(
|
||||
suffix = ""
|
||||
|
||||
source_id = task_data.get("source_id")
|
||||
for label in task_data.get("labels"):
|
||||
if label in TODOIST_TITLE_PREFIX_LABELS:
|
||||
prefix = label
|
||||
if label in TODOIST_TITLE_SUFFIX_LABELS:
|
||||
suffix = label
|
||||
for label in task_data.get("labels", []):
|
||||
if label in TODOIST_TITLE_LABELS:
|
||||
title = label
|
||||
|
||||
if not prefix and not suffix:
|
||||
logger.warning(
|
||||
|
||||
@ -201,7 +201,7 @@ class RecentScrobbleList(ListView):
|
||||
processed_finished__isnull=True,
|
||||
user=self.request.user,
|
||||
)
|
||||
data["counts"] = scrobble_counts(user)
|
||||
data["counts"] = [] #scrobble_counts(user)
|
||||
else:
|
||||
data["weekly_data"] = week_of_scrobbles()
|
||||
data["counts"] = scrobble_counts()
|
||||
|
||||
@ -1,10 +1,12 @@
|
||||
TODOIST_TITLE_PREFIX_LABELS = ["work", "home", "personal"]
|
||||
TODOIST_TITLE_SUFFIX_LABELS = [
|
||||
TODOIST_TITLE_LABELS = [
|
||||
"bug",
|
||||
"project",
|
||||
"feature",
|
||||
"errand",
|
||||
"chore",
|
||||
"admin",
|
||||
"computer",
|
||||
"farm",
|
||||
"baking",
|
||||
"meeting",
|
||||
"habit",
|
||||
"research",
|
||||
|
||||
@ -149,13 +149,6 @@ def emacs_webhook(request):
|
||||
post_data, user_id, started=task_in_progress, stopped=task_stopped
|
||||
)
|
||||
|
||||
if scrobble and task_in_progress and post_data.get("notes"):
|
||||
emacs_scrobble_update_task(
|
||||
post_data.get("source_id"),
|
||||
post_data.get("notes"),
|
||||
user_id,
|
||||
)
|
||||
|
||||
if not scrobble:
|
||||
logger.info(
|
||||
"[emacs_webhook] finished with no note or task found",
|
||||
@ -166,6 +159,13 @@ def emacs_webhook(request):
|
||||
status=status.HTTP_304_NOT_MODIFIED,
|
||||
)
|
||||
|
||||
if task_in_progress and post_data.get("notes"):
|
||||
emacs_scrobble_update_task(
|
||||
post_data.get("source_id"),
|
||||
post_data.get("notes"),
|
||||
user_id,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"[emacs_webhook] finished",
|
||||
extra={"scrobble_id": scrobble.id},
|
||||
|
||||
@ -0,0 +1,23 @@
|
||||
# Generated by Django 4.2.19 on 2025-06-16 14:20
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
("trails", "0004_alter_trail_run_time_seconds"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="trail",
|
||||
name="alltrails_id",
|
||||
field=models.CharField(blank=True, max_length=255, null=True),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="trail",
|
||||
name="gaiagps_id",
|
||||
field=models.CharField(blank=True, max_length=255, null=True),
|
||||
),
|
||||
]
|
||||
@ -9,9 +9,7 @@ from locations.models import GeoLocation
|
||||
BNULL = {"blank": True, "null": True}
|
||||
|
||||
|
||||
|
||||
class Trail(ScrobblableMixin):
|
||||
|
||||
class PrincipalType(models.TextChoices):
|
||||
WOODS = "WOODS"
|
||||
ROAD = "ROAD"
|
||||
@ -19,7 +17,7 @@ class Trail(ScrobblableMixin):
|
||||
MOUNTAIN = "MOUNTAIN"
|
||||
|
||||
class ActivityType(models.TextChoices):
|
||||
WALK= "WALK"
|
||||
WALK = "WALK"
|
||||
HIKE = "HIKE"
|
||||
RUN = "RUN"
|
||||
BIKE = "BIKE"
|
||||
@ -39,8 +37,14 @@ class Trail(ScrobblableMixin):
|
||||
)
|
||||
strava_id = models.CharField(max_length=255, **BNULL)
|
||||
trailforks_id = models.CharField(max_length=255, **BNULL)
|
||||
principal_type = models.CharField(max_length=10, choices=PrincipalType.choices, **BNULL)
|
||||
default_activity_type = models.CharField(max_length=10, choices=ActivityType.choices, **BNULL)
|
||||
alltrails_id = models.CharField(max_length=255, **BNULL)
|
||||
gaiagps_id = models.CharField(max_length=255, **BNULL)
|
||||
principal_type = models.CharField(
|
||||
max_length=10, choices=PrincipalType.choices, **BNULL
|
||||
)
|
||||
default_activity_type = models.CharField(
|
||||
max_length=10, choices=ActivityType.choices, **BNULL
|
||||
)
|
||||
|
||||
def get_absolute_url(self):
|
||||
return reverse("trails:trail_detail", kwargs={"slug": self.uuid})
|
||||
|
||||
61
vrobbler/apps/trails/sources/gaiagps.py
Normal file
61
vrobbler/apps/trails/sources/gaiagps.py
Normal file
@ -0,0 +1,61 @@
|
||||
from typing import Optional
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
|
||||
def get_description_from_soup(soup) -> Optional[int]:
|
||||
rating = None
|
||||
try:
|
||||
potential_desc = soup.find("div", id_="descriptionContainer")
|
||||
potential_rating = soup.find("div", class_="allmusic-rating")
|
||||
if potential_rating:
|
||||
rating = int(strip_and_clean(potential_rating.get_text()))
|
||||
except ValueError:
|
||||
pass
|
||||
return rating
|
||||
|
||||
|
||||
def scrape_gaia_trail(url):
|
||||
headers = {"User-Agent": "Mozilla/5.0"}
|
||||
r = requests.get(url, headers=headers)
|
||||
r.raise_for_status()
|
||||
soup = BeautifulSoup(r.text, "html.parser")
|
||||
|
||||
# Title
|
||||
title = soup.select_one("h1").get_text(strip=True)
|
||||
|
||||
# Stats are inside divs with classes like "flex" and label-value pairs
|
||||
stats = {}
|
||||
stat_blocks = soup.select("div.flex.flex-col.items-start.text-sm span")
|
||||
for i in range(0, len(stat_blocks) - 1, 2):
|
||||
label = stat_blocks[i].get_text(strip=True)
|
||||
value = stat_blocks[i + 1].get_text(strip=True)
|
||||
stats[label] = value
|
||||
|
||||
# Description (under 'Overview' header)
|
||||
overview_heading = soup.find(
|
||||
lambda tag: tag.name == "h2" and "Overview" in tag.text
|
||||
)
|
||||
if overview_heading:
|
||||
desc_parts = []
|
||||
for sib in overview_heading.next_siblings:
|
||||
if sib.name and sib.name.startswith("h"):
|
||||
break
|
||||
if sib.name == "p":
|
||||
desc_parts.append(sib.get_text(strip=True))
|
||||
description = "\n\n".join(desc_parts).strip()
|
||||
else:
|
||||
description = None
|
||||
|
||||
return {
|
||||
"url": url,
|
||||
"title": title,
|
||||
"stats": stats,
|
||||
"description": description,
|
||||
}
|
||||
|
||||
|
||||
# Example usage
|
||||
url = "https://www.gaiagps.com/hike/318136/"
|
||||
trail_data = scrape_gaia_trail(url)
|
||||
print(trail_data)
|
||||
@ -25,6 +25,7 @@ class VideoMetadata:
|
||||
60 # Silly default, but things break if this is 0 or null
|
||||
)
|
||||
imdb_id: Optional[str]
|
||||
tmdb_id: Optional[str]
|
||||
youtube_id: Optional[str]
|
||||
|
||||
# IMDB specific
|
||||
@ -35,6 +36,7 @@ class VideoMetadata:
|
||||
tv_series_id: Optional[int]
|
||||
plot: Optional[str]
|
||||
imdb_rating: Optional[str]
|
||||
tmdb_rating: Optional[str]
|
||||
cover_url: Optional[str]
|
||||
overview: Optional[str]
|
||||
|
||||
@ -59,6 +61,6 @@ class VideoMetadata:
|
||||
video_dict = vars(self)
|
||||
cover = None
|
||||
if "cover_url" in video_dict.keys():
|
||||
cover = video_dict.pop("cover_url")
|
||||
genres = video_dict.pop("genres")
|
||||
cover = video_dict.pop("cover_url", "")
|
||||
genres = video_dict.pop("genres", [])
|
||||
return video_dict, cover, genres
|
||||
|
||||
18
vrobbler/apps/videos/migrations/0023_video_tmdb_rating.py
Normal file
18
vrobbler/apps/videos/migrations/0023_video_tmdb_rating.py
Normal file
@ -0,0 +1,18 @@
|
||||
# Generated by Django 4.2.19 on 2025-06-13 15:04
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('videos', '0022_alter_video_run_time_seconds'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name='video',
|
||||
name='tmdb_rating',
|
||||
field=models.FloatField(blank=True, null=True),
|
||||
),
|
||||
]
|
||||
@ -2,7 +2,6 @@ import logging
|
||||
from typing import Optional
|
||||
from uuid import uuid4
|
||||
|
||||
import pendulum
|
||||
import requests
|
||||
from django.conf import settings
|
||||
from django.core.files.base import ContentFile
|
||||
@ -21,8 +20,8 @@ from scrobbles.mixins import (
|
||||
from taggit.managers import TaggableManager
|
||||
from videos.metadata import VideoMetadata
|
||||
from videos.sources.imdb import lookup_video_from_imdb
|
||||
|
||||
from vrobbler.apps.videos.sources.youtube import lookup_video_from_youtube
|
||||
from videos.sources.tmdb import lookup_video_from_tmdb
|
||||
from videos.sources.youtube import lookup_video_from_youtube
|
||||
|
||||
YOUTUBE_VIDEO_URL = "https://www.youtube.com/watch?v="
|
||||
YOUTUBE_CHANNEL_URL = "https://www.youtube.com/channel/"
|
||||
@ -208,6 +207,7 @@ class Video(ScrobblableMixin):
|
||||
next_imdb_id = models.CharField(max_length=20, **BNULL)
|
||||
imdb_id = models.CharField(max_length=20, **BNULL)
|
||||
imdb_rating = models.FloatField(**BNULL)
|
||||
tmdb_rating = models.FloatField(**BNULL)
|
||||
cover_image = models.ImageField(upload_to="videos/video/", **BNULL)
|
||||
cover_image_small = ImageSpecField(
|
||||
source="cover_image",
|
||||
@ -309,11 +309,13 @@ class Video(ScrobblableMixin):
|
||||
def get_from_imdb_id(
|
||||
cls, imdb_id: str, overwrite: bool = False
|
||||
) -> "Video":
|
||||
if "tt" in imdb_id:
|
||||
imdb_id = imdb_id[2:]
|
||||
video, created = cls.objects.get_or_create(imdb_id=imdb_id)
|
||||
if not created and not overwrite:
|
||||
return video
|
||||
|
||||
vdict, cover, genres = lookup_video_from_imdb(
|
||||
vdict, cover, genres = lookup_video_from_tmdb(
|
||||
imdb_id
|
||||
).as_dict_with_cover_and_genres()
|
||||
if created or overwrite:
|
||||
|
||||
@ -72,26 +72,30 @@ def lookup_video_from_imdb(
|
||||
video_metadata.video_type = VideoType.MOVIE.value
|
||||
series_name = None
|
||||
if imdb_result.get("kind") == "episode":
|
||||
series_name = imdb_result.get("episode of", None).data.get(
|
||||
"title", None
|
||||
)
|
||||
series, _ = Series.objects.get_or_create(name=series_name)
|
||||
video_metadata.video_type = VideoType.TV_EPISODE.value
|
||||
video_metadata.tv_series_id = series.id
|
||||
try:
|
||||
series_name = imdb_result.get("episode of", None).data.get(
|
||||
"title", None
|
||||
)
|
||||
except IndexError:
|
||||
series_name = None
|
||||
if series_name:
|
||||
series, _ = Series.objects.get_or_create(name=series_name)
|
||||
video_metadata.video_type = VideoType.TV_EPISODE.value
|
||||
video_metadata.tv_series_id = series.id
|
||||
|
||||
if imdb_result.get("runtimes"):
|
||||
video_metadata.run_time_seconds = (
|
||||
int(imdb_result.get("runtimes")[0]) * 60
|
||||
)
|
||||
|
||||
video_metadata.imdb_id = name_or_id
|
||||
video_metadata.title = imdb_result.get("title", "")
|
||||
video_metadata.imdb_id = imdb_result.get("imdbID")
|
||||
video_metadata.episode_number = imdb_result.get("episode", None)
|
||||
video_metadata.season_number = imdb_result.get("season", None)
|
||||
video_metadata.next_imdb_id = imdb_result.get("next episode", None)
|
||||
video_metadata.year = imdb_result.get("year", None)
|
||||
video_metadata.plot = imdb_result.get("plot outline")
|
||||
video_metadata.imdb_rating = imdb_result.get("rating")
|
||||
video_metadata.genres = imdb_result.get("genres")
|
||||
video_metadata.plot = imdb_result.get("plot outline", "")
|
||||
video_metadata.imdb_rating = imdb_result.get("rating", None)
|
||||
video_metadata.genres = imdb_result.get("genres", [])
|
||||
|
||||
return video_metadata
|
||||
|
||||
81
vrobbler/apps/videos/sources/tmdb.py
Normal file
81
vrobbler/apps/videos/sources/tmdb.py
Normal file
@ -0,0 +1,81 @@
|
||||
import logging
|
||||
|
||||
import pendulum
|
||||
from django.conf import settings
|
||||
from themoviedb import TMDb
|
||||
from tmdbv3api import TV, Movie, TMDb as TMDb_direct
|
||||
from videos.metadata import VideoMetadata, VideoType
|
||||
|
||||
TMDB_KEY = getattr(settings, "TMDB_API_KEY", "")
|
||||
|
||||
tmdb_direct = TMDb_direct()
|
||||
tmdb_direct.api_key = TMDB_KEY
|
||||
|
||||
tmdb = TMDb(key=TMDB_KEY, language="en-US", region="US")
|
||||
|
||||
TMDB_IMAGE_URL = "https://image.tmdb.org/t/p/original"
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def lookup_video_from_tmdb(
|
||||
name_or_id: str, kind: str = "movie"
|
||||
) -> VideoMetadata:
|
||||
from videos.models import Series
|
||||
|
||||
imdb_id = name_or_id
|
||||
if name_or_id.startswith("tt"):
|
||||
imdb_id = name_or_id[2:]
|
||||
|
||||
video_metadata = VideoMetadata(imdb_id=imdb_id)
|
||||
|
||||
tmdb_result = tmdb.find().by_imdb("tt" + imdb_id)
|
||||
|
||||
if not tmdb_result:
|
||||
logger.info(
|
||||
"[lookup_video_from_tmdb] no video found on tmdb",
|
||||
extra={"name_or_id": name_or_id},
|
||||
)
|
||||
return None
|
||||
|
||||
video_metadata = VideoMetadata(imdb_id=imdb_id)
|
||||
|
||||
media = None
|
||||
show = None
|
||||
if len(tmdb_result.movie_results) > 0:
|
||||
media = Movie().details(tmdb_result.movie_results[0].id)
|
||||
video_metadata.video_type = VideoType.MOVIE.value
|
||||
video_metadata.title = media.title
|
||||
video_metadata.cover_url = TMDB_IMAGE_URL + media.poster_path # TODO: enrich this with TMDB url
|
||||
video_metadata.year = pendulum.parse(media.release_date).year
|
||||
video_metadata.genres = [g.get("name", "") for g in media.genres]
|
||||
|
||||
if len(tmdb_result.tv_episode_results) > 0:
|
||||
video_metadata.video_type = VideoType.TV_EPISODE.value
|
||||
media = tmdb_result.tv_episode_results[0]
|
||||
video_metadata.title = media.name
|
||||
video_metadata.cover_url = TMDB_IMAGE_URL + media.still_path # TODO: enrich this with TMDB url
|
||||
video_metadata.episode_number = media.episode_number
|
||||
video_metadata.season_number = media.season_number
|
||||
video_metadata.year = media.air_date.year
|
||||
|
||||
series_imdb_id = TV().external_ids(media.show_id).imdb_id[2:]
|
||||
series, created = Series.objects.get_or_create(imdb_id=series_imdb_id)
|
||||
if created:
|
||||
show_data = TV().details(tv_id=media.show_id)
|
||||
series.name = show_data.name
|
||||
series.save()
|
||||
video_metadata.tv_series_id = series.id
|
||||
|
||||
if not media:
|
||||
logger.warning("Video not found on TMDB", extra={"imdb_id":imdb_id})
|
||||
return video_metadata
|
||||
|
||||
video_metadata.tmdb_id = media.id
|
||||
video_metadata.run_time_seconds = media.runtime * 60
|
||||
video_metadata.plot = media.overview
|
||||
video_metadata.overview = media.overview
|
||||
video_metadata.tmdb_rating = media.vote_average
|
||||
#video_metadata.next_imdb_id = imdb_result.get("next episode", None)
|
||||
|
||||
return video_metadata
|
||||
@ -2,6 +2,12 @@
|
||||
{% load naturalduration %}
|
||||
<tr {% if scrobble.in_progress %}class="in-progress"{% endif %}>
|
||||
<td>{% if scrobble.in_progress %}{{scrobble.media_obj.strings.verb}} now | <a class="right" href="{% url "scrobbles:finish" scrobble.uuid %}">Finish</a>{% else %}{{scrobble.timestamp|naturaltime}}{% endif %}</td>
|
||||
<td><a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj|truncatechars_html:45}}</a></td>
|
||||
<td>
|
||||
{% if scrobble.media_type == "Task" %}
|
||||
<p><em><a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj.title|truncatechars_html:45}} - {% if scrobble.logdata %}{% if scrobble.logdata.description %}{{scrobble.logdata.description}}{% endif %}{% endif %}</a></em></p>
|
||||
{% else %}
|
||||
<a href="{{scrobble.media_obj.get_absolute_url}}">{{scrobble.media_obj|truncatechars_html:45}}</a>
|
||||
{% endif %}
|
||||
</td>
|
||||
<td>{{scrobble.elapsed_time|natural_duration}}</td>
|
||||
</tr>
|
||||
|
||||
@ -57,6 +57,7 @@ dd {
|
||||
<div class="row header">
|
||||
<div class="cover image-wrapper">
|
||||
{% if object.imdb_rating %}<div class="caption">{{object.imdb_rating}}</div>{% endif %}
|
||||
{% if object.tmdb_rating %}<div class="caption">{{object.tmdb_rating}}</div>{% endif %}
|
||||
<img src="{% if object.cover_image %}{{object.cover_image.url}}{% else %}{% static 'images/no-video-cover.jpg' %}{% endif %}" width="400px" />
|
||||
<div class="caption-footer">{{object.year}}{% if object.tv_series %} | <b>S</b>{{object.season_number}} <b>E</b>{{object.episode_number}}{% endif %}</div>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user