Fix charts to do rolling day counts
This commit is contained in:
247
todos.org
247
todos.org
@ -2,6 +2,130 @@
|
||||
|
||||
A fun way to keep track of things in the project to fix or improve.
|
||||
|
||||
* Version 1.0.0
|
||||
** TODO [#A] Add django-storage to store files on S3 :improvement:
|
||||
** TODO Add a "finished_timestamp" so we don't rely on content length :improvement:scrobbling:
|
||||
|
||||
Essentially, we currently have the timestamp as when the content began
|
||||
scrobbling and then calculate the finish time from the length of the content.
|
||||
This works pretty well because we know how long most things are.
|
||||
|
||||
But in some cases, sports events or long podcasts, we may start mid-way through
|
||||
an event or finish halfway through but still want to mark it as done. In these
|
||||
cases, knowing the finish time could be useful, especially when interfacing with
|
||||
other scrobblers which may have different definitions of when a scrobble
|
||||
finishes or started.
|
||||
|
||||
** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
|
||||
CLOSED: [2023-03-24 Fri 00:31]
|
||||
The live view will be blank every Monday, no reason to tie it to a day of the
|
||||
week. It should be "the last 7 days"
|
||||
** DONE [#B] Implement a detail view for TV shows :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE [#B] Implement a detail view for Movies :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:04]
|
||||
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
|
||||
CLOSED: [2023-03-22 Wed 17:01]
|
||||
** DONE Add live chart view like Maloja :improvement:views:
|
||||
CLOSED: [2023-03-07 Tue 11:13]
|
||||
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:06]
|
||||
|
||||
An example:
|
||||
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
|
||||
|
||||
This is actually going to be moot because we can import from LastFM, and
|
||||
web-scrobbler integrates well with LastFM. The only thing to think through here
|
||||
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
|
||||
from Youtube (all videos get pushed, sigh).
|
||||
|
||||
* Version 0.11.4
|
||||
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
|
||||
** DONE [#A] Fix fetching artwork without release group :bug:
|
||||
CLOSED: [2023-01-29 Sun 14:27]
|
||||
|
||||
When we get artwork from Musicbrianz, and it's not found, we should check for
|
||||
release groups as well. This will stop issues with missing artwork because of
|
||||
obscure MB release matches.
|
||||
|
||||
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
|
||||
CLOSED: [2023-01-30 Mon 18:31]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
|
||||
:END:
|
||||
|
||||
If we play music from Jellyfin and the track reaches 90% completion, the
|
||||
scrobbling goes crazy and starts creating new scrobbles with every update.
|
||||
|
||||
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
|
||||
as complete for the following conditions:
|
||||
|
||||
- Play stopped and percent played beyond 90%
|
||||
- Play completely finished
|
||||
|
||||
But if we keep listening beyond 90, we should basically ignore updates (or just
|
||||
update the existing scrobble)
|
||||
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
|
||||
CLOSED: [2023-02-03 Fri 16:52]
|
||||
|
||||
An example of the format:
|
||||
#+begin_src csv
|
||||
,
|
||||
#AUDIOSCROBBLER/1.1
|
||||
#TZ/UNKNOWN
|
||||
#CLIENT/Rockbox sansaclipplus $Revision$
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Jackolantern’s Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
|
||||
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
|
||||
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
|
||||
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
|
||||
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
|
||||
,
|
||||
#+end_src
|
||||
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:10]
|
||||
|
||||
This would allow a few nice flows. One, you'd be able to record the play of an
|
||||
entire album by just dropping the muscibrainz_id in. This could be helpful for
|
||||
offline listening. It would also mean bad metadata from mopidy would not break
|
||||
scrobbling.
|
||||
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:11]
|
||||
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:09]
|
||||
|
||||
Given a UUID from musicbrainz, we should be able to scrobble an album or
|
||||
individual track.
|
||||
|
||||
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:10]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
|
||||
:END:
|
||||
|
||||
Maloja does this cool thing where artists and tracks get recorded as the top
|
||||
track of a given week, month or year. They get gold, silver or bronze stars for
|
||||
their place in the time period.
|
||||
|
||||
I could see this being implemented as a separate Chart table which gets
|
||||
populated at the end of a time period and has a start and end date that defines
|
||||
a period, along with a one, two, three instance.
|
||||
|
||||
Of course, it could also be a data model without a table, where it runs some fun
|
||||
calculations, stores it's values in Redis as a long-term lookup table and just
|
||||
has to re-populate when the server restarts.
|
||||
* Backlog
|
||||
** TODO [#C] Move to using more robust mopidy-webhooks pacakge form pypi :utility:improvement:
|
||||
*** Example payloads from mopidy-webhooks
|
||||
@ -296,126 +420,3 @@ A fun way to keep track of things in the project to fix or improve.
|
||||
#+end_src
|
||||
** TODO [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
|
||||
** TODO What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
|
||||
* Version 1.0.0
|
||||
** TODO [#A] Add django-storage to store files on S3 :improvement:
|
||||
** TODO Add a "finished_timestamp" so we don't rely on content length :improvement:scrobbling:
|
||||
|
||||
Essentially, we currently have the timestamp as when the content began
|
||||
scrobbling and then calculate the finish time from the length of the content.
|
||||
This works pretty well because we know how long most things are.
|
||||
|
||||
But in some cases, sports events or long podcasts, we may start mid-way through
|
||||
an event or finish halfway through but still want to mark it as done. In these
|
||||
cases, knowing the finish time could be useful, especially when interfacing with
|
||||
other scrobblers which may have different definitions of when a scrobble
|
||||
finishes or started.
|
||||
|
||||
** TODO Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
|
||||
The live view will be blank every Monday, no reason to tie it to a day of the
|
||||
week. It should be "the last 7 days"
|
||||
** DONE [#B] Implement a detail view for TV shows :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE [#B] Implement a detail view for Movies :improvement:views:
|
||||
CLOSED: [2023-03-22 Wed 17:05]
|
||||
** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:04]
|
||||
** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
|
||||
CLOSED: [2023-03-22 Wed 17:01]
|
||||
** DONE Add live chart view like Maloja :improvement:views:
|
||||
CLOSED: [2023-03-07 Tue 11:13]
|
||||
** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
|
||||
CLOSED: [2023-03-22 Wed 17:06]
|
||||
|
||||
An example:
|
||||
https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
|
||||
|
||||
This is actually going to be moot because we can import from LastFM, and
|
||||
web-scrobbler integrates well with LastFM. The only thing to think through here
|
||||
now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
|
||||
from Youtube (all videos get pushed, sigh).
|
||||
|
||||
* Version 0.11.4
|
||||
** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
|
||||
CLOSED: [2023-03-07 Tue 11:11]
|
||||
|
||||
** DONE [#A] Fix fetching artwork without release group :bug:
|
||||
CLOSED: [2023-01-29 Sun 14:27]
|
||||
|
||||
When we get artwork from Musicbrianz, and it's not found, we should check for
|
||||
release groups as well. This will stop issues with missing artwork because of
|
||||
obscure MB release matches.
|
||||
|
||||
** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
|
||||
CLOSED: [2023-01-30 Mon 18:31]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] => 0:31
|
||||
:END:
|
||||
|
||||
If we play music from Jellyfin and the track reaches 90% completion, the
|
||||
scrobbling goes crazy and starts creating new scrobbles with every update.
|
||||
|
||||
The cause is pretty simple, but the solution is hard. We want to mark a scrobble
|
||||
as complete for the following conditions:
|
||||
|
||||
- Play stopped and percent played beyond 90%
|
||||
- Play completely finished
|
||||
|
||||
But if we keep listening beyond 90, we should basically ignore updates (or just
|
||||
update the existing scrobble)
|
||||
** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
|
||||
CLOSED: [2023-02-03 Fri 16:52]
|
||||
|
||||
An example of the format:
|
||||
#+begin_src csv
|
||||
,
|
||||
#AUDIOSCROBBLER/1.1
|
||||
#TZ/UNKNOWN
|
||||
#CLIENT/Rockbox sansaclipplus $Revision$
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494944 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
75 Dollar Bill I Was Real I Was Real 4 1015 S 1740494990 64ff5f53-d187-4512-827e-7606c69e66ff
|
||||
311 311 Down 1 173 S 1740495003 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495049 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Down 1 173 L 1740495113 00476c23-fd9e-464b-9b27-a62d69f3d4f4
|
||||
311 311 Random 2 187 S 1740495190 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Random 2 187 L 1740495194 530c09f3-46fe-4d90-b11f-7b63bcb4b373
|
||||
311 311 Jackolantern’s Weather 3 204 L 1740495382 cc3b2dec-5d99-47ea-8930-20bf258be4ea
|
||||
311 311 All Mixed Up 4 182 L 1740495586 980a78b5-5bdd-4f50-9e3a-e13261e2817b
|
||||
311 311 Hive 5 179 L 1740495768 18f6dc98-d3a2-4f81-b967-97359d14c68c
|
||||
311 311 Guns (Are for Pussies) 6 137 L 1740495948 5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
|
||||
311 311 Misdirected Hostility 7 179 S 1740496085 61ff2c1a-fc9c-44c3-8da1-5e50a44245af
|
||||
,
|
||||
#+end_src
|
||||
** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:10]
|
||||
|
||||
This would allow a few nice flows. One, you'd be able to record the play of an
|
||||
entire album by just dropping the muscibrainz_id in. This could be helpful for
|
||||
offline listening. It would also mean bad metadata from mopidy would not break
|
||||
scrobbling.
|
||||
** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
|
||||
CLOSED: [2023-02-17 Fri 00:11]
|
||||
** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:09]
|
||||
|
||||
Given a UUID from musicbrainz, we should be able to scrobble an album or
|
||||
individual track.
|
||||
|
||||
** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
|
||||
CLOSED: [2023-03-07 Tue 11:10]
|
||||
:LOGBOOK:
|
||||
CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] => 1:30
|
||||
:END:
|
||||
|
||||
Maloja does this cool thing where artists and tracks get recorded as the top
|
||||
track of a given week, month or year. They get gold, silver or bronze stars for
|
||||
their place in the time period.
|
||||
|
||||
I could see this being implemented as a separate Chart table which gets
|
||||
populated at the end of a time period and has a start and end date that defines
|
||||
a period, along with a one, two, three instance.
|
||||
|
||||
Of course, it could also be a data model without a table, where it runs some fun
|
||||
calculations, stores it's values in Redis as a long-term lookup table and just
|
||||
has to re-populate when the server restarts.
|
||||
|
||||
@ -95,6 +95,8 @@ def live_charts(
|
||||
now = now_user_timezone(user.profile)
|
||||
tzinfo = now.tzinfo
|
||||
|
||||
seven_days_ago = now - timedelta(days=7)
|
||||
thirty_days_ago = now - timedelta(days=30)
|
||||
start_of_today = datetime.combine(now, datetime.min.time(), tzinfo)
|
||||
start_day_of_week = now - timedelta(days=now.today().isoweekday() % 7)
|
||||
start_day_of_month = now.replace(day=1)
|
||||
@ -105,6 +107,8 @@ def live_charts(
|
||||
period_queries = {
|
||||
"today": {"scrobble__timestamp__gte": start_of_today},
|
||||
"week": {"scrobble__timestamp__gte": start_day_of_week},
|
||||
"last7": {"scrobble__timestamp__gte": seven_days_ago},
|
||||
"last30": {"scrobble__timestamp__gte": thirty_days_ago},
|
||||
"month": {"scrobble__timestamp__gte": start_day_of_month},
|
||||
"year": {"scrobble__timestamp__gte": start_day_of_year},
|
||||
"all": {},
|
||||
|
||||
@ -102,10 +102,17 @@ class RecentScrobbleList(ListView):
|
||||
limit = 14
|
||||
artist = {"user": user, "media_type": "Artist", "limit": limit}
|
||||
# This is weird. They don't display properly as QuerySets, so we cast to lists
|
||||
data["chart_keys"] = {
|
||||
"today": "Today",
|
||||
"last7": "Last 7 days",
|
||||
"last30": "Last 30 days",
|
||||
"year": "This year",
|
||||
"all": "All time",
|
||||
}
|
||||
data["current_artist_charts"] = {
|
||||
"today": list(live_charts(**artist, chart_period="today")),
|
||||
"week": list(live_charts(**artist, chart_period="week")),
|
||||
"month": list(live_charts(**artist, chart_period="month")),
|
||||
"last7": list(live_charts(**artist, chart_period="last7")),
|
||||
"last30": list(live_charts(**artist, chart_period="last30")),
|
||||
"year": list(live_charts(**artist, chart_period="year")),
|
||||
"all": list(live_charts(**artist)),
|
||||
}
|
||||
@ -113,8 +120,8 @@ class RecentScrobbleList(ListView):
|
||||
track = {"user": user, "media_type": "Track", "limit": limit}
|
||||
data["current_track_charts"] = {
|
||||
"today": list(live_charts(**track, chart_period="today")),
|
||||
"week": list(live_charts(**track, chart_period="week")),
|
||||
"month": list(live_charts(**track, chart_period="month")),
|
||||
"last7": list(live_charts(**track, chart_period="last7")),
|
||||
"last30": list(live_charts(**track, chart_period="last30")),
|
||||
"year": list(live_charts(**track, chart_period="year")),
|
||||
"all": list(live_charts(**track)),
|
||||
}
|
||||
|
||||
@ -89,20 +89,18 @@
|
||||
<div class="row">
|
||||
<h2>Top Artist</h2>
|
||||
<ul class="nav nav-tabs" id="artistTab" role="tablist">
|
||||
{% for chart_name in current_artist_charts.keys %}
|
||||
{% for key, name in chart_keys.items %}
|
||||
<li class="nav-item" role="presentation">
|
||||
<button class="nav-link {% if forloop.counter == 2 %}active{% endif %}"
|
||||
id="artist-{{chart_name}}-tab" data-bs-toggle="tab" data-bs-target="#artist-{{chart_name}}"
|
||||
type="button" role="tab" aria-controls="home" aria-selected="true">
|
||||
{% if chart_name == "all" %}All Time{% else %}{% if chart_name != "today" %}This {% endif %}{{chart_name|capfirst}}{% endif %}
|
||||
</button>
|
||||
id="artist-{{key}}-tab" data-bs-toggle="tab" data-bs-target="#artist-{{key}}"
|
||||
type="button" role="tab" aria-controls="home" aria-selected="true">{{name}}</button>
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
|
||||
<div class="tab-content" id="artistTabContent" class="maloja-chart">
|
||||
{% for chart_name, artists in current_artist_charts.items %}
|
||||
<div class="tab-pane fade {% if forloop.counter == 2 %}show active{% endif %}" id="artist-{{chart_name}}" role="tabpanel" aria-labelledby="artist-{{chart_name}}-tab">
|
||||
{% for key, artists in current_artist_charts.items %}
|
||||
<div class="tab-pane fade {% if forloop.counter == 2 %}show active{% endif %}" id="artist-{{key}}" role="tabpanel" aria-labelledby="artist-{{key}}-tab">
|
||||
<div style="display:block">
|
||||
<div style="float:left;">
|
||||
<div class="image-wrapper" style="display:flex; flex-wrap: wrap; margin:0">
|
||||
@ -263,12 +261,10 @@
|
||||
<div class="row">
|
||||
<h2>Top Tracks</h2>
|
||||
<ul class="nav nav-tabs" id="trackTab" role="tablist">
|
||||
{% for chart_name in current_track_charts.keys %}
|
||||
{% for key, name in chart_keys.items %}
|
||||
<li class="nav-item" role="presentation">
|
||||
<button class="nav-link {% if forloop.counter == 2 %}active{% endif %}" id="track-{{chart_name}}-tab" data-bs-toggle="tab"
|
||||
data-bs-target="#track-{{chart_name}}" type="button" role="tab" aria-controls="home" aria-selected="true">
|
||||
{% if chart_name == "all" %}All Time{% else %}{% if chart_name != "today" %}This {% endif %}{{chart_name|capfirst}}{% endif %}
|
||||
</button>
|
||||
<button class="nav-link {% if forloop.counter == 2 %}active{% endif %}" id="track-{{key}}-tab" data-bs-toggle="tab"
|
||||
data-bs-target="#track-{{key}}" type="button" role="tab" aria-controls="home" aria-selected="true">{{name}}</button>
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
|
||||
Reference in New Issue
Block a user