Fix charts to do rolling day counts

todos.org

@@ -2,6 +2,130 @@
 
 A fun way to keep track of things in the project to fix or improve.
 
+* Version 1.0.0
+** TODO [#A] Add django-storage to store files on S3 :improvement:
+** TODO Add a "finished_timestamp" so we don't rely on content length :improvement:scrobbling:
+
+Essentially, we currently have the timestamp as when the content began
+scrobbling and then calculate the finish time from the length of the content.
+This works pretty well because we know how long most things are.
+
+But in some cases, sports events or long podcasts, we may start mid-way through
+an event or finish halfway through but still want to mark it as done. In these
+cases, knowing the finish time could be useful, especially when interfacing with
+other scrobblers which may have different definitions of when a scrobble
+finishes or started.
+
+** DONE Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
+CLOSED: [2023-03-24 Fri 00:31]
+The live view will be blank every Monday, no reason to tie it to a day of the
+week. It should be "the last 7 days"
+** DONE [#B] Implement a detail view for TV shows :improvement:views:
+CLOSED: [2023-03-22 Wed 17:05]
+** DONE [#B] Implement a detail view for Movies :improvement:views:
+CLOSED: [2023-03-22 Wed 17:05]
+** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
+CLOSED: [2023-03-22 Wed 17:04]
+** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
+CLOSED: [2023-03-22 Wed 17:01]
+** DONE Add live chart view like Maloja :improvement:views:
+CLOSED: [2023-03-07 Tue 11:13]
+** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
+CLOSED: [2023-03-22 Wed 17:06]
+
+An example:
+https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
+
+This is actually going to be moot because we can import from LastFM, and
+web-scrobbler integrates well with LastFM. The only thing to think through here
+now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
+from Youtube (all videos get pushed, sigh).
+
+* Version 0.11.4
+** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
+CLOSED: [2023-03-07 Tue 11:11]
+** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
+CLOSED: [2023-03-07 Tue 11:11]
+
+** DONE [#A] Fix fetching artwork without release group :bug:
+CLOSED: [2023-01-29 Sun 14:27]
+
+When we get artwork from Musicbrianz, and it's not found, we should check for
+release groups as well. This will stop issues with missing artwork because of
+obscure MB release matches.
+
+** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
+CLOSED: [2023-01-30 Mon 18:31]
+:LOGBOOK:
+CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] =>  0:31
+:END:
+
+If we play music from Jellyfin and the track reaches 90% completion, the
+scrobbling goes crazy and starts creating new scrobbles with every update.
+
+The cause is pretty simple, but the solution is hard. We want to mark a scrobble
+as complete for the following conditions:
+
+- Play stopped and percent played beyond 90%
+- Play completely finished
+
+But if we keep listening beyond 90, we should basically ignore updates (or just
+update the existing scrobble)
+** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
+CLOSED: [2023-02-03 Fri 16:52]
+
+An example of the format:
+#+begin_src csv
+,
+#AUDIOSCROBBLER/1.1
+#TZ/UNKNOWN
+#CLIENT/Rockbox sansaclipplus $Revision$
+75 Dollar Bill	I Was Real	I Was Real	4	1015	S	1740494944	64ff5f53-d187-4512-827e-7606c69e66ff
+75 Dollar Bill	I Was Real	I Was Real	4	1015	S	1740494990	64ff5f53-d187-4512-827e-7606c69e66ff
+311	311	Down	1	173	S	1740495003	00476c23-fd9e-464b-9b27-a62d69f3d4f4
+311	311	Down	1	173	L	1740495049	00476c23-fd9e-464b-9b27-a62d69f3d4f4
+311	311	Down	1	173	L	1740495113	00476c23-fd9e-464b-9b27-a62d69f3d4f4
+311	311	Random	2	187	S	1740495190	530c09f3-46fe-4d90-b11f-7b63bcb4b373
+311	311	Random	2	187	L	1740495194	530c09f3-46fe-4d90-b11f-7b63bcb4b373
+311	311	Jackolantern’s Weather	3	204	L	1740495382	cc3b2dec-5d99-47ea-8930-20bf258be4ea
+311	311	All Mixed Up	4	182	L	1740495586	980a78b5-5bdd-4f50-9e3a-e13261e2817b
+311	311	Hive	5	179	L	1740495768	18f6dc98-d3a2-4f81-b967-97359d14c68c
+311	311	Guns (Are for Pussies)	6	137	L	1740495948	5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
+311	311	Misdirected Hostility	7	179	S	1740496085	61ff2c1a-fc9c-44c3-8da1-5e50a44245af
+,
+#+end_src
+** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
+CLOSED: [2023-02-17 Fri 00:10]
+
+This would allow a few nice flows. One, you'd be able to record the play of an
+entire album by just dropping the muscibrainz_id in. This could be helpful for
+offline listening. It would also mean bad metadata from mopidy would not break
+scrobbling.
+** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
+CLOSED: [2023-02-17 Fri 00:11]
+** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
+CLOSED: [2023-03-07 Tue 11:09]
+
+Given a UUID from musicbrainz, we should be able to scrobble an album or
+individual track.
+
+** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
+CLOSED: [2023-03-07 Tue 11:10]
+:LOGBOOK:
+CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] =>  1:30
+:END:
+
+Maloja does this cool thing where artists and tracks get recorded as the top
+track of a given week, month or year. They get gold, silver or bronze stars for
+their place in the time period.
+
+I could see this being implemented as a separate Chart table which gets
+populated at the end of a time period and has a start and end date that defines
+a period, along with a one, two, three instance.
+
+Of course, it could also be a data model without a table, where it runs some fun
+calculations, stores it's values in Redis as a long-term lookup table and just
+has to re-populate when the server restarts.
 * Backlog
 ** TODO [#C] Move to using more robust mopidy-webhooks pacakge form pypi :utility:improvement:
 *** Example payloads from mopidy-webhooks
@@ -296,126 +420,3 @@ A fun way to keep track of things in the project to fix or improve.
 #+end_src
 ** TODO [#C] Consider a purge command for duplicated and stuck in-progress scrobbles :utililty:improvement:
 ** TODO What to do with Youtube videos from LastFM and web-scrobbler :bug:source:lastfm:
-* Version 1.0.0
-** TODO [#A] Add django-storage to store files on S3 :improvement:
-** TODO Add a "finished_timestamp" so we don't rely on content length :improvement:scrobbling:
-
-Essentially, we currently have the timestamp as when the content began
-scrobbling and then calculate the finish time from the length of the content.
-This works pretty well because we know how long most things are.
-
-But in some cases, sports events or long podcasts, we may start mid-way through
-an event or finish halfway through but still want to mark it as done. In these
-cases, knowing the finish time could be useful, especially when interfacing with
-other scrobblers which may have different definitions of when a scrobble
-finishes or started.
-
-** TODO Update weekly live chart to be 7-day continuous rather than weekly :views:bug:
-The live view will be blank every Monday, no reason to tie it to a day of the
-week. It should be "the last 7 days"
-** DONE [#B] Implement a detail view for TV shows :improvement:views:
-CLOSED: [2023-03-22 Wed 17:05]
-** DONE [#B] Implement a detail view for Movies :improvement:views:
-CLOSED: [2023-03-22 Wed 17:05]
-** DONE Add "service provider" to TV Series, and use that for source when available :bug:scrobbling:
-CLOSED: [2023-03-22 Wed 17:04]
-** DONE Add view for long-play content (books, video games) to restart them :views:improvement:
-CLOSED: [2023-03-22 Wed 17:01]
-** DONE Add live chart view like Maloja :improvement:views:
-CLOSED: [2023-03-07 Tue 11:13]
-** DONE [#C] Figure out how to add to web-scrobbler :improvement:scrobbling:
-CLOSED: [2023-03-22 Wed 17:06]
-
-An example:
-https://github.com/web-scrobbler/web-scrobbler/blob/master/src/core/background/scrobbler/maloja-scrobbler.js
-
-This is actually going to be moot because we can import from LastFM, and
-web-scrobbler integrates well with LastFM. The only thing to think through here
-now is what to do with all the garbage web-scrobbler sometimes pushes to LastFM
-from Youtube (all videos get pushed, sigh).
-
-* Version 0.11.4
-** DONE Add rudimentary video game scrobbling :improvement:content:videogames:
-CLOSED: [2023-03-07 Tue 11:11]
-** DONE Add ability to scrobble from KOReader statistics files :improvement:books:content:
-CLOSED: [2023-03-07 Tue 11:11]
-
-** DONE [#A] Fix fetching artwork without release group :bug:
-CLOSED: [2023-01-29 Sun 14:27]
-
-When we get artwork from Musicbrianz, and it's not found, we should check for
-release groups as well. This will stop issues with missing artwork because of
-obscure MB release matches.
-
-** DONE [#A] Fix Jellyfin music scrobbling N+1 past 90 completion percent :bug:
-CLOSED: [2023-01-30 Mon 18:31]
-:LOGBOOK:
-CLOCK: [2023-01-30 Mon 18:00]--[2023-01-30 Mon 18:31] =>  0:31
-:END:
-
-If we play music from Jellyfin and the track reaches 90% completion, the
-scrobbling goes crazy and starts creating new scrobbles with every update.
-
-The cause is pretty simple, but the solution is hard. We want to mark a scrobble
-as complete for the following conditions:
-
-- Play stopped and percent played beyond 90%
-- Play completely finished
-
-But if we keep listening beyond 90, we should basically ignore updates (or just
-update the existing scrobble)
-** DONE [#A] Add support for Audioscrobbler tab-separated file uploads :improvement:
-CLOSED: [2023-02-03 Fri 16:52]
-
-An example of the format:
-#+begin_src csv
-,
-#AUDIOSCROBBLER/1.1
-#TZ/UNKNOWN
-#CLIENT/Rockbox sansaclipplus $Revision$
-75 Dollar Bill	I Was Real	I Was Real	4	1015	S	1740494944	64ff5f53-d187-4512-827e-7606c69e66ff
-75 Dollar Bill	I Was Real	I Was Real	4	1015	S	1740494990	64ff5f53-d187-4512-827e-7606c69e66ff
-311	311	Down	1	173	S	1740495003	00476c23-fd9e-464b-9b27-a62d69f3d4f4
-311	311	Down	1	173	L	1740495049	00476c23-fd9e-464b-9b27-a62d69f3d4f4
-311	311	Down	1	173	L	1740495113	00476c23-fd9e-464b-9b27-a62d69f3d4f4
-311	311	Random	2	187	S	1740495190	530c09f3-46fe-4d90-b11f-7b63bcb4b373
-311	311	Random	2	187	L	1740495194	530c09f3-46fe-4d90-b11f-7b63bcb4b373
-311	311	Jackolantern’s Weather	3	204	L	1740495382	cc3b2dec-5d99-47ea-8930-20bf258be4ea
-311	311	All Mixed Up	4	182	L	1740495586	980a78b5-5bdd-4f50-9e3a-e13261e2817b
-311	311	Hive	5	179	L	1740495768	18f6dc98-d3a2-4f81-b967-97359d14c68c
-311	311	Guns (Are for Pussies)	6	137	L	1740495948	5e97ed9f-c8cc-4282-9cbe-f8e17aee5128
-311	311	Misdirected Hostility	7	179	S	1740496085	61ff2c1a-fc9c-44c3-8da1-5e50a44245af
-,
-#+end_src
-** DONE [#B] Allow scrobbling music without MB IDs by grabbing them before scrobble :improvement:
-CLOSED: [2023-02-17 Fri 00:10]
-
-This would allow a few nice flows. One, you'd be able to record the play of an
-entire album by just dropping the muscibrainz_id in. This could be helpful for
-offline listening. It would also mean bad metadata from mopidy would not break
-scrobbling.
-** DONE When updating musicbrainz IDs, clear and run fetch artwrok :improvement:
-CLOSED: [2023-02-17 Fri 00:11]
-** DONE [#A] Add ability to manually scrobble albums or tracks from MB :improvement:
-CLOSED: [2023-03-07 Tue 11:09]
-
-Given a UUID from musicbrainz, we should be able to scrobble an album or
-individual track.
-
-** DONE [#C] Implement keeping track of week/month/year chart-toppers :improvement:
-CLOSED: [2023-03-07 Tue 11:10]
-:LOGBOOK:
-CLOCK: [2023-01-30 Mon 16:30]--[2023-01-30 Mon 18:00] =>  1:30
-:END:
-
-Maloja does this cool thing where artists and tracks get recorded as the top
-track of a given week, month or year. They get gold, silver or bronze stars for
-their place in the time period.
-
-I could see this being implemented as a separate Chart table which gets
-populated at the end of a time period and has a start and end date that defines
-a period, along with a one, two, three instance.
-
-Of course, it could also be a data model without a table, where it runs some fun
-calculations, stores it's values in Redis as a long-term lookup table and just
-has to re-populate when the server restarts.

vrobbler/apps/music/aggregators.py

@@ -95,6 +95,8 @@ def live_charts(
         now = now_user_timezone(user.profile)
         tzinfo = now.tzinfo
 
+    seven_days_ago = now - timedelta(days=7)
+    thirty_days_ago = now - timedelta(days=30)
     start_of_today = datetime.combine(now, datetime.min.time(), tzinfo)
     start_day_of_week = now - timedelta(days=now.today().isoweekday() % 7)
     start_day_of_month = now.replace(day=1)
@@ -105,6 +107,8 @@ def live_charts(
     period_queries = {
         "today": {"scrobble__timestamp__gte": start_of_today},
         "week": {"scrobble__timestamp__gte": start_day_of_week},
+        "last7": {"scrobble__timestamp__gte": seven_days_ago},
+        "last30": {"scrobble__timestamp__gte": thirty_days_ago},
         "month": {"scrobble__timestamp__gte": start_day_of_month},
         "year": {"scrobble__timestamp__gte": start_day_of_year},
         "all": {},

vrobbler/apps/scrobbles/views.py

@@ -102,10 +102,17 @@ class RecentScrobbleList(ListView):
             limit = 14
             artist = {"user": user, "media_type": "Artist", "limit": limit}
             # This is weird. They don't display properly as QuerySets, so we cast to lists
+            data["chart_keys"] = {
+                "today": "Today",
+                "last7": "Last 7 days",
+                "last30": "Last 30 days",
+                "year": "This year",
+                "all": "All time",
+            }
             data["current_artist_charts"] = {
                 "today": list(live_charts(**artist, chart_period="today")),
-                "week": list(live_charts(**artist, chart_period="week")),
+                "last7": list(live_charts(**artist, chart_period="last7")),
-                "month": list(live_charts(**artist, chart_period="month")),
+                "last30": list(live_charts(**artist, chart_period="last30")),
                 "year": list(live_charts(**artist, chart_period="year")),
                 "all": list(live_charts(**artist)),
             }
@@ -113,8 +120,8 @@ class RecentScrobbleList(ListView):
             track = {"user": user, "media_type": "Track", "limit": limit}
             data["current_track_charts"] = {
                 "today": list(live_charts(**track, chart_period="today")),
-                "week": list(live_charts(**track, chart_period="week")),
+                "last7": list(live_charts(**track, chart_period="last7")),
-                "month": list(live_charts(**track, chart_period="month")),
+                "last30": list(live_charts(**track, chart_period="last30")),
                 "year": list(live_charts(**track, chart_period="year")),
                 "all": list(live_charts(**track)),
             }

vrobbler/templates/scrobbles/scrobble_list.html

@@ -89,20 +89,18 @@
         <div class="row">
             <h2>Top Artist</h2>
             <ul class="nav nav-tabs" id="artistTab" role="tablist">
-                {% for chart_name in current_artist_charts.keys %}
+                {% for key, name in chart_keys.items %}
                 <li class="nav-item" role="presentation">
                     <button class="nav-link {% if forloop.counter == 2 %}active{% endif %}"
-                            id="artist-{{chart_name}}-tab" data-bs-toggle="tab" data-bs-target="#artist-{{chart_name}}"
+                            id="artist-{{key}}-tab" data-bs-toggle="tab" data-bs-target="#artist-{{key}}"
-                            type="button" role="tab" aria-controls="home" aria-selected="true">
+                            type="button" role="tab" aria-controls="home" aria-selected="true">{{name}}</button>
-                        {% if chart_name == "all" %}All Time{% else %}{% if chart_name != "today" %}This {% endif %}{{chart_name|capfirst}}{% endif %}
-                    </button>
                 </li>
                 {% endfor %}
             </ul>
 
             <div class="tab-content" id="artistTabContent" class="maloja-chart">
-                {% for chart_name, artists in current_artist_charts.items %}
+                {% for key, artists in current_artist_charts.items %}
-                <div class="tab-pane fade {% if forloop.counter == 2 %}show active{% endif %}" id="artist-{{chart_name}}" role="tabpanel" aria-labelledby="artist-{{chart_name}}-tab">
+                <div class="tab-pane fade {% if forloop.counter == 2 %}show active{% endif %}" id="artist-{{key}}" role="tabpanel" aria-labelledby="artist-{{key}}-tab">
                     <div style="display:block">
                         <div style="float:left;">
                             <div class="image-wrapper" style="display:flex; flex-wrap: wrap; margin:0">
@@ -263,12 +261,10 @@
         <div class="row">
             <h2>Top Tracks</h2>
             <ul class="nav nav-tabs" id="trackTab" role="tablist">
-                {% for chart_name in current_track_charts.keys %}
+                {% for key, name in chart_keys.items %}
                 <li class="nav-item" role="presentation">
-                    <button class="nav-link {% if forloop.counter == 2 %}active{% endif %}" id="track-{{chart_name}}-tab" data-bs-toggle="tab"
+                    <button class="nav-link {% if forloop.counter == 2 %}active{% endif %}" id="track-{{key}}-tab" data-bs-toggle="tab"
-                            data-bs-target="#track-{{chart_name}}" type="button" role="tab" aria-controls="home" aria-selected="true">
+                            data-bs-target="#track-{{key}}" type="button" role="tab" aria-controls="home" aria-selected="true">{{name}}</button>
-                        {% if chart_name == "all" %}All Time{% else %}{% if chart_name != "today" %}This {% endif %}{{chart_name|capfirst}}{% endif %}
-                    </button>
                 </li>
                 {% endfor %}
             </ul>