[podcasts] Fix podcast data

PROJECT.org

@@ -1,6 +1,20 @@
 #+title: Vrobbler Project
 
 * Overview
+Vrobbler began humbly enough as a way to use Jellyfin's webhook to keep track of
+the shows and movies I was watching. More specifically, I broke my ankle a few
+days after Christmas in 2022 and spent the next four months very slowly
+recovering after surgical repair. So once I had the webhook working, and
+scrobbling videos, it was only a matter of time till I expaned it to mopidy to
+replicate LastFM. Then I added board games, books via KoReader, sports events,
+podcasts ... it just keeps going. Vrobbler is now a sort of Frankenstein's
+monster of scrobbling an entire life.
+
+I am still unconvinced I can keep this going, but being able to scrobble org
+tasks, Todoist tasks, web pages I've read and trails I've hiked has turned out
+to be sometimes cathartic and sometimes functional as I try to remember when I
+did a thing.
+
 * Features
 ** Beer
 *** Triggers
@@ -70,7 +84,6 @@ fetching and simple saving.
 **** Bookmarklet
 *** Metadata sources
 **** Scraper
-* Release history
 * Chores
 ** DONE Document various vrobbler features :chore:personal:project:vrobbler:documentation:
 :PROPERTIES:
@@ -79,71 +92,8 @@ fetching and simple saving.
 :LOGBOOK:
 CLOCK: [2025-07-09 Wed 09:55]--[2025-07-09 Wed 10:15] =>  0:20
 :END:
-* Backlog [3/23]
-** TODO [#A] Add classmethod for metadata fetching to tracks :vrobbler:feature:music:personal:project:
-:PROPERTIES:
-:ID:       bc4b45e5-4c65-13c5-ab7b-1937d3fbf5c2
-:END:
-:LOGBOOK:
-CLOCK: [2025-07-09 Wed 10:15]
-:END:
-
-** TODO Add importer class for IMAP imports :vrobbler:feature:imap:importers:project:personal:
-** TODO Add youtube link in place of IMDB on video detail page :vrobbler:feature:videos:personal:project:
-** TODO [#A] Tasks from org-mode should properly update notes and leave them out of the body :vrobbler:bug:tasks:
-** TODO [#A] Fix small bug in views for TV series where next episode is now None :vrobbler:bug:personal:videos:
-
-#+begin_src python
-ERROR    django.request:241 log_response Internal Server Error: /series/c24100d1-da45-4abe-86bf-27cfce9b1f89/
-Traceback (most recent call last):
-  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
-    response = get_response(request)
-               ^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
-    response = wrapped_callback(request, *callback_args, **callback_kwargs)
-               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
-    return self.dispatch(request, *args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/mixins.py", line 73, in dispatch
-    return super().dispatch(request, *args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
-    return handler(request, *args, **kwargs)
-           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/django/views/generic/detail.py", line 109, in get
-    context = self.get_context_data(object=self.object)
-              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-  File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/videos/views.py", line 33, in get_context_data
-    context_data["next_episode_id"] = "tt" + next_episode_id
-                                      ~~~~~^~~~~~~~~~~~~~~~~
-TypeError: can only concatenate str (not "NoneType") to str
-#+end_src
-
-** TODO [#A] Send periodic check notifications for board games and beer that ask if you're still scrobbling :vrobbler:personal:project:beers:boardgames:notifications:feature:
-** TODO [#A] Fix raw text webpage title not truncating to 254 chars :vrobbler:personal:bug:webpages:
-** TODO [#A] Fix koreader scrobble imports to use DST properly :vrobbler:personal:bug:books:imports:
-The page data has the canonical date something was read in it, but it seems to be an hour off. I traced this back to being off during DST, so we just need the importer to be aware of whether a user is using DST or not and roll back an hour for part of the year.
-
-Also, we'd need to adjust any old scrobbles that took place with DST off to roll them back by an hour.
-** TODO [#A] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
-** TODO [#B] Add AllTrails as a source for Trail data :vrobbler:trails:feature:personal:project:
-Pretty clear, I would love to make trails more useful. Historically I wasn't
-hiking a lot, which made the source for this a bit silly. But it's clear that
-AllTrails is the best source, though having TrailForks is nice to.
-** TODO [#B] Add `garmin_activity_id` to the TrailMetadataLog class :vrobbler:trails:feature:personal:project:
-Would be nice to have some loose connection to the actual event in my Garmin profile.
-** TODO [#B] Explore a way to add metadata editing to scrobbles after saving :vrobbler:spike:scrobbling:personal:project:
-Could be as simple as a JSON form on the scrobble detail page (do I have have one of those yet?).
-** TODO [#B] Explore a good way to show notes and descriptions from scrobbles to users :personal:project:scrobbling:vrobbler:spike:
-** TODO [#B] Add webdav syncing to retroarch imports :vrobbler:videogames:webdav:feature:project:personal:
-** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
-https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
-** TODO [#C] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal:
-Rather than pick up an existing Podcast using the podcast title in the mopidy
-file name, Vrobbler creates a new podcast with no enriched data. Not a big deal
-for my use as the volume of podcasts I listen to makes manual fixes easy. But
-it's annoying.
+* Backlog [3/27]
+** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
 ** TODO [#C] Move to using more robust mopidy-webhooks pacakge form pypi :utility:improvement:
 :PROPERTIES:
 :ID:       ab31fdc3-359c-1b1d-6b9d-546b476021ba
@@ -442,6 +392,133 @@ it's annoying.
 ** TODO [#C] Allow users to see tasks on calendar view :vrobbler:personal:project:templates:feature:
 https://codepen.io/oliviale/pen/QYqybo
 ** TODO [#C] Come up with a possible flow using WebDAV and super-productivity for tasks :personal:feature:project:vrobbler:tasks:
+** TODO [#B] Add importer class for IMAP imports :vrobbler:feature:imap:importers:project:personal:
+** TODO [#B] Clean up follow up notifications for board games and beer that ask if you're still scrobbling :vrobbler:personal:project:beers:boardgames:notifications:feature:
+
+- Note taken on [2025-09-30 Tue 09:32]
+
+  I added this feature in a very rough way, but now we should add "Action"
+  headers so that we can either Finish or Cancel the associated scrobble:
+
+  https://docs.ntfy.sh/publish/#send-http-request
+
+** TODO [#B] Add AllTrails as a source for Trail data :vrobbler:trails:feature:personal:project:
+Pretty clear, I would love to make trails more useful. Historically I wasn't
+hiking a lot, which made the source for this a bit silly. But it's clear that
+AllTrails is the best source, though having TrailForks is nice to.
+** TODO [#B] Add `garmin_activity_id` to the TrailMetadataLog class :vrobbler:trails:feature:personal:project:
+Would be nice to have some loose connection to the actual event in my Garmin profile.
+** TODO [#B] Explore a way to add metadata editing to scrobbles after saving :vrobbler:spike:scrobbling:personal:project:
+Could be as simple as a JSON form on the scrobble detail page (do I have have one of those yet?).
+** TODO [#B] Explore a good way to show notes and descriptions from scrobbles to users :personal:project:scrobbling:vrobbler:spike:
+** TODO [#B] Add webdav syncing to retroarch imports :vrobbler:videogames:webdav:feature:project:personal:
+** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
+https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
+** TODO [#B] Add youtube link in place of IMDB on video detail page :vrobbler:feature:videos:personal:project:
+** TODO [#B] Fix PuzzleLogData has no attribute form :vrobbler:puzzles:personal:project:logdata:
+** TODO [#B] Scrape ComicBookRoundUp ratings for comic book metadata :vrobbler:books:feature:comicbook:personal:project:
+
+- Note taken on [2025-09-25 Thu 10:51]
+
+  As an example https://comicbookroundup.com/comic-books/reviews/humanoids-publishing/the-history-of-science-fiction
+** TODO [#B] Add PuzzleLogData class with with_people and completed :vrobbler:feature:puzzles:logdata:personal:project:
+** TODO [#A] Add classmethod for metadata fetching to tracks :vrobbler:feature:music:personal:project:
+:PROPERTIES:
+:ID:       bc4b45e5-4c65-13c5-ab7b-1937d3fbf5c2
+:END:
+
+** TODO [#A] Tasks from org-mode should properly update notes and leave them out of the body :vrobbler:bug:tasks:
+** TODO [#A] Fix views for TV series where next episode is now None :vrobbler:bug:personal:videos:
+
+#+begin_src python
+ERROR    django.request:241 log_response Internal Server Error: /series/c24100d1-da45-4abe-86bf-27cfce9b1f89/
+Traceback (most recent call last):
+  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
+    response = get_response(request)
+               ^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
+    response = wrapped_callback(request, *callback_args, **callback_kwargs)
+               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
+    return self.dispatch(request, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/mixins.py", line 73, in dispatch
+    return super().dispatch(request, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
+    return handler(request, *args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/django/views/generic/detail.py", line 109, in get
+    context = self.get_context_data(object=self.object)
+              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/local/lib/python3.11/site-packages/vrobbler/apps/videos/views.py", line 33, in get_context_data
+    context_data["next_episode_id"] = "tt" + next_episode_id
+                                      ~~~~~^~~~~~~~~~~~~~~~~
+TypeError: can only concatenate str (not "NoneType") to str
+#+end_src
+
+** TODO [#A] Fix koreader scrobble imports to use DST properly :vrobbler:personal:bug:books:imports:
+
+- Note taken on [2025-09-25 Thu 10:37] \\
+
+  This may already be fixed ... need to check.
+
+- Note taken on [2025-02-25 12:34] \\
+
+  The page data has the canonical date something was read in it, but it seems
+  to be an hour off. I traced this back to being off during DST, so we just need
+  the importer to be aware of whether a user is using DST or not and roll back
+  an hour for part of the year. Also, we'd need to adjust any old scrobbles that
+  took place with DST off to roll them back by an hour.
+
+** TODO [#A] Allow scrobbling from the Food list page's start links :vrobbler:bug:food:scrobbling:personal:project:
+https://life.lab.unbl.ink/scrobble/e39779c8-62a5-46a6-bdef-fb7662810dc6/start/
+** TODO [#A] Puzzles (and all longplays) should have a "Completed?" column on their detail page :vrobbler:bug:puzzles:personal:project:
+** TODO [#A] Fix raw text webpage title not truncating to 254 chars :vrobbler:personal:bug:webpages:
+
+- Note taken on [2025-09-30 Tue 09:33]
+
+  This may have already been resolved ... need to just confirm it.
+* Version 27.0 [3/3]
+** DONE [#A] Fix bug where podcast scrobbling creates duplicate Podcast :project:vrobbler:scrobbling:podcasts:bug:personal:
+:PROPERTIES:
+:ID:       7377ef6c-5fa7-9e4e-9080-f9810a76118c
+:END:
+
+Rather than pick up an existing Podcast using the podcast title in the mopidy
+file name, Vrobbler creates a new podcast with no enriched data. Not a big deal
+for my use as the volume of podcasts I listen to makes manual fixes easy. But
+it's annoying.
+
+** DONE [#A] Allow reading comic books from readcomicsoline.ru :vrobbler:books:feature:comicbook:personal:project:scrobbling:
+:PROPERTIES:
+:ID:       7c7e9ecc-b675-68c3-764f-ef771ce5d88f
+:END:
+
+- Note taken on [2025-09-25 Thu 10:52]
+
+  Things to consider are whether we scrobble the issue on one page, send it to
+  archivebox? (yes), and how best to enrich the data
+
+** DONE [#A] Add RSS feed lookups to podcasts :vrobbler:personal:feature:podcasts:
+:PROPERTIES:
+:ID:       d60645b0-7578-97c1-0278-05bd9de4269c
+:END:
+
+- Note taken on [2025-10-14 Tue 10:08]
+
+  Turns out the Podcast plugin for mopidy does a pretty good job of showing the
+  latest file without having to scroll the bottom using only Muse to not parse
+  the podcast title name. BUT, now we're getting urls like this:
+
+  https://nsf.libsyn.com/rss#77e01251-cb20-4609-b577-d48e985d2e7b
+
+  This is great, because there's more context there, but it has to read out of
+  the RSS feed. We should add a check in the podcast util to sniff out the file
+  referenced in the # in that url and populate the info from there. This should
+  actually be much more reliable than the current state of the podcast lookup
+  which depends on the file to be name properly.
+
 * Version 26.0 [3/3]
 ** DONE Clean up templates for scrobble details :vrobbler:personal:bug:templates:
 :PROPERTIES:

poetry.lock

@@ -1602,6 +1602,21 @@ files = [
 [package.extras]
 devel = ["colorama", "json-spec", "jsonschema", "pylint", "pytest", "pytest-benchmark", "pytest-cache", "validictory"]
 
+[[package]]
+name = "feedparser"
+version = "6.0.12"
+description = "Universal feed parser, handles RSS 0.9x, RSS 1.0, RSS 2.0, CDF, Atom 0.3, and Atom 1.0 feeds"
+optional = false
+python-versions = ">=3.6"
+groups = ["main"]
+files = [
+    {file = "feedparser-6.0.12-py3-none-any.whl", hash = "sha256:6bbff10f5a52662c00a2e3f86a38928c37c48f77b3c511aedcd51de933549324"},
+    {file = "feedparser-6.0.12.tar.gz", hash = "sha256:64f76ce90ae3e8ef5d1ede0f8d3b50ce26bcce71dd8ae5e82b1cd2d4a5f94228"},
+]
+
+[package.dependencies]
+sgmllib3k = "*"
+
 [[package]]
 name = "filelock"
 version = "3.18.0"
@@ -4403,6 +4418,17 @@ enabler = ["pytest-enabler (>=2.2)"]
 test = ["build[virtualenv] (>=1.0.3)", "filelock (>=3.4.0)", "ini2toml[lite] (>=0.14)", "jaraco.develop (>=7.21) ; python_version >= \"3.9\" and sys_platform != \"cygwin\"", "jaraco.envs (>=2.2)", "jaraco.path (>=3.7.2)", "jaraco.test (>=5.5)", "packaging (>=24.2)", "pip (>=19.1)", "pyproject-hooks (!=1.1)", "pytest (>=6,!=8.1.*)", "pytest-home (>=0.5)", "pytest-perf ; sys_platform != \"cygwin\"", "pytest-subprocess", "pytest-timeout", "pytest-xdist (>=3)", "tomli-w (>=1.0.0)", "virtualenv (>=13.0.0)", "wheel (>=0.44.0)"]
 type = ["importlib_metadata (>=7.0.2) ; python_version < \"3.10\"", "jaraco.develop (>=7.21) ; sys_platform != \"cygwin\"", "mypy (==1.14.*)", "pytest-mypy"]
 
+[[package]]
+name = "sgmllib3k"
+version = "1.0.0"
+description = "Py3k port of sgmllib."
+optional = false
+python-versions = "*"
+groups = ["main"]
+files = [
+    {file = "sgmllib3k-1.0.0.tar.gz", hash = "sha256:7868fb1c8bfa764c1ac563d3cf369c381d1325d36124933a726f29fcdaa812e9"},
+]
+
 [[package]]
 name = "shellingham"
 version = "1.5.4"
@@ -5499,4 +5525,4 @@ cffi = ["cffi (>=1.11)"]
 [metadata]
 lock-version = "2.1"
 python-versions = ">=3.9,<3.12"
-content-hash = "3a483aefea0a3afebf187b17b7df72a158788024ca8121b512b39567fb5ec8ca"
+content-hash = "cd3b566597e09aa444f9af30f95f94f922bf3dca71fbd05c887fb10cbc11d7bf"

pyproject.toml

@@ -56,6 +56,7 @@ poetry-bumpversion = "^0.3.3"
 orgparse = "^0.4.20250520"
 tmdbv3api = "^1.9.0"
 themoviedb = "^1.0.2"
+feedparser = "^6.0.12"
 
 [tool.poetry.group.test]
 optional = true

tests/scrobbles_tests/conftest.py

@@ -115,6 +115,12 @@ def mopidy_podcast_request_data():
         mopidy_uri=mopidy_uri, artist="NPR", album="Up First"
     ).request_json
 
+@pytest.fixture
+def mopidy_podcast_https_request_data():
+    mopidy_uri = "podcast+https://feeds.npr.org/510318/podcast.xml#85b9c4c4-ae09-43d9-8853-31ccf43f68e6"
+    return MopidyRequest(
+        mopidy_uri=mopidy_uri, artist="NPR", album="Up First"
+    ).request_json
 
 class JellyfinTrackRequest:
     name = "Emotion"

vrobbler/apps/podcasts/models.py

@@ -143,43 +143,46 @@ class PodcastEpisode(ScrobblableMixin):
     def find_or_create(
         cls,
         title: str,
-        podcast_name: str,
         pub_date: str,
-        number: int = 0,
-        mopidy_uri: str = "",
-        producer_name: str = "",
+        episode_num: int = 0,
         run_time_seconds: int = 1800,
+        mopidy_uri: str = "",
+        podcast_name: str = "",
+        podcast_producer: str = "",
+        podcast_description: str = "",
         enrich: bool = True,
     ) -> "PodcastEpisode":
         """Given a data dict from Mopidy, finds or creates a podcast and
         producer before saving the epsiode so it can be scrobbled.
 
         """
+        log_context={"mopidy_uri": mopidy_uri, "media_type": "Podcast"}
         producer = None
-        if producer_name:
-            producer = Producer.find_or_create(producer_name)
+        if podcast_producer:
+            producer = Producer.find_or_create(podcast_producer)
 
-        podcast = Podcast.objects.filter(
-            name__iexact=podcast_name,
-        ).first()
-        if not podcast:
-            podcast = Podcast.objects.create(
-                name=podcast_name, producer=producer
-            )
-            if enrich:
-                podcast.fix_metadata()
+        podcast, created = Podcast.objects.get_or_create(name=podcast_name, defaults={"description": podcast_description})
+        log_context["podcast_id"] = podcast.id
+        log_context["podcast_name"] = podcast.name
+        if created:
+            logger.info("Created new podcast", extra=log_context)
+        if enrich and created:
+            logger.info("Enriching new podcast", extra=log_context)
+            podcast.fix_metadata()
 
-        episode = cls.objects.filter(
-            title__iexact=title, podcast=podcast
-        ).first()
-        if not episode:
-            episode = cls.objects.create(
-                title=title,
-                podcast=podcast,
-                run_time_seconds=run_time_seconds,
-                number=number,
-                pub_date=pub_date,
-                mopidy_uri=mopidy_uri,
-            )
+        episode, created = cls.objects.get_or_create(
+            title=title,
+            podcast=podcast,
+            defaults={
+                "run_time_seconds": run_time_seconds,
+                "number": episode_num,
+                "pub_date": pub_date,
+                "mopidy_uri": mopidy_uri,
+            }
+        )
+        if created:
+            log_context["episode_id"] = episode.id
+            log_context["episode_title"] = episode.title
+            logger.info("Created new podcast episode", extra=log_context)
 
         return episode

vrobbler/apps/podcasts/utils.py

@@ -1,7 +1,9 @@
 import logging
 import os
+from typing import Any
 from urllib.parse import unquote
 
+import feedparser
 from dateutil.parser import ParserError, parse
 from podcasts.models import PodcastEpisode
 
@@ -10,26 +12,85 @@ logger = logging.getLogger(__name__)
 # TODO This should be configurable in settings or per deploy
 PODCAST_DATE_FORMAT = "YYYY-MM-DD"
 
+def parse_duration(d):
+    if not d:
+        return None
+    if d.isdigit():
+        return int(d)
+    parts = [int(p) for p in d.split(":")]
+    while len(parts) < 3:
+        parts.insert(0, 0)
+    h, m, s = parts
+    return h * 3600 + m * 60 + s
+
+def fetch_metadata_from_rss(uri: str) -> dict[str, Any]:
+    log_context = {"mopidy_uri": uri, "media_type": "Podcast"}
+    podcast_data: dict[str, Any] = {}
+
+    try:
+        feed = feedparser.parse(uri.split("#")[0])
+        target_guid = uri.split("#")[1]
+    except IndexError:
+        logger.warning("Tried to parse uri as RSS feed, but no target found", extra=log_context)
+        return podcast_data
+
+    podcast_publisher = feed.feed.get("itunes_publisher")
+    podcast_owner = feed.feed.itunes_owner.get("name") if isinstance(feed.feed.itunes_owner, dict) else feed.feed.itunes_owner
+    podcast_other = feed.feed.get("managingeditor") or feed.feed.get("copyright")
+
+    podcast_data = {
+        "podcast_name": feed.feed.get("title", "Unknown Podcast"),
+        # "podcast_description": feed.feed.get("description", ""),
+        # "podcast_link": feed.feed.get("link", ""),
+        "podcast_producer": podcast_publisher or podcast_owner or podcast_other
+    }
+
+    for entry in feed.entries:
+        if target_guid in target_guid:
+            logger.info("🎧 Episode found in RSS feed", extra=log_context)
+            podcast_data["title"] = entry.title
+            podcast_data["episode_num"] = entry.guid
+            podcast_data["pub_date"] = entry.get("published", None)
+            podcast_data["run_time_seconds"] = parse_duration(entry.get("itunes_duration", None))
+            # podcast_data["description"] = entry.get("description", None)
+            # podcast_data["episode_url"] = entry.enclosures[0].href if entry.get("enclosures") else None
+            return podcast_data
+    else:
+        logger.info("Episode not found in RSS feed.")
+
+
+def parse_mopidy_uri(uri: str) -> dict[str, Any]:
+    podcast_data: dict[str, Any] = {}
 
-def parse_mopidy_uri(uri: str) -> dict:
     logger.debug(f"Parsing URI: {uri}")
+    if "https://" in uri:
+        return fetch_metadata_from_rss(uri)
+
+
     parsed_uri = os.path.splitext(unquote(uri))[0].split("/")
 
+    podcast_data = {
+        "title": parsed_uri[-1],
+        "episode_num": None,
+        "podcast_name": parsed_uri[-2].strip(),
+        "pub_date": None,
+    }
+
+
     episode_str = parsed_uri[-1]
-    podcast_name = parsed_uri[-2].strip()
     episode_num = None
     episode_num_pad = 0
 
     try:
         # Without episode numbers the date will lead
-        pub_date = parse(episode_str[0:10])
+        podcast_data["pub_date"] = parse(episode_str[0:10])
     except ParserError:
-        episode_num = int(episode_str.split("-")[0])
-        episode_num_pad = len(str(episode_num)) + 1
+        podcast_data["episode_num"] = int(episode_str.split("-")[0])
+        episode_num_pad = len(str(podcast_data["episode_num"])) + 1
 
         try:
             # Beacuse we have epsiode numbers on
-            pub_date = parse(
+            podcast_data["pub_date"] = parse(
                 episode_str[
                     episode_num_pad : len(PODCAST_DATE_FORMAT)
                     + episode_num_pad
@@ -39,41 +100,19 @@ def parse_mopidy_uri(uri: str) -> dict:
             pub_date = ""
 
     gap_to_strip = 0
-    if pub_date:
+    if podcast_data["pub_date"]:
         gap_to_strip += len(PODCAST_DATE_FORMAT)
-    if episode_num:
+    if podcast_data["episode_num"]:
         gap_to_strip += episode_num_pad
 
-    episode_name = episode_str[gap_to_strip:].replace("-", " ").strip()
+    podcast_data["title"] = episode_str[gap_to_strip:].replace("-", " ").strip()
 
-    return {
-        "episode_filename": episode_name,
-        "episode_num": episode_num,
-        "podcast_name": podcast_name,
-        "pub_date": pub_date,
-    }
+    return podcast_data
 
 
 def get_or_create_podcast(post_data: dict) -> PodcastEpisode:
+    logger.info("Looking up podcast", extra={"post_data": post_data, "media_type": "Podcast"})
     mopidy_uri = post_data.get("mopidy_uri", "")
     parsed_data = parse_mopidy_uri(mopidy_uri)
 
-    producer_dict = {"name": post_data.get("artist")}
-
-    podcast_name = post_data.get("album")
-    if not podcast_name:
-        podcast_name = parsed_data.get("podcast_name")
-    podcast_dict = {"name": podcast_name}
-
-    episode_name = parsed_data.get("episode_filename")
-    episode_dict = {
-        "title": episode_name,
-        "run_time_seconds": post_data.get("run_time"),
-        "number": parsed_data.get("episode_num"),
-        "pub_date": parsed_data.get("pub_date"),
-        "mopidy_uri": mopidy_uri,
-    }
-
-    return PodcastEpisode.find_or_create(
-        podcast_dict, producer_dict, episode_dict
-    )
+    return PodcastEpisode.find_or_create(**parsed_data)

vrobbler/apps/scrobbles/constants.py

@@ -18,6 +18,8 @@ PLAY_AGAIN_MEDIA = {
     "bricksets": "BrickSet",
     "trails": "Trail",
     "beers": "Beer",
+    "foods": "Food",
+    "locations": "GeoLocation",
 }
 
 MEDIA_END_PADDING_SECONDS = {
@@ -35,6 +37,7 @@ SCROBBLE_CONTENT_URLS = {
     "-t": ["https://app.todoist.com/app/task/{id}"],
     "-p": ["https://www.ipdb.plus/IPDb/puzzle.php?id="],
     "-l": ["https://brickset.com/sets/"],
+    "-c": ["https://readcomicsonline.ru"],
 }
 
 EXCLUDE_FROM_NOW_PLAYING = ("GeoLocation",)
@@ -50,6 +53,7 @@ MANUAL_SCROBBLE_FNS = {
     "-t": "manual_scrobble_task",
     "-p": "manual_scrobble_puzzle",
     "-l": "manual_scrobble_brickset",
+    "-c": "manual_scrobble_book",
 }
 
 

vrobbler/apps/scrobbles/scrobblers.py

@@ -56,18 +56,11 @@ def mopidy_scrobble_media(post_data: dict, user_id: int) -> Scrobble:
 
     if media_type == Scrobble.MediaType.PODCAST_EPISODE:
         parsed_data = parse_mopidy_uri(post_data.get("mopidy_uri", ""))
-        podcast_name = post_data.get(
-            "album", parsed_data.get("podcast_name", "")
-        )
+        if not parsed_data:
+            logger.warning("Tried to scrobble podcast but no uri found", extra={"post_data": post_data})
+            return Scrobble()
 
-        media_obj = PodcastEpisode.find_or_create(
-            title=parsed_data.get("episode_filename", ""),
-            podcast_name=podcast_name,
-            producer_name=post_data.get("artist", ""),
-            number=parsed_data.get("episode_num", ""),
-            pub_date=parsed_data.get("pub_date", ""),
-            mopidy_uri=post_data.get("mopidy_uri", ""),
-        )
+        media_obj = PodcastEpisode.find_or_create(**parsed_data)
     else:
         media_obj = Track.find_or_create(
             title=post_data.get("name", ""),
@@ -875,8 +868,13 @@ def manual_scrobble_webpage(
     )
 
     scrobble = Scrobble.create_or_update(webpage, user_id, scrobble_dict)
-    # possibly async this?
-    scrobble.push_to_archivebox()
+
+    if action == "stop":
+        scrobble.stop(force_finish=True)
+    else:
+        # possibly async this?
+        scrobble.push_to_archivebox()
+
     return scrobble
 
 

vrobbler/apps/scrobbles/utils.py

@@ -1,15 +1,18 @@
 import hashlib
 import logging
 import re
-from datetime import datetime, timedelta
+from datetime import date, datetime, timedelta
 from typing import TYPE_CHECKING, Optional
 from urllib.parse import urlparse
 from zoneinfo import ZoneInfo
 
+import pendulum
 import pytz
 from django.apps import apps
 from django.contrib.auth import get_user_model
 from django.db import models
+from django.db.models.fields.json import KeyTextTransform
+from django.db.models.functions import Cast, TruncDate
 from django.utils import timezone
 from profiles.models import UserProfile
 from profiles.utils import now_user_timezone
@@ -371,3 +374,37 @@ def fix_playback_position_seconds(*scrobbles: "Scrobble", commit=True) -> list["
                     scrobble.save(update_fields=["playback_position_seconds"])
 
     return updated_scrobbles
+
+
+def base_scrobble_qs(user_id: int) -> models.QuerySet:
+    """Base queryset with calories extracted as integer and day annotated."""
+    from scrobbles.models import Scrobble
+
+    return (
+        Scrobble.objects
+        .annotate(day=TruncDate("timestamp"))
+        .annotate(calories_int=Cast(KeyTextTransform("calories", "log"), models.IntegerField()))
+        .filter(calories_int__isnull=False, user_id=user_id)
+    )
+
+def get_daily_calories_for_user_by_day(user_id: int, date: date| str) -> int:
+    """Return total calories for a user on a specific day."""
+
+    if isinstance(date, str):
+        date = pendulum.parse(date)
+
+    qs = base_scrobble_qs(user_id).filter(day=date)
+    agg = qs.aggregate(total_calories=models.Sum("calories_int"))
+
+    return agg["total_calories"] or 0
+
+def get_daily_calorie_dict_for_user(user_id: int) -> dict[date, int]:
+    """Return {day: total_calories} for all days with scrobbles, in one query."""
+    qs = (
+        base_scrobble_qs(user_id)
+        .values("day")
+        .annotate(total_calories=models.Sum("calories_int"))
+        .order_by("day")
+    )
+
+    return {entry["day"]: entry["total_calories"] for entry in qs}

vrobbler/apps/scrobbles/views.py

@@ -2,26 +2,27 @@ import calendar
 import json
 import logging
 from datetime import datetime, timedelta
-from dateutil.relativedelta import relativedelta
 
-from django.shortcuts import redirect
 import pendulum
 import pytz
+from dateutil.relativedelta import relativedelta
 from django.apps import apps
 from django.contrib import messages
 from django.contrib.auth.mixins import LoginRequiredMixin
-from django.db.models import Count, Q, Max
+from django.core.paginator import EmptyPage, PageNotAnInteger, Paginator
+from django.db.models import Count, Max, Q
 from django.db.models.query import QuerySet
 from django.http import FileResponse, HttpResponseRedirect, JsonResponse
+from django.shortcuts import redirect
 from django.urls import reverse_lazy
 from django.utils import timezone
 from django.views.decorators.csrf import csrf_exempt
 from django.views.generic import DetailView, FormView, TemplateView
 from django.views.generic.edit import CreateView
 from django.views.generic.list import ListView
+from moods.models import Mood
 from music.aggregators import live_charts, scrobble_counts, week_of_scrobbles
-
-from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
+from pendulum.parsing.exceptions import ParserError
 from rest_framework import status
 from rest_framework.decorators import (
     api_view,
@@ -54,10 +55,10 @@ from scrobbles.tasks import (
     process_tsv_import,
 )
 from scrobbles.utils import (
+    get_daily_calories_for_user_by_day,
     get_long_plays_completed,
     get_long_plays_in_progress,
 )
-from moods.models import Mood
 
 logger = logging.getLogger(__name__)
 
@@ -146,7 +147,7 @@ class RecentScrobbleList(ListView):
             if date_str:
                 try:
                     date = pendulum.parse(date_str)
-                except:
+                except ParserError:
                     pass
             if date_str:
                 if date_str == "this_week" or "-W" in date_str:
@@ -230,6 +231,7 @@ class RecentScrobbleList(ListView):
                 user=self.request.user,
             )
             data["counts"] = []  # scrobble_counts(user)
+            data["daily_calories"] = get_daily_calories_for_user_by_day(self.request.user.id, date)
         else:
             data["weekly_data"] = week_of_scrobbles()
             data["counts"] = scrobble_counts()

vrobbler/templates/_scrobblable_list.html

@@ -12,6 +12,7 @@
     </thead>
     <tbody>
         {% for obj in object_list %}
+        {% if obj.title %}
         <tr>
             <td><a href="{{obj.scrobble_set.last.get_absolute_url}}">{{obj.scrobble_set.last.local_timestamp}}
             <td><a href="{{obj.get_absolute_url}}">{{obj}}</a></td>
@@ -20,6 +21,7 @@
             <td><a type="button" class="btn btn-sm btn-primary" href="{{obj.start_url}}">Scrobble</a></td>
             {% endif %}
         </tr>
+        {% endif %}
         {% endfor %}
     </tbody>
 </table>

vrobbler/templates/scrobbles/_last_scrobbles.html

@@ -105,6 +105,7 @@
         {% with scrobbles=Beer count=Beer_count time=Beer_time %}
         {% include "scrobbles/_scrobble_table.html" %}
         {% endwith %}
+        {% else %}
         <p>No beer today</p>
         {% endif %}
 

vrobbler/templates/scrobbles/_scrobble_table.html

@@ -4,7 +4,7 @@
 <div class="tab-pane fade show" id="latest-beers" role="tabpanel"
     aria-labelledby="latest-beers-tab">
     <div class="table-responsive">
-        {{count}} scrobble {% if time %}| {{time|natural_duration}}{% endif %}
+        {{count}} scrobble {% if time %}| {{time|natural_duration}}{% endif %}{% if daily_calories %}| {{daily_calories}} calories{% endif %}
         <table class="table table-striped table-sm">
             <thead>
                 <tr>