[release] Bump to version 51.4

- Clean up metadata comicbook enrichment

PROJECT.org

@@ -88,7 +88,7 @@ fetching and simple saving.
 *** Metadata sources
 **** Scraper
 
-* Backlog [0/15] :vrobbler:project:personal:
+* Backlog [0/16] :vrobbler:project:personal:
 ** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
 :PROPERTIES:
 :ID:       702462cf-d54b-48c6-8a7c-78b8de751deb
@@ -442,41 +442,6 @@ displayed in the template.
 
 ** TODO [#B] Add CSV endpoint for book scrobbles that LibraryThing can ingest :personal:project:books:feature:export:
 https://app.todoist.com/app/task/add-a-csv-endpoint-for-users-book-reads-that-library-thing-can-ingest-6X7QPMRp265xMXqg#comment-6X7QrXq6gJjMP4hg
-** TODO [#B] Scrape ComicBookRoundUp ratings for comic book metadata :vrobbler:books:feature:comicbook:personal:project:
-
-- Note taken on [2025-09-25 Thu 10:51]
-
-  As an example https://comicbookroundup.com/comic-books/reviews/humanoids-publishing/the-history-of-science-fiction
-** TODO [#B] Find page numbers for comic books from ComicVine :feature:books:
-:PROPERTIES:
-:ID:       79f867c3-1288-4143-b6bf-2a452983ee9f
-:END:
-** TODO [#B] Fix koreader scrobble imports to use DST properly :bug:books:imports:
-:PROPERTIES:
-:ID:       79758cba-a440-48b6-a637-efb88827acf2
-:END:
-
-- Note taken on [2025-09-25 Thu 10:37] \\
-
-  This may already be fixed ... need to check.
-
-- Note taken on [2025-02-25 12:34] \\
-
-  The page data has the canonical date something was read in it, but it seems
-  to be an hour off. I traced this back to being off during DST, so we just need
-  the importer to be aware of whether a user is using DST or not and roll back
-  an hour for part of the year. Also, we'd need to adjust any old scrobbles that
-  took place with DST off to roll them back by an hour.
-      
-  
-*** Description
-
-This is a long-standing problem when daylight saving time takes effect. Time is
-manually set on a KoReader device (or at least, always saved in local time). So
-whatever time KoReader reports, we need to know, given the date and the user
-profile's historic timezone, how many hours to adjust the KoReader time to get
-to GMT to save it in the database.
-
 ** TODO [#B] Make IMAP and WebDAV configurable :webdav:feature:imap:importers:
 :PROPERTIES:
 :ID:       b1426d92-2feb-4d15-9738-d5b7b0594f96
@@ -544,6 +509,156 @@ log a warning and move on.
 We should have a global view `/favorites/` that shows the logged in users's
 favorited media objects.
 
+** TODO [#B] Scrape ComicBookRoundUp ratings for comic book metadata :vrobbler:books:feature:comicbook:personal:project:
+
+- Note taken on [2025-09-25 Thu 10:51]
+
+  As an example https://comicbookroundup.com/comic-books/reviews/humanoids-publishing/the-history-of-science-fiction
+** TODO [#B] Find page numbers for comic books from ComicVine :feature:books:
+:PROPERTIES:
+:ID:       79f867c3-1288-4143-b6bf-2a452983ee9f
+:END:
+** TODO [#C] Implement loguru into project :feature:loguru:logging:
+:PROPERTIES:
+:ID:       efcd0c0a-db81-4518-9c23-5505d59e8ef5
+:END:
+
+*** Description
+
+Would be great to formalize how we log so we can search for errors and such more
+easily. And our exposure to PII is really low at this point in the project,
+so we can probably use backtrace=True and diagnose=True to help us root cause
+bugs faster.
+
+** TODO [#B] Add a /trends/ page that shows trends based on scrobble data :feature:trends:scrobbles:
+
+*** Description
+
+This project is a bit invovled. But we should add a top level URL `trends` that shows
+various trends as defined either in a static settings file, or dynamically via a database table.
+
+Examples of trends:
+
+- How often does the user:
+  + watch sports while doing a task?
+  + do a task while watching a video?
+  * how often do I do 
+  
+- trail_scrobble__average_heartrate per trail 
+- ...
+
+* Version 51.4 [1/1]
+** DONE [#A] Clean up metadata comicbook enrichment :bug:comics:books:metadata:
+:PROPERTIES:
+:ID:       cd875450-7117-78ca-8be4-9c8b73037dba
+:END:
+
+*** Description
+
+Still getting wonky results with some comicbooks. Would be nice to be able to
+tag a Book as a comicbook, and also gather volume information. I also noticed
+that some books that are found in OL never get their comicvine_id populated. We
+should make sure we always have comicvine_ids if available.
+
+
+* Version 51.3 [1/1]
+** DONE [#A] Improve speed of index and chart pages :bug:scrobbles:perf:
+:PROPERTIES:
+:ID:       031a23f8-7c02-4926-9884-6654ceca16c2
+:END:
+
+*** Description
+
+Over the last few releases, the home page and charts pages have gotten really
+slow.
+
+We should look into what's causing the slowness and maybe do more agressive
+query optimization or caching.
+
+
+* Version 51.2 [2/2]
+** DONE [#A] Fix bug where last page of book gets separate scrobble :bug:books:importers:koreader:
+:PROPERTIES:
+:ID:       e13e0b4c-461e-e5a9-c685-b972f4e262e5
+:END:
+
+*** Description
+
+The new KoReader code is working great to import books with the correct timezone
+and what-not. But it has a weird artifact of creating one extra scrobble for the
+last page read. Need to button that up.
+
+** DONE [#B] Fix metadata scraping for books :books:metadata:
+:PROPERTIES:
+:ID:       ea416a69-a8a8-4d05-b7d4-0a3470820e34
+:END:
+
+
+* Version 51.1 [1/1]
+** DONE [#A] Fix scrobbling comic books :books:scrobbles:bug:
+:PROPERTIES:
+:ID:       8dfbff19-3fa4-f3b8-21c7-7a416498000c
+:END:
+
+*** Description
+
+At some point logdata and log got confused, and now when you try
+to scrobble a comic book, it just throws errors. We should look
+into where the confusion happened and fix it.
+
+
+* Version 51.0 [3/3]
+** DONE [#B] Fix koreader scrobble imports to use DST properly :bug:books:imports:
+:PROPERTIES:
+:ID:       79758cba-a440-48b6-a637-efb88827acf2
+:END:
+
+- Note taken on [2025-09-25 Thu 10:37] \\
+
+  This may already be fixed ... need to check.
+
+- Note taken on [2025-02-25 12:34] \\
+
+  The page data has the canonical date something was read in it, but it seems
+  to be an hour off. I traced this back to being off during DST, so we just need
+  the importer to be aware of whether a user is using DST or not and roll back
+  an hour for part of the year. Also, we'd need to adjust any old scrobbles that
+  took place with DST off to roll them back by an hour.
+      
+  
+*** Description
+
+This is a long-standing problem when daylight saving time takes effect. Time is
+manually set on a KoReader device (or at least, always saved in local time). So
+whatever time KoReader reports, we need to know, given the date and the user
+profile's historic timezone, how many hours to adjust the KoReader time to get
+to GMT to save it in the database.
+
+** DONE [#A] Fix book scrobbles where page_data is a list :bug:books:scrobbles:
+:PROPERTIES:
+:ID:       35b323fa-ccc0-4009-b227-8a0f12bbd469
+:END:
+
+*** Description
+
+Comic scrobbling is currently kind of janky. Most of the problems boil down to them
+storing saved page data in a list of dicts rather than a dict keyed off of the page number that
+was read.
+
+We need to adjust comic scrobbling to use a dict of pages keyed off the page number, and also
+write a migration script that runs as a data migration to update any book scrobbles that may
+have page_data as a list.
+
+*** Example data
+#+begin_src python
+{"notes": null, "page_end": 27, "page_data": [{"notes": null, "end_ts": 1771815895, "duration": 14, "start_ts": 1771815881, "description": null, "page_number": "1"}, {"notes": null, "end_ts": 1771815908, "duration": 13, "start_ts": 1771815895, "description": null, "page_number": "1"}, {"notes": null, "end_ts": 1771815913, "duration": 5, "start_ts": 1771815908, "description": null, "page_number": "2"}, {"notes": null, "end_ts": 1771815933, "duration": 20, "start_ts": 1771815913, "description": null, "page_number": "3"}, {"notes": null, "end_ts": 1771815945, "duration": 12, "start_ts": 1771815933, "description": null, "page_number": "4"}, {"notes": null, "end_ts": 1771815983, "duration": 38, "start_ts": 1771815945, "description": null, "page_number": "5"}, {"notes": null, "end_ts": 1771816007, "duration": 24, "start_ts": 1771815983, "description": null, "page_number": "6"}, {"notes": null, "end_ts": 1771816011, "duration": 4, "start_ts": 1771816007, "description": null, "page_number": "7"}, {"notes": null, "end_ts": 1771816013, "duration": 2, "start_ts": 1771816011, "description": null, "page_number": "8"}, {"notes": null, "end_ts": 1771816052, "duration": 39, "start_ts": 1771816013, "description": null, "page_number": "7"}, {"notes": null, "end_ts": 1771816127, "duration": 75, "start_ts": 1771816052, "description": null, "page_number": "8"}, {"notes": null, "end_ts": 1771816134, "duration": 7, "start_ts": 1771816127, "description": null, "page_number": "9"}, {"notes": null, "end_ts": 1771816196, "duration": 62, "start_ts": 1771816134, "description": null, "page_number": "10"}, {"notes": null, "end_ts": 1771816262, "duration": 66, "start_ts": 1771816196, "description": null, "page_number": "11"}, {"notes": null, "end_ts": 1771816293, "duration": 31, "start_ts": 1771816262, "description": null, "page_number": "12"}, {"notes": null, "end_ts": 1771816322, "duration": 29, "start_ts": 1771816293, "description": null, "page_number": "13"}, {"notes": null, "end_ts": 1771816330, "duration": 8, "start_ts": 1771816322, "description": null, "page_number": "14"}, {"notes": null, "end_ts": 1771816368, "duration": 38, "start_ts": 1771816330, "description": null, "page_number": "15"}, {"notes": null, "end_ts": 1771816388, "duration": 20, "start_ts": 1771816368, "description": null, "page_number": "16"}, {"notes": null, "end_ts": 1771816482, "duration": 94, "start_ts": 1771816388, "description": null, "page_number": "17"}, {"notes": null, "end_ts": 1771816550, "duration": 68, "start_ts": 1771816482, "description": null, "page_number": "18"}, {"notes": null, "end_ts": 1771816567, "duration": 17, "start_ts": 1771816550, "description": null, "page_number": "19"}, {"notes": null, "end_ts": 1771816586, "duration": 19, "start_ts": 1771816567, "description": null, "page_number": "20"}, {"notes": null, "end_ts": 1771816597, "duration": 11, "start_ts": 1771816586, "description": null, "page_number": "21"}, {"notes": null, "end_ts": 1771816616, "duration": 19, "start_ts": 1771816597, "description": null, "page_number": "22"}, {"notes": null, "end_ts": 1771816640, "duration": 24, "start_ts": 1771816616, "description": null, "page_number": "23"}, {"notes": null, "end_ts": 1771816690, "duration": 50, "start_ts": 1771816640, "description": null, "page_number": "24"}, {"notes": null, "end_ts": 1771816702, "duration": 12, "start_ts": 1771816690, "description": null, "page_number": "25"}, {"notes": null, "end_ts": 1771816823, "duration": 121, "start_ts": 1771816702, "description": null, "page_number": "26"}, {"notes": null, "end_ts": null, "duration": null, "start_ts": 1771816823, "description": null, "page_number": "27"}], "page_start": 1, "pages_read": 27, "resume_url": null, "description": null, "koreader_hash": null, "long_play_complete": false}
+
+#+end_src
+** DONE [#A] Lichess imports do not set default visbility :boardgames:bug:importers:lichess:
+:PROPERTIES:
+:ID:       a78f7c72-a20a-8db2-cde0-d92a731d4fba
+:END:
+
 * Version 50.2 [2/2]
 ** DONE [#B] Koreader imports only import single-page scrobbles the next day :bug:books:importers:
 :PROPERTIES:

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "vrobbler"
-version = "50.2"
+version = "51.4"
 description = ""
 authors = ["Colin Powell <colin@unbl.ink>"]
 

vrobbler/apps/boardgames/sources/lichess.py

@@ -103,6 +103,7 @@ def import_chess_games_for_user_id(user_id: int, commit: bool = False) -> dict:
             "source": "Lichess",
             "timezone": user.profile.timezone,
             "log": log_data,
+            "visibility": "private",
         }
         if commit:
             Scrobble.objects.create(**scrobble_dict)

vrobbler/apps/books/koreader.py

@@ -1,9 +1,8 @@
 import logging
 import re
 import sqlite3
-from datetime import datetime, timedelta
+from datetime import datetime, timedelta, timezone
 from enum import Enum
-from zoneinfo import ZoneInfo
 
 import requests
 from books.constants import BOOKS_TITLES_TO_IGNORE
@@ -174,7 +173,7 @@ def build_book_map(rows) -> dict:
     return book_id_map
 
 
-def build_page_data(page_rows: list, book_map: dict, user_tz=None) -> dict:
+def build_page_data(page_rows: list, book_map: dict) -> dict:
     """Given rows of page data from KoReader, parse each row and build
     scrobbles for our user, loading the page data into the page_data
     field on the scrobble instance.
@@ -218,7 +217,6 @@ def build_scrobbles_from_book_map(book_map: dict, user: "User") -> list["Scrobbl
             pages_not_found.append(book_id)
             continue
 
-        should_create_scrobble = False
         scrobble_page_data = {}
         playback_position_seconds = 0
         prev_page_stats = {}
@@ -248,37 +246,48 @@ def build_scrobbles_from_book_map(book_map: dict, user: "User") -> list["Scrobbl
             end_of_reading = pages_processed == total_pages_read
             big_jump_to_this_page = (page_number - last_page_number) > 10
             is_session_gap = seconds_from_last_page > SESSION_GAP_SECONDS
-            if (is_session_gap and not big_jump_to_this_page) or end_of_reading:
-                should_create_scrobble = True
+            should_create_scrobble = (
+                is_session_gap and not big_jump_to_this_page
+            ) or end_of_reading
+
+            # Always accumulate the current page first
+            scrobble_page_data[page_number] = stats
 
             if should_create_scrobble:
-                if not scrobble_page_data:
-                    scrobble_page_data[page_number] = stats
+                # For end-of-reading, the current page is already accumulated
+                # and belongs in this scrobble. For a session gap, remove the
+                # current page from this scrobble — it starts a new session.
+                if is_session_gap and not end_of_reading:
+                    del scrobble_page_data[page_number]
+
                 scrobble_page_data = dict(
                     sorted(
                         scrobble_page_data.items(),
                         key=lambda x: x[1]["start_ts"],
                     )
                 )
-                try:
-                    first_page = scrobble_page_data.get(
-                        list(scrobble_page_data.keys())[0]
-                    )
-                    last_page = scrobble_page_data.get(
-                        list(scrobble_page_data.keys())[-1]
-                    )
-                except IndexError:
+
+                if not scrobble_page_data:
                     logger.error(
                         "Could not process book, no page data found",
                         extra={"scrobble_page_data": scrobble_page_data},
                     )
                     continue
 
+                first_page = next(iter(scrobble_page_data.values()))
+                last_page = scrobble_page_data[
+                    list(scrobble_page_data.keys())[-1]
+                ]
+
                 timestamp = user.profile.get_timestamp_with_tz(
-                    datetime.fromtimestamp(int(first_page.get("start_ts")))
+                    datetime.fromtimestamp(
+                        int(first_page.get("start_ts"))
+                    )
                 )
                 stop_timestamp = user.profile.get_timestamp_with_tz(
-                    datetime.fromtimestamp(int(last_page.get("end_ts")))
+                    datetime.fromtimestamp(
+                        int(last_page.get("end_ts"))
+                    )
                 )
 
                 scrobble = Scrobble.objects.filter(
@@ -316,16 +325,18 @@ def build_scrobbles_from_book_map(book_map: dict, user: "User") -> list["Scrobbl
                             timezone=tz,
                         )
                     )
-                # Then start over
-                should_create_scrobble = False
+                # Then start over for the next session
                 playback_position_seconds = 0
                 scrobble_page_data = {}
 
-            # We accumulate pages for the scrobble until we should create a new one
-            scrobble_page_data[page_number] = stats
+                # For session gaps, re-add the current page as the
+                # beginning of the next session's accumulation
+                if is_session_gap and not end_of_reading:
+                    scrobble_page_data[page_number] = stats
 
             last_page_number = page_number
             prev_page_stats = stats
+
     if pages_not_found:
         logger.info(f"Pages not found for books: {set(pages_not_found)}")
     return scrobbles_to_create
@@ -355,9 +366,6 @@ def process_koreader_sqlite_file(file_path, user_id) -> list:
 
     new_scrobbles = []
     user = User.objects.filter(id=user_id).first()
-    tz = ZoneInfo("UTC")
-    if user:
-        tz = user.profile.tzinfo
 
     is_os_file = "https://" not in file_path
     if is_os_file:
@@ -373,7 +381,6 @@ def process_koreader_sqlite_file(file_path, user_id) -> list:
         book_map = build_page_data(
             cur.execute("SELECT * from page_stat_data ORDER BY id_book, start_time"),
             book_map,
-            tz,
         )
         new_scrobbles = build_scrobbles_from_book_map(book_map, user)
     else:
@@ -390,7 +397,7 @@ def process_koreader_sqlite_file(file_path, user_id) -> list:
             _sqlite_bytes(file_path), max_buffer_size=1_048_576
         ):
             if table_name == "page_stat_data":
-                book_map = build_page_data(rows, book_map, tz)
+                book_map = build_page_data(rows, book_map)
                 new_scrobbles = build_scrobbles_from_book_map(book_map, user)
 
     logger.info(f"Creating {len(new_scrobbles)} new scrobbles")

vrobbler/apps/books/management/commands/cleanup_book_metadata.py

@@ -0,0 +1,301 @@
+import logging
+import time
+
+from django.core.management.base import BaseCommand
+from django.db import transaction
+
+from books.constants import READCOMICSONLINE_URL
+
+logger = logging.getLogger(__name__)
+
+MISSING_ALL = [
+    "cover",
+    "summary",
+    "isbn",
+    "pages",
+    "language",
+    "publisher",
+    "publish_year",
+]
+
+MISSING_GROUPS = {
+    "cover": lambda b: not bool(b.cover),
+    "summary": lambda b: not b.summary,
+    "isbn": lambda b: not b.isbn_13 and not b.isbn_10,
+    "pages": lambda b: b.pages is None,
+    "language": lambda b: not b.language,
+    "publisher": lambda b: not b.publisher,
+    "publish_year": lambda b: b.first_publish_year is None,
+}
+
+
+def _book_matches(book, flags):
+    if not flags:
+        return False
+    for flag in flags:
+        fn = MISSING_GROUPS.get(flag)
+        if fn and fn(book):
+            return True
+    return False
+
+
+class Command(BaseCommand):
+    help = "Backfill missing metadata on books from Google Books, OpenLibrary, and ComicVine"
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--commit",
+            action="store_true",
+            help="Commit changes to the database",
+        )
+        parser.add_argument(
+            "--batch-size",
+            type=int,
+            default=100,
+            help="Number of books to process per batch (default: 100)",
+        )
+        parser.add_argument(
+            "--sleep",
+            type=float,
+            default=0.5,
+            help="Seconds to sleep between API calls (default: 0.5)",
+        )
+        for flag in MISSING_ALL:
+            parser.add_argument(
+                f"--missing-{flag}",
+                dest="missing_flags",
+                action="append_const",
+                const=flag,
+                help=f"Process books missing {flag}",
+            )
+        parser.add_argument(
+            "--comics-only",
+            action="store_true",
+            help="Only process books with a readcomicsonline.ru URL",
+        )
+        parser.add_argument(
+            "--all",
+            action="store_true",
+            dest="all_missing",
+            help="Process books missing any metadata field",
+        )
+
+    def handle(self, *args, **options):
+        from books.models import Book
+
+        commit = options["commit"]
+        batch_size = options["batch_size"]
+        sleep_secs = options["sleep"]
+        flags = options.get("missing_flags") or []
+        comics_only = options["comics_only"]
+        all_missing = options["all_missing"]
+
+        if all_missing:
+            flags = MISSING_ALL
+
+        if not flags and not comics_only:
+            self.stdout.write(
+                "No filters specified. Use --all, --missing-*, or --comics-only."
+            )
+            return
+
+        qs = Book.objects.all()
+        if comics_only:
+            qs = qs.filter(readcomics_url__isnull=False)
+
+        if flags:
+            qs = [b for b in qs.iterator() if _book_matches(b, flags)]
+        else:
+            qs = list(qs)
+
+        total = len(qs)
+        self.stdout.write(f"Found {total} books to process")
+
+        if not commit:
+            self.stdout.write(
+                "Dry run — no API calls will be made. Use --commit to run lookups."
+            )
+            return
+
+        enriched = 0
+        skipped = 0
+        stats = {
+            "cover_fixed": 0,
+            "summary_fixed": 0,
+            "isbn_fixed": 0,
+            "pages_fixed": 0,
+            "language_fixed": 0,
+            "publisher_fixed": 0,
+            "publish_year_fixed": 0,
+        }
+
+        for batch_num, offset in enumerate(range(0, len(qs), batch_size)):
+            batch = qs[offset : offset + batch_size]
+            for book in batch:
+                result = self._enrich_book(book, sleep_secs)
+                if result:
+                    enriched += 1
+                    for key in stats:
+                        if result.get(key):
+                            stats[key] += 1
+                else:
+                    skipped += 1
+
+            self.stdout.write(
+                f"  Batch {batch_num + 1}: {offset + len(batch)}/{total} — "
+                f"enriched: {enriched}, skipped: {skipped}"
+            )
+
+        self.stdout.write(
+            f"\nResults (commit={commit}):\n"
+            f"  Books enriched: {enriched}\n"
+            f"  Books skipped:  {skipped}\n"
+            f"  Covers fixed:   {stats['cover_fixed']}\n"
+            f"  Summaries fixed:{stats['summary_fixed']}\n"
+            f"  ISBNs fixed:    {stats['isbn_fixed']}\n"
+            f"  Pages fixed:    {stats['pages_fixed']}\n"
+            f"  Languages fixed:{stats['language_fixed']}\n"
+            f"  Publishers fixed:{stats['publisher_fixed']}\n"
+            f"  Publish yrs fixed: {stats['publish_year_fixed']}"
+        )
+
+    def _enrich_book(self, book, sleep_secs):
+        from books.sources.comicvine import (
+            lookup_comic_from_comicvine,
+            lookup_issue_by_comicvine_id,
+        )
+        from books.sources.google import lookup_book_from_google
+        from books.sources.openlibrary import lookup_book_from_openlibrary as lookup_book_from_ol
+
+        title = book.original_title or book.title
+        author_name = book.author.name if book.author else None
+        book_dict = {}
+
+        cv_data = None
+        if book.comicvine_id:
+            cv_data = lookup_issue_by_comicvine_id(str(book.comicvine_id))
+        if not cv_data:
+            cv_data = lookup_comic_from_comicvine(title)
+        if cv_data:
+            book_dict.update(cv_data)
+
+        ol_data = lookup_book_from_ol(title, author=author_name)
+        time.sleep(sleep_secs)
+        google_data = lookup_book_from_google(title)
+
+        if ol_data:
+            for k, v in ol_data.items():
+                book_dict.setdefault(k, v)
+
+        if google_data:
+            for k, v in google_data.items():
+                if v:
+                    book_dict.setdefault(k, v)
+
+        if not book_dict:
+            return None
+
+        changed = self._apply(book, book_dict, title)
+        return changed
+
+    def _apply(self, book, data, title):
+        changed = {
+            "cover_fixed": False,
+            "summary_fixed": False,
+            "isbn_fixed": False,
+            "pages_fixed": False,
+            "language_fixed": False,
+            "publisher_fixed": False,
+            "publish_year_fixed": False,
+        }
+
+        update_fields = []
+
+        cover_url = data.pop("cover_url", "")
+
+        if data.get("summary") and not book.summary:
+            book.summary = data["summary"]
+            update_fields.append("summary")
+            changed["summary_fixed"] = True
+
+        if data.get("isbn_13") and not book.isbn_13:
+            book.isbn_13 = data["isbn_13"]
+            update_fields.append("isbn_13")
+            changed["isbn_fixed"] = True
+
+        if data.get("isbn_10") and not book.isbn_10:
+            book.isbn_10 = data["isbn_10"]
+            update_fields.append("isbn_10")
+            changed["isbn_fixed"] = True
+
+        if data.get("pages") and book.pages is None:
+            book.pages = data["pages"]
+            update_fields.append("pages")
+            changed["pages_fixed"] = True
+
+        if data.get("language") and not book.language:
+            book.language = data["language"]
+            update_fields.append("language")
+            changed["language_fixed"] = True
+
+        if data.get("publisher") and not book.publisher:
+            book.publisher = data["publisher"]
+            update_fields.append("publisher")
+            changed["publisher_fixed"] = True
+
+        if data.get("first_publish_year") and book.first_publish_year is None:
+            book.first_publish_year = data["first_publish_year"]
+            update_fields.append("first_publish_year")
+            changed["publish_year_fixed"] = True
+
+        if data.get("openlibrary_id") and not book.openlibrary_id:
+            book.openlibrary_id = data["openlibrary_id"]
+            update_fields.append("openlibrary_id")
+
+        if data.get("comicvine_id") and not book.comicvine_id:
+            book.comicvine_id = data["comicvine_id"]
+            update_fields.append("comicvine_id")
+
+        if data.get("issue_number") and book.issue_number is None:
+            book.issue_number = data["issue_number"]
+            update_fields.append("issue_number")
+
+        if data.get("volume_number") and book.volume_number is None:
+            book.volume_number = data["volume_number"]
+            update_fields.append("volume_number")
+
+        if data.get("volume") and not book.volume:
+            book.volume = data["volume"]
+            update_fields.append("volume")
+
+        if data.get("volume_comicvine_id") and not book.volume_comicvine_id:
+            book.volume_comicvine_id = data["volume_comicvine_id"]
+            update_fields.append("volume_comicvine_id")
+
+        if update_fields:
+            book.save(update_fields=update_fields)
+            self.stdout.write(f"  [ENRICHED] {book} — {', '.join(update_fields)}")
+
+        if cover_url and not book.cover:
+            book.save_image_from_url(cover_url)
+            if book.cover:
+                changed["cover_fixed"] = True
+                self.stdout.write(f"  [COVER] {book} — cover saved from source")
+
+        genres = data.pop("genres", data.pop("generes", []))
+        if genres:
+            existing = set(book.genre.names())
+            new_genres = [g for g in genres if g not in existing]
+            if new_genres:
+                book.genre.add(*new_genres)
+                self.stdout.write(f"  [GENRES] {book} — added {len(new_genres)} genres")
+
+        tags = data.pop("tags", [])
+        if tags:
+            existing_tags = set(book.tags.names())
+            new_tags = [t for t in tags if t not in existing_tags]
+            if new_tags:
+                book.tags.add(*new_tags)
+                self.stdout.write(f"  [TAGS] {book} — added {', '.join(new_tags)}")
+
+        return changed if any(changed.values()) else None

vrobbler/apps/books/management/commands/merge_koreader_scrobbles.py

@@ -0,0 +1,130 @@
+import logging
+
+from datetime import timedelta
+
+from django.core.management.base import BaseCommand
+from django.db import transaction
+
+from books.koreader import SESSION_GAP_SECONDS, fix_long_play_stats_for_scrobbles
+
+logger = logging.getLogger(__name__)
+SESSION_GAP = timedelta(seconds=SESSION_GAP_SECONDS)
+
+
+def _page_data_keys(pages):
+    return sorted(int(k) for k in (pages or {}))
+
+
+class Command(BaseCommand):
+    help = "Merge orphaned 1-page KOReader scrobbles into the preceding scrobble"
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--commit",
+            action="store_true",
+            help="Commit changes to the database",
+        )
+
+    def handle(self, *args, **options):
+        from scrobbles.models import Scrobble
+
+        commit = options["commit"]
+
+        qs = Scrobble.objects.filter(
+            media_type="Book", source="KOReader"
+        ).order_by("book_id", "timestamp")
+
+        if not qs.exists():
+            self.stdout.write("No KOReader book scrobbles found.")
+            return
+
+        merged = 0
+        affected_books = set()
+
+        # Group by book_id manually since we're iterating in order
+        book_scrobbles = {}
+        for s in qs:
+            book_scrobbles.setdefault(s.book_id, []).append(s)
+
+        if not commit:
+            self.stdout.write("Dry run — no changes will be saved. Use --commit to apply.")
+
+        for book_id, scrobbles in book_scrobbles.items():
+            batch_merged = 0
+            i = 0
+            while i < len(scrobbles) - 1:
+                current = scrobbles[i]
+                orphan = scrobbles[i + 1]
+
+                orphan_pages = orphan.logdata.page_data if orphan.logdata else {}
+                orphan_keys = _page_data_keys(orphan_pages)
+                if len(orphan_keys) != 1:
+                    i += 1
+                    continue
+
+                current_pages = current.logdata.page_data if current.logdata else {}
+                current_keys = _page_data_keys(current_pages)
+                if not current_keys:
+                    i += 1
+                    continue
+
+                orphan_page_num = orphan_keys[0]
+                current_last_page = current_keys[-1]
+
+                if orphan_page_num != current_last_page + 1:
+                    i += 1
+                    continue
+
+                # Check that the orphan is close enough in time
+                gap = orphan.timestamp - current.stop_timestamp
+                if gap > SESSION_GAP:
+                    i += 1
+                    continue
+
+                # Merge orphan into current
+                current_pages[str(orphan_page_num)] = orphan_pages[str(orphan_page_num)]
+                current.log["page_data"] = current_pages
+                current.log["pages_read"] = len(current_pages)
+                current.stop_timestamp = orphan.stop_timestamp
+                current.playback_position_seconds += orphan.playback_position_seconds
+
+                affected_books.add(book_id)
+
+                if commit:
+                    with transaction.atomic():
+                        current.save(
+                            update_fields=[
+                                "log",
+                                "stop_timestamp",
+                                "playback_position_seconds",
+                            ]
+                        )
+                        orphan.delete()
+
+                merged += 1
+                batch_merged += 1
+                scrobbles.pop(i + 1)
+
+            if batch_merged:
+                self.stdout.write(
+                    f"  Book {book_id}: merged {batch_merged} orphan scrobble(s)"
+                )
+
+        self.stdout.write(f"\nTotal orphans merged: {merged}")
+
+        if commit and affected_books:
+            self.stdout.write("Recalculating long_play_stats for affected books...")
+            for book_id in affected_books:
+                scrobbles_to_fix = (
+                    Scrobble.objects.filter(book_id=book_id, source="KOReader")
+                    .order_by("timestamp")
+                )
+                fix_long_play_stats_for_scrobbles(list(scrobbles_to_fix))
+
+            self.stdout.write(f"Fixed stats for {len(affected_books)} books.")
+
+        if not commit:
+            self.stdout.write(
+                f"\nWould merge {merged} orphan scrobble(s) across "
+                f"{len(affected_books)} book(s)."
+            )

vrobbler/apps/books/migrations/0037_book_volume_book_volume_comicvine_id.py

@@ -0,0 +1,23 @@
+# Generated by Django 4.2.29 on 2026-06-15 16:09
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("books", "0036_alter_book_genre_alter_paper_genre"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="book",
+            name="volume",
+            field=models.CharField(blank=True, max_length=255, null=True),
+        ),
+        migrations.AddField(
+            model_name="book",
+            name="volume_comicvine_id",
+            field=models.CharField(blank=True, max_length=255, null=True),
+        ),
+    ]

vrobbler/apps/books/models.py

@@ -4,6 +4,7 @@ from dataclasses import dataclass
 from datetime import datetime
 from typing import Optional
 from uuid import uuid4
+from zoneinfo import ZoneInfo
 
 import requests
 from books.constants import MediaSourceTag, READCOMICSONLINE_URL
@@ -19,6 +20,7 @@ from books.openlibrary import (
 from books.sources.comicvine import (
     ComicVineClient,
     lookup_comic_from_comicvine,
+    lookup_issue_by_comicvine_id,
 )
 from books.sources.google import lookup_book_from_google
 from books.sources.openlibrary import (
@@ -71,7 +73,7 @@ class BookLogData(BaseLogData, LongPlayLogData):
     _excluded_fields = {"koreader_hash", "page_data"}
 
     def avg_seconds_per_page(self):
-        if self.page_data:
+        if self.page_data and isinstance(self.page_data, dict):
             total_duration = 0
             for page_num, stats in self.page_data.items():
                 total_duration += stats.get("duration", 0)
@@ -149,6 +151,8 @@ class Book(LongPlayScrobblableMixin):
     first_sentence = models.TextField(**BNULL)
     # ComicVine
     comicvine_id = models.CharField(max_length=255, **BNULL)
+    volume = models.CharField(max_length=255, **BNULL)
+    volume_comicvine_id = models.CharField(max_length=255, **BNULL)
     readcomics_url = models.CharField(max_length=255, **BNULL)
     next_readcomics_url = models.CharField(max_length=255, **BNULL)
     issue_number = models.IntegerField(**BNULL)
@@ -173,6 +177,9 @@ class Book(LongPlayScrobblableMixin):
     genre = TaggableManager(through=ObjectWithGenres, blank=True, verbose_name="Genre")
 
     def __str__(self) -> str:
+        if not self.subtitle:
+            return self.title
+
         return f"{self.title} - {self.subtitle}"
 
     def save(self, *args, **kwargs):
@@ -184,12 +191,18 @@ class Book(LongPlayScrobblableMixin):
 
     @property
     def subtitle(self):
-        subtitle = self.author
+        subtitle_parts = []
+        if self.author:
+            subtitle_parts.append(self.author.name)
         if self.issue_number and "Issue" not in str(self.title):
-            subtitle += " - Issue {self.issue_number}"
+            subtitle_parts.append(f"Issue {self.issue_number}")
         if self.volume_number and "Volume" not in str(self.title):
-            subtitle += " - Volume {self.volume_number}"
-        return subtitle
+            subtitle_parts.append(f"Volume {self.volume_number}")
+        if len(subtitle_parts) > 1:
+            return " / ".join(subtitle_parts)
+        if len(subtitle_parts) == 1:
+            return subtitle_parts[0]
+        return ""
 
     @property
     def strings(self) -> ScrobblableConstants:
@@ -219,36 +232,24 @@ class Book(LongPlayScrobblableMixin):
     ) -> "Book":
         book, created = cls.objects.get_or_create(title=title)
 
-        if not created:
+        if not created and not overwrite:
             return book
 
         book_dict = lookup_comic_from_comicvine(title)
+        if not book_dict:
+            return book
 
-        if created or overwrite:
-            author_list = []
-            author_dicts = book_dict.pop("author_dicts")
-            if author_dicts:
-                for author_dict in author_dicts:
-                    if author_dict.get("authorId"):
-                        author, a_created = Author.objects.get_or_create(
-                            semantic_id=author_dict.get("authorId")
-                        )
-                        author_list.append(author)
-                        if a_created:
-                            author.name = author_dict.get("name")
-                            author.save()
-                            # TODO enrich author?
-                            ...
+        tags = book_dict.pop("tags", [])
+        genres = book_dict.pop("genres", book_dict.pop("generes", []))
 
-            for k, v in book_dict.items():
-                setattr(book, k, v)
-            book.save()
+        for k, v in book_dict.items():
+            setattr(book, k, v)
+        book.save()
 
-            if author_list:
-                book.authors.add(*author_list)
-            genres = book_dict.pop("genres", [])
-            if genres:
-                book.genre.add(*genres)
+        if genres:
+            book.genre.add(*genres)
+        if tags:
+            book.tags.add(*tags)
         return book
 
     @classmethod
@@ -282,24 +283,43 @@ class Book(LongPlayScrobblableMixin):
 
         book_dict = None
         source_tag = None
+        tried_comicvine = False
         if READCOMICSONLINE_URL in url:
             book_dict = lookup_comic_from_comicvine(title)
+            tried_comicvine = True
             if book_dict:
                 source_tag = MediaSourceTag.COMICVINE
-            book_dict["readcomics_url"] = get_comic_issue_url(url)
-            book_dict["next_readcomics_url"] = next_url_if_exists(
-                book_dict["readcomics_url"]
-            )
+                book_dict["readcomics_url"] = get_comic_issue_url(url)
+                book_dict["next_readcomics_url"] = next_url_if_exists(
+                    book_dict["readcomics_url"]
+                )
 
         if not book_dict:
-            book_dict = lookup_book_from_ol(title, author=author)
-            if book_dict:
+            book_dict = {}
+            ol_data = lookup_book_from_ol(title, author=author)
+            google_data = lookup_book_from_google(title)
+
+            if ol_data:
+                book_dict.update(ol_data)
                 source_tag = MediaSourceTag.OPENLIBRARY
 
-        if not book_dict:
-            book_dict = lookup_book_from_google(title)
-            if book_dict:
+            if google_data:
+                for k, v in google_data.items():
+                    if v:
+                        book_dict.setdefault(k, v)
                 source_tag = MediaSourceTag.GOOGLE_BOOKS
+                if ol_data and ol_data.get("cover_url"):
+                    book_dict["cover_url"] = ol_data["cover_url"]
+
+        # Always try ComicVine as a fallback — it may recognize books that
+        # OL/Google don't flag as comics
+        if not tried_comicvine:
+            cv_data = lookup_comic_from_comicvine(title)
+            if cv_data:
+                for k, v in cv_data.items():
+                    if v:
+                        book_dict.setdefault(k, v)
+                source_tag = MediaSourceTag.COMICVINE
 
         if not book_dict:
             logger.warning(
@@ -311,6 +331,7 @@ class Book(LongPlayScrobblableMixin):
         authors = book_dict.pop("authors", [])
         cover_url = book_dict.pop("cover_url", "")
         genres = book_dict.pop("genres", book_dict.pop("generes", []))
+        tags = book_dict.pop("tags", [])
 
         if authors:
             for author_str in authors:
@@ -330,6 +351,8 @@ class Book(LongPlayScrobblableMixin):
             book.save_image_from_url(cover_url)
             if genres:
                 book.genre.add(*genres)
+            if tags:
+                book.tags.add(*tags)
             book.authors.add(*author_list)
             if source_tag:
                 book.tags.add(source_tag.value)
@@ -367,9 +390,14 @@ class Book(LongPlayScrobblableMixin):
                     data = lookup_comic_from_locg(str(self.title))
 
             if not data and COMICVINE_API_KEY:
-                logger.warn(f"Checking ComicVine for {self.title}")
-                cv_client = ComicVineClient(api_key=COMICVINE_API_KEY)
-                data = lookup_comic_from_comicvine(str(self.title))
+                if self.comicvine_id:
+                    logger.warn(
+                        f"Checking ComicVine by ID for {self.title}"
+                    )
+                    data = lookup_issue_by_comicvine_id(str(self.comicvine_id))
+                if not data:
+                    logger.warn(f"Checking ComicVine for {self.title}")
+                    data = lookup_comic_from_comicvine(str(self.title))
 
             if not data:
                 logger.warn(f"Book not found in any sources: {self.title}")
@@ -407,10 +435,10 @@ class Book(LongPlayScrobblableMixin):
                 )
                 data.pop("pages")
 
-            # Pop this, so we can look it up later
+            # Pop these so they don't get passed to update()
             cover_url = data.pop("cover_url", "")
-
             subject_key_list = data.pop("subject_key_list", [])
+            tags = data.pop("tags", [])
 
             # Fun trick for updating all fields at once
             Book.objects.filter(pk=self.id).update(**data)
@@ -418,6 +446,8 @@ class Book(LongPlayScrobblableMixin):
 
             if subject_key_list:
                 self.genre.add(*subject_key_list)
+            if tags:
+                self.tags.add(*tags)
 
             if cover_url:
                 r = requests.get(cover_url)
@@ -463,8 +493,11 @@ class Book(LongPlayScrobblableMixin):
             if scrobble.logdata.page_data:
                 for page, data in scrobble.logdata.page_data.items():
                     if convert_timestamps:
-                        data["start_ts"] = datetime.fromtimestamp(data["start_ts"])
-                        data["end_ts"] = datetime.fromtimestamp(data["end_ts"])
+                        tz = None
+                        if scrobble.timezone:
+                            tz = ZoneInfo(scrobble.timezone)
+                        data["start_ts"] = datetime.fromtimestamp(data["start_ts"], tz=tz)
+                        data["end_ts"] = datetime.fromtimestamp(data["end_ts"], tz=tz)
                     pages[page] = data
         sorted_pages = OrderedDict(
             sorted(pages.items(), key=lambda x: x[1]["start_ts"])

vrobbler/apps/books/sources/comicvine.py

@@ -17,8 +17,10 @@ class ComicVineClient(object):
     account on https://comicvine.gamespot.com/ in order to obtain an API key.
     """
 
-    # All API requests made by this client will be made to this URL.
-    API_URL = "https://www.comicvine.com/api/search/"
+    # All API requests made by this client will be made to these URLs.
+    API_URL = "https://comicvine.gamespot.com/api/search/"
+    ISSUE_API_URL = "https://comicvine.gamespot.com/api/issue/4000-{issue_id}/"
+    VOLUME_API_URL = "https://comicvine.gamespot.com/api/volume/4050-{volume_id}/"
 
     # A valid User-Agent header must be set in order for our API requests to
     # be accepted, otherwise our request will be rejected with a
@@ -41,15 +43,12 @@ class ComicVineClient(object):
         "volume",
     }
 
-    def __init__(self, api_key, expire_after=300):
+    def __init__(self, api_key):
         """
-        Store the API key in a class variable, and install the requests cache,
-        configuring it using the ``expire_after`` parameter.
+        Store the API key in a class variable.
 
         :param api_key: Your personal ComicVine API key.
         :type  api_key: str
-        :param expire_after: The number of seconds to retain an entry in cache.
-        :type  expire_after: int or None
         """
 
         self.api_key = api_key
@@ -109,14 +108,17 @@ class ComicVineClient(object):
         :rtype:  dict
         """
 
-        return {
+        params = {
             "api_key": self.api_key,
             "format": "json",
             "limit": min(10, limit),  # hard limit of 10
             "offset": max(0, offset),  # cannot provide negative offset
             "query": query,
-            "resources": self._validate_resources(resources),
         }
+        validated = self._validate_resources(resources)
+        if validated:
+            params["resources"] = validated
+        return params
 
     def _validate_resources(self, resources):
         """
@@ -141,33 +143,35 @@ class ComicVineClient(object):
     def _query_api(self, params):
         """
         Query the ComicVine API's ``search`` resource, providing the required
-        headers and parameters with the request. Optionally allow the caller
-        of the function to disable the request cache.
+        headers and parameters with the request.
 
         If an error occurs during the request, handle it accordingly. Upon
         success, return the JSON from the response.
 
         :param params: Parameters to include with the request.
         :type  params: dict
-        :param use_cache: Toggle the use of requests_cache.
-        :type  use_cache: bool
 
         :return: The JSON contained in the response.
         :rtype:  dict
         """
 
-        # Since we're performing the identical action regardless of whether
-        # or not the request cache is to be used, store the procedure in a
-        # local function to avoid repetition.
-        def __httpget():
-            response = requests.get(self.API_URL, headers=self.HEADERS, params=params)
+        response = requests.get(self.API_URL, headers=self.HEADERS, params=params)
 
-            if not response.ok:
-                self._handle_http_error(response)
+        if not response.ok:
+            self._handle_http_error(response)
 
-            return response.json()
+        json_data = response.json()
 
-        return __httpget()
+        if json_data.get("status_code") != 1:
+            error_msg = json_data.get("error", "Unknown ComicVine API error")
+            logger.error(
+                "ComicVine API returned status_code %s: %s",
+                json_data.get("status_code"),
+                error_msg,
+            )
+            return {}
+
+        return json_data
 
     def _handle_http_error(self, response):
         """
@@ -195,15 +199,81 @@ class ComicVineClient(object):
 
         raise exception(message)
 
+    def get_issue(self, issue_id: str) -> dict:
+        """
+        Fetch a single issue by its ComicVine ID directly from the issue detail
+        endpoint, which returns richer data than the search endpoint.
+
+        :param issue_id: The ComicVine numeric ID for the issue (e.g. "538480")
+        :type  issue_id: str
+
+        :return: The full JSON response for the issue, or empty dict on failure.
+        :rtype:  dict
+        """
+        params = {
+            "api_key": self.api_key,
+            "format": "json",
+        }
+        url = self.ISSUE_API_URL.format(issue_id=issue_id)
+        response = requests.get(url, headers=self.HEADERS, params=params)
+
+        if not response.ok:
+            self._handle_http_error(response)
+
+        json_data = response.json()
+
+        if json_data.get("status_code") != 1:
+            error_msg = json_data.get("error", "Unknown ComicVine API error")
+            logger.error(
+                "ComicVine API returned status_code %s: %s",
+                json_data.get("status_code"),
+                error_msg,
+            )
+            return {}
+
+        return json_data.get("results", {})
+
+    def get_volume(self, volume_id: str) -> dict:
+        """
+        Fetch a single volume by its ComicVine ID from the volume detail
+        endpoint. Used to get publisher info and other volume-level metadata.
+
+        :param volume_id: The ComicVine numeric ID for the volume (e.g. "91273")
+        :type  volume_id: str
+
+        :return: The full JSON response for the volume, or empty dict on failure.
+        :rtype:  dict
+        """
+        params = {
+            "api_key": self.api_key,
+            "format": "json",
+        }
+        url = self.VOLUME_API_URL.format(volume_id=volume_id)
+        response = requests.get(url, headers=self.HEADERS, params=params)
+
+        if not response.ok:
+            self._handle_http_error(response)
+
+        json_data = response.json()
+
+        if json_data.get("status_code") != 1:
+            error_msg = json_data.get("error", "Unknown ComicVine API error")
+            logger.error(
+                "ComicVine API returned status_code %s: %s",
+                json_data.get("status_code"),
+                error_msg,
+            )
+            return {}
+
+        return json_data.get("results", {})
+
 
 def lookup_comic_from_comicvine(title: str) -> dict:
     original_title = title
 
     issue_number = None
-    volume_nubmer = None
     resource_type = "issue"
     if "Issue " in title:
-        resource_type = "issue"
         issue_number = title.split("Issue ")[1]
     volume_number = None
     if "Volume " in title:
@@ -215,48 +285,136 @@ def lookup_comic_from_comicvine(title: str) -> dict:
         logger.warning("No ComicVine API key configured, not looking anything up")
         return {}
 
-    client = ComicVineClient(api_key=getattr(settings, "COMICVINE_API_KEY", None))
+    client = ComicVineClient(api_key=api_key)
 
-    raw_results = client.search(title).get("results")
-    results = [r for r in raw_results if r.get("resource_type") == resource_type]
+    raw_results = client.search(title)
+    if not raw_results:
+        return {}
+    results = raw_results.get("results", [])
+    results = [r for r in results if r.get("resource_type") == resource_type]
     if not results:
         logger.warning("No comic found on ComicVine")
         return {}
 
     found_result = None
     for result in results:
-        if result.get("issue_number") == str(issue_number):
+        if issue_number is not None and result.get("issue_number") == str(issue_number):
             found_result = result
             break
-        if result.get("volume_number") == str(volume_number):
+        if volume_number is not None and result.get("volume_number") == str(volume_number):
             found_result = result
             break
 
     if not found_result:
         found_result = results[0]
 
-    logger.info("ComicVine results", extra={"results": results})
+    data_dict = _build_data_dict_from_issue(found_result, original_title)
+    _enrich_with_volume_data(client, data_dict)
+    return data_dict
 
-    if not found_result:
-        logger.warning("No matches found on ComicVine")
+
+def lookup_issue_by_comicvine_id(comicvine_id: str) -> dict:
+    """
+    Look up an issue directly by its ComicVine ID using the issue detail
+    endpoint. Returns richer data than the search-based lookup.
+
+    :param comicvine_id: The ComicVine numeric ID for the issue (e.g. "538480")
+    :type  comicvine_id: str
+
+    :return: A dict of extracted book metadata, or empty dict on failure.
+    :rtype:  dict
+    """
+    if not comicvine_id:
         return {}
 
-    title = found_result.get("name")
+    api_key = getattr(settings, "COMICVINE_API_KEY", "")
+    if not api_key:
+        logger.warning("No ComicVine API key configured, not looking anything up")
+        return {}
 
-    if found_result.get("volume"):
-        title = found_result.get("volume").get("name")
+    client = ComicVineClient(api_key=api_key)
+    issue_data = client.get_issue(comicvine_id)
+    if not issue_data:
+        logger.warning("No issue found on ComicVine for ID %s", comicvine_id)
+        return {}
+
+    data_dict = _build_data_dict_from_issue(issue_data, issue_data.get("name", ""))
+    _enrich_with_volume_data(client, data_dict)
+    return data_dict
+
+
+def _build_data_dict_from_issue(issue_data: dict, original_title: str = "") -> dict:
+    """
+    Build a book metadata dict from a ComicVine issue resource (either from
+    search results or issue detail endpoint). Both return the same shape of
+    issue data.
+
+    :param issue_data: The issue resource dict from ComicVine.
+    :param original_title: The original search term, if any.
+
+    :return: A dict of extracted book metadata.
+    :rtype:  dict
+    """
+    title = issue_data.get("name")
+    if issue_data.get("volume"):
+        title = issue_data.get("volume").get("name")
+
+    cover_url = None
+    if issue_data.get("image"):
+        cover_url = issue_data["image"].get("original_url")
+
+    volume_name = None
+    volume_cv_id = None
+    publisher_name = None
+    volume_data = issue_data.get("volume")
+    if volume_data:
+        volume_name = volume_data.get("name")
+        volume_cv_id = volume_data.get("id")
+        publisher_data = volume_data.get("publisher")
+        if publisher_data:
+            publisher_name = publisher_data.get("name")
 
     data_dict = {
         "title": title,
         "original_title": original_title,
-        "issue_number": found_result.get("issue_number"),
-        "volume_number": found_result.get("volume_number"),
-        "cover_url": found_result.get("image").get("original_url"),
-        "comicvine_id": found_result.get("id"),
-        "comicvine_data": found_result,
-        "summary": found_result.get("description"),
-        "publish_date": found_result.get("cover_date"),
-        "first_publish_year": found_result.get("cover_date", "")[:4],
+        "issue_number": issue_data.get("issue_number"),
+        "volume_number": issue_data.get("volume_number"),
+        "volume": volume_name,
+        "volume_comicvine_id": volume_cv_id,
+        "publisher": publisher_name,
+        "cover_url": cover_url,
+        "comicvine_id": issue_data.get("id"),
+        "summary": issue_data.get("description"),
+        "publish_date": issue_data.get("cover_date"),
+        "first_publish_year": (issue_data.get("cover_date") or "")[:4],
+        "tags": ["comicbook"],
     }
 
     return data_dict
+
+
+def _enrich_with_volume_data(client: ComicVineClient, data_dict: dict) -> None:
+    """
+    Follow-up a successful issue lookup by fetching the volume detail and
+    filling in publisher and other volume-level metadata that the issue
+    endpoint doesn't provide.
+
+    :param client: An initialised ComicVineClient instance.
+    :param data_dict: The data dict from an issue lookup (mutated in place).
+    """
+    volume_cv_id = data_dict.get("volume_comicvine_id")
+    if not volume_cv_id:
+        return
+
+    volume_data = client.get_volume(str(volume_cv_id))
+    if not volume_data:
+        return
+
+    publisher_data = volume_data.get("publisher")
+    if publisher_data:
+        publisher_name = publisher_data.get("name")
+        if publisher_name and not data_dict.get("publisher"):
+            data_dict["publisher"] = publisher_name
+
+    if not data_dict.get("volume"):
+        data_dict["volume"] = volume_data.get("name")

vrobbler/apps/books/sources/google.py

@@ -26,8 +26,6 @@ def lookup_book_from_google(title: str) -> dict:
     if not google_result:
         return {}
 
-    publish_date = pendulum.parse(google_result.get("publishedDate"))
-
     isbn_13 = ""
     isbn_10 = ""
     for ident in google_result.get("industryIdentifiers", []):
@@ -35,25 +33,25 @@ def lookup_book_from_google(title: str) -> dict:
             isbn_13 = ident.get("identifier")
         if ident.get("type") == "ISBN_10":
             isbn_10 = ident.get("identifier")
-    # TODO this may lead to issues with the first get if Google changes our title
-    # book_metadata.title = google_result.get("title")
-    # if google_result.get("subtitle"):
-    #    book_metadata["title"] = ": ".join(
-    #        [google_result.get("title"), google_result.get("subtitle")]
-    #    )
-    # book_dict["subtitle"] = google_result.get("subtitle")
     book_dict["authors"] = google_result.get("authors")
     book_dict["publisher"] = google_result.get("publisher")
-    book_dict["first_publish_year"] = publish_date.year
     book_dict["pages"] = google_result.get("pageCount")
     book_dict["isbn_13"] = isbn_13
     book_dict["isbn_10"] = isbn_10
-    book_dict["publish_date"] = google_result.get("publishedDate")
-    if len(book_dict["publish_date"]) == 4:
-        book_dict["publish_date"] = f"{book_dict['publish_date']}-1-1"
     book_dict["language"] = google_result.get("language")
     book_dict["summary"] = google_result.get("description")
     book_dict["genres"] = google_result.get("categories")
+
+    raw_date = google_result.get("publishedDate")
+    if raw_date:
+        try:
+            publish_date = pendulum.parse(raw_date)
+            book_dict["first_publish_year"] = publish_date.year
+        except Exception:
+            pass
+        book_dict["publish_date"] = raw_date
+        if len(raw_date) == 4:
+            book_dict["publish_date"] = f"{raw_date}-1-1"
     book_dict["cover_url"] = (
         google_result.get("imageLinks", {})
         .get("thumbnail", "")

vrobbler/apps/books/tests/test_koreader.py

@@ -44,7 +44,10 @@ def test_build_scrobbles_from_pages(get_mock, koreader_rows, demo_user, valid_re
     book_map = build_page_data(koreader_rows.PAGE_STATS_ROWS, book_map)
 
     scrobbles = build_scrobbles_from_book_map(book_map, demo_user)
-    # Corresponds to number of sessions per book ( 20 pages per session, 120 +/- 15 pages read )
+    # The test data generator adds the session-gap 3600s AFTER the trigger page
+    # (not before), so the first session includes 21 pages (1-21), and each
+    # subsequent session has 20 until the last. The last page is now included
+    # in the final scrobble instead of being orphaned.
     expected_scrobbles = 6 * len(book_map.keys())
     assert len(scrobbles) == expected_scrobbles
     assert len(scrobbles[0].logdata.page_data.keys()) == 21
@@ -52,7 +55,7 @@ def test_build_scrobbles_from_pages(get_mock, koreader_rows, demo_user, valid_re
     assert len(scrobbles[2].logdata.page_data.keys()) == 20
     assert len(scrobbles[3].logdata.page_data.keys()) == 20
     assert len(scrobbles[4].logdata.page_data.keys()) == 20
-    assert len(scrobbles[5].logdata.page_data.keys()) == 18
+    assert len(scrobbles[5].logdata.page_data.keys()) == 19
 
 
 def test_get_author_str_from_row():

vrobbler/apps/charts/migrations/0002_chartrecord_charts_char_user_id_1adcde_idx_and_more.py

@@ -0,0 +1,146 @@
+# Generated by Django 4.2.29 on 2026-06-12 16:09
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("charts", "0001_initial"),
+    ]
+
+    operations = [
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "album", "rank"],
+                name="charts_char_user_id_1adcde_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "track", "rank"],
+                name="charts_char_user_id_d18aab_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "video", "rank"],
+                name="charts_char_user_id_de9f0a_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "board_game", "rank"],
+                name="charts_char_user_id_d5d58f_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "book", "rank"],
+                name="charts_char_user_id_e877cf_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "food", "rank"],
+                name="charts_char_user_id_a0ad71_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "podcast", "rank"],
+                name="charts_char_user_id_846b80_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "trail", "rank"],
+                name="charts_char_user_id_54feba_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "album", "rank"],
+                name="charts_char_user_id_a3dc49_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "track", "rank"],
+                name="charts_char_user_id_4b01ab_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "video", "rank"],
+                name="charts_char_user_id_2ac9d2_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "board_game", "rank"],
+                name="charts_char_user_id_ba968a_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "book", "rank"],
+                name="charts_char_user_id_e66751_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "food", "rank"],
+                name="charts_char_user_id_d23f06_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "podcast", "rank"],
+                name="charts_char_user_id_be8122_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "week", "trail", "rank"],
+                name="charts_char_user_id_b94ea9_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "day", "artist", "rank"],
+                name="charts_char_user_id_406e0e_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "day", "album", "rank"],
+                name="charts_char_user_id_322b0d_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="chartrecord",
+            index=models.Index(
+                fields=["user", "year", "month", "day", "tv_series", "rank"],
+                name="charts_char_user_id_aa44b7_idx",
+            ),
+        ),
+    ]

vrobbler/apps/charts/models.py

@@ -60,10 +60,29 @@ class ChartRecord(TimeStampedModel):
             models.Index(fields=["user", "year", "geo_location", "rank"]),
             models.Index(fields=["user", "year", "food", "rank"]),
             models.Index(fields=["user", "year", "book", "rank"]),
-            models.Index(fields=["user", "year", "week", "artist", "rank"]),
-            models.Index(fields=["user", "year", "week", "tv_series", "rank"]),
             models.Index(fields=["user", "year", "month", "artist", "rank"]),
+            models.Index(fields=["user", "year", "month", "album", "rank"]),
+            models.Index(fields=["user", "year", "month", "track", "rank"]),
             models.Index(fields=["user", "year", "month", "tv_series", "rank"]),
+            models.Index(fields=["user", "year", "month", "video", "rank"]),
+            models.Index(fields=["user", "year", "month", "board_game", "rank"]),
+            models.Index(fields=["user", "year", "month", "book", "rank"]),
+            models.Index(fields=["user", "year", "month", "food", "rank"]),
+            models.Index(fields=["user", "year", "month", "podcast", "rank"]),
+            models.Index(fields=["user", "year", "month", "trail", "rank"]),
+            models.Index(fields=["user", "year", "week", "artist", "rank"]),
+            models.Index(fields=["user", "year", "week", "album", "rank"]),
+            models.Index(fields=["user", "year", "week", "track", "rank"]),
+            models.Index(fields=["user", "year", "week", "tv_series", "rank"]),
+            models.Index(fields=["user", "year", "week", "video", "rank"]),
+            models.Index(fields=["user", "year", "week", "board_game", "rank"]),
+            models.Index(fields=["user", "year", "week", "book", "rank"]),
+            models.Index(fields=["user", "year", "week", "food", "rank"]),
+            models.Index(fields=["user", "year", "week", "podcast", "rank"]),
+            models.Index(fields=["user", "year", "week", "trail", "rank"]),
+            models.Index(fields=["user", "year", "month", "day", "artist", "rank"]),
+            models.Index(fields=["user", "year", "month", "day", "album", "rank"]),
+            models.Index(fields=["user", "year", "month", "day", "tv_series", "rank"]),
         ]
 
     @property

vrobbler/apps/charts/views.py

@@ -438,12 +438,14 @@ class ChartRecordView(TemplateView):
         return context
 
     def get_available_years(self, user):
-        return list(
-            ChartRecord.objects.filter(user=user)
-            .values_list("year", flat=True)
-            .distinct()
-            .order_by("-year")
-        )
+        if not hasattr(self, "_available_years"):
+            self._available_years = list(
+                ChartRecord.objects.filter(user=user)
+                .values_list("year", flat=True)
+                .distinct()
+                .order_by("-year")
+            )
+        return self._available_years
 
     def get_period_type(self):
         date_param = self.request.GET.get("date")

vrobbler/apps/profiles/models.py

@@ -140,6 +140,11 @@ class UserProfile(TimeStampedModel):
         return history
 
     def get_timestamp_with_tz(self, timestamp):
+        from django.conf import settings
+
+        server_tz = ZoneInfo(settings.TIME_ZONE)
+        ref_dt = timestamp if timestamp.tzinfo is not None else timestamp.replace(tzinfo=server_tz)
+
         timezone = self.tzinfo
         if self.timezone_change_log:
             change_list = self.historic_timezone_changes
@@ -150,13 +155,13 @@ class UserProfile(TimeStampedModel):
                     end = None
 
                 if end:
-                    if start <= timestamp.replace(tzinfo=end.timezone) <= end:
+                    if start <= ref_dt <= end:
                         timezone = start.timezone
                 else:
-                    if start <= timestamp.replace(tzinfo=start.timezone):
+                    if start <= ref_dt:
                         timezone = start.timezone
 
-        return timestamp.replace(tzinfo=timezone)
+        return ref_dt.astimezone(timezone)
 
     def adjust_timezone_of_scrobbles(self, commit=False):
         current_dt = None

vrobbler/apps/scrobbles/importers/webdav.py

@@ -153,7 +153,7 @@ def scan_webdav_for_koreader(
     )
 
     if update_etag_only:
-        if last_import and remote_etag:
+        if last_import and remote_etag and hasattr(last_import, "webdav_etag"):
             last_import.webdav_etag = remote_etag
             last_import.save(update_fields=["webdav_etag"])
             logger.info(
@@ -163,7 +163,7 @@ def scan_webdav_for_koreader(
             )
         return 0
 
-    if last_import and last_import.webdav_etag and remote_etag:
+    if last_import and getattr(last_import, "webdav_etag", None) and remote_etag:
         if last_import.webdav_etag == remote_etag:
             logger.info(
                 "koreader stats file unchanged (ETag match)",

vrobbler/apps/scrobbles/migrations/0096_convert_book_page_data_to_dict.py

@@ -0,0 +1,35 @@
+from django.db import migrations
+
+
+def convert_page_data_to_dict(apps, schema_editor):
+    Scrobble = apps.get_model("scrobbles", "Scrobble")
+    for scrobble in Scrobble.objects.filter(media_type="Book").exclude(
+        log__page_data=None
+    ):
+        page_data = scrobble.log.get("page_data")
+        if isinstance(page_data, list):
+            new_page_data = {}
+            for entry in page_data:
+                page_num = entry.get("page_number")
+                if page_num is not None:
+                    try:
+                        page_num = int(page_num)
+                    except (ValueError, TypeError):
+                        continue
+                    new_page_data[page_num] = entry
+            scrobble.log["page_data"] = new_page_data
+            scrobble.save(update_fields=["log"])
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("scrobbles", "0095_backfill_null_visibility"),
+    ]
+
+    operations = [
+        migrations.RunPython(
+            convert_page_data_to_dict,
+            reverse_code=migrations.RunPython.noop,
+        ),
+    ]

vrobbler/apps/scrobbles/migrations/0097_scrobble_scrobbles_s_user_id_d367a7_idx.py

@@ -0,0 +1,19 @@
+# Generated by Django 4.2.29 on 2026-06-12 16:09
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("scrobbles", "0096_convert_book_page_data_to_dict"),
+    ]
+
+    operations = [
+        migrations.AddIndex(
+            model_name="scrobble",
+            index=models.Index(
+                fields=["user", "-timestamp"], name="scrobbles_s_user_id_d367a7_idx"
+            ),
+        ),
+    ]

vrobbler/apps/scrobbles/models.py

@@ -225,7 +225,9 @@ class KoReaderImport(BaseFileImportMixin):
 
         self.mark_started()
         try:
-            scrobbles = process_koreader_sqlite_file(self.upload_file_path, self.user.id)
+            scrobbles = process_koreader_sqlite_file(
+                self.upload_file_path, self.user.id
+            )
             self.record_log(scrobbles)
         except Exception as e:
             self.record_error(f"Import failed: {e}")
@@ -272,7 +274,9 @@ class AudioScrobblerTSVImport(BaseFileImportMixin):
 
         self.mark_started()
         try:
-            scrobbles = import_audioscrobbler_tsv_file(self.upload_file_path, self.user.id)
+            scrobbles = import_audioscrobbler_tsv_file(
+                self.upload_file_path, self.user.id
+            )
             self.record_log(scrobbles)
         except Exception as e:
             self.record_error(f"Import failed: {e}")
@@ -608,6 +612,30 @@ class EBirdCSVImport(BaseFileImportMixin):
             self.mark_finished()
 
 
+TYPE_FK_PREFETCHES: dict[str, tuple[str, ...]] = {
+    "Video": ("video",),
+    "Track": ("track", "track__artist_fk"),
+    "PodcastEpisode": ("podcast_episode", "podcast_episode__podcast"),
+    "SportEvent": ("sport_event",),
+    "Book": ("book",),
+    "Paper": ("paper",),
+    "VideoGame": ("video_game",),
+    "BoardGame": ("board_game",),
+    "GeoLocation": ("geo_location",),
+    "Trail": ("trail",),
+    "Beer": ("beer",),
+    "Puzzle": ("puzzle",),
+    "Food": ("food",),
+    "Task": ("task",),
+    "WebPage": ("web_page",),
+    "LifeEvent": ("life_event",),
+    "Mood": ("mood",),
+    "BrickSet": ("brick_set",),
+    "Channel": ("channel",),
+    "BirdingLocation": ("birding_location",),
+}
+
+
 class ScrobbleQuerySet(models.QuerySet):
     def with_related(self):
         return self.select_related("user").prefetch_related(
@@ -634,6 +662,13 @@ class ScrobbleQuerySet(models.QuerySet):
             "birding_location",
         )
 
+    def with_related_for_types(self, media_types: list[str]):
+        prefetches = []
+        for t in media_types:
+            if t in TYPE_FK_PREFETCHES:
+                prefetches.extend(TYPE_FK_PREFETCHES[t])
+        return self.select_related("user").prefetch_related(*prefetches)
+
 
 class ShareViewLog(TimeStampedModel):
     scrobble = models.ForeignKey(
@@ -774,6 +809,7 @@ class Scrobble(TimeStampedModel):
                     "is_paused",
                 ]
             ),
+            models.Index(fields=["user", "-timestamp"]),
         ]
 
     @classmethod
@@ -806,11 +842,14 @@ class Scrobble(TimeStampedModel):
     @classmethod
     def as_dict_by_type(cls, scrobble_qs: models.QuerySet) -> dict:
         scrobbles_by_type = defaultdict(list)
-        scrobbles = (
-            scrobble_qs.with_related()
-            if hasattr(scrobble_qs, "with_related")
-            else scrobble_qs
-        )
+
+        if hasattr(scrobble_qs, "with_related"):
+            media_types_present = list(
+                scrobble_qs.values_list("media_type", flat=True).distinct()
+            )
+            scrobbles = scrobble_qs.with_related_for_types(media_types_present)
+        else:
+            scrobbles = scrobble_qs
 
         for scrobble in scrobbles:
             scrobbles_by_type[scrobble.media_type].append(scrobble)
@@ -825,7 +864,7 @@ class Scrobble(TimeStampedModel):
 
         # Remove any locations without titles
         if "GeoLocation" in scrobbles_by_type.keys():
-            for loc_scrobble in scrobbles_by_type["GeoLocation"]:
+            for loc_scrobble in list(scrobbles_by_type["GeoLocation"]):
                 if not loc_scrobble.media_obj.title:
                     scrobbles_by_type["GeoLocation"].remove(loc_scrobble)
                     scrobbles_by_type["GeoLocation_count"] -= 1
@@ -936,6 +975,8 @@ class Scrobble(TimeStampedModel):
             logdata_cls = logdata.BaseLogData
 
         log_dict = self.log
+        if isinstance(log_dict, logdata.BaseLogData):
+            log_dict = log_dict.asdict
         if isinstance(self.log, str):
             # There's nothing stopping django from saving a string in a JSONField :(
             logger.warning(
@@ -1412,20 +1453,18 @@ class Scrobble(TimeStampedModel):
                 or scrobble_data["playback_status"] == "stopped"
             ):
                 if read_log_page:
-                    page_list = scrobble.log.get("page_data", [])
-                    if page_list:
-                        for page in page_list:
+                    page_data = scrobble.log.get("page_data", {})
+                    if page_data:
+                        for page_num, page in page_data.items():
                             if not page.get("end_ts", None):
                                 page["end_ts"] = int(timezone.now().timestamp())
                                 page["duration"] = page["end_ts"] - page.get("start_ts")
 
-                    page_list.append(
-                        BookPageLogData(
-                            page_number=read_log_page,
-                            start_ts=int(timezone.now().timestamp()),
-                        )
+                    page_data[read_log_page] = BookPageLogData(
+                        page_number=read_log_page,
+                        start_ts=int(timezone.now().timestamp()),
                     )
-                    scrobble.log["page_data"] = page_list
+                    scrobble.log["page_data"] = page_data
                     scrobble.save(update_fields=["log"])
                 elif "log" in scrobble_data.keys() and scrobble.log:
                     scrobble_data["log"] = scrobble.log | scrobble_data["log"]
@@ -1436,13 +1475,13 @@ class Scrobble(TimeStampedModel):
 
         if read_log_page:
             scrobble_data["log"] = BookLogData(
-                page_data=[
-                    BookPageLogData(
+                page_data={
+                    read_log_page: BookPageLogData(
                         page_number=read_log_page,
                         start_ts=int(timezone.now().timestamp()),
                     )
-                ]
-            )
+                }
+            ).asdict
 
         logger.info(
             f"[scrobbling] creating new scrobble",
@@ -1457,7 +1496,7 @@ class Scrobble(TimeStampedModel):
             "calories", None
         ):
             if media.calories:
-                scrobble_data["log"] = FoodLogData(calories=media.calories)
+                scrobble_data["log"] = FoodLogData(calories=media.calories).asdict
 
         scrobble = cls.create(scrobble_data)
         return scrobble
@@ -1780,33 +1819,27 @@ class Scrobble(TimeStampedModel):
         return False
 
     def calculate_reading_stats(self, commit=True):
-        page_data = self.log.get("page_data")
+        page_data = self.logdata.page_data
 
         if page_data:
-            # --- Sort safely by numeric page_number ---
-            def safe_page_number(entry):
-                try:
-                    return int(getattr("page_number", entry), 0)
-                except (ValueError, TypeError):
-                    return float("inf")
-
             if isinstance(page_data, dict):
-                logger.warning("Page data is dict, migrate koreader data")
-                return
+                valid_pages = sorted(int(k) for k in page_data.keys())
+                if valid_pages:
+                    self.log["page_start"] = min(valid_pages)
+                    self.log["page_end"] = max(valid_pages)
+                    self.log["pages_read"] = len(set(valid_pages))
+            elif isinstance(page_data, list):
+                valid_pages = []
+                for page in page_data:
+                    try:
+                        valid_pages.append(int(page["page_number"]))
+                    except (ValueError, TypeError):
+                        continue
 
-            page_data.sort(key=safe_page_number)
-
-            valid_pages = []
-            for page in page_data:
-                try:
-                    valid_pages.append(int(page["page_number"]))
-                except (ValueError, TypeError):
-                    continue
-
-            if valid_pages:
-                self.log["page_start"] = min(valid_pages)
-                self.log["page_end"] = max(valid_pages)
-                self.log["pages_read"] = len(set(valid_pages))
+                if valid_pages:
+                    self.log["page_start"] = min(valid_pages)
+                    self.log["page_end"] = max(valid_pages)
+                    self.log["pages_read"] = len(set(valid_pages))
         else:
             page_start = self.log.get("page_start")
             page_end = self.log.get("page_end")
@@ -1846,9 +1879,7 @@ class FavoriteMedia(TimeStampedModel):
     birding_location = models.ForeignKey(
         BirdingLocation, on_delete=models.CASCADE, **BNULL
     )
-    media_type = models.CharField(
-        max_length=20, choices=Scrobble.MediaType.choices
-    )
+    media_type = models.CharField(max_length=20, choices=Scrobble.MediaType.choices)
     sent_to_mopidy = models.BooleanField(default=False)
 
     class Meta:

vrobbler/apps/scrobbles/scrobblers.py

@@ -357,10 +357,7 @@ def manual_scrobble_book(
 
     if action == "stop":
         if url:
-            if isinstance(scrobble.log, "BookLogData"):
-                scrobble.log.resume_url = next_url_if_exists(url)
-            else:
-                scrobble.log["resume_url"] = next_url_if_exists(url)
+            scrobble.log["resume_url"] = next_url_if_exists(url)
             scrobble.save(update_fields=["log"])
         scrobble.stop(force_finish=True)
 
@@ -540,6 +537,7 @@ def email_scrobble_board_game(
             "playback_position_seconds": duration_seconds,
             "source": "BG Stats",
             "log": log_data,
+            "visibility": "private",
         }
 
         scrobble = None
@@ -1152,7 +1150,7 @@ def web_scrobbler_scrobble_video_or_song(
     artist_name = data_dict.get("artist")
     track_name = data_dict.get("track")
     tracks = Track.objects.filter(
-        artist__name=data_dict.get("artist"), title=data_dict.get("track")
+        artist_fk__name=data_dict.get("artist"), title=data_dict.get("track")
     )
     if tracks.count() > 1:
         logger.warning(