[importers] Add error_log to surface import errors
All checks were successful
build / test (push) Successful in 2m2s
All checks were successful
build / test (push) Successful in 2m2s
This commit is contained in:
66
PROJECT.org
66
PROJECT.org
@ -88,7 +88,7 @@ fetching and simple saving.
|
||||
*** Metadata sources
|
||||
**** Scraper
|
||||
|
||||
* Backlog [0/12] :vrobbler:project:personal:
|
||||
* Backlog [1/16] :vrobbler:project:personal:
|
||||
** TODO [#C] Create small utility to clean up tracks scrobbled with wonky playback times :vrobbler:personal:bug:music:scrobbles:
|
||||
:PROPERTIES:
|
||||
:ID: 702462cf-d54b-48c6-8a7c-78b8de751deb
|
||||
@ -500,6 +500,70 @@ needed import celery task. This is how the WebDAV celery task currently works.
|
||||
This would also be an opporunity to clean up the code around WebDAV imports
|
||||
and make them more re-usable for other import services.
|
||||
|
||||
** TODO [#A] Generate a report of tracks with mistmatched metadata :music:tracks:metadata:
|
||||
:PROPERTIES:
|
||||
:ID: 684b8cd2-a3c1-4995-ba9e-7abdb02c37f2
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
We should have a management command that outputs a CSV file of track IDs where the log["raw_data"] artist
|
||||
does not match the actual track artist. Specific cases are
|
||||
|
||||
** TODO [#A] Add an exception list of artists as a constant that are exempted from splitting :music:artists:metadata:
|
||||
:PROPERTIES:
|
||||
:ID: fd86a11a-73ec-470d-b5e3-2d90ba9137c8
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
Certain artists like "Simon & Garfunkel" are actually one artist. While we don't want to mess with splitting up
|
||||
tracks into featured artists, we should have a "LITERAL_ARTIST_TITLES" constant that can have exceptions like
|
||||
this put into it and then we stop trying to pull the artist apart when we run into it.
|
||||
|
||||
** TODO [#A] Before enriching anything, trust the POST data :feature:scrobbles:metadata:
|
||||
:PROPERTIES:
|
||||
:ID: db6b05f8-09f4-49f5-9838-fbacc9fe9cd0
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
Both Jellyfin and Mopidy provide a decent amount of metadata when they POST to our webhooks.
|
||||
|
||||
In most cases, we should be able to trust this data to created music tracks or videos rather
|
||||
than going to third-party services to enrich. Thus, for tracks and videos we should search in
|
||||
the local database for imdb_id or musicbrainz_id for the specific content and, if found, not
|
||||
enrich further.
|
||||
|
||||
If not found, tracks and videos from mopidy and jellyfin should be created as completely as
|
||||
possible using only the POST data from the webhooks, tagged the scrobble with "webhook-metadata-only"
|
||||
and start the scrobble. A separate celery task should be kicked off to enrich the track or video
|
||||
async with the POST data stored in the log["raw_data"] and used by the celery enrichment task
|
||||
to go try to enrich the media instance. Should this enrichment fail, tag the scrobble as "enrichment-failed"
|
||||
log a warning and move on.
|
||||
|
||||
** DONE [#A] Date parsing failing in eBird imports :birds:ebird:importers:
|
||||
:PROPERTIES:
|
||||
:ID: 72e0254f-39d8-4843-9857-623e0362d77e
|
||||
:END:
|
||||
|
||||
*** Description
|
||||
|
||||
On line 45 in the apps/birds/importer.py file, the import is thorowing this
|
||||
error:
|
||||
|
||||
#+begin_src python
|
||||
ValueError: time data 'Jun 7, 2026, 5:15 PM' does not match format '%B %d, %Y %I:%M %p'
|
||||
#+end_src
|
||||
|
||||
Historically other files starting on May 24 worked, so I suspect this is a
|
||||
problem of the date formatter expecting Long month names and zero-padded days
|
||||
and the only sample we had was a three-letter month (May) and days with two
|
||||
digites(24 through the 31)
|
||||
|
||||
We should also add a "error_log" to the importers so that errors tha occur are
|
||||
surfaced, even when 0 successful files were processed. And we should make sure
|
||||
all importers do this as well.
|
||||
|
||||
* Version 48.0 [2/2]
|
||||
** DONE [#B] Show team or player images on sport detail and scrobble detail :sports:templates:
|
||||
|
||||
Reference in New Issue
Block a user