[tasks] Clean up fields and simplify form
This commit is contained in:
@ -1,5 +1,30 @@
|
|||||||
#+title: CHANGELOG
|
#+title: CHANGELOG
|
||||||
|
|
||||||
|
* Version 65.6 [1/1]
|
||||||
|
** DONE [#B] TODOIST_* variable names flagged as TODOs (false positives) :tasks:metadata:cleanup:
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: b2fd3053-ce26-4cdb-bf37-578387fef6c7
|
||||||
|
:END:
|
||||||
|
|
||||||
|
*** Description
|
||||||
|
|
||||||
|
Task logdata now uses generic, source-agnostic field names instead of
|
||||||
|
per-source prefixed keys. `TaskLogData` gained `source_id`, `project_id`,
|
||||||
|
`state` and `raw_data` (the full remaining source payload kept for future
|
||||||
|
reference, matching the music track `raw_data` convention); the old `orgmode_*`
|
||||||
|
and `todoist_*` keys are gone. The todoist and emacs scrobblers and webhooks
|
||||||
|
normalize into the generic fields, and all `log__orgmode_id` / `log__todoist_id`
|
||||||
|
lookups became `log__source_id`. `Task.source_url_for_user()` now reads
|
||||||
|
`source_id` (fixing the `todist_id` typo and the never-matching log `source`
|
||||||
|
check). New `convert_orgmode_task_log_to_generic` /
|
||||||
|
`convert_todoist_task_log_to_generic` helpers (wired into the
|
||||||
|
`convert_task_log_data` management command) migrate existing scrobble logs.
|
||||||
|
The scrobble "Edit Log" form no longer renders the read-only fields (labels,
|
||||||
|
source_id, project_id, state, raw_data) as disabled inputs — they are excluded
|
||||||
|
from the form and preserved on save — and the form now defaults to being
|
||||||
|
toggled open. Task labels render as read-only badges in the detail view
|
||||||
|
instead.
|
||||||
|
|
||||||
* Version 65.5 [2/2]
|
* Version 65.5 [2/2]
|
||||||
** DONE [#A] Fix database backup bug where running in celery task fails :bug:backups:celery:tasks:
|
** DONE [#A] Fix database backup bug where running in celery task fails :bug:backups:celery:tasks:
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
@ -34,35 +59,6 @@ The gitea action, Drone pipeline, Makefile deploy target, and the rc.d script
|
|||||||
itself are updated so every deploy restarts the three celery workers plus
|
itself are updated so every deploy restarts the three celery workers plus
|
||||||
celerybeat.
|
celerybeat.
|
||||||
|
|
||||||
* Version 65.4 [2/2]
|
|
||||||
** DONE [#B] Rename the default,charts celery worker to vrobbler_celery_charts :celery:deploy:
|
|
||||||
:PROPERTIES:
|
|
||||||
:ID: 92686951-8c87-4eae-8c48-1bed87567e93
|
|
||||||
:END:
|
|
||||||
|
|
||||||
*** Description
|
|
||||||
|
|
||||||
The FreeBSD rc.d worker that consumes the `default,charts` queues is renamed
|
|
||||||
from `vrobbler_celery` to `vrobbler_celery_charts`, matching the naming of its
|
|
||||||
companion workers (`vrobbler_celery_priority`, `vrobbler_celery_background`).
|
|
||||||
The gitea action, Drone pipeline, Makefile deploy target, and the rc.d script
|
|
||||||
itself are updated so every deploy restarts the three celery workers plus
|
|
||||||
celerybeat.
|
|
||||||
|
|
||||||
** DONE [#A] Fix database backup bug where running in celery task fails :bug:backups:celery:tasks:
|
|
||||||
:PROPERTIES:
|
|
||||||
:ID: 4117482f-4774-49b8-93b2-5b97adb194bd
|
|
||||||
:END:
|
|
||||||
|
|
||||||
*** Description
|
|
||||||
|
|
||||||
The `backup_database` celery task failed with `No such file or directory
|
|
||||||
'pg_dump'` even though the manual management command worked. FreeBSD rc.d
|
|
||||||
celery workers run under daemon(8) with a minimal PATH that omits
|
|
||||||
`/usr/local/bin`. The task now resolves `pg_dump` via `shutil.which()`,
|
|
||||||
falling back to the common PostgreSQL install locations, instead of relying
|
|
||||||
on the inherited PATH.
|
|
||||||
|
|
||||||
* Version 65.3 [1/1]
|
* Version 65.3 [1/1]
|
||||||
** DONE [#B] Trail scrobbles without names should try to use trial heads loc :trails:metadata:
|
** DONE [#B] Trail scrobbles without names should try to use trial heads loc :trails:metadata:
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
|
|||||||
47
PROJECT.org
47
PROJECT.org
@ -505,21 +505,6 @@ This may be a no-op situation as we've moved away from Google Books. But maybe w
|
|||||||
- =Book.find_or_create()= only matches by =original_title=, which is fragile.
|
- =Book.find_or_create()= only matches by =original_title=, which is fragile.
|
||||||
Should also match by Google Books ID or author name (like Track does).
|
Should also match by Google Books ID or author name (like Track does).
|
||||||
|
|
||||||
** TODO [#B] TODOIST_* variable names flagged as TODOs (false positives) :tasks:metadata:cleanup:
|
|
||||||
*** Description
|
|
||||||
|
|
||||||
Need to clean up how we store task data in logdata.
|
|
||||||
|
|
||||||
*** Implementation
|
|
||||||
|
|
||||||
Should clean up the task logdata dict so that we have fewer source-specific names, try to generalize task detail implementation from the input source, while keeping the full task payload for future reference.
|
|
||||||
|
|
||||||
- Files:
|
|
||||||
- ~vrobbler/settings.py~ (lines 72-73) -- =TODOIST_CLIENT_ID=, =TODOIST_CLIENT_SECRET=
|
|
||||||
- ~vrobbler/apps/scrobbles/constants.py~ (line 39) -- =TODOIST_TASK_URL=
|
|
||||||
- ~vrobbler/apps/tasks/models.py~ (line 13) -- =TODOIST_TASK_URL=
|
|
||||||
- These are variable/constant names for the Todoist integration, not actual
|
|
||||||
TODOs. No action needed, but worth noting they show up in searches.
|
|
||||||
** TODO [#B] Add AllTrails as a source for Trail data :trails:feature:
|
** TODO [#B] Add AllTrails as a source for Trail data :trails:feature:
|
||||||
:PROPERTIES:
|
:PROPERTIES:
|
||||||
:ID: 39313362-cdfe-46e7-bbd4-9139a65c0b3c
|
:ID: 39313362-cdfe-46e7-bbd4-9139a65c0b3c
|
||||||
@ -626,5 +611,37 @@ The Edit log form should have from top to bottom:
|
|||||||
- People (which should be similar to the Bird widget on BirdLocation and allow setting per user score, win true/false, rank, new true/false, seat_ordrer)
|
- People (which should be similar to the Bird widget on BirdLocation and allow setting per user score, win true/false, rank, new true/false, seat_ordrer)
|
||||||
- Expansion ids (which should a multi-select widget of expansions for this game)
|
- Expansion ids (which should a multi-select widget of expansions for this game)
|
||||||
- Location (which should be a drop down of BoardGameLocations for this user)
|
- Location (which should be a drop down of BoardGameLocations for this user)
|
||||||
|
** DONE [#B] TODOIST_* variable names flagged as TODOs (false positives) :tasks:metadata:cleanup:
|
||||||
|
:PROPERTIES:
|
||||||
|
:ID: b2fd3053-ce26-4cdb-bf37-578387fef6c7
|
||||||
|
:END:
|
||||||
|
*** Description
|
||||||
|
|
||||||
|
Need to clean up how we store task data in logdata.
|
||||||
|
|
||||||
|
*** Implementation
|
||||||
|
|
||||||
|
Task logdata now uses generic, source-agnostic field names instead of
|
||||||
|
per-source prefixed keys. `TaskLogData` gained `source_id`, `project_id`,
|
||||||
|
`state` and `raw_data` (the full remaining source payload kept for future
|
||||||
|
reference, matching the music track `raw_data` convention); the old
|
||||||
|
`orgmode_id`/`orgmode_state`/`orgmode_properties`/
|
||||||
|
`orgmode_drawers`/`orgmode_timestamps` and `todoist_id`/`todoist_project_id`
|
||||||
|
keys are gone. The todoist and emacs scrobblers and webhooks normalize into the
|
||||||
|
generic fields, and all `log__orgmode_id` / `log__todoist_id` lookups became
|
||||||
|
`log__source_id`. `Task.source_url_for_user()` now reads `source_id` (fixing the
|
||||||
|
`todist_id` typo and the never-matching log `source` check). New
|
||||||
|
`convert_orgmode_task_log_to_generic` / `convert_todoist_task_log_to_generic`
|
||||||
|
helpers (wired into the `convert_task_log_data` management command) migrate
|
||||||
|
existing scrobble logs. The scrobble "Edit Log" form no longer renders the
|
||||||
|
read-only fields (labels, source_id, project_id, state, raw_data) as disabled
|
||||||
|
inputs — they are excluded from the form and preserved on save — and the form
|
||||||
|
now defaults to being toggled open. Task labels render as read-only badges in
|
||||||
|
the detail view instead.
|
||||||
|
|
||||||
|
- Files:
|
||||||
|
- ~vrobbler/settings.py~ (lines 108-109) -- =TODOIST_CLIENT_ID=, =TODOIST_CLIENT_SECRET=
|
||||||
|
- ~vrobbler/apps/scrobbles/constants.py~ (line 74) -- =TODOIST_TASK_URL=
|
||||||
|
- These are variable/constant names for the Todoist integration, not actual
|
||||||
|
TODOs. No action needed, but worth noting they show up in searches.
|
||||||
|
|
||||||
|
|||||||
@ -105,7 +105,7 @@ def test_emacs_scrobble_update_task_stores_cleaned_notes():
|
|||||||
timestamp=datetime(2026, 6, 1, 10, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2026, 6, 1, 10, 0, tzinfo=timezone.utc),
|
||||||
log={
|
log={
|
||||||
"title": "My Org Task",
|
"title": "My Org Task",
|
||||||
"orgmode_id": "org-123",
|
"source_id": "org-123",
|
||||||
"notes": {},
|
"notes": {},
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|||||||
223
tests/tasks_tests/test_task_logdata.py
Normal file
223
tests/tasks_tests/test_task_logdata.py
Normal file
@ -0,0 +1,223 @@
|
|||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from django.contrib.auth import get_user_model
|
||||||
|
from django.urls import reverse
|
||||||
|
from scrobbles.models import Scrobble
|
||||||
|
from tasks.models import Task
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def user(db):
|
||||||
|
return get_user_model().objects.create(username="testuser")
|
||||||
|
|
||||||
|
|
||||||
|
def make_todoist_task():
|
||||||
|
return {
|
||||||
|
"source_id": "1234567890",
|
||||||
|
"labels": ["chore", "inprogress"],
|
||||||
|
"project_id": "9876543210",
|
||||||
|
"title": "Do the dishes",
|
||||||
|
"description": "Wash everything",
|
||||||
|
"updated_at": "2026-08-23T12:00:00",
|
||||||
|
"raw_data": {"todoist_type": "item", "todoist_event": "updated"},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def make_emacs_task():
|
||||||
|
return {
|
||||||
|
"source_id": "b2fd3053-ce26-4cdb-bf37-578387fef6c7",
|
||||||
|
"state": "STRT",
|
||||||
|
"labels": ["chore"],
|
||||||
|
"description": "Do the dishes",
|
||||||
|
"body": "Some details\n*** Description\nWash everything",
|
||||||
|
"notes": {},
|
||||||
|
"properties": {"ID": "b2fd3053-ce26-4cdb-bf37-578387fef6c7"},
|
||||||
|
"drawers": {"PROPERTIES": {"ID": "b2fd3053-ce26-4cdb-bf37-578387fef6c7"}},
|
||||||
|
"timestamps": ["2026-08-23 12:00"],
|
||||||
|
"updated_at": "2026-08-23T12:00:00",
|
||||||
|
"source": "Org-mode",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
class TestTodoistScrobbleTaskLogData:
|
||||||
|
def test_stores_generic_logdata(self, user):
|
||||||
|
from scrobbles.scrobblers import todoist_scrobble_task
|
||||||
|
|
||||||
|
scrobble = todoist_scrobble_task(
|
||||||
|
make_todoist_task(), user.id, started=True, user_context_list=["Chore"]
|
||||||
|
)
|
||||||
|
|
||||||
|
log = scrobble.log
|
||||||
|
assert log["source_id"] == "1234567890"
|
||||||
|
assert log["project_id"] == "9876543210"
|
||||||
|
assert log["title"] == "Do the dishes"
|
||||||
|
assert log["description"] == "Wash everything"
|
||||||
|
assert log["labels"] == ["chore"]
|
||||||
|
assert "inprogress" not in log["labels"]
|
||||||
|
assert log["raw_data"] == {
|
||||||
|
"todoist_type": "item",
|
||||||
|
"todoist_event": "updated",
|
||||||
|
}
|
||||||
|
assert "todoist_id" not in log
|
||||||
|
assert "todoist_label_list" not in log
|
||||||
|
|
||||||
|
def test_finish_matches_in_progress_scrobble_by_source_id(self, user):
|
||||||
|
from scrobbles.scrobblers import todoist_scrobble_task
|
||||||
|
|
||||||
|
task = Task.find_or_create("Chore")
|
||||||
|
Scrobble.objects.create(
|
||||||
|
user=user,
|
||||||
|
task=task,
|
||||||
|
media_type=Scrobble.MediaType.TASK,
|
||||||
|
source="Todoist",
|
||||||
|
in_progress=True,
|
||||||
|
timestamp=datetime(2026, 8, 23, 10, 0, tzinfo=timezone.utc),
|
||||||
|
log={"source_id": "1234567890", "title": "Chore"},
|
||||||
|
)
|
||||||
|
|
||||||
|
scrobble = todoist_scrobble_task(
|
||||||
|
make_todoist_task(), user.id, stopped=True, user_context_list=["Chore"]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert not scrobble.in_progress
|
||||||
|
assert scrobble.played_to_completion
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
class TestEmacsScrobbleTaskLogData:
|
||||||
|
def test_stores_generic_logdata(self, user):
|
||||||
|
from scrobbles.scrobblers import emacs_scrobble_task
|
||||||
|
|
||||||
|
scrobble = emacs_scrobble_task(
|
||||||
|
make_emacs_task(), user.id, started=True, user_context_list=["chore"]
|
||||||
|
)
|
||||||
|
|
||||||
|
log = scrobble.log
|
||||||
|
assert log["source_id"] == "b2fd3053-ce26-4cdb-bf37-578387fef6c7"
|
||||||
|
assert log["state"] == "STRT"
|
||||||
|
assert log["title"] == "Do the dishes"
|
||||||
|
assert log["description"] == "Wash everything"
|
||||||
|
assert log["raw_data"] == {
|
||||||
|
"properties": {"ID": "b2fd3053-ce26-4cdb-bf37-578387fef6c7"},
|
||||||
|
"drawers": {"PROPERTIES": {"ID": "b2fd3053-ce26-4cdb-bf37-578387fef6c7"}},
|
||||||
|
"timestamps": ["2026-08-23 12:00"],
|
||||||
|
"source": "Org-mode",
|
||||||
|
}
|
||||||
|
assert "orgmode_id" not in log
|
||||||
|
assert "orgmode_state" not in log
|
||||||
|
assert "orgmode_properties" not in log
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
class TestConvertTaskLogToGeneric:
|
||||||
|
def test_orgmode_conversion(self, user):
|
||||||
|
from vrobbler.apps.tasks.utils import (
|
||||||
|
convert_orgmode_task_log_to_generic,
|
||||||
|
)
|
||||||
|
|
||||||
|
task = Task.find_or_create("My Task")
|
||||||
|
scrobble = Scrobble.objects.create(
|
||||||
|
user=user,
|
||||||
|
task=task,
|
||||||
|
media_type=Scrobble.MediaType.TASK,
|
||||||
|
source="Org-mode",
|
||||||
|
in_progress=True,
|
||||||
|
timestamp=datetime(2026, 8, 23, 10, 0, tzinfo=timezone.utc),
|
||||||
|
log={
|
||||||
|
"title": "My Task",
|
||||||
|
"orgmode_id": "org-123",
|
||||||
|
"orgmode_state": "STRT",
|
||||||
|
"orgmode_properties": {"ID": "org-123"},
|
||||||
|
"orgmode_drawers": {"PROPERTIES": {"ID": "org-123"}},
|
||||||
|
"orgmode_timestamps": ["2026-08-23 12:00"],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
convert_orgmode_task_log_to_generic(commit=True)
|
||||||
|
|
||||||
|
scrobble.refresh_from_db()
|
||||||
|
log = scrobble.log
|
||||||
|
assert log["source_id"] == "org-123"
|
||||||
|
assert log["state"] == "STRT"
|
||||||
|
assert log["raw_data"] == {
|
||||||
|
"properties": {"ID": "org-123"},
|
||||||
|
"drawers": {"PROPERTIES": {"ID": "org-123"}},
|
||||||
|
"timestamps": ["2026-08-23 12:00"],
|
||||||
|
}
|
||||||
|
assert "orgmode_id" not in log
|
||||||
|
assert "orgmode_state" not in log
|
||||||
|
|
||||||
|
def test_todoist_conversion(self, user):
|
||||||
|
from vrobbler.apps.tasks.utils import (
|
||||||
|
convert_todoist_task_log_to_generic,
|
||||||
|
)
|
||||||
|
|
||||||
|
task = Task.find_or_create("My Task")
|
||||||
|
scrobble = Scrobble.objects.create(
|
||||||
|
user=user,
|
||||||
|
task=task,
|
||||||
|
media_type=Scrobble.MediaType.TASK,
|
||||||
|
source="Todoist",
|
||||||
|
in_progress=True,
|
||||||
|
timestamp=datetime(2026, 8, 23, 10, 0, tzinfo=timezone.utc),
|
||||||
|
log={
|
||||||
|
"title": "My Task",
|
||||||
|
"todoist_id": "1234567890",
|
||||||
|
"todoist_project_id": "9876543210",
|
||||||
|
"todoist_type": "item",
|
||||||
|
"todoist_event": "updated",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
convert_todoist_task_log_to_generic(commit=True)
|
||||||
|
|
||||||
|
scrobble.refresh_from_db()
|
||||||
|
log = scrobble.log
|
||||||
|
assert log["source_id"] == "1234567890"
|
||||||
|
assert log["project_id"] == "9876543210"
|
||||||
|
assert log["raw_data"] == {
|
||||||
|
"todoist_type": "item",
|
||||||
|
"todoist_event": "updated",
|
||||||
|
}
|
||||||
|
assert "todoist_id" not in log
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.django_db
|
||||||
|
class TestTaskEditLogForm:
|
||||||
|
def test_form_excludes_readonly_fields(self):
|
||||||
|
from tasks.models import TaskLogData
|
||||||
|
|
||||||
|
form = TaskLogData.form()
|
||||||
|
for excluded in ("labels", "source_id", "project_id", "state", "raw_data"):
|
||||||
|
assert excluded not in form.base_fields
|
||||||
|
|
||||||
|
def test_post_preserves_excluded_fields(self, user, client):
|
||||||
|
task = Task.objects.create(title="Test Task", description="Test description")
|
||||||
|
scrobble = Scrobble.objects.create(
|
||||||
|
user=user,
|
||||||
|
task=task,
|
||||||
|
media_type=Scrobble.MediaType.TASK,
|
||||||
|
log={
|
||||||
|
"notes": ["Original note"],
|
||||||
|
"description": "Original description",
|
||||||
|
"labels": ["work", "urgent"],
|
||||||
|
"source_id": "org-123",
|
||||||
|
"raw_data": {"properties": {"ID": "org-123"}},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
url = reverse("scrobbles:detail", kwargs={"pk": scrobble.id})
|
||||||
|
|
||||||
|
client.force_login(user)
|
||||||
|
response = client.post(
|
||||||
|
url,
|
||||||
|
{"description": "Updated description", "notes": "Updated note"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 302
|
||||||
|
|
||||||
|
scrobble.refresh_from_db()
|
||||||
|
assert scrobble.log["description"] == "Updated description"
|
||||||
|
assert scrobble.log["labels"] == ["work", "urgent"]
|
||||||
|
assert scrobble.log["source_id"] == "org-123"
|
||||||
|
assert scrobble.log["raw_data"] == {"properties": {"ID": "org-123"}}
|
||||||
@ -82,12 +82,12 @@ def form_from_dataclass(dataclass):
|
|||||||
form_fields[f.name] = override_fields[f.name]
|
form_fields[f.name] = override_fields[f.name]
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
if f.name in dataclass._excluded_fields:
|
||||||
|
continue
|
||||||
|
|
||||||
required = f.default is None and f.default_factory is None
|
required = f.default is None and f.default_factory is None
|
||||||
form_fields[f.name] = django_form_field_from_type(f.type, required=required)
|
form_fields[f.name] = django_form_field_from_type(f.type, required=required)
|
||||||
|
|
||||||
if f.name in dataclass._excluded_fields:
|
|
||||||
form_fields[f.name].disabled = True
|
|
||||||
|
|
||||||
form_cls = type(f"{dataclass.__name__}Form", (forms.Form,), form_fields)
|
form_cls = type(f"{dataclass.__name__}Form", (forms.Form,), form_fields)
|
||||||
|
|
||||||
if "notes" in form_cls.base_fields and "notes" not in override_fields:
|
if "notes" in form_cls.base_fields and "notes" not in override_fields:
|
||||||
|
|||||||
@ -1,7 +1,10 @@
|
|||||||
from django.core.management.base import BaseCommand
|
from django.core.management.base import BaseCommand
|
||||||
|
|
||||||
from vrobbler.apps.tasks.utils import (
|
from vrobbler.apps.tasks.utils import (
|
||||||
convert_old_boardgame_log_to_new,
|
convert_old_boardgame_log_to_new,
|
||||||
|
convert_orgmode_task_log_to_generic,
|
||||||
convert_tasks_notes_list_to_dict,
|
convert_tasks_notes_list_to_dict,
|
||||||
|
convert_todoist_task_log_to_generic,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@ -21,3 +24,5 @@ class Command(BaseCommand):
|
|||||||
print("No changes will be saved, use --commit to save")
|
print("No changes will be saved, use --commit to save")
|
||||||
convert_tasks_notes_list_to_dict(commit)
|
convert_tasks_notes_list_to_dict(commit)
|
||||||
convert_old_boardgame_log_to_new(commit)
|
convert_old_boardgame_log_to_new(commit)
|
||||||
|
convert_orgmode_task_log_to_generic(commit)
|
||||||
|
convert_todoist_task_log_to_generic(commit)
|
||||||
|
|||||||
@ -715,7 +715,7 @@ def todoist_scrobble_task_finish(
|
|||||||
) -> Optional[Scrobble]:
|
) -> Optional[Scrobble]:
|
||||||
scrobble = Scrobble.objects.filter(
|
scrobble = Scrobble.objects.filter(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
log__todoist_id=todoist_task.get("todoist_id"),
|
log__source_id=todoist_task.get("source_id"),
|
||||||
in_progress=True,
|
in_progress=True,
|
||||||
played_to_completion=False,
|
played_to_completion=False,
|
||||||
).first()
|
).first()
|
||||||
@ -737,7 +737,7 @@ def todoist_scrobble_update_task(
|
|||||||
scrobble = Scrobble.objects.filter(
|
scrobble = Scrobble.objects.filter(
|
||||||
in_progress=True,
|
in_progress=True,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
log__todoist_id=todoist_note.get("task_id"),
|
log__source_id=todoist_note.get("task_id"),
|
||||||
).first()
|
).first()
|
||||||
|
|
||||||
if not scrobble:
|
if not scrobble:
|
||||||
@ -775,16 +775,14 @@ def todoist_scrobble_task(
|
|||||||
stopped: bool = False,
|
stopped: bool = False,
|
||||||
user_context_list: list[str] = [],
|
user_context_list: list[str] = [],
|
||||||
) -> Scrobble:
|
) -> Scrobble:
|
||||||
title = get_title_from_labels(
|
title = get_title_from_labels(todoist_task.get("labels", []), user_context_list)
|
||||||
todoist_task.get("todoist_label_list", []), user_context_list
|
|
||||||
)
|
|
||||||
task = Task.find_or_create(title)
|
task = Task.find_or_create(title)
|
||||||
|
|
||||||
timestamp = pendulum.parse(todoist_task.pop("updated_at", timezone.now()))
|
timestamp = pendulum.parse(todoist_task.pop("updated_at", timezone.now()))
|
||||||
in_progress_scrobble = Scrobble.objects.filter(
|
in_progress_scrobble = Scrobble.objects.filter(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
in_progress=True,
|
in_progress=True,
|
||||||
log__todoist_id=todoist_task.get("todoist_id"),
|
log__source_id=todoist_task.get("source_id"),
|
||||||
task=task,
|
task=task,
|
||||||
).last()
|
).last()
|
||||||
|
|
||||||
@ -792,9 +790,9 @@ def todoist_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[todoist_scrobble_task] cannot stop already stopped task",
|
"[todoist_scrobble_task] cannot stop already stopped task",
|
||||||
extra={
|
extra={
|
||||||
"todoist_type": todoist_task["todoist_type"],
|
"todoist_type": todoist_task["raw_data"]["todoist_type"],
|
||||||
"todoist_event": todoist_task["todoist_event"],
|
"todoist_event": todoist_task["raw_data"]["todoist_event"],
|
||||||
"todoist_id": todoist_task["todoist_id"],
|
"todoist_id": todoist_task["source_id"],
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
@ -803,9 +801,9 @@ def todoist_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[todoist_scrobble_task] cannot start already started task",
|
"[todoist_scrobble_task] cannot start already started task",
|
||||||
extra={
|
extra={
|
||||||
"todoist_type": todoist_task["todoist_type"],
|
"todoist_type": todoist_task["raw_data"]["todoist_type"],
|
||||||
"todoist_event": todoist_task["todoist_event"],
|
"todoist_event": todoist_task["raw_data"]["todoist_event"],
|
||||||
"todoist_id": todoist_task["todoist_id"],
|
"todoist_id": todoist_task["source_id"],
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
return in_progress_scrobble
|
return in_progress_scrobble
|
||||||
@ -815,19 +813,15 @@ def todoist_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[todoist_scrobble_task] finishing",
|
"[todoist_scrobble_task] finishing",
|
||||||
extra={
|
extra={
|
||||||
"todoist_type": todoist_task["todoist_type"],
|
"todoist_type": todoist_task["raw_data"]["todoist_type"],
|
||||||
"todoist_event": todoist_task["todoist_event"],
|
"todoist_event": todoist_task["raw_data"]["todoist_event"],
|
||||||
"todoist_id": todoist_task["todoist_id"],
|
"todoist_id": todoist_task["source_id"],
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
return todoist_scrobble_task_finish(todoist_task, user_id, timestamp)
|
return todoist_scrobble_task_finish(todoist_task, user_id, timestamp)
|
||||||
|
|
||||||
todoist_task["title"] = todoist_task.pop("description")
|
labels = todoist_task.pop("labels", [])
|
||||||
todoist_task["description"] = todoist_task.pop("details")
|
|
||||||
labels = todoist_task.pop("todoist_label_list", [])
|
|
||||||
todoist_task["labels"] = [l for l in labels if l.lower() != "inprogress"]
|
todoist_task["labels"] = [l for l in labels if l.lower() != "inprogress"]
|
||||||
todoist_task.pop("todoist_type")
|
|
||||||
todoist_task.pop("todoist_event")
|
|
||||||
|
|
||||||
scrobble_dict = {
|
scrobble_dict = {
|
||||||
"user_id": user_id,
|
"user_id": user_id,
|
||||||
@ -925,7 +919,7 @@ def emacs_scrobble_update_task(
|
|||||||
scrobble = Scrobble.objects.filter(
|
scrobble = Scrobble.objects.filter(
|
||||||
in_progress=True,
|
in_progress=True,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
log__orgmode_id=emacs_id,
|
log__source_id=emacs_id,
|
||||||
source="Org-mode",
|
source="Org-mode",
|
||||||
).first()
|
).first()
|
||||||
|
|
||||||
@ -984,7 +978,7 @@ def emacs_scrobble_task(
|
|||||||
stopped: bool = False,
|
stopped: bool = False,
|
||||||
user_context_list: list[str] = [],
|
user_context_list: list[str] = [],
|
||||||
) -> Scrobble | None:
|
) -> Scrobble | None:
|
||||||
orgmode_id = task_data.get("source_id")
|
source_id = task_data.get("source_id")
|
||||||
title = get_title_from_labels(task_data.get("labels", []), user_context_list)
|
title = get_title_from_labels(task_data.get("labels", []), user_context_list)
|
||||||
|
|
||||||
task = Task.find_or_create(title)
|
task = Task.find_or_create(title)
|
||||||
@ -993,7 +987,7 @@ def emacs_scrobble_task(
|
|||||||
in_progress_scrobble = Scrobble.objects.filter(
|
in_progress_scrobble = Scrobble.objects.filter(
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
in_progress=True,
|
in_progress=True,
|
||||||
log__orgmode_id=orgmode_id,
|
log__source_id=source_id,
|
||||||
task=task,
|
task=task,
|
||||||
).last()
|
).last()
|
||||||
|
|
||||||
@ -1001,7 +995,7 @@ def emacs_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[emacs_scrobble_task] cannot stop already stopped task",
|
"[emacs_scrobble_task] cannot stop already stopped task",
|
||||||
extra={
|
extra={
|
||||||
"orgmode_id": orgmode_id,
|
"source_id": source_id,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
@ -1010,7 +1004,7 @@ def emacs_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[emacs_scrobble_task] cannot start already started task",
|
"[emacs_scrobble_task] cannot start already started task",
|
||||||
extra={
|
extra={
|
||||||
"orgmode_id": orgmode_id,
|
"source_id": source_id,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
return in_progress_scrobble
|
return in_progress_scrobble
|
||||||
@ -1020,7 +1014,7 @@ def emacs_scrobble_task(
|
|||||||
logger.info(
|
logger.info(
|
||||||
"[emacs_scrobble_task] finishing",
|
"[emacs_scrobble_task] finishing",
|
||||||
extra={
|
extra={
|
||||||
"orgmode_id": orgmode_id,
|
"source_id": source_id,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
in_progress_scrobble.stop(timestamp=timestamp, force_finish=True)
|
in_progress_scrobble.stop(timestamp=timestamp, force_finish=True)
|
||||||
@ -1036,12 +1030,13 @@ def emacs_scrobble_task(
|
|||||||
)
|
)
|
||||||
task_data["labels"] = task_data.pop("labels")
|
task_data["labels"] = task_data.pop("labels")
|
||||||
|
|
||||||
task_data["orgmode_id"] = task_data.pop("source_id")
|
task_data["state"] = task_data.pop("state")
|
||||||
task_data["orgmode_state"] = task_data.pop("state")
|
task_data["raw_data"] = {
|
||||||
task_data["orgmode_properties"] = task_data.pop("properties")
|
"properties": task_data.pop("properties"),
|
||||||
task_data["orgmode_drawers"] = task_data.pop("drawers")
|
"drawers": task_data.pop("drawers"),
|
||||||
task_data["orgmode_timestamps"] = task_data.pop("timestamps")
|
"timestamps": task_data.pop("timestamps"),
|
||||||
task_data.pop("source")
|
"source": task_data.pop("source"),
|
||||||
|
}
|
||||||
|
|
||||||
scrobble_dict = {
|
scrobble_dict = {
|
||||||
"user_id": user_id,
|
"user_id": user_id,
|
||||||
|
|||||||
@ -1414,10 +1414,11 @@ class ScrobbleDetailView(DetailView):
|
|||||||
if form.is_valid():
|
if form.is_valid():
|
||||||
data = form.cleaned_data.copy()
|
data = form.cleaned_data.copy()
|
||||||
|
|
||||||
for field_name, field in form.fields.items():
|
# Preserve log keys not editable in the form (e.g. _excluded_fields
|
||||||
if field.disabled:
|
# like source ids, raw payloads, or importer-only fields).
|
||||||
original_value = (self.object.log or {}).get(field_name)
|
for key, value in (self.object.log or {}).items():
|
||||||
data[field_name] = original_value
|
if key not in data:
|
||||||
|
data[key] = value
|
||||||
|
|
||||||
if data.get("with_people_ids") is not None:
|
if data.get("with_people_ids") is not None:
|
||||||
data["with_people_ids"] = [p.id for p in data["with_people_ids"]]
|
data["with_people_ids"] = [p.id for p in data["with_people_ids"]]
|
||||||
|
|||||||
@ -5,37 +5,29 @@ from typing import Optional
|
|||||||
from django.apps import apps
|
from django.apps import apps
|
||||||
from django.db import models
|
from django.db import models
|
||||||
from django.urls import reverse
|
from django.urls import reverse
|
||||||
|
from scrobbles.constants import TODOIST_TASK_URL
|
||||||
from scrobbles.dataclasses import BaseLogData
|
from scrobbles.dataclasses import BaseLogData
|
||||||
from scrobbles.mixins import LongPlayScrobblableMixin, ScrobblableConstants
|
from scrobbles.mixins import LongPlayScrobblableMixin, ScrobblableConstants
|
||||||
|
|
||||||
BNULL = {"blank": True, "null": True}
|
BNULL = {"blank": True, "null": True}
|
||||||
|
|
||||||
TODOIST_TASK_URL = "https://app.todoist.com/app/task/{id}"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class TaskLogData(BaseLogData):
|
class TaskLogData(BaseLogData):
|
||||||
title: Optional[str] = None
|
title: Optional[str] = None
|
||||||
labels: Optional[list[str]] = None
|
labels: Optional[list[str]] = None
|
||||||
|
|
||||||
orgmode_id: Optional[str] = None
|
source_id: Optional[str] = None
|
||||||
orgmode_state: Optional[str] = None
|
project_id: Optional[str] = None
|
||||||
orgmode_properties: Optional[dict] = None
|
state: Optional[str] = None
|
||||||
orgmode_drawers: Optional[list] = None
|
raw_data: Optional[dict] = None
|
||||||
orgmode_timestamps: Optional[list] = None
|
|
||||||
|
|
||||||
todoist_id: Optional[str] = None
|
|
||||||
todoist_project_id: Optional[str] = None
|
|
||||||
|
|
||||||
_excluded_fields = {
|
_excluded_fields = {
|
||||||
"labels",
|
"labels",
|
||||||
"orgmode_id",
|
"source_id",
|
||||||
"orgmode_state",
|
"project_id",
|
||||||
"orgmode_properties",
|
"state",
|
||||||
"orgmode_drawers",
|
"raw_data",
|
||||||
"orgmode_timestamps",
|
|
||||||
"todoist_id",
|
|
||||||
"todoist_project_id",
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@ -86,9 +78,25 @@ class TaskLogData(BaseLogData):
|
|||||||
|
|
||||||
md = markdown.Markdown(extensions=["extra"])
|
md = markdown.Markdown(extensions=["extra"])
|
||||||
allowed_tags = [
|
allowed_tags = [
|
||||||
"p", "br", "strong", "em", "a", "ul", "ol", "li",
|
"p",
|
||||||
"code", "pre", "blockquote", "h1", "h2", "h3", "h4", "h5", "h6",
|
"br",
|
||||||
"hr", "img",
|
"strong",
|
||||||
|
"em",
|
||||||
|
"a",
|
||||||
|
"ul",
|
||||||
|
"ol",
|
||||||
|
"li",
|
||||||
|
"code",
|
||||||
|
"pre",
|
||||||
|
"blockquote",
|
||||||
|
"h1",
|
||||||
|
"h2",
|
||||||
|
"h3",
|
||||||
|
"h4",
|
||||||
|
"h5",
|
||||||
|
"h6",
|
||||||
|
"hr",
|
||||||
|
"img",
|
||||||
]
|
]
|
||||||
|
|
||||||
notes = self.notes
|
notes = self.notes
|
||||||
@ -160,8 +168,8 @@ class Task(LongPlayScrobblableMixin):
|
|||||||
url = ""
|
url = ""
|
||||||
scrobble = self.scrobbles(user_id).first()
|
scrobble = self.scrobbles(user_id).first()
|
||||||
if scrobble:
|
if scrobble:
|
||||||
if scrobble.log.get("source") == "todoist":
|
if scrobble.source == "Todoist" and scrobble.logdata.source_id:
|
||||||
url = TODOIST_TASK_URL.format(id=scrobble.logdata.todist_id)
|
url = TODOIST_TASK_URL.format(id=scrobble.logdata.source_id)
|
||||||
return url
|
return url
|
||||||
|
|
||||||
def subtitle_for_user(self, user_id):
|
def subtitle_for_user(self, user_id):
|
||||||
|
|||||||
@ -63,6 +63,48 @@ def convert_old_todoist_log_to_new(commit=False):
|
|||||||
print(f"Updated {scrobbles.count()} todoist tasks logs")
|
print(f"Updated {scrobbles.count()} todoist tasks logs")
|
||||||
|
|
||||||
|
|
||||||
|
def convert_orgmode_task_log_to_generic(commit=False):
|
||||||
|
scrobbles = Scrobble.objects.filter(source="Org-mode", log__has_key="orgmode_id")
|
||||||
|
for scrobble in scrobbles:
|
||||||
|
scrobble.log["source_id"] = scrobble.log.pop("orgmode_id")
|
||||||
|
scrobble.log["state"] = scrobble.log.pop("orgmode_state")
|
||||||
|
|
||||||
|
raw_data = {}
|
||||||
|
for key, payload_key in (
|
||||||
|
("orgmode_properties", "properties"),
|
||||||
|
("orgmode_drawers", "drawers"),
|
||||||
|
("orgmode_timestamps", "timestamps"),
|
||||||
|
("orgmode_body", "body"),
|
||||||
|
):
|
||||||
|
if key in scrobble.log:
|
||||||
|
raw_data[payload_key] = scrobble.log.pop(key)
|
||||||
|
scrobble.log["raw_data"] = raw_data
|
||||||
|
|
||||||
|
print(f"Updating scrobble {scrobble.id}")
|
||||||
|
if commit:
|
||||||
|
scrobble.save(update_fields=["log"])
|
||||||
|
print(f"Updated {scrobbles.count()} orgmode task logs to generic logdata")
|
||||||
|
|
||||||
|
|
||||||
|
def convert_todoist_task_log_to_generic(commit=False):
|
||||||
|
scrobbles = Scrobble.objects.filter(source="Todoist", log__has_key="todoist_id")
|
||||||
|
for scrobble in scrobbles:
|
||||||
|
scrobble.log["source_id"] = scrobble.log.pop("todoist_id")
|
||||||
|
if "todoist_project_id" in scrobble.log:
|
||||||
|
scrobble.log["project_id"] = scrobble.log.pop("todoist_project_id")
|
||||||
|
|
||||||
|
raw_data = {}
|
||||||
|
for key in ("todoist_type", "todoist_event"):
|
||||||
|
if key in scrobble.log:
|
||||||
|
raw_data[key] = scrobble.log.pop(key)
|
||||||
|
scrobble.log["raw_data"] = raw_data
|
||||||
|
|
||||||
|
print(f"Updating scrobble {scrobble.id}")
|
||||||
|
if commit:
|
||||||
|
scrobble.save(update_fields=["log"])
|
||||||
|
print(f"Updated {scrobbles.count()} todoist task logs to generic logdata")
|
||||||
|
|
||||||
|
|
||||||
def convert_notes_to_dict(commit=False):
|
def convert_notes_to_dict(commit=False):
|
||||||
from scrobbles.utils import make_note_timestamp
|
from scrobbles.utils import make_note_timestamp
|
||||||
|
|
||||||
@ -129,9 +171,9 @@ def convert_old_boardgame_log_to_new(commit=False):
|
|||||||
notes = [notes]
|
notes = [notes]
|
||||||
if isinstance(notes, list):
|
if isinstance(notes, list):
|
||||||
key_ts = scrobble.stop_timestamp or scrobble.timestamp
|
key_ts = scrobble.stop_timestamp or scrobble.timestamp
|
||||||
scrobble.log["notes"] = {make_note_timestamp(key_ts): "\n".join(
|
scrobble.log["notes"] = {
|
||||||
str(n) for n in notes
|
make_note_timestamp(key_ts): "\n".join(str(n) for n in notes)
|
||||||
)}
|
}
|
||||||
count += 1
|
count += 1
|
||||||
if commit:
|
if commit:
|
||||||
scrobble.save(update_fields=["log"])
|
scrobble.save(update_fields=["log"])
|
||||||
|
|||||||
@ -2,11 +2,14 @@ import json
|
|||||||
import logging
|
import logging
|
||||||
|
|
||||||
from django.utils import timezone
|
from django.utils import timezone
|
||||||
from django.views.decorators.csrf import csrf_exempt
|
|
||||||
from django.utils.decorators import method_decorator
|
from django.utils.decorators import method_decorator
|
||||||
|
from django.views.decorators.csrf import csrf_exempt
|
||||||
from profiles.models import UserProfile
|
from profiles.models import UserProfile
|
||||||
from rest_framework import status
|
from rest_framework import status
|
||||||
from rest_framework.authentication import BaseAuthentication, TokenAuthentication
|
from rest_framework.authentication import (
|
||||||
|
BaseAuthentication,
|
||||||
|
TokenAuthentication,
|
||||||
|
)
|
||||||
from rest_framework.exceptions import AuthenticationFailed
|
from rest_framework.exceptions import AuthenticationFailed
|
||||||
from rest_framework.permissions import IsAuthenticated
|
from rest_framework.permissions import IsAuthenticated
|
||||||
from rest_framework.response import Response
|
from rest_framework.response import Response
|
||||||
@ -118,30 +121,35 @@ class TodoistWebhookView(APIView):
|
|||||||
|
|
||||||
if is_item_type and is_updated and (task_started or task_stopped):
|
if is_item_type and is_updated and (task_started or task_stopped):
|
||||||
todoist_task = {
|
todoist_task = {
|
||||||
"todoist_id": event_data.get("id"),
|
"source_id": event_data.get("id"),
|
||||||
"todoist_label_list": event_data.get("labels"),
|
"labels": event_data.get("labels"),
|
||||||
"todoist_type": todoist_type,
|
"project_id": event_data.get("project_id"),
|
||||||
"todoist_event": todoist_event,
|
"title": event_data.get("content"),
|
||||||
|
"description": event_data.get("description"),
|
||||||
"updated_at": event_data.get("updated_at"),
|
"updated_at": event_data.get("updated_at"),
|
||||||
"todoist_project_id": event_data.get("project_id"),
|
"raw_data": {
|
||||||
"description": event_data.get("content"),
|
"todoist_type": todoist_type,
|
||||||
"details": event_data.get("description"),
|
"todoist_event": todoist_event,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
if is_note_type and is_added:
|
if is_note_type and is_added:
|
||||||
task_data = event_data.get("item", {})
|
task_data = event_data.get("item", {})
|
||||||
todoist_note = {
|
todoist_note = {
|
||||||
"task_id": event_data.get("item_id"),
|
"task_id": event_data.get("item_id"),
|
||||||
"todoist_id": event_data.get("id"),
|
"source_id": event_data.get("id"),
|
||||||
"todoist_label_list": task_data.get("labels"),
|
"labels": task_data.get("labels"),
|
||||||
"todoist_type": todoist_type,
|
"title": task_data.get("content"),
|
||||||
"todoist_event": todoist_event,
|
"description": task_data.get("description"),
|
||||||
"updated_at": task_data.get("updated_at"),
|
"updated_at": task_data.get("updated_at"),
|
||||||
"posted_at": event_data.get("posted_at"),
|
"posted_at": event_data.get("posted_at"),
|
||||||
"details": task_data.get("description"),
|
|
||||||
"notes": event_data.get("content"),
|
"notes": event_data.get("content"),
|
||||||
"is_deleted": (
|
"is_deleted": (
|
||||||
True if event_data.get("is_deleted") == "true" else False
|
True if event_data.get("is_deleted") == "true" else False
|
||||||
),
|
),
|
||||||
|
"raw_data": {
|
||||||
|
"todoist_type": todoist_type,
|
||||||
|
"todoist_event": todoist_event,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
if (is_added and not todoist_note) or (is_updated and not todoist_task):
|
if (is_added and not todoist_note) or (is_updated and not todoist_task):
|
||||||
|
|||||||
@ -77,6 +77,13 @@
|
|||||||
</p>
|
</p>
|
||||||
<h2>{{ object.logdata.title }}</h2>
|
<h2>{{ object.logdata.title }}</h2>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
{% if object.media_type == "Task" and object.logdata.labels %}
|
||||||
|
<p>
|
||||||
|
{% for label in object.logdata.labels %}
|
||||||
|
<span class="badge bg-secondary me-1">{{ label }}</span>
|
||||||
|
{% endfor %}
|
||||||
|
</p>
|
||||||
|
{% endif %}
|
||||||
{% if object.media_type == "AgentSession" and object.logdata.title %}
|
{% if object.media_type == "AgentSession" and object.logdata.title %}
|
||||||
</p>
|
</p>
|
||||||
<h2>{{ object.logdata.title }}</h2>
|
<h2>{{ object.logdata.title }}</h2>
|
||||||
@ -300,7 +307,7 @@
|
|||||||
<button class="btn btn-secondary mb-3" type="button" data-bs-toggle="collapse" data-bs-target="#editLogForm">
|
<button class="btn btn-secondary mb-3" type="button" data-bs-toggle="collapse" data-bs-target="#editLogForm">
|
||||||
Edit Log
|
Edit Log
|
||||||
</button>
|
</button>
|
||||||
<div class="collapse mb-3" id="editLogForm">
|
<div class="collapse show mb-3" id="editLogForm">
|
||||||
<form method="post" class="needs-validation mt-3" novalidate>
|
<form method="post" class="needs-validation mt-3" novalidate>
|
||||||
{% csrf_token %}
|
{% csrf_token %}
|
||||||
{% for field in log_form %}
|
{% for field in log_form %}
|
||||||
|
|||||||
Reference in New Issue
Block a user