Generated: 2026-07-26T08:39:49.527356297Z | Window: 24h | Each log pattern includes up to 10 full entries for root cause tracing

CTO Triage Table

Service Health Matrix

Mobile Crashes (24h)

Per-Status Latency (Gap 1)

GPU Capacity (Gap 3)

Top Error Endpoints

Slowest Endpoints

Abnormal Log Patterns — Full Trace (20 patterns, up to 10 logs each)

Each pattern shows up to 10 raw log entries with timestamps, pod names, and full text for code tracing. Click "Show logs" to expand.

P	Signal	Tier	Impact	Code Entry Point
P1	Desktop-backend 5xx P99 90s timeout cascade	user_facing	Desktop users frozen for 90s	Desktop-backend upstream call chain
P1	iOS crashes 3,676 (153/hr)	critical	6.5x Android rate	Crashlytics → http_pool_manager.dart
P1	Backend-integration truncated responses (408x)	user_facing	Request timeouts / app exit	backend-integration Cloud Run timeout
P2	WebSocket auth: 627x Token expired + 24x cert errors	user_facing	Disconnected from transcription	utils/other/endpoints.py:WebSocket auth
P2	Conversation finalization: 196x stale reconciliation	user_facing	Orphaned conversations	services/conversation_finalization.py:135
P2	GPU capacity: 6/7 allocated, 1 remaining	user_facing	Transcription/diarization at risk if demand spikes	GKE autoscaler, node pools

Service	Status	Issues
desktop-backend	CRITICAL	No issues
backend-integration	CRITICAL	No issues
backend-sync	DEGRADED	No issues
pusher	DEGRADED	No issues
diarizer	HEALTHY	No issues
parakeet	HEALTHY	No issues
vad	HEALTHY	No issues
backend-listen	HEALTHY	No issues
agent-proxy	HEALTHY	No issues

Platform	Total	Per Hour
iOS	3,676	153/hr
Android	563	23/hr

Service	Status	Count	P50	P99	Analysis
desktop-backend	2xx	2,446	379ms	20.3s	Normal
	5xx	60	90.0s	90.2s	Gateway timeout — 100% are 90s waits (4.4x vs 2xx)
backend	2xx	40,767	45ms	2.8s	Normal
backend-sync	2xx	282	776ms	29.0s	Normal
backend-integration	2xx	1,362	11ms	1.3s	Normal

Metric	Value	Status
GPU Allocation	6 allocated / 7 total (1 remaining)	WARNING
VRAM (max)	23.5%	OK

Node (pool)	VRAM %
vad-pool-v2-47a302d1-71u5	23.5%
diarizer-pool-037ee134-qt5h	22.6%
parakeet-pool-c1f29adf-9lzh	16.3%
parakeet-pool-c1f29adf-kwxz	16.1%
vad-pool-v2-47a302d1-brjg	10.1%
nllb-translation-poo-bf276826-g5f6	5.1%
vad-pool-v2-47a302d1-5jr2	0.0%

Service	Path	Count	Tier
backend-integration	/v1/mcp/sse	35,114	internal
backend-integration	/v1/mcp/conversations?limit=3	30,346	internal
backend-sync	/v2/sync-local-files	27,019	user_facing
pusher	https://audio.soundbrain.cc/webhook	23,100	known_noise
pusher	https://omi-assist-api-prod.dakom.workers.dev/omi	20,316	known_noise
pusher	https://omi.supermemory.ai/webhook	10,548	known_noise
backend-sync	/v2/audio-merge-jobs/run	6,732	user_facing
backend-sync	/v2/sync-jobs/run	3,532	user_facing
backend-sync	/v2/sync-local-files	2,426	user_facing
backend-sync	/v1/conversation-finalization-jobs/run	2,113	user_facing

Service	Path	Avg Latency	Count
pusher	/v1/trigger/listen	17.5s	48,313
parakeet	/v2/transcribe	3.5s	91,823
parakeet	/v3/stream	12.0s	3,873
parakeet	/v1/transcribe	296ms	1,540

#1 [CRITICAL] backend-listen — 627x in 24h

utils.other.endpoints

ERROR:utils.other.endpoints:WebSocket auth failed: code=4001 error=Token expired, <HEX> < <HEX>

Show 10 logs ▼

#2 [CRITICAL] backend-integration — 408x in 24h

Truncated response body. Usually implies that the request timed out or the application exited before the response was fi

Show 10 logs ▼

#3 [CRITICAL] backend-integration — 127x in 24h

services.conversation_finalization

ERROR:services.conversation_finalization:stale processing conversation reconciliation query failed

Show 10 logs ▼

#4 [CRITICAL] backend-integration — 125x in 24h

/app/services/conversation_finalization.py:135

Traceback (most recent call last):

Show 10 logs ▼

[1] 2026-07-26 08:41:11 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[2] 2026-07-26 08:41:09 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[3] 2026-07-26 08:41:07 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[4] 2026-07-26 08:41:05 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[5] 2026-07-26 08:41:05 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[6] 2026-07-26 08:41:04 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[7] 2026-07-26 08:41:03 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[8] 2026-07-26 08:41:03 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[9] 2026-07-26 08:41:03 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[10] 2026-07-26 08:41:03 pod:backend-integrationTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir

#5 [WARNING] backend-sync — 72x in 24h

/app/services/conversation_finalization.py:135

Traceback (most recent call last):

Show 10 logs ▼

[1] 2026-07-26 08:39:40 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[2] 2026-07-26 08:39:40 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[3] 2026-07-26 08:39:40 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[4] 2026-07-26 08:39:40 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[5] 2026-07-26 08:39:40 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[6] 2026-07-26 08:38:56 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[7] 2026-07-26 08:38:29 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[8] 2026-07-26 08:34:39 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[9] 2026-07-26 08:34:39 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir
[10] 2026-07-26 08:34:39 pod:backend-syncTraceback (most recent call last):
  File "/app/services/conversation_finalization.py", line 135, in reconcile_stale_processing_conversations
    sweep = jobs_db.get_stale_processing_orphan_candidates(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/database/conversation_finalization_jobs.py", line 1055, in get_stale_processing_orphan_candidates
    page = list(query.stream())
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/google/cloud/fir

#6 [WARNING] backend-sync — 69x in 24h

services.conversation_finalization

ERROR:services.conversation_finalization:stale processing conversation reconciliation query failed

Show 10 logs ▼

#7 [WARNING] pusher — 61x in 24h

utils.app_integrations

ERROR:utils.app_integrations:App 01K8FRZA3XV2T4N1VD9CMTW8HV is not proactive_notification or data invalid CInQQMaPJvZfaP

Show 10 logs ▼

#8 [WARNING] backend-listen — 24x in 24h

utils.other.endpoints

ERROR:utils.other.endpoints:WebSocket auth failed: code=4001 error=Certificate for key id <HEX> not found.

Show 10 logs ▼

#9 [INFO] backend-listen — 14x in 24h

utils.stt.streaming

ERROR:utils.stt.streaming:Parakeet WS recv error: no close frame received or sent

Show 10 logs ▼

#10 [INFO] crashlytics — 13x in 24h

uid=<HEX> platform=ios device=iPhone17,2 version=1.0.544 package

uid=<HEX> platform=ios device=iPhone17,2 version=1.0.544 package:omi/backend/http/http_pool_manager.dart - HttpPoolManag

Show 3 logs ▼

#11 [INFO] pusher — 12x in 24h

utils.webhooks

ERROR:utils.webhooks:Error sending audio bytes to developer webhook:

Show 10 logs ▼

#12 [INFO] pusher — 9x in 24h

utils.app_integrations

ERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data in

Show 10 logs ▼

[1] 2026-07-26T08:45:08 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[2] 2026-07-26T08:45:03 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[3] 2026-07-26T08:44:58 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[4] 2026-07-26T08:44:56 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[5] 2026-07-26T08:44:50 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[6] 2026-07-26T08:44:49 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[7] 2026-07-26T08:44:45 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[8] 2026-07-26T08:44:43 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[9] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[10] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2

#13 [INFO] crashlytics — 8x in 24h

uid=<HEX> platform=ios device=iPhone18,2 version=1.0.544 package

uid=<HEX> platform=ios device=iPhone18,2 version=1.0.544 package:omi/backend/http/http_pool_manager.dart - HttpPoolManag

Show 3 logs ▼

#14 [INFO] pusher — 7x in 24h

utils.app_integrations

ERROR:utils.app_integrations:Plugin integration error:

Show 10 logs ▼

#15 [INFO] backend-sync — 6x in 24h

utils.llm.conversation_processing

ERROR:utils.llm.conversation_processing:Error determining memory discard: Invalid json output: discard = False

Show 10 logs ▼

#16 [INFO] backend-sync — 6x in 24h

utils.sync.pipeline

ERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknow

Show 10 logs ▼

[1] 2026-07-26 08:40:57 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[2] 2026-07-26 08:40:56 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[3] 2026-07-26 08:40:54 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[4] 2026-07-26 08:34:05 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[5] 2026-07-26 08:34:04 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[6] 2026-07-26 08:34:00 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[7] 2026-07-26 08:33:54 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[8] 2026-07-26 08:30:49 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[9] 2026-07-26 08:30:47 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio
[10] 2026-07-26 08:30:45 pod:backend-syncERROR:utils.sync.pipeline:event=sync_transcription_job outcome=invalid_input status=failed provider=unknown model=unknown lane=backfill reason_code=sync_invalid_audio

#17 [INFO] backend-listen — 5x in 24h

routers.listen.runtime

ERROR:routers.listen.runtime:Listen event delivery failed type=RuntimeError

Show 10 logs ▼

#18 [INFO] pusher — 5x in 24h

utils.app_integrations

ERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data in

Show 10 logs ▼

[1] 2026-07-26T08:45:08 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[2] 2026-07-26T08:45:03 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[3] 2026-07-26T08:44:58 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[4] 2026-07-26T08:44:56 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[5] 2026-07-26T08:44:50 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[6] 2026-07-26T08:44:49 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[7] 2026-07-26T08:44:45 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[8] 2026-07-26T08:44:43 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[9] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[10] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2

#19 [INFO] backend-sync — 5x in 24h

utils.conversations.finalizer

ERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=MVNZQHN42xeKJsXjfm2F5ZpK1sx1 conversa

Show 10 logs ▼

[1] 2026-07-26 08:38:59 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=pO8xDyw2Dqbya8SlnFbJADDEcPp1 conversation=d73bf6e3-ab9d-47ce-b18f-a2909a1d1239 failure=processing_failed
[2] 2026-07-26 08:23:36 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=MBCS9r3MamPDLFzBKVJnKSPfiGj1 conversation=f86c75e4-fa6a-4d1f-b877-fc656f9e4e25 failure=processing_failed
[3] 2026-07-26 08:19:34 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=78EXTWjud1exlKA6kfMZe3UKtEk2 conversation=bb736d59-6438-49c2-81c3-ad5eac520bef failure=processing_failed
[4] 2026-07-26 08:19:29 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=78EXTWjud1exlKA6kfMZe3UKtEk2 conversation=bb736d59-6438-49c2-81c3-ad5eac520bef failure=processing_failed
[5] 2026-07-26 08:19:18 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=78EXTWjud1exlKA6kfMZe3UKtEk2 conversation=bb736d59-6438-49c2-81c3-ad5eac520bef failure=processing_failed
[6] 2026-07-26 07:53:35 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=UP37FUOhldTcQnWs3caGMrx0pBH2 conversation=e9763e2b-6774-4cca-95cc-e373aa03f2bc failure=processing_failed
[7] 2026-07-26 07:52:48 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=pO8xDyw2Dqbya8SlnFbJADDEcPp1 conversation=13a27101-000b-4c0e-8c86-719845a00374 failure=processing_failed
[8] 2026-07-26 07:47:02 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=UP37FUOhldTcQnWs3caGMrx0pBH2 conversation=b50319ac-2163-44df-b93e-c1802bbffd81 failure=processing_failed
[9] 2026-07-26 07:46:53 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=rO8kKgieEDVkFrcOFrSELuoqNYu2 conversation=d53aac06-1ed1-40fe-9b4f-d70bf97509ed failure=processing_failed
[10] 2026-07-26 07:38:11 pod:backend-syncERROR:utils.conversations.finalizer:persisted conversation finalization failed uid=m29RMXeV3OafdZZSRgmYF6JcKXE2 conversation=ad09a63c-4530-4a81-963a-5620dcfed498 failure=processing_failed

#20 [INFO] pusher — 5x in 24h

utils.app_integrations

ERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data in

Show 10 logs ▼

[1] 2026-07-26T08:45:08 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[2] 2026-07-26T08:45:03 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[3] 2026-07-26T08:44:58 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[4] 2026-07-26T08:44:56 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[5] 2026-07-26T08:44:50 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[6] 2026-07-26T08:44:49 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[7] 2026-07-26T08:44:45 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[8] 2026-07-26T08:44:43 pod:pusher-58b94bc575-4b7qvINFO:utils.app_integrations:trigger_realtime_integrations supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ status: 500 results: Internal Server Error
[9] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2
[10] 2026-07-26T08:44:34 pod:pusher-58b94bc575-4b7qvERROR:utils.app_integrations:App supermemory-for-omi-01JE0442S4S20TRAHVYAXGQ2RQ is not proactive_notification or data invalid m29RMXeV3OafdZZSRgmYF6JcKXE2

Check	Status	Details
Grafana Alerts	CRITICAL	76 alert instances, 7 firing, 1 pending
Cloud Run 5xx	WARNING	8427 total 5xx across 10 services
Pod Lifecycle	CRITICAL	3061 events (Unhealthy: 1803, Killing: 324, Started: 323)
Autoscaler	WARNING	1134 events (1134 FailedScheduling, 0 scale failures)
Deploys	WARNING	18 deploy runs, 2 failed
Node Maint	OK	no events
HPA	OK	no rescales

Tier	Weight	Active Signals
critical_user_facing	3.0x	GPU exhaustion, transcription pipeline
user_facing	1.0x	Desktop timeouts, sync 5xx, WebSocket auth, iOS crashes
internal	0.3x	MCP SSE 401s, Cloud Armor
known_noise	0.1x	Pusher webhooks, firmware 404s

Gap	Status	Coverage
1. Per-status latency	LIVE	beast --per-status + Grafana 5 panels + 2 alerts
2. Route tiers	LIVE	beast --tier-filter + 19 route rules
3. GPU capacity	LIVE	beast --gpu + Grafana 10 panels + 3 alerts

Omi 24h Health Report — Full Trace Edition

CTO Triage Table

Service Health Matrix

Mobile Crashes (24h)

Per-Status Latency (Gap 1)

GPU Capacity (Gap 3)

Top Error Endpoints

Slowest Endpoints

Abnormal Log Patterns — Full Trace (20 patterns, up to 10 logs each)

Infrastructure Health

Signal Tier Classification (Gap 2)

Detection Gap Coverage