Files

Emiliano Frascaro 52cfffb83f Aggiunge monitoraggio ops: health check, log scanner, dashboard e notifiche.

Introduce incidenti con dedup, job Sidekiq ogni 3 min, scan log cron, /up/deep,
dashboard /admin/ops e push ntfy; include Sentry opzionale e fix test suite.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-06-12 08:16:47 +02:00

3.0 KiB

Raw Blame History

Monitoraggio ops — Match Live TV

Sistema integrato per health check continui, analisi log, dashboard incidenti e notifiche push.

Componenti

Componente	Ruolo
`Ops::HealthMonitorJob`	Sidekiq — health check ogni ~3 min
`Ops::HealthChecks`	Disco, DB, Redis, MediaMTX, Garage, Sidekiq, HTTP pubblico
`Ops::LogScanner`	Pattern regex su log Docker (cron ogni 10 min)
`Ops::Incident`	DB — incidenti con deduplicazione
`Ops::Notifier`	Push ntfy + email opzionale
`/admin/ops`	Dashboard incidenti
`GET /up/deep`	Health check esteso (JSON)

Setup notifiche ntfy (telefono)

Installa l'app ntfy su Android/iOS
Scegli un topic segreto (es. matchlivetv-ops-eminux-a8f3b2)
Iscriviti al topic nell'app
Sul server, in infra/.env:

OPS_NTFY_URL=https://ntfy.sh/matchlivetv-ops-eminux-a8f3b2
OPS_NOTIFY_SEVERITIES=critical
OPS_NOTIFY_COOLDOWN_MINUTES=30

Test:

docker compose -f docker-compose.prod.yml exec -T rails bundle exec rails ops:test_notify

Variabili ambiente

Vedi infra/.env.production.example.

Variabile	Default	Descrizione
`OPS_NTFY_URL`	—	URL POST ntfy
`OPS_NTFY_TOKEN`	—	Bearer token (topic protetto)
`OPS_ALERT_EMAIL`	—	Fallback email
`OPS_HEALTH_INTERVAL_SECS`	180	Intervallo health job
`OPS_HEALTH_TOKEN`	—	Token per `/up/deep`
`OPS_DISK_CRIT_PERCENT`	90	Soglia critical disco
`OPS_LOG_SUBSCRIBER`	false	Cattura 500 in-process

Cron produzione

cd /opt/matchlivetv/infra && bash scripts/install_production_cron.sh

Schedule	Task
ogni 3 min (Sidekiq)	`Ops::HealthMonitorJob`
`/10 * * *`	`scan_ops_logs.sh` → `ops:ingest_logs`

Uptime Kuma (consigliato)

Watchdog esterno se Rails è completamente giù:

HTTP: https://www.matchlivetv.it/up
HTTP deep: https://www.matchlivetv.it/up/deep (header X-Ops-Token)
TCP: porta 1935 (RTMP)
Notifica: stesso topic ntfy

Workflow rilascio

cd backend && bundle exec rspec
Ciclo patch/test fino a 0 failure
git commit → git push
bash scripts/deploy/sync_to_server.sh
Rebuild: docker compose -f docker-compose.prod.yml up -d --build rails sidekiq
Migrate: docker compose exec -T rails bundle exec rails db:migrate
Cron: bash scripts/install_production_cron.sh
Smoke test:
- curl -sf https://www.matchlivetv.it/up
- bundle exec rails ops:health_check (nel container)
- Apri /admin/ops

Sentry (opzionale)

Imposta SENTRY_DSN dopo bundle install. Gli errori creano anche incidenti ops.

Troubleshooting

Problema	Azione
Nessuna push	Verifica `OPS_NTFY_URL`, test con `ops:test_notify`
Troppi alert	Aumenta `OPS_NOTIFY_COOLDOWN_MINUTES`, usa Mute 24h in admin
Sidekiq stale	Verifica container `sidekiq` e log Sidekiq
Log scanner vuoto	Controlla `/opt/matchlivetv/log/cron-ops.log`

3.0 KiB Raw Blame History