Add ranking snapshot performance path

This commit is contained in:
devRaGonSa
2026-06-09 08:13:42 +02:00
parent df84a11f53
commit 60334417d2
11 changed files with 1126 additions and 32 deletions

View File

@@ -1,7 +1,7 @@
---
id: TASK-188-audit-ranking-and-stats-query-performance
title: Audit ranking and stats query performance
status: pending
status: done
type: research
team: Arquitecto de Base de Datos
supporting_teams:
@@ -85,12 +85,21 @@ Before completing the task ensure:
## Outcome
Document:
- measured baseline results
- environment limitations
- the most likely root cause of slow public reads
- the follow-up recommendation that should feed `TASK-189` and `TASK-190`
- Audit completed in `docs/ranking-stats-performance-audit.md`.
- Measured baseline captured for the six required endpoint probes.
- Environment limitation documented: backend HTTP server was not running locally, so request timing used in-process route resolution and SQL tracing over SQLite.
- Current runtime weekly/monthly ranking windows on `2026-06-09` are empty because the latest materialized `admin-log-match-ended` data ends on `2026-05-20T23:21:45.816Z`.
- Slowest measured endpoint is `/api/stats/players/{player_id}` because it issues 16 SQL statements, including repeated window counts and two ranking-position subqueries.
- Primary future performance risks identified:
- full scans on `rcon_materialized_matches` for `source_basis + time-window` filters
- a full scan on `rcon_match_player_stats` for player-detail-by-`player_id`
- temp B-trees for grouping, `COUNT(DISTINCT)` and ordering in leaderboard/search queries
- Follow-up recommendation for `TASK-189`:
- add time-window indexes on `rcon_materialized_matches`
- add a direct `player_id` index on `rcon_match_player_stats`
- keep annual snapshot indexes as-is
- Follow-up recommendation for `TASK-190`:
- move weekly/monthly public ranking to snapshot-backed reads with controlled runtime fallback
## Change Budget

View File

@@ -1,7 +1,7 @@
---
id: TASK-189-add-ranking-materialized-read-indexes
title: Add ranking materialized read indexes
status: pending
status: done
type: backend
team: Arquitecto de Base de Datos
supporting_teams:
@@ -80,12 +80,24 @@ Before completing the task ensure:
## Outcome
Document:
- indexes added
- why each index was chosen
- observed improvement or inability to measure it
- residual performance gaps that still require snapshot-based reads
- Added SQLite/PostgreSQL-compatible indexes in the materialized storage initialization path:
- `idx_rcon_materialized_matches_source_window_text`
- `idx_rcon_materialized_matches_target_source_window_text`
- `idx_rcon_materialized_matches_external_source_window_text`
- `idx_rcon_match_player_stats_player_id_match`
- Kept existing annual snapshot indexes unchanged because the annual read path was already using matching indexes.
- Did not add a plain `player_name` B-tree because the current search query uses `LOWER(player_name) LIKE '%term%'` with a leading wildcard, so the audit did not justify it as a useful narrow index.
- Post-index validation confirmed plan improvement:
- weekly count queries now use the new `source_basis + window` covering index
- the stats player-detail aggregate now narrows through the match window index and then probes stats by `(target_key, match_key, player_id)`
- Before/after timing was captured for representative endpoints:
- `/api/stats/players/{player_id}` weekly improved from `8.924 ms / 4.494 ms SQL` to `4.300 ms / 1.043 ms SQL`
- `/api/ranking` weekly `kills` showed no meaningful change in this dataset because the active weekly window on `2026-06-09` is empty
- Validation scripts executed successfully:
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- Remaining gap:
- weekly/monthly public ranking still does repeated runtime counting and grouped aggregation per request, so snapshot-backed reads remain necessary in `TASK-190` and `TASK-191`
## Change Budget

View File

@@ -1,7 +1,7 @@
---
id: TASK-190-design-weekly-monthly-ranking-snapshots
title: Design weekly monthly ranking snapshots
status: pending
status: done
type: documentation
team: Arquitecto de Base de Datos
supporting_teams:
@@ -98,12 +98,34 @@ Before completing the task ensure:
## Outcome
Document:
- the proposed read-model schema
- refresh policy
- fallback policy
- transition notes for implementation in `TASK-191`
- Snapshot design documented in `docs/ranking-snapshot-read-model-plan.md`.
- Proposed read model uses:
- `ranking_snapshots`
- `ranking_snapshot_items`
- The plan explicitly covers:
- `timeframe` weekly/monthly/annual
- `server_id`
- `metric`
- `window_start`
- `window_end`
- `generated_at`
- `source`
- `snapshot_status`
- `item_count`
- `limit_size`
- per-item ranking and player fields
- Refresh policy defined:
- weekly current every `5` to `15` minutes
- monthly current every `15` to `30` minutes
- previous week/month stable once closed
- annual manual or daily
- Fallback policy defined:
- serve snapshot when `ready`
- return controlled `missing` or use runtime fallback only by configuration when snapshot is absent
- never recalculate by default on every public request
- Transition notes prepared for `TASK-191`:
- weekly/monthly snapshot-first read path
- annual remains on the existing annual snapshot implementation until a dedicated migration task consolidates storage
## Change Budget

View File

@@ -1,7 +1,7 @@
---
id: TASK-191-serve-ranking-from-snapshots-with-runtime-fallback
title: Serve ranking from snapshots with runtime fallback
status: pending
status: done
type: backend
team: Backend Senior
supporting_teams:
@@ -99,12 +99,35 @@ Before completing the task ensure:
## Outcome
Document:
- final read-path behavior
- fallback conditions
- validation results
- any remaining operational dependency for snapshot generation
- Implemented weekly/monthly `/api/ranking` as snapshot-first:
- snapshot `ready` rows are served from `ranking_snapshots` + `ranking_snapshot_items`
- annual requests remain on the existing annual snapshot path
- Added controlled runtime fallback behavior:
- fallback is used only when the weekly/monthly snapshot is missing
- fallback is controlled by `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED`
- default remains enabled for transition
- setting the variable to `false` returns controlled `snapshot_status='missing'` with empty items
- Response metadata now distinguishes:
- snapshot-ready
- snapshot-missing
- runtime-fallback
- Metadata exposed on ranking responses now includes:
- `source`
- `snapshot_status`
- `generated_at`
- `freshness`
- `fallback_used`
- `window_start`
- `window_end`
- Validation completed:
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- Validation script now proves:
- normal route-contract behavior
- snapshot-ready behavior using temporary fixture rows
- snapshot-missing behavior with runtime fallback disabled
- Remaining operational dependency:
- snapshot tables now exist on first access, but snapshot rows are not generated automatically yet; a future generator/job still needs to populate weekly/monthly snapshots for production use
## Change Budget