Add ranking snapshot performance path
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: TASK-188-audit-ranking-and-stats-query-performance
|
||||
title: Audit ranking and stats query performance
|
||||
status: pending
|
||||
status: done
|
||||
type: research
|
||||
team: Arquitecto de Base de Datos
|
||||
supporting_teams:
|
||||
@@ -85,12 +85,21 @@ Before completing the task ensure:
|
||||
|
||||
## Outcome
|
||||
|
||||
Document:
|
||||
|
||||
- measured baseline results
|
||||
- environment limitations
|
||||
- the most likely root cause of slow public reads
|
||||
- the follow-up recommendation that should feed `TASK-189` and `TASK-190`
|
||||
- Audit completed in `docs/ranking-stats-performance-audit.md`.
|
||||
- Measured baseline captured for the six required endpoint probes.
|
||||
- Environment limitation documented: backend HTTP server was not running locally, so request timing used in-process route resolution and SQL tracing over SQLite.
|
||||
- Current runtime weekly/monthly ranking windows on `2026-06-09` are empty because the latest materialized `admin-log-match-ended` data ends on `2026-05-20T23:21:45.816Z`.
|
||||
- Slowest measured endpoint is `/api/stats/players/{player_id}` because it issues 16 SQL statements, including repeated window counts and two ranking-position subqueries.
|
||||
- Primary future performance risks identified:
|
||||
- full scans on `rcon_materialized_matches` for `source_basis + time-window` filters
|
||||
- a full scan on `rcon_match_player_stats` for player-detail-by-`player_id`
|
||||
- temp B-trees for grouping, `COUNT(DISTINCT)` and ordering in leaderboard/search queries
|
||||
- Follow-up recommendation for `TASK-189`:
|
||||
- add time-window indexes on `rcon_materialized_matches`
|
||||
- add a direct `player_id` index on `rcon_match_player_stats`
|
||||
- keep annual snapshot indexes as-is
|
||||
- Follow-up recommendation for `TASK-190`:
|
||||
- move weekly/monthly public ranking to snapshot-backed reads with controlled runtime fallback
|
||||
|
||||
## Change Budget
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: TASK-189-add-ranking-materialized-read-indexes
|
||||
title: Add ranking materialized read indexes
|
||||
status: pending
|
||||
status: done
|
||||
type: backend
|
||||
team: Arquitecto de Base de Datos
|
||||
supporting_teams:
|
||||
@@ -80,12 +80,24 @@ Before completing the task ensure:
|
||||
|
||||
## Outcome
|
||||
|
||||
Document:
|
||||
|
||||
- indexes added
|
||||
- why each index was chosen
|
||||
- observed improvement or inability to measure it
|
||||
- residual performance gaps that still require snapshot-based reads
|
||||
- Added SQLite/PostgreSQL-compatible indexes in the materialized storage initialization path:
|
||||
- `idx_rcon_materialized_matches_source_window_text`
|
||||
- `idx_rcon_materialized_matches_target_source_window_text`
|
||||
- `idx_rcon_materialized_matches_external_source_window_text`
|
||||
- `idx_rcon_match_player_stats_player_id_match`
|
||||
- Kept existing annual snapshot indexes unchanged because the annual read path was already using matching indexes.
|
||||
- Did not add a plain `player_name` B-tree because the current search query uses `LOWER(player_name) LIKE '%term%'` with a leading wildcard, so the audit did not justify it as a useful narrow index.
|
||||
- Post-index validation confirmed plan improvement:
|
||||
- weekly count queries now use the new `source_basis + window` covering index
|
||||
- the stats player-detail aggregate now narrows through the match window index and then probes stats by `(target_key, match_key, player_id)`
|
||||
- Before/after timing was captured for representative endpoints:
|
||||
- `/api/stats/players/{player_id}` weekly improved from `8.924 ms / 4.494 ms SQL` to `4.300 ms / 1.043 ms SQL`
|
||||
- `/api/ranking` weekly `kills` showed no meaningful change in this dataset because the active weekly window on `2026-06-09` is empty
|
||||
- Validation scripts executed successfully:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- Remaining gap:
|
||||
- weekly/monthly public ranking still does repeated runtime counting and grouped aggregation per request, so snapshot-backed reads remain necessary in `TASK-190` and `TASK-191`
|
||||
|
||||
## Change Budget
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: TASK-190-design-weekly-monthly-ranking-snapshots
|
||||
title: Design weekly monthly ranking snapshots
|
||||
status: pending
|
||||
status: done
|
||||
type: documentation
|
||||
team: Arquitecto de Base de Datos
|
||||
supporting_teams:
|
||||
@@ -98,12 +98,34 @@ Before completing the task ensure:
|
||||
|
||||
## Outcome
|
||||
|
||||
Document:
|
||||
|
||||
- the proposed read-model schema
|
||||
- refresh policy
|
||||
- fallback policy
|
||||
- transition notes for implementation in `TASK-191`
|
||||
- Snapshot design documented in `docs/ranking-snapshot-read-model-plan.md`.
|
||||
- Proposed read model uses:
|
||||
- `ranking_snapshots`
|
||||
- `ranking_snapshot_items`
|
||||
- The plan explicitly covers:
|
||||
- `timeframe` weekly/monthly/annual
|
||||
- `server_id`
|
||||
- `metric`
|
||||
- `window_start`
|
||||
- `window_end`
|
||||
- `generated_at`
|
||||
- `source`
|
||||
- `snapshot_status`
|
||||
- `item_count`
|
||||
- `limit_size`
|
||||
- per-item ranking and player fields
|
||||
- Refresh policy defined:
|
||||
- weekly current every `5` to `15` minutes
|
||||
- monthly current every `15` to `30` minutes
|
||||
- previous week/month stable once closed
|
||||
- annual manual or daily
|
||||
- Fallback policy defined:
|
||||
- serve snapshot when `ready`
|
||||
- return controlled `missing` or use runtime fallback only by configuration when snapshot is absent
|
||||
- never recalculate by default on every public request
|
||||
- Transition notes prepared for `TASK-191`:
|
||||
- weekly/monthly snapshot-first read path
|
||||
- annual remains on the existing annual snapshot implementation until a dedicated migration task consolidates storage
|
||||
|
||||
## Change Budget
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: TASK-191-serve-ranking-from-snapshots-with-runtime-fallback
|
||||
title: Serve ranking from snapshots with runtime fallback
|
||||
status: pending
|
||||
status: done
|
||||
type: backend
|
||||
team: Backend Senior
|
||||
supporting_teams:
|
||||
@@ -99,12 +99,35 @@ Before completing the task ensure:
|
||||
|
||||
## Outcome
|
||||
|
||||
Document:
|
||||
|
||||
- final read-path behavior
|
||||
- fallback conditions
|
||||
- validation results
|
||||
- any remaining operational dependency for snapshot generation
|
||||
- Implemented weekly/monthly `/api/ranking` as snapshot-first:
|
||||
- snapshot `ready` rows are served from `ranking_snapshots` + `ranking_snapshot_items`
|
||||
- annual requests remain on the existing annual snapshot path
|
||||
- Added controlled runtime fallback behavior:
|
||||
- fallback is used only when the weekly/monthly snapshot is missing
|
||||
- fallback is controlled by `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED`
|
||||
- default remains enabled for transition
|
||||
- setting the variable to `false` returns controlled `snapshot_status='missing'` with empty items
|
||||
- Response metadata now distinguishes:
|
||||
- snapshot-ready
|
||||
- snapshot-missing
|
||||
- runtime-fallback
|
||||
- Metadata exposed on ranking responses now includes:
|
||||
- `source`
|
||||
- `snapshot_status`
|
||||
- `generated_at`
|
||||
- `freshness`
|
||||
- `fallback_used`
|
||||
- `window_start`
|
||||
- `window_end`
|
||||
- Validation completed:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- Validation script now proves:
|
||||
- normal route-contract behavior
|
||||
- snapshot-ready behavior using temporary fixture rows
|
||||
- snapshot-missing behavior with runtime fallback disabled
|
||||
- Remaining operational dependency:
|
||||
- snapshot tables now exist on first access, but snapshot rows are not generated automatically yet; a future generator/job still needs to populate weekly/monthly snapshots for production use
|
||||
|
||||
## Change Budget
|
||||
|
||||
Reference in New Issue
Block a user