comunidadhll/docs/ranking-stats-performance-audit.md

Ranking And Stats Performance Audit

Scope

Audit date:

• 2026-06-09

Target endpoints:

• /api/ranking?timeframe=weekly&server_id=all&metric=kills&limit=20
• /api/ranking?timeframe=weekly&server_id=all&metric=kd_ratio&limit=20
• /api/ranking?timeframe=monthly&server_id=all&metric=kills_per_match&limit=20
• /api/ranking?timeframe=annual&year=2026&server_id=all&metric=kills&limit=20
• /api/stats/players/search?q=Chi&limit=10
• /api/stats/players/3530e19cf8a9dd6a9fada4599592fbf8?timeframe=weekly

Environment And Method

• Backend HTTP server was not running on http://127.0.0.1:8000 during the audit.
• Request timing was measured in-process through app.routes.resolve_get_payload(...).
• SQL timing was measured by instrumenting the SQLite read connections used by:
  • backend/app/rcon_historical_leaderboards.py
  • backend/app/rcon_historical_player_stats.py
  • backend/app/rcon_annual_rankings.py
• EXPLAIN QUERY PLAN was obtained successfully from SQLite.
• Storage inspected:
  • backend/data/hll_vietnam_dev.sqlite3

Important coverage note:

• The latest materialized match rows for source_basis="admin-log-match-ended" end on 2026-05-20T23:21:45.816Z.
• The weekly/monthly public requests on 2026-06-09 therefore read windows with 0 qualifying matches.
• Measured latency is real for the current dataset, but it understates future production cost because the active public windows are empty.

Measured Baseline

Endpoint	Avg request ms	Avg SQL ms	SQL queries	Result
/api/ranking weekly kills	4.045	0.882	6	snapshot_status=ready, items=0
/api/ranking weekly kd_ratio	3.801	0.837	6	snapshot_status=ready, items=0
/api/ranking monthly kills_per_match	3.612	0.816	6	snapshot_status=ready, items=0
/api/ranking annual kills	3.326	0.629	3	snapshot_status=ready, items=2
/api/stats/players/search?q=Chi&limit=10	6.731	3.593	3	items=10
/api/stats/players/{player_id} weekly	8.924	4.494	16	weekly profile payload

Slowest endpoint:

• /api/stats/players/{player_id} weekly
• Main reason: 16 SQL statements per request, including repeated window-count queries and two ranking subqueries.

Relevant Table Size And Coverage

Row counts:

• rcon_materialized_matches: 58
• rcon_match_player_stats: 3,824
• rcon_annual_ranking_snapshots: 2
• rcon_annual_ranking_snapshot_items: 2

Rows actually used by runtime ranking/stats:

• rcon_materialized_matches with source_basis="admin-log-match-ended": 22

Observed covered range for runtime ranking/stats:

• first materialized match end/start: 2026-05-19T11:16:10.281Z
• latest materialized match end/start: 2026-05-20T23:21:45.816Z

Current public window coverage:

• weekly window 2026-06-01T00:00:00Z to 2026-06-08T00:00:00Z: 0 matches
• monthly window 2026-06-01T00:00:00Z to 2026-06-09T23:59:59Z: 0 matches

Annual snapshot presence:

• 2026 / all-servers / kills / ready / limit_size=2 / source_matches_count=22

Existing Indexes

rcon_materialized_matches

• unique (target_key, match_key)
• non-unique (target_key, ended_at, ended_server_time)

rcon_match_player_stats

• unique (target_key, match_key, player_id)
• non-unique (target_key, match_key)

rcon_annual_ranking_snapshots

• unique (year, server_key, metric)
• non-unique (year, server_key, metric)
• non-unique (status)

rcon_annual_ranking_snapshot_items

• unique (snapshot_id, ranking_position)
• unique (snapshot_id, player_id)
• non-unique (snapshot_id, ranking_position)
• non-unique (snapshot_id, player_id)

Current Query Shapes

Weekly/monthly ranking runtime path:

• 4x COUNT(*) over rcon_materialized_matches to choose weekly/monthly windows
• 1x aggregate join from rcon_match_player_stats to rcon_materialized_matches
• 1x source-range query over rcon_materialized_matches

Annual ranking path:

• 1x snapshot lookup in rcon_annual_ranking_snapshots
• 1x item count in rcon_annual_ranking_snapshot_items
• 1x ordered item read in rcon_annual_ranking_snapshot_items

Stats player search path:

• 1x grouped search query joining rcon_match_player_stats to rcon_materialized_matches
• 1x latest-name lookup for returned player_id set
• 1x servers_seen lookup for returned player_id set

Stats player detail path:

• 12x COUNT(*) window-selection queries over rcon_materialized_matches
• 1x player aggregate query
• 1x source-range query
• 2x ranking-position subqueries using grouped leaderboard logic

Execution Plan Findings

SQLite EXPLAIN QUERY PLAN showed:

Ranking count queries:

• SCAN matches

Weekly/monthly leaderboard aggregate queries:

• SCAN matches
• SEARCH stats USING INDEX idx_rcon_match_player_stats_match (target_key=? AND match_key=?)
• USE TEMP B-TREE FOR GROUP BY
• USE TEMP B-TREE FOR count(DISTINCT)
• USE TEMP B-TREE FOR ORDER BY

Stats search primary query:

• SCAN matches
• SEARCH stats USING INDEX idx_rcon_match_player_stats_match (target_key=? AND match_key=?)
• USE TEMP B-TREE FOR GROUP BY
• USE TEMP B-TREE FOR count(DISTINCT)
• USE TEMP B-TREE FOR ORDER BY

Stats player detail aggregate query:

• SCAN stats
• SEARCH matches USING INDEX sqlite_autoindex_rcon_materialized_matches_1 (target_key=? AND match_key=?)
• USE TEMP B-TREE FOR count(DISTINCT)

Stats player detail ranking subquery:

• SCAN stats USING INDEX sqlite_autoindex_rcon_match_player_stats_1
• SEARCH matches USING INDEX sqlite_autoindex_rcon_materialized_matches_1 (target_key=? AND match_key=?)
• USE TEMP B-TREE FOR GROUP BY
• USE TEMP B-TREE FOR count(DISTINCT)
• USE TEMP B-TREE FOR ORDER BY

Annual snapshot lookup:

• SEARCH rcon_annual_ranking_snapshots USING INDEX sqlite_autoindex_rcon_annual_ranking_snapshots_1 (year=? AND server_key=? AND metric=?)

Root Cause Assessment

Current measured latency is low because the runtime weekly/monthly windows are empty on 2026-06-09, not because the runtime read path is already efficient at scale.

The most likely root causes for future slowdown are:

• no index that matches source_basis + time window on rcon_materialized_matches
• repeated public-request window counting against rcon_materialized_matches
• grouped leaderboard/search queries that spill to temp B-trees for GROUP BY, COUNT(DISTINCT) and ORDER BY
• no direct player_id read index for the stats player-detail aggregate path

The main non-performance operational issue exposed by the audit:

• runtime weekly/monthly ranking currently has stale coverage for the current public window because the latest materialized match data ends on 2026-05-20, while the requests were evaluated on 2026-06-09

Recommendation For TASK-189

Priority indexes justified by the current plans:

• add a time-window read index on rcon_materialized_matches keyed by source_basis plus the runtime time field used by public reads
• add a scoped time-window index on rcon_materialized_matches that also includes target_key
• add a direct player_id index on rcon_match_player_stats

Concrete candidate coverage to evaluate in implementation:

• rcon_materialized_matches(source_basis, ended_at)
• rcon_materialized_matches(source_basis, started_at)
• rcon_materialized_matches(target_key, source_basis, ended_at)
• rcon_materialized_matches(target_key, source_basis, started_at)
• rcon_match_player_stats(player_id)

Notes:

• (target_key, match_key) is already covered and should be preserved.
• annual snapshot item lookup is already covered by (snapshot_id, ranking_position) and (snapshot_id, player_id).
• a simple player_name B-tree is not likely to materially help the current search query because the query uses LOWER(...) LIKE '%term%' with a leading wildcard. If search becomes a real bottleneck later, a normalized-prefix strategy or FTS is more promising than a plain index.

TASK-189 Applied Indexes

Applied in storage initialization for both SQLite and PostgreSQL:

• idx_rcon_materialized_matches_source_window_text
• idx_rcon_materialized_matches_target_source_window_text
• idx_rcon_materialized_matches_external_source_window_text
• idx_rcon_match_player_stats_player_id_match

Not added:

• plain player_name index

Reason:

• the current search query uses LOWER(player_name) LIKE '%term%', so a simple B-tree on player_name is not expected to materially help the current search shape

TASK-189 Post-Index Check

Observed plan change:

• weekly count query now uses SEARCH matches USING COVERING INDEX idx_rcon_materialized_matches_source_window_text (source_basis=? AND <expr>>? AND <expr><?)
• stats player-detail aggregate now uses:
  • SEARCH matches USING INDEX idx_rcon_materialized_matches_source_window_text (...)
  • SEARCH stats USING INDEX sqlite_autoindex_rcon_match_player_stats_1 (target_key=? AND match_key=? AND player_id=?)

Before/after timing probe:

• /api/stats/players/{player_id} weekly
  • before: request 8.924 ms, SQL 4.494 ms
  • after: request 4.300 ms, SQL 1.043 ms
• /api/ranking weekly kills
  • before: request 4.045 ms, SQL 0.882 ms
  • after: request 4.724 ms, SQL 0.881 ms
  • interpretation: no meaningful change in this dataset because the current public weekly window on 2026-06-09 is empty and the table is still small

Residual gap after indexes:

• weekly/monthly public ranking still performs repeated runtime counting and grouped aggregation per request
• snapshots remain the correct next step for predictable public ranking latency

Recommendation For TASK-190

Weekly/monthly public ranking should move to snapshot-backed reads for the same reason annual ranking is already cheap:

• runtime ranking currently performs repeated window counting and grouped aggregation on every public request
• the empty-window result on 2026-06-09 hides that structural cost
• snapshot-backed weekly/monthly reads would make public ranking latency predictable and remove repeated runtime recomputation from the request path

Recommended transition policy:

• serve weekly/monthly ranking from snapshots when snapshot_status=ready
• keep runtime fallback controlled and optional during migration
• do not recalculate heavy weekly/monthly aggregates by default on every public request

Commands Executed

Repository and task context:

• Get-Content AGENTS.md
• Get-Content ai/architecture-index.md
• Get-Content ai/repo-context.md
• Get-Content ai/tasks/pending/TASK-188-audit-ranking-and-stats-query-performance.md
• Get-Content ai/orchestrator/backend-senior.md
• Get-Content ai/orchestrator/database-architect.md

Relevant code inspection:

• Get-Content backend/app/routes.py
• Get-Content backend/app/payloads.py
• Get-Content backend/app/rcon_historical_leaderboards.py
• Get-Content backend/app/rcon_historical_player_stats.py
• Get-Content backend/app/rcon_annual_rankings.py
• Get-Content backend/app/config.py
• Get-Content backend/app/postgres_rcon_storage.py
• Get-Content backend/app/sqlite_utils.py
• Get-Content docs/global-ranking-page-plan.md
• Get-Content scripts/run-stats-validation.ps1

Environment inspection:

• Get-ChildItem backend/data -Force
• try { (Invoke-WebRequest -UseBasicParsing -TimeoutSec 3 http://127.0.0.1:8000/health).Content } catch { $_.Exception.Message }

Measurement and database inspection:

• inline Python against backend/data/hll_vietnam_dev.sqlite3 for endpoint timing, SQL timing, row counts, PRAGMA index_list, PRAGMA index_info and EXPLAIN QUERY PLAN

Limitations

• No live HTTP timing was captured because the backend process was not running locally during the audit.
• The current runtime weekly/monthly windows were empty on 2026-06-09, so the measured latency is a lower bound, not a stressed production baseline.
• The database is large on disk because the repository stores many other domains, but the materialized ranking/stats tables used by these endpoints are currently small.