rgonsal/comunidadhll
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: persist and diagnose scoreboard correlation candidates

Browse Source
This commit is contained in:
devRaGonSa
2026-05-21 14:45:49 +02:00
parent 6383d08eba
commit f0820a1da7
13 changed files with 872 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

200
ai/tasks/done/TASK-147-persist-scoreboard-list-candidates.md Normal file
View File

@@ -0,0 +1,200 @@
---
id: TASK-147
title: Persist public scoreboard list matches as RCON candidates
status: done
type: backend
team: Backend Senior
supporting_teams:
- Arquitecto Python
roadmap_item: rcon-full-data
priority: high
---
# TASK-147 - Persist public scoreboard list matches as RCON candidates
## Goal
Make `scoreboard_candidate_backfill` persist safe rows into
`rcon_scoreboard_match_candidates` directly from `/api/get_scoreboard_maps`
list payloads, so public scoreboard URLs are available for RCON correlation
even if `/api/get_map_scoreboard` detail fetching fails, returns a different
shape, or lacks fields.
## Background
A real Foy match existed in the scoreboard public list:
- scoreboard id: `1562115`
- server_number: `2`
- server slug: `comunidad-hispana-02`
- map: Foy Warfare
- start: `2026-05-20T20:54:11+00:00`
- end: `2026-05-20T22:24:11+00:00`
- score: allied `4` / axis `1`
- expected URL: `https://scoreboard.comunidadhll.es:5443/games/1562115`
But it was not available as an RCON scoreboard candidate until manually
inserted. After manual insert and materialization, the match detail page
correctly showed `Ver en Scoreboard`.
The backfill currently reads `/api/get_scoreboard_maps`, collects IDs, fetches
detail payloads, and calls `upsert_historical_match` from detail payloads. The
list payload already contains enough trusted data for a safe candidate:
`id`, `start`, `end`, `map.id`, `map.pretty_name`, `result.allied`,
`result.axis`, and `server_number`.
## Constraints / DO NOT BREAK
- Do not remove PostgreSQL migration logic.
- Do not switch back to SQLite.
- Do not break recent match cards.
- Do not show `Ver partida` in recent cards.
- Do not change detail-page scoreboard button behavior except enabling it when
`match_url` exists.
- Do not expose unsafe URLs.
- Do not generate scoreboard URLs from player names.
- Do not commit runtime DB files.
## Allowed Changes
- `backend/app/scoreboard_candidate_backfill.py`
- Existing backend storage/read-model helpers needed for the safe candidate
upsert path.
- Existing trusted scoreboard origin helpers needed to preserve URL safety.
- Focused backend scoreboard tests and fixtures/mocks.
- This task file when it moves through the task workflow.
## Implementation Requirements
1. Inspect:
- `backend/app/scoreboard_candidate_backfill.py`
- `backend/app/postgres_rcon_storage.py`
- `backend/app/postgres_display_storage.py`
- `backend/app/historical_storage.py`
- `backend/app/rcon_admin_log_materialization.py`
- `backend/app/rcon_historical_read_model.py`
- `backend/app/scoreboard_origins.py`
2. Add or reuse a safe upsert function for
`rcon_scoreboard_match_candidates`.
3. During `scoreboard_candidate_backfill`, for every list match inside the
requested window:
- build a candidate from the list payload
- validate `server_number` / `server_slug` mapping
- build `match_url` only from trusted scoreboard base URL and numeric
external match id
- persist candidate idempotently into
`rcon_scoreboard_match_candidates`
4. This candidate upsert must happen before `fetch_match_details`.
5. `fetch_match_details` must remain only for historical match/player stat
enrichment.
6. If `fetch_match_details` fails, the candidate row must still exist.
7. Add counters to the JSON report:
- `list_candidates_inserted`
- `list_candidates_updated`
- `list_candidates_skipped`
- detail candidate/detail match counters if useful
8. Keep existing historical upsert behavior.
9. Preserve trusted URL allowlist behavior.
10. Do not create candidates for untrusted origins.
11. Do not reintroduce Comunidad Hispana #03 as an active visible target.
12. Add tests covering a list payload equivalent to Foy `1562115`.
13. The test must prove:
- list payload alone persists candidate
- `match_url` is
`https://scoreboard.comunidadhll.es:5443/games/1562115`
- failure of detail fetch does not prevent candidate persistence
- operation is idempotent
## Files to Read First
- `AGENTS.md`
- `ai/architecture-index.md`
- `ai/repo-context.md`
- `ai/orchestrator/backend-senior.md`
- `backend/app/scoreboard_candidate_backfill.py`
- `backend/app/postgres_rcon_storage.py`
## Expected Files to Modify
- Backfill/storage helper files needed for the safe list-candidate upsert.
- Focused scoreboard tests covering list-only persistence and detail failure.
- This task file after moving it through the workflow.
Keep the task within the allowed changes. If a storage boundary requires an
additional backend file from the inspected list, document why in the task
outcome.
## Validation
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"`
- `python -m app.storage_diagnostics`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- `docker compose --profile advanced up -d --build backend frontend postgres`
- `docker compose exec backend python -m app.scoreboard_candidate_backfill --server comunidad-hispana-02 --from 2026-05-20T00:00:00Z --to 2026-05-21T23:59:59Z --max-pages 5 --page-size 100`
- Query `rcon_scoreboard_match_candidates` and verify external match id
`1562115` exists for `comunidad-hispana-02`.
- Invoke the detail endpoint for
`comunidad-hispana-02:1779310451:1779315851:foywarfare` and verify
`match_url` is present.
- Review `git diff --name-only` and confirm the changed files match this task.
## Manual Verification
1. Open:
`http://localhost:8080/historico-partida.html?server=comunidad-hispana-02&match=comunidad-hispana-02%3A1779310451%3A1779315851%3Afoywarfare`
2. Verify `Ver en Scoreboard` appears.
3. Verify it opens:
`https://scoreboard.comunidadhll.es:5443/games/1562115`
4. Verify recent cards still do not show the public scoreboard button.
## Commit Message
`fix: persist scoreboard list matches as rcon candidates`
## Outcome
Document validation, candidate upsert decisions, counters added to the JSON
report, and any follow-up task instead of widening this task.
- Persisted list payload candidates before detail fetch through a focused
PostgreSQL upsert for `rcon_scoreboard_match_candidates`.
- Built list candidate URLs only from the trusted scoreboard origin catalog and
numeric public match IDs. List rows are accepted only when the selected
historical server slug/base URL/server number and payload `server_number`
agree with the trusted origin.
- Kept detail fetching for the existing `historical_*` match/player enrichment
path. The backfill report now exposes `list_candidates_inserted`,
`list_candidates_updated` and `list_candidates_skipped` independently from
the detail match counters.
- Added focused regression coverage for the Foy list row `1562115`; detail
fetch failure still leaves the list candidate present, and a second run
updates the same candidate key instead of creating a duplicate.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with pre-existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `python -m app.storage_diagnostics` from `backend/`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
passed; the script reports no product integration tests configured for its
platform-only scope.
- `docker compose --profile advanced up -d --build backend frontend postgres`
- `docker compose exec backend python -m app.scoreboard_candidate_backfill
--server comunidad-hispana-02 --from 2026-05-20T00:00:00Z --to
2026-05-21T23:59:59Z --max-pages 5 --page-size 100`
reported list candidate inserts/updates with no errors.
- Queried PostgreSQL and verified external match id `1562115` exists for
`comunidad-hispana-02` with URL
`https://scoreboard.comunidadhll.es:5443/games/1562115`.
- Invoked `/api/historical/matches/detail` for
`comunidad-hispana-02:1779310451:1779315851:foywarfare`; payload `found`
was true and `match_url` used the expected trusted URL.
- Scope review used `git diff --name-only`; changed implementation files stay
within this task plus this workflow task file.
## Change Budget
- Prefer fewer than 5 modified files.
- Prefer changes under 200 lines when feasible.
- Split follow-up work into a new task if the scope grows.

203
ai/tasks/done/TASK-148-relink-existing-rcon-matches-to-scoreboard-candidates.md Normal file
View File

@@ -0,0 +1,203 @@
---
id: TASK-148
title: Relink existing RCON matches to scoreboard candidates
status: done
type: backend
team: Backend Senior
supporting_teams:
- Arquitecto Python
roadmap_item: rcon-full-data
priority: high
---
# TASK-148 - Relink existing RCON matches to scoreboard candidates
## Goal
Add a deterministic relink flow that can update or resolve scoreboard URLs for
already-materialized RCON matches after new scoreboard candidates are inserted.
## Background
After manually inserting the Foy `1562115` candidate, the match detail page
showed the expected scoreboard button. Future backfills may insert candidates
after RCON matches already exist, so the system must relink existing RCON
matches with null or missing `match_url` dynamically or via materialization.
Known Foy regression target:
- RCON match key:
`comunidad-hispana-02:1779310451:1779315851:foywarfare`
- candidate external match id: `1562115`
- expected URL: `https://scoreboard.comunidadhll.es:5443/games/1562115`
## Constraints / DO NOT BREAK
- Do not relax trusted scoreboard URL allowlist.
- Do not link to unsafe origins.
- Do not choose ambiguous candidates silently.
- Do not break old Carentan match:
`comunidad-hispana-02:1779178461:1779183861:carentanwarfare`
- Do not break Kharkov:
`comunidad-hispana-02:1779315955:1779319098:kharkovwarfare`
- Do not re-add public scoreboard buttons to recent match cards.
- Do not expose correlation debug text in normal UI.
## Allowed Changes
- Existing RCON materialization, historical read-model, and RCON storage
modules needed for deterministic candidate relinking.
- `backend/app/scoreboard_candidate_backfill.py` only when needed to trigger
or document the relink sequence.
- A focused backend relink command module or a narrowly scoped command option
on existing materialization CLI.
- Focused backend scoreboard correlation tests and fixtures/mocks.
- This task file when it moves through the task workflow.
## Implementation Requirements
1. Inspect current correlation logic:
- `backend/app/rcon_admin_log_materialization.py`
- `backend/app/rcon_historical_read_model.py`
- `backend/app/postgres_rcon_storage.py`
- `backend/app/scoreboard_candidate_backfill.py`
2. Add or improve a function that matches RCON materialized matches to
`rcon_scoreboard_match_candidates` using:
- same server slug / external server id
- normalized map identity
- overlapping or close time window
- score allied/axis where present
- winner where present
3. Tolerances:
- allow reasonable time drift between RCON and scoreboard, at least
+/- 15 minutes, preferably configurable or centralized
- for matches with null `started_at` / `ended_at` but `closed_at` plus
`duration_seconds`, derive a correlation window
4. When a best safe candidate is found, make `match_url` available in detail
payload.
5. Prefer deterministic scoring:
- exact map match
- close end time
- score match
- same server
- reject ambiguous ties unless one candidate is clearly better
6. Add a command or option to relink existing matches without needing new
ingestion:
- acceptable command module:
`python -m app.rcon_scoreboard_relink`
- acceptable alternative: an option inside
`app.rcon_admin_log_materialization`
7. The relink command must print JSON summary:
- `matches_scanned`
- `candidates_scanned`
- `matches_linked`
- `matches_skipped_no_candidate`
- `matches_skipped_ambiguous`
- `errors`
8. Ensure `scoreboard_candidate_backfill` can optionally trigger relink at the
end, or document the command sequence.
9. Add regression coverage for the Foy case:
- RCON match key
`comunidad-hispana-02:1779310451:1779315851:foywarfare`
- candidate external match id `1562115`
- expected `match_url`
`https://scoreboard.comunidadhll.es:5443/games/1562115`
10. Ensure the detail endpoint returns `match_url` after relink.
## Files to Read First
- `AGENTS.md`
- `ai/architecture-index.md`
- `ai/repo-context.md`
- `ai/orchestrator/backend-senior.md`
- `backend/app/rcon_admin_log_materialization.py`
- `backend/app/rcon_historical_read_model.py`
## Expected Files to Modify
- Existing RCON correlation/materialization/storage code needed for relinking.
- One relink CLI entry point or one existing CLI option.
- Focused scoreboard correlation regression tests.
- This task file after moving it through the workflow.
Keep correlation diagnostics out of this task except the JSON relink summary
required here. Use TASK-149 for detailed diagnostic output and docs.
## Validation
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- `docker compose --profile advanced up -d --build backend frontend postgres`
- `docker compose exec backend python -m app.scoreboard_candidate_backfill --server comunidad-hispana-02 --from 2026-05-20T00:00:00Z --to 2026-05-21T23:59:59Z --max-pages 5 --page-size 100`
- Run the new relink command.
- Use `Invoke-WebRequest` for the Foy detail payload and verify `found` is
true and `match_url` is present.
- Use `Invoke-WebRequest` for the Carentan detail payload and verify the
existing `match_url` is still present.
- Review `git diff --name-only` and confirm the changed files match this task.
## Manual Verification
1. Foy detail page shows `Ver en Scoreboard`.
2. Scoreboard button opens the `1562115` public URL.
3. Recent match cards still do not show public scoreboard links.
4. No `Detalle no disponible` state appears for the validated detail page.
5. Player filters and external profile links still work.
## Commit Message
`fix: relink rcon matches to scoreboard candidates`
## Outcome
Document correlation scoring choices, time-window derivation, ambiguity
handling, relink command output, validation, and any follow-up task instead of
expanding scope.
- Added `python -m app.rcon_scoreboard_relink` to scan already-materialized
ended RCON matches against the same trusted candidate resolution used by the
detail read model. The command reports `matches_scanned`,
`candidates_scanned`, `matches_linked`, `matches_skipped_no_candidate`,
`matches_skipped_ambiguous` and `errors` as JSON.
- Kept URL availability dynamic in the read model instead of adding a second
persisted URL column to RCON matches. The new correlation resolution summary
preserves the existing deterministic map/time/duration/score scoring path,
returns explicit low-confidence/no-candidate/ambiguous reasons, and exposes
the selected external candidate only to command callers.
- Reused the existing materialized window derivation for closed-at plus
server-time duration cases so relink and detail payloads evaluate the same
effective correlation window.
- Added a Foy regression for materialized match key
`comunidad-hispana-02:1779310451:1779315851:foywarfare`: relink reports a
safe link and detail returns
`https://scoreboard.comunidadhll.es:5443/games/1562115`.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
passed; the script reports no product integration tests configured for its
platform-only scope.
- `docker compose --profile advanced up -d --build backend frontend postgres`
- Re-ran `app.scoreboard_candidate_backfill` for the requested May 20-21
window. Candidate refresh succeeded (`list_candidates_updated: 11`), while
detail enrichment returned a partial status because SQLite reported
`database is locked` for detail match `1562104`.
- `docker compose exec backend python -m app.rcon_scoreboard_relink --server
comunidad-hispana-02` reported 19 scanned matches, 13 linked matches, 6
no-candidate skips, 0 ambiguous skips and no errors.
- HTTP detail verification returned `found: true` and the expected Foy URL.
Existing Carentan detail key
`comunidad-hispana-02:1779178461:1779183861:carentanwarfare` also kept a
trusted `match_url`.
- Scope review used `git diff --name-only`; relink changes stay within the
correlation/read-model command path, focused scoreboard tests and this task
file.
## Change Budget
- Prefer fewer than 5 modified files.
- Prefer changes under 200 lines when feasible.
- Split follow-up work into a new task if the scope grows.

182
ai/tasks/done/TASK-149-scoreboard-correlation-diagnostics.md Normal file
View File

@@ -0,0 +1,182 @@
---
id: TASK-149
title: Add scoreboard correlation diagnostics
status: done
type: backend
team: Backend Senior
supporting_teams:
- PM
roadmap_item: rcon-full-data
priority: medium
---
# TASK-149 - Add scoreboard correlation diagnostics
## Goal
Make missing scoreboard links diagnosable without exposing debug clutter in the
normal UI.
## Background
When the Foy match had `match_url` null, investigation required manual
inspection of public pages, candidate tables, and historical tables. Backend
diagnostics should explain whether:
- no candidates exist
- candidates exist but map/time/score mismatch
- candidates were ambiguous
- candidate URL was unsafe
- relink was not run
The known Foy diagnostic target is:
- server: `comunidad-hispana-02`
- RCON match key:
`comunidad-hispana-02:1779310451:1779315851:foywarfare`
- expected candidate when available: external match id `1562115`
## Constraints / DO NOT BREAK
- Do not add visible debug text to `historico-partida.html`.
- Do not add timeline/source/debug metadata back to normal UI.
- Do not change recent card layout.
- Do not require internet for unit tests.
- Use fixtures/mocks for tests.
## Allowed Changes
- A focused backend scoreboard correlation diagnostic command.
- Existing backend correlation/read helpers only when needed to reuse safe
candidate selection output.
- Focused backend tests using fixtures or mocks.
- Documentation explaining the missing-scoreboard-button debug sequence.
- This task file when it moves through the task workflow.
## Implementation Requirements
1. Add backend diagnostic capability for a given RCON match:
- preferred command:
`python -m app.scoreboard_correlation_diagnostics --server comunidad-hispana-02 --match comunidad-hispana-02:1779310451:1779315851:foywarfare`
- optional alternative: also expose it inside `storage_diagnostics`, but
never in normal frontend payloads
2. Diagnostic output must be JSON.
3. Include:
- `rcon_match_key`
- `server`
- `map`
- `started_at` / `ended_at` / `closed_at` / `duration_seconds`
- score
- candidate search window
- `candidate_count`
- top candidate summaries:
- `external_match_id`
- `started_at`
- `ended_at`
- map
- score
- `match_url`
- `correlation_score`
- `rejection_reason` if rejected
- `selected_candidate` if any
- `final_reason`
4. Add a specific diagnostic for the known Foy case.
5. Add docs explaining how to debug a missing scoreboard button:
- run `scoreboard_candidate_backfill`
- run relink
- run `scoreboard_correlation_diagnostics`
- inspect detail endpoint
6. Keep diagnostics out of normal UI.
7. Do not emit raw sensitive data.
8. Do not emit unsafe URLs.
## Files to Read First
- `AGENTS.md`
- `ai/architecture-index.md`
- `ai/repo-context.md`
- `ai/orchestrator/backend-senior.md`
- `backend/app/rcon_historical_read_model.py`
- Backend correlation/relink code delivered by TASK-148.
## Expected Files to Modify
- One backend diagnostic command module or the existing diagnostics module.
- Existing safe correlation/read helper files only if necessary.
- Focused backend diagnostic tests using fixtures/mocks.
- One focused documentation file for the debug workflow.
- This task file after moving it through the workflow.
Do not add frontend debug surfaces or widen the normal detail payload for this
task.
## Validation
- `python -m compileall backend/app`
- `python -m app.scoreboard_correlation_diagnostics --server comunidad-hispana-02 --match comunidad-hispana-02:1779310451:1779315851:foywarfare`
- `python -m app.storage_diagnostics`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- `node --check frontend/assets/js/historico-partida.js`
- `node --check frontend/assets/js/historico.js`
- `node --check frontend/assets/js/historico-recent-live.js`
- Review `git diff --name-only` and confirm the changed files match this task.
## Manual Verification
1. Run diagnostics for Foy and verify it explains why `1562115` is selected or
why no candidate is selected.
2. Run diagnostics for a match without public candidate and verify it explains
no candidates found.
3. Normal frontend pages do not show diagnostics.
## Commit Message
`chore: add scoreboard correlation diagnostics`
## Outcome
Document diagnostic fields, known Foy output behavior, documentation added,
validation performed, and any follow-up task instead of expanding normal UI
payloads.
- Added `python -m app.scoreboard_correlation_diagnostics` for one
materialized RCON match. JSON output includes match key, server, map,
timestamps, duration, score, candidate search window, safe top candidates,
selected candidate and `final_reason`.
- Candidate summaries expose trusted public `match_url` values only. Untrusted
URLs are omitted and receive `unsafe-url`; non-scoring candidates report
`map-or-window-mismatch` where applicable.
- Kept diagnostics out of normal detail and frontend payloads. The command
reuses the materialized correlation window and safe candidate scoring that
relink/detail use.
- Added `docs/scoreboard-correlation-debugging.md` with the missing-button
sequence: candidate backfill, relink scan, diagnostics command and detail
endpoint check.
- Extended focused Foy coverage so diagnostics select external match id
`1562115` with the expected Foy candidate summary.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `python -m app.scoreboard_correlation_diagnostics --server
comunidad-hispana-02 --match
comunidad-hispana-02:1779310451:1779315851:foywarfare` from `backend/`
returned `final_reason: linked` and selected candidate `1562115`.
- `python -m app.storage_diagnostics`
- `node --check frontend/assets/js/historico-partida.js`
- `node --check frontend/assets/js/historico.js`
- `node --check frontend/assets/js/historico-recent-live.js`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
exited successfully after reporting its platform checks. Its later local
route probe printed a SQLite `database disk image is malformed` traceback
from `historical_storage`; this task did not alter or repair runtime DB
files.
- Scope review used `git diff --name-only`; diagnostics changes stay in the
correlation command/test/doc path and this task file.
## Change Budget
- Prefer fewer than 5 modified files.
- Prefer changes under 200 lines when feasible.
- Split follow-up work into a new task if the scope grows.