From 7f1aee3d4aad7ef1bbc1a042c6aa0a66a457630d Mon Sep 17 00:00:00 2001 From: MingxuanGame Date: Sun, 14 Sep 2025 07:58:37 +0000 Subject: [PATCH] docs(agent): update agent instruction --- .github/copilot-instructions.md | 49 +-------- AGENTS.md | 187 ++++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 48 deletions(-) mode change 100644 => 120000 .github/copilot-instructions.md create mode 100644 AGENTS.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md deleted file mode 100644 index f496480..0000000 --- a/.github/copilot-instructions.md +++ /dev/null @@ -1,48 +0,0 @@ -## g0v0-server – AI Coding Agent Quick Guide - -Focus: High‑performance osu! (v1 / v2 / lazer) API emulator. Python (FastAPI + SQLModel/MySQL + Redis + APScheduler) + Rust (`packages/msgpack_lazer_api` for MessagePack encode/decode). - -### Core Architecture (know these before editing) -1. Entry: `main.py` – FastAPI app + lifespan startup/shutdown orchestration (fetcher init, GeoIP, schedulers, cache/email/download health checks, Redis message system, stats, achievements). Add new long‑running startup logic here only if it is foundational; otherwise prefer a scheduler or on‑demand service. -2. Routers (`app/router/`): - - `v1/` & `v2/`: Must mirror official endpoints only (no custom additions). Keep prefix via parent `router = APIRouter(prefix="/api/v2")` etc. - - `notification/`: Chat & notification subset (also “official only”). - - `auth.py`: Auth / token flows. - - `private/`: Server‑specific or internal endpoints (place all custom APIs here; split by concern with small modules + include via `private/router.py`). -3. Services (`app/service/`): Stateless or stateful domain logic (pp calc, rankings, caching, daily challenge, email, geoip, etc.). If you need background computation, create a service + (optionally) scheduler wrapper. -4. Schedulers (`app/scheduler/`): Start/stop functions named `start_*_scheduler` + matching `stop_*`. Lifespan wires them; stay consistent (see `cache_scheduler`, `database_cleanup_scheduler`). -5. Database layer (`app/database/`): SQLModel models; large models (e.g. `score.py`) embed domain helpers & serialization validators. When adding/altering tables: generate Alembic migration (autogenerate) & review for types / indexes. -6. Caching: Redis accessed via dependency (`app/dependencies/database.get_redis`). High‑traffic objects (users, user scores) use `UserCacheService` with explicit key patterns (`user:{id}...`). Reuse existing service; do not invent new ad‑hoc keys for similar data. -7. Rust module: `packages/msgpack_lazer_api` – performance critical (MessagePack for lazer). After Rust edits: `maturin develop -R` (release) inside dev container; Python interface consumed via import `msgpack_lazer_api` (see usage in encode/decode paths – keep function signatures stable; update `.pyi` if API changes). - -### Common Task Playbooks -Add v2 endpoint: create file under `app/router/v2/`, import router via `from .router import router`, define FastAPI path ops (async) using DB session (`session: Database`) & caching patterns; avoid blocking calls. Do NOT modify existing prefixes; do NOT add non‑official endpoints here (place in `private`). -Add background job: put pure logic in `app/service/_job.py` (idempotent / retry‑safe), add small scheduler starter in `app/scheduler/_scheduler.py` exposing `start__scheduler()` & `stop__scheduler()`, then register in `main.lifespan` near similar tasks. -DB schema change: edit / add SQLModel in relevant module, then: `alembic revision --autogenerate -m "feat(db): add "` → inspect migration (indexes, enums) → `alembic upgrade head`. -pp / ranking recalculation: standalone scriptable path via env `RECALCULATE=true python main.py` (see `app/service/recalculate.py`). Keep heavy loops async + batched (`asyncio.gather`) like `_recalculate_pp` pattern. -User data responses: Use `UserResp.from_db(...)` and cache asynchronously (`background_task.add_task(cache_service.cache_user, user_resp)`). Maintain ≤50 result limits where existing code does (see `v2/user.py`). - -### Configuration & Safety -Central settings: `app/config.py` (pydantic-settings). Secrets default to unsafe placeholders; lifespan logs warnings if not overridden (`secret_key`, `osu_web_client_secret`). When introducing new config flags: add field + defaults, document in README tables, and reference via `settings.` (avoid reading env directly elsewhere). - -### Conventions / Quality -Commit messages: Angular format (type(scope): subject). Types: feat/fix/docs/style/refactor/perf/test/chore/ci. -Async only in request paths (no blocking I/O). Use existing redis/service helpers instead of inline logic. Prefer model methods / helper functions near existing ones for cohesion. -Cache keys: follow established prefixes (`user:` / `v1_user:` / `user:{id}:scores:`). Invalidate via provided service methods. -Error handling: Raise `HTTPException` for client issues; wrap external fetch retries (see `_recalculate_pp` loop with capped retries + sleep). Avoid silent except; log via `logger`. - -### Tooling / Commands -Setup: `uv sync` → `pre-commit install`. -Run dev server: `uvicorn main:app --reload --host 0.0.0.0 --port 8000` (logs managed by custom logger; uvicorn default log config disabled). -Rust rebuild: `maturin develop -R`. -Migrations: `alembic revision --autogenerate -m "feat(db): ..."` → `alembic upgrade head`. - -### When Modifying Large Files (e.g. `score.py`) -Keep validation / serializer patterns (field_validator / field_serializer). Add new enum-like JSON fields with both validator + serializer for forward compatibility. - -### PR Scope Guidance -One concern per change (e.g., add endpoint OR refactor cache logic—not both). Update README config tables if adding env vars. - -Questions / ambiguous patterns: prefer aligning with closest existing service; if still unclear, surface a clarification comment instead of introducing a new pattern. - -— End of instructions — diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 120000 index 0000000..be77ac8 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1 @@ +../AGENTS.md \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..3ddac59 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,187 @@ +# AGENTS.md + +> Guidelines for using automation and AI agents (GitHub Copilot, dependency/CI bots, and in-repo runtime schedulers/workers) with the g0v0-server repository. + +--- + +## API References + +This project must stay compatible with the public osu! APIs. Use these references when adding or mapping endpoints: + +- **v1 (legacy):** [https://github.com/ppy/osu-api/wiki](https://github.com/ppy/osu-api/wiki) +- **v2 (OpenAPI):** [https://osu.ppy.sh/docs/openapi.yaml](https://osu.ppy.sh/docs/openapi.yaml) + +Any implementation in `app/router/v1/`, `app/router/v2/`, or `app/router/notification/` must match official endpoints from the corresponding specification above. Custom or experimental endpoints belong in `app/router/private/`. + +--- + +## Agent Categories + +Agents are allowed in three categories: + +- **Code authoring / completion agents** (e.g. GitHub Copilot or other LLMs) — allowed **only** when a human maintainer reviews and approves the output. +- **Automated maintenance agents** (e.g. Dependabot, Renovate, pre-commit.ci) — allowed but must follow strict PR and CI policies. +- **Runtime / background agents** (schedulers, workers) — part of the product code; must follow lifecycle, concurrency, and idempotency conventions. + +All changes produced or suggested by agents must comply with the rules below. + +--- + +## Rules for All Agents + +1. **Human review required.** Any code, configuration, or documentation generated by an AI or automation agent must be reviewed and approved by a human maintainer familiar with g0v0-server. Do not merge agent PRs without explicit human approval. +2. **Single-responsibility PRs.** Agent PRs must address one concern only (one feature, one bugfix, or one dependency update). Use Angular-style commit messages (e.g. `feat(api): add ...`). +3. **Lint & CI compliance.** Every PR (including agent-created ones) must pass `pyright`, `ruff`, `pre-commit` hooks, and the repository CI before merging. Include links to CI runs in the PR. +4. **Never commit secrets.** Agents must not add keys, passwords, tokens, or real `.env` values. If a suspected secret is detected, the agent must abort and notify a designated human. +5. **API location constraints.** Do not add new public endpoints under `app/router/v1` or `app/router/v2` unless the endpoints exist in the official v1/v2 specs. Custom or experimental endpoints must go under `app/router/private/`. +6. **Stable public contracts.** Avoid changing response schemas, route prefixes, or other public contracts without an approved migration plan and explicit compatibility notes in the PR. + +--- + +## Copilot / LLM Usage + +> Consolidated guidance for using GitHub Copilot and other LLM-based helpers with this repository. + +### Key project structure (what you should know) + +- **App entry:** `main.py` — FastAPI application with lifespan startup/shutdown orchestration (fetchers, GeoIP, schedulers, cache and health checks, Redis messaging, stats, achievements). + +- **Routers:** `app/router/` contains route groups. Important routers exposed by the project include: + + - `api_v1_router` (v1 endpoints) + - `api_v2_router` (v2 endpoints) + - `notification` routers (chat/notification subsystems) + - `auth_router` (authentication/token flows) + - `private_router` (internal or server-specific endpoints) + + **Rules:** `v1/` and `v2/` must mirror the official APIs. Put internal-only or experimental endpoints under `app/router/private/`. + +- **Models & DB helpers:** + + - SQLModel/ORM models live in `app/models/`. + - DB access helpers and table-specific helpers live in `app/database/`. + - For model/schema changes, draft an Alembic migration and manually review the generated SQL and indexes before applying. + +- **Services:** `app/service/` holds domain logic (e.g., user ranking calculation, caching helpers, notification/email logic). Heavy logic belongs in services rather than in route handlers. + +- **Schedulers:** `app/scheduler/` contains scheduler starters; implement `start_*_scheduler()` and `stop_*_scheduler()` and register them in `main.py` lifespan handlers. + +- **Caching & dependencies:** Use injected Redis dependencies from `app/dependencies/` and shared cache services (follow existing key naming conventions such as `user:{id}:...`). + +- **Rust/native extensions:** `packages/msgpack_lazer_api` is a native MessagePack encoder/decoder. When changing native code, run `maturin develop -R` and validate compatibility with Python bindings. + +### Practical playbooks (prompt patterns) + +- **Add a v2 endpoint (correct):** Add files under `app/router/v2/`, export the router, implement async path operations using DB and injected caching dependencies. Do **not** add non-official endpoints to v1/v2. +- **Add an internal endpoint:** Add under `app/router/private/`; keep route handlers thin and move business logic into `app/service/`. +- **Add a background job:** Put pure job logic in `app/service/_job.py` (idempotent, retry-safe). Add scheduler start/stop functions in `app/scheduler/_scheduler.py`, and register them in the app lifespan. +- **DB schema changes:** Update SQLModel models in `app/models/`, run `alembic revision --autogenerate`, inspect the migration, and validate locally with `alembic upgrade head` before committing. +- **Cache writes & responses:** Use existing `UserResp` patterns and `UserCacheService` where applicable; use background tasks for asynchronous cache writes. + +### Prompt guidance (what to include for LLMs/Copilot) + +- Specify the exact file location and constraints (e.g. `Add an async endpoint under app/router/private/ ... DO NOT add to app/router/v1 or v2`). +- Ask for asynchronous handlers, dependency injection for DB/Redis, reuse of existing services/helpers, type annotations, and a minimal pytest skeleton. +- For native edits, require build instructions, ABI compatibility notes, and import validation steps. + +### Conventions & quality expectations + +- **Commit message style:** `type(scope): subject` (Angular-style). +- **Async-first:** Route handlers must be async; avoid blocking the event loop. +- **Separation of concerns:** Business logic should live in services, not inside route handlers. +- **Error handling:** Use `HTTPException` for client errors and structured logging for server-side issues. +- **Types & linting:** Aim for `pyright`-clean, `ruff`-clean code before requesting review. +- **Comments:** Avoid excessive inline comments. Add short, targeted comments to explain non-obvious or "magical" behavior. + +### Human reviewer checklist + +- Is the code async and non-blocking, with heavy logic in `app/service/`? +- Are DB and Redis dependencies injected via the project's dependency utilities? +- Are existing cache keys and services reused consistently? +- Are tests or test skeletons present and runnable? +- If models changed: is an Alembic migration drafted, reviewed, and applied locally? +- If native code changed: was `maturin develop -R` executed and validated? +- Do `pyright` and `ruff` pass locally? + +### Merge checklist + +- Run `uv sync` to install/update dependencies. +- Run `pre-commit` hooks and fix any failures. +- Run `pyright` and `ruff` locally and resolve issues. +- If native modules changed: run `maturin develop -R`. +- If DB migrations changed: run `alembic upgrade head` locally to validate. + +### Tooling reference + +``` +uv sync +pre-commit install +pre-commit run --all-files +pyright +ruff . +maturin develop -R # when native modules changed +alembic revision --autogenerate -m "feat(db): ..." +alembic upgrade head +uvicorn main:app --reload --host 0.0.0.0 --port 8000 +``` + +### PR scope guidance + +- Keep PRs focused: one concern per PR (e.g., endpoint OR refactor, not both). +- Update README/config docs when adding new environment variables. +- If unsure about conventions, align with the closest existing service and leave a clarifying comment. + +--- + +## Performance Tips + +Below are practical, project-specific performance tips derived from this repository's architecture (FastAPI + SQLModel/SQLAlchemy, Redis caching, background schedulers, and a Rust-native messagepack module). + +### Database + +- **Select only required fields.** Fetch only the columns you need using `select(Model.col1, Model.col2)` instead of `select(Model)`. + +```py +stmt = select(User.id, User.username).where(User.active == True) +rows = await session.execute(stmt) +``` + +- **Use **``** for existence checks.** This avoids loading full rows: + +```py +from sqlalchemy import select, exists +exists_stmt = select(exists().where(User.id == some_id)) +found = await session.scalar(exists_stmt) +``` + +- **Avoid N+1 queries.** Use relationship loading strategies (`selectinload`, `joinedload`) when you need related objects. + +- **Batch operations.** For inserts/updates, use bulk or batched statements inside a single transaction rather than many small transactions. + +- **Indexes & EXPLAIN.** Add indexes on frequently filtered columns and use `EXPLAIN ANALYZE` to inspect slow queries. + +- **Cursor / keyset pagination.** Prefer keyset pagination for large result sets instead of `OFFSET`/`LIMIT` to avoid high-cost scans. + +### Caching & Redis + +- **Cache hot reads.** Use `UserCacheService` to cache heavy or frequently-requested responses and store compact serialized forms (e.g., messagepack via the native module). + +- **Use pipelines and multi/exec.** When performing multiple Redis commands, pipeline them to reduce roundtrips. + +- **Set appropriate TTLs.** Avoid never-expiring keys; choose TTLs that balance freshness and read amplification. + +- **Prevent cache stampedes.** Use early recompute with jitter or distributed locks (Redis `SET NX` or a small lock library) to avoid many processes rebuilding the same cache. + +- **Atomic operations with Lua.** For complex multi-step Redis changes, consider a Lua script to keep operations atomic and fast. + +### Background & Long-running Tasks + +- **BackgroundTasks for lightweight work.** FastAPI's `BackgroundTasks` is fine for quick follow-up work (send email, async cache write). For heavy or long tasks, use a scheduler/worker (e.g., a dedicated async worker or job queue). + +- **Use schedulers or workers for heavy jobs.** For expensive recalculations, use the repository's `app/scheduler/` pattern or an external worker system. Keep request handlers responsive — return quickly and delegate. + +- **Throttling & batching.** When processing many items, batch them and apply concurrency limits (semaphore) to avoid saturating DB/Redis. + +### API & Response Performance + +- **Compress large payloads.** Enable gzip/deflate for large JSON responses