Local Development

RAG is driven by mise tasks (which load the env) and dependencies are managed with uv. It builds on Railway with Railpack (railpack.json) and starts with python -m app.temporal.worker.

Prerequisites

• mise — provisions Python 3.12, goose, the Railway CLI, and more.
• Docker — for the local pgvector Postgres.
• Railway access — mise run env pulls config from Railway.
• A GITHUB_TOKEN — the huddlesurety-api SDK is git-installed from a private repo; the build rewrites https://github.com/ to use the token.

Setup

1. Install tools + dependencies:

   mise install     # provisions the toolchain; uv resolves Python deps

2. Pull environment variables from Railway:

   mise run env     # writes .env.json (gitignored); mise auto-loads it

3. Start local Postgres (pgvector):

   docker compose -f docker-compose-pgvector.yaml up

4. Migrate:

   mise run db:migrate    # goose up, against app/db/

5. Run the worker:

   mise run run           # python -m app.temporal.worker

Local DB compose path

The mise run db:local task references ./db/docker-compose.yaml, but the pgvector compose file actually lives at the repo root as docker-compose-pgvector.yaml. Use the root file directly (step 3) until the task path is fixed.

Tasks

Task	What it does
mise run run	Run the Temporal worker
mise run db:migrate	goose up (schema document)
mise run db:status	goose migration status
mise run db:nuke	goose reset
mise run db:ui	Postgres TUI (rainfrog)
mise run env	Pull env vars from Railway → .env.json
mise run log	Tail Railway logs

Environment

Runtime config comes from Railway via mise run env → .env.json (gitignored); .env.local holds dev overrides. Key variables:

• API: BASE_URL (Go API), KEY_AUTH_SECRET (service-key auth)
• DB: DATABASE_URL
• Temporal: TEMPORAL_SERVER_URL, TEMPORAL_TASK_QUEUE_RAG, DEPLOYMENT_ENVIRONMENT
• LLM gateway: LITELLM_PROXY_API_BASE, LITELLM_PROXY_API_KEY — both required (the worker holds no provider key; the gateway does). See LLM Gateway.
• Models: VERTEX_MODEL (the extraction model — the name is legacy; it now names a LiteLLM gemini/… model), EMBEDDING_MODEL, EMBEDDING_DIMENSION, RERANK_MODEL (a voyage/… gateway alias), RERANK_ENABLED
• Tuning: RAG_*, CHUNK_*, FUZZY_MATCH_THRESHOLD
• OTel: OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_PROTOCOL

Model names look forward-dated, and now carry a `gemini/` prefix

The configured models (e.g. gemini/gemini-3.1-flash-lite, gemini/gemini-3.5-flash) are what the service actually runs — not a typo. Since the LiteLLM migration (#142) the provider prefix moved from google/·openai/ to gemini/. As of #145 those calls no longer go direct to Gemini with a GOOGLE_API_KEY; they route through the LiteLLM gateway, where the worker authenticates with a per-environment virtual key (LITELLM_PROXY_API_KEY) and holds no provider credential. .env.example still lists Vertex/GCP and S3 access keys the current code doesn’t use (it goes through the gateway and presigned URLs). Three declared-but-unused dependencies now: google-genai (was always unused), openrouter (dead since the LiteLLM migration), and voyageai (dead since reranking moved to litellm.rerank through the gateway) — no code imports any of them.

Conventions

• Python 3.12, type hints throughout. Dataclasses for Temporal activity I/O (must serialize across the boundary); pydantic models for domain types and LLM structured-output schemas.
• Lazy imports inside activities to keep the workflow sandbox light.
• Per-call LLM timeouts + retry — every litellm.completion call sets an explicit timeout (LLM_TIMEOUT_MS 30s for the extractor, FILL_LLM_TIMEOUT_MS 36s for the filler, FIND_LLM_TIMEOUT_MS / FINAL_FIND_LLM_TIMEOUT_MS 30s for the finders). This is load-bearing: LiteLLM’s own default request_timeout is 6000s, so an un-timed call can hang for ~27 min. The finder calls route through a shared services/llm_retry.py completion_with_retry(timeout_s, …) — 429s honor the server delay (capped at 15s), 5xx/transport errors back off exponentially, and a client-side Timeout is not retried. The filler additionally downshifts reasoning effort on a timeout/truncation (medium → low) rather than failing outright. No explicit temperature is passed anymore — LiteLLM applies Gemini 3’s default of 1.0 (structure stays pinned by the strict json_schema).
• Memory hygiene — activities free memory (gc + malloc_trim) in finally, and never pass images across activity boundaries (they re-download/re-render).
• Settings via pydantic-settings with an env-var fallback for unlisted keys.

There are no committed tests (the tests/ and benchmark/ dirs are gitignored), and the repo has no in-repo CI.