Skip to content
Huddle

Overview

The RAG service is a Python Temporal worker — not an HTTP API server (it exposes only a minimal :8080 health endpoint for the platform healthcheck). It polls one task queue and runs the durable workflows that turn bond documents into structured data: parsing PDFs, embedding and retrieving content, extracting field values, and finding & filling bond forms. It reads and writes everything through the Go API via the git-installed huddlesurety-api Python SDK.

  • Language: Python 3.12, managed with uv
  • Orchestration: Temporal Python SDK (worker)
  • PDF parsing: PyMuPDF (fitz) text-layer extraction + Tesseract OCR for scanned pages
  • LLMs & embeddings: LiteLLM SDK for extraction, fill, and bond-finding, with Voyage AI reranking (litellm.rerank) — all routed through the self-hosted LiteLLM gateway, which holds the provider keys (the worker carries only a virtual key)
  • Retrieval: hybrid dense (pgvector) + BM25 (ParadeDB pg_search) fused with Reciprocal Rank Fusion
  • Database: PostgreSQL with pgvector, via psycopg3

Project structure

Section titled “Project structure”
  • Directoryapp/
    • Directorycore/
      • config.py Settings + get_huddle_client
      • db.py Process-wide psycopg3 connection pool
      • api_client.py Single-ID → full-context resolvers (activity-only)
      • idconvert.py ULID (Go/S3) ↔ UUID (DB)
      • logging.py Structured JSON logging
    • Directorymodels/ Pydantic + dataclass models (DOMElement, RAGChunk, …)
      • …
    • Directoryservices/
      • parser.py PDF → DOMElements (PyMuPDF text-layer OR Tesseract OCR)
      • text_cleaner.py OCR-noise cleanup
      • chunker.py DOMElements → RAGChunks
      • embeddings.py Gemini embeddings via the LiteLLM gateway (batched, rate-limited)
      • query_embedding_cache.py Postgres cache for query embeddings
      • vectorstore.py Hybrid dense + BM25 (RRF) search; chunk CRUD
      • elementstore.py document.element CRUD
      • reranker.py Voyage rerank-2.5-lite via litellm.rerank (gateway)
      • rag_extractor.py Field-extraction orchestrator
      • grounding.py Quote → element match; evidence PNG render + upload
      • bond_finder.py Bid-bond finder
      • final_bond_finder.py Final-bond finder
      • bond_filler.py Vision-LM form filler
      • pct.py Bond-percentage coercion/clamping (coerce_pct)
      • llm_retry.py completion_with_retry — timeout + backoff wrapper for litellm.completion
      • prompts.py LLM prompts + JSON schemas
    • Directoryo11y/ OTel + Temporal identity interceptor + cost table
      • …
    • Directorydb/ goose migrations (schema “document”)
      • …
    • Directorytemporal/
      • worker.py Entry point — python -m app.temporal.worker
      • workflows.py The 6 workflow definitions
      • Directoryactivities/ Activities, one file per workflow
        • …
  • pyproject.toml uv dependencies
  • mise.toml Tasks
  • railpack.json Railway (Railpack) build config