A Framework-Independent, Multilingual RAG Platform

This is a personal, self-initiated backend project, not professional production experience. It has no production traffic, no users, and no frontend; it's feature-complete through its backend lifecycle phase, with UI validation intentionally deferred.

Problem

Most RAG tutorials stop at "upload a document, ask a question." I wanted to explore the parts that stop being optional once you treat retrieval infrastructure as something you'd actually operate: what happens when a document is deleted, when an embedding model changes, when the index and the source of truth disagree, or when you want to swap out the orchestration framework without rewriting the API.

Context

A self-initiated backend project, built and iterated on solo, with no company or production deployment behind it. It runs entirely on local infrastructure: FastAPI, PostgreSQL, Redis, Qdrant, and Ollama for local LLM/embedding inference. No external API calls are required.

Constraints

• Multilingual from the start: Hebrew and English content and queries, including cross-language retrieval, not just English with translation bolted on.
• No cloud dependency requirement: the whole stack needed to run locally via Docker Compose.
• Avoid framework lock-in: the retrieval/generation logic shouldn't be inseparable from a specific orchestration library.
• Solo project, so scope had to be sequenced deliberately (the project is explicitly phased, with the backend lifecycle frozen pending future UI/E2E work) rather than built as one unbounded effort.

My Role

Sole designer and implementer: architecture, backend implementation, test strategy (unit, integration, and Testcontainers-based backend end-to-end), and documentation.

Architecture / Approach

Three storage systems with distinct, explicit roles: PostgreSQL as the lifecycle authority (source of truth for document state and metadata), object storage as the original-content authority, and Qdrant as a rebuildable derived vector index, never treated as a system of record. Above that sits a provider-abstraction layer for embeddings, LLM, and vector-store access, and a RagEngine abstraction that lets a custom orchestrator and an optional LangChain-backed engine share one public API and streaming (SSE) contract.

Key Decisions

• Qdrant as derived, not authoritative. Treating the vector index as rebuildable, with PostgreSQL owning lifecycle state, means re-indexing or recovering from a corrupted index is a safe, well-defined operation instead of a data-loss risk.
• Two interchangeable RAG engines behind one contract. A custom orchestrator and a LangChain-based engine both implement the same RagEngine interface, so the framework choice is swappable rather than load-bearing, validated with behavioral parity checks across multilingual scenarios.
• Versioned embeddings and vector collections. Embedding configuration and vector collections are versioned explicitly, so an embedding-model or chunking change can't silently mix incompatible vectors, laying the groundwork for safe re-indexing.
• Provider abstraction over hard-coding one LLM vendor. The embedding/LLM/vector-store layer is built so additional providers can be added later via provider stubs without touching the ingestion or retrieval pipeline.
• Deployment-readiness as a first-class phase (Phase 2.10). Fail-fast configuration validation, bounded exponential backoff with jitter on provider calls, structured JSON logging with request correlation IDs, and cooperative worker shutdown are treated as part of the system, not an afterthought bolted on before a hypothetical launch.

Trade-offs

Building the lifecycle layer (observe, recover, delete, deduplicate, upgrade, reconcile) before a frontend existed meant the project stayed backend-only for longer than a demo-first approach would have, a deliberate trade-off documented in the project's own backend-freeze decision, prioritizing a correct foundation over a faster-to-show UI. Framework independence via a shared RagEngine contract also means more abstraction than a single-framework implementation would need, justified here because comparing a custom engine against a LangChain-based one was itself part of the point.

Impact / Outcome

A backend that supports document upload and async ingestion across multiple file types, streaming source-attributed chat, hash-based upload deduplication, full async document deletion, build-ahead zero-downtime re-indexing, and read-only reconciliation/audit reporting across all three storage systems, validated with unit, integration, migration, and backend end-to-end test suites, gated by a make verify pre-commit hook.

Lessons

The interesting engineering problems in RAG systems show up after the first successful query: what happens when documents change, when models change, or when the derived index drifts from the source of truth. Designing for those cases from the start, even in a solo project, forced real architectural decisions instead of demo shortcuts.