Guide

Install and first start

The recommended way to run ParsRAG is Docker Compose with the Ollama already installed on your computer: one app container serves the interface and API, Qdrant stores the vectors, and your own Ollama writes the answers. Ollama inside Docker and API models are the other ways to run it.

What you need

Docker Desktop on Windows or macOS, or Docker Engine with Compose on Linux, and 8–16 GB of RAM.
Ollama installed on the computer, for the recommended setup. Without it, run Ollama inside Docker or use an API model.
Disk space for the images and model weights.

The complete setup

The recommended setup uses the Ollama already installed on your computer, with the models you already have and its GPU. The app reaches it at host.docker.internal, so no Compose profile or override file is needed and the app container needs no GPU. On a Mac this is the way to generate on the GPU, because Docker can't pass a Mac's GPU to a container.

Install Ollama and pull a model. On a GPU with about 6 GB of memory, a 3–4B model runs fully on the GPU.
Copy .env.example to .env and set LLM_MODEL_NAME to the model you pulled. Its defaults already point the app at the Ollama on this computer: OLLAMA_DOCKER_BASE_URL=http://host.docker.internal:11434.
Linux only: Ollama listens on 127.0.0.1 by default, where containers can't reach it. Make the Ollama service listen on the Docker bridge address, for example OLLAMA_HOST=172.17.0.1:11434. Windows and macOS need nothing.
Start ParsRAG, open http://127.0.0.1:8000 (or the port in PARSRAG_PORT), upload documents and ask.

ollama pull qwen2.5:7b
Copy-Item .env.example .env
docker compose up -d --build
ollama ps

On macOS or Linux, copy the settings file with cp .env.example .env. While an answer is being written, ollama ps shows where the model runs; 100% GPU means it runs entirely on the GPU. The port is bound to 127.0.0.1 unless you change PARSRAG_BIND_HOST.

Ollama has no authentication. On Linux, prefer the bridge address: OLLAMA_HOST=0.0.0.0 opens Ollama to every network the computer is on, so if you use it, allow port 11434 only from Docker's networks.

Other ways to run

Setup	When to use it	How
Ollama in Docker on CPU	No Ollama on the computer; portable on every system	The local-model profile, with OLLAMA_DOCKER_BASE_URL=http://ollama:11434 and OLLAMA_MODEL in .env
NVIDIA GPU	Ollama in Docker with faster embeddings and generation, on Windows or Linux	Add the compose.gpu.yaml overlay
Low-VRAM NVIDIA	Cards around 6 GB that can't hold both models	Add compose.gpu-low-vram.yaml: embeddings on CPU, Ollama context 2048
AMD GPU	Supported Linux ROCm hosts	Add compose.amd.yaml (PyTorch 2.6 with ROCm wheels)
API model	No suitable local hardware	Set LLM_PROVIDER=api and the endpoint in .env, or switch to it in Settings

docker compose --profile local-model up -d qdrant ollama
docker compose --profile local-model run --rm ollama-init
docker compose --profile local-model up -d --build app

The first start

The embedding model (intfloat/multilingual-e5-base) and the Hybrid reranker (about 100 MB) download into the model cache.
With the computer's own Ollama, ParsRAG never pulls models. With Ollama in Docker, ollama-init pulls the configured model once into its own volume.
Until everything is ready, /health/ready answers 503 with status preparing, and the interface shows that it is preparing the model and document store.
If Ollama is off when the app starts, the app still becomes ready. Questions report that the model service is unreachable until Ollama starts, then work without a restart.
After the first downloads, a local Ollama setup can run offline.

Without Docker

Native development needs Python 3.12, Node.js 22, a running Qdrant, Ollama or an OpenAI-compatible endpoint, and Tesseract with Persian and English data for OCR. Copy .env.example to .env and remove the # from the lines under “Running without Docker” at its end: the Qdrant address, the Ollama URL, the settings file and OCR_ENABLED=1.

py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements-runtime.txt -c requirements.lock
Set-Location frontend
npm ci
npm run build
Set-Location ..
.\.venv\Scripts\python.exe -m uvicorn backend.main:app --host 127.0.0.1 --port 8000

In a native run, OCR stays off until OCR_ENABLED=1 is set; the Docker setup turns it on.

Day-to-day

docker compose ps
docker compose logs -f app
docker compose down

docker compose down keeps your data. docker compose down -v permanently removes vectors, settings saved in the app, caches and the weights of Ollama in Docker.