Guide
How documents are read
Before anything can be cited, ParsRAG turns each file into located passages: text with the file name and the page, slide, paragraph, table or image it came from.
- Each page becomes its own section, so citations can name the page.
- Persian pages are rebuilt in reading order from glyph positions: right-to-left lines, embedded Latin and digits, mirrored brackets and the ezafe mark on its letter.
- A page goes to OCR when it has no text, when its text layer is garbled, or when it has under 40 characters and contains images.
OCR
- Tesseract with Persian and English data, on grayscale pages rendered at 200 dpi, without contrast stretching.
- A second, single-block pass runs when the first reading is weak (mean confidence below 75), and the more confident reading wins.
- Pages turned 90, 180 or 270 degrees, and pages skewed by 2 degrees or more, are detected and read again.
- Pictures and screenshots also get a sparse-text pass on an enlarged black-and-white copy. It adds text the page layout pass skipped, such as status panels, dark toolbars and text on tinted panels.
- Pictures in Word and PowerPoint are read several at a time (OCR_WORKERS: every CPU core, at most 8, by default), with the same text as reading them one by one.
- Limits: 30 scanned pages per PDF and 200 images per Word or PowerPoint file, 45 seconds per page and 40 million pixels per image by default. Images under 10,000 pixels, such as icons, are skipped.
Word, PowerPoint and other files
- Word: paragraphs are grouped under their headings, up to 180 words, and each passage carries its heading path. Commands and code in one-cell boxes stay with the sentence that introduces them.
- Word tables become Markdown tables with row records, cited by number and the paragraph they follow, for example “paragraph 120 · table 3”.
- Images in Word and PowerPoint: the text read from each image is its own passage with the image's caption, cited as “paragraph 120 · image 7” or “slide 25 · image 9”. It also carries the text that introduces the image, and answers say when a fact comes from an image.
- PowerPoint: one section per slide, including its tables.
- HTML and XML keep only visible text; JSON is validated and re-written readably.
Passages
- Text is split into passages of about 512 tokens with a 50-token overlap, and each document is embedded in batches sorted by length.
- For consecutive PDF pages, an extra passage joins the last 90 words of one page with the first 90 of the next, so a fact split by a page turn is cited as a page range.
- Persian spelling is normalized in passages and questions: Arabic Yeh and Kaf, presentation forms, diacritics and digits. The ezafe hamza is kept.
- For every image read with OCR, a WebP copy at most 1280 px wide is kept so the source can show it.