Building a Scanned Document RAG Pipeline with FastOCR and LangChain

The Silent Failure of Scanned PDFs in RAG

Retrieval-Augmented Generation (RAG) relies on high-quality text chunks stored in vector databases. However, enterprise document repositories—including legal contracts, medical reports, and historical records—are predominantly image-based scans.

When developers run pypdf.PdfReader.extract_text() or fitz.open(), the parser looks for embedded font streams. Scanned PDFs contain only raw image bitstreams, resulting in empty strings or corrupted whitespace. The pipeline silently indexes zero tokens, and downstream LLMs fail to retrieve relevant context.

Why Raw Text Beats Synthesized Markdown for Vector Stores

Many modern OCR services attempt to “guess” semantic markdown—inserting synthetic tables, bold markers, and header tags. In vector search, however, artificial markdown tags introduce syntactic noise, distort character split boundaries, and artificially skew cosine similarity scores.

FastOCR delivers clean, raw extracted text preserving native line breaks and paragraph separations without synthetic hallucination. This ensures that text chunkers like LangChain’s RecursiveCharacterTextSplitter calculate chunk boundaries based on natural language rather than parser-generated syntax.

Step-by-Step Implementation

1. Install Dependencies

pip install fastocr langchain-core langchain-text-splitters langchain-community langchain-openai chromadb

2. Load and Chunk the Scanned Document

from fastocr import FastOCR
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings

# Initialize FastOCR client (reads FASTOCR_API_KEY from environment)
client = FastOCR()

# Extract raw text from multi-page scan in one line
raw_text = client.extract_text("contract_scan.pdf")

# Wrap into LangChain Document with metadata
doc = Document(
    page_content=raw_text,
    metadata={"source": "contract_scan.pdf", "extractor": "FastOCR API v1"}
)

# Chunk text for embeddings
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
chunks = splitter.split_documents([doc])

# Index into Chroma vector database
vectorstore = Chroma.from_documents(chunks, embedding=OpenAIEmbeddings())

3. Build the Retrieval Question-Answering Chain

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

prompt = ChatPromptTemplate.from_template("""
Context from scanned archive:
{context}

Question: {question}

Answer concisely based strictly on the extracted text:
""")

chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | ChatOpenAI(model="gpt-4o", temperature=0)
    | StrOutputParser()
)

response = chain.invoke("What are the payment terms defined in Section 4?")
print(response)