Building a Scanned Document RAG Pipeline with FastOCR and LangChain

The Silent Failure of Scanned PDFs in RAG

Retrieval-Augmented Generation (RAG) relies on high-quality text chunks stored in vector databases. However, enterprise document repositories—including legal contracts, medical reports, and historical records—are predominantly image-based scans.

When developers run pypdf.PdfReader.extract_text() or fitz.open(), the parser looks for embedded font streams. Scanned PDFs contain only raw image bitstreams, resulting in empty strings or corrupted whitespace. The pipeline silently indexes zero tokens, and downstream LLMs fail to retrieve relevant context.

Why Raw Text Beats Synthesized Markdown for Vector Stores

Many modern OCR services attempt to “guess” semantic markdown—inserting synthetic tables, bold markers, and header tags. In vector search, however, artificial markdown tags introduce syntactic noise, distort character split boundaries, and artificially skew cosine similarity scores.

FastOCR delivers clean, raw extracted text preserving native line breaks and paragraph separations without synthetic hallucination. This ensures that text chunkers like LangChain’s RecursiveCharacterTextSplitter calculate chunk boundaries based on natural language rather than parser-generated syntax.

Step-by-Step Implementation

1. Install Dependencies

pip install langchain-fastocr langchain-core langchain-text-splitters langchain-community langchain-openai chromadb

2. Load and Chunk the Scanned Document

langchain-fastocr (source on GitHub) wraps the FastOCR Python SDK. FastOCRLoader creates the document, uploads the PDF, waits for processing, and returns one LangChain Document per file with the OCR text as page_content. Sign in and create an API key at /developers/keys, then set it as FASTOCR_API_KEY.

from langchain_fastocr import FastOCRLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings

# One Document per file; metadata includes source, file_name and pages_billed
documents = FastOCRLoader("contract_scan.pdf").load()

# Chunk text for embeddings
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
chunks = splitter.split_documents(documents)

# Index into Chroma vector database
vectorstore = Chroma.from_documents(chunks, embedding=OpenAIEmbeddings())

Each chunk keeps its file’s metadata, so answers can cite the source file. Pass a list of paths to load a folder of scans, or output="pdf" to also save a searchable PDF beside each input.

3. Build the Retrieval Question-Answering Chain

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

prompt = ChatPromptTemplate.from_template("""
Context from scanned archive:
{context}

Question: {question}

Answer concisely based strictly on the extracted text:
""")

chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | ChatOpenAI(model="gpt-4o", temperature=0)
    | StrOutputParser()
)

response = chain.invoke("What are the payment terms defined in Section 4?")
print(response)