Shipping a document intelligence API with Gemini and FastAPI

The brief was simple: take a document — any document, a PDF or a URL or raw text — and turn it into something a machine can search and retrieve. I have never written a brief that turned out to be as complicated as that one.

The hard part was not the extraction. It was deciding what to extract, how to structure it, and how to store it in a way that would still be useful when the query came in six months later.

The ingestion model

We settled on three input types, each handled differently.

A file upload is processed page by page. Each page is rasterised to a PNG and passed to Gemini 2.5 Flash Vision with a structured prompt — identify all major components and their relationships, describe spatial layout and flow directions, extract table structures with complete values and headers, capture all numerical annotations.

async def chunk_to_section_pdf(chunk: Chunk) -> schemas.SectionRecord:
    for item_bytes in chunk.page_items:
        link = await upload_file(item_bytes, 'image/png', 'png')
        item_desc = await gemini_client.models.generate_content(
            model='gemini-2.5-flash-preview-04-17',
            contents=[
                f"""Analyze this technical diagram in exhaustive detail:
                1. Identify all major components and their relationships
                - List every visible element with precise measurements
                - Describe spatial relationships between elements
                2. For tables: extract complete structures with headers and values
                3. Capture all numerical values and annotations"""
            ]
        ).text

A link is stored as an external reference — name, size, and type recorded, content not fetched. Raw text is UTF-8 encoded, byte-counted for size, and stored directly. No extraction needed.

Exactly one input

The API enforces exactly one input type per request. Not at least one — exactly one.

inputs = [bool(obj_in.file), bool(obj_in.link), bool(obj_in.text)]
if inputs.count(True) != 1:
    raise errors.link_or_file_or_text_required

A caller should never be uncertain about which input the API used. Counting truthy values and raising on anything other than one is the cleanest way to express that constraint.

The section model

The output of the pipeline is a Section record: name, content, summary, section number, a relations field for entities and their connections, and a meta_data field for identifiers and keywords extracted for retrieval. Every section is linked to its source via a foreign key.

Security

Client secrets are generated with secrets.token_hex(32) — a 64-character hexadecimal string from Python's cryptographically secure random number generator. Client IDs are str(uuid.uuid4()).

class ClientCredentials(HasOwnerMixin, HasDeleteMixin, Base):
    name: Mapped[str]
    client_id: Mapped[str] = mapped_column(
        default=lambda: str(uuid.uuid4())
    )
    client_secret: Mapped[str] = mapped_column(
        default=lambda: secrets.token_hex(32)
    )

These are not interchangeable. UUID4 is random but not designed for use as a secret. secrets.token_hex gives you full cryptographic randomness. For anything that functions as a password, use secrets.