The document-to-knowledge-graph pipeline is fundamentally broken
The market is obsessed with the sexy stuff, autonomous agents, reasoning engines, sophisticated orchestration. Meanwhile, the unsexy foundation layer is completely broken. ⭕
And that foundation layer? It's the only thing that determines whether your agent actually works.
Here's the technical problem killing agentic AI reliability and that a great company like Lettria solves:
The document-to-knowledge-graph pipeline is fundamentally broken :
Layer 1: Document Parsing Hell
You can't feed a 400-page PDF with mixed layouts into a vision-language model and expect consistent structure. Here's why:
Reading order detection fails on multi-column layouts, nested tables, and floating elements
Vision LLMs hallucinate cell boundaries on complex tables (financial statements, technical specs)
You need bbox-level segmentation with preserved coordinate metadata for traceability
Traditional CV models (Doctr, Detectron2, YOLO) outperform transformers on layout detection and run on CPU
Optimal approach requires model routing: PDF Plumber for text extraction, specialized table parsers for structured data, VLMs only as fallback
Without preserving document_id → page_num → bbox_coords → chunk_id mapping, you lose provenance permanently
Layer 2: Ontology Generation Collapse
RDF/OWL ontology creation isn't prompt engineering. It's semantic modeling:
You need 5-6 levels of hierarchical abstraction (not flat entity lists)
Object properties require explicit domain/range specifications (rdfs:domain, rdfs:range)
Data properties need typed constraints (xsd:string, xsd:integer, xsd:date)
Relationships must follow semantic web standards (owl:ObjectProperty, owl:DatatypeProperty)
LLM might output syntactically valid Turtle that violates semantic consistency
Proper approach: 8-9 specialized LLM calls with constraint validation, reasoner checks, and ontologist-in-the-loop verification
Without this, your knowledge graph has edges connecting semantically incompatible nodes
Layer 3: Text-to-RDF Extraction Failure
Converting natural language to structured triples while maintaining schema compliance is where frontier models crater:
GPT-4/Claude achieve ~60-70% F1 on entity extraction, ~50-60% on relation extraction (measured on Text2KGBench)
They hallucinate entities not in your ontology
They create relations violating domain/range constraints
Context window limitations force truncation (32K tokens = ~10-15 pages with full ontology)
A specialized 600M parameter model fine-tuned on 14K annotated triples across 19 domain ontologies hits 85%+ F1
Why? Task-specific loss functions, schema-aware training, constrained decoding
The compounding effect destroys reliability
Your agent's reasoning is irrelevant when it's operating on a knowledge graph where 73% of nodes/edges are wrong, incomplete, or unverifiable.
Without bidirectional traceability (SPARQL query → triple → chunk_id → bbox → source PDF), you can't deploy in regulated environments. Period. | 13 comments on LinkedIn
The document-to-knowledge-graph pipeline is fundamentally broken