Loaders: PDF, HTML, Text, Web

1. Loading is Interpretation

A "Loader" in LangChain connects to a data source (File, API, Database) and returns a list of Document objects.

This sounds simple, but it is lossy. When you load a PDF, you lose vector graphics, exact positioning, and font sizes. When you load HTML, you lose CSS visibility rules (hidden text might be loaded).

You are translating "Visual Layout" into "Linear Text."

2. TextLoader (The Baseline)

The simplest loader. It reads a file .txt and returns one Document.

from langchain_community.document_loaders import TextLoader

loader = TextLoader("./note.txt")
docs = loader.load()

Use Case: Code, Markdown,logs. 1:1 fidelity.

3. WebBaseLoader (The Scraper)

Extracting text from the web is messy. Websites have navbars, ads, and footers. Using WebBaseLoader (packaged with BeautifulSoup) allows you to target specific <div> tags.

from langchain_community.document_loaders import WebBaseLoader

loader = WebBaseLoader("https://example.com")
docs = loader.load()
# Warning: This often loads "Privacy Policy" and "Login" text.

Pro Tip: Always post-process web content. The "noise" ratio is high.

4. PyPDFLoader (The Nightmare)

PDF is a print format, not a data format. It doesn't know what a "paragraph" is; it only knows "character 'A' at position x=10, y=20".

Loaders try to reconstruct the text flow, but they fail on:

1. Multi-column layouts: It might read across columns (combining line 1 of col A with line 1 of col B).
2. Tables: It extracts cell contents locally, destroying the row/column relationship.

from langchain_community.document_loaders import PyPDFLoader

loader = PyPDFLoader("./report.pdf")
docs = loader.load_and_split() 
# By default, it splits by Page number.

5. Summary

Loading is not passive. It is an active choice about what to keep and what to throw away. For PDFs and Complex HTML, "Standard" loaders are rarely enough for production. You often need specialized tools (like Unstructured.io or Azure Layout).

Key Intuition: "Fidelity lost at loading is lost forever."