Free lesson

Inspect PDFs before ingestion

Inspect PDFs before ingestion — A private PDF knowledge assistant. Create three fixtures; Inspect visible and extracted text; Define an ingestion decision. Practical assignment: Lab evidence.

Demo edition. The materials are undergoing editorial review.

Topics covered

  • Create three fixtures
  • Inspect visible and extracted text
  • Define an ingestion decision

Create three fixtures

Create a one-page fictional policy PDF by printing a document. Also export a screenshot as PDF and prepare a harmless text file renamed .pdf.

A PDF is a container for page layout, not a guarantee of readable sentences. A scan may have no text layer; a table may extract in the wrong order; an OCR layer may contain plausible spelling errors. This course therefore treats ingestion quality, source visibility and lifecycle as product requirements. Use only invented workshop documents until every page can be traced back to its original file.

Inspect visible and extracted text

Open each file manually. Try selecting the cancellation sentence, then copy it into a text editor. Record page order, columns and whether 48 hours survives.

A copied sentence is a quick screening test, not certification of every page. Include one numeric rule and one negation in your inspection because OCR errors there change decisions. Record the actual page number so a later reviewer can reproduce the same comparison.

Run processing in a separate local directory with synthetic files and explicit limits. Save the raw file under an application-generated identifier rather than a user-supplied path. Preserve a page manifest containing extraction status, revision and processing method. The retrieval project from the prerequisite course consumes approved page text only; it must not receive every uploaded byte merely because upload completed successfully.

// Fictional page:
// Cancellation requires 48 hours notice.
// Bring a washable apron.
// Expected: both sentences remain intact after extraction.

Define an ingestion decision

Route readable text to extraction, image-only pages to OCR review and invalid files to rejection. Empty extraction is a status, not an empty policy.

Test a normal document, an image-only page and a file that should be rejected. Keep the original visible during review; a fluent answer cannot reveal that OCR changed a number. Access checks apply to both retrieved text and original downloads. Document deletion must also remove active derived chunks and cached answers, while backup retention follows an explicit separate policy. Record the exact behavior you verified.

Practical assignment

Lab evidence

Continue to learning

Course outline

  1. Inspect PDFs before ingestion
  2. Extract page text and metadata
  3. Use OCR without hiding uncertainty
  4. Handle uploads with bounded processing
  5. Ground answers and enforce source access
  6. Maintain, replace and remove documents