AI-Assisted Document Analysis with Human Review
POC and staging-oriented document-analysis workflows combining deterministic extraction, bounded AI assistance, review-safe states, and evidence-backed outputs.
Confidential Context
This case study is sanitized. Client data and proprietary integrations are omitted. Work performed at Lean Tech. Public details are sanitized; customer names, source documents, internal identifiers, prompts, URLs, and proprietary calculations are omitted.
Outcomes
- Separated confirmed findings, review candidates, missing-source cases, and policy alerts
- Preserved uncertain signals for human review instead of forcing unsupported conclusions
- Combined deterministic extraction with bounded AI fallback and versioned prompt-run evidence
- Generated review packets and evidence artifacts for stakeholder validation
Problem
The project started as a web proof of concept for comparing contracts with billing and operational records. The source material mixed PDFs, spreadsheets, database records, and manual review notes. A single model response could not safely resolve differences in document scope, missing amendments, ambiguous matches, or incomplete operational evidence.
The core engineering problem became preserving evidence and uncertainty through the pipeline: deterministic comparisons should remain distinct from AI suggestions, and unresolved items should enter review queues rather than disappear or become precise-looking conclusions.
Status
The repository supports a POC with staging-oriented tooling and internal analysis deliverables. It contains deployment scripts and Supabase-backed run infrastructure, but the available evidence does not justify describing the complete system as a production platform or claiming autonomous decisions.
Approach
- Frontend and review model: extended a Vite/React interface and TypeScript analysis layer that separates confirmed, review, missing-source, contract-update, and policy-alert states.
- Backend: worked with Supabase/PostgreSQL, Edge Functions, private artifact records, authentication boundaries, and reviewer-feedback data.
- Extraction and comparison: combined deterministic invoice/workbook parsing and matching heuristics with bounded AI fallback for document extraction and discrepancy review.
- Asynchronous processing: contributed run, artifact, correction, and worker foundations with explicit status transitions rather than treating a long document analysis as one request-response call.
- Evaluation: added fixture-backed regression cases, prompt-run logging, benchmark scripts, and review packets that preserve the evidence behind a result.
- Human review: routed incomplete contract stacks, weak matches, missing sources, and ambiguous pricing questions into review-oriented states instead of forcing exact conclusions.
My Contribution
Git history attributes commits to me across contract and bill extraction, deterministic signal merging, contract matching, discrepancy analysis, prompt versioning, reviewer feedback, regression tooling, and the asynchronous run/artifact foundation. The original prototype used generated and team-provided components, so I describe this as extending and hardening a shared system rather than sole ownership of the full architecture.
What I Delivered
- Review workbooks and evidence packets that retain source references, confidence, and unresolved questions.
- Deterministic and AI-assisted extraction paths with explicit fallback and error handling.
- Review-state logic that prevents missing sources or ambiguous scope from appearing as finalized findings.
- Local benchmark and regression workflows for testing extraction, matching, and comparison changes before broader use.
- A Supabase-backed foundation for asynchronous runs, generated artifacts, and later human decisions.
Why It Matters
The important lesson was that document AI cannot be evaluated only by whether it produces an answer. A reliable workflow must also preserve source lineage, identify missing evidence, distinguish deterministic facts from model suggestions, expose uncertainty, and make human review an explicit system state.
Skills Demonstrated
- Python/Pandas analysis over heterogeneous PDF, spreadsheet, database, and API inputs.
- TypeScript/React review interfaces and Supabase/PostgreSQL-backed workflow state.
- Deterministic extraction, matching, and comparison combined with bounded AI assistance.
- Asynchronous job and artifact handling with status validation and retry-aware workers.
- Fixture-backed regression testing, prompt evaluation, and evidence-preserving review outputs.
- Human-in-the-loop design for ambiguous, incomplete, or policy-sensitive results.
Safe Public Summary
AI-assisted document-analysis POC combining Python/Pandas workflows, a React/Supabase review interface, deterministic extraction and comparison, asynchronous run infrastructure, and evidence-backed human-review queues. The public summary intentionally omits customer data, proprietary rules, internal documents, prompts, and financial results.