How we know
Methodology & Architecture
Provenance-aware conformity and inconsistency analytics.
Where do the documents come from?
Every claim traces to a public annual report, sustainability report, or investor filing, retrieved directly from the issuing institution or a verified public archive. In the intelligence community, this rigorous standard of relying entirely on publicly available, verifiable data is known as Open Source Intelligence (OSINT). We record the exact retrieval date alongside each source. More importantly, every insight the system generates is hard-linked to the original source text via a strict 'provenance graph'. This physical link prevents the system from 'hallucinating' or inventing citations.
How does the system decide something changed?
Instead of standard AI chunking, which loses context, PACIA consolidates thousands of pages into a structured claim graph. For each topic, we compare the specific passage side by side across years: same commitment, different number; same commitment, softer wording; or no difference. That comparison produces the redline you see on each claim page.
Who checks the system's own work?
No single model's read is taken as final. A panel of specialized AI agents — a multi-agent debate framework — reviews every candidate mutation. One extracts the change, one checks context, one actively tries to disprove the finding, and a judge decides whether the disagreement was resolved. This orchestration of foundation models is highly token-efficient, borrowing techniques from coding agents to keep costs minimized while maximizing accuracy.
How do we know the AI isn't hiding mistakes?
Total transparency. Every step of the analysts' internal debate — every challenge, defense, and shifted confidence score — is recorded instantly in a tamper-proof, append-only audit ledger. While simplified summaries are presented for brevity, the actual structural debate is securely stored and accessible for full institutional auditing.
How does the system remember the past?
It uses a continuously updating 'Tiered Memory'. As new claims emerge year over year, the system automatically consolidates past stances and resolves timeline conflicts into a unified community memory. This ensures the system doesn't lose context over long timelines, constantly learning how an institution's narrative evolves.
What are the system's limitations?
Language models can misread ambiguous phrasing, and a change in report structure can look like a claim change when it isn't. We surface confidence levels for exactly this reason, we show 'no change' cases with the same visual weight as flagged ones, and unresolved debates are labeled as such rather than hidden. This tool flags candidates for scrutiny — it is not a legal or regulatory finding. (A formal paper detailing our full methodology is coming out soon).
PACIA — Provenance Aware Conformity and Inconsistency Analytics. Architecture and methodology subject to change as the system evolves. For inquiries regarding the forthcoming paper or research collaboration, please use the Feedback page.
