Browse the documentation
AI-Soda Brain

Verify sources and metric definitions

A reader needs to know where an answer came from, what it covers and how to check it. Review the source, its date and the metric definition together. Even a correct calculation can answer the wrong question.

Technical guide · November 20263 min read

Identify the kind of source

Evidence for a structured query typically includes the queried object, period, filters and aggregation. Evidence from a document includes its title, version and page or section. A business metric also needs a definition and an applicable scope. A generic source label does not tell the reader which of these checks they can perform.

A figure from an earlier report may be a snapshot taken when that report was produced. For a fresh query, check when the underlying source was last updated. A historical report and a live database may represent different points in time.

Ask four questions

  1. Does the source actually contain the cited fact? Open the original passage or detailed records to check.
  2. Do the period, units, currency and organizational scope match the question?
  3. Does the answer mix observed facts, calculated results and recommendations? Label them separately.
  4. Do missing data or an unconfirmed definition affect the conclusion? Gather the missing material before making a decision if necessary.

How one name can produce two figures

In a synthetic example, “revenue” could be grouped by the date an order was confirmed or the date payment was received. Both queries could be correct and return different values. Choose the business definition first, then align the filters to it.

Dimension to checkWhat to record
PeriodCalendar month or business cycle, and whether the period is complete
Amount definitionOrder, invoice or payment amount, and tax treatment
Record statusTreatment of cancellations, refunds and reversals
Verification trailSource location and data version at the time of generation

When a citation will not open

Record the citation location and the time of the issue. Check permissions, whether the file moved or was deleted, whether the index refreshed and whether the document version changed. You may retain the existing summary, but label the citation as currently unverifiable until the source can be checked.

When reporting an issue, include the smallest relevant excerpt and a synthetic example that reproduces it. A complete customer file or business export is rarely an appropriate default attachment.

Try “permissions,” “data source,” “manufacturing” or “API.”