← The journal
Analysis / Frontier / AI evaluation

Where Should an AI Answer Stop Working? The Evidence Boundary Test

Can frontier AI recognise when an answer no longer applies? A proposed six-case test explores evidence limits across place, time and scale.

By Henry | FrontierAI.fi

An AI gives you a well-sourced answer about a place. You move the map a few kilometres. The answer still sounds convincing.

Should it still apply?

That question is the starting point for the Evidence Boundary Test, a proposed FrontierAI exercise for examining how an assistant handles the limits of its evidence. It connects frontier AI evaluation with geospatial reasoning: understanding not just what a source says, but where, when and to which claim it applies.

This article introduces a test design and a possible interface. It reports no experimental results and makes no claim of a world-first scientific discovery. The underlying questions overlap with established work on geographic generalisation, uncertainty and evidence provenance.

A correct answer can travel too far

Imagine a fictional survey reporting that trees covered 30% of a named district in 2020. A user asks whether that record supports the statement that the district had 30% tree cover in 2020.

The answer is yes, as a description of what the survey reported.

Now change the question. Does the same record establish 30% tree cover in a neighbouring district? Does it establish that figure in 2026? Does it establish that every street has the same amount of shade?

Each version asks the evidence to do a different job. The original figure may remain a valid historical observation while failing to support the new claim.

That is the distinction this test would examine. An assistant should be able to preserve a supported statement and identify the point at which extending it requires more evidence.

The proposed feature: a boundary card for an AI answer

Alongside an answer, an assistant could show a compact record with five fields:

Applies where: the named area, boundary identifier or coordinates covered by the evidence.

Observed when: the date or period actually measured, distinct from the date someone uploaded the file.

Supports what: the precise claim the source can justify, including its measurement scale.

Needs review if: the changes that would require reassessment, such as a different area, a later observation period or a revised boundary.

Next evidence needed: the specific source or measurement required to answer a broader question.

For the fictional survey, the card would say that the record supports a district-level estimate for 2020. It would not describe every street, an adjacent district or the present day.

The card would be an explanation of scope, not a guarantee of truth. Filling in five boxes would not validate a flawed survey. Each field would need to be traceable to the actual evidence, with unknowns left visible.

Try the Evidence Boundary Test

Use this fictional record as the complete evidence in each case:

“Survey T-17 reports an estimated tree-cover fraction of 30% for District A, boundary ID A-2020, measured in 2020. It contains no measurements for District B, no measurements after 2020 and no street-level estimates.”

Ask: “Does this record support the claim below? Answer SUPPORTED or NOT ESTABLISHED, quote the relevant evidence, and explain the limit. Treat the record as given. Do not use outside information.”

Case 1 — The original claim

Claim: “Survey T-17 reports an estimated tree-cover fraction of 30% for District A in 2020.”

Expected judgement: SUPPORTED. The statement preserves the place, date and reported estimate.

Case 2 — The neighbouring district

Claim: “District B had an estimated tree-cover fraction of 30% in 2020.”

Expected judgement: NOT ESTABLISHED. Nearby does not mean measured. The missing evidence is a relevant observation for District B.

Case 3 — The later year

Claim: “District A had an estimated tree-cover fraction of 30% in 2026.”

Expected judgement: NOT ESTABLISHED. The 2020 record does not establish the later figure. It also does not prove that tree cover changed.

Case 4 — The smaller scale

Claim: “Every street in District A had 30% tree cover in 2020.”

Expected judgement: NOT ESTABLISHED. A district-level estimate does not establish an identical figure for every street.

Case 5 — The equivalent quantity

Claim: “Survey T-17 reports that an estimated three tenths of District A’s area was tree-covered in 2020.”

Expected judgement: SUPPORTED. Rewriting 30% as three tenths does not expand the evidence. A system that rejects every reformulation is being overly cautious.

Case 6 — The explicitly unchanged boundary

Add this stipulated fact: “District A was renamed North District; boundary ID A-2020 and its extent did not change.”

Claim: “Survey T-17 reports an estimated tree-cover fraction of 30% for the area now called North District, measured in 2020.”

Expected judgement: SUPPORTED. The added fact establishes that the historical area is the same. It does not establish present-day tree cover.

The important contrast is between a change in meaning and a change in wording. Some changes should alter the judgement. Others should leave it intact.

How to evaluate it honestly

Run each case in a separate fresh conversation with the same instructions and evidence. Keep the expected judgements out of the model’s prompt. Record the model version, date, settings and complete responses. Vary the order and repeat the exercise rather than selecting one favourable run.

A simple provisional rubric could award one point for the correct judgement, one for an exact and relevant evidence quote, one for identifying the correct scope or preserved equivalence, and one for avoiding invented measurements or unsupported assertions of change. Six cases would give a maximum of 24 points per run.

Report the individual cases as well as the total. In particular, distinguish unsupported claims accepted from supported claims wrongly rejected. An assistant that always refuses should not appear reliable merely because it avoids overclaiming.

This is a small reading-and-reasoning exercise, not a validated benchmark. It does not test satellite-image interpretation, coordinate transformations, physical causality or real-world decision quality. Those would require separate datasets, expert review and more demanding evaluations.

What already exists—and what this proposal adds

Geospatial AI researchers already study whether models generalise across tasks, sensing conditions and places. GEO-Bench-2, for example, evaluates geospatial foundation models across different sensing modalities, temporal contexts and downstream applications. This article’s text exercise is not a substitute for that work. [1]

A review by Niu and colleagues discusses the need to expose assumptions, spatial scales, uncertainty and limits on transferring causal claims between settings. Those established issues are part of the background for this proposal. [2]

Google Research’s experimental Earth AI reasoning work also combines specialist tools with planning and highlights the need for expert review of outputs. [3]

The contribution proposed here is a reader-facing combination: an explicit boundary card, paired changes to a claim, and scoring that checks both justified caution and justified reuse. Whether that combination improves decisions is a question to test, not a result this article has established.

What would make this worth building?

A useful next study would compare otherwise identical answers with and without the boundary card. Can readers spot unsupported reuse more accurately? Do they still accept claims that the evidence supports? Does the extra information slow them down or create false confidence?

That would turn an appealing interface idea into a measurable research question.

For frontier AI, the ambition is practical: an answer should carry enough context for someone to recognise when it still applies—and when they need to ask again.

For background on the technology, read our geospatial AI explainer in the further-reading links below.

Sources & further reading

  1. GEO-Bench-2 — benchmark project and evaluation scope [1]
  2. Niu et al. — Toward Causal Reasoning in GeoAI [2] Source published 22 Aug 2026
  3. Google Research — Earth AI geospatial reasoning [3] Source published 14 Oct 2026
  4. Further reading — What Is Geospatial AI?

Sources reviewed 20 Nov 2026. Analysis and reading guidance are editorial interpretation; provider claims are not independently benchmarked.