Query fan-out › Research and methodology
Research and methodology
From QueryFanout.wiki, the technical reference and research archive for query fan-out
The Query Fan-Out Observatory is the research programme behind this site. It records how AI search systems behave when given prompts of different kinds, keeps observed behaviour strictly separate from inference and illustration, and publishes the raw records on the Data page so anyone can check them or re-run them.
Status. Version 0.1.0 of the dataset covers the launch batch: eleven prompts on Perplexity plus two repeat runs. Google AI Mode was attempted first and turned out not to be observable by automation, so its rows will be added by manual capture. Batches are added as they are run and the version number increments each time. Nothing on this page describes a result that is not in the dataset.
Contents
Research question
Google has said that AI Mode uses query fan-out. Google has not said how many subqueries it issues, whether every request fans out, or what decides that. The Observatory asks a narrower, answerable set of questions:
- For which classes of prompt does AI Mode expose evidence of fan-out, and for which does it not?
- When subqueries are exposed, how many are there, and how do they relate to the prompt?
- How stable is the behaviour across repeat runs of the same prompt?
- How do the sources cited in the response relate to the exposed subqueries?
Three evidence classes
Every fan-out recorded anywhere on this site belongs to exactly one class. This is the single most important rule of the project.
| Class | Meaning | How it is captured |
|---|---|---|
| OBSERVED | The subqueries were directly exposed by the platform and captured as displayed. | Screenshot or saved page state of the platform's own "searching for" or similar disclosure, taken at the time of the test. |
| RECONSTRUCTED | The subqueries were not shown. They are inferred from what the platform retrieved or cited, or from the structure of the response. | Cited sources and response sections are mapped back to the information needs they answer. Always labelled as inference. |
| SYNTHETIC | The decomposition was generated by a model or a person as a content-planning simulation. | Produced by a fan-out simulation tool or by hand. Never presented as platform behaviour. |
A data row can carry only one class. A page can show all three, but never in the same table without the label on every row.
Prompt classes
Prompts are drawn from eleven classes so that results can be compared by prompt type rather than reported as one undifferentiated average.
| Code | Class | Example shape |
|---|---|---|
simple | Simple factual | A single fact with one well-known answer |
longtail | Long-tail informational | A specific how-to or explanation with a narrow audience |
multifaceted | Multifaceted | One prompt that plainly contains several distinct information needs |
comparison | Comparison | Two or more named options weighed against each other |
commercial | Commercial recommendation | "Best X for Y" with constraints |
ambiguous | Ambiguous | A prompt with more than one plausible intent |
rare | Rare | A topic with very little published material |
fresh | Freshness-sensitive | The answer changed within the last few weeks |
local | Local | Depends on a place |
product | Product | A specific product, model or SKU |
followup | Follow-up or session | A second turn that depends on the first |
Fields recorded per test
These are the columns of the published CSV and the keys of the JSON records.
| Field | Type | Meaning |
|---|---|---|
id | string | Stable record identifier, e.g. qfo-0001 |
prompt | string | The exact prompt text submitted |
prompt_class | enum | One of the eleven class codes above |
platform | string | e.g. google-ai-mode, google-ai-overview, chatgpt-search, perplexity |
timestamp_utc | ISO 8601 | When the test was run |
location | string | Country and region of the tester, since results are localised |
logged_in | boolean | Whether the session was signed in |
fanout_observable | boolean | Whether the platform exposed any subqueries during or after the response |
observed_subqueries | list | The subqueries exactly as displayed; empty unless fanout_observable is true |
subquery_count | integer or null | Number of observed subqueries; null when not observable |
sources_retrieved | integer or null | Sites the platform reported consulting, where it reports a number |
sources_cited | integer | Distinct domains cited in the final response |
cited_domains | list | The cited domains |
repeat_runs | integer | How many times the identical prompt was submitted in the session window |
repeat_consistency | float or null | Share of observed subqueries that recurred across repeat runs |
overlap_with_plain_search | float or null | Share of cited domains that also appear in the top 10 of a plain web search for the original prompt |
evidence_class | enum | OBSERVED, RECONSTRUCTED or SYNTHETIC |
evidence_file | string | Filename of the screenshot or saved state supporting the record |
notes | string | Anything the fields cannot express |
Test protocol
- Prompts are written in advance for each class and frozen before testing begins. Prompts are not edited after a result is seen.
- Each prompt is submitted in a fresh AI Mode session with no prior turns, unless the class is
followup, in which case the first turn is recorded as its own row. - The tester captures the platform's own disclosure of what it searched, if any, at the moment it is displayed. If nothing is displayed,
fanout_observableis false and the row can only ever be RECONSTRUCTED. - The final response is saved and cited domains are counted from the citations the platform itself attaches, not from links inferred by the tester.
- The identical prompt is re-run at least twice more within the same hour. Subqueries and citations are compared and consistency scored.
- A plain web search for the original prompt is run at the same time so overlap with ordinary ranking can be computed.
- The row is written with its evidence class and evidence file before the next prompt is tested.
Known limitations
- Disclosure is at the platform's discretion. What AI Mode shows about its own searching has changed over time and may differ by account, location and device. A false
fanout_observablemeans the platform did not show it, not that fan-out did not happen. - Observed subqueries may be a subset. Even when subqueries are displayed there is no way to confirm the display is complete.
- Results are localised and personalised. Rows record location and login state for this reason. A different tester may see different behaviour.
- Sample sizes are small. The launch batch is a few dozen prompts. Class-level patterns are reported only where they hold across the whole class, and never as percentages of Google's traffic.
- Reconstruction is inference. Mapping citations back to information needs is a judgement made by the tester. It is published so that it can be disputed.
Batch log
- 2026-09-08Methodology published. Batch 1 run on Perplexity: eleven frozen prompts, one per class, plus two repeat runs of the multifaceted prompt, 03:08 to 03:20 UTC (evening of 7 September, Colorado). 13 rows. Google AI Mode attempted and found not observable by automation (Chrome-internal page; headless access blocked); see Measurement. Dataset version 0.1.0 published.
Categories:
- Information retrieval
- AI search
- Google AI Mode