Measurement
Part of Attribution measurement built for England around five separate jobs
Six tests before a marketing attribution benchmark is used
Six admission tests decide whether an outside figure can serve as a marketing attribution benchmark for England, and an internal series if none passes.
Result: an England marketing-attribution benchmark is not established. The official sources reviewed did not measure a defined England population using one compatible attribution event, denominator and method. Broad advertising, business or digital-use figures cannot fill that gap.
The desk review closed on 6 September 2026. Its source universe covered current UK government evaluation and data-quality guidance, Office for Statistics Regulation standards, and relevant ICO measurement constraints. It excluded supplier marketing aggregates, unattributed web summaries and figures without a reproducible population or method. No product, dataset or respondent-level record was accessed.
Official sources reviewed, all accessed on 6 September 2026:
- Office for Statistics Regulation, quality standard.
- ICO, final guidance on storage and access technologies.
- HM Treasury, Quality in Policy Impact Evaluation (QPIE) guidance.
- Government Analysis Function, uncertainty guidance.
What to take away
- The official sources reviewed did not establish an England marketing-attribution benchmark: none measured one compatible population and method.
- Write the benchmark record before searching, and leave a candidate unresolved if any field changes the comparison.
- Give every outside figure six admission tests and write the reason for each answer.
- Where external admission fails, build an internal reference series and publish the admission log rather than a fabricated target.
- A future benchmark becomes usable only when a named statistician verifies its population, method, uncertainty and decision relevance.
Write the benchmark record before searching
A candidate benchmark must resolve ten fields before it is compared with anything. Write the record before searching, so the search cannot choose the answer, and treat any field left open as a reason to leave the candidate unresolved.
Benchmark record fields
- User and action informed
- Eligible population defined
- England geography and multi-region
- Event, numerator, denominator, unit
- Observation and reporting periods
- Attribution job and rule version
- Source, sampling, collection route
- Late, duplicate, missing, refunded states
- the user and action it is meant to inform;
- the eligible organisation, campaign, person, session, order or other population;
- England geography and treatment of multi-region activity;
- event, numerator, denominator or unit;
- observation and reporting periods, with timezone;
- attribution job and rule or model version;
- data source, sampling or collection route;
- late, duplicate, missing, unmatched, refunded and corrected states;
- adjustment, weighting and uncertainty method; and
- publisher, funding, conflicts, access date and expiry.
If any field changes the meaning of the comparison, leave the candidate unresolved rather than guessing it.
Run each source through six admission tests
Give every candidate source one row and six questions: does the population match, is the geography genuinely England, is the metric identical, is attribution comparable, can quality be assessed, and is the record current and independent enough. Write the reason for each answer.
Source admission tests
Admission question
- Population match
- Eligibility and denominator
- Genuine England
- Collection and location rule
- Identical metric
- Event, unit, exclusions
- Comparable attribution
- Rule, model, design version
- Quality assessable
- Missingness, bias, uncertainty
- Current and independent
- Date, publisher, conflicts
Required evidence
- Population match
- Genuine England
- Identical metric
- Comparable attribution
- Quality assessable
- Current and independent
| Candidate source | Does the population match? | Is geography genuinely England? | Is the metric identical? | Is attribution comparable? | Can quality be assessed? | Is the record current and independent enough? | Decision |
|---|---|---|---|---|---|---|---|
| Undated supplier platform-credit summary page | Rejected: no eligibility or denominator | Rejected: no collection or location rule | Rejected: click credit and order unit not stated | Rejected: configured rule version absent | Rejected: missingness and bias not reported | Rejected: no date and publisher has commercial interest | Rejected |
| Your candidate | Admit, reject or unresolved, with the reason | Admit, reject or unresolved, with the reason | Admit, reject or unresolved, with the reason | Admit, reject or unresolved, with the reason | Admit, reject or unresolved, with the reason | Admit, reject or unresolved, with the reason | Admit, reject or unresolved |
Write the reason in the cell, not only a mark. In the worked row the reasons are no eligibility or denominator, no collection or location rule, click credit and order unit not stated, configured rule version absent, missingness and bias not reported, and no date plus a publisher with a commercial interest.
The Office for Statistics Regulation's questions to guide thinking about quality are ones analysts producing statistics can use in considering quality at each stage of production. They are written for official statistics, but they give a useful discipline for questioning an outside figure's inputs and limits.
A shared label does not make values comparable
A shared label does not make two values comparable. A supplier's platform-credit rate is bounded by its customer set, observed journeys, identity route and configured rule, so it cannot become an England causal benchmark.
An MMM estimate depends on inputs, specification and variation. An experiment estimates an effect for the studied population and conditions. These values are not interchangeable even when each is labelled attribution.
Three attribution values
Platform credit
- Bounded by
- Customer set and rule
- Estimates
- Platform-credit rate
- Can benchmark England causal?
- No
MMM estimate
- Bounded by
- Inputs and variation
- Estimates
- Model-based effect
- Can benchmark England causal?
- No
Experiment
- Bounded by
- Studied population
- Estimates
- Causal effect
- Can benchmark England causal?
- No
HM Treasury's QPIE guidance distinguishes observed change from causal impact in its government setting. A benchmark for platform credit cannot be used as a benchmark for incremental effect.
Collection differences matter too. The ICO's final storage and access technologies guidance covers technologies such as tags and pixels. Permission and configuration can change the observable population, so a comparison must disclose that boundary rather than treating unobserved people as equivalent non-events.
Build an internal reference without calling it normal
Where external admission fails, use the buyer's own consistent series, and do not call it normal. Freeze the metric card, source and query versions, eligible population and correction policy.
A metric card records the metric name, formula, numerator, denominator, unit, eligible population, source table, query version and correction policy. Mark method changes and avoid describing past performance as an industry norm.
The Government Analysis Function's guidance on communicating quality, uncertainty and change sets out principles for telling users about all three. It is aimed mainly at official-statistics producers but is useful for explaining quality and change to other readers of numbers.
Publish the admission log, not a fabricated target. The log shows which sources were rejected and why. A future benchmark becomes usable only when a named statistician verifies its population, method, uncertainty and decision relevance.
Before you act
- Define the user, population, event and denominator before searching.
- Apply the source admission tests to each candidate source.
- Reject comparisons that mix platform credit with incremental effect.
- Disclose permission and configuration boundaries affecting the observable population.
- Freeze the metric card, source and query versions.
- Publish the admission log.
Common questions
What has to be settled before a source is tested?
Write the benchmark record first: the user and action, population, England geography, event and unit, periods, attribution job and version, data route, late and corrected states, uncertainty method and provenance. If any field changes the meaning of the comparison, leave the candidate unresolved rather than guessing it.
Which tests does an outside figure have to pass?
Population match, genuine England geography, an identical metric, comparable attribution, assessable quality, and a record that is current and independent enough. In the worked row every test was rejected, so the decision was rejected, and each cell gave its reason.
What should a reader do when no external source passes the admission tests?
Use the buyer's own consistent series. Freeze the metric card, source and query versions, eligible population and correction policy. Mark method changes and avoid describing past performance as an industry norm. Publish the admission log, not a fabricated target.
