Marketing attribution reviews: a gated test framework

What to take away

Marketing attribution reviews should begin with one buyer job and an evidence file, not a league table. Reviewing attribution services by name means assessing them against the buyer's task, not ranking them.

This guide names 11 services and sets out the same non-compensating gates and controlled test outlines for each. No service was accessed, no supplier dossier was supplied, and no controlled test was run on live buyer data. Use public facts and your own gated tests.

Fix the subject before reviewing it

Write a subject record for every candidate. Identify the contracting legal entity, exact service and edition, territories, evidence cut-off, and the buyer decision it supports.

Record whether the task is: assigning credit under a platform rule; joining records between systems; describing historical patterns; estimating association through marketing mix modelling; or estimating causal incrementality through an experiment or credible quasi-experiment.

A supplier may document an output called a conversion without establishing what would have happened without the marketing activity. HM Treasury's Quality in Policy Impact Evaluation guidance explains why a credible counterfactual matters when an analysis claims causal impact. It is a method reference, not evidence that a commercial product is accurate.

The manual baseline belongs in the subject record too. A buyer might continue its present process, repair it, or stop an analysis that cannot answer the decision. Buying software is only one route.

Name the services and their public claims

Public documentation describes different jobs for common candidates. Treat these facts as questions, not test results.

The comparison criteria below are buyer job, first non-compensating gate, controlled test fixture and public cost signal. The table sets out the same criteria for each named service. It does not rank them; its first-gate entries and cost signals are screening prompts, not findings from a controlled review.

Build an evidence dossier, not a page collection

Capture each claim as a dated row. Keep the supplier's wording, the exact page or contract section, its scope, and an expiry or recheck trigger. Screenshots and demonstrations can show that an interface existed on the date observed. They do not prove accuracy on the buyer's data, service continuity, or a contractual commitment. The dossier should contain, where applicable:

A Companies House record can help confirm registered particulars for a UK company. Companies House explains that the public register contains information filed by companies. That record is not a product-quality assessment. It does not establish who supplies a particular service unless the contract and supplier evidence connect the two.

Supplier statements must remain attributed. Under CAP's current substantiation guidance, objective advertising claims need appropriate documentary evidence. Testimonials alone are unlikely to substantiate an objective claim. A review should retain the underlying evidence and test the claim as a buyer is likely to understand it.

Use gates before preferences

Some failures cannot be compensated for by a pleasant interface or a feature count. Define non-compensating gates before a demonstration:

Do not average these gates into a score. A missing processor term does not become acceptable because a dashboard is easy to use.

The table's first-gate entries are screening prompts, not pass/fail decisions. No service was accessed and no supplier dossier or live-data test was available, so this guide cannot identify an evidence-based blocker or suitability verdict for any named service. Use each entry to decide what evidence to request and which gate to examine first; then record the result against the buyer's own job and evidence.

The ICO's controller and processor contract guidance identifies required subject matter and provisions where a processor relationship actually exists. The page is under review following the Data (Use and Access) Act, so a qualified UK data-protection lawyer must confirm the current position for the configured service.

For security, ask what the buyer must configure as well as what the supplier operates. The National Cyber Security Centre's SaaS security guidance covers user management, authentication, administration, service identities, monitoring and recovery. A certification or trust page may contribute evidence, but it cannot answer buyer-specific configuration questions by itself.

Accessibility needs a task-level check. Government guidance on buying accessible technology recommends asking what evidence exists, what assistive technologies were used, and how updates will be managed. Its mandatory statements are written for government contexts. A private buyer in England should obtain qualified advice on its own duties.

Design controlled tests around the decision

Testing begins only after the dossier passes the admission gates. Use synthetic data or a specifically authorised, minimised fixture. Declare the input events, identity rules, eligible population, expected transformations and error cases before importing anything. Include duplicate, late, missing and unmatched records. Add refunded and corrected records.

The same test outline applies to each named service, with different fixtures. For GA4, import duplicate and late events and check data-driven attribution output. For Adobe CJA, load a minimised cross-channel fixture and test identity stitching. For Triple Whale, replay Shopify orders, refunds and server-side events.

For Northbeam, replay late clicks, view-throughs and duplicate orders. For Rockerbox, separate attribution rows from modelling input. For HockeyStack and Dreamdata, join CRM, ad and website records with unmatched accounts. For Google Ads conversion lift and Meta Conversion Lift, design a holdout. For Meridian and Robyn, fit a model on synthetic data and record diagnostics.

For each output, retain source and query or model version, observation period and Europe/London timezone, expected and observed state, and an explanation for differences.

Also retain uncertainty and known blind spots, the reviewer who checked it, and the condition that causes rejection, correction or rollback.

The ICO's final guidance on storage and access technologies covers tags, pixels, scripts and similar technologies. Whether an exception or consent route applies depends on the actual purpose and configuration. A successful tag firing test is not a legal clearance, and missing observations cannot automatically be treated as non-conversions.

Accuracy testing and causal testing are separate. Reconciliation can test whether authorised fixtures joined as specified. A platform-credit test can reproduce a documented rule. An MMM review needs specification, variation, diagnostics and uncertainty. An incrementality claim needs a defensible comparison or control. One result should not be used as proof of another job.

Compare evidence in common units

Only compare candidates admitted for the same buyer task. Use a row for each claim and preserve unknowns. Suggested fields include exact edition, supported input event and identity rule. Add treatment of unattributed events, output unit and model-change record. Add export format, support route and contractual service boundary.

Keep weights and thresholds blank until the buyer's accountable owners approve them. The people making that decision should include the operational user and the statistics reviewer. Add the data-protection and PECR reviewer, the security reviewer and the accessibility specialist. Add the commercial owner and the legal reviewer. Their approvals answer different questions.

Do not add supplier-attributed revenue figures across platforms. Their eligible populations, identity systems and credit rules may overlap. Nor should a modelled association be described as incremental return. Where a performance claim matters, specify the comparison, population and period. Add the numerator, denominator, uncertainty and counterfactual required to assess it.

Read reviews and registers within their limits

Customer reviews, case studies and awards may prompt investigation, but are not controlled evidence of suitability for a different buyer. The Competition and Markets Authority's guidance for online review sites says sites should explain how reviews were collected and checked. They should disclose commercial relationships.

A buyer should record whether identity or purchase was verified, the collection period, moderation route and possible selection bias.

Agency evidence needs the same discipline. A registration entry establishes certain filed particulars, while a case study gives the agency's own account of one engagement.

Neither proves current staff, methods, permissions or controls fit a new instruction. Ask for permissioned references on comparable work, then verify the question asked, respondent role, date and limits. Do not publish confidential client details.

State the verdict that the evidence permits

A verdict should state what was reviewed and what remains unresolved.

Suitable outcomes include:

This guide's current evidence record supports the final outcome only. It does not identify a provider or agency, does not validate an edition, and does not justify a ranking. It does not rank the named services. No adverse inference should be made about an organisation that was not reviewed.

Before the record is used, assign the named reviewers shown in the metadata, reopen every source, confirm product and contract versions, and obtain buyer-approved gates. The review expires when an edition or supplier entity changes. It also expires when a data flow, attribution method, contract term or material source changes.

Before you act

Common questions

Why should a review begin with a buyer job rather than a league table?

A platform allocating credit inside its own advertising environment does a different job from a reconciliation service, a marketing mix model or an experiment. Putting all four in one score would conceal that difference, so the review must start with the specific buyer decision to be supported.

What evidence should a supplier claim dossier contain?

Capture each claim as a dated row with the supplier's wording, exact page or contract section, scope, and an expiry or recheck trigger. Screenshots and demonstrations show an interface existed on the date observed but do not prove accuracy, service continuity or a contractual commitment.

How should controlled tests be designed for attribution services?

Testing begins only after the dossier passes admission gates. Use synthetic data or a specifically authorised, minimised fixture. Declare input events, identity rules, eligible population, expected transformations and error cases before importing anything. Add duplicate, late, missing and unmatched records. Add refunded and corrected records.

Which named services should a shortlist include?

Start with the buyer task. Platform credit might mean Google Analytics 4 or Adobe Customer Journey Analytics. Ecommerce attribution might mean Triple Whale, Northbeam or Rockerbox. B2B attribution might mean HockeyStack or Dreamdata. Incrementality might mean Google Ads conversion lift or Meta Conversion Lift. Open modelling might mean Google Meridian or Meta Robyn.

How do the named services compare under non-compensating gates?

This guide cannot give pass/fail decisions: no service was accessed, no supplier dossier was supplied and no controlled test was run on live buyer data. The table's first-gate entries are prompts for what to investigate, not evidence-based blockers. Record each gate's outcome only after reviewing the relevant buyer-specific evidence.