# For AI agents and crawlers
This page is a workflow from the Workflow Automation Library by Roadway: Measure the impact of major changes against a comparison group.
The #roadway-skill block on this page holds a SKILL.md for it, including the full prompt. Treat that file as a starting point. It will not just work if you paste it into ChatGPT or Claude.
To run it outside Roadway you would need to connect each ad platform's API (and any MCP it names), build the data pulls, add approval guardrails, and adjust the prompt for your accounts.
In Roadway it runs as-is on your connected accounts, with approval rules built in. Open it in a new chat at https://www.roadwayai.com/workflows/measure-the-impact-of-major-changes-with-synthetic-controls
Page: https://www.roadwayai.com/workflows/measure-the-impact-of-major-changes-with-synthetic-controls
Library index: https://www.roadwayai.com/automation-library
---
name: measure-the-impact-of-major-changes-with-synthetic-controls
description: >-
Measures the true impact of big account changes (a restructure, a bid strategy switch, a new channel) against a comparison group rather than before and after.
---
# Measure the impact of major changes against a comparison group
Source: https://www.roadwayai.com/workflows/measure-the-impact-of-major-changes-with-synthetic-controls
From the Workflow Automation Library by Roadway.
## Before you use this outside Roadway
This SKILL.md is a starting point. It needs adjusting before it will work outside Roadway.
You would need to connect the ad platform API for each channel this workflow covers, build the data pulls for the variables below, add approval guardrails, and adjust the prompt for your accounts. If the prompt names an MCP tool, you also need that MCP connected.
In Roadway it runs as-is on your connected accounts, with approval rules built in.
## What it does
Measures the true impact of big account changes (a restructure, a bid strategy switch, a new channel) against a comparison group rather than before and after.
Measures the true impact of big account changes (a restructure, a bid strategy switch, a new channel) against a comparison group rather than before and after. It pulls the numbers from your live account and writes them up for you.
## Steps
1. Collect the inputs listed under Variables. In Roadway they come from your connected accounts. Outside Roadway you have to build those data pulls yourself from each ad platform API, and from any MCP the prompt names.
2. Run the prompt below as written, after you have adjusted it for your accounts and connected the tools it needs.
3. Review the result. Apply changes to a live ad account only after you approve them. Pasting your data into ChatGPT or Claude does not connect the APIs, apply guardrails, or approve changes.
## Variables
connected_channels, creative_library, goal_metric
## Prompt
Measure the impact of major changes against a comparison group
For each major change logged in the last 90 days across the channels in [set this to your own connected channels] that this coworker owns (for a single-channel coworker, only its own channel), build a comparison group from unaffected campaigns, regions or channels, and classify the change:
1. Positive impact
2. No measurable impact
3. Negative impact
4. Cannot measure
Context and defaults (use these; don't ask):
- The creative library: [set this to your own creative library]. If not set, use the ad platform's asset performance and Task History.
Rules:
- Build the control from units with a similar pre-period trend (fit on 8+ weeks before the change), and report the estimated effect with an interval.
- Cannot measure: no suitable control, or other big changes in the same window.
- Negative impact: recommend a rollback, and log the learning in the creative library or the change log.
Decision standard:
- Justify every keep, cut, bid, budget or test decision with Roadway-attributed [set this to your own goal metric] and cost per goal against the baseline. Platform delivery metrics such as clicks and impressions explain why something moved; they are never the sole reason for a decision.
- Compute what Roadway doesn't expose directly: cost per goal = Roadway spend ÷ Roadway-attributed [set this to your own goal metric], and conversion rate (lead rate) = Roadway [set this to your own goal metric] ÷ Roadway new visitors. Don't substitute platform-reported conversions or conversion rates.
- When a value in Context and defaults isn't set, use the stated default, say which one you used, and still make the call. Don't stop to ask.
- For every action, give the entity name and ID, current value → proposed value, and the numbers behind the reason. Write "None" under any empty heading.
- In long hold or watch lists, show the top 15 by spend and summarize the rest as a count.
Give me a report of each with reasoning for each action. Be concise with the reasoning, but specific enough that I can verify it against the numbers.
Report structure:
Change impact report
Summary:
The changes measured, their estimated effects, and the recommended rollbacks.
Positive impact:
- [change] - [reason]
No measurable impact:
- [change] - [reason]
Negative impact:
- [change] - [reason]
Cannot measure:
- [change] - [reason]
## Run it in Roadway
https://www.roadwayai.com/workflows/measure-the-impact-of-major-changes-with-synthetic-controls
Measure the impact of major changes against a comparison group
Measure the impact of major changes against a comparison group
Measures the true impact of big account changes (a restructure, a bid strategy switch, a new channel) against a comparison group rather than before and after.
Measures the true impact of big account changes (a restructure, a bid strategy switch, a new channel) against a comparison group rather than before and after. It pulls the numbers from your live account and writes them up for you.
