Your attribution model says paid LinkedIn drove 40 demo requests last quarter. Great news, right? Maybe. But here's the question most B2B marketing teams never actually answer: would those 40 people have requested a demo anyway, even if they'd never seen your ad?
That's the gap attribution models cannot close. They tell you which channels were present when a conversion happened. They distribute credit across touchpoints according to rules or algorithms. What they cannot tell you is whether removing a channel from the equation would have changed anything at all.
This is where incrementality testing comes in. It's the methodology that moves marketing measurement from correlation to causation, from "this channel was there" to "this channel made it happen." For B2B SaaS teams making serious budget decisions across long sales cycles and multiple touchpoints, that distinction is not a minor nuance. It's the difference between scaling what works and pouring budget into channels that are simply along for the ride.
This guide breaks down what incrementality testing is, how it works in practice, when to run it, and how to connect it to your broader attribution strategy. Whether you're questioning the true value of your retargeting campaigns or trying to justify a demand generation investment to leadership, this is the framework you need.
Causation vs. Correlation: The Problem at the Heart of Marketing Measurement
Let's start with a scenario that will feel familiar. A prospect visits your website after seeing a LinkedIn ad, reads a case study, searches your brand name on Google, clicks a branded search ad, and books a demo. Your last-click model credits Google. Your first-touch model credits LinkedIn. Your linear model splits credit across every touchpoint.
None of these models ask the harder question: what if you had turned off the LinkedIn campaign entirely? Would this person still have found you, searched your brand, and converted? Quite possibly, yes.
This is the difference between correlation and causation. Correlation means a channel was present during a conversion. Causation means the channel was responsible for it. Attribution models, by design, measure correlation. They map what happened. They do not measure what would have happened in the absence of a specific marketing activity.
Incrementality in marketing is defined as the measurable lift in conversions that is directly caused by a specific marketing activity, above and beyond what would have occurred organically. Think of it as your baseline demand versus your ad-driven demand. Some portion of your pipeline would come in regardless of whether you ran a single campaign. Incrementality testing isolates exactly how much of your pipeline is truly ad-generated.
Why does this matter for budget allocation? Because spending on channels that only capture existing intent, rather than creating new intent, is a costly mistake that attribution models can mask for months or even years.
Consider branded search. A prospect who already knows your product and types your brand name into Google is expressing intent that almost certainly exists independent of your ads. If you're running branded search campaigns and attributing every resulting conversion to that campaign, you may be paying for clicks that would have arrived as direct traffic anyway. The channel looks efficient. The attribution report looks clean. But the incremental value may be far lower than the numbers suggest.
For B2B SaaS teams managing limited budgets and pressure to generate net-new pipeline, this distinction is critical. Every dollar allocated to a channel that captures existing demand is a dollar not invested in channels that could generate genuinely new pipeline. Incrementality testing gives you the evidence to make that call with confidence rather than assumption.
The Mechanics Behind an Incrementality Test
At its core, an incrementality test is a controlled experiment. The logic is straightforward: take a defined audience, split them into two groups, show ads to one group and not the other, then compare conversion rates between the two. The difference in conversion rates represents the incremental lift from your advertising.
The group that sees the ad is called the exposed group or test group. The group that does not see the ad is called the holdout group or control group. The holdout group serves as your counterfactual: it shows you what conversion behavior looks like in the absence of the marketing activity you're testing.
After the test period ends, you compare three key metrics across both groups.
Incremental Conversions: The number of conversions in the exposed group that exceed the conversion rate of the holdout group, scaled to the size of the exposed audience. This tells you how many conversions your campaign actually caused, rather than just captured.
Incremental Revenue: The revenue tied specifically to those incremental conversions. In B2B SaaS, this might be expressed as incremental pipeline value or incremental closed-won revenue, depending on where in the funnel your test is focused.
Cost Per Incremental Conversion: Your total ad spend for the test period divided by the number of incremental conversions. This is the true cost of a conversion your campaign actually drove, and it's often significantly higher than the cost per conversion your attribution model reports.
There are two primary approaches to structuring the holdout group, and the right choice depends on your campaign type and technical setup.
Geo-Based Holdouts: You assign entire geographic regions to either the test or control condition. One set of markets sees your campaign, another comparable set does not. This approach is practical when user-level audience suppression isn't technically feasible, and it works well for broad awareness campaigns or display advertising. The tradeoff is less precision, since regional differences in market conditions can introduce noise into your results.
User-Level Holdouts: You randomly split your target audience at the individual level, suppressing ads for the holdout segment within the ad platform itself. This approach is more precise and is generally preferred for B2B campaigns on platforms like LinkedIn or Meta, where audience-level controls are available. It reduces the risk of confounding variables that can affect geo-based tests.
One important practical consideration: a valid incrementality test requires sufficient conversion volume to produce statistically significant results. If your monthly demo request volume is low, a short test period may not generate enough data to distinguish real lift from random variation. For smaller B2B SaaS teams, this means planning tests carefully around periods of higher activity or extending the test window to accumulate enough data.
Why Standard Attribution Models Give You an Incomplete Picture
Attribution models are useful tools. They help you understand the customer journey, identify which channels appear frequently in conversion paths, and allocate budget based on touchpoint patterns. But they have a structural limitation that no amount of model sophistication can fully overcome: they assign credit, they don't establish cause.
Last-click attribution gives all credit to the final touchpoint before conversion. First-touch attribution gives all credit to the first interaction. Linear attribution models spread credit evenly. Time-decay models weight recent touchpoints more heavily. Each model has its logic, but none of them can answer the counterfactual question: would this conversion have happened if this touchpoint had never occurred?
This limitation becomes especially costly in one specific scenario: retargeting.
Retargeting campaigns target users who have already visited your website, engaged with your content, or interacted with your brand in some way. By definition, these are people who are already aware of you and likely already considering your product. They are, in many cases, going to convert regardless of whether they see another ad.
When a retargeted user converts, attribution models credit the retargeting campaign. The campaign looks highly efficient because conversion rates among retargeted audiences are naturally high. But the incrementality of that campaign may be very low, because most of those conversions would have happened through organic channels anyway.
This pattern plays out across B2B SaaS marketing in ways that can quietly distort budget decisions for months. A retargeting campaign targeting trial users who are already in active evaluation looks like a top performer in your attribution dashboard. A branded search campaign capturing people who already know your product and were going to find you regardless looks indispensable. Meanwhile, the demand generation campaign that actually introduced your brand to net-new buyers gets undervalued because it sits earlier in the funnel and touches fewer conversions directly.
The longer your sales cycle, the more pronounced this problem becomes. In B2B SaaS, where deals can take months to close and involve multiple stakeholders, the gap between touchpoint credit and actual causal contribution is wide. Attribution models trace the path. Incrementality testing tells you which steps on that path actually mattered.
Knowing When to Run a Test
Incrementality testing is a powerful tool, but it's not something you run on every campaign at all times. Designing a meaningful test requires the right conditions, a clear hypothesis, and a willingness to accept a short-term tradeoff.
The right conditions start with conversion volume. You need enough conversions across both your test and holdout groups to reach statistical significance. If your campaign generates only a handful of conversions per month, the test results will be too noisy to act on. A general rule: the lower your conversion volume, the longer your test period needs to be. For B2B SaaS teams with modest monthly demo or trial volumes, planning tests over six to eight weeks rather than two is often necessary.
You also need stable campaign performance before you begin. Running an incrementality test during a period of major creative changes, budget fluctuations, or seasonal demand shifts introduces variables that make it harder to isolate the effect of the channel itself. Establish a baseline period of consistent performance, then launch the test.
A clearly defined hypothesis is equally important. The best incrementality tests are built around a specific question about a specific channel. Common testing scenarios for B2B SaaS teams include:
Branded Search Incrementality: Are the conversions attributed to your branded search campaigns truly incremental, or would those prospects have found you through direct traffic or organic search anyway? This is one of the most common and high-value tests for B2B SaaS companies, because branded search often appears highly efficient in attribution reports while potentially capturing mostly organic intent.
LinkedIn Retargeting Lift: Is your LinkedIn retargeting campaign actually lifting demo request rates among warm audiences, or are those audiences converting at a high rate because they were already going to convert? Testing this can reveal whether retargeting spend is generating real pipeline or simply claiming credit for it.
Demand Generation Channel Validation: Is a new awareness or demand generation campaign generating net-new pipeline, or is it reaching prospects who would have entered your funnel through other channels regardless? Understanding assisted conversions can help contextualize where demand generation efforts appear across the full customer journey.
There is one tradeoff you have to accept going in: the holdout group deliberately withholds ads from potential buyers. Some of those buyers may not convert during the test period as a result. You are accepting a potential short-term dip in attributed conversions in exchange for longer-term clarity on where your budget is actually working. For most B2B SaaS teams, that tradeoff is well worth making, but it requires buy-in from leadership before the test begins.
How Incrementality Testing Fits Into Your Attribution Stack
A common misconception is that incrementality testing replaces multi-touch attribution. It doesn't. The two approaches answer different questions and work best when used together.
Multi-touch attribution maps the customer journey. It shows you which channels and touchpoints appear across your conversion paths, how often they appear, and in what sequence. This is valuable for understanding how buyers move through your funnel and which channels are consistently present in high-value journeys. Choosing the best attribution model for your ad campaigns is itself a strategic decision that shapes how you interpret every touchpoint in your data.
Incrementality testing validates whether the channels in that journey are actually driving outcomes. Think of it as a layer of causal verification on top of your attribution data. Attribution tells you the story. Incrementality testing confirms which chapters of that story actually caused the ending.
Used together, these approaches give you a much more complete and actionable picture of marketing performance than either can provide alone. You can use attribution data to identify which channels look influential, then use incrementality testing to confirm or challenge that appearance with causal evidence.
But here's the prerequisite that many teams overlook: incrementality tests are only as reliable as the conversion data feeding them. If your tracking is incomplete, if conversions are being missed in either the test or holdout group, your results will be distorted. A channel might appear to have low incrementality simply because conversions in the exposed group are being undercounted.
This is where server-side tracking and first-party data become critical. Browser-based tracking is increasingly unreliable due to ad blockers, browser privacy restrictions, and cookie limitations. Server-side tracking and Conversion API integrations capture conversion events directly from your server, bypassing client-side data loss and ensuring both your test and holdout groups are measured accurately.
This is the foundation Cometly is built on. By connecting your ad platforms, CRM, and website through server-side tracking and Conversion API integrations, Cometly gives B2B SaaS teams a complete, accurate view of every touchpoint across the customer journey. When you're ready to run an incrementality test, you need to know that your baseline conversion data is clean and complete. Cometly's real-time pipeline and revenue attribution means you're not designing tests on top of incomplete data. You're working from a single, reliable source of truth that captures every conversion event, from first ad click to closed-won revenue, so your test results reflect reality rather than tracking gaps.
Turning Test Results Into Budget Decisions
Running an incrementality test is only valuable if you're prepared to act on what you find. The results will typically fall into one of two categories: a channel has meaningful incremental lift, or it doesn't.
If a channel shows strong incrementality, that's a signal to invest more confidently. You have causal evidence that this channel is generating conversions that wouldn't have happened otherwise. Scale budget with confidence, and use the cost per incremental conversion as your true efficiency benchmark rather than the cost per conversion your attribution model reports. Tracking attributed revenue alongside incremental results gives you a fuller picture of where genuine growth is coming from.
If a channel shows low incrementality, the right response is not necessarily to eliminate it immediately. Low incrementality means the channel is capturing conversions that would likely have occurred anyway, but it doesn't always mean the channel has zero value. Some channels play important roles in brand visibility or competitive defense even if their direct causal contribution to conversions is limited. The appropriate response is to reduce spend, reallocate the freed budget to higher-incrementality channels, and potentially retest with a refined audience or creative strategy before making a final call.
Incrementality findings should also feed back into how you think about attribution model selection. If your testing reveals that retargeting has lower true lift than your last-click model suggests, that's a signal to move toward a data-driven or position-based attribution model that doesn't over-weight the final touchpoint. Your attribution model should reflect the causal reality your incrementality tests are revealing, not the other way around.
Perhaps most importantly, incrementality testing should be a recurring practice, not a one-time audit. Markets shift, buyer behavior evolves, and the incremental value of a channel today may be different six months from now. Building a testing cadence into your quarterly planning process, rotating through your highest-spend channels systematically, ensures that your budget allocation stays grounded in current causal evidence rather than historical assumptions.
For B2B SaaS teams managing complex, multi-channel campaigns across long sales cycles, this kind of disciplined, evidence-based approach to budget allocation is what separates teams that consistently grow pipeline from teams that are perpetually guessing.
The Bottom Line on Incrementality Testing
Knowing which channels get credit for your conversions is not the same as knowing which channels drive your growth. Attribution models are essential tools for understanding the customer journey, but they cannot answer the question that matters most for budget decisions: would this conversion have happened without this ad?
Incrementality testing closes that gap. By comparing conversion behavior between an exposed group and a holdout group, it provides causal evidence for marketing decisions rather than correlational patterns. For B2B SaaS teams navigating long sales cycles, multiple stakeholders, and pressure to prove marketing's contribution to pipeline, that causal evidence is invaluable.
If you're not sure where to start, pick one channel you suspect might be over-credited in your attribution reports. Branded search and retargeting are the two most common candidates. Define a clear hypothesis, ensure your conversion tracking is complete and accurate, and design a simple holdout test with a defined measurement window.
Accurate attribution data is the foundation of any reliable incrementality test. You need clean, complete conversion data across every touchpoint to design meaningful tests and interpret results with confidence. Cometly gives B2B SaaS teams exactly that: a real-time, connected view of every touchpoint from first ad click to closed-won revenue, built on server-side tracking that captures what browser-based tools miss.
When you're ready to move beyond vanity attribution and measure what actually moves the needle, Get your free demo and see how Cometly gives your team the data foundation to run incrementality tests that produce results you can actually act on.
