A NOTE BY TONY MCCLUNGEST. 2026

Pour & Loss

Wine and beverage numbers, one layer down.

THE CURRENT EDITIONNo. 06 / November 202615 min read

Your influencer report has never met a control group

Wine judges creators on the sales it can attribute to them. Every time researchers have put an attributed number next to an experiment, the attributed number came out wrong by multiples, and the best influencer study in print never ran the experiment.

Every week this industry produces more reporting than any owner has time to read, and most of it gets compressed into a headline that says almost nothing about your business. This note goes the other direction. One layer down, behind the numbers, with a bias toward what an owner should do about them. It arrives every other week. It will not always be comfortable.

01

When an attributed number meets a control group, it shrinks

In 2012, eBay switched off its paid search ads for non-brand keywords in 68 of the country's 210 Nielsen media markets for 60 days, and kept them running in 65 matched markets. Tom Blake, Chris Nosko and Steven Tadelis published the result in Econometrica in January 2015, under the title "Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment."

The sales credited to those ads, meaning purchases made within 24 hours of a paid click, fell by more than 72 percent in the dark markets. Total sales barely noticed. The authors' estimate is that the entire non-brand search program added 0.66 percent to sales, with a 95 percent confidence interval running from minus 0.42 to plus 1.74. Zero sits comfortably inside that range.

Then they ran the return on investment two ways (Table 1). Regress sales on ad spending, the way most companies read their own data, and the eBay numbers produce an ROI of 4,173 percent. Add controls for market and day and it's still 1,632 percent. Use the experiment instead and the ROI is minus 63 percent, with a confidence interval from minus 124 to minus 3. Same company, same ads. One method says the program was a money machine. The other says it lost about 63 cents of every dollar.

The ads moved new and lapsed customers. The people who clicked most, and who absorbed most of the spend, were eBay's frequent buyers, who would have bought anyway. The brand-keyword test was starker still. When eBay stopped buying ads on searches containing its own name on Microsoft and Yahoo, 99.5 percent of the lost clicks came back through the free search listing. eBay had been paying to show an ad to people who had just typed the word eBay.

The authors add a fair caveat: their result may not hold for small, new companies without brand recognition. Plenty of wineries are closer to unknown than eBay. That cuts in a specific direction. The ads moved people who didn't already know where to find you, and were wasted on people who did.

The second document is bigger. Brett Gordon and Florian Zettelmeyer of Kellogg, with Neha Bhargava and Dan Chapsky of Facebook, published "A Comparison of Approaches to Advertising Measurement" in Marketing Science (online April 2019). They took 15 US advertising campaigns that Facebook had run as randomized trials in 2015, covering 500 million user-experiment observations and 1.6 billion ad impressions. Then they threw away the control groups and asked the observational methods the industry actually uses to recover the answer from what was left.

They couldn't. In half the studies, every method missed the lift in purchases by a factor of three or more; seven of the fourteen campaigns that tracked checkouts missed by that much. The errors ran mostly high and occasionally low. One retailer's comparison of exposed and unexposed customers showed a 316 percent lift where the experiment found 73. On one e-commerce campaign the closest observational estimate was 1,306 percent. The experiment said 2.4.

Those methods use propensity scores, census data and Facebook's own behavioral signals. They are far more sophisticated than anything a winery runs. The number a winery can see from a creator post is the crude version: the redemptions on a code, the orders from a tracked link, the club sign-up somebody tagged "social" at the counter. It counts everyone who used the door, including everyone who was already on their way in.

Picture a club member who has bought every fall release for six years. A creator she follows posts your code in October, and she uses it, because who wouldn't. Your report books a new sale to the influencer program. What actually happened is you paid a creator to give your best customer a discount.

In thirty years I've sat through a lot of marketing reviews, and I can't remember one that came with a control group.

02

Exhibit: What the attributed number said, and what the experiment said

Campaign

Attributed or observational estimate

Experimental estimate

Source

eBay non-brand paid search, US, 2012

ROI of 4,173% (sales regressed on spend)

ROI of -63% (95% CI -124% to -3%)

Blake, Nosko and Tadelis, Table 1

Facebook study 4, omnichannel retailer, 2015

316% lift in purchases (exposed vs. unexposed)

73% lift

Gordon et al., Sec. 7.2.1 and Table 6

Facebook study 9, e-commerce, 2015

1,306% lift (closest observational method)

2.4% lift

Gordon et al., Sec. 7.2.3

Influencer posts, nano tier (bottom quartile), 2018-2021

Return on influencer spend of 17.85 (coded revenue, estimated fees)

Never tested

Beichert et al., Table 2

Influencer posts, macro tier (top quartile), 2018-2021

Return on influencer spend of 4.67 (coded revenue, estimated fees)

Never tested

Beichert et al., Table 2

03

The best study in the field measured coded sales, and says so

The most serious attempt to put revenue and cost against influencer posts is "Revenue Generation Through Influencer Marketing," by Maximilian Beichert, Andreas Bayerl, Jacob Goldenberg and Andreas Lanz, in the Journal of Marketing (vol. 88, no. 4, July 2024). It deserves its reputation. Most influencer research measures likes. This team measured euros, and they went to the trouble of running field studies on top of a large dataset.

The dataset came from a firm the authors describe as one of Europe's leading direct-to-consumer fashion companies: 2,808 influencer-specific discount codes created between 2018 and 2021, posted by 1,698 Instagram influencers, carrying 1,881,533 products and more than €17 million in revenue. They sorted influencers into quartiles of their own sample. The bottom quarter, which they call nano, ran from 1,219 to 8,496 followers. The top quarter, macro, ran from 49,845 to 1.4 million.

Table 2 has the strongest numbers. Revenue per follower was €0.265 for nano, €0.080 for micro and €0.049 for macro. Return on influencer spend, revenue less fee divided by fee, was 17.85, 5.98 and 4.67. On the authors' estimated fees, every tier brought in more coded revenue than it cost. The macro tier had about 32 times the followers of the nano tier and earned less than a fifth as much per follower. Three field studies with 319 paid influencers, two in German sneakers in 2020 and one in UK makeup in 2021, found the small creators ahead by 17 to 114 times on revenue per follower.

Revenue in this study means sales that came through the influencer's code, or in the makeup study through a personal swipe-up link. In the main dataset the authors didn't observe how many people saw each post or what each influencer was paid; both were inferred from industry sources (p. 46). So the per-reach figures and the return on spend sit on estimated denominators. No group of followers was ever held back from a post. The field studies compared small creators against big ones, recruited first come, first served; neither was compared against no post at all. The authors are candid that endogeneity "still poses a threat to our results" (p. 53), and they call for long-run work on customer lifetime value.

The abstract says small creators win by an order of magnitude. In the main dataset the gap is about five times per follower and about four times on return on spend; the order of magnitude comes from the field studies. And return on influencer spend is calculated on revenue, before cost of goods, and every coded order in the main dataset carried a 15 to 30 percent discount. Run it through your own margin before you call it profit. At a 50 percent gross margin, a return on influencer spend of 1.0 is breakeven.

None of that makes the tier finding wrong. It may well be right, and the direction matches what engagement data would predict. A 2024 meta-analysis in the Journal of the Academy of Marketing Science by Meizhi Pan, Markus Blut, Arezou Ghiassaleh and Zach Lee pooled 1,531 effect sizes from 251 papers. It found only five effect sizes linking follower count to sales, a pooled correlation of 0.10 with a confidence interval from minus 0.06 to 0.24, and no significant effect of any influencer characteristic on sales (Table 4). The research base on creators and actual sales is that thin. Beichert's team built the best part of it. What they measured is revenue that arrived through a code. Whether any of it was new revenue, nobody has tested.

04

Wine is a harder case than fashion, and measures less

What follows is my inference from how these studies attribute sales. No published study tests it in wine.

Every sales-linked influencer study I could find depends on a code or link redeemed in a store the brand owns. Wine breaks that in four places. A winery's most valuable customers are its most frequent ones, and a code posted to a creator's audience will find some of them; that is the eBay pattern exactly. Wine is age-gated, and under TTB Industry Circular 2024-1 (October 30, 2024), a post a winery pays a creator to make can count as an advertisement that has to carry the mandatory statements. Direct shipping stops at state lines, so a code can only convert where you can ship. And most volume moves through three tiers, where a post that sends someone to a retail shelf leaves no code behind at all.

Put those together and the attributed number in wine can be wrong in both directions at once. It overcounts the regulars who were coming anyway and undercounts the shelf. In fashion e-commerce the error mostly runs high. In wine you can't tell which way it runs, which is a worse position to budget from.

The industry's own data says most wineries aren't measuring at all. Silicon Valley Bank's 2021 Direct-to-Consumer Wine Survey (May 2021, 460 producers reporting 2020 data) found that 82 percent of wineries were unsure of their return on digital marketing spend. Spending on social and search together averaged 1 to 1.5 percent of sales, and about 20 percent of new club members joined through digital channels. Those shares are self-reported, and the survey doesn't ask how anyone measured them. The minority that tracked a return reported strong ones. I'd want to see the method before I believed a number from that group either.

I looked for a published figure on what US wineries spend on influencers and creators, separate from other social spend. SVB's 2021 direct-to-consumer survey doesn't break it out, and I found nothing from Wine Institute, the Sonoma State Wine Business Institute or the DTC platforms that does. There is no wine number for this line. That absence is part of the finding: an industry can't overpay or underpay for something it has never counted.

05

Every objection ends at the same missing test

"It's brand building. You can't measure it in a quarter." This is the strongest objection and it has real evidence behind it. Raj Sethuraman, Gerard Tellis and Richard Briesch's meta-analysis in the Journal of Marketing Research (June 2011) put the average long-term advertising elasticity at 0.24 against 0.12 in the short term. Carryover is real. And Gordon's Facebook study excluded brand-building campaigns measured by surveys (footnote 6), so its finding speaks to conversion campaigns. That cuts both ways, and I'll say so: the Facebook evidence doesn't condemn brand campaigns, and nothing in it supports them either. The answer is where the carryover evidence comes from. Sethuraman's estimates come from 56 studies of television, print and combined advertising published between 1960 and 2008. Instagram didn't exist. Borrowing that carryover for creator posts is an assumption. Verdict: the answer holds. The objection borrows its evidence from other media.

"Gen Z discovers wine on social. If you're not there, you're invisible." Wine Market Council and Material released "Wine, Reframed" on September 2, 2026, from a survey of 900 US alcohol consumers aged 21 to 65, split across three generations and three drinker segments. In the release, Material's director says Gen Z is discovering wine through social media. Nine cells from 900 people is about 100 per cell if they're even, and the release doesn't print the question. The full report is members-only. More to the point, discovery is a different event from a purchase, and a paid post converting a buyer is a third thing again. Verdict: partly holds. Being present on social is defensible. Paying creators on the theory that they convert has no wine evidence behind it.

"Our club sign-ups prove it works." This is the eBay problem with a membership card. A sign-up that arrives through a creator's link tells you the person used the link. It doesn't tell you whether they were a customer last year, or would have joined at the next release. Verdict: the answer holds, unless you have run a holdout.

"Every channel is unmeasurable. Why hold creators to a higher standard?" Concede it, because it's true. Randall Lewis and Justin Rao analyzed 25 randomized display-ad experiments run at Yahoo between 2007 and 2011 for major US retailers and brokerages ("The Unfavorable Economics of Measuring the Returns to Advertising," Quarterly Journal of Economics, 2015). The median retail campaign's confidence interval on ROI was more than 100 percentage points wide. Only three of the 25 could reliably tell a wildly profitable campaign from one that broke even, and the median campaign would have needed to be nine times larger to do it. Then turn it around. If nothing is precisely measurable, the right response is a budget small enough to test and a report that treats the attributed figure as unverified. Lewis and Rao also say their results are least likely to apply to products with low baseline awareness, which describes a lot of small wineries. That's an argument for testing: the effect a small winery is looking for may be big enough to see. Verdict: the objection is correct, and it supports this issue.

"Big creators buy awareness small ones can't." Beichert's team concedes this directly: when the goal is awareness and immediate revenue isn't, large creators probably still make sense (p. 56). Verdict: unresolved. Nobody has run the test that would settle it, for wine or anything else.

06

Wine's best evidence for an endorser is twenty years old

The best evidence that an endorsement sells wine is about critics, and it's getting old. Richard Friberg and Erik Grönqvist's "Do Expert Reviews Affect the Demand for Wine?" (American Economic Journal: Applied Economics, January 2012) tracked weekly sales at Sweden's state retail monopoly and found that a favorable review in the Swedish press lifted demand 6 percent at peak, the week after it ran, and the effect stayed significant for more than 20 weeks. The data cover 526 wines and 5,093 reviews from 2002 to 2007, in a market that doesn't map cleanly onto American three-tier. Every other sales test I could find uses data from the same era, and nobody has rerun one on recent data.

What has been measured since points away from the critic. In Applied Economics (October 2024), Stephen Bazen, Jean-Marie Cardebat and Magalie Dubois ran 36,970 French red wines through a price model and found that Vivino's community ratings have a larger effect on price than expert scores, a result that holds with the top-end wines taken out. That's prices, not sales, and it can't say which way the cause runs. But in that market prices track the crowd more closely than the critic, and nobody has measured what the crowd does to a sale either.

Both of those voices are unpaid. A creator the winery pays is running an advertisement. Assuming it works because reviews used to is the same mistake as reading a code report as proof.

07

What I'd actually do

Three moves before the year closes. None requires capital.

Run your next creator push as a test. Give it a direct-to-consumer code, then hold part of it back: if the content runs as paid partnership ads, show them in some of your shipping states and not others; if it's organic, alternate weeks on and off. Judge it on total direct orders in the on markets against the off markets, never on code redemptions. Ask your Meta and TikTok reps whether your account qualifies for a conversion lift study on any paid amplification. TikTok documents one, Facebook offered one free to qualifying advertisers when the Gordon study ran, and nothing I found in either platform's documentation says whether an alcohol account can use it. What this reveals is your own ratio between what the code says and what actually moved.

Match last year's code redeemers against your customer file. Count how many were club members or had bought from you in the prior twelve months. That's the eBay split done with a spreadsheet you already own, and it shows how much of your creator line was a discount handed to people who were coming anyway.

Size the creator budget to what you can read, and move it down the follower curve. Split it across several smaller creators, each on a separate code, rather than one large deal you can't evaluate. That gives you a comparison inside your own data on the direction Beichert found, and it puts a ceiling on what an unreadable bet can cost.

Until a winery runs that test and publishes it, every creator report in this business is an unchecked number, and nobody can say which way it's off.

Tony

Sources this issue

Blake, Nosko and Tadelis, "Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment," Econometrica 83(1), January 2015 (accepted manuscript, August 2014); Gordon, Zettelmeyer, Bhargava and Chapsky, "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science 38(2), published online April 4, 2019; Beichert, Bayerl, Goldenberg and Lanz, "Revenue Generation Through Influencer Marketing," Journal of Marketing 88(4), July 2024; Pan, Blut, Ghiassaleh and Lee, "Influencer Marketing Effectiveness: A Meta-Analytic Review," Journal of the Academy of Marketing Science 53(1), published online October 12, 2024; Lewis and Rao, "The Unfavorable Economics of Measuring the Returns to Advertising," Quarterly Journal of Economics 130(4), 2015; Sethuraman, Tellis and Briesch, "How Well Does Advertising Work? Generalizations from Meta-Analysis of Brand Advertising Elasticities," Journal of Marketing Research 48(3), June 2011; Friberg and Grönqvist, "Do Expert Reviews Affect the Demand for Wine?," American Economic Journal: Applied Economics 4(1), January 2012 (abstract and web appendix); Bazen, Cardebat and Dubois, "The role of customer and expert ratings in a hedonic analysis of French red wine prices: from gurus to geeks?," Applied Economics 56(46), October 2024; Silicon Valley Bank, 2021 Direct-to-Consumer Wine Survey Report, Results and Benchmarks, May 2021; Wine Market Council and Material, "Wine, Reframed" release, September 2, 2026; TTB Industry Circular 2024-1, "Use of Social Media in the Advertising of Alcohol Beverages," October 30, 2024; TikTok Business Help Center, "About Conversion Lift Study," April 2025.

A Statement About P&L

Pour & Loss is a note by Tony McClung, published every other week, looking behind the wine and beverage numbers with a bias toward what an owner should do about them.

Tony brings thirty years in senior commercial roles in wine and beverage, an Advanced Sommelier qualification, and a master’s in economics. He runs True Origin Collective, a strategic advisory practice serving wineries and beverage businesses.

More about Tony