EVENT MEASUREMENT
IN THE AI AGE

Travel back in time with us for a moment. It’s 2014 and EM has just released a comprehensive event measurement report. We find that marketers are laser-focused on big data. Quantity is the name of the game as measurement meets the digital age. Or, as, Michael Curmi, head of experiential marketing at Chrysler, told us: “Technology has allowed us to really have an ease of collecting, of volume, and putting our finger on the metrics for everything we do now.”

Ten years (and one devastating global pandemic) later, the meteoric rise of artificial intelligence has enabled the industry to focus on collecting the right data—and make it actionable. Marketers are prioritizing metrics that inform strategic decision-making. They’re using new, and much improved, technology and taking cues from neuroscience. As a result, they’re gathering meaningful intel from, and about, their audiences, then reimagining their events based on attendees’ needs.

The shift is all about quality over quantity in every sense. Consider trade show data from EM’s EventTrack 2024 study, in which 66 percent of exhibitors said on-site engagement has become the most critical metric, followed closely by leads, at 64 percent. “It’s less about driving badge scans and more about getting the right people in,” says Kate Alter, trade show and event manager at Philips.

It’s a similar story on the b-to-c side, with brands leaning heavily on social media engagement, and experimenting with activations on newer digital platforms, like Roblox, to meet the right audiences exactly where they are—then measure the interactions that matter.

Indeed, a new ROI era has dawned. So we’ve surveyed marketers and scanned the measurement landscape to find out how refined metrics, AI-optimized tools and an evolving landscape of influence are changing the way experience builders work to prove a tangible return on their investments. This is event measurement in the AI age.

Security regulations like the General Data Protection Regulation (GDPR) and the California Privacy Rights Act have required marketers to learn the intricacies of managing personal information in recent years. Now, with the proliferation of artificial intelligence, comes a new wave of legislation to untangle. At the forefront is the European Union’s AI Act, the world’s first comprehensive legal framework for the development and use of the technology, which took effect Aug. 1. And more guardrails are poised to follow as businesses grapple with the growing list of complexities—and threats—that AI presents. 

“AI is not just one risk to manage; it’s a bundle of risks you need to manage, starting with confidential and personal information,” Howie Cockrill, general counsel and evp-group operations at The Opus Group, explained during EM’s “AI for Event Marketers” webinar in August. 

For a breakdown of dos and don’ts, check out five expert tips on how to approach data security responsibly in the GenAI era. 

  1. Disclose AI interactions to users.

Attendees, whether digital or in-person, should always be informed that they’re interacting with AI if any personal data is being collected. Getting consent is paramount to complying with legal regulations and maintaining trust. 

  2. Have a human validate outputs.

Outputs, which are the responses that AI conjures based on the information it’s been fed, shouldn’t go unchecked to ensure quality control. A human should be closely monitoring the way the tech is behaving and have the ability to shut it down or adjust settings if things go awry. 

For instance, when Coca-Cola and agency partner Momentum Worldwide activated the Coke AI Studio festival experience, their application of ChatGPT enabled fans to create their own unique bands, name included. But the team noticed that GPT started supplying obvious knock-offs of real band names.

“Out of every 100 names, we would see some very curious things bubble up,” Megan Cross, executive creative director, US, at Momentum, said in the August webinar. “Names like ‘Illmatic,’ even if you put five Ls in, it is still one of the most iconic rap albums of all time. So we really had to make sure that we were going in and checking the work because AI does not yet, off the shelf, have the capacity for nuance or suggestion in the same way that a human would.”

  3. Don’t share brand assets or IP.

You’ve likely heard it before, but we’ll say it again: Don’t feed proprietary or confidential information to an AI system. Doing so could allow that sensitive data to be accessed by the public, and for agencies, that might also result in breaching a client contract.

“The copyright angle on GenAI is super complicated,” Cockrill says. “Does the AI tool itself have any rights to your GenAI outputs? Can you pass ownership rights along to the client? Could that content infringe on the IP of third parties? These are all tricky questions that are extremely context- and situation-dependent.”

One way to ensure a company’s IP isn’t made public is to leverage tools that allows for brand-specific GenAI models that can’t be accessed by outsiders. 

“Whether we’re using Bedrock, Llama or ChatGPT, we can swap those [AI platforms] out with a brand’s own version of those things,” says Brian Gates, svp-industry strategy at RainFocus. “This is where you get the added security and privacy element—you’re not putting that into a public model. You’re putting it directly into their model; it’s their data. It’s already under the restrictions and policies that that brand has.”

“This is a mindset shift. Step one is realizing that you can use AI in your work, and step two is thinking about how to do it mindfully.”

-Howie Cockrill, General Counsel & EVP-Group Operations,
The Opus Group

 4. Train employees.

AI threats exist from every angle, meaning anyone from the organization who touches AI should be properly trained on risk mitigation. 

“One thing AI [systems] will do is insert themselves and try and communicate as if they’re the person you think they are. So it’s staying awake and realizing that everyone you’re interacting with could actually be AI,” says Mitch Thrower, ceo of Events.com. “Train the people that work for you on how to avoid attacks, because AI is going to make it more important. Make sure you have a password with everyone in your event family. That becomes a way of you communicating who you’re actually talking to.”

  5. Create your own policy.

Creating a proprietary AI policy is about balance. When The Opus Group began developing its own framework, the effort was based on common sense and tied to existing standards on one hand, but needed to account for the broad spectrum of applications and risk on the other.

“[The policy] needed to help us bring risk to a non-zero level that we’re comfortable with. It needed to prevent us from breaching contracts and breaking laws. And truthfully, it needed to pass muster with client legal and procurement for RFPs and audits,” says Cockrill. “This is a mindset shift. Step one is realizing that you can use AI in your work, and step two is thinking about how to do it mindfully.”

^{-Kait Shea}

Ahead of this summer’s D23: The Ultimate Disney Fan Event in August, Disney surprised fans with the announcement that one of its most popular ticketed presentations was going to be livestreamed for the first time… exclusively in Fortnite. During “Horizons: Disney Experiences Showcase,” Josh D’Amaro, chairman of Disney Experiences, was joined on stage by creative leads from across The Walt Disney Company to discuss Disney’s expansive collaboration with Epic Games, Fortnite’s developer, in which Disney has invested $1.5 billion. 

In front of a sold-out crowd of about 12,000 in Anaheim’s Honda Center and 1.2 million Fortnite players who joined the livestream on a specially created island in the game, they shared how Disney storytelling would be integrated into Fortnite and unveiled Disney, Star Wars and Marvel characters and skins. Players who visited the “Disney Horizons Live” island for 10 minutes received a free in-game item and XP, and a replay of the stream on a loop was available for a week after.

Through the livestreamed Epic Games announcement, Disney tapped an audience 100-times the size of the arena attendance and reached gamers and Gen Zers who may not have made it through the doors of the D23 venue, as tickets to the highly coveted fan event can get quite pricy and typically sell out within a day. Just as Disney cracked open the door to livestreaming on a gaming platform, brands are navigating new online spaces and elevating virtual experiences hosted on platforms that have been perfected since the pivots of 2020. 

In conversations with event marketers, we’ve discovered a divide in experiential priorities and preferences. Post-pandemic, some brands went all in on in-person experiences, opting to let go of virtual components, while others, particularly global tech brands, have continued to offer hybrid programs to allow customers and users to join virtually from around the world. These decisions on event formats or platforms don’t just come from executive leadership or budgets; brands are making an effort to collect the right data, listen to their attendee bases and shape event experiences to best fit their needs. 

HYBRID IS HERE TO STAY

A month ahead of SAP’s hybrid SuccessConnect, taking place Oct. 28-30 in Lisbon, Portugal, in-person registrations sold out, but the free virtual experience remained open to interested attendees. The annual customer event brings together h.r. business leaders, SuccessFactors users and prospects to explore the latest SAP human capital management (HCM) technologies through keynotes, sessions and demos. Integral to the event’s recent iterations has been the virtual experience with live and on-demand content across three days.

Melissa Vilders, head of global events strategy and experience, SAP HCM Solutions, SAP Global Marketing, says SuccessConnect Virtual offers many benefits to registrants, such as the opportunity to fit the programming into their own schedule and save money by not having to book travel and accommodations. Live feeds from the show floor, real-time polls and interactive learning components are in place to make sure virtual attendees still participate and feel the energy from the in-person event. 

“From the time we started in 2020 all the way to now in 2024, it’s been constant change and improvement to keep designing a program that fits the expectations of our customers,” Vilders says. “Even the length of the program used to be much longer, but with the data and feedback we collected, we made it shorter, and shorter doesn’t mean that it’s less powerful; it just means that it’s really the best of the best that we are delivering for the virtual attendees.”

REGISTRATION DOESN’T EQUAL ENGAGEMENT

SAP monitors several key metrics to quantify the value its audience places on SuccessConnect Virtual. The No. 1 metric is, of course, the virtual attendance rate, which is compared against in-person attendee numbers, and Vilders specifies that this is not the same as the number of registrations because “it’s easy to register for a free event and just not come, so we really look at the actual attendance.” 

Engagement metrics include the number of virtual sessions attended, chat interaction, polling participation and the number of questions asked in live Q&As. Post-event, SAP reviews the number of views on session playbacks, time spent in a session and dropout rates to understand which sessions were most effective. “Is 50 percent of the audience tuning out after 20 minutes?” Vilders says.

For SAP, surveys are crucial to gathering direct feedback from attendees, and yes, Vilders and her team read every comment to see what worked well and identify enhancements. With this year’s event being held in Portugal, SAP is looking at the geographic locations of virtual attendees to get a sense of how widespread the virtual attendance is and determine if a future in-person event would make sense in a particularly concentrated area of attendees. 

WHAT HAPPENED TO THE METAVERSE?

Somewhere attendees are not is the metaverse. At a time when everyone was trapped inside and on endless Zoom meetings, the metaverse seemed like an appealing space to virtually connect with friends and fellow users. But once pandemic lockdowns were lifted and people found a new appreciation for experiencing in-person togetherness and live events, the metaverse’s appeal and excitement hit a downturn. In October 2022, data aggregator DappRadar found that metaverse platform Decentraland had just 38 active users in 24 hours, while The Sandbox had 522 daily users—despite both having over $1 billion valuations. 

Wired cites the November 2022 release of ChatGPT as the spark that led tech companies, including Meta, to shift their capital and resources toward AI innovation, either minimizing or altogether abandoning the metaverse. Additionally, many chief metaverse officers have taken on different positions or left, such as Michael White, who was appointed to lead Disney’s metaverse efforts in 2022 but left after the brand shut down the division a year later. Disney then announced the launch of an AI task force.

ROBLOX REIGNS SUPREME

In a world where the metaverse is barely hanging on, Roblox is thriving. The gaming platform allows users of all ages and skill levels to play user-created games or produce their own through Roblox Studio. Inside the virtual world that hosts 79.5 million daily users, gamers can get social by making friends and interacting with game developers, and brands are buying into Roblox’s positioning as a marketing tool, popping up their own immersive experiences within the platform. 

In June, IKEA opened a virtual store on Roblox through The Co-Worker game, where players can explore the brand’s home furnishings showroom and visit the famous Swedish Food Market and Bistro. In advance of the launch, IKEA recruited 10 virtual coworkers for paid roles at the IKEA hourly rate for a London employee, £13.15, from more than 178,000 applicants. Alongside the paid players, Roblox users can become a coworker or team leader to develop their career at their own pace in positions inspired by real IKEA jobs, including sales, interior design and f&b.

Roblox opened “The Brands Experience” in September, its own virtual world that presents case studies from popular brands that have previously launched their own Roblox experiences, and features a virtual tour of the platform’s ad offerings and gamified marketing challenges. Interested brands can jump into the experience, developed with creative studio Sawhorse, to move through campaigns from adidas, Universal Pictures, Walmart and Paris Hilton’s 11:11 Media, while learning how to activate on Roblox from a user POV. Truly “powering imagination.”

Top Metrics for Virtual Programming

SAP’s Melissa Vilders lists eight key metrics for measuring the value of a hybrid event’s virtual component.

☑️ Number of virtual attendees
☑️ Number of live sessions attended
☑️ Participation in live chats, polls and Q&As
☑️ Number of views (on-demand session playbacks)
☑️ Time spent viewing a session on-demand
☑️ Satisfaction scores in post-event surveys
☑️ Geographic locations of virtual attendees
☑️ Website traffic pre-, during and post-event

^{-Juanita Chavarro Arias}

At some point over the last decade, nearly everyone has heard that the average human’s attention span is eight seconds—shorter than that of a goldfish. The data was published in a 2015 Microsoft-led study and subsequently picked up by a host of major news outlets. But when the stat was debunked two years later, the damage had already been done. The myth had embedded itself into pop culture and business strategies alike.

There’s no refuting that people are living busier lives, and research shows that we’re exposed to thousands of advertising messages per day. That makes competition for attention high, but some event professionals argue that humans’ capacity to focus hasn’t shrunk as much as it needs to be earned. (You still with us?)

“You have to be able to earn people’s intentional attention,” says Pablo Rosas, executive director-strategy at media company ATTN. “I have a 12-year-old who reads 300-, 400-, 500-page books—it isn’t a generational thing. We all binge-watch things on Netflix. Clearly, we can pay attention to things for more than eight seconds. So first, you have to stop trying to solve the wrong problem. The problem isn’t that people can’t pay attention for a long time... The real challenge becomes, how do you earn it?”

On the other hand, Dean Armintrout, senior director-global event marketing at TransUnion, believes that humans’ addiction to the quick dopamine hits that social media provides has translated into the way that attendees want to experience a business event, particularly when it comes to how they consume content.

“If you’re just listening to a speaker for half an hour, you’re toast,” says Armintrout. “Every 10 or 15 minutes, you need something to visually shift, whether it’s shifting to video or shifting to another speaker on the other side of the stage. Your mind constantly needs that, and I guarantee you people 100 years ago didn’t need that. We created our own monster in that sense.”

Indeed, attention is a hot industry topic. In fact, beginning in September, the International Advertising Bureau’s Attention Task Force started drafting attention measurement guidelines in collaboration with the Media Rating Council. The pair plan to release one of the guides in Q1 2025. 

In conversations with authorities in the field, they, too, pointed to an inflection point in the attention economy. So for a better grasp on the landscape, we tapped the experts for best practices on earning attention at b-to-b events, insights on the modern metrics that matter, new tools worth trying and strategies in action. 

THE NEUROSCIENCE

The science of attention is just that—a science. Neuroscience, the study of the nervous system and brain, is foundational to understanding attention spans. And the more marketers know about how the human brain works, the better equipped they are to earn and retain their attendees’ concentration. 

Experts have widely agreed that there are four types of attention, but the way they specifically define those categories is a bit varied. One event-based interpretation comes from ATTN, which formulated its four-part Attention Framework after hiring a research organization to do a comprehensive study on attention and conducting in-depth interviews with a range of neuroscientists. The resulting framework defines the four types as instinctual, emotional, cognitive and planned. 

The Lowdown:

• Instinctual attention is triggered by visceral elements, like loud noises or flashing lights.
• Emotional attention is about making someone feel deeply enough that they stay engaged.
• Cognitive attention engages the brain in some light problem-solving that the team at ATTN informally calls the “Wait, what? Effect.”
• Planned attention involves making a decision to engage in an activity. 

And because the Attention Framework is founded on neuroscience, its application is universal, and can be leveraged for both business and consumer events, Rosas says. 

On the b-to-b side, he encourages marketers to ask themselves: “What part of the day or what part of the session or what physical location do I want to use for instinctual? Then, why do I want people to feel something, and what is the thing that I want them to feel? And then, when is the moment I want to provoke them to problem-solve with me and think through something along with what’s happening at the event? And the planned is, OK, when do I want people to make a decision in some way?”

After studying a host of Cannes Lions entries this year, ATTN discovered a McDonald’s consumer program that serves as a prime example of how the framework can be used because it triggered all four attention types. And the strategy was incredibly simple. 

Over the spring, the fast food giant planted first-of-their-kind scented billboards that emitted the aroma of warm McDonald’s french fries in the Dutch cities of Utrecht and Leiden whenever a consumer passed within sniffing distance. Each (unbranded) installation was located roughly 650 feet from a restaurant location, and featured a compartment that held fries, along with a ventilation system that amplified their scent. (TBWA\Neboko and Raúl&Rigel handled.)

The billboards activated instinctual attention with the smell of the fries, emotional attention because people generally love eating them, cognitive attention because they had to ponder why they smelled the food and where it was coming from, and planned attention because many of them then decided to go purchase the fries at a restaurant a few hundred feet away. 

TIPS & FUNDAMENTALS

It goes without saying that there’s no one-size-fits-all solution for earning, retaining and measuring attention. But there are some fundamental best practices event marketers can lean on. So, straight from the b-to-b trenches, we bring you five core insights.

➡ TUNE INTO ATTENDEES' INTERESTS.

If you want to know how to earn an audience’s attention, you have to know who they are and what interests them. Even brands who have hosted annual conferences for years may not know their attendee base as well as they could. 

“I always go back to, who is your attendee?” says Katie Moser, director-marketing at GoGather. “Get a very, very deep-dive understanding of who your attendee is and what is interesting to them. Really understanding who’s coming to your event, what they want to get out of your event, how they like to consume content, are they coming to network, are they coming to learn? There are so many factors, but it’s the place that event managers need to spend a lot of time diving into and understanding.”

➡ OFFER A MIX OF BITE-SIZED CONTENT AND DEEP DIVES.

For Armintrout, shortening breakout sessions is critical to maintaining attendees’ attention because it mimics the way they like to consume content in their personal lives. And for sessions that need to be longer, creating consistent shifts in the way the content is delivered is essential. 

“These 15-minute, 20-minute content sessions with another 10 minutes for Q&A are the perfect sweet spot,” he says. “It gives [attendees] time to then ask questions and engage on their own accord. And then if it is a longer general session, you’ve got to break it up with videos or other entertainment or shifting of speakers... It’s the reality of how we are delivering content and how people are consuming it, so we’ve definitely got to keep that in mind at our events.”

On the other side of the token, Moser advises against completely overhauling your conference format. There are certain b-to-b event staples that attendees have come to expect, she says, and one of them is the opportunity to deeply explore a topic.

“There’s a necessity for those longer deep-dive sessions, and I don’t think people don’t want more deep dive content, because that’s why you go to events,” Moser says. “You’re not attending an event just to get information that you could get on your phone. You’re going to learn things that are hands-on or deeper expertise that you may not be able to get just from being online... Then [it’s] having some of those smaller sessions sprinkled throughout the show floor that people can go to and wander around and get a bit of that format like you would on social media.”

➡ KEEP PRODUCT DEMOS TO FIVE MINUTES.

It might sound counterintuitive, but these days, shorter demos tend to make attendees more apt to stay on the floor and engage with a brand a little longer, or opt in for more info. As Armintrout puts it, if it takes you more than around five minutes to describe the basics of your product or service, you’re likely to lose prospects’ interest. It’s about delivering a short, engaging hook to get them interested in your story, and telling that story fast.

“If you have a five-minute demo to show me, I’m all in. Give me the nuts and bolts of how this is going to work, how it’s going to affect my business, why I should want to partner with you,” he says. “If you start showing me a 15- to 20-minute demo, it’s too much. People want those quick interactions... If it’s crisp enough, you’re going to engage me, and I’m going to ask questions. And that’s where you’re going to realize a deeper connection in your conversation.”

➡ BUILD IN 'OFFICE' TIME.

The reality of the modern workforce is that people will check their work email at your events. They’ll take phone calls and meetings. But if time for those activities is built directly into the agenda at strategic points, attendees will be more likely to devote their full attention to the event content.

“If you have three hours of sessions back-to-back, it can be hard to retain anything. And you’re going to start to lose people,” Moser says. “So figure out how to build in good, interesting breaks for people where they can get up, they can move, they can interact, they can spend some time on their phone so they get that side time to answer emails. Just trying to keep in mind what your modern attendee is interested in and needs [in order] to be able to stay focused.”

➡ ELEVATE YOUR A/V GAME

At Quest Software’s annual NXT sales conference this year, an upgraded a/v strategy impacted how well the audience was able to concentrate, according to Alyson Strickler-Vazquez, senior event and marketing specialist at Quest. To modernize the experience and make content feel more engaging and dynamic, the brand installed LED screens in every breakout room and leveraged giant projection-mapped LED cubes on stage to add “dimensionality” to keynote presentations, among tactics.

“The resounding feedback has been that they’re much more attentive now to what we’re saying,” Strickler-Vazquez says. “They’re able to see the screens better. It’s a better presentation. We’re keeping them awake. Obviously, that’s combined with a lot of other things that we do, but I would equate most of that to what we’re doing from a digital aspect and all the a/v that we’re bringing into the room.”

Case Study: Tennessee Department of Tourist Development Offers ‘Sound Advice’

Listening to music is an innately emotional experience, but the Tennessee Department of Tourist Development (TDTD) took things up a notch over the spring along NEEDTOBREATHE’s The CAVE World Tour. Across nine U.S. tour stops, the organization highlighted Tennessee’s musical legacy by combining tunes and tech to transform concertgoers’ brainwaves into tailored travel recommendations. Dubbed “Sound Advice,” the activation used electroencephalogram (EEG) technology to measure attendees’ subconscious affinity toward different Tennessee locales based on their respective sounds.

Fittingly, the activation coincides with TDTD’s recent brand refresh and new tagline, “Tennessee Sounds Perfect.” How it worked: Participants stepped inside a custom-built booth, where they strapped on one of MUSE’s research-grade EEG headbands and listened to authentic sounds and songs from around the state. More than 20 iconic destinations were featured, including Dollywood, Graceland, The Grand Ole Opry and Ruby Falls. Consumers’ brain activity then determined which sounds resonated most, and TDTD provided personalized recommendations for their next Tennessee getaway via an email that listed the top destinations their brainwaves reacted to. (Agencies: VML; FlyteVu)

METRICS MAGIC

There’s no shortage of attention-span measurement methods, platforms and tools out there. There’s also no “right” way to analyze attention, and marketers have plenty of options, from exit surveys to biosensors to lead generation to social listening to session analytics. 

That said, combining metrics has proven to give event professionals a more thorough and actionable understanding of audience attention and engagement levels. There’s a lot to choose from, but the following options should get you going in the right direction.

➡ BIOSENSORS & WEARABLES

Wearable technology and biosensors that measure elements like attendees’ brain activity, heart rate variability, changes in sweat gland activity and movement around the event are all science-backed ways of analyzing attention. And there are a number of approaches you can take, depending on the vendor you select, but bear in mind that it likely won’t be cheap. 

Providers like iMotions, which lever-ages a neuroscience-backed methodology and focuses on the four types of attention, and Emotiva, an AI platform that uses computer vision and machine learning algorithms to analyze emotional responses, are among measurement platforms making headlines.

➡ VIDEO ANALYTICS

Video analytics software can help marketers identify how many audience members are engaged in content at a given time, and in real time, as well as their emotional state. Eye-tracking technology, in particular, is growing in popularity among event profs looking for specific insights into how many people are paying attention to the content, how deeply focused they are, and when their concentration wanes.   

At the 2024 Experiential Marketing Summit in Las Vegas, event measurement partner These Guys used its AI-supported video technology to analyze audience attention and sentiment at various sessions and keynotes. 

Among results, the organization reported that during the last few moments of the opening keynote, attendees were still engaged. Among the stats: 45 percent of the audience expressed excitement, 31 percent displayed focus and 21 percent showed joy, while only 2 percent were confused and 1 percent expressed doubt.

^{-Kait Shea}