Typically, analysis for experiential marketing programs happens in one of two different ways, by overall date or by individual market. Overall is clearly an evaluation of all available data. Market Analysis looks at data unique to specific locations. The object of Market Analysis is to determine if certain products or messages resonate better in certain locales.
Suppose you are promoting a fruity drink. Comparing responses can reveal how well people respond to the drink in warm or cold climates. This comparative analysis is great for rendering project-based insights. It tends to be the focus of much of our research.
The certainty of this approach is supported by the fact that, when looking at market-based data, we tend to have a robust “N size” (number of entries or surveys). So, what should you do when you want to refine the scale to look at a small portion of the entire market? We recently received a request to look at sets of individual events to see if there was any conclusion that could be drawn from them. Because of the small scale, we had to be very careful about several issues.
ISSUE #1: THE N SIZE
The most obvious issue is the small number of surveys from a single event. At PortMA, we set a minimum threshold of 20 N for analysis. That is, any single event must have at least 20 responses. With that minimum, we can be comfortable that our assertions will be true for that event. However, we will not generalize the small sampling for the whole.
ISSUE #2: GENERALIZATION
When considering an individual event, it is important to reinforce that any number of factors might make that event unique. Avoid generalizing results. Clients tend to get excited when they see data that matches their expectations. Make sure they understand that they should not draw conclusions from any single event.
ISSUE #3: SPECIFICITY
Similar, it is important to try to account for what might be unique about an event and how that unique factor may be driving the results. Is the event directed primarily at women? If your product is targeting men, you’ll probably get unexpected results. Was there some outside, mitigate factor that caused a unique subset of attendees? Check it out. Taking into account how susceptible a single event is to outside influences will help you to avoid pitfalls when examining singular events.
Singular event Market Analysis can generate misleading results. While doing so may provide interesting insights, we would recommend erring on the side of caution.