I have been asked by several people to provide more detail with respect to the points I raised in “A Mix Modeling Manifesto.” While we are working on an academic publication, that could take months (or even years!) to get published. As a result, I have decided to post some brief notes as follow-ups to my previous article. I will address each point in an individual installment in hopes that we can continue the focused discussions we have had so far.
I must first address the fact that we have seen no side-by-side comparisons between the approach we are using and other approaches like path modeling and agent-based modeling. We are currently recruiting an academic team to pursue just that, so please let me know if you have seen any research along these lines.
How are marketing effects being modeled?
As mentioned in the previous document, we no longer live in a marketplace where mix models can only consider the direct impacts of marketing activities on sales. Today’s offline advertisements are generating traffic online, where consumers are exposed to paid search and display ads. This online behavior may be in the form of consumers’ searching for more information about the product through their search engine of choice or talking to their friends about the ad or product through social media outlets. How is your modeler or vendor accounting for the role of search and consumer-generated media in their mix models? Are they considering both direct and indirect impacts of marketing on consumer behavior? Y, that is to say consumer response, can no longer be considered a function of all the X variables at the same time!
We have recently completed a consulting project, which we are permitted to publish academically and to also present as a case study. Our client is a large e-commerce company with a wide range of marketing mix activities, both online and offline. One interesting thing to note about this client is that until 2009, they solely engaged in paid search advertising. As a result, they know that their paid search advertising works, because they have been able to directly attribute effects to it in the absence of all other marketing.
In 2009, the company added television, out-of-home, and online display advertising to their marketing mix. Since they were no longer able to directly attribute consumer behavior to a specific form of marketing, they engaged us to conduct a marketing mix analysis for them. As part of the process, we estimated two different sets of model. The first was a traditional marketing mix model, in which everything was estimated in a single equation with the outcome variable as a function of all the marketing activities (and other exogenous factors) occurring simultaneously. In the other, we estimated a vector autoregressive (VAR) model in which the outcome variable (i.e., company performance), organic search activity (e.g., Google search activity), and the marketing mix variables could all be treated as endogenous variables. External factors were treated as exogenous variables in the VAR approach.
First, let me make a quick observation with respect to the role which organic search plays in today’s consumer decision-making process. With advertisements for everything from doctors and lawyers to soft drinks and snack foods, from automobiles and telephones to airlines and insurance agencies directing people in their offline media to look for them on the web, organic search has arisen as a huge intermediary in the consumer behavior process. While most marketing mix models fail to capture the role which organic search plays in measuring marketing effects, we have repeatedly demonstrated its significance in the decision-making process and thus in measuring marketing mix effectiveness.
In the traditional, single-equation mix model approach, we estimated a model with the outcome variable as a function of television, out-of-home, paid search, and online display advertising. While we were able to estimate very robust models, which explained very high degrees of variability, there is NO face validity to the findings. Television advertising was the only marketing activity to have significant and positive effects on the performance variable, no matter how many decay rates we tested to attempt to get other marketing activities to demonstrate significant effects.
How can this be the case, when the company has thrived up until 2009 with no other marketing activity than paid search? Can you imagine trying to explain these results to their CMO? However, I have heard so many mix modelers (on both the client and supplier sides) talk about their inability to find significant effects for online marketing efforts. They typically blame it on low levels of spending for online media, but the real problem is the way in which they are estimating their models!
In the VAR approach, we were able to demonstrate significant effects for all the marketing activities which the firm had employed. Some of these had direct effects on company performance, while others only had indirect effects by generating organic search activity, which in turn has an impact on the company’s performance. Interestingly enough, television advertising demonstrated both direct and indirect effects. These results have much more face validity, as the model takes into account the various paths from marketing to consumer behavior to business performance.
In closing, our marketing mix models must accurately reflect consumer behavior, and this simply can no longer be accomplished with a “single-equation” view of the marketplace. While a few companies are trying other approaches to address this complexity (e.g., agent-based models, path models), we continue to leverage advances in econometric time series modeling. Econometric models have a long history in sales response modeling and are widely accepted throughout industry and academia. However, we must employ models which can measure marketing effects and consumer behavior in the way they actually occur in the marketplace.
Please feel free to contact me at craig.stacey@mproductivity.com with questions or comments.
How are marketing effects being modeled?
As mentioned in the previous document, we no longer live in a marketplace where mix models can only consider the direct impacts of marketing activities on sales. Today’s offline advertisements are generating traffic online, where consumers are exposed to paid search and display ads. This online behavior may be in the form of consumers’ searching for more information about the product through their search engine of choice or talking to their friends about the ad or product through social media outlets. How is your modeler or vendor accounting for the role of search and consumer-generated media in their mix models? Are they considering both direct and indirect impacts of marketing on consumer behavior? Y, that is to say consumer response, can no longer be considered a function of all the X variables at the same time!
We have recently completed a consulting project, which we are permitted to publish academically and to also present as a case study. Our client is a large e-commerce company with a wide range of marketing mix activities, both online and offline. One interesting thing to note about this client is that until 2009, they solely engaged in paid search advertising. As a result, they know that their paid search advertising works, because they have been able to directly attribute effects to it in the absence of all other marketing.
In 2009, the company added television, out-of-home, and online display advertising to their marketing mix. Since they were no longer able to directly attribute consumer behavior to a specific form of marketing, they engaged us to conduct a marketing mix analysis for them. As part of the process, we estimated two different sets of model. The first was a traditional marketing mix model, in which everything was estimated in a single equation with the outcome variable as a function of all the marketing activities (and other exogenous factors) occurring simultaneously. In the other, we estimated a vector autoregressive (VAR) model in which the outcome variable (i.e., company performance), organic search activity (e.g., Google search activity), and the marketing mix variables could all be treated as endogenous variables. External factors were treated as exogenous variables in the VAR approach.
First, let me make a quick observation with respect to the role which organic search plays in today’s consumer decision-making process. With advertisements for everything from doctors and lawyers to soft drinks and snack foods, from automobiles and telephones to airlines and insurance agencies directing people in their offline media to look for them on the web, organic search has arisen as a huge intermediary in the consumer behavior process. While most marketing mix models fail to capture the role which organic search plays in measuring marketing effects, we have repeatedly demonstrated its significance in the decision-making process and thus in measuring marketing mix effectiveness.
In the traditional, single-equation mix model approach, we estimated a model with the outcome variable as a function of television, out-of-home, paid search, and online display advertising. While we were able to estimate very robust models, which explained very high degrees of variability, there is NO face validity to the findings. Television advertising was the only marketing activity to have significant and positive effects on the performance variable, no matter how many decay rates we tested to attempt to get other marketing activities to demonstrate significant effects.
How can this be the case, when the company has thrived up until 2009 with no other marketing activity than paid search? Can you imagine trying to explain these results to their CMO? However, I have heard so many mix modelers (on both the client and supplier sides) talk about their inability to find significant effects for online marketing efforts. They typically blame it on low levels of spending for online media, but the real problem is the way in which they are estimating their models!
In the VAR approach, we were able to demonstrate significant effects for all the marketing activities which the firm had employed. Some of these had direct effects on company performance, while others only had indirect effects by generating organic search activity, which in turn has an impact on the company’s performance. Interestingly enough, television advertising demonstrated both direct and indirect effects. These results have much more face validity, as the model takes into account the various paths from marketing to consumer behavior to business performance.
In closing, our marketing mix models must accurately reflect consumer behavior, and this simply can no longer be accomplished with a “single-equation” view of the marketplace. While a few companies are trying other approaches to address this complexity (e.g., agent-based models, path models), we continue to leverage advances in econometric time series modeling. Econometric models have a long history in sales response modeling and are widely accepted throughout industry and academia. However, we must employ models which can measure marketing effects and consumer behavior in the way they actually occur in the marketplace.
Please feel free to contact me at craig.stacey@mproductivity.com with questions or comments.
RSS Feed
