What’s On Your Conjoint Wish List? Q&A with Author Dr. Chris Chapman and Professor Eric Bradlow
In GBK’s recent webinar on conjoint analysis, Dr. Chris Chapman, author of Quantitative User Experience Research: Informing Product Decisions by Understanding Users at Scale, and Professor Eric Bradlow, Vice Dean, Analytics at Wharton and GBK Collective Co-Founder, shared best practices for how to apply conjoint and discrete choice modeling in real world business situations.
Not only is conjoint analysis used to more accurately predict what features, functionality and pricing are important to customers, but also as a tool to optimize decisions across areas – from product portfolio optimization to measuring brand equity.
In the follow up Q&A below, Chris and Eric share additional insight on the power of conjoint analysis to predict customer behavior and the types of business problems conjoint can help solve. Chris also shares his conjoint wish list.
Eric: In our recent webinar, we explored a number of best practices for conjoint analysis. To summarize again for our readers, what is conjoint analysis and what types of decisions or trade off choices can it help to inform?
Chris: At its core conjoint is a compositional approach. Products are made up of attributes and levels. Anytime there are multiple attributes with trade-offs and a choice is required, conjoint can be an invaluable decision making tool. Perhaps the most common use of conjoint by companies is to maximize the value they deliver customers by measuring their interest in a product relative to what it costs to make the product.
So for example, if I were an automotive engineer, I might be wondering, how important is it to have a larger cargo space in the minivan? How do people consider that compared to the seating area? There's a trade off decision to make. If I make the whole product larger, it'll get lower gas mileage and cost more. Or I could use some seating space to create more cargo area.
To make these kinds of design decisions, it’s very helpful for product designers to know which attributes people prefer, how many people prefer, and also how much they're willing to pay for it. That allows the product designer to not only deliver more value to the customer, but also determine what’s going to be most profitable for the company relative to the cost of production.
Price establishes a common framework for this underlying and rather nebulous utility value that goes into the formulas for estimating this. Specifically, product managers are often interested in things like the willingness to pay for a higher level feature. For example, if I add a fancy sound system or a hybrid engine to a car, what will people pay for that? Will they pay enough that we can recover the cost to engineer and produce it? Conjoint also lets us answer questions related to brand equity such as how do we compare to a different brand? If I'm the product manager for Acme, I may wonder about, am I perceived as being worth more or less than Bravo or Charlie?
Eric: You bring up some great points. One of the first areas I teach in my core marketing class at Wharton is how to apply conjoint analysis to determine brand equity. When you have three cars from GM, Toyota and Ford with nearly identical features, how much is GM worth versus the Toyota and Ford. How would you approach using conjoint to measure brand equity over time?
Chris: A typical model for measuring brand equity using conjoint would be to conduct a tracking-based survey on a quarterly basis, or regular interval, where people are repeating a conjoint to assess brand equity. So brand is one of the attributes in a conjoint study. One aspect of this that can become difficult however is when the attributes you’re assessing change.
One of the assumptions of any conjoint study is that any attribute not shown has zero effect, meaning it is the same for all the choices. So if you start assessing a brand relative to a set of attributes that change over time, the brand value itself may change over time. So it's very important to think about what you want to measure longitudinally, and which parts you want to bring in and out of an assessment regime, due to timely product decisions.
“At its core conjoint is a compositional approach. Products are made up of attributes and levels. Anytime there are multiple attributes with trade-offs and a choice is required, conjoint can be an invaluable decision making tool.”
Eric: In addition to the areas we’ve discussed such as optimizing product decisions, determining price sensitivity and brand equity, what are some other common business applications for conjoint?
Another key area is around maximizing customer value. In other words, the goal of conjoint isn’t to maximize the company’s profit, but rather how to maximize profit relative to delivering the highest value to the customer. For example, it may be that a particular feature costs almost nothing for the company to deliver, but is highly valued by the customer.
Another benefit of choice-based conjoint as it's being implemented is what's known as hierarchical Bayesian models. This is where you get an estimate from every survey respondent on the level of value they place on a feature or price sensitivity. And by understanding the respondent level estimates, you can apply segmentation methods to determine how many people value that combination of features and at what price. From there, you can design a model to estimate what to expect from each of your target segments. Or apply clustering methods to determine what are the groups of people that go together empirically based on these data (i.e. which groups of people seek similar benefits).
Eric: Another area we touched on in our webinar is how choice-based conjoint is changing the way companies conduct segmentation. And by the way, GBK runs lots of segmentation studies, but not necessarily every one of them based on conjoint. Can you share your perspective on what areas of segmentation conjoint is best suited for?
Chris: Sure, this actually gets into one of my wish list areas which is to improve segmentation inside conjoint. One of the most fundamental misunderstandings of segmentation is that it’s a method that can be applied to a data set to get some answer. And the ugly truth is that with any data set, you can always apply a clustering method to get some answer.
So, the question is how useful is the answer going to be for the business. If the company needs to better understand customer psychographics to inform advertising or how to message a new product or service to them, then it may be better to segment based on psychographic or demographic variables.
But if we're thinking about estimating product demand across a portfolio space, and how much to make of one product versus another, or how we expect to perform versus the competition, then segmentation based on a conjoint analysis might fit that problem.
“The first item on my wishlist for conjoint is modeling which experiential features, product designs or form factors are most appealing to customers.”
Eric: Conjoint has been around for 50 years. Literally, the first paper in marketing was Green and Rao, 1971. What do you see as the emerging areas for conjoint moving forward? Or to ask another way, what's on your wish list for conjoint?
Chris: Sure. When you and I first met, you had published a paper with your wish list for conjoint, which is a great resource for folks. And I'm not going to go through the areas there. Luckily, I think several of them have been addressed now, which is wonderful. And it gets to this integration between the academic and practitioner side. But speaking from a practitioner wish list, there are three things that come up, over and over in my work and dealing with stakeholders and consulting with folks across industry.
The first item on my wishlist for conjoint is modeling which experiential features, product designs or form factors are most appealing to customers. So if it's important to touch and feel the product, how do we bring that into the conjoint analysis. It may surprise folks out there to know that many products – ranging from automotive doors to computer keyboards – have lead weights added to them, not because they do anything functional, but because they make the product feel better. The door closes with a thunk, the keyboard feels solid. So there are a number of features that define our actual physical interaction with products we can better understand. That's number one.
The second area on my wishlist is improving how we think about segmentation with conjoint. So one of the things that's underlying most models using conjoint analysis is that you are sampling from a population and there is a multivariate normal distribution of some sort across that population. But if you believe that there are multiple segments included within the population you're drawing from, then to treat them as a single population in this underlying model is, in fact, incorrect.
And so there approaches to doing this using latent class modeling that have been in the literature. But I think, coming from the clustering and machine learning community, there are more modern approaches you could apply from mixture models and so on. I won't go into the technical details, but I think we have an opportunity to bring those into conjoint analysis so we can do better segmentation at the same time we're estimating people's preferences.
The next item on my conjoint wish list is to develop a design and underlying statistical model that takes distributed decision making into account. This is especially important in the B2B space, where no single person inside an enterprise company makes a decision to buy a product. It’s an entire team that evaluates and decides whether or not to choose a product.
So there are many other kinds of decision paradigms out there other than just I choose. Surprisingly, there hasn’t been much work done in this area. And I think that relates to the difficulty level – from underlying statistical assumptions about stable units and exchangeability to other technical matters. But designing an effective model for distributed decision making is a ripe area of incredible importance in the B2B space.