Choice-Based Conjoint Workshop October, 2010 With information provided by Sawtooth Software

Choice-Based Conjoint Workshop
October, 2010
With information provided by
Sawtooth Software
Outline
•
•
•
•
•
Introduction to Conjoint Analysis
Formulating Experiments
Conjoint Methods -types
Use CBC software
Introduction to CBC analysis
• Introduction to Conjoint Analysis
– Identify Your Goal
– Design your experiment – 5 stages
– Interpreting part-worths and importance
– A brief introduction to market simulations
Different Perspectives, Different Goals
• Public wants all of the most desirable features of
environmental assets at lowest possible cost
• Providers want to maximize welfare by:
1) minimizing costs of providing features
2) providing products/services that offer greater overall value than
other alternatives
Demand or Preference Side of Equation
• Focus first on demand/preference side of the
equation
• After figuring out what consumer wants, next assess
whether it can be built/provided in a cost- effective
manner
Products/Services are Composed of
Features/Attributes
• Olive Oil
– Source, Price, Aroma, Size
• Forest Harvesting Program
Live trees after harvesting, Cost , Dead trees
after harvesting, % of forest set aside from
harvest
• Plastic Bag Management
– Cost/tax, % wildlife impact, durability, waste
Breaking the Problem Down
• If we learn how consumer values the components of
a product, we are in a better position to design those
that improve profitability
• If we learn how the public values the components of
environmental goods and services, we are in a better
position to design those goods and services to
maximize societal welfare
How to Learn What Customers/Public Want?
• One way: Ask Direct Questions about preference
–
–
–
–
What Brand do you prefer?
How much would you pay for it?
What color do you prefer
What size of container would you like?
• Answers often trivial and unenlightening (e.g.
respondents prefer low fees to high fees, medium
size than large etc..)
How to Learn What Is Important?
• One way: Ask Direct Questions about Importances
– How important is it that you get the << >> that you want?
Stated Importances
• Importance Ratings often have low discrimination:
Stated Importances
• Answers often have low discrimination, with most
answers falling in “very important” categories
– If they were not important, we probably wouldn’t have
included them in the research!
• Answers sometimes useful for segmenting market,
but still not very actionable
– We still don’t exactly what product they want
What is Conjoint Analysis?
• Research technique developed in early 70s
• Measures how buyers value components of a
product/service bundle
• Dictionary definition-- “Conjoint: Joined together,
combined.”
• Marketer’s catch-phrase-- “Features CONsidered
JOINTly”
How Does Conjoint Analysis Work?
• We vary the product/service features (independent variables) to build
many (usually 12 or more) product concepts
• We ask respondents to rate/rank or choose among a subset of those
product concepts (dependent variable)
• Based on the respondents’ evaluations of the product concepts, we
figure out how much unique value (utility) each of the features
(attributes) added
• (Regress dependent variable on independent variables; estimated
betas equal to part worth utilities.)
Important Early Articles
•
•
•
•
•
Luce, Duncan and John Tukey (1964), “Simultaneous Conjoint Measurement: A
New Type of Fundamental Measurement,” Journal of Mathematical Psychology, 1,
1-27
Green, Paul and Vithala Rao (1971), “Conjoint Measurement for Quantifying
Judgmental Data,” Journal of Marketing Research, 8 (Aug), 355-363
Johnson, Richard (1974), “Trade-off Analysis of Consumer Values,” Journal of
Marketing Research, 11 (May), 121-127
Green, Paul and V. Srinivasan (1978), “Conjoint Analysis in Marketing: New
Development with Implications for Research and Practice,” Journal of Marketing,
54 (Oct), 3-19
Louviere, Jordan and George Woodworth (1983), “Design and Analysis of
Simulated Consumer Choice or Allocation Experiments,” Journal of Marketing
Research, 20 (Nov), 350-367
Traditional, Less-Effective Questions
• How important is horsepower to you in a
vehicle?
• How important is fuel efficiency to you in a
vehicle?
• Which is more important to you, horsepower
or fuel efficiency?
What’s So Good about Conjoint?
• More realistic questions:
Which product would you prefer . . .
- 210 horsepower
or
- 140 horsepower
- 17 MPG
- 28 MPG
• If choose left, you prefer Power. If you choose right, you
prefer Fuel Economy
• Rather than ask directly whether you prefer Power over Fuel
Economy, we present realistic tradeoff scenarios and infer
preferences from your product choices
What’s So Good about Conjoint? (cont.)
• When respondents are forced to make difficult
tradeoffs, we learn what they truly value
• These values (utility scores) are associated with
specific and actionable attribute levels relevant to
the problem at hand
Building a Model
Building a Model
• Inputs:
– Attributes
– Levels
– Respondents
– Prior Knowledge
– External Data
– Experimental
Design
– Conjoint Method
• Outputs:
– Utility Scores for each
level
– Importance Scores for
each attribute
– Ability to perform
Simulations
Defining Attributes
• Attributes are independent aspects of a product or a
service (Brand, Price, Size, Color etc.)
• How many attributes?
• -Depends on research objectives
– One rule of thumb was that no more than 6 0r 7 attributes
is too much
• May cause respondents to simplify, looking only at 2-3 most
important
• Attributes should be independent, mutually exclusive
– Brand, quality and product life expectancy may all measure
the same thing
• Each attribute has varying degrees, or “levels”
– Cost: $1, $2, $3
Rules for Formulating Attribute
Levels
• Attributes are assumed to be mutually
exclusive
– Attribute: Add-on features
– Level 1= Sun roof
– Level 2= GPS system
– Level 3=DVD player
– If you define levels in this way, you cannot
determine the value of providing 2 or 3 of these
features at the same time (or none of them)
Solutions
• 8 level Attribute:
• Features
–
–
–
–
–
–
–
–
None
Sunroof
GPS system
DVD Player
Sunroof, GPS
Sunroof, DVD
GPS, DVD
Sunroof, GPS, DVD
• 3 Binary Attributes:
– Sunroof:
• None
• Sunroof
– GPS System
• None
• GPS
– DVD Player
• None
• DVD Player
Rules for Formulating Attribute
Levels
• Levels should have concrete/unambiguous
meaning
• “very expensive” vs “ costs $575”
• “weight: 5-7 kilos” vs “ weight 6 kilos”
• -One description leaves meaning up to
individual interpretation, while the other does
not
Rules for Formulating Attribute
Levels
• Don’t include too many levels for any one
attribute
– The usual number is about 3-5 levels per attribute
– Make sure levels from your attributes can
combine freely with one another without resulting
in utterly impossible combinations (very unlikely
combinations OK)
Attribute Examples
Cost
$1
$2
$3
Brand
A
B
C
…or graphics as well can be levels.
Color
Red
Black
Blue
Suggestions for Determining Which
Attributes & Levels to Include
• Talk to all stakeholders
• Focus Groups
• Search of competitors websites, sales materials
Other Inputs into the Model
Conjoint Utilities (Part Worths)
• Numeric values that reflect how desirable different
features are:
Feature
Vanilla
Chocolate
Utility
2.5
1.8
25¢
35¢
50¢
5.3
3.2
1.4
• The higher the utility, the better
Interpreting Conjoint Utilities
• Interval scaled data (no ratio operations!)
• You cannot compare one level from one attribute
with one level from another attribute, since conjoint
utilities are scaled to an arbitrary constant within
each attribute (often zero-centered)
• You CAN compare differences between two levels of
one attribute versus two levels of another attribute
(an addition operation)
Conjoint Importances
• Ratio scaled data
• Measure of how much influence each attribute has on people’s
choices
• Best minus worst level of each attribute, then percentaged:
Vanilla - Chocolate
25¢ - 50¢
(2.5 - 1.8) =
(5.3 - 1.4) =
0.7
3.9
----Totals: 4.6
15.2%
84.8%
-------100.0%
• Importances are directly affected by the range of levels you
choose for each attribute
Market Simulations
• Make alternative program/services scenarios and predict
which program/services respondents would choose
• Accumulate (aggregate) respondent predictions to make
“Shares of Preference” (some refer to them as “market
shares”)
Market Simulation Example
• Predict market shares for 35¢ Vanilla cone vs. 25¢ Chocolate
cone for Respondent #1:
Vanilla (2.5) + 35¢ (3.2)
Chocolate (1.8) + 25¢ (5.3)
= 5.7
= 7.1
• Respondent #1 “chooses” 25¢ Chocolate cone!
• Repeat for rest of respondents. . .
Market Simulation Results
• Predict responses for 500 respondents, and we might see
“shares of preference” like:
35%
65%
Vanilla @ 35¢
Chocolate @ 25¢
• 65% of respondents prefer the 25¢ Chocolate cone
So you want to do a conjoint…..
Step 1: Begin With the End in Mind
• What is the objective of the research?
– How much will public be willing to pay for biological
control feature?
– Will farmers switch and adopt a different varieties?
• The better you define the root problem, the
better your research will be!
Step 2: Plan Your Analysis
• Identify how clients need to use data
• Deliver analysis plan to clients as part of research
proposal
– Makes sure that objectives and deliverables are clear
Step 3: Define Attributes and Levels
• How many attributes? Depends on research
objectives
– More than 6 attributes may cause respondents to
simplify, looking only at 3-4 most important
• Attributes should be independent, mutually
exclusive
– Brand, quality, product life expectancy may all
measure the same thing
Rules for Formulating
Attribute Levels
• Levels are assumed to be mutually exclusive
Attribute: Add-on features
level 1: Manual
level 2: Biological Control
level 3: Chemical
– If you define levels in this way, you cannot determine the
value of providing two or three of these features at the
same time (or none of them)
Rules for Formulating
Attribute Levels
• Don’t include too many levels for any one attribute
– The usual number is about 3 to 5 levels per attribute
• One temptation is to include many levels for price, so we can
estimate people’s preferences for each
– Better approach usually is to interpolate between fewer
more precisely measured levels for “not asked about”
prices
• Cover the range of probable values
Rules for Formulating
Attribute Levels
• Make sure levels from your attributes can combine freely with
one another without resulting in utterly impossible
combinations (very unlikely combinations OK)
Representing Levels
• Text
– “High Performance Sports Car”
• Pictures / Graphics
Medium
• Sample boards
Low
High
– Allows respondents to touch or feel samples for tactile attributes
(towel softness, greeting card paper quality, etc.)
• “Null” Level – Has Stereo vs. __________
Step 3: Choose a conjoint method
Step 4: Identify Research Constraints
• Sample issues
• Sample Population >200
• Length of survey (how long can I keep their
attention)
• Fielding issues
• Budget
• Client sophistication
Example of a Pair of Soda Product Profile
Scenarios
Attributes
Cost
Color
Program A:
$.50
Red
Program B:
$.25
Silver
Size
.33 l
.75 l
Sugar level
Diet/light
Regular
How to tell if you are in over your
head…
• You should be okay if…
– Small number of attributes
– Attributes freely combine with one another
– Large sample, even after adjusting for subgroup
analysis
– Your client can describe the analysis, attributes in
one paragraph or less, and you can then explain it
to a six year old with little difficulty!
Section 2
• Intro to Choice-Based Conjoint (Discrete
Choice Modeling)
Setting Up CBC Interview (Definitions)
Concept 1
Concept 2
Concept 3
How many concepts per task?
How many tasks per survey?
Concept 4
Task
How Many Concepts per Task?
• Generally, 2 to 5 concepts are used
• Attribute text length, graphical representation affect the
decision
How Many Tasks per Survey?
• Respondents are expensive to recruit. It makes sense to
ask respondents multiple choice tasks.
• Respondents take about 7 minutes on average to answer
20 tasks (~20 seconds per task)
• CBC is very flexible in terms of how many tasks to include.
Minimum is just one task! (but you’ll need huge sample
size, and will face limitations in analysis)
• Typical choice is 12 to 18 choice tasks
Test CBC Design
• What is a Design? The sum total of information about the
attribute levels being shown in the CBC tasks across all
respondents. (The independent variable matrix)
• If you use ANY prohibitions, or use few questionnaire
versions, you MUST test your design
• Failure to test the design can invalidate your study
• CBC/Web automatically tests your design when it generates
the design file--pay attention to the test!
The “None” Concept
• Pros:
– Respondents aren’t forced to choose a product concept
that they really don’t like
– Lets you capture information about whether respondents
(or segments) are more or less interested in buying the
product concept
• Cons:
– Choices of “None” provide much less information for
estimating utilities than other choices (reduces the
effectiveness of your sample size)
– “None” utilities and Shares of Preference for “None” are
difficult to interpret
Design Stage of CCE
Exercise
• Decide on topic
• Discuss how you are going to decide the
attributes and the levels
• Begin to think about attitudes
• Assign team members tasks for the above
• Be efficient
`