Assessing the Quality of Data from Stated-Preference Surveys

See the research:

Performance of a comprehension question in discrete-choice experiment surveys (DCE)

Mansfield C, Poulos C, Boeri M, Hauber B


Developing methods to assess the quality of data from stated-preference surveys like discrete-choice experiments is an active research area. Respondents need to understand the survey questions if they are going to provide reliable data. It is considered best practice to include some comprehension questions in stated-preference surveys. These questions teach respondents how to interpret text and graphics and may help identify respondents who misunderstand the survey and might not be providing reliable responses to the choice questions.

We reviewed 46 stated-preference surveys, mostly discrete-choice experiments, and documented the percentage of respondents who failed the risk grid comprehension question by recruiting source and respondent type. Across these studies, the highest failure rates were in studies where patients were the respondents and in studies recruited through online market research panels, but there was a wide range of results across the studies. Based on these results, we recommend a variety of approaches for analyzing the responses to the comprehension questions and modeling respondents who answer incorrectly. 

Read more about this research by following this link.