Evidence: Recently in “Tobacco Regulatory Science,” Vogel et al., reported findings of an experimental study designed to assess the “effects of sponsorship disclosures on perceptions of e-cigarette Instagram influencer posts” among young adults. After recruiting young adult e-cigarette users, the authors assigned participants to 1 of 3 experimental conditions that varied the clarity of sponsorship disclosure (clear, ambiguous, and no disclosure) on simulated Instagram influencer posts. The dependent variables (brand attitudes, brand use intentions, and vaping intentions) were all measured after participants viewed each Instagram posts designed for their specific experimental condition. The authors reported “sponsorship disclosures did not significantly affect brand attitudes, brand use intentions, or vaping intentions.” While the analysis that produced these results were free from major flaws, the study design, specifically its inclusion criteria, possessed severe limitations.
Researchers should choose observations based on the independent variable(s) and not the dependent variable(s) of their study. In other words, a study’s inclusion criteria should not prevent variation of the dependent variable. By requiring participants to have used e-cigarettes, Vogel et al., limited the variation of their dependent variable(s). They admitted that “these criteria represent the subset of young adults who are most likely to see e-cigarette influencer posts on Instagram, as the platform tailors its suggested content to users’ interests and behavior.” It has been shown that study selection rules that are correlated with the dependent variable attenuate estimates of relationships on average. This means that statistical relationships will be closer to zero than they really are. If Vogel et al., had selected participants based on age alone, their study design would not have introduced any bias between the independent and the dependent variables. The inclusion criteria of e-cigarette use truncated the range of values the dependent variable could take, biasing the study and casting doubts on its conclusions.