Start with the question. Then choose the method.

Research requests often arrive as tools

Run a survey. Organize a focus group. Conduct a sensory test. Add biometrics. Do a home-use study. Review the literature.

Any of those methods may be appropriate. None of them tells us what uncertainty must be reduced or what someone will do differently when the result arrives.

The first useful sentence is usually a decision statement: “We need to decide ___ by ___ because ___.” Then ask the uncomfortable follow-up: what would change if the result were positive, null, mixed or inconclusive?

If every plausible result leads to the same action, the study may be collecting reassurance rather than decision evidence.

Define what you are studying before choosing the measure

“Experience,” “acceptance,” “engagement,” “satisfaction,” “preference” and “performance” are not measures. They are broad words that must be translated into something that can be observed or measured. Is the organization trying to understand detection, intensity, description, liking, forced preference, choice, repeated use, switching, trust, adherence or adoption?

A method can be rigorous and still be wrong for the specific concept being studied. A forced-choice task answers a different question from a liking scale. A one-time rating answers a different question from continued-use behaviour. A qualitative interview can illuminate mechanisms and language but cannot estimate population prevalence.

Define the comparison and the boundary

A result only becomes interpretable against a baseline, alternative, comparison condition, or what would likely have happened otherwise. The population, context and time horizon also matter. Evidence belongs to the people, products, conditions and duration that produced it. Generalization should be planned rather than assumed.

Finally, define the action threshold. Statistical significance can be relevant, but it is not automatically the threshold for a product, claim, regulatory, clinical or commercial decision. Equivalence bounds, minimum important differences, feasibility constraints and decision costs may matter more (Lakens et al., 2018; Wasserstein et al., 2019).

Choose the smallest credible evidence path

Once the decision, uncertainty, what must be measured, comparison, population, context, time, threshold and boundary are explicit, the method usually becomes much easier to choose. Sample size, for example, should be justified by the type of conclusion and decision required rather than by convention alone (Lakens, 2022).

The goal is not the largest study or the most fashionable technology. It is the smallest credible evidence path capable of moving the decision forward.

Start with the question. The method follows.