Сауалнамаларды алдын ала тестілеу әдісі

Field research method
Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people. Cognitive interviewing generally collects the following information from participants: evaluations on how the subject constructed their answers; explanations on what the subject interprets the questions to mean; reporting of any difficulties the subject had in answering the questions; and anything else that reveals the circumstances to the subject's answers. Cognitive pretesting is considered essential in testing the validity of an interview, test, or questionnaire.

Sample size is a very important topic in pretests. Small samples of 5 15 participants are common. While some researchers suggest that it is best if the sample size is at least 30 people and more is always better, the current best practice is to design the research in rounds to retest changes. For example, when pretesting a questionnaire, it is more useful to conduct 3 rounds of 9 participants than 1 round of 27. There are two different methods of telling participants about the questionnaire: participating pretests and undeclared pretest. When recruiting hard to reach respondents and respondent characteristics via purposive sampling, community based recruitment (word of mouth, endorsement from community leaders) works better than advertisements.

Cognitive interviewing is regularly practiced by U. S. Federal Agencies, including the Census Bureau, National Center for Health Statistics (NCHS), and the Bureau of Labor Statistics. The NCHS maintains a database of U. S. and international agencies that have conducted cognitive interview projects and contributed reports to their depository, such as the National Science Foundation and GESIS – Leibniz Institute for the Social Sciences. Cross cultural cognitive interviewing is practiced to evaluate survey question equivalence and sources of difficulties, as well as to repair problems related to translation. Because of differences in communication styles and cultural norms, adaptations are needed in protocol setup and design, and verbal probing.

In October 2016, the U. S. Office of Management and Budget (OMB) issued Statistical Policy Directive No. 2 Addendum: Standards and Guidelines for Cognitive Interviews that included seven standards for cognitive interviews conducted by or for U. S. Federal studies. Another standard proposed by researchers is the Cognitive Interviewing Reporting Framework (CIRF) that applies a 10 category checklist to make clear what was done during the cognitive interviews and how conclusions were made based on procedures and results of those interviews. In addition, a project management approach is recommended when managing cognitive interviewing studies. For translated surveys, cognitive interviewing techniques, participant selection and recruitment, and project management approach must be adapted to increase their fit for use.