Term
|
Definition
-a set of questions designed to generate the data necessary for accomplishing the objectives of the research project
-STANDARDIZES wording/sequencing of questions
-imposes uniformity on data gathering process |
|
|
Term
| Primary role of questionnaire |
|
Definition
-provide the necessary decision making information
-is the questionnaire fails to provide insights, then discard or revise
-questionnaire must be approved by management or client |
|
|
Term
| Criteria for good questionnaire |
|
Definition
-Does it consider the respondent?
-Will the topic interest the respondent?
-Does the language fit the type of respondent?
-Environment
-Questionnaire length |
|
|
Term
|
Definition
| -going through each questionnaire to make certain that skip patterns were followed and required questions were answered |
|
|
Term
|
Definition
| -the sequence in which questions are asked, based on the respondent's answer |
|
|
Term
|
Definition
| -the process of grouping and assigning numeric codes to the various responses to a question |
|
|
Term
| Step #1 (questionnaire development process) |
|
Definition
-determine survey objectives, resources and constraints
-outlines of the decision making information sought through the questionnaire |
|
|
Term
| Step #2 (questionnaire development process) |
|
Definition
-determine the data collection method
-Internet, telephone, mail, self-administration, face-to-face |
|
|
Term
| Step #3 (questionnaire development process) |
|
Definition
-open ended questions (respondent replies in their own words)
-closed ended questions (respondent must choose from a list of answers)
-hybrid (the respondent can choose between a list of answers or write in answer if it isn't listed |
|
|
Term
| Advantages to open-ended questions |
|
Definition
-potential for in depth responses
-can provide good quotes in consumers' own words
-can be used to provide insights into close ended terms
-response not limited to choices provided by research
-final "additional comments" question may pick up ideas, feelings, etc. that might otherwise by missed |
|
|
Term
| Disadvantages to open-ended questions |
|
Definition
-coding can be time consuming
-content analysis
-respondents often are not willing to craft longer, well thought out answers
-may bias towards more articulate respondent |
|
|
Term
|
Definition
-close ended question based off of open ended responses
-provide a list of likely answers |
|
|
Term
|
Definition
-elaborate depending upon respondents answer
-only possible in telephone and personal interviews
-in mail surveys, how much space to allocate for answers? |
|
|
Term
|
Definition
-fixed response/structured
-respondent selects from list of responses |
|
|
Term
| Advantages to closed-ended questions |
|
Definition
-list of responses may jog memory
-easier coding
-unlike open ended, no bias in favor of the articulate |
|
|
Term
| Dichotomous closed-ended questions |
|
Definition
-ask the respondent to choose between two answers
-prone to large amount of measurement error due to only two choices
-wording of question is critical
-responses do not allow for intensity of feeling |
|
|
Term
| Multiple choice closed-ended questions |
|
Definition
-make it clear if more than one choice is possible ("check all that apply")
-choices must be exhaustive and mutually exclusive
-order of answers
-position bias (tendency to pick first or last)
-layout (horizontal, vertical) |
|
|
Term
| Disadvantages to multiple choice questions |
|
Definition
-must develop list of choices
-too long a list can confuse respondent or cause a loss of interest |
|
|
Term
|
Definition
-provide list of most likely answers
-end list with "other" choice |
|
|
Term
|
Definition
-entire group of people about whom information is needed
-aka: population of interest |
|
|
Term
|
Definition
| -collection of data obtained from or about EVERY MEMBER of the population of interest |
|
|
Term
|
Definition
-subset of all the members of a population of interest
-portion that represents everybody |
|
|
Term
|
Definition
-list of population elements from which units to be sampled can be selected or a specified procedure for generating such a list
-a list of all elements in the population |
|
|
Term
|
Definition
-samples in which every element of the population has a known, nonzero likelihood of selection
-everyone has the same chance |
|
|
Term
|
Definition
-samples in which specific elements from the population have been selected in a nonrandom manner
-don't have the same chance of being picked |
|
|
Term
| Probability sample advantages |
|
Definition
-sampling error can be computed
-survey results can be projected to the total population
-more expensive than non-probability sample |
|
|
Term
| Non-probability sample advantages |
|
Definition
-gathered much more quickly and efficiently
-reasonably representative if collected in a careful, thorough manner |
|
|
Term
|
Definition
-can be calculated because we know the size of the population and the sample size
-sample size = the identified and selected population subset for the survey, chosen because it represents the entire group |
|
|
Term
|
Definition
| -a value that accurately portrays or typifies a factor of a complete population, such as average age or income |
|
|
Term
|
Definition
| -the chance that the sample selected does not accurately represent the entire population |
|
|
Term
|
Definition
| -purest form of probability sampling known and equal probability is computed |
|
|
Term
|
Definition
| = sample size/population size |
|
|
Term
| Drawing simple random sample |
|
Definition
-requires a COMPLETE listing of the population (difficult if not impossible)
-an organization that has a list of customers or members may draw the random sample using its computer software |
|
|
Term
|
Definition
-uses a fixed skip interval to draw elements from a numbered population
-obtain listing of the population and number it
-determine a skip interval
-select names based on skip interval, using a random starting point |
|
|
Term
|
Definition
| = population size/sample size |
|
|
Term
| Advantages of systematic sampling |
|
Definition
-more simple
-less time consuming
-less expensive |
|
|
Term
|
Definition
-probability sample that is forced to be more representative through simple random sampling of mutually exclusive and exhaustive subsets
-original pop. divided into two+ mutually exclusive and exhaustive subsets of strata
-identify salient demographic
-select SRS from each new subset |
|
|
Term
| Stratified samples more ______ ______ than SRS |
|
Definition
-stratified samples more statistically efficient than SRS
-a source of variation is eliminated
-forces sample to be more representative of the sub-groups in the population |
|
|
Term
| Why isn't stratified sampling used more often? |
|
Definition
-the necessary information may not be available
-researcher may not be able to identify key basis on which to stratify
-cost and time |
|
|
Term
|
Definition
-U.S. Census uses a cluster sampling approach (multi-stage area probability sample)
-choose counties within a state, select residential blocks within counties, select households within those blocks
-rarely used in marketing, advertising or PR research |
|
|
Term
| Difference between stratified and cluster sampling |
|
Definition
| -cluster sampling requires that all people within the exclusive sub group are sampled |
|
|
Term
|
Definition
-efficient, effective and less expensive method when a representative sample is not needed
-use of convenience samples growing faster than probability sampling
-more databases available with consumers (including low incidence groups) |
|
|
Term
| Convenience samples (cont.) |
|
Definition
-may be referred to as purposive or judgment sampling
-selection criteria based on researcher's judgment
-convenience samples are, statistically, non-representative
-best used for asking cursory questions ("Please rate the smell of this shampoo.") |
|
|
Term
|
Definition
| -focus of quota sample is on meeting targets for different demographic groups (same proportions as in the population) |
|
|
Term
| Key difference of quota samples from stratified samples |
|
Definition
-respondents not selected randomly
-basis for selecting the sample is the researcher's judgment |
|
|
Term
|
Definition
-Internet surveys...
-additional respondents obtained through referral from initial respondents
-used when incidence rate is low
-inexpensive
-check for similar demographics
-ex. Email to friends and ask them to send it along... |
|
|
Term
| Concerns with Internet samples |
|
Definition
-users of the Internet are not representative of the general population
-no comprehensive and reliable source of email addresses exists
-companies that compile e-mail databases usually contain less than 3% of the U.S. population |
|
|
Term
|
Definition
-the larger the sample, the smaller the sampling error
-larger samples cost more money
-sample size may be based on desired sampling error, similar studies or past experience |
|
|
Term
| Ideal number of subgroups |
|
Definition
| -the sample should contain at least 100 respondents in each major subgroup and 20-50 respondents in each minor subgroup |
|
|
Term
|
Definition
-How accurate do estimates have to be?
-How confident must decision makers be that the true population value lies within the confidence intervals?
-Sample size is often determined by the available budget
-Alternative data collection approaches may have to be explored that fit the budget |
|
|
Term
| Importance of normal distribution |
|
Definition
-Many variables have probability distributions that are close to the normal distribution
-used for theoretical reasons
-Central Limit Theorem |
|
|
Term
|
Definition
-regression towards the mean
-for any population the distribution of sample means (or proportions) will approach a normal distribution as sample size increases |
|
|
Term
| Characteristics of normal distribution |
|
Definition
-bell shaped with one mode = median = mean
-symmetric around the mean
-defined by its mean |
|
|
Term
|
Definition
-dispersion of the population/sample
-1 SD = 99%
-2 SD = 95%
-3 SD = 68% |
|
|
Term
|
Definition
-Proportional property of the normal distribution
-Feature that the number of observations falling between the mean and a given number of standard deviations from the mean is the same for all distributions |
|
|
Term
| Standard normal distribution |
|
Definition
-transformation of normal distribution
-mean = 0 and standard deviation = 1
-this distribution follows a known Z scale |
|
|
Term
| Sampling distribution of the mean |
|
Definition
-standard error of the mean - standard deviation of a distribution of sample means
-i.e. How close the sample mean is to the population mean
-given the same population standard deviation, why does a larger sample yield smaller sampling error? |
|
|
Term
| Inferences based on one sample |
|
Definition
-68% probability estimate will fall within +/- 1 standard error
-34% + 34% = 68% (68% of sample means fall within this range)
-What percentage of means falls within +/- 2 standard errors? |
|
|
Term
| Determining sample size for a SRS |
|
Definition
-an estimate of the population standard deviation
-measure of "spread" of values
-acceptable level of sampling error
-norm among researchers tends to be a maximum error range of +/- 5%
-desired level of confidence that the sample will fall within a certain range of the true population values (norm is at 95% - 2SE) |
|
|
Term
| Validation - data analysis procedure |
|
Definition
-were interviews conducted as specified
-the process of making sure that you measured what you wanted to measure
-recontact a small percentage of respondents (10-20%) to see if there was interviewer fraud
*Was the person actually interviewed?
*Was the respondent actually qualified?
*Was the interview conducted in the required manner?
*Did the interviewer cover the entire survey? |
|
|
Term
| Editing - data analysis procedure |
|
Definition
-Process of ascertaining that questionnaires were filled out properly and completely
-Did the interviewer fail to ask or record answers for certain questions?
-Were skip patterns followed?
-Were responses to open-ended questions recorded verbatim; was probing done?
-New technology eliminates most of this step (CATI and web surveys) |
|
|
Term
| Coding - data analysis procedure |
|
Definition
-process of grouping and assigning numeric codes to the various responses to a question
-in phone, personal interview or intercept surveys
-answers may be pre-coded on the questionnaire - interviewer simply circles the answer |
|
|
Term
| Data entry - data analysis procedure |
|
Definition
-process of converting information to an electronic format
-CATI and web based surveys may automatically sort the data in to analysis formats
-paper surveys may be scanned by a data entry device and then the data is transferred to an electronic file for analysis |
|
|
Term
| Logical cleaning of data - data analysis of procedure |
|
Definition
-final computerized error check of data
-error checking routines
-marginal report |
|
|
Term
|
Definition
| -computer programs that accept instructions from the user to check for logical errors in the data |
|
|
Term
|
Definition
| -computer generated table of the frequencies of the responses to each question, used to monitor entry of valid codes and correct use of skip patterns |
|
|
Term
| One way frequency tabulation and statistical analysis |
|
Definition
-the most basic tabulation
-a table showing the numbering of respondents choosing each answer to a survey question |
|
|
Term
| Cross tabulation tabulation and statistical analysis |
|
Definition
-a common way of setting up cross tabulation tables is to use columns to represent factors such as demographics and lifestyle characteristics
-are expected to be predictors of the state of mind, behavior, or intentions data shown as rows of the table |
|
|
Term
| Independent variable is the... |
|
Definition
|
|
Term
| Dependent variable is the... |
|
Definition
|
|
Term
| Cross tabulation and statistical analysis |
|
Definition
| -examination of the responses to one question relative to the responses to one or more other questions |
|
|
Term
| Three different percentages that can be calculated in a cross tabulation table |
|
Definition
1. column percentage - based on the column total
2. row percentage - based on the row total
3. total percentages - based on the table total |
|
|
Term
| Measure of central tendency and dispersion |
|
Definition
-non metric scales
-metric scales |
|
|
Term
|
Definition
-most statistical analysis CANNOT be used
-nominal
-ordinal |
|
|
Term
|
Definition
-most statistical analysis CAN be used
-interval
-ratio |
|
|
Term
|
Definition
| -sum of the values for all observations of a variable divided by the number of observations |
|
|
Term
|
Definition
-value below which 50% of the observations fall
-NOIR |
|
|
Term
|
Definition
-value that occurs most frequently
-NOIR |
|
|
Term
|
Definition
-measure how "spread out" the data are
-they include the standard deviation, variance and range
-Variance - how far variables are from the mean
-Range - of possible given answers |
|
|
Term
|
Definition
-most efficient means of summarizing characteristics of large sets of data
-measures of central tendency
-measures of dispersion |
|
|
Term
|
Definition
-the number of observations in a statistical problem that are not restricted or are free to vary
- (n-1)
-number of people we can compare to |
|
|
Term
| Question wording - clarity |
|
Definition
-clear instructions
-appropriate reading level
-avoid abbreviations/jargon
-use the language of the target group
-avoid double barreled questions |
|
|
Term
| Double-barreled questions |
|
Definition
| Are you satisfied with the COST and AVAILABILITY of tickets? |
|
|
Term
|
Definition
-leading questions that give away the research goal or sponsor identity
-"In the past 30 days, how often did you jeopardize your family's health by undercooking beef?" |
|
|
Term
|
Definition
-consider the respondent's willingness to answer the question
-if possible, avoid embarrassing sensitive questions
-Counter-biasing statement |
|
|
Term
| Counter-biasing statement |
|
Definition
-preface an embarrassing question with a statement indicating, for example, how many people engage in the behavior
-ask question about a third person: "How often do your friends drink enough alcoholic beverages to get drunk?" |
|
|
Term
|
Definition
-Introduction
-Screeners
-Warm up, easy question(s)
-Main body of questionnaire
-Demographic questions |
|
|
Term
| Questionnaire flow - funnel approach |
|
Definition
-general warm-up
-specific in-depth
-general wind-down |
|
|
Term
| Questionnaire introduction |
|
Definition
| -identify purpose, instructions, etc. |
|
|
Term
|
Definition
-identify qualified respondents (to establish quotes)
-"Are you 18 years of age or older?"
-"Have you lived in Florida for at least six months?"
-Provides a basis for estimating survey cost and incidence rate |
|
|
Term
|
Definition
| -How much does it cost per valid survey |
|
|
Term
|
Definition
-simple, interesting, non-threatening
-pertain to all respondents |
|
|
Term
|
Definition
-meat of the survey
-questions that focus on objectives
-general first and then more specific questions
-more difficult/complicated questions
-at the end of this section place particularly sensitive questions |
|
|
Term
|
Definition
| -intrusive, personal questions last |
|
|
Term
|
Definition
| -sequence in which questions will be asked based on answers (skip to Q #...) |
|
|
Term
| Other considerations in questionnaire flow and layout |
|
Definition
-insert "prompters" (encouraging statement) at strategic points
-allow adequate space for open-ended responses in mail and self administered Q's (3 to 5 lines)
-put instructions in capital letters
-consistent font spacing, alignment, check boxes, blanks, etc. |
|
|
Term
|
Definition
-Is a question necessary? Does it serve a purpose?
-"nice to know" phenomenon
-ask client to prioritize - drop question with least priority
-too often client asks for extensive number of demographic Qs |
|
|
Term
| Evaluate questionnaire length |
|
Definition
-Is the questionnaire too long?
-time it out (at least 5) under different scenarios |
|
|
Term
| Split questionnaire design |
|
Definition
-when questionnaire is too long and sample is large
-questionnaire split into "core component" and numerous subcomponents
-respondents given the core plus one or more randomly assigned subcomponents |
|
|
Term
| Evaluate questionnaire accomplishments |
|
Definition
-Will the questions provide the information needed to accomplish the research objectives?
-Next to each question write the research objective number
-drop any questions without an objective
-add questions to fulfill objectives |
|
|
Term
|
Definition
-obtain approval from all relevant parties
-distribute copies to all parties with authority
-obtain managerial (or client) approval |
|
|
Term
|
Definition
-pretest and revise
-administer multiple times in real world situation
-look for confusion, misinterpretation, etc.
-revise - do a second pretest
-re-obtain managerial/client approval |
|
|
Term
|
Definition
-prepare final questionnaire copy
-precise typing, instructions, spacing, numbering and precoding must be set up, monitored and proofread
-for mail, self-administered and Internet questionnaires - make sure it looks professional |
|
|
Term
|
Definition
-supervisor's instructions
-detailed guidelines for field service firm - parameters of what is required/expected
-sampling and sample size, quotas, validation procedures, etc. |
|
|
Term
| Field management services |
|
Definition
-intermediary between research supplier and field service firm
-leaves design and analytical elements to full-service companies and ad agency research staffs
-coordinates data collection but does not do data collection
-tells firm who they should ask |
|
|
Term
|
Definition
-all question formats
-randomize questions and response sets
-piping
-option to require response before respondent can move on
-skipping/branching capability
-ability to include images/graphics - branding the survey
-control the "look" and "feel" |
|
|
Term
|
Definition
-form of probing
-integrate earlier answers in later questions
-"What is your favorite type of ice cream?" Vanilla. "What do you like about vanilla?" |
|
|
Term
|
Definition
-form of probing
-integrate earlier answers in later questions
-"What is your favorite type of ice cream?" Vanilla. "What do you like about vanilla?" |
|
|
Term
|
Definition
-form of probing
-integrate earlier answers in later questions
-"What is your favorite type of ice cream?" Vanilla. "What do you like about vanilla?" |
|
|
Term
| Images and graphics on questionnaires |
|
Definition
| avoid bc it slows the survey down |
|
|
Term
| Major reasons why people terminate telephone interviews |
|
Definition
-subject matter (not interesting or sensitive questions)
-redundant or difficult to understand questions
-lengthy questionnaire
-changing the subject during an interview |
|
|
Term
|
Definition
-common for means or percentages to be mathematically different
-however, it is not common for numbers to be statistically different
-moreover, even if numbers are statistically different, the difference does not automatically imply it is practical to deviate from current expectations and/or actions |
|
|
Term
|
Definition
-when numbers are not exactly the same
-this does not mean the difference between the two numbers is either important or statistically significant |
|
|
Term
|
Definition
| -the difference is large enough for it to be unlikely to have occurred because of chance or sampling error |
|
|
Term
| Managerially important difference |
|
Definition
-the difference between two numbers may be statistically different
-yet so small the difference has little practical significance
-depends on how big your sample is |
|
|
Term
|
Definition
|
|
Term
| Null hypothesis and null hypothesis are... |
|
Definition
|
|
Term
|
Definition
-rejection of the null hypothesis when, in fact, it is true
-WORST |
|
|
Term
|
Definition
| -failure to reject the null hypothesis when, in fact, it is false |
|
|
Term
| Example of Type I vs. Type II error |
|
Definition
| -Worse to say someone liked the movie when they didn't, than not counting someone who does like the movie. |
|
|
Term
|
Definition
-The probability of making a correct decision by not rejecting the null hypothesis when, in fact, it is true.
-percent chance that it will be correct |
|
|
Term
| If the number calculated from the comparison falls at an "end" of the spectrum... |
|
Definition
| then the difference between the numbers is statistically different. |
|
|
Term
|
Definition
| how far each observation from the mean, then find the average difference |
|
|
Term
|
Definition
| square root of the variance |
|
|
Term
|
Definition
-compare the alpha level to the P value
-probability value of an observation occurrence
-P<.05 or p<.05
-5% or 1% chance |
|
|
