Advantages and Disadvantages of a survey

Advantages	Disadvantages
Useful in describing the characteristics of a large population	Surveys can rarely capture the totality and complexity of the context of social life
Make large samples feasible	Response rate may affect generalizability
Flexibility through multiple questions	Inflexible in that the design must stay the same throughout
Standardized questions produce high reliability	Standardization produces low validity
The act of studying any topic may effect it	The act of studying any topic may effect it

Survey research is weak on validity and strong on reliability in general.

Here are the different types of surveys most commonly used:

• Self-administered: respondent fills it out- researcher does not play a role.

Ex. Customer service survey.

• Mailed questionnaires: follow-up mailings and maybe even pre-mailings are important here. Are addresses accurate? Transient population samples.
• Telephone surveys: one way to conduct telephone surveys is through the use of CATI (computer assisted telephone interviewing). Phone bank: instant clarification available to research assistants by supervisor, automatically prepares data for analysis, outsiders can buy questions, safety for researcher.
• On-line surveys: This can be done via e-mail or you can just have a web page set up that you direct people to after an event for them to fill the survey out.
• Face-to-face surveys: one-on-one--- reaction to researcher?
• Secondary Analysis: survey data collected by one researcher is reanalyzed another (usually for a different purpose). Ex. Census data.
• New technologies: CAPI (computer assisted personal interviewing, researcher uses laptop with software and researcher enters in results), CASI (computer assisted self interviewing, researcher brings computer to respondents home and respondent fills it out), CSAQ (computerized self-administered questionnaire, ex. ATMs or e-mail: respondent self selects to participate or not), TDE (touchtone data entry), VR (voice recognition)

Efficiency vs. Effectiveness of some of these approaches:

	Phone survey	Mail survey	Face-to-face survey	E-mail survey
Cost	Mid	High	Mid	Low
Response rate	High	Low	High	Mid
Situation control	Mid	Low	High	Low
Detailed responses	Mid	Mid	High	Mid
speed	High	Low	Low	High

Here are some tips for designing surveys:

• Internal Validity & Reliability:
• Choose appropriate question formats for the information you want: closed or open ended? How vs. What vs. Does vs. Are vs. Will – pay attention to your intentions in how you phrase the start of a question. Direction- “you” vs. “some” – Repeat question format/structure.
• Choose appropriate response formats: Likert Scale (pizza is the best food ever: strongly agree, agree, neutral, disagree, strongly disagree). Drawbacks of neutral? Positives of no opinion & don’t know response options. Are you offering the response you want to measure?- index: yes, no, don’t know. Open ended vs. closed ended response options? Offer similar response patterns to similar questions.- don’t reverse response options from question to question. Placing logic into your response options “If yes, then…” may simplify, but many complicate. Avoid: big matrix response option (MPA survey), check all that apply, and pick three. Think ahead to all the possible combinations of answers you might get from respondents.
• Survey must be appealing and easy to read (clear instructions are helpful) while being useful for the researcher in recording and coding data.
• Ordering and flow of the questions can affect the response: surveys should be funnel like, move from the general to the specific: questions about personal demographic information last. Contingency questions can aide, but also interrupt the flow. ALWAYS PILOT THE QUESTIONNAIRE.
• Respondents/Subjects/Objects/Participants must be competent, knowledgeable about subject and willing to answer.
• Keep it short and to the point: avoid respondent and researcher fatigue--- one way to do this is to make sure questions are relevant to the respondent. Also, limit the number of questions you ask and the amount of response options.
• Avoid vague terms: one question will not mean the same thing to all respondents (define terms clearly)
• Avoid leading questions: push polling during elections (ex. H. Clinton voted to cut your personal prescription coverage, are you going to vote for McCain?)
• Avoid trigger words: emotionally biased, ex. Hate, Love
• Avoid confusing questions:
• No double barreled: two questions in one
• No double negatives: Do you not believe that Obama is not going to be the next President?
• Socially desirable?: survey about child abuse may not be socially desirable and the researcher needs to be prepared to deal with that.
• Investment/Give back is crucial: what is in it for the respondents or community as a whole?

You, the researcher: your presence may reduce the amount of “don’t know” responses & allows you to observe respondents, guard against confusing questions, record responses exactly as given (includes spelling), be familiar with the survey, be safe.

For further information, some great resources are:

Creative Research Systems

http://www.surveysystem.com/sdesign.htm

Survey research method key definitions http://www2.chass.ncsu.edu/garson/PA765/survey.htm

Pew Research Center: Information in the public interest http://pewresearch.org/

Survey Construction http://www.apssa.uiuc.edu/content/conducting_surveys/conducting_surveys.html

University of Michigan Survey Research Center (sample projects) http://www.isr.umich.edu/src/projects.html

Why select the survey method? http://www.socialresearchmethods.net/kb/survey.htm

Cooperative Institutional Research Program (surveys of Higher Education) http://www.gseis.ucla.edu/heri/freshman.html

Conducting Surveys http://www.managementhelp.org/commskls/surveys/surveys.htm

Southern Opinion Research http://www.southernopinion.com/surveyresearch/surveyresearch.html

Georgia State Poll Phone Survey http://aysps.gsu.edu/srp/georgiastatepoll/index.htm

Annotated Survey Research Bibliography http://www.bettycjung.net/Surveys.htm