# Survey design
## Overview

This learning resource is about how to design surveys (or questionnaires) in the social sciences.
Surveys are commonly used in psychology, health, marketing, sociology, governance, demographics, and many other disciplinary areas.
Survey research is an efficient way of gathering data to help answer a research question. The main challenge is developing reliable and valid measures and sampling representative data.
Survey design is critical in determining the quality of research. The potential for poor design is vast - whether intentionally on the part of the researcher or unintentionally. For example, watch this 2 min. episode of Yes, Minister about politicians trying to get the poll results they want.

## Before designing a survey

It can be very tempting to press ahead with designing a survey. But first, be clear about the purpose of the study and the research methodology.
Before designing a survey, develop a research proposal which clearly explains the:
* research purpose
* research questions
* hypotheses
* Research design: Experimental, quasi-experimental, non-experimental
* Sampling method
* Target constructs - operationally define the:
** independent variables
** dependent variables
Have the research proposal peer reviewed and modify as appropriate. Before designing a survey, it is helpful, and generally recommended, to clearly establish a research proposal and to get this proposal peer-reviewed (and/or reviewed by a supervisor). Investment in developing the proposal is generally returned many-fold.
The seven Ps apply to survey design: Prior preparation and planning prevents piss-poor performance.
Poor research results and conclusions emerge from poor data, which is often due to poor survey design. Hence, a well-conducted survey research project should exhibit:
* clarity in the project's purposes (and specific research questions and hypotheses)
* careful development of well-worded questions with appropriate response formats and/or
* a well designed and implemented sampling method

## Designing a survey
### Initial draft survey

* Create separate sections for each main purpose/research question/hypothesis
* Within each section, brainstorm ways data about topic/question could be obtained and draft items (questions) which you expect can provide a reliable and valid measure of the target constructs; items may also be obtainable from previous surveys. Start off with lots more possible questions/items (based on the operational definitions) than will actually be used; this way, you can cull and refine, using only the best items
* For each consider, brainstorm
* Add an informed consent statement, a coversheet, and an instructions page
* Get the draft survey critically reviewed by others, then redraft etc.
* Get assistance with high quality word-processing skills (if you don't have them) to tweak the essay so that it looks professional
* Pre-test the survey (on convenient others), redraft etc.
* Pilot test the survey (on target population), redraft etc.
* Use the survey in a major study

### Survey structure

* Cover letter
* Informed consent
* Ethics complaints
* Sections containing survey questions
** Personal details / demographics
** One section per major topic
* Debrief information

### Types of surveys

Types of surveys are:
* Hard copy
* Electronic
* Face to face
* Telephone
What are the strengths and weaknesses of the following modes of administration?
| Hard-copy (paper and pencil) | Structured interviews: Face to face |
| Online | Structured interviews: Telephone |

### Types of questions

It is surprisingly difficult to develop a "good" survey question or item. Consider each of the following aspects of survey questions, their pros and cons, and with examples:
* Objective vs. subjective
* Close-ended vs. open-ended
* Leading and loaded questions
* Positive-, negative-, and double-negative-wording

### Types of data

Surveys can be used to collect:
* quantitative and/or
* qualitative
data.

### Response formats

In survey design, it is important to understand the different types of survey response formats and their implications in terms of levels of measurement for quantitative data analysis.
Some commonly used response formats include:
| Measurement scale | Level of measurement | Example |
| Dichotomous | Binary | Yes or No |
| Graphical rating | Continuous | Mark any point on a line |
| Idiographic | Ordinal/Likert | Use symbols/pictures instead of words and numbers |
| Likert scale | Interval | Equally-spaced intervals, usually 3 to 9 intervals |
| Multiple choice or Multichotomous | Categorical/nominal | Yes, No, or Maybe |
| Multiple response | Binary | Tick all that apply |
| Ranking | Ordinal | Compare items to each other by placing them in order of descending preference |
| Semantic differential | Interval | Put two words at opposite ends of a scale with interval marks |
| Verbal frequency | Ordinal | How often ... Never, Sometimes, Often, Always |