# Survey research and design in psychology/Assessment/Lab report/Feedback/2011

General feedback about the lab report (2011)
* This page provides a summary of commonly-repeated feedback about lab reports.
* This should be read in conjunction with the a priori lab report guidelines and the marking criteria.

## Marking distribution

* N = 71
* Mean = 70, SD = 19.5, Skewness = -1.3, Kurtosis = 2.0
* Median = 75
* Mode = 72
* Min = 0
* Max = 99

## Title

* Did it clearly and unambiguously communicate the main content of the report? (Weaker titles tended to be more vague and to lack specific reference to the main variables analysed.)
* Longer titles generally provide more appropriate detail than shorter titles (try to mention the key variables or questions).
* A few reports didn't provide an APA style title page.

## Abstract

* Should be one paragraph in length.
* Typically too much focus on the Intro/Method and not enough on the Results and Discussion.
* Strength and direction of results were often not indicated.
* Often there was no mention of implications or recommendations.
* Statistical results (i.e., with symbols and numbers) should not generally be reported in the abstract unless they are particularly pertinent (e.g., a notable effect size).
* References should not be reported in the abstract unless they are particularly pertinent (e.g., to draw attention to a key theory which is being tested).

## Introduction

* There is generally one major issue → Did the introduction lead directly to clearly expressed, logically-derived hypotheses?
* For example, weaker introductions might have reviewed some literature but this wasn't necessarily related directly to justifying each of the hypotheses
* Present one hypothesis or research question for each statistical test to be conducted. This wasn't always done.

## Method

* There was sometimes a tendency towards plagiarism of the sample lab report.

### Participants

* Often very basic profiles of the sample were provided, e.g., N, and n and percentage of males and females, with the average age (and SD and range; what about the median?)
* Better sections provided more thoughtful description of the sample e.g,. more description of the cultural context (so that an outside/naive reader can better understand the sample).
* Further description of the sample could have been provided for example by using other demographic information (many reports simply reported on gender and age).

### Measures

* Generally the instrument purpose, development and structure was well explained.
* Sometimes the descriptions were overly based on the sample report.
* Weaker sections tended to lack sufficient description of the proposed factors, e.g.,
** Include a table summarising the proposed factor names, definitions, with example items.
* Put more emphasis on the measurement of variables used in the Results than on other aspects of the survey.
* Make sure to provide a citation and reference to the survey.

### Procedure

* What kind of sampling technique was used? (Hint: It was not random - this was a common mistake - it was convenience sampling, with systematic selection.)
* Make sure to provide a citation and reference to the administration guidelines (otherwise, how can someone replicate the study?).
* How did your administration process go? (response rate, reasons for refusal, anomalies)
* Information about data entry, conventional statistical software (e.g., SPSS), data collation, etc. is not necessary.

## Results

* There's no need to mention what software you used for well-known and commonly available data analysis techniques, such as used in this study.

### Factor analysis

* Generally this was well done.
* Some reports seemed to remove items and factors simultaneously. The recommended approach is to work out how many factors first, then work out which items to retain/drop.
* Borderline plagiarism of the sample lab report was not uncommon.
* Assumptions
** How many cases per variable were there? (Sometimes not reported)
** Despite what some books say, normality is not an assumption; linear relations are an assumption. Normality tends to enhance linearity. As factor analysis is based on correlations between variables, if r does not accurately measure the strength of a relationship between two variables then the analysis is degraded (Francis, 2007).
* Use item descriptions rather than item names (item names are arbitrary and unfamiliar to an outside reader).

### Qualitative analysis

* Approx. 80% of reports adopted a quantitative approach to the qualitative data (i.e., categorising and counting). About 20% used thematic analysis. The thematic analyses were generally very good. The quantitative analyses which used multiple response analysis were generally also very good. The best multiple response analyses also provided some thick description of the main themes.
* Good analyses tended to identify/suggest more themes/categories than weaker analyses. However, categories with very small sample sizes (esp. < 5) are likely to be unreliable and would have better subsumed into broader categories.
* Some QAs did not suggest any additional categories to those identified in the factor analyses of the quantitative items, which generally did not indicate that an indepth analysis of the qualitative data had been undertaken.
* Some content analyses did not use multiple response analysis (e.g., they summarised the multiple responses as separate variables).

### Multiple linear regression

* Often more care should have been taken to prepare the variables for MLR. For example, sometimes the IVs should have been recoded as dichotomous or dummy-coded. Also, GPA required some careful screening and recoding.

## Discussion

* The key task is to demonstrate depth of understanding about the results and the implications for theory, research, and practice.
* Were the directions of relations between constructs clearly explained/understood?
* New findings should not be presented in the results.
* Limitations
* The most common problem with recommendations and conclusions was their lack of specificity (vagueness).