Metodologiczne problemy badań typu self-report
Authors: Andrzej Siemaszko  
Published: July 2024

Self-report studies are gradually becoming the predominating current of empirical research in criminology. This is particularly the case with etiological studies of deviant behaviour in young persons. However, the present popularity of self-report studies is not accompanied by improvement of their methodological aspect. No important development of the methodology of these studies has occured since the pioneer works of Short and Nye. It is the fundamental aim of the present paper to point to these of the methodological questions on which the further development of self-report studies will depend most. In Chapter I, the first works have been discussed in which the self-report methods were applied. The works anaiyzed are those of Murphy at al. Porterfield, as well as Wallerstein and Wy1e. Particular attention has been given to the methodological and substantive aspects of the series of studies carried out by Short and Nye.             Chapter II contains the presentation of findings of the Polish self-report studies.             The first attempt at a self-report study was made in the early 1960s by Malewska and Muszyński. A national random sample of pupils of the sixth grade of primary school n =2,222) was examined by means of an anonymous questionnaire. The basic aim of the study was to define the children's attitude towards the ownership rights and the situations in which violation of these rights is admissible. Besides, the authors were interested in how children perceived given situations to be thefts. Thus the question whether the respondents ever happened to take another person’s property was but a fragment of the questionnaire which served another purpose in its essence. To the question: "How often do you happen to take another person's property?", 0.8 per cent of the children answered ,,very often," 4.2 pet cent - "often," 26.6 per cent - "sometimes," and. 34.5 per cent-,,seldom’’             Like Malewska and Muszyński, also Szemińska and Gołąb aimed at defining the moral sense of young persons: pupils of primary schools (n=61) and inmates of educational institutions (n= 64), asking also about the extent and structure of deviant behaviour. The respondents answered anonymously in writing.             The two compared groups of boys differed from each other considerably as far as both the frequency and the seriousness of thefts commited was concerned. While the majority of "delinquents" admitted a large number of thefts, the "nondelinquents" 'in their vast majority owned up to 1-2 thefts at most, mostly of small objects they stole from their classmates or next of kin with the intention to use these objects themselves." The study of Szemińska and Gołąb raises doubts, both as regards its merits and methodology. Among other things, in spite of the fact that various offences were committed by both of the discussed groups, the authors use a dichotomic pair of notions: delinquent and non-delinquent, failing to put these words in quotation marks which are necessary in this situation.             In the years 1976-1977, Ostrihanska and Wójcik conducted a large self-report study of a random sample of pupils of grades 3-8 of Warsaw primary schools. 50 schools were selected at random, in which the study was carried out by means of a questionnaire in 120 classes, also randomly selected (n=3,177, of which there were 1,631 boys and 1,546 girls). The self-report study was part of a broader research programme aimed at estimating the extent of social maladjustment in the youth and defining its causes.             Among other things, the questions concerned the following phenomena: school failures, truancy, running away from home, drinking alcohol, taking drugs, free riding, destroying another person’s property, other acts against property (including thefts, frauds, robbery-,,taking something from a younger child by constraint"). The possible answers were as follows: ,,never", ,,once'’, ,,2-3 times’’, ,,4-10 times’’, ,,more frequently’’ .             As expected, both the extent and intensity (frequency of perpetration) of deviant behaviour were higher in the group of boys as compared with girls. For instance, as few as 15.3 per cent of the eldest boys (aged 15) stated that they never took another person's property while the percentage of non-stealing girls among the eldest group was over two times higher (38.0 per cent) In this age group 16.3 per cent of boys stole a dozen or more times, while percentage of girls who committed multiple thefts amounted to as few as 1.4 Instead, no greater differences were found between boys and girls who admitted having stolen once.             In boys, the most frequent were thefts from allotments and gardens (35.2 per cent), thefts from parents (22.8 per cent) and thefts in self-service shops 18.1 per cent). On the other hand the most seldom were thefts from cellars (5.8 per cent), and thefts of wine in shops (9.7 per cent).            Taking another person’s property ranked fourth among the types of deviant acts included IN the study The first position was taken by lies (88.8 per cent of answers in the affirmative). Near1y 90 per cent of boys admitted having chribbed anothe child’s exercise, 25.2 per cent- having destroyed property 2.8 per cent ran away from home, and 2.4 per cent tock drugs. There was an upward tendency with age: elder boys admitted a greater number of deviant acts, and actuallv commited these acts more frequently.             The next self-report study was conducted by Ostrowska and Siemaszko in 1979. It included 2,991 pupils of Warsaw secondary schools (1,197 boys and 1,795 girls). Non-random selection was applied. Young persons of the first and last grades were examined by means of anonymous questionnaire. Among other variables, it contained a list of 42 questions about various types of deviant behaviour, acts of infringement of disciplinary regulations, transgressions and offences, from most trivial (like stealing a ride or failing to return change from shopping) to comparatively serious like house-breaking or robbery. All questions had the same set of possible answers: "never," "once of twice," several times," "a dozen or more-times," ,,more frequently." The examined young persons were characterized by rather a high level of deviance. In the group of boys for instance,539 persons (30.9 per cent) admitted having perpetrated a half of the 42 acts included in the questionnaire at least once, 2.8 per cent of them having committed 27 to 31 acts, and 2.3 per cent-32 to 42 acts. Thus together, 6 per cent of the examined boys were highly deviant. Since the study also revealed a close relationship between the number of acts committed and the frequency of their perpetration, the abovementioned 6 per cent of the examined persons (about 300 boys) are "multiple recidivists" in the interpretation used in self-report studies. Considerable differences in the level of deviant behaviour were found in respect of age and sex. For instance,  the level of deviance in the group of elder boys was four times higher on average as compared with younger girls.             Among the most widespread acts there were free riding (94,6 per cent of girls and 96. 1 per cent of boys), failure to return change from shopping (79.6 and 84.1 per cent respectively), petty frauds in shops (67.6 and 84.0 per cent respectively), and failure to return a found object to its owner (69.9 per cent of girls and 83.8 per cent of boys). Aggressive acts were relatively frequent, particularly among boys. Battery "without an explicit causes” was comitted by 20.2 per cent of boys and 6.5 per cent of girls.11.6 per cent of boys and 2.8 per cent of girls participated in affrays in which dangerous weapons were used. Among thefts, comparatively less serious acts predominated. 16.9 per cent of girls and 31.6 per cent of boys admitted having perpetrated petty thefts. 8.2 per cent of girls and 14.6 per cent of boys stole change from call-boxes. Serious thefts were committed by 1.6 per cent of girls and 4.6 per cent of boys.             Ostrowska and Siemaszko repeated their study in 1981 on a random sample of students of secondary schools in five typically agricultural provinces. 2,144 persons (1,702 boys and 420 girls) aged 14-19, students of 29 schools, were examined. They young persons who participated in the study went to :89 classes that were selected at random. The extent and structure of deviant behaviour were examined by means of a questionnaire identical to the one applied in the previous study Also the way in which the study was carried out in the classes was the same.             In the group of acts termed insubordination, the most widespread one was smoking at under: 14:78,2 per cent of boys and 44,8 per cent of girls admitted it. Somewhat less than 10per cent of the examined persons admitted having run away from home, 2 per cent of them having run away several times. Over 20 per cent of the respondents admitted having had their identity papers checked by the police (30 per cent of boys) and somewhat less than 7 per cent took drugs.             Among various types of dishonest behaviour the most widespread one was free riding- over 80 per cent. Nearly a half of the examined persons admitted having failed to return a borrowed object; 7 per cent of them did it repeatedly Also nearly 50 per cent of the respondents stole money from their parents:15 per cent of them did it several times, and 5.6 per cent-more frequently.             In the group of offences, thefts predominated. 24.9 per cent of girls and 32.4 per cent of boys admitted having stolen an object or money to the value of under 100 złotys (the percentage amounting to 38.6 in the  group of eldest boys); nearly 20 per cent of them repeatedly stole money from their parents.             About 25 per cent of the examined persons committed shop- lifting, the percentage of shop-lifters in the group of eldest boys exceeding 40. The acts of breaking into cellars, recesses, attics etc., were committed by 15 per cent of the respondents 6.1 per cent of girls and 17.2 per cent of boys. The most seldom offences against property were: robbery (2.4 per cent of girls, 10.1 per cent of boys), stealing from call-boxes (6.6 per cent of girls, 7.8 per cent of boys), thefts of money to the amount of 500-1000 złotys (6 .2 per cent of girls, 7.0  per cent of boys), failure to pay the bill in a restaurant (3.3 per cent of girls, 5.6 per cent of boys) and thefts of over 1 000 zlotys (2.8 per cent of girls and 5.6 per cent of boys). Among aggressive behavior, brawls and beatings prevailed (25 per cent of girls and 50 per cent of' boys). In Chapter III the most important methodological problems related to self-report studies are discussed. In self-report studies, both direct (e. g. ,,have you stolen), and indirect and euphemistic questions (e, g. ,,have you ever happened to take and not to give back. ") can be found. The indirect questions undoubtedly less  threatening. Yet on the other hand, those asked directly are probably  easier to interpret explicitly. There is no proof as to the superiority of any of these ways of asking. However indirect and euphemistic questions prevail in self-report studies.             The degree of abstractness of questions varies. The good point of clearcut questions (e.g. ''have you ever taken and failed to give back some article in a supermarket") is that the highly detailed formulation may help the respondent to recall an event which the researcher is interested in. On the other hand, their weak point is that the respondent cannot be relied upon to admit having acted in another, very similar yet not identical way. Unfortunately, the majority of self-report questionnaires contain questions about inseparate classes of phenomena. Hence the danger of one and the same act being counted several times.             In self-report studies, the number of questions about deviant behavior is an important problem. One should bean it in mind that the deviant acts taken into account by the researcher are always nothing but a certain sample of the totality of such acts, the parameters of which are usually unknown (e.g. Christie et al.). The greater the number of acts taken into account, the more standard the "sample of acts" seems to be with respect to the "totality of acts." There are great differences as regards the number of acts included: from several (e.g.  Hirschi, Dentler and Monroe) up to several dozen (e.g. Gibson).             Today time limits are usually introducted as regards the period between the act and the moment of examination one year as a rule), though Short and Nye introducted no limits as regards the period during which the respondents committed the admitted acts. Shorter periods can also be found. (e.g. Simone et al, - 2 months, Lipton and Smith - 18 months). The limits are among the most important problems in self-report studies, since it is on them that the estimation depends on the level of deviance of the entire examined group, as well as the precise estimation of the separate respondents levels of  deviance. The views on the optimum time limits are not uniform. Different sets of possible answers to the questions about deviant behaviour can be found: from most precise (e.g. "once," "twice," etc.) to most general and ambigous (e.g. "seldom," ,,frequently"). A strictly enumerative set of answers may be methodologically correct only in the case of a short period (one year or less). In the remaining cases, this set may be misleading as one hardly expects the examined persons to remember past events with such accuracy.             The questions about deviant behaviour may constitute a separate block (nay a separate questionnaire), or they may be put among other questions. There are no studies showing the good and weak points of each of these two solutions. It seems more proper however, to "mask" the aim of the study by interlarding the questions about deviant behaviour with those neutral or concerning "acts of kindness."             When the level or "depth" of the examined person's deviant involvement is defined, an important problem emerges: acts with different "charges of deviance are taken into account here. Therefore, one can either try and attach different weights to them, or treat all of them as equally serious. Christie et al. ranked acts according to the judges opinion. Morash weighted them with the use of Selling and Wolfgang's scale of seriousness of offences. In Hindelang's study, the weight of acts was defined by specialists by means of a fivepoint scale. Hepburn weighted deviant acts basing on appraisals done by the examined persons themselves. However in the vast majority of self-report studies, no weigh ting procedure is applled. As shown by Farrington, weighting procedures fail to contribute substantially to the increase in accuracy of measurement.             An anonymous questionnaire, though most frequentlv applied, is not the only method of gathering information about unrecorded deviant behaviour. E, g. Gold (and other researchers who applied Gold’s scale) employed a questionnaire interview In Belson's study a card sorting procedure was applied. This method of gathering information is particularly popular in England (see also Gibson, Farrington, West, Morash, Shapland). Hirschi examined his respondents with a signed questionnaire. Should the differences in veracity of answeers of a signed and anonymous questionnaire prove to be inessential (and there is much to be said for it, e.g. Krohn, Waldo and Chiracos), it would be advisable to use the signed version (because of the possibility of comparing the separate sociometric choices or comparing the findings with external sources of information).             The main objection raised to self-report studies concerns the doubtful veracity of the data gathered this way (Dentler, Liska).             A relatively small number of studies concerned the reliability of self-report studies, e.g. the stability of findings in time. This is the most difficult problem in the case of a strictly anonymous questionnaire as the separate respondents cannot be retest. Only global distributions are compared then (e.g the scores of respondents in a given class) Siemaszko finds no valid differences between the distributions answers about deviant acts between a test and a retest which took place there months later. Dentler and Monroe found that 92 per cent of answers to a test and a retest two weeks 1ater were consistent, yet the respondents could still have remembered their previous answers in this case. Belson conducted a retest after a shorter period still: one week. The percentage of consistent answers amounted to 88. Also Farrington’s study revealed rarther a high degree of consistency in spite of the two year's interval. The percentage of mistakes in the test or retest was 3.2 The tendency to inconsistent answers was less explicit if the general scores of the examined persons on the deviance scales were analyzed and not the proportion of their affirmative and negative answers to the separate questions (11.5 per cent of the, examined persons found themselves in another quartile than Before). The results obtained by Shapland were parallel. The results seem to point to a high stability of self-report questionnaires in time.             Hardt and Peterson-Hardt distinguish the following methods of defining the validity of self-report questionnaires: comparing with external sources of information, comparing with a known group, lie scales, and defining face validity.             The most frequent method of defining the validity of questionnaires used in examination of unrecorded deviant behaviour is the comparison of the respondents' answers with other reliable sources of information. Erickson and Empey found that none of their respondents concealed their contact with the police or an offence with which they were charged. According to Gold, the probability of contacts with the police diminishes monotonically together with decrease of frequency of offences admitted during the examination. Gibson, Morrison and West found a high consistency between  offences revealed by means of the self-report method and the contents of the police files. Hindelang found a distinct positive interdependence between high scores in the deviance scale and having a record in the police files. Farrington, as well as Farrington and West, examined the so-called predictive validity of self-report questionnaires. It appeared that those of the examined persons who score highest in deviance scales at the moment. A, have records in the police files much more frequently at the moment B. Gould compared the scores in the Short,/Nye scale with those in the recorded crime scale, finding a high, positive and valid interdependence.             Results of self-report tests were also compared with other sources of information (teachers, colleagues, social workers, etc.). As shown by Jessor, Graves, Hanson and Jessor, results of the self-report tests tally with appraisals of the degree of deviant involvement made by teachers and colleagues of the examined persons. Also Gould compared the respondents' statements with appraisals of their behaviour made by their colleagues and teachers. The interdependence proved to be as expected. Hardt and Peterson-Hardt compared statements in which the examined persons admitted having robbed parkometers with the official data concerning the extent of these thefts. The respondents appeared to have answered truthfully.             In many studies scores of school children and of institutionalized youth were  compared. As demonstrated already by Short and Nye, although the inmates of reformatories scored somewhat higher than students of normal schools, nevertheless the profiles of distributions and their structure were analogous. Voss found the correlates of deviance in groups of school children and institutionalized youth to be parallel. This finding was confirmed in many other studies. The only exception here is the parents socio-economic status. Uniformity of views could not have been reached as yet as to whether the positive interdependence between the socio-economic status and deviant behaviour found in the majority of self-report studies is artificial or real (see i.a. Tribble, Axenroth, Hindelang Hirschi and Waise).             Much can be said about the validity of a self-report questionnaire only on the grounds of the distributions of answers to the separate questions, Siemaszko found the percentage of affirmative answers to decrease monotonically with the increase of seriousness of the act and its scarcity in the general population. In the same study the percentage of affirmative  answers to the question about being checked by the police was found to be higher than that concerning detention: also the level of deviance of elder as compared with younger and boys as compared with girls proved higher, These results agree with theoretical expectations, Hardt and Peterson-Hardt found the percentage of affirmative answers to the questions about acts commited during  the last year to be generally lower than it is the case with questions that concerned also acts commited longer before. Not all of self-report questionnaires contain lie scales. Moreover, the researchers are not in agreeement as to the usefulness of such scales this type of studies (i.a. Farrington, Smart, Hardt, Peterson- Hardt). I seems that lie scales should be employed Questions should however be avoided  which  might be correlated with deviant behaviour, as in such case there is the danger of the lie scale becoming the reverse of that of deviance.             The popularity of self-report studies was determined by the effectiveness of this method (relatively low cost 1ittle time consuming, promptitude and the possibility of examining large samples) Today, self-report studies have become popular in spite of the fact that many important methodological problems have not been solved yet.