Executive Summary This report presents findings from the final evaluation of projects funded in Round 3 of the Respectful Relationships initiative. The approach to evaluating Round 3 projects included both an evaluation of project design and content against the National Association of Services Against Sexual Violence (NASASV) standards (Process Evaluation), and an evaluation of participant outcomes in achieving positive changes in awareness and attitudes towards respectful relationships in the targeted population (Outcomes Evaluation). The 11 projects targeted diverse participant groups including mainstream primary and secondary school students, students in flexible learning options environments, Indigenous young people, young people from culturally and linguistically diverse (CALD) backgrounds in school settings, and young people in community or other institutional settings such as detention. Participants ranged in age from 8 to 24 years. Each project delivered respectful relationships education programs in multiple venues. Overall, the process evaluation found that projects demonstrated appropriate approaches to: Community and school consultation, Cultural inclusion and relevance for culturally specific projects, Overall training and support for project facilitators. A major component of the process evaluation was a comparison of the performance of each project with the standards outlined by the National Association of Services Against Sexual Violence (NASASV). The evaluation found that overall, projects demonstrated compliance with the six NASASV standards. The main specific findings of the process and outcome evaluations were: Most projects clearly articulated a feminist approach to understanding the gendered nature of violence perpetration. Others were reluctant to use and articulate feminist theory in their project design due to a perceived negative impact of feminist ideals. Theories of change were used to develop program content but the relevance of these theories to the efficacy of the project was not demonstrated for individual projects. All projects used considerable time and resources to ensure that the content and delivery of their projects were inclusive, relevant and culturally sensitive to the diverse range of students that they work with. Projects generally demonstrated comprehensive program development and delivery with extensive community consultation and stakeholder engagement particularly when working with Indigenous students and communities. One-off projects that were designed for a target population with specific needs and challenges were successful in facilitating initial engagement with populations and organisations that had not previously been engaged with respectful relationships education.
Attitudes towards women for adolescents () Procedure Survey respondents completed questionnaires prior to commencing their Respectful Relationships program and directly after completing the program. Where possible, participants also completed a follow-up questionnaire 2-5 months after the conclusion of their program. The pre-post questionnaire design aimed to measure whether young people’s attitudes on different topics relating to respectful relationships changed after taking part in a Respectful Relationships program, and whether this was maintained over a longer period of time. By measuring attitudinal change, the evaluation examined the effectiveness of each project in changing attitudes around sexual, physical and emotional abuse in relationships. See Table 5 for a summary of questionnaire types used for evaluating each project and for detail concerning stages of surveys completed for each project (pre-program, post-program and follow-up). Table 5 Questionnaire type and waves completed by project Project organisation Questionnaire type Interrelate (NSW) Primary school Ipswich Women’s Centre Against Domestic Violence (QLD) Original Uniting Communities (SA) Original Vocational Partnerships Group Inc. (QLD) Abridged Council For Domestic and Family Violence (WA) Original Women’s Health West (VIC) Primary school cohort Primary school ESL course cohort (young adult) Abridged YWCA NSW (NSW) Primary school cohort Primary school Secondary school cohort Abridged YWCA of Canberra (ACT and SA) Primary school Pre-
11 Individual characteristics of individuals that participated in wave 1 and 2 only by organisation (data from wave 1) Project organisations Number of schools that took part Number of respondents (wave 1 and 2 only) Individual characteristics Age (mean) Gender (female) Aboriginal and/or Torres Strait Islander status (ATSI) Country of birth (Australia) English spoken at home Primary schools Interrelate 16 417 11.61 223 (54.0%) 56 (13.7%) 407 (97.8%) 409 (98.1%) Women’s Health West 4 237 12 138 (58.2%) 7 (3.0%) 179 (76.8%) 199 (85.8%) YWCA NSW 2 3 11 2 (66.7%) 0 (0.0%) 3 (100.0%) 2 (66.7%) YWCA of Canberra 3 36 11.25 19 (55.9%) 1 (2.3%) 27 (75.0%) 27 (77.1%) Total 25 693 11.72 382 (55.6%) 64 (9.4%) 616 (89.5%) 637 (92.7%) Secondary and post-secondary schools Ipswich Women’s Centre Against Domestic Violence 2 17 15.14 13 (76.5%) 0 (0.0%) 13 (76.5%) 14 (82.4%) Vocational Partnerships Group 2 12 14.4 6 (50.0%) 2 (16.7%) 11 (91.7%) 11 (91.7%) Women’s Council 1 32 14.75 18 (56.3%) 0 (0.0%) 15 (48.4%) 19 (59.4%) Health West 2 11 19.5 4 (36.4%) 0 (0.0%) 0 (0.0%) 1 (10.0%) YWCA NSW 5 72 12.76 50 (70.4%) 17 (24.3%) 68 (95.8%) 68 (95.8%) Total 12 144 14.02 91 (63.6%) 19 (13.5%) 107 (76.4%) 113 (79.6%)
Scales and paired t-test results – Primary schools 22 Scale summary and t-test results for “Response to Anger Scale” for primary schools by organisations Scale Summary (all respondents) Paired t-test (same respondents across waves 1 and 2 only) n mean sd min max n mean mean diff se t df Interrelate Family Centres Wave1 (pre) 565 2.54 0.34 1.31 3.00 411 2.54 -0.03 0.02 -1.83 410 Wave2 (post) 505 2.52 0.37 1.15 3.00 2.51 0.02 Wave3 88 2.61 0.31 1.46 3.00 n/a n/a n/a n/a n/a n/a Women's Health West Wave1 (pre) 279 2.53 0.32 1.25 3.00 236 2.53 0.04 0.02 2.07* 235 Wave2 (post) 347 2.58 0.36 1.15 3.00 2.57 0.03 Wave3 18 2.63 0.24 2.08 3.00 n/a n/a n/a n/a n/a n/a YWCA NSW (pre) 5 2.34 0.25 2.15 2.62 2 2.38 0.35 0.23 3 1 Wave2 (post) 3 2.67 0.16 2.54 2.85 2.73 0.12 YWCA of Canberra (pre) 42 2.60 0.35 1.33 3.00 34 2.57 0.10 0.06 1.97 33 (post) 37 2.65 0.24 2.00 3.00 2.67 0.04 23 2.68 0.23 1.92 3.00 n/a n/a n/a n/a n/a n/a
Table 24 T-test results for “Response to Anger Scale” and “Conflict Resolution Skills Scale” for primary schools by organisations for same respondents across wave 1 and 3 only Paired t-test (same respondents across waves 1 and 3 only) Response to Anger Scale Conflict Resolution Skills n mean mean diff se t df n mean mean diff se t df Interrelate Family Centres Wave1 (pre) 84 2.59 0.02 0.03 0.66 83 80 2.33 0.05 0.05 1.33 79 Wave3 (follow-up) 84 2.61 0.03 80 2.38 0.05 Health West (pre) 12 2.52 0.04 0.09 0.46 11 12 2.19 0.20 0.13 1.68 11 Wave3 (follow-up) 12 2.56 0.07 12 2.39 0.13 YWCA of Canberra Wave1 (pre) 23 2.59 0.09 0.08 1.11 22 23 2.50 -0.12 0.09 -1.06 22 Wave3 (follow-up) 23 2.68 0.05 23 2.39 0.06 Scales and paired t-test results – Secondary and Post-secondary schools Table 25 Scale summary and t-test results for “Acceptance of Dating Abuse Scale” for secondary and post-secondary schools by organisations Summary (all respondents) Paired t-test (same respondents across waves 1 and 2 only) n mean sd min max n mean mean diff se t df
Wave 3 .05 (.04) .20 (.10) -.13 (.10) Intercept 2.12 (.06)*** 2.04 (.16)*** 2.41 (.10)*** Variance components Student (between students within school) .10 (.02) .12 (.06) .03 (.02) Residual (within student) .06 (.01) .06 (.02) .11 (.02) LR test vs. linear regression chi2(1) = 93.78*** chi2(1) = 203.17*** chi2(1) = 3.15* Person period 236 35 63 Number of individuals 84 12 22 *p<.05, **p<.01, ***p<.001 Regression analysis of scale scores – Secondary schools Table 35 Acceptance of Dating Abuse Scale for secondary school: Multilevel regression analysis with random effects for student level and where applicable school level Unstandardized Coefficient (SE) Ipswich Centre Against Domestic Violence Vocational Partnerships Group Women's Council for Domestic and Family Violence YWCA NSW Female .04 (.20) -.01 (.24) -.03 (.11) .19 (.09)* Wave 2 .06 (.09) -.27 (.19) .21 (.10)* .06 (.04) Intercept 3.60 (.18)*** 3.75 (.20)*** 3.53 (.10)*** 3.50 (.08) Variance components School (between schools) n.a. n.a. n.a. <.01 (.01) Student (between students) .08 (.05) .07 (.08) <.01 (<.01) .09 (.02) Residual (within student) .07 (.02) .21 (.08) .18 (.03) .06 (.01) LR test vs. linear regression chi2(01) = 4.95* chibar2(01) = 0.89 chibar2(01) = 0.00 chi2(2) = 20.24***
Residual (within student) .09 (.03) .26 (.07) LR test vs. linear regression chibar2(01) = 14.15*** chibar2(01) = 2.24 Person period 33 63 Number of individuals 17 32 *p<.05, **p<.01, ***p<.001 Table 38 Response to Anger Scale for secondary school: Multilevel regression analysis with random effects for student level and where applicable school level Unstandardized Coefficient (SE) Ipswich Centre Against Domestic Violence Vocational Partnerships Group Women's Council for Domestic and Family Violence YWCA NSW Female .34 (.10)** .29 (.16) .14 (.09) .20 (.07) Wave 2 .02 (.04) -.05 (.11) .08 (.04) -.01 (.03) Intercept 2.41 (.09)*** 2.47 (.13)*** 2.53 (.07)*** 2.44 (.07)*** Variance components School (between schools) n.a. n.a. n.a. .01 (.01) Student (between students) .03 (.01) .04 (.04) .05 (.02) .05 (.01) Residual (within student) .01 (<.01) .08 (.03) .03 (.01) .04 (.01) LR test vs. linear regression chibar2(01) = 10.65*** chibar2(01) = 1.56 chibar2(01) = 14.21*** chi2(2) = 29.81*** Person period 33 24 63 139 Number of individuals 17 12 32 71 Number of schools n.a. n.a. n.a. 5 *p<.05, **p<.01, ***p<.001 Table 39 Conflict Resolution Skills Scale for secondary school: Multilevel regression analysis with random effects for student level and where applicable school level Unstandardized Coefficient (SE)