What are the main data item outputs of the Study? Geographical and coverage of survey cohorts Consistency with good practice in longitudinal survey design to measure change over time Type of survey data outputs Literature review reflecting on ‘representativeness’ versus ‘variability’ Desktop review of data bank fields and data Comparison to data outputs for selected comparator studies What flexibility has there been to expand/modify existing questions? Number and type of changes to survey questionnaires to enhance relevance Key informant interviews Desktop review of selection of recent survey questionnaires Has it been possible to make new inquiries of the existing database? Number of new analyses responding to current information needs that draw on data bank Number of sub-studies that draw on data bank Desktop review of papers and sub-studies Quality of the ALSWH - Area of inquiry – 1.2 Timeliness
What checking is undertaken to ensure accuracy of data? of error measurement undertaken Measures to test questions for clarity and sensitivity Desktop review of standard Study measures undertaken to check data accuracy Quality of the ALSWH - Area of inquiry – 1.4 Coherence Research question Indicator Data source/approach-primary Data source/approach-Secondary How much variation is in reporting of findings across survey periods? Proportion of common reporting items across survey waves Desktop review of reporting topics over the life of the Study How consistent are the Study methods with similar longitudinal studies Use of standard variables and classifications, where available Key informant interviews Desktop review of Study methods and commentary on consistency with standards of good practice Quality of the ALSWH - Area of inquiry – 1.5 Interpretability Research question Indicator Data source/approach-Primary Data source/approach-Secondary
How well documented is information about the statistical measures and processes of data collection for external data users? Type and format of publicly available information about statistical measures and processes/methods of the Study Desktop review of information available detailing Study statistical methods and processes Quality of the ALSWH - Area of inquiry – 1.6 Accessibility Research question Indicator Data source/approach - Primary Data source/approach - Secondary How accessible is the Study information to a range of end users, including appropriateness of the meta-data? Number and type of approaches used to provide access to the Study findings and summary data by target audience Measures to facilitate data user independent management and interrogation of the data bank, including processes for gaining access Resources required for external users to access data Key informant interviews Stakeholder interviews and surveys Desktop review of the products designed to allow access to findings and data bank Desktop review of how study data are available and accessible
Contribution of the Study to the evidence base for women’s health in Australia – 2.1 Building the evidence base Research question Indicator Data source/approach – Primary Data source/approach – Secondary To what extent is the information collected through the Study contributing to the evidence base for women’s health in Australia? Number of peer-reviewed articles using Study data and number of citations relating to these articles Women’s health policy at a state/territory and/or federal level that utilise Study data and/or reports Number of conferences using Study data and/or reports Desktop review of and government website information What strategies are in place to promote data usage for building an evidence base? Type of approaches designed to embed data outputs in recognised evidence base to inform decisions Proportion of stakeholders who consider the is a recognised contributor to the evidence base Stakeholder interviews and surveys Desktop review of Study approaches to ensure findings contribute to the evidence base Literature review for success factors in establishing research findings as part of the evidence base to inform decisions
Contribution of the to public health research practice – 3.1 Public health research practice Research question Indicator Data source/approach – Primary Data source/approach - Secondary What strategies are in place for influencing public health research practice? Documented processes for disseminating findings to the wider public health research community in Australia Number and reach of research conferences attended to present on Study findings Established avenues for influencing research agendas/strategies of government and research grants bodies Contribution of the to quality public health research practice, including: Number of student projects completed using ALSWH data Reports of studies that have adopted ALSWH research methods or that ALSWH informed their study design Key informant interviews Stakeholder interviews and surveys Desktop review of approaches to engage public health researchers in Australia in awareness of Study findings Desktop review of external researchers accessing Study data bank Desktop review of methods used to add to the quality of public health research practice through sharing of information about conduct of the Study Literature review for success factors in promoting longitudinal findings, and research findings more broadly, to influence research practice
How the Study has affected participants Area of inquiry Research question Indicator Data source/approach – Primary Data source/approach – Secondary 4.1 Lifestyle changes Has the Study data been affected through lifestyle changes precipitated by involvement of participants in the Study? Prevalence of lifestyle risk factors in the general population compared to Participant feedback over time on their lifestyle Analysis of qualitative information related to lifestyle changes from a sub-sample of surveys for selected cohorts over the last 10 years Key informant interviews Desktop review of major reports produced from the Study on adherence to health guidelines, health behaviour changes over time 4.2 Survey fatigue Has Study participation been affected by survey fatigue? of survey attrition Participant feedback on ability to complete the survey Number, size and frequency of additional survey requests (sub-studies) utilising sample Analysis of qualitative information in sample of surveys completed in last survey wave for all cohorts for reference to survey burden Study staff interviews relating to outcome of first reminder call for key themes relating to delay in responding Key informant interviews Review of attrition rates and rationale for all cohorts over life of Study Review of information on withdrawal from contained in the participant database Desktop review of undertaken, response rates and strategies to avoid disruption of Study sample
5. Value for money of the ALSWH Area of inquiry Research question Indicator Data source/approach – Primary Data source/approach - Secondary 5.1 Measurement of Study costs What are the associated with the Study for the last 3-5 years? Operating expenses Labour Other costs Capital costs Activity expenses Survey development Implementation of survey Survey analysis Reporting Marketing and Promotion Information requests of Study staff Desktop review of annual reports 5.2 Measurement of Study outputs What outputs does the Study deliver? Number of peer-reviewed articles using Study data and number of citations relating to these articles Women’s health policy at a state/territory and/or Key informant interviews Stakeholder surveys Desktop review of relevant previously collected secondary data under Key Theme 2.
federal level that utilise Study data and/or reports 5.3 Measurement of selected comparator survey costs What are the costs associated with delivering selected comparator studies? See ‘Measure of costs’ Desktop review of annual and financial reports for selected comparator studies 5.4 Measurement of selected comparator outputs What outputs do selected comparator studies deliver? Number of publications study data Number of citations for publications using study data Number of state/territory and/or federal health policy documents utilising study data Consideration of survey length, funding, topics(s) and size Desktop review (e.g. publications on selected comparator study websites, number of citations) 5.5 Efficiency of the Study compared to selected similar longitudinal studies How does the cost to output ratio compare between the Study and Study outputs as a proportion of total costs for ALSWH and selected Desktop review using secondary data previously
6.3 Communication Frequency, nature and of communication about Study findings As listed above 6.4 Survey frequency Do surveys need to be conducted more or less frequently? Do more or less questions need to be Should the cohorts continue to be surveyed in their current form? Optimal frequency of data collection As listed 6.5 Questionnaire coverage As listed above Size of questionnaire commensurate with data utilisation As listed above 6.6 Data collection methods As listed Good practice in survey data collection methods relevant to ALSWH As listed above 6.7 Other As listed above Other indicators of improved value arising from evaluation findings As listed above