Source: https://oag.parliament.nz/2011/central-government/part4.htm
Timestamp: 2020-08-08 00:43:52
Document Index: 3020691

Matched Legal Cases: ['art 4', 'art 4', 'art 2', 'art 3', 'art 5', 'art 6', 'art 4', 'art4']

Part 4: Career Services — Office of the Auditor-General New Zealand
Central government: Case studies in reporting forecast performance information
Part 4: Career Services
Part 2: Why good quality reporting of performance information is important
Part 3: Elements of the performance story
Part 5: Ministry of Economic Development
Part 6: New Zealand Customs Service
Appendix 1: Public entities that received a good grade for their service performance information
Appendix 2: What auditors considered when grading service performance information
Appendix 3: Glossary
Part 4: Career Services Central government: Case studies in reporting forecast performance information. https://oag.parliament.nz/2011/central-government/part4.htm https://oag.parliament.nz/@@site-logo/logo.png
Central government: Case studies in reporting forecast performance information.
Career Services is a Crown entity. It was established under the Education Act 1989 to maintain and make available information and advice about occupations and post-compulsory education and training. It receives revenue of more than $16 million each year. More than 90% of this funding comes from Vote Education as a Non-Departmental Output Expense.
The SOI explained the importance of good decision-making about education, training, and career choices
Its 2010/11 SOI was a compact document that used an inviting layout and provided several useful diagrams.
The SOI explained the importance of good decision-making about education, training, and career choices, noting that the recession had made good information and advice to support such choices more important than ever. Figure 1 reproduces Career Services' description of its strategic context.
Strategic context for Career Services
Raising New Zealanders' awareness of where to get career decision-making support is an important step in giving everyone the opportunity to maximise their potential. Our ongoing challenge is to continue to extend our reach by making the best use of our low-cost web-based and phone-based services, while ensuring that personal assistance is available to the groups and individuals who need it most.
Young people in particular need support to navigate through the range of learning and career options open to them. Career Services is committed to supporting government initiatives such as the Youth Guarantee, to ensure young people are making well-informed decisions about their futures. We make the most of the latest technologies to deliver services with particular appeal to young people (such as personalised online My Career Spaces and our texting services). We also focus on building the career skills of parents and teachers so they can support young people as they map out their career futures.
As the country emerges from the recession, it becomes even more important to have a highly skilled workforce, to help lift productivity and contribute to economic growth. It is vital, too, that those affected most by current unemployment levels – such as young people, Māori communities and Pasifika families – are not left behind.
The first step in raising skill levels in the workforce of today and tomorrow lies in the ability to make good decisions about work and learning. New Zealanders need to know how to find independent and accurate information about jobs, education and training. As importantly, they need to make well-informed decisions that take their circumstances, skills and interests into account. They need to develop the job resilience that will help them avoid long periods of unemployment.
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 2.
Career Services set out its own place within the wider public sector work, education, and skills development spheres, showing how it contributes to Government priorities and actions (see Figure 2).
Career Services set out its own place within the wider public sector work, education, and skills development spheres, showing how it contributes to Government priorities and actions
How Career Services contributes to Government priorities and actions
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 6.
Career Services presented a brief environmental scan to set out the significant issues and challenges affecting the services it provides and its responses to those challenges.
Career Services' environmental scan
TABLE 1: HOW OUR PRIORITIES AND SERVICES RESPOND TO THE ENVIRONMENTAL ISSUES
Issue Response Our Strategic Priorities Our Services
Lifting public
Continuing to generate more innovative and cost-effective ways of delivering services Integrated self-help for all
Evidence channelling users primarily to web-based self-help and 0800 advice line services – the most cost-effective ways of reaching the greatest number of users
delivering more group guidance
exploring new services such as online guidance, virtual networking and wikis; seeking innovative ways to harness emerging technologies.
has the skills
it needs Ensuring people are aware of our services
Working with young people and their influencers, as well as schools, tertiary providers and ITOs Awareness
Young people and their influencers
Partnerships providing freely available online tools to help people match their skills and interests to jobs or school subject choices
enabling web users to click through to job vacancies and watch video clips about particular occupations
building others’ capability (such as parents and teachers) to support young people to make learning and career decisions.
Guarantee Helping young people identify career pathways; building the capability of schools, parents and whānau to support young people Young people and their influencers providing group training sessions to build the skills of parents and whānau, teachers, career professionals and others to help guide young people through learning and career decisions
providing advice and support to schools to help build effective career education programmes for students.
Māori success Further developing our whānau-based approach, based on what we have learnt from the Whānau Decision-making Project Young people and their influencers
Evidence using the results of the Whānau Decision-making Project to improve our support for young Māori making career decisions
providing group guidance sessions for Māori secondary school students to help them develop their career decision-making skills and awareness of education and work opportunities.
population Web-based and phone-based services for people of all ages Integrated self-help for all all age service, personalised and differentiated to meet people’s individual needs.
ethnic diversity Tailoring resources and services to meet the needs of different groups Integrated self-help for all
Partnerships providing targeted sections of the website to meet the needs of particular groups, such as Māori, Pasifika, migrants and refugees.
providing group guidance sessions (and smaller peer support sessions) for migrants and refugees to inform them about New Zealand workplaces and employers’ expectations.
people Reaching more young people through group work, capability-building and tailored web resources
influencers providing texting, webchat and online tools (such as CV 4 Me) aimed at helping young people with learning and career decisions; personalised online My Career Spaces
exploring virtual networking and wikis that appeal to young people
working with the secondary school, tertiary and industry training sectors to ensure students can access the information and advice they need to make
Helping people maximise their potential
Integrated self-help for all up-to-date web-based database of occupations that reflects changes to job descriptions, salaries and labour market trends
working closely with government agencies, both at the national and regional level to support relevant actions in the labour and education sectors, such as the Re-Start package.
a recession;
national debt Tailoring services to people impacted by unemployment
Finding ways to work smarter Integrated self-help for all
Evidence tailoring sections of our website to workers facing job uncertainty and redundancy
providing services to workers made redundant, through our 0800 advice line with additional guidance available from career consultants for those needing extra support
working with businesses (for example, to support those facing closure or redundancies)
supporting local employers and staff at the local level.
unemployment Tailoring services for people facing unemployment Integrated self-help for all
Partnerships working with the Ministry of Social Development to support initiatives to assist businesses and workers facing closure or downsizing
working closely with the Department of Labour to ensure our information about occupations and local labour markets is as up-to-date as possible.
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 12-14.
Specifying and presenting the outcomes framework
The previous year's result for each outcome measure was usefully set out as comparative information
Career Services' narrative discussion made its overall purpose and direction clear. A Results Framework diagram (see Figure 4) identified elements, including Career Services' contribution to the overall education sector outcome and government priorities, and set out its primary outcome – "Many more New Zealanders can make sound career decisions".
The Results Framework diagram also set out Career Services' four outcome statements (for example, "More people using our web-based service can match their skills and interests to jobs") that were the immediate and direct outcomes (or the impacts) of Career Services' outputs.
Career Services' results framework diagram
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 16.
Main measures and targets for outcomes and impacts
The SSP contained 12 outcome measures for the four outcome statements. These sat alongside the service performance measures for each output. The previous year's result for each outcome measure was usefully set out as comparative information (see Figure 5). The measurement method for all the measures is an independent annual evaluation.
Main measures and targets for outcomes and impacts and their relationship to outputs
We provide web-based information and tools (such as quizzes and wizards) to help people understand their skills and ambitions, find out more about the labour market, explore training and job opportunities and develop career plans and CVs.
Results/Measures
Target 2010/11
Target 2012/13
How it will be measured
% of people using web-based self-help resources who can identify a good match between personal skills and interests and career/job choice 88%
[achieved 87% in 2008/09] 90% Through an independent annual evaluation
% who have made a career-related decision aided by our services 70%
[new measure] 80%
Quantity measures
Number of visits from New Zealand-based people accessing web-based career information 2,500,000
[2,150,488 in 2008/09] 2,750,000 Google Analytics: website visits from NZ-based computers
Number of My Career Space registrations 20,000 new registrations [new measure] 50,000 My Career Space
% of people accessing online information who find it useful and meaningful 80% [79% said it was useful in 2008/09] 85% Through an independent annual evaluation
% of people who say website is easy to use and up-to-date 75% [72% said it was easy to find their way around in 2008/09] 80% Through an independent annual evaluation
Notes on Measures
Outcome: We would expect some small improvements over time.
Quantity: By way of comparison, the Retirement Commissioner's financial planning website, www.sorted.org.nz, received 1.2 million visits during the whole of 2008. Individuals may visit our website more than once and unfortunately it is not currently possible to determine the number of unique visitors. We would expect the number of visits to continue to increase over the next two to three years, but for numbers to eventually reach a maximum level that would be sustained over time.
The number of My Career Space users is a new measure to gauge the use of this recently developed resource for people to save their career information online. Over time we will use the My Career Space information to learn more about our users and improve our services.
Quality: The words "Up-to-date" have been added for 2010/11.
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 30.
In our view, the focus on impacts as a layer that is close to the delivery of the output is useful and appropriate for Career Services
The main measures are close in focus to the delivery of the output and also focus on the perceptions of Career Services' clients. For example, the measure "% of people using web-based self-help resources who can identify a good match between personal skills and interests and career/job choice" focuses on the clients of Career Services.
In our view, the focus on impacts as a layer that is close to the delivery of the output is useful and appropriate for Career Services, given its role within the wider public sector work, education, and skills development spheres, its organisational resources, and its overall size and scale.
The SOI also describes a three-year evaluation programme that includes an evaluation of key assumptions, such as how web-based services help people to identify a good match between personal skills and interests, and career/job choices (see Figure 6). Immediately after the three-year evaluation programme, Career Services reported some initial evaluation results that are useful for understanding the impact that its activities have had to date.
Information about Career Services' evaluation strategy
We have developed an outcomes framework and indicator matrix that help us monitor the results we are achieving. The diagram below shows some key indicators in each of the four Results-based accountability categories.
Further work on measures is being undertaken jointly with other agencies.
Our three year external evaluation programme is aimed at ensuring our services are meeting users' needs.
The programme is based on:
continuing to build on a core of information about the effectiveness of our services, whether it meets people's needs and how we can improve services
examining users' total experience with our integrated services: web, phone, face-to-face
each year, having more in-depth investigations into results for a particular client group (e.g. young Māori) or a particular product (eg. texting services).
Each year, we will evaluate:
how well the web-based services help people to identify a good match between personal skills and interests and career/job choice
whether people accessing web-based services find them useful and meaningful and easy to use.
Assisted advice
whether people accessing our 0800 Advice Line know which steps to take next and find that the advice meets their needs.
how well group sessions enable students to gain a greater understanding of how to make effective learning and career decisions
how well group sessions enable migrants and refugees to understand how to make effective career decisions in a New Zealand context
how well one-to-one guidance helps people identify relevant learning and career pathways.
Capability-building services
how well the skills building sessions prepare people to help others with career decisions
whether assistance to schools meets their needs.
At the end of each year, we re-examine evaluation priorities in light of survey results and new service developments. For 2009/10, we focused particularly on the whānau decision-making project.
For 2010/11, we will focus on secondary school students. We will:
follow up with a group of young Māori and Pasifika students who attended group guidance sessions to find out what progress they have made with respect to their career plans, six months after the guidance session
repeat the Career Decision-making Abilities evaluation that was conducted in 2008, tracking secondary students as they exit school, across a range of factors related to their career-related decisions.
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 22.
Career Services also states its intention to measure its contribution to government priorities for the use of Career Services by target groups (such as young people, and young Māori and their whānau). Comparative or baseline information on these matters could usefully be included in the SOI.8
Linking outcomes to impacts to outputs
The Results Framework diagram also sets out the relationships between Career Services' outputs and impacts, and the target groups and drivers that shape its output delivery.
After Career Services' brief environmental scan of the significant issues and challenges affecting its services, the SOI sets out Career Services' intended response. These intended responses link to Career Services' model for delivering services to individuals and groups, and its improvement intentions for output service effectiveness and efficiency.
The SOI (see Figure 7) notes that Career Services ensures value for money for New Zealanders by, for example:
channelling people, as far as appropriate, to the lower-cost web-based and telephone-based services; and
gathering evidence about what people want and which aspects of its services work best.
Career Services' integrated delivery model for services to individuals and groups
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 19.
Specifying outputs and output classes
Career Services has one output class – Provision of Information and Advisory Services. It identifies four outputs within the output class, and provides indicative costs for each output. Figure 8 describes the services delivered within each output.
Description of the services delivered within each of Career Services' outputs
Our self-help web-based resources provide people with:
up-to-date, comprehensive and independent job, course and training, and labour market information
good practice career education resources for teachers and careers advisers
online career tools such as ‘CV 4 Me' (a CV Wizard), skill matching quizzes and ‘My Career Space' (where people can save their CVs and other career-related information)
We also offer live webchat, so anyone in New Zealand can get instant online answers to their questions about courses and occupations.
our 0800 advice line, which provides general career information; advice about careers, jobs and training courses; needs assessment; and referral to in-depth guidance if callers' needs can't be met in other ways
a texting service for requesting career information (external evaluation results showed that two-thirds of participants surveyed would not have contacted Career Services if the texting service didn't exist.
follow-up calls to check whether people need further information or help, an opt-in appointment reminder service and a tips of the day service.
CV feedback for young people who have used our online CV 4 Me tool.
customised career planning in a group setting to secondary school Māori and Pasifika students, prospective tertiary students, migrants and refugees and other groups.
individual face-to-face career planning assistance to targeted people, including young Māori and Pasifika people, workers who have been made redundant and recent migrants
telephone guidance, so people can access in-depth personal career guidance without having to visit one of our offices. This makes our services more accessible to people who live in remote locations, have care-giving responsibilities and/or have a disability (this service is also available outside normal office hours)
online access to a career consultant.
Capability building services
We work with a range of organisations and groups to build their ability to support others – especially young people – with learning and career decisions.
In particular, we provide:
customised advice and professional development assistance to schools.
customised advice to parents, whānau and others who support young people to make career decisions.
Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, pages 3-4.
The grouping of these outputs into one class seems appropriate to us because all outputs relate to providing information, guidance, and advice. All the outputs are externally focused and cover the significant goods and services that Career Services provides.
The important feature of Career Services' output performance information is that the focus and weight of the measures is on relevant dimensions of the quality of services
Performance measures and targets for outputs
Career Services identifies output measures as either quantity or quality measures. All the measures include specific criteria or means for measuring performance. Although quantity measures have a demand dimension to them (such as the number of website visits), this information is relevant to Career Services' effectiveness and efficiency improvement intentions.
The important feature of Career Services' output performance information is that the focus and weight of the measures is on relevant dimensions of the quality of services (in particular, whether services meet people's needs).
8: "Every young person has the skills and qualifications to contribute to their and New Zealand's future" and "Māori enjoying success as Māori".
ISBN 978-0-478-32688-8 (print)
ISBN 978-0-478-32689-5 (online)
PDF (351KB, 69 pages)