good evening everybody i'm brian and i'm back again with ask the experts our topic tonight is results based accountability and government very happy to have here two guests senator diane snowing and susan from as the who is the chief performance officer in the state of vermont welcome i'm excited about this topic because it's one that we've been working together on with the government accountability work and i think just to tee it up we should start with what action in one of the examples i like is that we all use it the same process or similar process every day for example with the that they're responsible for and they work through policy issues and it is that kind of straightforward approach but it's based on mark friedman's book trying hard isn't good enough and he lays out this very straightforward process and i say straightforward but i don't mean easy it's simple to understand but the work is hard and it's a been very effective internationally globally in helping communities get from talk to action and really get good results for making change making things better so one of the reasons i'm interested in is to get a better handle on how we're spending taxpayer dollars and where we're making and investments and what kind of outcomes are impacts we're getting for those investments so your role the chief performance up maybe you could just talk about what that is and how you're working in government i had and working with strategic planning i'm working with the executive branch efficiencies but one of my primary duties is after the legislature passed at twenty six setting out some initial population outcomes for the state my role is to try to train all the departments and agencies in using r b a a three to five year program of reporting on programs and or activities for those who may not have read act one eighty six the outcomes the high level outcomes are easy to understand and really what state government was meant to do well educated work force good job accessible health care so you know there are obviously many others but but using mark friedman's methodology i train fifty eight people across the state and from a pilot program last year that had thirteen programs across eleven departments we're going to have more than thirty programs across at least that many departments this year and we're tying it to the budget which i think goes to your point about resources it's we're very good at saying how much we do it's times we're good at saying how well we do it but we're not historically we haven't been really good at saying is anybody better off for our actions than are our programs actually succeeding in the results we hope that they have so i thought we should talk a little bit about why we want to do this with the legislature think for many of the reasons you mentioned it's very important to understand how to track the dollars we spend and what they accomplish and as tyler dollars get very tight we need to have more and more information  about the kinds of changes that are actually happening the interest started in well before two thousand and eight but most recently in two thousand and eight when the government accountability committee was formed and we went back and looked at all of the materials that were on file from strategic planning to efficient government to making government more effective and a lot of good work was done but it never seemed to really last it didn't seem to be part become part of how things were done in r b a in vermont actually goes back to the one nine hundred ninety s when con hogan was the secretary of human services and he worked with marc freedman then and we got some of the outcomes that we have now in law at that time in they've evolved still pretty similar but valving and as the government accountability committee reviewed all the past work we came to understand that one of the most important things was to choose a methodology and stick with it and really try to help everybody speak the same language so that we could communicate in a straightforward fashion and really start being comparative across programs to things that work well and understanding what we might need to do to make them work better and we did put into statute the ten outcomes or eight i'm sorry eight outcomes and even the process for the legislature the administration and the community to work together on this because this is not just something the legislature does and it's not something the administration does but it really is the entire community working together to make things better and it comes from that we we all want things to be better we want children to have better lives we want families to be stronger we want the things that government supplies to make a difference so could you just talk about what you're doing specifically in your agency about developing and cultivating skills within state government sure i've done many training sessions all day long seminars with what we what the statute refers to as the performance accountability liaison pals for short and so i've trained them in the use of this i go from agency to department to department and work with teams of people on the programs that they're looking to improve results on help them decide on what are good performance measures help them decide how that program fits into the indicators that are in act one eighty six as well as the state wide strategic plan which by the way mirror each other very well so so that's a plus and the departments and agencies most notably human services to date are also working with their nonprofit and outside providers and partners to get these outcomes and results into the contracts and grants that we give them because in many cases it's outside providers or outside partners that actually run many of the programs of state government so we want them to be part of this too and actually even spoken in front of several of the nonprofit groups  it's it's hard work as senator smelling said but the more people go to training the more exercises we do the more speaking i do the more comfortable people become so i think it's beginning to be a groundswell both in the public sector both of the nonprofits and and community partners and within the agencies and departments and the legislature to just a little some tiny step back if i think it's important people understand that what r b a does is talk about accountability i mean that's really what it's all about in different interpretations of how you apply that but it divides accountability into two in two ways one is population level accountability where all efforts lead to certain outcomes and that's how you talk about the big picture and then in each and every program and organization you have performance measures which help you understand whether those programs are working so i just i think it's important because so many people feel like well how can my one program make any difference in the larger outcome and it's really collective impact that we're talking about for population change so v a is actually divided into two segments they top segment is the population accountability which talks about outcomes or results and indicators which are sort of broad measures of whether you're achieving things or not for example recidivism rate now there's no one program that is solely responsible for recidivism rate so that's when you get down into the performance accountability which are the individual programs and your strategies for how you're going to end your individual program affect risk and there are three in use performance measures to judge that too to display results and there are three types of performance measures how much do we do how well do we do it and is anybody better off historically the state is great at how much do we do we've got that down pat case loads volumes i mean we know that but we have not been as experienced at actually documenting is anybody better off which is the real goal that we want for any program whether it's a smoking cessation program you want people to stop smoking a training program you want them to finish the training gain new skills and potentially get a better job so all of the programs have accountability for their individual peak but they're not obviously responsible for the overall indicator they're just responsible for their piece if the indicator is unemployment rate the training program is accountable for the people that come through their training program but they can't be responsible for unemployment rate across the state and i think that's a very and orton's part for people to understand and it's also something that you know frightens people a little bit because they're afraid and senator selling said well how can i be responsible  interesting thing i think to note is that at that population level it's sort of the policy level of faith in that within the legislature we're talking about what are our population level outcomes or what are what are the impacts that we want for these investments that that's the high level but then that then nested in there underneath there are all of these programs that hopefully are actually driving that bigger outcome and there may be many of them as you said with the case of recidivism there may be many programs in state government that actually impact that and when those programs here established either from grants from the federal government or funding from from the legislature there was a reason that program was established and there was an intended goal  tended result for that program and you know since we do have limited resources and i think we always will this is a way we can assure selves that we are getting the best bang for our buck the other thing that  i wondered if you could comment on that is you know how we're approaching it within the legislature we've been talking with as committee chairs about the  moving this whole culture into the legislature and thinking of some strategies about how to do that so i wonder if you if you could just share what you're thinking in terms of the legislative group but we don't quite a bit of training with the legislature we had friedman himself come into training which was really a lot of fun and very exciting and we've done smaller trainings with different groups and we're intending to do more of that we'd like to have committees meet with their associates in the administration and in the community and have a facilitated conversation using r b a which would help everybody put it into practical use so they can understand it a little bit better and i just need to add that part of it but part of it's important to understand that this is a communication technique and that it's not you know many people hear the word accountability and they think about somebody is going to make me look bad or i might end up looking bad when i didn't realize it and this is much more about sharing the knowledge that every individual has in the work that they do and how to create new bridges to do programs better to get a better result so i just wanted to reassure people that it's really very it's about being it more informed about the programs that you're working on and taking those different pieces of knowledge and putting them together and coming up with a better result so we're working with the committees of jurisdiction every community in the legislature has an area that they're responsible for and they work through policy issues and many committees contribute their recommendations based on the budget so the hope is that we can get all committees and their members and all legislators informed about the basics of examples of of actual programs within their area of expertise you know we have some general learning programs that we use we use as senator snelling said the leaking roof we also used the local fire department stuff and that's how you you help them understand the basics but they really begin to get it when you start talking about specific programs in their area of expertise so you start talking about in the theoretical academic exercise that actually shows them how they can see whether a program is achieving the results that they intended when they legislated for that program so this this whole concept and that's a little bit lingo but turn the curve that there has to be a baseline that's right some baseline data so is part of your work establishing what those baselines are so we could then within the legislative process say how need to understand where you've been and where you're going and you know mark friedman says because you get natural resistance from people we don't have that data we have made collecting that data and if you don't have anything else you have right now you have the data you have right now and you start from today and you start with the programs that you do have data on might not be the top program you'd really like to be doing but you've put in place a plan for how to begin to get the data you need on a particular program and in the meanwhile you work on the program that you have data for and start with what you have and work toward where do you want to go and it is data data is the critical element in all of this and that may  sound harsh and and very numeric but the kind of data that we're talking about especially with is anybody better off is than monetary per se it's more like if i have a training program how many people completed the training program and of those people that completed the training program what percentage of them got a better job or got a job which is the point of that training program so it's not about i  can do more if you give me more money it's about actually being able to say what you did with the money and if you had more money exactly what you'd be able to do not just the theoretical i could do more one of the things i appreciate appropriations is the thinking around not not just the fear base which is what will be cut but where we invest more because we're getting the kinds of results that we're looking for to look at both both sides of the coin and one of the things  in my working with some of the outside providers and partners of the agencies and departments is saying to them you know imagine we'll use training programs again we have various training programs all over the state in different areas of the state  what if you could get all of them together in a room and talk about who's having success with a certain element of the program well how come your program is having better results in a particular area attendance than my program i think that there's a lot to be learned between the prova among the providers and partners and state departments and agencies about what other people are doing in very similar programs it may not even be exactly the same but you can learn a lot and try some new things that are not costly to do marc freedman always includes a low cost as one of the elements and i like that about to do that it is and it isn't you know first of all it's serious but it doesn't have to  be you know without humor also and i think one of the things that happens when you bring disparate groups of people together you actually want to have some people that are so completely stayed in what we're doing now because they'll ask that question you won't ask well why do you do it that way well because we've always done it that way and it gets you thinking about other things where you're just so used to doing things the way you do them that you think well you don't think so you don't think that there's any other way to do it and you know i've had that a experience facilitating meetings for departments i'm not the expert on what their program is i'm just their stor another part and we can put those two things together and and expanding both of our understanding of things and i find that's what makes this fun i mean it's also work and i don't mean to minimize it but some of the conversations i've been involved in was with mixed groups of people there are a lot of fun yeah because people you know you see the light go on a nasty people come up with really creative ideas and that are not costly i think we all want it to to be to be making a difference to make things better and i know there are just very many committed legislators and people in the ministration and in the community who are working very hard to see things change for children and families or  hope variety of programs and this is a way to share their knowledge and feel good about what you know and also keep learning are not also think what is things that's important that we're trying to do is to try to connect head the budget efforts with the strategic planning effort and the performance accountability effort historically those three things have been done very separately budgets over here the strategic plans over there and performance accountability somewhere else well really they should overlap and inform each other and in the r b a can be used to do that and at one hundred six is driving us to not just say where our programs are and what results are they achieving but tie that into the actual budget dollars for that program which is something we've never really done before so i think if nothing else it'll be a lot more information and a more granular level for the legislature to work with for the public to understand where the money's going and what the investment is right so that the concept of from talk to action i know that you senator snelling talk about that a lot like that it's quite frustrating people in government and people looking at government to see a lot of talk and we talk about accountability we're interested in the kind of military we've talked about it wrong time but actually getting to action about it is one of the underlying well you're just perfect example right now talking about education spending and how do we improve quality of opportunities for every student and it's something we can all talk about and talk about and talk about and talk about how do we get to possible solutions do we get to how do we make those changes and if you use something as methodical and as analytical as r b a you can actually get to an action plan and start making those changes and that's very exciting in a whole variety of arrogance it's like a tool in the tool kit to get it into action it's a tool and it took at and i'll just say this out of people out there my no other  results based methodology used that are used for efficiency tools that are used and r b a is not exclusive in the way that act when eighty six was written it allows agencies and departments to use other methodology to for example  wane but what it what we will do is we will sort of standardize the language so we don't have multiple sets of language we'll call outcomes outcomes in this and you know and performance measures performance measure so that we can at least regardless of whether a department in state decides to use which works wonderfully with business process efficiencies they can still talk about their results and r b a speak so that everyone will understand when you say outcome or when you say indicator you know exactly what that means and sharing the same language is really a critical piece and if you just think about it is any common language you know we if we were to for all speaking english are all speaking spanish or french at least we know what that language is and what we what our intention is when we use certain words in the detail you know the devil's in the detail i used recidivism as an example before but when i say that what do i mean how long out of prison is one if one commits a crime again what period of time do we count is it three months six months a year and in vermont statutorily it's three months so we know that one but if i say high school graduation rate and my counting seniors only my counting how many students went into freshman year and went through all four years it's a get it can get complicated if you do not and then there's a piece in r b i called the story behind you know the baseline and you explain exactly what you're measuring how you're measuring it so that you know when i say high school graduation rate exactly how i'm calculating th so if you had one message for people about why we're doing this are being a work in and approaching government accountability right now well some of us came up here on the street and why are you why do you want to do  but yeah no i mean from my perspective it's because you're able to see what government is accomplishing i actually believe in good government and using  n and engaging the community of citizens to try who want to make life in vermont better then you can actually measure that using  a and to me that's a very exciting otherwise you can speculate you can tell stories you can have lots of other pieces but you're not actually seeing the results and once you do you can make things even better so i guess i would say what i like find the most engaging about it is that you get to use the data to back up your decisions that you're not making decisions  anecdotally they are not making decisions because of the whim of the moment you're actually looking at trends over time analyzing the data so that you know where your investments need to be spent or not employees in particular they want to do the best job they can and they do do the best job they can it's just another tool to help them track how their programs are performing and take you know simple steps that can improve the outcomes results and we're doing this in vermont staying around state government they're also doing it and other states we have some testimony from connecticut their work on appropriations with the results bas with many many people from all over africa as well as the rest of the world who've been using r b a one in particular i was very impressed with that was from leeds england where leeds decided that they wanted to be a child friendly city and within two years they had me substantial progress in improving circumstances for children and families and an example of that was that they had substantially  reduced children in foster care in two years and i just you know so it's happening it's being used everywhere the experience of coming going to africa and exchanging that with people from zimbabwe and mozambique and other african countries where the difference between the america in africa and what we have to work with with with so gigantic to see how much they have to overcome and yet here they are using r b a and we were using our we could share our stories and we could share our success and talk about it so that was very exciting and other states are working on this also i it's the end of january i'm going to a conference sponsored by the national association of state budget officers and before i became c p o i was deputy commissioner of finance doing budgets for the state and states are working on this and i'm going to be on a panel of people to talk about what we're doing in vermont and how we're approaching it yeah and there's a lot of interest and many states i think people look at it as a huge undertaking and i think sometimes it makes them afraid but and taking the time to be on the program that's all for this evening i will sign up now and good night