Developing a Data and Evaluation Framework
for Urban Promise Zones - Session 1
Janet Pershing: This is Janet Pershing speaking. I'm going to be helping facilitate the session
today. And I'd like to welcome you all to this webinar. We're glad you're all here, and we arelooking forward to a series of four of these sessions over the course of the next eight weeks or so.
First thing I'd like to do here is to get you a little bit oriented to the electronic platform we'reusing and let you understand some of the ways that we plan to interact. So you can see here wehave a number of different mechanisms for involving you in what we're doing here in thesession. As the host mentioned, if you have technical difficulties, you're going to use the chatfunction to speak with her. But the other functions here we will be introducing to you as we go.
We'll be doing some polls and quizzes. We will have a white board function where you'll be able
to write in some short answers. We've got some exercises. We'll be having some sharing fromone of your fellow Promise Zones. The San Diego Promise Zone is going to be sharing some of
their experiences. We'll have some open question-and-answer sessions and we'll have somediscussion.
Now, in order to do a number of those exercises, we are going to need to have you able to speakon the line. We have all been part of webinars where there's been the dog barking in the
background, the doorbell ringing, the hold music. We are really interested in not having any of
those things disrupt us today. So I'm going to ask you all right now if you would please use yourown mute function on your phone or computers to mute yourself.
In just a moment, I'm going to ask the host, Chantel, to unmute all of the lines. And what thatwill do is let everyone unmute themselves at will and break into the conversation. If, by any
chance, you accidentally come off mute and there is background noise, Chantel will mute you
manually. At that point, you will not be able to unmute yourself. So if you would like to be
unmuted, if Chantel has had to mute you manually – you should use that chat feature to let her
know your name and that you want to be unmuted so that she can take care of that for you.
So with that, please make sure you have muted yourselves. And Chantel, go ahead and unmute
the group so we can speak to each other when the time comes. All right.
So with that, I'd like to move on to a little bit of an overview of what you're going to behaving in
the webinar. And the first thing we're going to do is have a poll to figure out a little bit about
who is on the call. So we'd like to do a quick poll if you can bring that up for us, Chantel. What
you can do is look in the lower right-hand corner of your screen. You will see that there is a pollquestion there. It's asking you what your role is in the Promise Zones initiative. We'd like to get alittle sense of who's on the call.
So choose your Promise Zone role. And do make sure that you hit the submit button that's in thelower right-hand corner. If you just hit the radio button, your selection will not be registered. So
go ahead and let us know who's on the call. Great. Looks like the responses are rolling in here.
All right. I think that gets us pretty close.

~~~ Noble Transcription Services - 714.335.1645 ~~~Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Chantel, why don't you share those results so folks can see who's here. It looks like we havequite the predominance of Vista volunteers – 29 percent of you – a couple of researchers andevaluators, one program administrator, and some HUD staff. We do have quite a few more folks
who are registered for the webinar than who have actually joined. So we are hoping that we will
have additional people joining us over the course of the introductory material here. And we will
include them as they come on.
So I'm glad to see we have a number of different perspectives represented. I think that'll be really
important when we get to some of the conversation. So let's go, if we can, to the next slide. We'regoing to talk about learning outcomes. Essentially, what we are hoping is going to happen overthe course of these four webinars is that you are going to end up with a data and evaluation
framework, that each of the Promise Zones is going to have their framework and be ready to go.
I know that a number of you do, in fact, have either full or partial frameworks at this time. Andso for those of you who already have some work done on those, this'll be an opportunity toreflect and refine those. And for others of you who haven't really gotten a good start, this'll be
your chance to make that happen.
And we do hope that those of you who are further along in the process will be able to share someof experiences and learning with those who are just beginning. So what I'd like to do now isanother poll. In this case, we want to get a sense of where you are in your capacity to develop adata and evaluation framework.
So again, looking at lower right-hand corner, and please select the option that fits you best. Are
you in really great shape? Are you doing pretty well on your framework? Are you not too
confident about it? Or are you struggling a little bit with it? And we may have people at all
different levels and that's fine.
So go ahead and choose your favorite whether you're not started, whether you're rock stars whoare in great shape. All right. We're looking pretty good. Why don't we go ahead and close thatpoll.
So it looks like we've got quite a mix. We've got nobody who's really struggling. But we've got
quite a few who are not too confident and only one who feels like they're in really great shape.
So a little bit of a range and definitely folks who have some work to do on their frameworks,
which means you're in the right place. This'll be our chance to help you really get to the end ofthat process by the end of June.
So our next slide here shows you what the four-part series is going to consist of. This session
we're really going to be taking a look at working with your partners because that's such a crucialpart of how you as a Promise Zone will be forming your data and evaluation framework. And
we're going to talk a lot today about the theory of change and logic models, which really are thefoundation of what you're going to be doing with your framework.
Session 2 we're going to actually dig into that concept of data and evaluation framework. Exactly
what is this thing? What does it look like? What are the pieces? How are we going to get there?
And we will really look at the different kinds of evaluation that you may want to plug into your~~~ Noble Transcription Services - 714.335.1645 ~~~2

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

framework. Session 3 is going to look at data. We're going to look at the data sources, kinds ofdata you may be getting, and really talk about some systems for collecting and maintaining thedata that you need to collect.
And finally, in session 4, we're going to talk about some best practices for analyzing data. We'regoing to look at data visualization. And we're going to think about how you can use that to drive
decision-making and to really engage new partners and inform people about what you're doingwith your Promise Zone. So that's the lay of the land for the four sessions. It is certainly the casethat many of these topics are interrelated and we may get to a topic where you have a questionthat we're going to be covering in a later session. If that's the case, feel free to ask the question. If
we know we're going to be tackling it later, we may ask you to hold the question and we'll put it
in a parking lot until we get to that later session.
In addition to the four webinars that you know about, we are going to be making sure that you
get the kind of support that you need. There are a couple other supports that HUD is making sureof being provided you to. The first is office hours: There's going to be one-on-one help available
to each Promise Zone as you complete your homework. So as you complete each session's
homework, it's going to be building toward that framework, and there will be evaluation experts
available to you to tailor your questions and to help you finish your pieces of work towardgetting your framework together.
In addition, there are going to be four peer-to-peer learning sessions. Those four sessions aregoing to be designed to connect Promise Zone staff so that you can learn from each other andhopefully create a network that will be able to support you as you go forward beyond the timingof these webinars and these peer learning sessions.
So you can get a look here at your TA provider team. I'm Janet Pershing, as I noted at the
beginning. I'm here with Xiaodong and Kasia. All three of us are with ICF and we are going to
be your webinar trainers. We will then have office hours and the peer-to-peer sessions, thenthose will be provided by Trish, Jennifer, and Manolya. They are all with AIR and they will
manage both of those pieces – the office hours and the peer-to-peer sessions.
We work for two different firms, but we are partnering here to make sure that you get the support
you need in all dimensions. So for webinar information, you're going to be getting that from thethree trainers you see at the top. And you will meet Trish, Jennifer, and Manolya as you get into
the office hours and the peer-to-peer sessions.
So let me just give you a couple of thoughts about the role we are hoping that you will play. Firstand foremost, as everyone knows who's participated in a webinar, you get more out of it if you
stay engaged and you don't multitask. So if you can convince yourself to turn off your e-mail andput your phone down and really focus on the content, you'll get more out of it, as you well know.
We also really hope that those of you on the call will actively share your experience and ask
questions. Really a lot of the value of this – while we have some great material to share with you– will come from you sharing your thoughts with others about your questions, the strategies
you've tried, the things that work for you, and the things you're still struggling with.
~~~ Noble Transcription Services - 714.335.1645 ~~~3

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

We also really want to make sure that you are focused on doing the homework. The homework isvery much tailored to getting you through this process of developing a framework. If you do thehomework in the chunks that we've broken it out into, none of the weeks will have a huge load ofwork. You'll be able to get it done. And at the end of the process, you can pull it all together intoa framework. And absolutely take advantage of those office hours to help you get through thehomework and get input at whatever stage of development you're in from an outside person who
can give you their expertise.
So with that, let me give you a minute if you'd like to unmute yourself, if you have any questions
that you want to ask at this point about what we're doing and how we're going. You may nothave any questions, but please feel free to unmute now if you do.
Charlene Walla: Hi. This is Charlene Walla [ph] and I'm from USC, the research evaluation
partner for the Los Angeles Promise Zones.
Janet Pershing: Hi.
Charlene Walla: Hi. I just wanted you all to know I can't see what you're doing but I'm listening
so I can't [inaudible] picture –Janet Pershing: Is that because you're not at a computer and cannot see the slides? Or is that
because your slide's not working for you?
Xiaodong Zhang: She might be under session info instead of PZ webinar.
Jess Yuen: Yeah. Hi, Janet. This is Jess Yuen down in San Diego. Kayleigh is also having
trouble logging in from San Diego, as am I. Kayleigh has a blank screen – a blank window.
[talking over each other]
Speaker 4: And Atlanta's having the same issue also. We can't see anything after you launch it.
Janet Pershing: All right.
Speaker 5: I'm experiencing the same thing.
Janet Pershing: Why don't we hang on for just a minute, let me consult with our technical teamhere and see what we can do.
Xiaodong Zhang: FYI, other people are having problems, the same issues. You can tell. Okay.
Just tell them that.
Speaker 6: It opens in a different program, so if people are still looking in the spot where theyclicked on the link, you won't be able to see anything. You have to look on the bottom to see if
there's another program open.
Jess Yuen: I just get a blank white box opening. It's a separate – but it just says, fail on the top inpart of the bar. I tried it on two different browsers and two different computers so –~~~ Noble Transcription Services - 714.335.1645 ~~~4

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Speaker 4: I've done the same thing and just getting the same result.
Xiaodong Zhang: I'm one of the presenters. We're starting from slide number 13. You allreceived a glossary in advance and we asked you to look it over to see if you have any questions,especially the terms with asterisks. We want to take a minute to check on how well you did inabsorbing the information. Chantel, if you can bring up the quiz. And we have two true-and-false
quiz questions.
The first one is data visualization is a term referring for very specific sets of display that can beused to analyze data that require a specialized software to generate – true or false? The answer iscoming in. People are still working on it. Draw a picture of the data. Yeah. Let's close the poll. It
looks like a lot of people are still undecided. This is a little bit tricky. The correct answer is false
because data visualization, it doesn't really have a finite number of display. And more
importantly, although it is true that some would require specialized software, it is not always thecase.
Second one. Chantel, if you can bring up the poll, the quiz. It reads: The important distinction
between primary and secondary data is timing. Primary data are data collected prior to the startof an evaluation while secondary data are data collected after the evaluation – true or false?
Speaker 5: False. [Inaudible] data are data collected directly from whatever study.
Janet Pershing: This is Janet. While you're thinking, I will just jump in to say we've had several
people kind of muttering under their breath in the background clearly not on mute. Chantel ismuting you, so if you have a little trouble being heard later, make sure to use the chat function to
ask her to unmute you. In the meantime, please everybody, make sure you are muted.
Xiaodong Zhang: Okay. Let's close the poll. This time a lot more people ventured out. Theygave their results. And of all the ones that we've received, they all answered correctly. It's false.
The important distinction between primary and secondary data is not when – it is the data source,
specifically, who's collecting data. The primary data are usually collected from the evaluator or
the program people where the secondary data are data collected from somebody else andprovided to the evaluator and to the team.
Let's move on to the next exercise. We're going to talk about the glossary terms for a few
minutes. And Chantel, if you could bring up the white board function, and for those people who
can access the WebEx, I would encourage you to try it. Please type in any confusing terms thatyou wanted to be sure to clarify by writing them down on the white board. And do that byselecting – if you can see on the upper left corner there's a T on the toolbar. If you click on that T
and then click on the white board, you can type in your response.
Janet Pershing: And I will just add for those of you who don't have access to WebEx at thistime, if you would like to just say out loud any terms that you'd like to have explored, please feel
free to do that and we can jot them down here.
Xiaodong Zhang: Yeah. And if people are writing down their answers, I just want to clarify how
this glossary can be used. You can notice that the one that is marked with asterisks are for the~~~ Noble Transcription Services - 714.335.1645 ~~~5Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

most part representative of terms that we either define or they are directly related to the webinar.
The other terms are related to evaluation and they have specific meaning that you might comeacross while you are looking at the references or start digging more into the world of evaluation.
So the key point here is that we would expect you to understand those terms with asterisks at theend of the webinar four sessions. And the rest of them will be used for your future references.
Janet Pershing: Looks like we had a couple of people experimenting with the white board but
nobody having any questions about any of the definitions. One last chance for any verbal
statements about those if anybody wants to chime in. Great, if they're all really clear.
Okay. We have someone who doesn't have a glossary. So two ways that you can access theglossary for those who registered prior to yesterday I believe. It should have been attached toyour – there should have been an e-mail explaining that you should review it in advance, and it
should be attached there. If not, we will give you a URL at the end of this webinar that will takeyou to a link where you can actually access the glossary.
I think rather than trying to spell out a URL here when not everybody has got access to be able tosee things on the screen, I'm not going to do that right now. We will get that to you afterwards
and you can take a quick look after the session is over. You can definitely let us know of any
terms you'd like to have explained further after the session and we can tackle them next week.
Xiaodong Zhang: Now we have some results. Thanks for those responses. And we won't be able
to review them right now, but we will clarify them as we are going through the session. For
example, we're going to talk about logic model today and we're going to talk about Group C
indicators in more detail in session 3 and descriptive statistic in more detail is session 4. So wewill keep those in mind as we go through those related content.
Chantel, if we can move to the next slide. This painting was specifically designed for the PZ in
the placed-based environment. Here is a list of place-based initiatives that you might be familiarwith. And as you can see, many of them are from HUD, but there are agencies involved like theDepartment of Education, Department of Justice, and SAMHSA.
To help prepare for the webinar, we also conducted a quick application review just to learn about
what the PZs are doing. The main finding is they're quite diverse. I just want to highlight a few
findings. So for example, most PZs address crime, education, economic development, housing;and some tackled health, transportation, and infrastructure issues.
The lead agency tends to be mayor's office or city office or city department, but it could also
include some of the nonprofits. Other partners may include implementation partners, support
partners. But for some other PZs, they are more organized by subzones or action teams. At thetime of the applications, very few PZs have data and evaluation framework developed. Althoughsome have provided samples of their data sharing, data usage agreements and some have hired
third-party evaluators. Next slide.
The cornerstone of the place-based initiative is this idea of collective impact. Now, here's a quick
review of the collective impact theory with key elements of faces in a collaborative chain. I'm
not going to read them all, but I just want to piece them out for your reference. Next slide.
~~~ Noble Transcription Services - 714.335.1645 ~~~6Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Because PZ is such a complex endeavor involving so many partner organizations, it is paramountto engage these partners effectively. Here are some principles to help engage your partners. We
will be coming back to this over and over in the webinar, because you can't really do it alone.
You really need your partners. For today we're going to focus only on the first bullet. And we'll
circle back to the others in later webinars. Back to Janet.
Janet Pershing: Great. Thanks. So next up, is doing a little bit of sharing about engaging partners
and stakeholders. If you're going to be doing collective action, one of the things you really haveto be doing is engaging the other folks who have to be involved for you to succeed and to do thatwell. And what we'd like to do is to invite Kayleigh to help share some of her experiences aroundthis in San Diego. Kayleigh, I know you were having some difficulty accessing the visual. I hopethat you are able to unmute yourself now and speak.
Kayleigh Creswell: Hi. Yeah. I'm here. Can you hear me?
Janet Pershing: Absolutely. Thank you so much and go right ahead.
Kayleigh Creswell: No problem. Hi. I'm Kayleigh Creswell. I'm the data associate AmeriCorps
Vista for the San Diego Promise Zone. So San Diego is a third-round Promise Zone, but I'm sureas many of you can attest to, Vista and Promise Zone is a steep learning curve. So I'm happy toshare with you the challenges that we've faced so far in San Diego.
So just to clarify some of our language that we use in San Diego, we have six goal areas. Those
goal areas meet every month in working groups. And those working groups are usually chairedby two co-chairs. And we also have two research consultants that help us with our work. So oneof the first lessons that we learned around engaging our stakeholders is that we really had toshow value.
So we had to have a clear value add – or we tried to have a clear value add at every meeting that
we're inviting our partners and stakeholders to. We also try not to play favorites, which is easy to
do if you have a large partner that seems like they can do everything very quickly. I think it's
important not to overlook smaller partners that might have that big expertise or actual resourcesthat you need. So San Diego does best personally by, of course, our preference points our hugevalue add.
We also do grant alignment meetings, so the Vistas look up grants that we think our partners
might be competitive to and invite them around the table to see how they might work together toapply for that grant. We also do activity alignment in our monthly meetings, of course, data andknowledge sharing, which is what these classes are all about. That's a huge value add especiallyto our smaller partners that might not have that capacity at their organization. And also
reminding our partners that Promise Zone is a network, we also hold all-partner meetings, which
are open to the public and, of course, all of our partners and that's once every four months.
So that's more doable for all our smaller partners who maybe can attend a long monthly meeting
every month. And it's an opportunity to get everyone connecting and back together. In the future,
we're looking to meet around funding opportunities more often to have that as a value add with a~~~ Noble Transcription Services - 714.335.1645 ~~~7

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

caution that we don't have any mission drift. So we're not going after funding just because it's
there but because our partners are able to and it aligns with our mission.
Another lesson that we've learned around engaging stakeholders has been to clarify their roles.
So we've learned that unclear roles can lead to apathy at best and messy misunderstandings at
worse. So you'll have those that are afraid to step on toes so they won't get involved. And thenyou'll have those all too willing to step on toes. So it's important that we remind our partners andour stakeholders that the Promise Zone team is really the backbone and that they're the drivers of
the work.
So in San Diego, we've done that through an MOU, which is a memorandum of understanding
and that's signed by all of our organizations and our partners. And that really clarifies our roles
both on the side of partners and the Promise Zone, makes everything clear. We also have writtenroles and responsibilities for the co-chairs that just says, the consultants, etc. we don't have thosesigned but they are very clear and laid out and written down.
And finally, as far as clarifying roles, as we've moved from planning to implementation, ourconsultants used to have a lot of support. But our co-chairs have recently taken over agenda
writing, meeting facilitation with the best of the support. And this support often looks likereminder e-mails, sending past documents to use as templates, reminding co-chairs of [??]
activities and note taking.
There's a few things that might be overlooked or silly, but we found them very useful as far as
engaging stakeholders and that is appreciation and patience. So it's people who engage withpositivity. So that might be personal e-mails thanking them for data or input during meeting. The
personal invitation meetings are always talking about who's the best to bring around the table.
And in the future, we're really looking to spotlight their work in newsletters or blogs as we
increase our capacity as a team.
Circling back to that patience I mentioned, I think it's really easy to get frustrated as partners or
stakeholders don't get back to you right away. But you have to remember that especially talking
as a Vista perspective, we spend 40 hours a week or maybe more working on this. And ourpartners and stakeholders really vary in the amount of time that they can dedicate. And no one
wants to contribute to something that makes them feel guilty. So we try to be encouraging,honoring our stakeholder's time by being on time to meetings, earning that trust by notrescheduling the meetings, and making their job easier by offering drafts, bringing printouts, andthen bringing info to those who couldn't attend in the first place.
I think one of our most important challenges that we've actually faced recently is balancing
consistency with novelty. So personally, we have six meetings a month every month. Our
meetings are at the same time of the month every month in the same room. If any changes are
made, it's across the board on all of our forms of communication. We've had challenges with that
before where partners say, oh, I showed up to this meeting. I didn't know you changed it. I'm nothappy with this. And we have to apologize.
That's not an ideal situation. So we've learned that lesson. At the same time, that can get dull
doing the same thing over and over again. It's been great because people show up every time.
~~~ Noble Transcription Services - 714.335.1645 ~~~8Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

They know what's expected of them. It's easy for them to plan in their schedule. But as we moveinto implementation, we're trying to create some more variety in our meetings, so doing breakout groups. We are planning up some of our first joint meetings across our working groups. And
doing worksheets and activities within those meetings really gets the energy level up.
And then one of the most important things has only happened once in the San Diego Promise
Zone as far as the events that we have planned, but nonbusiness events – so getting to know yourstakeholders and partners as people and not just seeing them when you're trying to get down tobusiness. So in the month of last December, we canceled all of our meetings – so all of our six
meetings, or co-chair meetings – and had a holiday party instead.
And that was a real paradigm shift for everyone. These partners that had seemed stern at first
were actually really friendly in this kind of more relaxed venue. Actually, one of our co-chairs
offered for one of our Vistas to spend Christmas with them because she had no Christmas plans
and no family in town. And since then, it's really served to deepen trust between the partners andthe Vista team, the Promise Zone team, between the partners. And I feel like it's really served toaccelerate that work.
We also work to get out to our partners' events, like their galas and fundraisers, their graduations
if they're in the education sector. In the future, we're looking to build that one-on-one
relationship between the Vista and the co-chair to increase that bond further. So those are someof the lessons that we've learned so far. So I will open it back up to the group. And I'm interested
in who's had similar challenges and maybe how you guys have faced them or challenges that I
haven't mentioned or any questions about what I mentioned. Thank you.
Janet Pershing: Please feel free to unmute yourselves and respond. If you're not able to getyourself off mute, make sure that you chat with Chantel. She may have muted you if you had
some background noise. Anybody have thoughts or reactions for Kayleigh? I'm curious about
whether others have done some of the kinds of interesting and unique things that San Diego did –the holiday party as an example. Any other Promise Zones have some interesting techniques thatyou've tried or that you're thinking of trying to engage your stakeholders?
Katherine: Hello?
Janet Pershing: Yeah. Please go ahead.
Katherine: Hi. This is Katherine from the North Promise Zone [ph]. My question was in regard
to the co-chairs if Kayleigh could answer. What was the strategy to best identify two co-chairsthat had time and availability to be the co-chairs for the working group?
Kayleigh Creswell: Hi. Yeah. That's a great question. As far as I understand, they were involved
before I arrived. But it was mostly on a volunteer basis. So a lot of it was getting the region's
nonprofits and other stakeholders excited about what the Promise Zone had to offer. And thenthe people who are excited by collective impact really came to the forefront and became thedrivers and then our co-chairs.

~~~ Noble Transcription Services - 714.335.1645 ~~~9Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Jess Yuen: Yeah, Kayleigh. This is Jess Yuen. I am the HUD community liaison for the San
Diego Promise Zone. Kayleigh is exactly correct here. As the working groups were being stoodup, there was sort of a really intense look at who were the partners who had been involved
especially through the two application processes that the City of San Diego had led? Who were
our implementing partners?
And then in most cases, going to leadership at those organizations and saying, we still want you
guys to participate. We want you to help lead this work. Will you participate in this? And either
having them participate or them designate somebody from their organizations to participate, it
was very much on a – looking at the people who were doing the work and who aligns with thework of that particular working group and having them lead.
We are going through some transitions. We asked in the beginning for co-chairs to commit to ayear. And with that commitment, they were to work with our consultants and the businesses who
were being on-boarded at the time in order to do a strategic plan. We have now recently goneback to those co-chairs and said, as we're moving into implementation we want to make sure that
this is the work that you still want to lead.
We may have some shifts in co-chairs coming up. We had, for example, one co-chair who was
working on both access to health care and healthy foods and also our safety working groups. And
she has come and said, safety and informed work is really where my heart is and my organizationis, so I will stay on safety and I will help you identify somebody to take my spot on the healthworking group.
So we are seeing some shifting of the co-chairs, but I think that's normal and expected. But thoseco-chairs really have been the drivers of the work. They are the ones especially the co-chairs of
our education working group and economic development and creating jobs working groups.
Those co-chairs are engaged to the point where they are the ones reaching out to their colleagues
at other organizations and inviting them and asking them to attend.
Dan Queue: Hi. This is Dan Queue [ph] from the Philadelphia Promise Zone. I have a follow-up
question to that. I know in our logic model in our application we identify a public and private
partner and I'm wondering if that held true for the San Diego Promise Zone as well.
Kayleigh Creswell: Yeah. So now especially that we're going into implementation, we are reallylooking at especially industry partners. So a lot of our first steps into implementation arefocusing on disconnected youth, so those youth that aren't in school or in a job. So it reallybrought our jobs and education group together. But we recognize the importance of hiringsectors at the table telling us what they need and what they want as far as training these young
individuals.
So we've actually found that we've really formed an organic network. We already have a few
community colleges around the table and those community colleges already have those strong
connections with industry partners that we're looking to reach out to. So I'm not sure if they werelisted specifically. We definitely had educational institutions and health institutions as well as
nonprofit partners. And along the way as we logic modeled, we did come up with large
industries. But now that we're getting down to the nitty-gritty, it's definitely as inviting specific~~~ Noble Transcription Services - 714.335.1645 ~~~10

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

names around the table. It's really become an organic process of inviting through people who are
already around the table.
Janet Pershing: I'm going to jump in real quick. We're at the end of our time for this segment but
it's been really informative. So I'm going to get one more question on the table before we wrap
this up. Kayleigh, we know that some of the Promise Zones had some trouble reaching out toeducational partners especially in high schools because of personally identifiable information
issues. So the question is: How has relationship building improved your access or do you still
have trouble communicating with those partners – those education partners in particular?
Kayleigh Creswell: Yeah. Of course. I think it goes back to using the network that you alreadyhave. So for us personally, our school district was very involved, maybe not the schools
themselves. So it was about asking those partners for a warm hand-off to who else we would liketo get data from. So for example, we were having a person from administration come from theschool district and he was very involved.
And when it came time to gathering the data about after-school programs that the school district
was connected with, it was about going to him and asking him for that warm connection, because
I have tried to reach out specifically for data, just cold reaching out. And you are faced with that,what do you want? Are you trying to sell us something? Are you writing a story for a paper that's
going to make us look bad? So it's much easier if you can find someone who's first- or secondand even third- or fourth-degree connected to who you want to reach out to.
Janet Pershing: Great. Thanks.
Ms. Shanike: I'm – I have a statement on this. So I'm Shanike [ph] from the Philadelphia
Promise Zone and I support the education committee. And here we actually have someone fromthe school district sit on as a co-chair for the education committee and to engage principals who
actually hold meetings at their schools. And we invite principals to speak about their school inthe meetings when our education meeting is hosted at their school. And we've actually been
recently seeing an increase in engagement by principals hosting meets at their school, having
connections within their staffs.
Janet Pershing: Great. Fantastic. Thanks for sharing that. Sounds like that there may be a longer
conversation that could benefit the group if you all get together after some of your peer-to-peer
sharing to talk about education in particular. I'm going to need to stop our conversation there, I'mafraid, to get through the rest of the material. But thank you very much, Kayleigh. And we'll beback to you later in the session. Let me at this point turn it over to Kasia, if I may.
Kasia Razynska: Thanks, Janet. For those of you following along with our Power Point, we are
now around slide 20. And in this section of our presentation we will be exploring what a theoryof change and what a logic model is. I will start the conversation by talking through a theory ofchange and then I will pass it over to Xiaodong, who will explore the logic model.
Before we get into this whole section, however, we would like to take another quick poll to get asense of how comfortable you are with your understanding of theory of change and logic models.
You are amongst friends, so we please ask you to be honest. And let us know the poll is now~~~ Noble Transcription Services - 714.335.1645 ~~~11

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

open. It seems like we have a range of responses with some folks saying that they only have avague understanding all the way to folks who feel extremely comfortable with theories of change
and logic models. However, just because you know what a theory of change is doesn't mean thatyou necessarily like them. So we will hopefully by the end of this session convince you they arevery useful tools that can really help you engage your partners.
So what exactly am I talking about when I say a theory of change? In plain language, a theory ofchange or a TOC is a way to check your logic and see why you think what you're proposing will
work. You identify a problem and then you develop a hypothesis to solve this problem. A theoryof change keeps you focused on strategies that deal with the problem you want to address. It is agood way to start – and a good way of starting a logic model is to ask yourself – in a theory ofchange is to ask yourself who, what, how, and why.
You start by identifying your target population or who you need to help. You then identify theoutcomes that you are trying to reach for this population – the what you are trying to do. And
finally, you use a process called backwards mapping. Once you have identified those outcomes,
you start thinking about the strategies that you can implement to try to reach the outcomes thatyou've proposed. The benefit of doing it this way is that you ensure that these strategies thatyou're proposing will ultimately lead you to the outcomes that you proposed.
A lot of times when you're developing a theory of action you might think, here's my populationand here are some strategies that I think will help me. And at the end you realize, wait. Thoseweren't the outcomes that I wanted. So by doing this process of backwards mapping and thinking
first about the outcomes that you want for this population, it helps you focus those strategies tospecifically hit the outcomes that you want to focus.
And the last thing in our framework is the why. If you look at that picture on the slide, the why
are the greens arrows around the outcome strategies and population. The why is the reason for
how you'll get to the what. It's the process through which you will move from your strategies toyour eventual outcomes.
Now that we have a good foundation on what a theory of change is, why are we proposing thatyou start your process of building an evaluation framework with a theory of change? Sometimes
practitioners find the idea of a theory of change a little academic. However, I want to convince
you that it really does have a lot of value. A theory of change helps you articulate explicitly what
you are thinking about and what you are assuming. It also helps to bring you and your partners
all to the table and to articulate and share those assumptions that you might have.
Some of those might be different for different groups. And so by developing a theory of change,
you can make sure that everybody is on the same page as to what their assumptions are about the
problem that you're trying to solve. And the final benefit of having a good theory of change isthat once it's agreed upon, it makes it clear to everybody exactly what you want to measure inyour evaluation framework and allows you to do so both effectively and efficiently.
So now that we have discussed what a theory of change is why it's beneficial to use a theory ofchange, let's start thinking about how we would actually create this theory of change. And it canbe extremely overwhelming to just to start with nowhere. So what we want to propose to you is a~~~ Noble Transcription Services - 714.335.1645 ~~~12

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

technique of writing a theory of change that will contain all the needed pieces. And we arecalling this technique Pit-B. That's easy to remember if you think of a pit bull. So what is thisPit-B technique that I would like to [inaudible] for you?
Well, you start by identifying your problem. Not only do you identify the problem, but you also
list all the assumptions that you are making that go along with that problem. Once you've
identified the problem and all the assumptions, you then go through developing an if/thenstatement. And when you create this if/then statement, this is where you create your workable
hypothesis about the strategies – which are the if's – and then the outcomes – the then's – of what
will happen as a result of the strategies that you're implementing.
The final step of this process is creating a very good because-statement. This is the causal linkthat explains the logic to you why you think that the hypothesis will eventually lead you to theoutcomes that you're expecting to reach. Let's look at an example.
Suppose that we have a problem and our problem is that students do not read at grade level. We
believe that one of our assumptions is that the reason students don't read at grade level is thatthey are not receiving enough individual attention in the classroom. There might be otherassumptions, but this is the assumption that we are – there might be other reasons why they're
not reading at grade level, but this is the assumption we are going to for simplicity [?] try to deal
with in this problem.
So an example of an if/then statement might be: If we provide 30 minutes of one-on-one tutoring
and then after school, then the reading level of the students with improve. In order to check that
hypothesis, we then develop that because-statement – why do we believe this hypothesis is true?
This is our final check of logic.
So by providing individualized one-on-one tutoring to the students, we are able to give them oneon-one attention that they're not receiving in the classroom. More individualized attention helps
them improve their reading because the tutors are able to tailor their teaching to the specificindividual student needs. This logic makes sense. And therefore, ultimately we can say that as aresult of our strategies, we can reach our outcomes and students' reading level will improve.
So now that I've walked you through this example, we want to put you to work again. We will
have a short exercise in which we want you to be able to identify these critical components of the
Pit-B model. You will see for those of you who can see our screen – you'll see a potential
scenario come up. I'll give you a minute to familiarize yourself with this text. As you're reading
this, I would like to bring your attention to statements that are bolded. The statements that arebolded are the beginnings of either a problem – an if-, a then-, or a because-statement.
As you read them, keep this model in mind and you'll see a poll come up eventually in which wewill ask you to identify both the problem-statement and the because-statement. I'll give you acouple of minutes to look at it and read the scenario. And for those of you who are following
along with the Power Point, we are now on slide 25. So I'm going to give you one more minute.
And you will now see a poll come up. What we want you to do is we want you to identify whichof the words – the bolded words identify the problem-statement first.
~~~ Noble Transcription Services - 714.335.1645 ~~~13

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

We're still having some folks write in. Okay. We're going to close the poll now. It looks likeeverybody who responded identified the correct statement, which is great news. And I think it
was a very easy, because usually the first thing you do is you identify the problem. So obviously
those of you who got it right, the first sentence is the right answer. Now we're going to ask you
to identify the because-statement or the logic. Again, what we want you to do is we want you to
hit the statement that starts the because-statement.
So it looks like this question was a little bit more difficult. For those of you who answered at D,congratulations. You got the right answer. But because there were several votes for different
statements, let's walk through this problem piece-by-piece. So what you see before you issections of the problem if/then and because-statements highlighted in the color corresponding to
the model. So what you see in red, starting with adolescents in our city, that is the problem we'reidentifying. Not only are we identifying the problem, we are also identifying those assumptions
about our problem.
And here you see we believe adolescents did not know how to make good food choices and soon. That is where we're identifying our assumptions. Following this we have our if/thenstatement. The if represents the strategies that we are going to be using and the then represents
the outcome we're hoping to reach. So if adolescents learn about healthy shopping and cooking,then those students will become healthier. The outcome we're trying to reach is students arehealthier. The strategy we're going to use is we're going to help them learn about shopping andcooking.
And that final piece, the piece that folks have a little bit of a hard time identifying – the giveaway here is that it starts with the word, because. And this is where we're testing our logic. Why
do we believe that introducing a strategy is going to help us reach our ultimate outcomes? Webelieve this because – we believe that once students learn about how to choose better food
options and develop positive – they will develop positive attitudes towards eating.
I just wanted to draw your attention to the very last statement that ultimately this will – thestatement that starts with, ultimately this will reduce adolescent obesity and related ailments, thatstatement is a final step in which we check – if our logic is correct, do we really solve theproblem? And here I believe that we do just through the theory of change.
It's easy to develop the theory of change. It doesn't necessarily 100 percent mean that the
program will work. However, one thing that I also wanted to highlight is that a theory of change
is just hypothesis. Even if your particular program is grounded in evidence-based practices thatwere implemented somewhere else does not necessarily mean that the same program will be
successful in your particular setting.
This is why it is very important at the very beginning list all the assumptions that are particular toyour specific case, which will be slightly different. And a part of the – or the purpose of the data
evaluation framework is to test whether your hypotheses will hold up in your particular setting.
The last thing I wanted to mention before I pass it on is I wanted to talk about some common
pitfalls that we have seen in theories of change that we have seen in our experience.

~~~ Noble Transcription Services - 714.335.1645 ~~~14

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

One common pitfall is creating a circular logic and basically stating, "My hypothesis will work
because my hypothesis will work" – basically restating the hypothesis in different words. For
example, let's say we're trying to implement a drop-out prevention program saying simply, we
will introduce a drop-out prevention program and as a result of the drop-out prevention program,we will help the problem of drop-outs. That introduces a lot of circular logic and doesn't reallyexplain in clarity what the because is in this situation.
Another common pitfall is having your theory of change be too focused on the process. Rather
than really thinking about the link between strategies and outcomes, these theories of change
tend to focus way more on exactly the details between the strategies that you're trying to propose.
Another pitfall is including too much history of the program in your theory of change.
Remember, what you're trying to produce is a statement that explains your logic, not give a
history lesson of why this is the program that you chose.
And finally – and this goes back to a point I made previously – a lot of folks try to in the
because-statements provide too much information that reads like a literature review to show thatthe program that your implementing is evidence-based, that it's been successful in previous
iterations, and just providing too much of that literature review context. This is supposed to be ashort statement that explains why this particular program is going to work in your context rather
than providing an overview of all literature available in this program. And with that said, I'm
now going to pass it over to Janet.
Janet Pershing: For those of you who are following along in the slide there, go ahead and moveto slide 30. We hope to have a little bit of discussion right here, but I think we're going to skip
that for the time being and do some Q&A and discussion about this at the end if we have a little
time. I just want to make sure we don't cut ourselves short on some of the other content.
So when you look at slide 30, one of the key takeaways that I hope we will have some time to
discuss later is making sure that you figure out how you're actually going to use your theory ofchange. It's as you know, good to articulate a beautiful theory if you don't actually use it to guide
your work. And so in the webinars that are coming up, we are going to be talking about a variety
of ways to apply that theory of change in your work and to the various elements of yourframework. And in fact, I think we'll hear a little bit more about that from Kayleigh when we
start thinking about her logic model in a little while.
So with that, let me ask Xiaodong if he can go ahead and talk to us about the logic model and we
can maybe discuss theory of change and logic model together when you're done.
Xiaodong Zhang: Sounds good, Janet. A related topic to theory of change is this idea of logic
model. Some people see these as two different tools with the theory of change focusing on thewhy-question and the logic model focusing on the how-question. But I really see them as thesame thing in that the logic model is simply a visual representation of theory of change.
The why logic model: The key value of having a logic model is really to help us unpack thecomplexity and to clearly articulate a program theory. The process of developing a logic model isalso a process to establish common understanding among partners. It also provides anopportunity to uncover holes in the program theory. Let me give you an example using a healthy~~~ Noble Transcription Services - 714.335.1645 ~~~15

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

food program as an example. And we will use the same example throughout the discussion oflogic model.
Let's say the program goal is to improve children's health. In conceptualizing what needs to bedone, different folks came up with a long list of activity. One of them was to provide training forteacher well-being. Upon closer examination, partners concluded that this activity won't directly
contribute to the goal and, hence, they decided to drop this particular activity.
Conversely, if a program outcome is to improve children's knowledge about healthy food and yet
the program does not have any specific activity that is explicitly helping children to attain theknowledge, then there's a problem there. So these are two examples of how theory of change canhelp us uncover the holes in the program.
We are on slide 33. Here's an example of a template for a logic model. In the next few slides,
we're going to give a step-by-step illustration of what each component of each element means. I
just want to emphasis that this is our template and our ways of doing logic model. Others may
have had their own template or slightly different ways of defining and labeling things, which is
totally fine. What is important is that we have to go through a systematic way, a systematic
thought process in developing our program theory, a logic model.
So here's a quick overview of the key component of the logic model. If you take a look at theslide, on the left-hand side we start with inputs. And this is kind of a tabulation of the resources.
The next element is called activity. The third column is labeled as output. And the final sections
are outcomes defined by short-term, medium-term, and long-term. Let's not forget about thearrows that show the connection of all these different components and different elements.
I'm going to talk about each one of them in more detail in the next few slides. And the firstelement is called input. The input may include financial, human, organizational, and community
resources. This is especially important to tabulate these resources here because the PZs have lotsof pieces that have to be coordinated. Next slide.
The second element of a logic model is called activities. Sometimes it's also known as strategies.
They are the tools and actions that we hypothesize that will bring about intended change. In thisexample, we provide a few sub-bullet and sample activity for our healthy food example. And
these activities may include: providing workshops, providing counseling, and providing referrals
to food programs and resources.
The third element is known as output. Outputs are direct tabulation of activities. People often
compare output to widgets or bean counting. In our healthy food example, the output may
include number of trainings provided. But they don't really tell us whether the kids are learning
or not. So going back to the point, output what we are doing and not what happens because of it.
The fourth element is called outcomes. So the name refers to the expected change resulting fromthe program. These are our ultimate goals. If you don't see the outcomes materialize after the
program is completed, either or both of these two things are happening: First, the theory might beflawed and/or second, the program is not implemented as designed. Since the PZs are 10-year
initiatives, we offer some guidance on how to define short-, medium-, and long-term.
~~~ Noble Transcription Services - 714.335.1645 ~~~16

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

So for example, we arbitrarily say anything happens within the first three years are short-term,
between three years and five years are medium, and after five years are long-term. But PZs may
have their own criteria, which again, is totally fine. The bottom line is for long-term investment
strategy like Promise Zones. You don't have to wait for 10 years to see what happened. You
really want to be on top of evaluating the activity or whether the short-term outcomes arematerializing so you can make mid-course correction. Next slide.
And now we're on slide 38. In doing the logic model, we often find that people are confused
about the difference between output and outcomes, especially since both elements can be
expressed numerically or can be quantified. Here's a recap of the key differences, both in termsof definition or in terms of examples. I'm not going to read them, but the key takeaway is thatjust listing the output is not enough. For example, we cannot just give out 40 books without
trying to determine if this activity improves student learning or not. Next slide.
Now that we have defined different elements of a generic logic model, let's try to put it together
by going through our healthy food example and clarify what they mean as it related to thehealthy food. So for example, in terms of input, the input could include grants, money, staff
capability, and the curriculum that they use to educate children about healthy food. The activity
may include delivering the curriculum or delivering the training program. The output could
include the number of workshops provided, number of people or children that have been trained.
The short-term outcome may include the improved knowledge, skills, and attitudes as a result ofparticipating in the training. The medium-outcome could be reflected in improved healthy foodchoice and healthy behavior. And the long-term in this case would be the improved health in thecommunity. So this is just a very quick example of how we put it together. Next slide.
In the previous slide, the logic model that we have shown were developed from left to rightstarting with the input and thinking through the activity, resulting output, and ultimate outcomes.
Another way to do it – which Kasia illustrated and she was a strong proponent of that approach –is called backward mapping that's displayed in this graph. In this case, we start from point B topoint A, which is the long-term outcome and try to back into what needs to be done to get there
from point A.
As Kasia mentioned, there are several benefits of doing this backward mapping exercise. So for
example, we can dream big by envisioning an end point. Secondly, it also helps you focus onwhat input that will get you what you want and not necessarily be constrained with what you
have right now. But again, both ways will work fine.
For Promise Zones – now we are at slide 41 – this might not be as tidy. So for example, you may
find yourself in a situation where the partner has already several certain activities being
implemented when everything gets started. In that case, so they are at point A – so in that case,you really need to think about what outcomes those activities are likely to lead to and how thatwould support your desired long-term outcome in point B. And then you really need to think
about what may be missing according to your theory of change, your logic model, and go backand think about what supplementary input and activity might be needed to help you get there. So
we go from point B back to point C.
~~~ Noble Transcription Services - 714.335.1645 ~~~17

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

You may also find that some of these outcomes would require collaboration of activity between
different partners and between different sector to achieve. So for example, in the case of
reducing poverty, it's not just a mission of, let's say, Department of Economic Development. It
would really require a different partner from, let's say, education, human services, to bring about
the intended change. Next slide and we're on slide 42.
So this is an example of a logic model from a choice neighborhood program, which is also aplace-based initiative from HUD. It's actually a logic model, but in order to fit the logic model in
one slide, we kind of simplified the logic model by focusing on only two sectors. The first sector,
the first row is about housing and the second one is about community development. I really want
you to take a closer look and see how different activities leads to outcome and how they drive
different output and how the different output would drive different outcomes eventually leading
us to the long-term outcome, which is to transform our neighborhood.
In the next slide I'm going to show how this logic model is connected to the theory of changeusing our Pit-B model. So we kind of have it nicely color-labeled. Essentially, all the activities
here are all if-statements – the activities including demolishing, rehabilitating, or constructing
housing, or a different kind of a community improvement activity – these are our if-. If these
activities are taking place, then we will see our long-term outcome, which is the fact that thewhole neighborhood has been transformed. And everything happening in between from theoutput, the short-term, the medium-outcome – these are our because-statements. These are what
it takes from the if- to our ultimate long-term outcome.
I just want to mention that the input section is not in here even though they are kind of equallyimportant. The next slide 44, I just want to point out in the resource section – which Janet isgoing to talk about at the end of the webinar – you will find a link to another choiceneighborhood logic model. We kind of present – I have a screenshot on this slide. And you can
see this logic model is formatted very differently. But it essentially includes the same
information that we've been discussing.
So for example, if you'll take a look and try to find where is the problem? Where's the problem
section? The problem is really the need section in this logic model. And the ifs are labeled as
strategy. Then is our goal. And because is our outcome/metric section. I want to show you thenhere not only because it is a concrete logic model used by a place-based initiative, but just toillustrate the fact that the logic model can be done differently as long as we go through the same
thought process.
I want to wrap up by – I want to conclude by giving you some tips to creating a logic model. So
for example, the first one is, it's absolutely okay to borrow and adapt from a similar program thatyou're implementing. And if you are creating a logic model, make sure that you focus on the PitB way of thinking.
And the third one is, when we create logic model especially for a place-based initiative with somany different partners, it's probably advisable to start small. And that different partner may
create their own logic model and then you can all sit together and see to what extent collectively
those logic models lead to a logic model at the PZ level and to what extent some of the activity
needs to be connected so that we don't work in silo. Finally, and the most important point is that~~~ Noble Transcription Services - 714.335.1645 ~~~18

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

the logic model is a living document. And don't be afraid to adjust it as you learn through theimplementation. Back to Janet.
Janet Pershing: Thank you. So we want to do some question and answer here. But I think what
we're going to do – because I don't want to short change Kayleigh's time – is go to her first andthen we will circle back to questions you may have. And I know we've gotten some actually
through the system. We'll circle back to those after we've heard from Kayleigh. So Kayleigh, if
we could, I would love to go ahead and turn this over to you. Talk a little bit about some of thechallenges you've faced in dealing with theory of change and logic models and making sure thatthey stay active and not duck be on the shelf.
Kayleigh Creswell: Yeah. Absolutely. So I think one of the first things we tried to do while wewere planning logic modeling and getting partners around the table is making sure there was
early investment, so making sure everyone was contributing and doing so early on, reallyensuring that they felt ownership of what they were contributing to. So we spent about 10
months logic modeling and gaining partners through that whole process. And that's going to vary
deeply. But that was a very cognitive decision on the Promise Zone team then to be strategic andcareful so we could have this document that we could refer back to again and again and really be
our anchor.
We also spent two entire meetings picking our indicators and/or sources of data. And again, thatwas driven by our partners. We wanted to make sure that they felt like they were in charge andthey had ownership of this process. So you really see how we tried to weave data and indicators
into our planning and logic modeling. So those happened within that 10 months. Another way wepromoted early investment was, as we mentioned, those inputs. While they can be funding or
curriculum, most of all of our inputs are partners. And those partners directly correlate toactivities that they've themselves volunteered to contribute to or that their organizations are wellknown for.
Also, we mentioned the importance of being flexible – so it's important to us that our logic
model is not written in stone. If I had a dollar for every time we called it a living document, mystipend would probably be triple what it is. So it's a jumping off point for new initiatives. If it
was written in stone, I feel like it would be a big turn off for more partners to join us andstrengthen our initiatives, so it's very flexible.
So for example, both our jobs in education have a focus on disconnected youth. So what we
ended up doing with our logic model is we took one subgoal from each working group, a subgoalfrom jobs and a subgoal from education. We took their logic models and then we combined andreordered a subgoal from their logic model. And now we have four steps and activities underthat. And we plan to use that as a guide as we go forward working on a specific population. And
that meeting took a half an hour and a couple follow-up meetings. But it was very easy because
we had already had all of our thoughts written down and who was going to contribute to this in
the logic model that we had spent time previously on.
Third, it's very important to have well-recorded baby steps, we've found. Logic models for
Promise Zones can be huge and overwhelming. The San Diego Promise Zone has six goal areas,so our logic and each one has a logic model. So that's pages and pages and pages. So as we~~~ Noble Transcription Services - 714.335.1645 ~~~19

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

moved into implementation, we were very careful to prioritize what we were wanting to tackle
first and then time line the rest.
We also were careful to keep track of the changes we made so that there were no groundhog's
day meetings, that we weren't coming into a meeting saying, what did we change in the logic
model last time? Who's going to contribute to this? It was very clear, like, meeting before lastwhat had changed and what we were going to work on in this meeting to keep us from being
overwhelmed and keep the work on track.
Fourth, to keep the logic model from sitting on the shelf is – we find it helpful that it has a voice,that someone be the official manager or kind of advocate for the logic model. So that needs to be
our consultants, our research consultants. And in implementation that responsibility has shiftedto the co-chairs in the Vistas. It also needs to be someone that believes it's a good tool. You don't
want someone that's perhaps more action-oriented in their work and doesn't believe in a lot ofplanning. You want someone that likes to strategize and believes this is a really good method of
implementation and planning.
And finally, just presence, just having it at the meeting – so printouts of our logic models arepresent at every single meeting and they are used. We use them as reference as we're talking
about what work we're trying to get done. And in the agenda, the logic model and the things
that's in the logic model are explicitly mentioned. So as we're going through our agenda for thatday, it will say, we're looking at subgoal B in the logic model today and C's activities. So it's
very specifically called out.
So those are the things that we found. We rely heavily on our logic model. And I think all for the
better. It keeps us centered. It keeps us from getting distracted. So that's the end of how we've
looked at logic modeling. So my question to the group would be: If you have a logic model, do
you find it helpful, divisive or neutral? And if you don't have a logic model, what are the guiding
values or the document that you're using instead of that?
Janet Pershing: Anybody game to respond to Kayleigh's question?
Jeremy Stroud: This is Jeremy Stroud from the Spokane Tribal Promise Zone. I use something
that's kind of similar to what you're referring to, or at least I've tried to start. The only difference
I think between mine is that this is showing a one segment model that's a progression through a
one specific challenge. What I'm trying to do is understand all of the challenges and more
importantly, how each one of those challenges relate to one another.
Tribal governments are trying to balance or judge where they're going to spend valuable
resources and assets. So I'm trying to model that in a reasonable way given 50, 60, 100 different
challenges, goals, and direction. Any advice there?
Kayleigh Creswell: Yeah. Absolutely. I can speak to that a little bit. So this is a very what was
shown as kind of your foundation for first filling out a logic model. So what we've done is –again, that sounds very overwhelming and I know that's the challenge that all Promise Zone
faces. We're trying to tackle everything at once within 10 years, granted – so what we've done issplit things into goals and subgoals and then fill out the previous template that you've seen in~~~ Noble Transcription Services - 714.335.1645 ~~~20

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

those subgoals. So it helps break things up, keep it organized. And then as far as seeing how
things are related in relationship, that's also something we face as a challenge.
I mentioned how we're having a joint meeting coming up between our jobs and our education
working groups and how they've taken two subgoals and combined them and reordered them. So
in this meeting coming up, we'd really like to see those relationships as well. So what we plan on
doing is calling all of our relevant partners together that served those disconnected youth – either
in jobs or education or industry – and having them – doing an activity with them where they
typically place themselves, maybe be a sticky note and we can compile that later into something
more professional so it's an interactive experience.
But having them somehow place themselves along that continuum as far as what's coming down
the pipeline as far as, for example, I'm having 24 graduates on my certification program. And
perhaps an industry can go, well, I plan on hiring 5 new employees with these skills and then so
we can see kind of what's coming up and make those connections while we still have time toplan, so there's a warm hand off, that there's no gap that these disconnected youth are falling into.
So I think that might be a good example of how you can see where those connections are in thoserelationships while still using a logic model.
Jess Yuen: I'm also going to drop in here. This is Jess Yuen again, the community liaison for
San Diego. Throughout that whole planning process that Kayleigh referred to earlier – thestrategic planning process – we were doing very similar types of relationship building throughout
the entire thing. So every time we were looking at the logic model, figuring out the activities, it
was always an open question of: Who is not around the table that needs to be around the table?
Who was around the table but are not participating for whatever reason? How do we engage
them? Who is the appropriate person of the room who already has that relationship? Can theyreach out? Can they do a warm hand-off, things like that?
And then when we got to the conversation about data indicators and how we weave them into thelogic model, it was a very concerted effort to make sure that all of the partners were engaged in
the brainstorming process and in the prioritization process of figuring out what indicators wewere going to track, how do we layer them together? It was a very robust effort in order to usethat as a tool for maintaining and building relationships.
Janet Pershing: I think we've got a couple of questions that have come in that kind of have asimilar theme. So I'm going to try to put these together. And Kayleigh, we'll let you have the first
crack at it but others may want to chime in.
There's a bit of a question about as you're putting together a theory of change and logic model,
you're doing this around programs that are being implemented in many cases by your partners.
So what do you do if you have a partner who doesn't want to engage because they're afraid they
may not accomplish it and they're going to be held accountable for it? What do you do if you
want to put together a logic model or a theory of change, but you're not the entity that is runningthe program? Do you do that in isolation? How do you bring those folks in? How do you partnerwith them on that? Do you mandate that they do it?

~~~ Noble Transcription Services - 714.335.1645 ~~~21

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

So that whole process of working with your partners around the theory of change and the logic
model, have any thoughts that you might share on that front?
Kayleigh Creswell: Yeah. I'm sorry. In this example, what's the reason for them not wanting tocontribute?
Janet Pershing: In one case, they're afraid that they may not succeed and that they might be held
accountable for the results that they've not been able to achieve if they put it into a logic model.
Kayleigh Creswell: Yeah. That's a really interesting challenge. I can't think of a specific time
where we've faced that super clearly. My first thoughts would be that that's more of a one-on-one
conversation and that has a lot to do with trust building and that we've mentioned as far as
engaging our partners. I would say we absolutely don't mandate anything. We want people to becontributing because they want to contribute. We think that that's the best way to get work done.
That's where the best work comes from and people are actually engaged in participating on theirown behalf.
But I would think that that would be – in our case, it would probably be in a meeting with ourdirector, our liaison, maybe the appropriate Vista and it would be talking through those concerns
and asking them what kind of support they would need to feel comfortable contributing.
Janet Pershing: And I think that the basic process that you just described there very likely would
address the other question, which was a more general one about essentially how do you involve a
Promise Zone, an entity that got a preference based on Promise Zones and do you engage withthem and how? I'm hoping that kind of got at that question as well.
Got time for just a couple more here so who's got thoughts either questions specifically for
Kayleigh and the San Diego Promise Zone or anything you wanted to reflect back on? There wasone other that came in that I think I'll mention about theory of change and logic model moregenerally. And the question was: It looked like we demonstrated the theory of change and logic
model have pretty different purposes. And so if you start trying to think about them as a single
thing – as some people do – that seems kind of confusing.
So I guess my clarification would be, I think if you think about the theory of change as what you
think is going to be happening and why, and the logic model is really how are you going to
implement that? What are the actions you're going to take and the output you're going to create to
get where you're going in the long-term to achieve the vision in your theory of change? I thinkthat's how they fit together as a cohesive unit. So I hope that answers that question.
See if anybody else has got a question that you'd like to do verbally. I'm not hearing anything. I
will go ahead and just read another question that came in here on the screen. We mentioned thathaving partners can be input considering that PZs have limited funding, if any. Can we expand a
bit more about using quantifying your human resources? And Kayleigh, I think you spoke
specifically to that about the importance of thinking about the partners and their time
contributions. Would you like to expand a little on how you had to quantify that piece?

~~~ Noble Transcription Services - 714.335.1645 ~~~22

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Kayleigh Creswell: Yeah. Sure. Personally, we don't have any quantifier next to the partners andthe inputs, but human capital is huge. They're the cogs that run the whole Promise Zone. As far
as taking full advantage of that, we've been talking a lot about asset that mapping. So we've beenusing some different tool for that and reaching out to our partners. If you have a local 211, that's
a great source. They have an organized database that's accessed online that says all of the local
nonprofits at least and the services and the populations that they're serving. So that's a good way
to clearly see in one place where your partner's strengths lie, but nothing beats directly reachingout to those partners.
And also, I would recommend all of our Vistas sign up for several different newsletters from ourpartners. So it's a great way to keep up on what's going on in our partner organizations,especially because as people are coming to our meetings, they only represent one person insometimes what are huge organizations. So no one can know everything that's going on. It might
not fall under their purview. So that's a great way to get a big-picture look on some of the things
coming down the pipeline and recent successes and resources that your partners could possiblyoffer that maybe the representative that they're sending isn't quite aware of yet.
Janet Pershing: Great. Thanks, Kayleigh. I did have another one come in here, a bit of anobservation in some sense basically pointing out that we need to make sure that if we think about
this, we're framing it with the emphasis on the fact that the PZs don't do or implement these
programs. We're putting together a theory of change and a logic model that is really for programs
that our partner organizations are implementing and then in some cases are already underway.
And that's a really important observation as we talk about how you involve your partners inthinking through your theory of change and developing your logic model.
Obviously, they are the ones who are out there on the ground doing this work and you need tomake sure that they are in sync with you about what you're putting in your model. At the sametime, as we notice, there are going to be multiple different segments of activities that you hopeare going to come together for an overall goal, that collective action output. And so as the PZ,
you really need to be able to bring it together and synthesize that material that's coming in fromall of your partners. So I think it's an important observation that this is not all within your directcontrol. So thank you very much for sharing that thought.
I think we've got time for maybe two more questions if anybody has one or an observation. If
you don't have a question, that's fine. You can just share a thought. Just one of my questions andthis is one depending on how many of you have actually gotten yourself to the point of having atheory of change and a logic model: Have any of you found that you have had an opportunity touse it? We heard Kayleigh describe that it is on the table every time they have a meeting. Theyuse it very regularly in her community. Anybody else find themselves using it actively in thatkind of a way or in another way?
Angela: Hi. This is Angela from Place B, the HUD community liaison. And I think I have moreof a not necessarily an answer to that because our Promise Zone is still developing our logic
models. But my question is more to San Diego. Were your partners the one developing the logic
models and actually putting the graphics together? Or was it your backbone team, because what
we've found is that we're finding ourselves and our Vista team putting the logic models and thetheory of change together and not necessarily our partners.
~~~ Noble Transcription Services - 714.335.1645 ~~~23

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

Kayleigh Creswell: Yeah. Absolutely I can speak to that. So at each of our meetings, it's all ofour partners, the backbone team, as well as our consultants. I have to give a lot of credit to ourconsultants. They're the experts on a lot of strict modeling. But I don't think consultants arenecessarily a necessity. What the consultants serve to do is a lot of reflection. So they'll say
things like, so what I'm hearing is this, kind of summarizing the group discussion.
So a lot of reflection and then they'll be editing the logic model on site. When those reflections
are approved by the group, when the group says, the partners say, yes. That is what we're saying
or not quite. We need a work method that – and then the consultant will write down thosechanges in real time.
[talking over each other]
Jess Yuen: I was saying the consultants for the San Diego Promise Zone did a really great job ofworking with the backbone in order to guide the Promise Zone partners through the logic
modeling process, and as Kayleigh mentioned, doing a lot of that. I hear this in making sure thateverybody's on the same page getting consensus and then turning that into what you see in frontof you. And over time – because consultants aren't forever – that work really has transitioned tothe backbone team in partnership with the co-chairs. And that's how I think moving forward it's
going to be.
Angela: So just as a follow-up to that, are there any kits or advise or Promise Zones that don't
have a consultant where it's like our backbone team I know this is great that we're doing this data
TA in this webinar to learn more about these logic models and the theory of change. But for
Promise Zones who don't have consultants, then we're really just relying on the liaisons ordirector and our Vista team to put these pieces together. Are there any tips on how we should be
putting these together or involving our co-chairs?
[talking over each other]
Kayleigh Creswell: Yeah. I will mention first, I think the most important thing is to make surethe logic model isn't sacred somehow, that your partners can't touch it or you need to do
everything perfectly. It's a tool. And it doesn't require a certificate or a degree to be able to messwith it. You do have to think carefully about what's an output, what's an input, what's an activity?
Are these things supporting one another? But it's very assessable to anyone who wants to use thetool.
So as we did have professional help, but I don't think it's necessary. After we went through our
logic modeling, the Vistas kind of sat down and made our own logic model. And we found it areally helpful exercise personally. So it's very attainable for anyone who wants to approach it.
And I think it's important.
I know there were some challenges that got brought up as we were logic modeling because as
Jess mentioned just recently that we've transitioned to the co-chairs having more autonomy overthe logic model itself where the co-chairs would be like, we need the consultant and we're notsure we're doing this correctly. So I think just encouraging everyone to really get their hands~~~ Noble Transcription Services - 714.335.1645 ~~~24

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

dirty in the logic model and ensuring that there's no special intense training needed in logic
modeling.
Janet Pershing: Jess, did you want to get in a word?
Jess Yuen: Yeah. I just wanted to say really the consultant's the most useful add for them interms of our process was for them to really just guide people and keep them moving along and toclarify. I mean, the questions that most of the time they would basically ask is: We're looking at
inputs right now. What are some of the inputs that the group, you subject matter experts have?
People would say things. They would write it down.
And then they would say, well, when you said this, did you mean that? Does everybody agree?
Those were the types of questions that the consultants were asking. And it was really just to
guide people through the logic model and then standardizing it across these different workgroups. I think it's eminently like Kayleigh said – eminently possible for you to do that without
outside help.
Janet Pershing: And I'm going to need to cut us off there on the questions just because of time.
Some great discussion. And Kayleigh, thank you so much and you too, Jess, for jumping in withall those thoughts to share. I'd like to move on to slide 48, if I could, and talk a little bit about the
homework. I mentioned at the beginning that we're going to be working through a homeworkassignment after each of the four webinars. And by the end of the process, it would give you all
the pieces that you need for your framework. We did have a question asking if you could have apreview of what the homework is.
So I'll just give you the high level here. We can maybe get one written out for our next session.
But basically, we're going to be starting by thinking about the logic modeling and theory ofchange that we tackled this week between now and the next session. In the next session, we're
really going to be thinking about evaluation design and evaluation strategies, so you'll do somethinking about that. The following one you'll be thinking about data and what kind of data you
will be collecting and your strategies for collecting and managing it. And then after the final
session, you'll really be thinking about analysis and data visualization and how you're going tocommunicate your results.
So at a high level, that's what the homework is going to look like. You're really going to be
developing those building blocks of the data and evaluation framework. I'm going to repeat
something that Xiaodong said because I'm really afraid people will walk away and be worried
about this. We in the homework are giving you a format so that if you don't already have what
you need to develop – if you haven't already begun, you don't face a blank sheet of paper. You've
got a form that you can fill out. It'll guide you through step-by-step.
That said, if you already have a fully fleshed-out document or a partially developed document or
you just like another style better, you are under no obligation to use our template. It's simply atool to support you if you want it.
The other thing I'd mention is that with regard to some of the questions about if you don't have aconsultant how do you go about doing this, while you don't have a long-term consultant at your~~~ Noble Transcription Services - 714.335.1645 ~~~25

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

disposal, you do have office hours at your disposal. So you want to make sure as I talked about atthe beginning those office hours, folks are there who have evaluation expertise who are available
to support you as you are doing this homework. So I really want to emphasize that you shouldmake good use of them. And we'll talk in a minute about how to access that assistance.
So slide 49 gives you the first of two pieces of the homework for this week. What we're asking
you to do is very much implement what you heard today to describe your theory of change andthen to build or refine your logic model. You will find in the homework that there's a blank spacefor you to write that theory of change. Obviously, we're going to recommend that you think
about that Pit-B model that Kasia talked through. And there's going to be a Power Point that's got
the little boxes and arrows just like you see them here.
Now, surely, your program is not going to have exactly this number of boxes with the arrows
going in this direction. Obviously, you will need to adapt it. But the fact that it's already laid out
in a Power Point template means you can just drag and drop things and copy the pieces shouldmake your life a lot easier if you want to go ahead and use this as your model.
As somebody mentioned earlier, doing a single logic model for all of the pieces of what you're
doing maybe overwhelming. You may need to do a separate chunk by goal or by subgoal. And
that's fine. You can definitely do a separate page. And ultimately in your thinking down the road
you're going to want to think about how does that all come together to meet your broader goals.
But it's fine as a starting point certainly to think about doing a logic model that's fit by goal or by
subgoal for your organization.
I'll also just say that we're asking you to do this particular exercise, come up with your theory ofchange, and draft that logic model in a couple of weeks. Obviously, Kayleigh talked about
spending 10 months refining [inaudible] it's an ongoing process. You're not going to do that in
two weeks. What you will be able to do we hope is to really think about what you already know
about your theory of change and the pieces that you know about your logic model and get those
captured into a framework that can then be refined as you work with your partners over time. So
that's your first task is to think about those two big chunks.
The second task is on slide 50 and that's just to think a little bit about the barriers that may makeit difficult to implement your logic model. And to kind of brainstorm about those we've
definitely heard that there are some Promise Zones that have gotten to the point of coming up
with their theory change logic model and then feel kind of stuck about the implementation. And
that's the last thing we want to see is people stopping after you've put your work into thedevelopment. So we'd like you to think about the barriers that you're seeing and then really think
about some possible solutions.
I think we've heard a lot of great creative ideas from Kayleigh during her discussion today. Other
people chimed in with a couple of other alternatives. Think about those. Talk to some of yourcolleagues. And if you can't think of a possible solution, again, use those office hours. Talk to thefolks who are going to be there to support your thinking about evaluation, about their thoughts
about how you might be able to overcome whatever barriers you are seeing.

~~~ Noble Transcription Services - 714.335.1645 ~~~26

Developing a Data and Evaluation Framework for Urban Promise Zones - Session 1, 5/2/18

On slide 51, you will see that there is a link to the HUD Exchange. If you go to the HUD
Exchange, you will find that you can go to programs and find Promise Zones. And on thePromise Zone's page, you will find on the right-hand side that there is a link to the data andevaluation framework process. That's where you are going to find the glossary. Some of you
were not able to identify that at the beginning of the session today. The glossary is there. You
will also find the instructions written for the homework. I just talked you through the two maintasks, but it's written there. The logic model template is there and a little template for the barriers
analysis is also there.
One tip that I'd like to give you is that when you open those documents, depending on yourbrowser and the temperament of your computer, it may not want to open. Therefore, instead of
opening the document, if you run into any problem, choose "save as." You choose the save-as
document and you should be able to get to it without any difficulty.
So that's the homework. I do want to just point out the resources that we've been alluding to on
slide 52. Again, it's that HUD Exchange Promise Zone webinar page – you find all of thematerials for these webinars. The office hours, if you scroll down on the HUD Exchange
Promise Zone page, you get to a place where there's a link on that page where you can sign up
for office hours. The AIR folks have got hours on Tuesday, Wednesday, and Thursday of theweek following each of these webinars. So you can go on there and find yourself a block of timeand get signed up.
You'll also want to be watching out for the peer-to-peer learning. May 9th will be the first of
those sessions that AIR is going to be sponsoring. They'll be bringing folks together for somepeer-to-peer sharing. You want to make sure that you catch that. And then we will be back here
for the second of the four content webinars on May the 16th.
Slide 53 gives you some references that we used as we put together the materials today. If you
have interest in seeing anything in more depth, they give you some links that you can try there.
That would include the last item, which is the screenshot that Xiaodong showed you about thelogic model that looked rather different but had many of the same pieces of content, just to give
you an idea of different ways that you may want to think about and format your work tocommunicate with your communities over time.
And that's all we have for you today. We are right at the top of the hour so I'm going to wrap it
up. Thank you all very much for your attention. And we hope that we will see you back here next
week. If you have questions about the homework, you can go ahead and reach out about those.
And we're going to make sure that we take a look at the technology and see if we can figure out
what folks are running into so that we don't have that barrier again next time around.
So with that, I'll thank you all very much for your participation and we will see you in two
weeks.
(END)~~~ Noble Transcription Services - 714.335.1645 ~~~27