what i want to talk about is how this technology sort of dictating how we go about living our lives this is what a camera looked like at the time american with photographs people in public and the outcome of that was that basically if you can in fact photograph people in public without their permission this is what the telephone look like at the time american jurisprudence was happening with wiretapping and whether or not it was you needed a person's consent to actually record their records their conversation and it came out the opposite of photographs it came out that in fact you do need a and sure enough that's exactly what did happen there were these cases of various kinds of incidents for example there was a parent who put a video recorder on her son and used it to record how the bus driver was treating her son very poorly and and capture the entire thing on tape now for those of us who are parents were like go and then the police came and arrested her not the police not the bus driver on felony charges there was the example of the protests happening at boston university and someone began videotaping the police being very aggressive with the protester and the police  the protester turn around and arrest on wiretapping for wiretapping the person who was videotaping so nowadays many other questions have come up around this clash between this dissonance we had between photographing people in public and recording them but but it all started with an interesting technology design decision one for sony having no camcorder and today almost every cell phone still has no mute option so it reminds us that not only did the technology design happen and we began changing our laws around it so that mothers didn't go to jail and so that we can in fact videotape the police in the in the in the actions that they do in public but we had to actually change our laws to make that happen or change the way cases were interpreted and so this means that technology is having a profound of perfect just based on the decisions that are being made and so this is why i say this point about technology really is the new policy maker let me give you some other examples this is  sleep number bed it's an air mattress and so basically the way sleep number of works is you have these chambers and you can basically how much air you want to put in different determines how soft or hard the mattress and so and so that's the basic idea of how it works not particularly not a particularly controversial issue except this is the new sleep number bed with sleep i q sleep i q you can pay for it in google for a few hundred dollars more and when you wake up in the morning you can use your app to go see how well your night was and it gives you an overall score of how good your sleep was now throughout the night it's measuring day it's measuring everything that happens on that bed so for example a cat a fifteen pound cat ok a low fat cat it actually picks up his breathing and the fact that when and where he is physically is on the bed that's how sensitive it is it picks up heartbeat and temperature and so forth in the bed and the information that gets captured by the sleep number bed is not contained at the bed it's not like i get the go look at the bed and see what's the answer i have to it's all shipped out through my wife and i to go to a server maintained by sleep number that i then connect to with an app which means this information going outside the home  design decision made by the company this is the apple watch and i have one here and so the uncle watch also is monitoring everything i do it has it has of quite a few different instruments in it so for example it can actually track where in free space my hand is doing what movements it's making its tracking my heartbeat it's doing a lot of the same kinds of things as the sleep number bed but apple made a different decision the information on the apple watch isn't going to assume a repository but even tells me when i need to move around isn't going to a central repository instead the information is stored locally on my phone a very different kind of design decision the impacts of those two those differences whether the data get shipped out to a central repository and then is not is totally beyond my control or whether at least it's on my phone has a different impact and on how i'm on my own privacy in the issues that come with that but notice i had no say in either one so i think one thing is clear from all of these examples is that they underscore this idea that we live in a first came around and introduce the word technocrat means someone who took a government position because they were the best in their field as opposed to a political appointment or some other reason but you know the idea of having the best expert in education be the head of the education department that was how tech knockers who was first introduced and the biggest so the biggest society of technocrats is actually communist china where people are are sort of a technocrat in a particular position but i could argue that we now live in a different kind of technology i would call it took not proceed two point zero and it's basically where technology design is basically dictating how we're going to live our lives now what kind of scary about it is there's no master plan this is not us though everyone is so looking valley has a plan is just how it's working out and this kind of replication that we saw like with the camcorder where without the mute button everything that comes after it keeps the same design and so the sort of replication is also happening and so the question is what do we do about it and what else is what are there examples are there i said the privacy was on the first wave so let me give you some other examples many you know a couple of years ago i was sitting in my office and i was being interviewed by a reporter and we needed to look up an old paper and so he types in my name into google  and find the old paper and when i clicked return up popped some ads and along with it came the result that we're looking for the paper and the ripped and i said well here's the paper click on it and the guy says oh no i'm much more interested in ad can you tell me about these arrests you've been having and i said well i've never been arrested he said i said and so we began searching more and more and we kept getting this interesting this is true seen issue that if i t it was somebody as your last name we would get these arrest ads and so he has a why it's how you know american guy and he jumps to the conclusion that it's because you have one of those black sounding names he says and i said look i'm a computer scientists computers aren't like that they don't they don't have these kinds of bias is no one in the computer would do that and so i kept playing around and i notice that well if you type in the time into images you do in fact s black faces and if you type in time yes you do in fact see white faces and so and the more i kept trying to prove him wrong the more i could so eventually i just broke down and said ok let's i took like a month and all out experiment and which i captured one hundred twenty thousand ad deliveries using g p s from around the country and was able to document that in fact these arrests that showed up much more often with baby names who's with people whose first names are given more often to black babies that the white babies and that the difference between those was almost eighty twenty and eighty twenty in this kind of context is very significant because the department of justice has a disparate impact assessment that they do and they say what is the difference at least eighty twenty and if it's eighty twenty then in fact they would argue that it's a grounds for discrimination investigation so it's not that everything just kind of clarify discriminate all the time we have student discounts we give elderly discounts the discrimination itself is not a problem but historically in american society we have certain protected groups that we protect in certain situations like housing and employment and so forth and this particular issue was in violation of that because one of the things that  potential employers do is they google a person's name and so if it's having this kind of disparate impact it's actually impacting on one of our protected history historical protections there are many reasons that could account for this so the way google search engine worked is they put the in this case the company a put down five different ad text that would show up and they said if you somebody searches for revenue glover it had to be a first name and a last name and it didn't matter whether or not the person had on the rest record it only matter that it was the name of a real person how much they would pay if the ads clicked on and then  algorithm would choose one of these texts to deliver the one that it chooses is the one that makes it the most money which is kind of an interesting effect because it could be that we were seeing this effect because of the choices that the company made it could be that all five of these had a rest in them or it could be that a couple of them were neutral but people tended to click more often on the arrest ads for black names and not as much for white names so it's kind of  began whole conversation for several reasons this case around around what's became known as algorithmic accountability the idea of how do i study an algorithm and know whether or not it's having a disparate impact or having a particular outcome and if so what what what what should should or could or is going to be done about it and so the reason i show you this is because intent doesn't really matter in these cases it just matters that it is in fact the effect so i hope you agree with me or it loose allow me to go on anyway we live in a way that we do in fact live in a tech not krissy and that all of these types of values are up for grabs so let me give you another example the pittsburgh courier was once the country's largest black newspaper and so whenever they wanted to give a noun like this from for newman's they had a staff who would review the ad copy before hand to decide so use me to decide whether or not the ad was appropriate for this audience but today the pittsburgh courier looks like this to website and google primarily chooses what and it's will show up on and so that's kind of an interesting shift around what kind of ads might show up and the fact that they no longer have control i mean hundredth anniversary now a lot of people like congressman cyber club cliburn earl graves and i say executives a lot of michael jordan shaquille o'neal a lot of really well known black men have come through this fraternity and so this was their hundredth year anniversary and the question was what kind of ads showed up on their website while they were asked for graduate degrees and that sounded pretty reasonable  and there were also ads about taking vacations that's on that pretty good but then there are also ads like this and sort of sort of these unwanted ads implying criminal records and so forth and then there are also ads for credit cards so they have to see includes the ability to credit card reporting act as cover the credit reporting act is covered by the f t c and so we became to focus on  what kind of bad showed up here and how do they fit in the scheme of credit card ads so we looked at on the far right are the most highly praised cards the most the top praise card would be american express blue on this side are the most harshly criticized cards cards that if you look on line people will always say please avoid these cards at all costs where you see highlighted in orange of the o y examples that only the ants for cards on the left showed up you make a side by side which is kind of interesting because you could say then where did the american express blue cards show up so now on the right we see all the places that american express blue cards showed up and oh my goodness they look like a lot of educational sites which only goes to beg the question even more as to why those that ads never showed up on the this is again underscores the idea or the issues that algorithms are making decisions and that we somehow need a way to think about how do we hold them accountable how do we assess them so there's a good and a back to that were the good thing about that work as we began to say that we could use technology and experiments on technology to see what a black box piece of technology was doing was it i into the rules that we expect of society or was it not it hearing to them and then began to think about how to address them so let me share with you then another another another example does also work from the f t c we were interested in and we heard a lot of times people say there is one internet we all share the same internet but we became really interested in the question of well is it really the same internet do we really have the same internet experiences motivated in the back of our minds is them a good fight example what are what is everyone's internet experience really like and so when we look at the internet meaning all of the places that we use com score data so we look at all of the places that some number of households in the united states had visited and purchased from their entire online activity done from those machines and we knew the demographics of those households we began seeing some interesting patterns so one pattern that we saw is that the first ten sites roughly tend to be everyone is sort of the same facebook google and so forth dominate the top of the list for everyone but after around number ten you see huge shifts in which sites people visit more often and they break down around demographic lines so we created an index called the exclusivity index which is the percentage of people in one demographic group divided by the sum of the percentages by all the other demographics to close to that ratio is to one the more exclusive that one domain is to that particular demographic so what you see here are only are only two of the demographic of whether or not you have children or not in the household and you can see it's a really bright divide but there are places that you go you people go when they have children that they don't go when you don't have children and so the red line that you see going across the top there is two standard deviations up so we consider something that's way up there very much more exclusive in terms of reporting and so what kind of  sites they look like well these are the kinds of sites that people who have children tend to have go to and people who don't have children have no idea what you saw its owner here's another example based on just race again we see sort of sort of a modeled effect we see some that are rising very high up above the standard deviation cut off and what do those domains look like well for latino's you know vision is number one  but the number two is footlocker and it's kind of interesting that of a higher percentage of blacks this is a foot locker than any other race i think that's really interesting it underscores for us that you can always use the name as a predictor of what websites may be more exclusive to one group than another and i'm certainly not going to even touch the idea that legacy dot com is the most exclusive for  whites that's another study so if you know about that people or have experience you not one internet but there are different internets or different internet experiences then this was very insightful as a tool at the c so another thing that the f t c does is fraud so now the way fraud normally works is we lose a lot of money we report today you have to see you lose a lot of money on the same fraud you reported to the f t c eventually somebody notices wow there's a fraud out there that's costing people lot of money let's go see if we can track it down when they go to track it down the frost fraudster is long gone and so now it becomes again a game of chasing that person down to try to recover some of the funds and what we were able to show is that and also the other thing that the f t c had learned over time is that fraud is specific to ethnicity so there's this fraud that we all see on our spam that's peanuts compared to the fraud that the f t c typically deals with which is a very very small if somebody really wants to separate you from your money they get into your trust network because the more the more they come from through a friend through a church through a school organization the more likely it is you're going to trust them with a larger amount of money and so these frauds tend to be fit into these same kind of demographic groups that have different internet experiences and so we began doing is using the internet experiences to identify where the frauds are likely to occur and so what you see and read were signals for which we were we believe were highly so that's a lot of you know she she works talk about another example when what we're starting to see now are on the one hand they're kind of the same kind of creepy latanya examples as always but on the other hand there's something a little different about these than my normal creepiness what what's different about this though is there's a kind of empowerment happening a kind of use of the technology that allows us to see something that we didn't see before and do something that we couldn't do before whatever but i have no idea who all the people that you might be giving the data to so we began tracking all of the places that the information may be going from a patient who visits a hospital and we did this through breach notices and through freedom of information act requests and if you actually visit the site the data map dot org and you click on one of these notes a list for you the actual companies involved and the basis on which we made and that's kind of interesting that more than half of the flows aren't even covered by hipaa so that was kind of interesting and then another thing that we saw was this discharge data now how many people have ever heard of hospital discharge data so most people have not how many people have ever been to a hospital or gone to a doctor ok you're in it right so most states passed a law that every time you go to a hospital or to a doctor's office a copy of the information primarily a copy of the bill goes to the state and the state then can sell or give away the information and you can see that this idea of discharge data is contributing a lot of information in the state amount to a lot of different places i just want to point out including my my performer employer but that's another story in fact almost every state collects the data thirty three of the states sell it or give it away and  big darkness over there for you know for washington state but only three of them do so in a way this district us hope because they don't have to adhere to help of the other thirty states don't and they use the standard that was less secure than hepa so we were interested in well maybe hippest too strong right maybe that maybe hippest screwing up the data more than it needs in the name of anonymity and or it could be that these thirty states have it wrong they're putting people's data at risk so for fifty dollars i purchased a fool years ago this house for the state of washington and it came with about three hundred fields of record for every hospitalization at least ninety nine point nine nine nine percent of all hospitalizations in the state of washington in that year were in the data this is a sample of the kind of data that i found there it includes an information about the hospital the type of admission how long the person was in the hospital where they went when they were discharged how they paid for it what were the diagnosis codes what were the procedures that were done to the person the person's age and m year their gender the five digit zip code and which they lived and their race and ethnicity and we were interested in who was not just only answering the question around the question a couple was it  more revealing or not we also had another agenda too and that agenda was whether or not employers could be using the data whether or not banks could be using the data who could actually use the data to learn more information about employees or people to whom they've given credit so we could ask ourselves what does an employer know about someone who went to the hospital well they know that they took time off from work and they may actually the mall the company is they may know a lot more about sort of the procedure that was being done they know the person's age and they know there's a code so they know quite a few things about the they may know the general purpose and so we said well how could we find that out about people here how can we have the same kind of knowledge that a friend could know about you or family member who knew about you were an employer can know about you and so we turn to newspaper articles and harvard's kind of cheap they don't actually have all five of the archives that are of newspapers they only pay for one so we only had the single archive that had about eight newspapers that were saved for for the state of washington so we had a sample that we could head against and so we were interested in taking these kinds of stories a mis case a motorcycle accident and we wanted to match it up and see if it matched exactly to the health records you know and so what i mean by match i don't mean i don't mean i'm sorry computer science i don't mean anything interesting i mean simply match like straight mess like this word is there this word is there right not no probability no computational linguistics to play just a straight match so we take the words we were trying to extract them out we said does it match if it matches one and only one record we said that that's a hit and if it doesn't match any records we take out one field at a time and see if it uniquely matches so in the belief that the newspaper may have gotten one field wrong and so this was our criteria so how does a match of work well so sixty two years so it's like man matches to zip code happened on saturday afternoon and we know the date of the newspaper the person's name in this case is wrong jamieson he had a motorcycle accident and he went to that hospital and so how well do we do well we took the news stories and then we went to public records to get because they always talk about like soap so flick city or a particular name of a city so we translated that into zip code by looking up the person's name and then we came over here and did our exact matching using the zip code plus the other information that was in the news article matched against the hospital data and so we found that in a bout a little less than half or three percent of the time we got an exact match and that's exactly the kind of same information that employers know creditors know and so forth now one of the things we did to go to confirm that we were correct is we gave all of the names to a reporter from bloomberg news under the understanding that he he and his editor greed they would not report the name of anyone unless that person agreed to it but on the other hand we wanted them to contact everyone to say whether or not we were correct and and he was able to confirm that every single match was in fact correct and they included congressional members included professional athletes included businessmen it included a lot of people who weren't that confortable getting a phone call from a reporter asking about a hospitalization because one of the things that hiding out in that data are things that have nothing to do with why you went to the doctor so if you if there's drugs in your system you want there because you got in a car accident but that will still tell us whether or not there were drugs in your system will tell us whether or not you had sexually transmitted diseases even though you didn't go for anything you were there because your arm is broken right and so so a lot of other kinds of information we're hiding in the data as well so who's been buying this data so we issued for your request this  freedom of information act requests to all of the states and ask who have you been selling the state or who you've been giving it to and here we really expected to see that the universities topping the list that is you know i know my colleagues are getting it all the time public health for many great fantastic studies and so in some ways we thought this was kind of a diety request to put all of all this effort all this effort into these freedom of information requests expecting researchers and research universities to top the list but that's not who does top the list who tops the list are the analytic companies remember the consensus that we all click through the click through agreements that say yes i agree that it goes to business analytic companies who are these analytic companies well are the analytic companies the producers of data products they make their money by making products that link data together in different ways and so wasn't if you think about is not so surprising to find data company that sells a lot of prescription data back to pharmacies pharmaceutical companies and so forth and so is a bit of a surprise who is actually getting the us now because i'm here in the state of washington i do want to tell you a little bit about what happened so you can imagine one day the person who is in the department of public health gets a phone call from me that says i've written a five year data it doesn't usually make a person quote happy so she but i have to tell you she was fantastic she was really fantastic i've given many of these phone calls and they don't usually go so well but in her case she says look i'm stuck between a rock and a hard place i have to protect the privacy but we have these laws in this in washington state there require more transparency and if i redact more data i'm going to get a fuss i'm going to get yelled at and lawsuits and a lot of pressure politically that i'm trying to hold data back and so she felt that the only way she could go forward was in fact if the law changed so in fact the law changed was pretty amazing and so now if you try to get the data the laws change so that she now provides the data in a in three different ways you can go to the website and you can just get aggregate statistics you can still pay fifty dollars and get this version of the same data set but some of the fields in which this experiment was space are now been more aggregated or you can promise your firstborn child and get an i r b review and so forth and they'll give you more detail data and s sort of tiered access to data i think was a great outcome because it doesn't stop anyone with a legitimate reason to get the data from getting the data but at the same time it doesn't make it as casual so you would think then that is sort of thirty states releasing the data this way we would know how how many of you right twenty nine ok i tell you it's one state at a time so so i climbed that we live in a tech nakor see that every value is up for grabs that we can take the technology and harness it for good and so does that mean that if we did it on a large scale if we did with you could we in fact save the world and so i became when i left off to see that was i was really interested in answering that question how do we save the world because i'd ha big left a big imprint on the f t c itself and that imprint had to do with being a technologist and being able to unpack the technology from the business practice being able to use technology to give us new insights into where the problems were that are of interest like in the fraud example and also been able to do social science experiments to show to sickle evidence of problems that were occurring and so that was the f t c while i was there but how do we make this on a larger scale how do we get to so if the f t c wanted to start a bureau of technologists who would they hire who knows how to do that how do we how do we bring them the masses and and and the fever began to spread it wasn't others in other parts of the government wanted somebody doing something like that for them to and civil society organizations begin saying well we we need someone to do that for us or like for example they would say that arrest that thing was a discrimination case we are the we were the ad because for against discrimination but we don't know how to do that we need t do that too and all of a sudden i could see an alternative to the tech knock receive that live in one that shored up our historical protections by giving them the knowledge in the technologists that they need to do their same jobs so i became really focused on that i taught a class and the students in the class did a project and we expected about three of the projects to be good enough to take down to the f t c to show them to regulators and in fact i made a sort of a bet with the students as i will any of you do a project that's good enough in consumer protection i will take you down i will fly you down my actual words will fly you down to d to meet with regulators and in fact we took twenty six students yes a budget buster because they reminded me about the fly part  what it was really an amazing experience it was supposed to last for two hours and lasted for almost for the regulators who you have to realize many of the regulators are lawyers who are aging and the students who are the closest to the technology we're talking about apps and problems that the regulators have not really heard of and even if they did hear of it they hadn't seen this way of thinking about it or exposing it or testing for it and so it became pretty ecstatic for both the regulators but also for the students who felt like they in fact had kind of saved the world of that ad real world impact so we didn't want to lose that momentum so we decided ok you know we're going to do we're going to polish up these papers and we're going to publish them and so we're going to have a new journal called harvard papers on technology science and so i then reached out to fifty of my colleagues from around the world and i said look we need people to  papers and when they all said yes without any of the explanation about the students or anything like that when they just said yes we realize that this is so much bigger than the harvard thing and so we launched this paper th called technology science and so let me show you some of the kinds of works that have come out in technology so i hands so one of them is by daniel daniel is the one who wrote he said look i can do you one better than your fraud by looking at these websites i can chip track brought for you in real time not something that you have to now go find the frost or later i'm going to use twitter streams and track it in real time that's a huge big change shift for the f t c right and so that was kind of cool so his papers there and the technology has been given to the f t c another example were students who showed them when you visited princeton review website and person review as you may recall does online tutoring for s a t s course when you visit their website they required a zip code before they would give you a price and so the students mined out all thirty two thousand zip codes what was the price given for each of those zip codes by the princeton review and what they were able to show was this kind of regional effect that people who live in massachusetts pay the highest possible price of people who live in washington state generally play the lowest possible price and clearly people in massachusetts don't like that but that's at this scale proposal into julia angwin a proposed look at took the data and began looking at a much closer level and when she looked at a really close level it was an average of code in fact it was the zip codes more often of asians so that if you were asian you were more to almost twice as likely to pay the higher price are interesting and so pro publica is paper is also on technology science another paper this particular paper shows you that the same hotel the same rental car is the different price you know we're all used to flights being we're used to being on the point where no two people pay the same price for the ticket for that seat but we're not used to poll tells having such different prices and we're not used to car rentals having different prices depending on what country i ask the price from and so this is what gets documented here using v p n two students of a different one this is air b n b study it showed so there are already been earlier air b n b study in new york city that showed that if you were a black host you tended to earn twelve percent less your prices tend to be around twelve percent less for an equivalent housing than white house and so the students were interested in repeating this but it is said they chose the berkeley area in california and they were interested in comparing whites and asians and they showed since contacted us about some strategies that they think will combat that concern this is a study that got a lot of media this was our first paper published this was  iran for four years facebook messenger had a privacy leaking problem that whenever you sent out a message it would give out your g p s location and not just your friends it could be to the friends of friends and so forth and so so oh boy that was good their house out and so iran decided to make this graphical so he built a chrome extension that visually would show you all of all of the locations of people who were sending facebook messenger text messages and so as soon as he did that nine days later facebook changed it they changed it where you had to opt in there so that being on everyone this particular story has also gotten a lot of traction because he had gotten a summer internship to work at facebook and so they waited till he shows up and then they rescinded his offer it's ok he ended up with a better job and a lot of fame and articles around the world because of that story and even an op ed on time and time magazine so he's good but i'm not your first book was so good they certainly did not when this is also worked on a while we were at the f t c the what you see on the left are the top most popular apps and what you see on the right of the domains to which is sending personal data and so if you visit the website will show you for each of the domains for each of the apps what personal information is actually sense but it's kind of amazing how much information is going and to whom it's going and the fact that most of the apps are giving the information to third parties that you another study this is also by students who looked at facebook privacy policy over time and measured it against these various measurements that were used on health data privacy policies originally developed by microsoft pricewaterhouse coopers and patient privacy rights but they applied it to facebook's privacy policies over time to show what we all know in this sort of the quality of their privacy policies were going down and the only time it stopped that progression was when there was intervention by the f t c and out of public outcry this is from two of my students who actually now have a company here in seattle and what they were able to show was that at first on the portable care act the health exchanges the large companies stayed out where the large companies decided it wasn't going to go away and they decided to come in they raised the prices by over seventy percent and so one of the things that happened shortly after the paper came out was the center for medicare and medicaid now requires more documentation was as i understand it requires more documentation if an insurance company is going to charge a higher price this is a real benefit cation experiment of that i am as healthy there to the people that i said is the prescription data this is stated that they collected from south korea and we were able to reverse  eer it to get what's the equivalent of the korean social security number or the national id number that's similar we showed vulnerabilities another student sort of vulnerabilities in iceland's number using apps and we talked about this we've also published some remedies proposed remedies this is sort of like the tear system that we talk about the washington state adopted but just on a more generalized scale now what's really interesting about those that's not even all of the stories that have been published it's not even the most popular it's not necessarily the most anything it's just a sample and we only started in august so when i say that the world comes down to you and your ability to save the world i really do mean that because these studies every one of these experiments a simple experiment has had a huge impact this huge impact all over the place and is something that everyone in this room could do so i'm going to stop there and open it up for questions because i sort of went on for a while and i'll give us a little more time to talk so this is a great question so one of the things that's kind of so so normally when computer science joins another discipline they create their own sort of body of work that been goes on so for example computer science class with biology has given us bioinformatics and then it goes on to study it and so this is that time in which computer science is often joining with some other disciplines to tackle problems in an area and it produces its own body or its own discipline own kind of problems to solve this one is kind of a trick tricky one this technology science because there's no one discipline that could join with another discipline if you look at the breath of problems that the students address some of them were done economic analysis some of them were statistical analysis some of them some of i didn't show but they were algorithms that you had to either analyze or new technologies that had to be built some of them were legal analysis to make the case that this was this was was actually in violation of a particular law and so what makes that hard for technology science is that it's not as some people as taking two disciplines and fusing them together and i can't even say the gee if you wanted to study discrimination all you need is to take these two disciplines and put them together instead the problem seems to drive one who has the best tool to provide an answer or insight into it so your question about how do we train people is a fantastic one that i have no idea of the answer in general but i have an idea of how to get to the answer so the w while i was at the at the the f t c many of the government agencies so how do we get people that can do this how do we produce this work force and so i began a conversation with the national science foundation and marcello was the had the most insight around what this was like he said you know it's like robotics many years ago when we started robotics he said funding robotics work people came from all kinds of disciplines there were mechanical engineers there were psychologists there was just a just a just a huge collection of different disciplines coming together and that's very difficult to accomplish at a graduate level which is one of the reasons i'm very proud of the university of washington it's a very difficult thing to do it a graduate level but he says but the way they did it in drop boxes they did it through the undergraduate program so they gave research money and then they would include undergraduates and the undergraduates could do aren't tainted by disciplinary silos and would tackle these problems fearlessly and that's what we've certainly have experience with this and so that's our path to getting there but every school will march with their own feet i know that you guys have a very different thing going on here which i think is very special eventually though if it's to be a discipline it will have all it will come down to a set of problems that are the most important ones to solve and it will come down to some methods that are the most important methods to help solve those problems that's what makes a discipline right now you know but i don't we're not there i don't know what they are you know but i think this journey this journey the university of washington was on well before i ever urgent utter the words technology science is a fantastic journey and in fact may very well have a huge impact on the future there are a lot of questions and matt so so so so one of the questions was basically in the world that we're entering the internet of things isn't that like putting this on steroids yes it's like putting this on steroids so so therefore imbedded in that comment there's a kind of sense of hopelessness so isn't the game over right technology one in the story we're now living a different society and the answer is no because we can't we don't know what that society is we don't know how to live in a world where all of the values that people have died for and fought for and so forth just came up for grabs arbitrarily right it's certainly not a democracy anymore what is  and so and that's probably not the way to get us there she usually through social movement and so forth and so that and if we allow social movement it could end up being a revolt in my heart and in my core i'm a computer scientist i love technology so that so for me the answer is i want society to have the benefits of this technology i want to enjoy my apple watch i want to enjoy what i don't want i want to enjoy the sleep number bed without it sending data over to sleep number right and sleep number sends it to so the trick is to figure out how to harmonize technology and society and that ultimately is the goal of this war is to figure out where are the problem areas how do i demonstrate the policy problem areas so that the right communities public interest groups whether they be advocates or whether they be consumer facing government can now perform their functions better and more importantly if we produce a lot of these technologists it will be the companies themselves because no company wants to spend millions of dollars to have their technology disrupted you know intel corp many years ago built the pentium chip many of you were quite young when they did this they had this revolutionary idea that they were going to put a serial number inside the processor that went into the the the personal computers at the time and their reason for doing that was simply they said look if we could track every every packet that left a particular device then if if you broke into a computer or if you did something or another we would through self record be able to know it was you and so that they said this is a great idea and they had and you know for intel to get a chip in to get a serial number into a chip to be able to fabricate the boards to send the boards to microsoft so microsoft can use use those features happen a long time ago at the time the the president and c e o is making this announcement the ships are already sailing from japan and china back to the united states the boat has left the dock he makes this announcement there's a huge public outcry mean just a huge public outcry and eventually intel stock began to go like this and at that point they abandon the project that cost them millions and millions of dollars to this day intel is intel is extremely sensitive around privacy because no company wants to have the disruption so that means that if we produce these technologists the same ones that we talked about going into government the same ones we talk about going into the public interest groups they also need to go and be product managers in the companies because there they can help navigate the technologies very early on away from these kinds of pitfalls so that's another answer to your question you have other questions but i'm going to talk to somebody else now well this is a particularly difficult conversation difficult issue so let me first to level so why this is a difficult issue so often i will say that mark zuckerberg could create facebook in cambridge massachusetts but never in cambridge england and the reason for that is the way privacy laws work in the united states so basically in the united states if you're if you have a neck and body and the privacy law is a little circle that's going to its laws is going to cover your body in the united states should we have two thousand one hundred eighty seven privacy laws they're a little bit he circles and they pretty much back up on top of each other leaving almost the entire body exposed if you talk about privacy in the european union it's like a cloak the drapes the entire body so from a privacy standpoint seems really comprehensive and the other one seems kind of like the wild wild west but if you're doing innovation then when because there's no privacy law forbidding it you can do whatever you think is the right thing whatever you think will sell and it's good until it isn't right and if you're in england and the first things are berg would have to do is go get permission from the data protection agency is it ok for me to hold personal data and use personal data on a lot of other people see it and you know what that would just take so long and so laborious he would just go find something else to do and so this relationship between innovation and our peace mil privacy laws is huge it is one of the reasons innovation is happening here more so than elsewhere so it's not as easy to see as to say we can look like europe because if we look like europe then it's not clear the first of all it has to go through a political process is not what we want it's it's part of what you're asking me what do i think can happen and just politically i just can't see that ever happening i don't ever see a comprehensive privacy policy in the united states unless some huge disaster happens it would have to be pretty catastrophic on the other hand there is the image of the apple watch a different design decision a different way to do something a different market value on privacy technologies that people are thinking about developing in ways that empower the individual giving individual control more and more we see examples of that on the research the graduate students are doing and so i think that these things become game changers for the game changers may have to come through that kind of market lines because right now the e u can cause a bit of obstacles but at the end of the day the e u wants the same technology and so you know it has to be a change on the terms  the technology to be the biggest change in terms of consumers and we can shore up our protections and that will be good and making companies think twice about things we can put new people t companies to help navigate products away from these problems and i think that kind of thing will help and create in a market value that's more user gives the user more power and is more privacy sensitive in the case of privacy i think is another way but comprehensively i think if that's your goal it's a really tough call