this hotel and then the newspapers were put in our rooms the next morning with the big headlines, people started to fly to DC and other places like... to do more important things than having conferences... I was actually there for a paper on FISA 702 and how this issue is raised also from a European perspective.

[3:51'31] I'm going to give a few comments and each of them will have something like a question and then I guess you'll do it like the other time when everybody can just pick and choose, you know... You can take the ones that you feel would add something to the discussion.

[3:51'47] I wanted to start with something which is like the grey zone. This is not my term, this is a term of Bob Litt, who's an NSA general counsel. Bob Litt had this comment in a panel celebrating the first year of Snowden revelations: there has been a long history of cooperative relationships between American business and American government in the interest of protecting the nation and its citizens. Companies have not been asked to do anything illegal, they have their own lawyers, they are pretty good at protecting their own interest but just as you talked about technological gaps that NSA looks to fill, there are legal gaps. There can be an area of space between what is specifically authorized by statutes and what's specifically prohibited by law. And then there is a grey area in-between, where we have been very successful over the years in securing voluntary cooperation. I think it has been an unquestionable loss for our ability to protect the nation if companies would stop this kind of voluntary cooperation".

[3:52'53] My question is: Do you recognize this observation? Is there something like that? Has there been a shift in the willingness of the companies you operate in? I think it definitely connects to the issue of encryption because you can basically think of a scale where you have end-to-end encrypted products, you can call it adversarial design (you design your products in a way that you already anticipate being in this relationship with unlawful access where actually law can't help you). You can also think of a very different stance in designing these types of services or products which you could call anticipatory design where we are going to have these types of services - governments are going to be very interested in that - and organize around that we know that we'll be able to provide the lawful access that will be requested from us. That's my first question.

[3:53'54] The second question is on the differences in legal systems that have been discussed quite a lot already. The difficulties of resolving these challenges have become quite significant because of the height of involvement of courts, specially in this area of intelligence surveillance. You are basically seeing levels of courts involvement that are quite unique. For law enforcement this is maybe not so unique but for intelligence issues I think it is. There is a very sharp difference in legal approach between the EU and the US as regard standing to sue, so basically the ability to bring cases on privacy issues that come from lawful access dynamics.

[3:54'50] On the EU side, there is a very broad ability to sue. Now we have the Charter of Fundamental Rights which basically makes everything that happens to your data a Fundamental Rights issue. What is also legally and politically very relevant is that we have a main court that interprets these provisions in light of the European integration project. That means that the Luxembourg Court is super interested in claiming that it is actually protecting European citizenship and you see this in the number of rulings. So it is also a political project that we are running into in Europe at the moment.

[3:55'37] On the US side, and I am just focusing on the US although we could look at other relations [?], there are very strict constitutional limits on your ability to sue in surveillance cases, and civil society has run into that a number of times, for instance in the Clapper case<fnref n="184" />. So these are, I think, very deep constitutional differences and divides that cannot be easily overcome. Now especially that the Court has become a key player that really articulates these differences very heavily and you see that in the Snowden cases. So my question is: do you believe the current level of scrutiny of lawful access by the courts system, in particular in the intelligence context, is temporary or do you think we will continue to have such courts scrutiny in the future?

[3:56'40 - Alex Macleod] Just want to mention one thing... [about Marc Mossé having to leave at 6:30 sharp]

[Some time during which Joris chooses his next question]

[3:57'09 - Joris von Hoboken] On the legal issues on surveillance law reform, I am wondering what the panel thinks about the need to broaden the conversation a little bit: what types of data are we talking about? We have the tradition of having a surveillance debate which is quite heavily focused on communications, I mean, traditionally those are telecommunication companies and of course we have seen a shift about which types of services are being offered by cloud companies and different types of architecture. I think it makes a lot of sense to keep focusing on communication surveillance but there are a lot of other types of datasets and other services that have quite enormous privacy implications, you can think for instance of the services that are offered in educational contexts.

<footnote n="184">184. Clapper vs. Amnesty International. https://www.oyez.org/cases/2012/11-1025</footnote>
[3:58'05] I am wondering if we shouldn't broaden the conversation beyond that kind of strict focus on consumer-oriented communication services. Just as an example: the laws are typically over focused on those types of services, just if mention could be made of the Stored Communications Act in the US, we have a new Dutch intelligence law, there was the news today that this new intelligence bill will be the subject of a general advisory referendum, which is I think quite unique for such thing as an intelligence act and raises all sort of interesting political questions. That surveillance law has new rules for communication surveillance but it has also kind of hidden, in-between, draconian provisions on access to other types of data systems. For instance like BD access to complete datasets that are held in the private and the public sector and also the possibility to obtain direct access to such information systems holding this kind of data.

[3:59'22] My question is: do you agree that the debate is focusing too much on communications content and related data and not addressing the privacy implications of other services enough and what could we do to bring those other types of privacy issues into focus in the debate so that we move forward on how to design of the laws. Thank you.

[3:59'45 - Alex Macleod] So, would somebody like to start, there? Gail?

[3:59'49 - Gail Kent] I can probably answer two of them very briefly. Firstly I think there is a debate on the difference between them in reverse order. I think there is a debate already on the different types of data particularly under the Budapest Convention of the Council of Europe. Because that comes from the cybercrime perspective, that's not just about communications data, that's also about if you are trying to investigate the cybercrime investigation [??]. I think that has already started but I'd agree it probably needs to go further because all the different companies that are here have different types of data because they provide different services.

[4:00'26] In terms of resolving the difference between different types of law, and the role the courts are playing, I think that's where, it feels to me that there is quite an easy almost cut and paste between what European judgements on surveillance legislation have been and what good standards may be. So somehow the ruling on Safe Harbor and the Privacy Shield actually helped a lot because I think it outlined eight different standards for what good legislation might look like and what the Fundamental Rights Agency look like and it looked at what different European surveillance legislations looked like and one of those was the ability you highlighted to have standards and to go and plead your case. And that's obviously one of the things that came out as part of the Privacy Shield discussions that Maud talked about earlier, the ability to go to some body in the States and claim that something is happening that shouldn't be.

[4:01'35] And then probably the most important one for me is your first question about should we be anticipating law enforcement's requirements in terms of how a service might be used. I think one of the first things I said is that it's part of my job and I have thought long and hard and really intellectually about this and I don't believe that we should. And the reason I don't believe that we should is that law enforcement, like companies, like technology, is constantly evolving or perhaps it should be evolving. We shouldn't be at the point where we just always expect we should be doing things because it makes law enforcement's life easier because that's a pretty weird state for us all to be living in if we're doing it so that we allow or we're enabling better surveillance. So I think that from that perspective it's you know it's a strange angle to come from.

[4:02'29] I think the other thing is that what one of the things that I'm thinking about WhatsApp is that... You know, Jan and Brian, when they founded WhatsApp and when security was a really big part of it, they did it for, Brian for security reasons and Jan, because he came from Ukraine and absolutely understood what it meant to have a government looking at your communication. But even us wouldn't have thought that end-to-end encryption was the most important thing when users were using it. Maybe it was the sort of delightfulness and the ease that WhatsApp can be used was the reason that it is so popular.

[4:03'16] But actually going out and talking to people and doing research and doing questionnaires onto what actually matters to people is no, that end-to-end encryption really does matter to people. It matters to people that for the first time they can have private, secure communications that used to only be available to governments. That level of security matters for those that are communicating with doctors. It matters for people that are using it for business purposes, and there are a lot of people that do use it for business purposes including politicians, that matters to you know husbands and wives having conversations.

[4:03'55] And even if I think about using it myself I like that level of assurance that it's not available to anyone, that nobody can get that data. And I think if we start exactly the same sort of like putting a freeze on what you can do in encryption, if we start limiting what companies can do for technology reasons then we really really, we're shooting ourselves in the foot and we're stopping great innovation and technology that will probably improve all of our lives.

[4:04'27 - Alex Macleod] Thank you. Is there anybody else on the panel? I that case I would throw it open to the floor. Is there somebody that would like to make a comment or ask a question to all members of the [panel]? Yes, please.

[4:04'50 - unnamed participant] My question is about e-evidence. I am wondering whether disagreements or tensions
might arise member states and the Commission and Internet companies because when I listen to you I understand that you all want to have a framework. So where are the disagreements?

[4:05'10 - Acadia Senese] I'll let my European colleagues take that one...

[4:05'18 - Marc Mossé] To know the disagreements we should know what the proposal is. So we do not know what the proposal is yet but the point is... I am not trying to escape the question [laugh from several people]... My point is more simple here. What is the efficient way to get access to evidence in terms of public safety that protects privacy? To do so there is a mechanism of mutual recognition that we know pretty well in Europe as a mechanism of cooperation between the member states, which means that, if a request from a member state is fully compliant with the law of this member state, it should be welcome in the member state that will execute this order. It's a mechanism which is pretty simple when you describe it, but it's a little more complicated to put in place. This mechanism of mutual recognition is one of the angles. The other aspect is, as we discussed earlier in the panel, when it's possible the government should go first to the customer, rather than to the company, just in order to avoid to put us in a difficult situation most of the time.

[4:06'46] It would be easier to get the data from the customer if there is a need to do so. If not, again, it should be done in a way that both legal systems of the states, which are the state asking for the data and the state where the data is stored, are agreed on the legal requirements. This is where the mutual recognition mechanism is maybe, I would say, the most effective one. Again, we have seen some lines drawn in the non-paper that has been issued in December 2nd, 2016, if I'm right. We'll need to see further details to be sure that it works well, but again what we should absolutely avoid is a mechanism where service providers are confronted to a sort of contradictory injunctions. We should be in the position that if we agreed on the request, we respect both legal systems.

[4:08'02 - Maud Sacquet] Perhaps just one point. You would be surprised how much, even though 28 member states can agree on a framework, they can then disagree on the way to think. That's often why it takes so much time to actually get any kind of legislative acts in Brussels. It's fascinating to see on which issues you have some coalitions between some countries and in another issue the coalition changes completely. On this point, we haven't seen the proposal so it's a bit difficult to say but you have to think for example that in some countries law enforcement can get content data while in some others they can't. There is very important difference into what kind of data and mechanism that works in every system and so I'm guessing that where the blocks will come in the definition is especially around this issue that what can get in every country and should you as, I don't know, an Irish police officer be able to hand more data to your French colleague about an Irish citizen than you would be able to do under Irish law. Very practically, this is the kind of issues that I'm guessing will be discussed.

[4:09'22 - Marc Mossé] We have seen already something working with the EIO, which is the Executive [in fact European] Investigation Order. This is a directive from 2014 or so, that should be now in force in 2017. So it can work, this is mutual recognition mechanism as well. Just think about mutual recognition as a way to ensure that both systems are respectful of the principles of each member state. You have to be sure that when I'm acting under this legal system I am sure that this request will be totally compliant with the legal system of the other member states. This is where there is still room for negotiation. 28 -still 28- member states can work.

[4:10'27] There is still another option, which is the enhanced cooperation, if that fails with 28 or 27 member states. This is one option, it exists in the treaty, nothing very extraordinary, and we know some possibility: Schengen is an enhanced cooperation, the EIO I think is an enhanced cooperation as well, so there are different options that could be decided in the coming months because again, following the report of December 2016, the proposal could come in 2018 or beginning of 2019 if I am right.

[4:11'08 - Didier Bigo] It was just a remark about what may be the case if you remember in 2005 how people were so pleased to succeed to have this agreement between the EU and the US about the legal agreements [probably "assistance" or "cooperation"). But what happened was that the NSA bypassed completely all law enforcement agencies. So we have discussed, and that's crucial, about law enforcement agencies and their practices but we still have - we'll invite you again - on foreign intelligence and how the protection given by the Fourth Amendment to their own citizens in the US is not applying for others, the foreigners (for them, they are the citizens of other countries). And we know that, as long as we have this kind of elements it will be very complicated. I think that at the EU level it's almost the same kind of understanding.

[4:12'35] So I was surprised that the question of the distinction for you, in your practice, you have spoken a lot about the requests which come on crime, or on evidence of terrorism, but not so much about the large data where you have no warrant, regarding suspicion, on different case, the way it has worked through different possibilities and the UN... I was surprised that this has not come. So it's an invitation, because we will have three conferences. Today was the first one, we will have a second one with more discussion about what is going on at the UN level and we had a lot of expectations, two or three years ago, regarding that, but that has disappeared. Or what is going on with, now, the CJEU and EU-Canada PNR and what are the facts now with EU-Canada PNR that the definition of privacy is now changing,
at the same moment there is the e-directive. And we will have another one in almost a year, in September. I hope that most of you can come back, we'll have a couple or three days regarding that, where we will discuss also other people, but I thought it was really crucial to have all this discussion in details, for the law enforcement and the way you have framed that. It was important to say that before Marc leaves for you to have organized [to come here] and to have spent so much time. It was more a compliment than a question as such but also an invitation to come back, for most of you.

[4:14'58 - Gail Kent] Can I just say that? I think that you were suggesting that companies were providing bulk access for data and that's not true. So there...

[4:15'05 - Didier Bigo] I was not. I was referring to the case which was in 2013. The way it has been organized. So with PRISM and with the different activities, you have, willingly or unwillingly, distribution of the information by the private companies. So we can continue about that, no or yes, it will be one of the elements and that's why that's very important to come back to that because in 2005 the agreement was that the US could have passed through all the legislations that it had at the time. So it's not a blame but it's the way it has been done at that time and we may come back to that because you have different views about what happened at that time and what is Snowden is a question of foreign intelligence and not using bulk because you know that bulk signifies something different. So we need to come back to what happened and how a form of oversight, and not a national oversight, could create the conditions for this - I don't like the term balancing - but this demands constraint to have a high level of privacy and nevertheless to have safety and security. The terminology of balance is a very US way of thinking, or UK way, you know that the European Court of Justice for example asked for a very different thing from balance. We can continue on this later.

[4:17'10 - Marc Mossé] I am sorry because I have my train... You have re-opened a very interesting debate that could take a little bit more time... just two things: on the "balance", I use often this word because it's maybe a bad translation of my constitutional background which is in the jurisprudence of the Conseil Constitutionnel we talk about conciliation, which is a way of balancing both principles but at the end of the day I come back to one very important principle, which is: there is no hierarchy between the Fundamental Rights: you always have to balance those in a way that you respect the constitutional heritage. This is how works the jurisprudence of, I would say, most of the courts across the world.

[4:18'02] I have a third book for you, which is called "les entretiens de Provence", written by 4-5 judges from different supreme courts. In fact, it was Robert Badinter, Chief justice in France, Stephen Breyer and Antonio Cassese, another judge, that spent one week all together talking about how supreme courts take a decision, not only how they take the decision itself, but how they go to the decision, what's the process, how they interact intellectually to get the decision. What you can see when you read this book, which is very interesting, really, with this dialogue between judges from supreme courts from all over the world is really how you can, how you have to balance between those fundamental rights.

[4:19'09] But, again, it is a very interesting debate and I would love to continue on this and, I have seen Gail react, we do not give any bulk of data as a principle, this is why this New York warrant case is so important, because beyond the grey zone we want to get clarity. We had a singer in France - you do not know him he is Eric Charden, maybe just a few here know him - who had a song which was "the world is grey, the world is blue", so maybe we went from a grey world and going to a blue one and I am going to [get] my train, thank you.

[4:19'54 - Alex Macleod] Immediately I want to thank our panel, which have given us their time and their energy, so thank and have a safe comeback.

[4:20'11 - Félix Tréguer] Thank you very much for this wonderful discussion, now we'll be moving to the concluding remarks, so we'll give the floor to Ron Deibert from the Citizen Lab at the University of Toronto, for ten minutes, and then we'll hear Mr. Thierry Delville, who is the ministerial delegate for security industries and cyberthreats at the Ministry of Interior, in France, the so-called "cyberprefect".

[4:20'35 - Ron Deibert] OK, hello everybody. I have the unenviable task of going at the end of a long day with very interesting discussion - not as enviable as the actual last speaker although. People were asking and talking about their Snowden moments so I might as well say mine... I published a book called "Black Code", which is about very much the subject matter about surveillance, in 2013 in fact the very day that Snowden fled to Hong Kong. And you know that could have been a disaster. But ultimately I think it worked in my favor and I didn't get too much wrong.

[4:21'14] I would just like to say a couple of things. Maybe what was overlooked mostly. And to that end I'd like to begin with what I think was an obvious thing that wasn't said right from the outset and that is what is it that we're talking about here in terms of the services that these companies provide and why we're in this situation to begin with. While the panels were going on I googled the mission statements of each of the companies that are up on the table. So, Facebook as I think it still "bring the world closer together"?

[4:21'48 - Gail Kent] We just changed that: "bring communities closer together".
[4:22'01 - Ron Deibert] Google: still “organizing the world's information, making it universally accessible”. Microsoft (I couldn't believe it): “be what's next”. Is that actually? That's why he [Marc] left because it can't be real. But you know, I think these mission statements I understand why their mission statement may actually obscure what it is really that these companies do. And I think it's important for us to define it from the outset that we live in a personal data economy. And these companies provide the infrastructure for commercial surveillance and behavioral engineering. That's what it is about. In spite of the mission statements.

[4:22'38] And then I think it's useful when you reflect on that for a minute and what that means, especially for private sector-government relations, to ask yourself what is the mission statement of states. There are many definitions we could go to you could think of Max Weber. But since we're in France and I'm here with Didier Bigo, I think I'd go to Foucault and say “the state is the organized practices through which subjects are governed”. Is that right, Didier, would you agree? More or less? So when you compare those too, the infrastructure for commercial surveillance and behavioral engineering to push advertisements and profit ultimately make money and then to the organized practices through which subjects are governed, you can see there's a convergence of interests at a very fundamental level here.

[4:23'27] This is where big data meets Big Brother and that's the crux of the issue of why there is such an important connection here for governments, for states, for law enforcement, for intelligence. What these companies do is an irresistible proxy, an attraction for social control for everything that they're interested in doing: that's part of their mandate and that's why we have all of these issues about what is the relationship between the two.

[4:24'00] And I thought it was interesting that Mark said that the rule of law is our North Star and I was glad to hear Gail say “Well, that's true but there are some some complicated issues around following the rule of law, especially when the context is different”. Of course, the rule of law varies country to country, and if you think China, for example, to state the elephant in the room, they have a new cybersecurity law (I think it went into effect in June of this year) that requires companies to host data locally to effectively police all of their users.

[4:24'36] At Citizen Lab, we've been doing extensive research on China-based mobile applications where we reverse engineer them or do experiments to understand what type of filtering or surveillance they had baked into their applications or that are done on the server side by the companies. And what we find is of course quite astonishing that there is deeply baked-in censorship and surveillance because they're following local laws in China. This is not so much an issue yet for Google or Facebook of course, although I did watch with interest the news about Facebook in discussions with China, eager to enter the market even going so far as to develop some kind of algorithm that would satisfy these requirements.

[4:25'24] So it's important to keep that in mind that there is this variation happening. And then also the trend lines: most of the discussion was about compliance with US, UK, European law. But there is a major demographic shift happening worldwide right now. The vast majority of Internet users, and hence the customers of these companies, are coming from the global south, now and certainly into the future. The sad reality is that most of these countries are not democratic countries. Most of them are authoritarian or hybrid regimes, sliding back into autocracy where the rule of law means something completely different.

[4:26'06] At Citizen Lab, we've just done several (I think five) consecutive reports where we've unearthed commercial spy were sold to the Mexican government, used to spy on journalists, activists, human rights defenders and so on. No law was broken there at all. NSO group, the Israeli spyware company that supplied this technology to the Mexican authorities, rightly claims that they follow Israeli and local law. So the issues that we're talking about here I would have liked to have seen been discussed more with the horizon in mind about what it means down the road.

[4:26'43] The other thing I'll bring up is around fake news and disinformation. I am surprised that topic didn't come up at all considering that both of these companies, I think today Google came out with the news of both the advertisements and Facebook did before that. Again here there's a convergence in a really odd way, I think a troubling way, which is where the company's revenue model, which rests on pushing advertisements and surveilling users in a very fine grained way. And I recall the Facebook experimentation that was done a few years back on manipulating people's emotions. This is actually fertile ground for political manipulation and disinformation and psychological operations.

[4:27'35] And I think we're going to be seeing soon a real problem around, that not only for the users but for the companies in terms of how they deal with that issue. And I saw Facebook today just released a post about how they will not have their platform used for the manipulation of elections which is commendable, but it makes me think maybe it's like a Band-Aid over you know flesh eating disease that maybe these companies, given the nature of their their platforms and how they operate at a core level, are really amplifying a kind of tribalism. And that really can't be escaped because of the nature of the way in which advertisements are pushed to siloed communities and increasingly will be manipulated in the future.
[4:28'27] The last thing I want to bring up was around targeted digital attacks. Another subject that didn't come up today when I think about security issues in this area, it's the one that I feel is most pressing, most urgent. At Citizen Lab we've been tracking what we call a silent epidemic of targeted espionage against civil society groups, human rights defenders worldwide. A lot of this campaigning takes place through the platforms of these companies through the misuse of their applications. To their credit, all of them, I think the ones that are represented here on the table want to do something about it and have taken some steps to do something about it. But I do believe that more could be done. One simple thing is requiring two-factor authentication by default.

[4:29'17] Now it's interesting to think about why that hasn't been done yet. Banks do it and I think it has to do with a principle: when companies are left holding the bag because their infrastructure is misused, in a certain way then they're incentivized to do something about it. But when users are left holding the bag the companies are incentivized to do something and I think that's the case here, unfortunately. So I hope steps will be taken to rectify that in the future. I would end it there, I really enjoyed the discussion and thank you Didier for the invitation.

[4:30'43 - Thierry Delville] Good evening everybody. I am Thierry Delville, the French delegate for the Ministry of Interior in charge of industrial security and cyber threats. Thank you for the invitation, I suppose I am the last to speak, so I will be short. I am very happy and honoured to be invited to speak in this chamber with so many researchers and experts. I recognise Mr Mossé, we were working together on different subjects. As a representative of the French Ministry of Interior I am aware of the sensitivity of such a subject, which refers to the very foundation of democracy whose strength is to know how to manage permanently the difficult balance between preservation of our individual freedoms and defence of the safety of the same individuals.

[4:31'46] Studying the techniques of interception and analysis of communication, the use that the services and agencies make of them, and the political and legal controversies they provoke seems to be at the heart of the reflection you are addressing. As well the analysis of the reconfiguration of the contemporary logics of surveillance wants to redefine the limits of democracy and questions to the sovereignty of states. (That's a good subject that you are studying). Obviously my remarks will not address the substance of all the issues that have already been or will be discussed at the next session, but allow me to give you some general thoughts on the evolution of security and the place of the states, I mean France in particular, but it will remain valid for most European democracies and countries as well.

[4:32'42] Digital transformation is at the heart of the society's major challenges: big data, clouds, smart TVs... we are speaking about smart and safe cities. All those developments lead or have led to the development of new economic actors (some of them are here, I suppose), who have developed and are able to work themselves on research and development, without external help. This is a big difference with the whole industry: unlike the pre-existing model, in the military shield for example, a large digital industry exists and developed without necessarily resorting to state budget, and that's a big difference in terms of relations with them. Interception tools and law - I will briefly make a shorts review of the context in France - the techniques of investigation in the digital age are seen as interferences in private life but they are necessary to characterize offenses, to protect the economic interests of a country and therefore they legislate or frame them very precisely.

[4:34'03] The use of interception of correspondence [through] the electronic communication or phone tapping is a procedure frequently used in criminal offence proceedings. The cases and the conditions under which such communications may be intercepted are provided for in different articles. Maybe I won't list the code of criminal procedure but you know the law, I suppose, the 1991 important law following the condemnation of France by the European Court of Human Rights.

[4:33'38] In practice, the interception of correspondence is carried out by requisitioning by any qualified official, service or organisation under the authority or supervision of the Minister of interior in charge of telecommunications, or any qualified agents of a network operator or access provider. The requisition of electronic communication operator put in place technical elements to intercept communication and collect data on their customers. Government, on the other hand, must have the technical tools to receive and decode this data. Nowadays [it requires] digital and human resources to exploit them.

[4:35'21] The legislator has intervened in many many occasions to shape the regime applicable to the judicial interception. There is a law with important modifications and evolutions in 2004. And the French legislator has adapted the means of the interception in the digital age with procedures very framed legally. It may also be geolocalization form of digital surveillance, infiltration, data capture with the aim of recovering numerical indices, enabling the compilation of indices, with which we subsequently characterize criminal offences for example.

[4:36'10] In the administrative or intelligence field, the legal framework is different and it is necessary to underline the recent evolution introduced by the law in 2015 on intelligence, which creates a the CNCTR. I suppose you know this organization, an independent administration who advise on their requests formulated by an internal ministry, like defence, justice, economy or finance, for customs or for border control. It's made mandatory prior to the prime minister
decision [?]. That's the context in our legislation.

[4:36'56] In a second time, I would like to speak about the interception tools and the digital revolution. Digital transformation is apparently a bargain for foreign intelligence or law enforcement agencies as a multiplication of sensor and the storage of the information in the servers or in the cloud and different things that are likely to be able to answer the needs of an investigation.

[4:37'23] In fact, several factors make today's task of the state services less easy than it seems. In reference to what happened a few years ago: a few years ago, in a number of cases, we are witnessing the development of a genuine capacity asymmetric between the means available to offenders and those used by the representatives of the rule of law. So development of end-to-end encryption is just as much a security factor for the confidentiality of information, it is increasingly exploited as a tool of concealment by large criminal organizations.

[4:38'03] The issue of the location of data and the legal status of the data is a considerable issue. Extraterritorial issues are very much [occupying?] the debates and legislative development and make it increasingly necessary to establish bilateral or multilateral agreements to facilitate information communication under the control of judges. So time is short for terrorist investigation or organized crime cannot be satisfied with the long communication time which can sometimes lead to a response after three hundred have died, to have an answer to our request.

[4:39'03] Legal interpretation specific to the major players on the Internet with whom we work at national level in particular I run actually quarterly like this [?]. We created a group two years ago, after the first attack in 2015, and we are sharing information with GAF A and telecom operators and their correspondents in France. We have also the same information-sharing at the European level and we see clearly that on the communication of data certain interpretation, linked for example to the difference of interpretation of hate speech, exist really between us and the operators, and sometimes between operators themselves.

[4:40'00] [It's our position?] that the technological development of communication tools that respond to strong economic imperatives should not take place within a framework that cannot be regulated or simply apprehended by the state services. The nation, as in the case of a terrorist threat or a flagrant crime, especially when the life of a person is at stake, it is essential that all means of interception can be activated within the framework of the law and regulation in force.

[4:40'35] In order to achieve lasting results, it is important that all economic actors, starting with industry, telecommunication regulation bodies, the agency responsible for their control and certification of tools should be committed to these requirements just as strongly as in the success of the new product line. This dialogue, if it exists, is nevertheless a major challenge to be [?] for the next years.

[4:41'08] I would conclude my remarks on the issue of cyber security interception. We have different concepts like data capture, the development of private offering of tools that are real tools for spying and capturing information remotely, is today a particularly important challenge. This issue of cyber defence is discussed at the same time as actors manipulated by states seeking to destabilize targets by this means. It is also about cybercrime, with actors who can sometimes be the same and who use the same tools to attack targets for the more directly lucrative purposes.

[4:41'51] The new reality that I can see is supplanting the old and the disappearance announce in a more or less short period of time of the classic currencies to the profit of the digital transfer or even the crypto money must encourage the states to wrestle in their way of fighting on equal terms against these new forms of delinquency. The challenge is, on the one hand to encourage companies and individuals to protect themselves. This is the meaning of the Military Programme Act of 2013 (I suppose you know this law) for operators of vital importance.

[4:42'35] This is also the meaning of the NIS directive which will operate next May. This is this also the meaning of the GDPR EU directive [not a directive in fact] that will allow as May 2018 to strongly condemn any economic actor who will not protect this personal data. On the other hand it is important that states organize themselves from what constitutes a new field of [maneuver?] of defence and security are formed more than in the physical world in the logic constitute a continuum and the technical expertise and ability to stay in the race are more challenging everyday. Thank you for your attention and I wish you on this exciting topic an excellent work.

[4:43'45 - Didier Bigo] Do you want to add some final comments? Or have we exhausted the topic? It was just a chance maybe to have answers to some of these elements that we have given.

[4:44'00 - Gail Kent] One very final sentence. We have seen in Facebook that we are 1% done. I don't think that any of us think we have all the solutions there, this is why this sort of conversations are really important.

[4:44'22 - Acadia Senese] As a follow-up on my very first answer, I think of this day, which was, I think, hours ago
now, but I was asked about the Snowden disclosures and I said my colleagues were quite surprised at the disclosure and
frustrated with the government's failure to correct the inaccuracies. My fear is that those inaccuracies have persisted.
And you know I think it's important that I say that you know Google doesn't do bulk, doesn't respond to bulk request for
data. For its part during the Snowden disclosures Google's involvement was responding to compel legal process. So
with that I think will be a segway to the future conversations that we have here. But I did want to end on that.

[4:45'07 - Didier Bigo] I have seen that it was also the next just for the people who had the courage to stay, I mean all these elements, we'll have two other conferences. One quite soon, which will continue on what are the forms of activism which have existed, what are the possibilities on encryption-decryption logic, what has been called for example countersurveillance. Why does that mean? Does it means something?

[4:45'53] For certain people it's a strong belief that it means something. For others it's almost “we don't see the difference between surveillance and counter surveillance. Maybe it's just a loop”. And that's one of the questions. We will have also, and maybe it's interesting to have all your voices before to have that. A lot of discussion about the different legal actions. We'll have a couple of lawyers coming from Privacy International. We will have all the people coming from the US and [?] and so on.

[4:46'41] I think it's always very important to have this first panel, because we have found your position, despite the diversity, was very strong in terms of saying the same form of narrative about what a private [company] have done, what they were obliged to do, what they have never done, and you insisted. Some people are saying “yes they have done” so we'll have a final conference on September 2018. So we are trying also to investigate more about that, with two or three days.

[4:47'23] Regarding panel with private actors, I hope that you will come back. We'll also have persons coming from activism and also persons who are in charge, at the of policy level, to intervene, also at the EU, at UN level. So we will have national, I hope, we are eager that you will come back also, we will have these two days, three days if we are lucky, but two days where we'll have this simultaneous discussion. The idea is really to try to, as I say, to deconstruct the stereotypes, to put more elements about what are the evidence people have.

[4:48'21] Could we have evidence based on this logic? Because we have seen that very quickly [we] can come back to belief. So what are the elements that can be put on the table?
We know that we will not have one story, it's also constructed depending on the positions that people are occupying.
Stories are complex but nevertheless I think it's very important.

[4:48'53] The second idea is nevertheless true. I think it's something most of the actors agree - not all. What are the conditions under which it is possible to have a better oversight? And the way to construct a bridge where this idea of trust will not be claimed because, it will be obvious in practice. Because the more you speak about trust, and that's really one of the law of sociology, it's because you are obliged to speak about trust, because nobody is trusting. It's about distrust, the practice is distrust, so you speak about trust. You never speak about trust as alone. If you have never divorced you don't understand what I am saying.

[4:49'44] The discussion about trust is always a question which will be put in practice, between the reluctance of people we'll see [many doubts in this sentence]. That distrust is the practice and we have seen a lot of discourse about trust. The discourse is not the proof that it works better or it will work better, it's because you have a problem and we need to come back to the roots of that.

[4:50'18] When we have MLAT, for example they are used. They are not bypassed. And that's also one of the key elements that we have to discuss certainly more to build at least some legal framework where the idea is that the legal framework is there to avoid to go in depth into practices, but that it could be operational and limit some of the elements about the form of intrusive surveillance which are not into this legal framework.

[4:51'02] So that's the general idea. I have tried to put it in a nutshell and I hope everyone will be sufficiently interested to come to the other conference. So we'll have the conference in December, we will send you more elements about this one soon. So it will be on the 18th of December. We will have the other one in September 2018.

[4:51'40] In the meantime we have also a running seminar with researchers and PHD students where, if you are interested, you can also register. You can come in. Thank you very much. Thank you also for the organizer because I've spoken a lot but all the work has been done by Félix Tréguer, who has really organized everything.