From: OpenReview.net/Anonymous_Preprint/Paper346/Authors
title: Thank you all!
comment: Thank you for participating in the discussion. We will now disable public commenting, but still leave the opportunity to leave comments just visible to the committee who prepared this draft. Simply login in OpenReview, write your post, and set "readers" to "OpenReview Anonymous Preprint Paper346 Authors". Thank you!

From: ~Ben_Zevenbergen1
title: Comments on the text
comment: This reads as a good start. Below I've added some comments on particular sections. Overall, I worry about two things:
1) The choice of topics addressed appear to be a snapshot of the topics currently popular in discussions around the social impacts of AI technology. The topics are important, though I wonder whether a more general approach would better stand the test of time.
2) I wonder if the document has the support and thereby legitimacy of the community. Judging by some comments in these threads, I doubt that's the case right now. While the conference is free to impose any rules it wants (within reason), it would be shame for this effort to fall short of expectations because those governed by it don't recognize it as a legitimate guide. I'd be happy to help think through a process that could increase the legitimacy and support.


Specific comments:

“With such great opportunity to impact the life of people comes also a great responsibility to ensure research has an overall positive effect in our society.”
- It would be useful to make a stronger connection between impact and responsibility. Right now it reads more like an opinion that others might want to disagree with. Further, how do suggest we understand "overall positive effect" on our society? Some might claim that the benefits of their work will, over the long term, strongly outweigh any negative effects of its production today. This type of clarity might/will be necessary to gain support from the community.

Also relevant for:
“First, AI research is worthwhile only to the extent that it can have a positive net outcome for our society."
“Second, since the assessment of whether a research practice or outcome is ethical might be difficult to establish unequivocally, transparent and truthful communication is of paramount importance.”


“[…] higher standards of ethical conduct”
- This is a little pedantic but not unimportant: some reference to what the standards of ethical conduct are would be helpful. You could link to professional guidelines in other fields as an analogy, to illustrate what you're working towards. 


“For example, any new dataset making use of human information must provide a detailed explanation of why it was necessary to do so, including if there was an assessment by an institutional review board (IRB).
- What is human information? Only personal data/personally identifiable data, or more? If left ambiguous and broad on purpose, this will confuse people and lead to different standards.
- What about when data about people’s devices is used, is that human data?


“some internal review process, during which researchers reflect on ethical considerations prior to the design and implementation of the research project.”
- What do you envisage being reviewed here? A review process is typically/inherently at the end of something. If it’s prior to a design and implementation stage, it should probably not be called a review process, unless there’s something concrete to review. 
- Further, the word “some” leaves much space for what this looks like. In the current working, a person working alone could sit down for 5 minutes and reflect on their work, agree it’s fine, and continue (if other processes are not available to them)


“Ensure research topics, activities and methods minimise direct harm to the environment.”
“3.2.7 Environment”
Would be good to cal' this the 'natural environment.'

“Researchers should consider, for example, whether their research is going to negatively impact the environment by promoting fossil fuel extraction, deforestation, or by increasing pollution.”
- There are many more topics that could be included here. For example, Animal Welfare is an important topic, both in the natural environment and for the use of AI systems in in farms of various scales.

“For data that is already publicly available and without any copyright claim, researchers should be considerate of the subjects in the data and their privacy concerns.”
- The copyright section is relatively very long and should be shortened/more concrete. This particular sentence should be moved to the privacy section. 

“Researchers are expected to take a subject-based approach to consent […]”
- The consent part might need its own section. These considerations are relevant for other types of subjects of consent, too. 

“A.4.3 Social considerations”
- This section is a wall of text that looks like legalese. Would dit not be worth making this topic less intimidating and easy to read, especially for an audience that is not inherently familiar with the topics mentioned?

From: ~Martin_Hofmann2
title: Agreed
comment: I think the original author of the comment wanted to stress that enforcing moral requirements is the wrong way. Not that ethical discussions should not be a part of scientific communication for engineers. In my opinion, the Code of Ethics draft would be better if it stresses the understandability of the artifact and reduces the requirements for ethical elaborations of the authors. Let us go for overall understandable science with open-minded contemplation without the "pretence of knowledge."

From: ~Martin_Hofmann2
title: Misconceptions
comment: Thank you for this intense discussion.

Only referring to a written codex of NeurIPS grammar rules would prove their existence. What was indeed referred to were codices from outside NeurIPS. 
But, why not tell reviewers explicitly that they can and should review for morals? Society has plenty of moral guidelines. Morality is inconsistent worldwide and so. One might think something is right or wrong because one belongs to a particular society; why should one's view be appropriate? Democracy is our one and only way to decide what is right or wrong. Anything else will inevitably result in might makes right. That this is true is easy to see in the dialectic of the last comment, where an argumentum ad verecundiam is used; an argument that is usually used to avoid meaningful arguments and force down views.

Please don't get me wrong. I appreciate the comment. What I intended to stress were the requirements on what is morally acceptable. I want to say that this is a hazardous statement because this opens doors to the dark ages. And as I pointed out in my other comment, there are at least two different aspects of an "artifact" or "piece of science." One aspect is what science is allowed to sacrifice for progress. This question is easier to answer. But even here is no clear moral decision by ethics. One other aspect is the probable impact of an artifact. Writing down eight pages of maths makes the broader impact so ambiguous that nobody can ever discuss ethical implications. That's my point. I don't want to see an author's own opinion on their cryptical convoluted artifact; I like the author to clarify the matter of the artifact and decide by myself what the ethical implications might be. This needs a clear and understandable style from the beginning to the end (except for a statement about exploitation, safety, data use, etc. ...), not just a statement at the end. 

I agree with the comment, but in the end, I would rather see understandable artifacts, not only ethics statements (except for how the artifact was created).

From: ~David_Manheim1
title: Standard English is far better documented than any version of ethical standards for technology and ressearch
comment: First, I think you're mistaken in claiming that there is no clear minimal standard for grammar at NeurIPS - it's just so well agreed upon in almost all areas that it's invisible. I'm guessing that there isn't standardization on edge cases - Oxford commas or not, or perhaps even British or American spelling conventions - though journals do have more explicit rules - but reviewers are explicitly told they can and should review for language. But we don't have similar standards for ethics, so instead of being able to point to Strunk and White or the New York Times style guide, it makes sense to spell out what is expected. 

Second, appeals to democracy are incorrect as a matter of ethics. Something isn't ethical because of a vote, and as you pointed out elsewhere, individuals have moral responsibilities. Yes, having engineers decide things for others is a real problem, and participatory decisionmaking is an important tool for governance. But to be clear, these standards aren't being drafted by engineers working alone, legislating rules. First, Iason Gabriel, who helped draft the rules, is a well-regarded academic ethicist and political theorist, currently focusing on ethics in machine learning. Second, the process we are currently participating in, of open review, allows people with backgrounds in public policy, ethics, and other topics to weigh in on the standards. And third, the bulk of what you seem to object to aren't ethical standards for what to allow, they are standards for what authors should discuss. And as you said in a comment below, "[Talking] about the consequences of a piece of science, on the other hand, is pointless without a proper understanding of what this piece is about," and so having authors with exactly that proper understanding be the people to start these discussions seems not only acceptable, but critical.

From: ~David_Manheim1
title: Science is a social process, and so are both norms for accuracy, and norms for ethics
comment: I agree with you that people have incentives to mislead others, and reviewers obviously need to be aware of that. But there isn't a tension between the need for accuracy and the need for ethics! Both are part of closely related *social processes* of review. I agree that " there will be a moral judgment on everything that is discussed," which is exactly why clarity about standards is needed, and I think it what this type of document provides.

The alternative to explicit rules about ethical standards for what should be accepted isn't a focus on only technical accuracy and clarity, it's unclear and unevenly applied rules. That said, there are legitimate debates to be had about what those standards should be - and this conference is certainly not where political and social tensions around inequality, silencing debate, and governance of technologies will be settled, but burying the debate inside of opaque and arbitrary reviews is unfair to authors. 

Yes, discussions of morality will involve subjective opinions and debatable claims. But the proposed standards do not suggest that technologies which are likely to be harmful should be banned from the conference, they require discussion of the questions, and more engagement in debates about the "the consequences of a piece of science." So I clearly agree that the debates are useless "without a proper understanding of what this piece is about." But that is exactly why Yann's original claim, that the conference should avoid these discussions, and as he said in a different comment, should leave the decisions to "governments and international institutions," without a technical understanding, and worse, who are organizationally and legally incapable of making decisions about what is or is not unethical, is so troubling to me.

From: ~Martin_Hofmann2
title: This discussion is arbitrary
comment: I may have to learn how to write clearly. I meant that if I had invented the world-destroying killer drone making me the universe's leader, I would tell you the exact opposite. And if my work is trivial, I would write things in that statement that say to the reader that it is an important matter for humankind. On the other hand, significant inventions imply an infinite variety of ethical aspects. These reasons make ethical statements an arbitrary point. One can write anything into it, and there will be a clear incentive to cover the true ethical impact for many more than the reasons I gave before. We should, therefore, not enforce high-quality, ethical statements but clearly and understandable written papers. 

Engineers are moral beings, and so are the reviewers. So, there will be a moral judgment on everything that is discussed. This discussion is a sign of morally acting people, and it is good. And it may be good to have a moral codex that forbids exploitation and harmful inventions. But these are, again, technical definitions. And I have no problem with moral limits like exploitation and direct harm to experimentees or the exposure of private data. These are clearly things that belong to how a piece of science is created. It's the question if we want research to do immoral things for progress.

Talking about the consequences of a piece of science, on the other hand, is pointless without a proper understanding of what this piece is about and engineers covering up whats possibly a problem.

From: ~Martin_Hofmann2
title: There is no codex for 'minimal requirements for grammar'
comment: There is no NeurIPS codex for '"minimal requirements" for grammar,' and accordingly, there is none needed for ethics. Ethics do not decide what's good or bad; there are no such categories. Ethics try to look at all possible aspects of a matter and analyze what concrete consequences they might bring. What people here mean is morale. You want 'very minimal requirements' on what is 'morally' acceptable. The moral is not a matter of engineers, but one of the politicians, and we have democracy to decide on that, not scientists. And it is very good not to have engineers decide how society should work.

From: ~Harrison_Zhu1
title: Thoughts on a few potentially contentious issues and lack of diversity
comment: I think this proposal is a good start with some reasonable propositions, such as suggestions to mitigate risks against fraud, malicious actors, copyright etc. Although, I agree with many other comments mentioning that a lot of what is stated is “wish”, mixed with a few mandates, and should be communicated more clearly. I also agree that certain views, such as from other parts of the world or demographics, may have been neglected. The general feel of the proposal seems to reflect (I may be wrong) the ethical values of the mainstream research community in the US. Considering that the NeurIPS aspires to be a global community, my impression is that this document may seem too US-centric.

For example, section 3.2.7 on the environment, as a global community, most countries recognise climate change is a serious issue, but the economic lifeline of many countries are still fossil fuel or pollution-intensive industries (e.g. semiconductors, manufacturing of electronics etc…). I agree that research papers should explicitly mention these climatic drawbacks (perhaps in the ethics section) but it would not be reasonable to discriminate against or reject papers on these bases. This may have the counter-productive effect of discouraging researchers from these countries from participating in NeurIPS. 

The last sentence of section 3.2.8 seems quite broad and I am worried that it will be quite divisive, especially “privacy” and “speech”. On privacy for example, facial recognition in the public is widely adopted by law enforcement in many countries, but it is not legal in the EU in most cases - would a paper on facial recognition algorithms in public domains constitute to a violation of section 3.2.8? For speech, an example would be fake news or hate speech detection algorithms. Censorship of hate speech is legally enforced in many countries, such as the UK and China, whereas it would be a violation of the constitution in other places such as the US. These are complex political and cultural issues in which every region has a different opinion on and it would be best not to have "one rule for all". 

That being said, I still support the initiative of ethical disclosures, even if it means disclosing something that the authors may not agree with. For example, it could be structured like “We identify this as an ethical risk in regions X, but not regions Y”.

From: ~Marc'Aurelio_Ranzato1
title: some thoughts on trade-offs
comment: Thank you for your feedback.

My understanding of your comment is that there is a clear trade-off. On the one hand, we want to be very clear and actionable in our asks. On the other hand, ethical judgements are certainly nuanced and context dependent, as you mentioned in your example. This is precisely why we opted for erring on the side of being "vague" and instead of providing a list of mandates, we provided a mostly educational material. The draft proposes axes of ethical evaluations and only asks for a) reflection along these axes and b) communication of such reflection.

Based on your remarks though, it does seem that the current draft is not sufficiently clear and that we need to restructure it to make clearer the distinction between what we require and what the axes of assessment are (letting reviewers and program chairs set the bar).
We'll definitely take your feedback into account when revising. Thank you!

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: Thank you for your feedback.

The current document does not have many mandates, beyond the request to reflect along the provided axes and communicate the outcome of such reflection. Even the section you referred to, 2.1.4 "There should be an internal review process for research projects involving human subjects", we ask to go to IRB *when possible*, as many institutions do not have an IRB. In fact, even if there is an IRB the section concludes with the following recommendation "Authors are encouraged to communicate transparently about the decision to enter this process or not", which is about communication rather than 'you must necessarily go through IRB before submitting'... 

That said, your comment resonates with others in terms of making the document more clear and better structured, and we will definitely take your feedback into account when revising in this direction.

As for the lack of diversity, thank you for raising the issue. Please see my response to "Concerned Asian"; in short, we'll definitely improve on this.

Finally, I really like your suggestion to further discuss this at a workshop on ethics or related topic. I think it would be helpful to continue this discussion at such a dedicated event.

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: You are right and we will improve this. 
The present committee has worked on the current draft, but this is not the end, just the beginning. I'll do all is in my power to increase the diversity of this committee, assuming this effort gets renewed. The other members of the current committee share the same sentiment too.

Thank you for raising this concern.

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: Thank you for the encouragement and for the suggestions.
Your feedback is rather inline with Pascale's: We probably need to restructure the draft to make clearer the actual asks. Currently, these are i) reflect along the provided axes and ii) communicate the outcome of such reflection. 
We'll revise accordingly. Once again, thank you.

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: Thank you for these public as well as the private comments you shared with us, which I generally agree with.
They will be definitely taken into account when we revise the draft.
Thank you.

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: Thank you for sharing your constructive feedback. I can understand how we might need to make this material more accessible, indeed. We'll have to think about how to re-organize it and structure it the way you suggested.
Thank you.

From: ~Marc'Aurelio_Ranzato1
title: unclear about the IRB suggestion
comment: Thank you Jitendra for sharing your perspective.

I also concur with Stella B. and I am not sure about what the alternative proposal would be for people who are not so fortunate to have an IRB in their institutions. It also seems to me that we are confounding two things: a) the discussion about the draft of code of ethics with b) the ethical review process which may rely at least in part on ethical guidelines or a code of ethics.

Even if everybody could refer to an IRB, I'd argue that having a document like this might be helpful in our present time where AI applications are more pervasive and impact our society in so many different ways. 
As I mentioned in my reply to Yann, the document is essentially providing several axes of reflection, and it contains mostly educational material (as opposed to a hard list of dos and donts). I personally learned a lot while working on this and I hope other people might benefit from it too. The only real asks from this documents are two: a) reflect and b) communicate transparently the outcome of that reflection. It'll be up to the reviewing committee and program chairs to draw the line as for what is deemed acceptable.

From: ~Marc'Aurelio_Ranzato1
title: thank you
comment: Thank you for pointing this out. I think you made a good point. We'll revise and clarify accordingly.

From: ~Marc'Aurelio_Ranzato1
title: response about draft being too prescriptive and on lack of legitimacy
comment: Thank you Yann for your comments.

I agree with you that an ethical guideline should “contain items that are non controversial and obviously shared by every reasonable author” and I believe that this guideline falls nicely within this category —  despite being more than one page long. 
What part did you find controversial or legislating on your political, moral, etc. beliefs?

The reason why this guideline is longer than one page is not because it is a long and detailed list of dos and donts, but because we thought to provide several axes of ethical reflections. There are only two major asks: 1) reflect if any of these ethical considerations are applicable to your research and b) communicate transparently about the outcome of that reflection. 
It will be the job of the program chairs to decide what is acceptable and what is not. At least now there is more transparent communication about what are the criteria and authors have material to inform themselves and better prepare.

Let me make an analogy. Imagine you are a car maker. We ask you to communicate the car specs along some axes like color, weight, speed… but we do not tell you what the maximum weight or speed should be. This draft is not prescriptive at all, in my opinion.

What I found nice about this is that now authors have a place to go to if they intend to learn more about what ethics in AI means. Similarly program committees have a reference for things to consider, although eventually the degree of how ethical/unethical a given practice is, is certainly nuanced, context dependent and often times subjective.

As an example, you may wonder when it is fine to scrape data from the internet. Sec 2.3.4 tells you that it might be ok depending on the situation and it provides further pointers.  It does not say that is necessarily always fine nor always wrong, because it isn't.

Regarding legitimacy: This is a a draft, it is not an approved code of ethics. It has been prepared by volunteers appointed by the Board. This committee has no legislative power whatsoever. It will be up to the Board to decide what to do with it and eventually to come up with policies. I believe all venues (ICLR, ICML, journals, etc.) have their own policies which participants need to abide to.

Thanks for reading this very long response, hope it helps clarifying a bit.

From: OpenReview.net/Anonymous_Preprint/Paper346/Authors
title: Conversation guidelines and how to reach out privately
comment: Thank you to all of those who have read and commented on the draft. 

Unfortunately, we have seen some posts in this discussion that are not very constructive and sometimes even offensive. We sympathize with the people who were involved and apologize for that. 

We thought then it is best to provide some clarifications and establish a set of rules. 

In order to contribute to the discussion, please:
1) Be considerate and respectful of others' opinions.
2) Help us make this discussion inclusive and welcoming, as we are all here to learn from each other and please refrain from any derogatory comments. 
3) Comments should be specific to the draft. Please reference specific sections, pages or quotes directly when possible or otherwise speak to the content of the draft. 

From now on, we reserve the right to remove any comment that violates these basic principles.

*Private commenting*: If you wish to send us feedback privately, now you can do so by setting “visibility” to “OpenReview Anonymous Preprint Paper346 Authors” (as opposed to "everyone").

Deadline: We will freeze commenting by June 15th, 2022. Afterwards we will work on a revision of the draft taking into account the feedback we have received.


Context: 

First and as stated in the introduction, this is a working document. The document in its current form has not yet been endorsed by the NeurIPS Board officially, but is simply an initial draft, prepared by volunteers appointed by the NeurIPS organization to begin this effort. The reason why we are asking for your feedback is precisely to help us come up with a better proposal for a code of ethics, and establish an opportunity for the broader community to chime in with their perspectives and feedback. 

Second, the document is currently drafted to be more of an educational and norm-setting document, than a prescriptive document. The goal is to articulate within the document the expected ethical standards for machine learning research practice, to provide guidance for researchers to properly engage in ethical reflection & communicate those risks broadly. This can then inform researcher reflections and decision-making, as well as provide more concrete references for ethical reviewers engaged in assessing the quality of such ethical reflections.  We hope it prompts our community to think critically about the harms they can cause through their research process or outcomes, and communicate transparently about such risks, as well as attempt to address these issues thoughtfully.

It is the program committee of the conference that will have the duty on any given year to set the bar for what is considered acceptable within this framework.

From: ~Haydn_Belfield1
title: Concrete and useful document for authors and reviewers - to prompt reflection on research ethics and societal impact 
comment: For context, I was an NeurIPS ethics reviewer for the most recent conference. The experience was very interesting personally – and the feedback from the authors I reviewed was that they appreciated having questions and issues flagged for them to consider. While I found the May 2021 guidelines helpful, it makes sense to develop the guidelines so they’re even more helpful to the community.

Overall, I find that this Code of Ethics is concrete, useful and action-guiding. It will be useful for me as both a reviewer and an author. It achieves its aim of “offering material of reflection and further reading, as opposed to providing a strict set of rules”. I also appreciate that it will be living document, kept up-to-date.

We surely all agree that authors submitting to NeurIPS cannot ignore research ethics or the societal impact of their work. The ethics review process is a useful way to support our community to do so – by providing an opportunity to reflect on research ethics and societal impact. In order for that process to occur in a transparent manner, its helpful to provide guidance to authors and reviewers about expectations, and topics to consider. This newer set of guidelines adds additional clarity to this process. While not being overly prescriptive, it helpfully flags important issues authors and reviewers should reflect on and consider. Many other research communities face the same issues, and have adopted very similar processes for setting expectations and considering questions of  research ethics and societal impact. I deeply appreciate that this process is being led by the community itself, rather than coming in from outside – it reflects the longstanding (and enviable) self-governing nature of the community.

From: ~Inioluwa_Deborah_Raji1
title: Clarification 
comment: The purpose of this document is not to police ethical judgements (ie. decide on what is "good" or "bad") but rather serve as a resource for ethical research practice, to set standards for what possible ethical issues researchers should reflect on, communicate about and address as they complete projects. Since ML researchers do engage in various practices that require serious ethical considerations (ie. collecting human data, building tools in high stakes scenarios, building tools operating at scale, etc.), it is important for them to be held accountable to some basic responsibility for further reflection on ethical issues. This document is meant to provide guidance for such reflections. 

The document is meant to be a starting point for researchers to consider and communicate about a broad range of possible concerns, at no point does it position itself as a tool for actual ethical judgement.

From: ~Karthik_Abinav_Sankararaman1
title: Is this an ideal or prescription?
comment: Thanks for putting this document together. I think its a great gesture to allow discussion and pass on an early draft to the community. I have some comments. First, I think the document is a bit too vague. Are these ideals we want to strive for or are they actions that every author, reviewer program committee needs to take today? For instance, can a reviewer reject my paper because it violates their interpretation of of this document? Making this clear(er) would help for the individual (such as choosing to steer their paper in a direction that matches the expectation or deciding altogether neurips is not the right venue). Second, and my biggest qualm is this seems like a document written by folks in western country trying to uphold a western standard for an international conference. I am of the opinion that "my conference my rules", but if you want neurips to be an international conference, then it may make sense to have guidelines that cater to the international community. A concrete example where I can say something is on environmental aspects. Environment is an extremely complicated geopolitical problem. The reason we have Paris climate accord and other climate agreements is to give different countries different environmental subsidies. Hoping to have a single broad stroke on environment is going to end up hurting many countries where their infrastructure is built around this international agreement. Not everyone can have access to GPUs situated in carbon zero data centers. Thus, a document that does not account for these geopolitical nuances would become (a) either irrelevant quickly (b) self-select the type of papers and people submitting to neurips. A related nit is that I also find some of the choice of examples used for environmental impact a bit odd. If we are including extraction of fossil fuel, I think a bigger elephant in the room is methane emissions and extraction of red-meat through animal husbandry being a major source of it. If/when neurips becomes in-person conference, we may want to think how this will affect food choices, such as serving red-meat.

From: ~Mike_Cook1
title: Followup
comment: I understand what you're trying to say, I do. But in your comment above you seem to imply that the Russian army is an example of a Bad army, by saying that people who are anti-weapons would change their mind if it invaded Europe. I would imagine most people would agree that what's happening in Ukraine right now is not good. Would you be okay, then, if NeurIPS published papers from a Russian government agency on using AI to build strategic command tools to help that army model and plan how best to invade Europe, or reported on the use of AI to gain an advantage in the ongoing conflict?

I realise it's hard to answer dispassionately given we're debating a code of ethics, but I think a lot of people would feel uncomfortable about that. Yet a lot of AI conferences do receive (and publish) work along exactly the same lines from military organisations in other countries like the US and the UK.

In your longer reply below you say that a code of ethics should only contain "non-controversial" statements that "every reasonable author" would agree on, but surely you can see that even that description is extremely subjective and biased (am I an unreasonable author if I think we should consider the potential for AI systems to be discriminatory?). There is no way not to take a stand on *something*. Even if NeurIPS were to do what you propose and make zero judgements on submitted work (which no conference does anyway) that would itself be a massive political declaration, and a huge statement about the role of scientists.

AI researchers are not used to having such a huge impact on the real world. The field is not prepared for it, its conferences and processes were not designed for it, but the field's role in the world has changed and we need to adjust.

From: ~Yingzhen_Li1
title: Need to be clear about the purpose of the document, and incorporate feedback from a broader community
comment: I'm glad to see this draft, and the respected authors' efforts are much appreciated. 

The reasons that I support to have a "code of ethics" document:
- I think AI researchers have the responsibility to ensure the ethical use of this technology. 
- The code of ethics document, if drafted well, can openly communicate the ethical values that the community agrees on.
- The code of ethics document, if drafted well, can be useful to aid education for future AI researchers/innovators.

That said, I do have some concerns regarding the draft:
- Less clear about the purpose: the board needs to clarify if they expect the Code of Ethics to have similar regulation effect on the NeurIPS community as the current Code of Conduct. 
- Continuing the above point, I feel the current draft is a mix of "wish" and "mandate". E.g., section 2.2 "environment" seems to be a "wish" while section 2.1.4 "internal review process" is clearer as "mandate". I suggest a clear separation of the two, if the board wants to make any "mandate".
- If the board expect it to be a "mandate", I agree with Pascale Fung and others, that the code needs to be more actionable for the average researcher. 
- If the board wants to make any "mandate", I suggest to have a section about "compliance" to clearly explain the minimal compliance and/or consequences of violations. 

I am also concerned that currently there is no Asian (nor Latinos, Muslins, correct me if I'm wrong) involved in this decision process (corroborated by "Concerned Asian"). It is problematic not to include views from this wider community if the board wants to publish a guideline on AI ethics for the whole NeurIPS community.

I very much hope that this document in its final form will represent a minimal list of values that is shared by the broader community. I'm optimistic that we do have some of them (e.g., not to use AI to harass others). But at the same time, we may have different views on the detailed practice of AI ethics, and I'm not a fan of "one-size-fits-all" detailed mandates. In sum, I suggest the final document to describe general principles/values like the ACM's Code of Ethics, but we can have e.g., workshops at the conference to discuss suggested actions, like the "mitigation strategies" in section 3.3.1.

Thank you for your efforts again.

From: ~David_Manheim1
title: Perhaps you want to check the facts, or the draft?
comment: "Einstein... invented nuclear energy and weapons... We can not wait for the next breakthrough to be ethically assessed in advance." But that manifestly wasn't the position of Einstein when talking about nuclear weapons. For instance, in 1939, Einstein wrote a letter to Truman advocating that he prevent Germany from getting access to resources needed to pursue atomic weapons, and other scientists involved certainly agreed that secrecy regarding technologies with applications to weapons was necessary. And perhaps you are correct that "Everything that is conceivable will one day be thought," (though basic combinatorics, physical limits, and the finite expected lifetime of the universe seem to indicate otherwise,) but that doesn't absolve the thinkers of ethical responsibility now.

But regardless of the factual issues, ethics in the conduct of research, which is the entire first half of the draft, is part of the process which absolutely must be assessed in advance. You may disagree with the need to register human subjects experiments, or minimise direct harm to living beings, or pay fair wages to your graduate students, or consider privacy protection, but if you're pushing for "open ethical discussion" you should support this standard, which only requires you "reflect upon" or "consider" them, right? And by that same logic, asking participants to "reflect upon" and "discuss" possible issues in the future seems to be pushing for the same thing - so I'd wonder what sections specifically you object to in the standard.

From: ~Pedro_Domingos1
title: And the NeurIPS board is overstepping its own
comment: Conferences are not even professional societies, even less companies, and even less governments or churches. You can't abdicate a responsibility you don't have. Leave well enough alone.

From: ~David_Manheim1
title: Organizations have responsibilities.
comment: Leaving everyone to make their own decisions is abdication of responsibility on the part of the conference organizers.

I wouldn't work for a company whose ethics statement was "Our employees might do bad things, but that's on them - we're just here for the money." And I wouldn't want to attend or submit to a conference whose program committee did the equivalent, which seems to be what you're advocating.

From: ~Pedro_Domingos1
title: We all do, of course
comment: You're very confused. I'm not saying that ethics is someone else's problem. On the contrary, I'm saying that it's for each one of us to decide, not for some conference ethics board to mandate. Clear?

From: ~David_Manheim1
title: If not NeurIPS, who does have this "authority" to make ethical decisions?
comment: Coming from a technologist, the idea that ethics is someone else's problem is frankly alarming and disturbing, especially without specifying who does have the responsibility. Furthermore, this purposeful ducking of responsibility runs counter to ethical norms in engineering, in scientific publishing, and in ethics generally. If an organization isn't responsible for the decisions it makes about what to publish, and whether it is ethically acceptable, who could be?

Yes, there will be discussions and debates about what is a political versus ethical concern, but mandating that the conference avoid those discussions is itself a strong "free speech absolutist" position. And if that is your position, you should say that there should be no constraints on what type of research is disallowed on ethical grounds, rather than appealing to lack of authority. Otherwise, your position is unclear, simultaneously claiming that the conference lacks authority, but that (evidently) ethical rules would be acceptable. So if the conference lacks authority, can you say who has it?

From: ~Yann_LeCun1
title: Quick a
comment: The point of my post is that neither NeurIPS, (nor I) have any legitimacy or competence to decide for society what piece of basic technology is going to be good or bad and who should be allowed to use it.

I mean, most technologies can be used for good and bad. 
A few things that were originally published at NeurIPS have eventually been used simultaneously for wonderful things and for pretty terrible things.

From: ~Pedro_Domingos1
title: NeurIPS has zero authority to decide what is ethical research
comment: And if the research is scientifically invalid, it should be rejected on those grounds. Rejecting work that some find "ethically problematic" on the grounds that it may also be scientifically invalid is perverse. (Rather, it leads to prejudging as scientifically invalid research you don't like for political reasons, which is even more alarming . . . and you've just illustrated.)

From: ~Pedro_Domingos1
title: Enough is enough
comment: Don't be disingenuous (or terminally naive). NeurIPS is the biggest AI conference in the world, and its dictates about what is and isn't acceptable research are already having a chilling effect on the field. Enough is enough.

From: ~Timnit_Gebru1
title: Stats on IRB
comment: Here's a paper discussing the percentage of papers at ML and related conferences with any sort of ethical review or IRB: https://t.co/CKdQ2bmaV9?amp=1. Surprise suprise CVPR had the lowest :) Why is it ok for us to reject papers because we don't think they are "novel" but any type of ethics is off limits?

I created a new institute precisely because the only path I see for myself is to not be dependent on these types of conferences since even the slightest hint of researchers having to be responsible for thinking about their work brings so much backlash. We got death threats for suggesting that NIPS should change its name even. Its changed now but after many people paid a price,  especially Anima Anandkumar.

From: ~Stella_Biderman1
title: I think you’re missing my point
comment: I think that the question of the efficacy of IRBs is largely irrelevant. Under widely accepted standards in the US (where I am many submitters to NeurIPS live, though I can’t speak to other countries) the overwhelming majority of AI research projects are not subject to IRB oversight.

The fact that NeurIPS is doing this indicates that they don’t think the existing enforcement mechanisms are sufficient, so leaving it up to the existing mechanisms isn’t a solution. NeurIPS doesn’t have the ability to coerce IRBs to review papers that are exempt from IRB approval, and making the large number of people who are currently submitting papers without IRB approval getting a brief note saying “this is exempt from IRB approval” doesn’t advance any of NeurIPS’s goals.

Saying “you should use IRBs” isn’t very productive advice unless you have a proposal for actually getting the IRBs to do the work. Do you have one?

From: ~Concerned_Asian1
title: Why No Asian Representation On Committee?
comment: This Committee have ZERO Asians. No East Asians, No Indians, no South East Asians, etc. No representation is very worrying. How come many researchers at NeurIPS are Asians, but no Asians on committee?

From: ~Mike_Cook1
title: Quick q
comment: Just out of interest, could you list the nations you think would be okay to equip with autonomous anti-tank weapons and which ones aren't okay? It might be helpful to have a list of approved military organisations.

From: ~Mike_Cook1
title: A Promising Start
comment: I'm really glad to see NeurIPS make this move. I was disappointed to see AAAI remove the requirement for ethics disclosures this year, as we desperately need examples to be set by the biggest conferences in our area. I have been impressed by some of the moves I've seen NeurIPS make in the past, and small changes made here will encourage other conferences in AI to do the same. 

I think the document is a great starting point for discussions. Personally I would like to see more concrete steps taken - for example, my reading of the document is that it actually forbids very little (despite the exaggerations of some other comments on here). It seems to leave a lot up to the individual reviewers of a paper, which is potentially fine but I think as a field we do need to re-examine what we expect from reviewers, how we train them to do the job, and how we evaluate the effectiveness of our review process. Having stronger and clearer guidelines in an ethics document like this would be helpful.

Regarding the comments made here by senior academics who should know better: this is a conference, it's not a public forum, and NeurIPS is not your local government. This ethics statement does not impinge on anyone's ability to do research, it simply sets a standard for NeurIPS participants as a place to *publish* that research. Everyone should feel very free to go and research their smart missile systems that magically and specifically target The Bad Guys, and then find a venue that is willing to publish that work. NeurIPS isn't even ruling itself out as a place you can do that! It's basically just asking you to think twice before deciding you are well-equipped enough to choose which nation to equip with war crimes robots.

Pretending that AI's beautiful diversity will be harmed by a document saying "Try not to murder people" is an absolute joke, and anyone claiming such is very well aware of it. Equally, pretending that this document is the first time an AI conference has been political is another very obvious misdirection. Our conferences already are political, they just happen to be political in a way that this angry minority are comfortable with, currently.

I religiously avoid large AI conferences precisely because they are mired in this kind of thinking. I would desperately like there to be an AI conference in my field that I am proud to attend, to talk about and to advertise to others. I really hope you continue work on this document and keep improving NeurIPS as a positive research space. Thanks for doing this, and for putting up with the backlash.

From: ~David_Manheim1
title: This is a call for no standards, not a request for outside guidance. If NeurIPS isn't self-monitoring, no-one will do it for them.
comment: "I don't think it is the place of NeurIPS to decide a priori what constitutes an acceptable trade-off...leave that up to the  governments and international institutions to deal with."

You claim it's someone else's problem, but you must already know that they can't address this directly, especially in this venue - for a wide variety of reasons, from legal problems for western countries not allowing prior restraint on free speech, to lack of capacity to pre-screen manuscripts that are submitted, to what I'm sure would be a far greater outcry if any government body or NGO attempted to impose standards on what research could be published. And if NeurIPS doesn't get to decide what constitutes an acceptable trade-off on any given dimension - ethical, scholarly, or grammatical - it is an invitation not to have any standard at all. If a conference can say "we're not grammarians, but this writing isn't acceptable for presentation at this conference," why should it not be able to impose similar very minimal requirements on what it deems ethically acceptable to present?

From: ~David_Manheim1
title: The ethicisist have entered the chat
comment: The idea that ethicists are opposed to engineers' self-imposed codes of ethics is not just laughable, but obviously and evidently so. Ethicists focusing on technology have called for exactly this type of reflection and self-monitoring on many occasions, and the calls for further ethical review has already shown that lacking such reflection is strictly worse than having it. Furthermore, the idea that the discussion now needs to be opened to "scholars of ethics," or that you need to cater to and simplify your work for "the ones that can make educated sense of what we do" reflects a deep ignorance of what scholars of technological ethics are doing and have already done.

From: ~Jitendra_Malik2
title: In defense of IRBs
comment: In computer vision, data set collection typically falls under IRB purview, as also user studies. Overall it makes sense to stick to the IRB machinery as it has met the test of time in many disciplines, including in our sister disciplines of psychology and neuroscience. It aligns with the practices of granting agencies and journals as well. What is in the scope of IRBs is obviously debatable, so if we feel that more AI related research activities need to be brought under their purview, by all means we should discuss that. And obviously the norms of what is considered "acceptable" by IRBs can also evolve over time. The IRB mechanism has a quasi-judicial character, with the reviewers acquiring serious competence and experience in making these judgments. This is how it should be - telling some one that a research project needs to be modified or abandoned is a serious decision for which the bar should be high and not capricious. In my experience at Berkeley, people may complain about the extra effort required in getting IRB permissions, but the legitimacy of the process is widely accepted.

In contrast, NeurIPS has a hard time even managing the basic process of reviewing scientific merit of papers in a consistent and fair fashion (I am not blaming the organizers; a lot of this is intrinsic noise exacerbated by the rapid growth of the community). To take on an additional burden, where there  is considerable debate on values (see Yann's example of precision weapons) seems unwise.

From: ~Stella_Biderman1
title: IRBs don’t oversee most ML research
comment: The overwhelming majority of ML research doesn’t ever go before an IRB. I don’t have NeurIPS-specific data, but I would be blown away if 25% of papers submitted to NeurIPS had received IRB approval. Regardless of how successful an IRB would be at this task (and I don’t think they’d be particularly successful), people don’t submit their research plans to IRBs.

From: ~Emily_M._Bender1
title: NeurIPS is absolutely within its authority to do this
comment: I will be sending the authors more detailed comments via email but here want to point out a few high level things:

1) The purpose of NeurIPS is to run conferences and publish papers. As such, NeurIPS 100% has the authority to set the standards for publication/presentation in its venues. 

2) The notion that ethics reviews are in some way at odds with "scientific" or "technical" reviews is a red herring. In a large number of cases, we find that ethically problematic work is also scientifically invalid (e.g. digital physiognomy), and the questions that ethical review raises helps to expose this.

3) I appreciate that the document points to the ACM Code of Ethics and the Toronto Declaration. I think it would be helpful to clarify what force these have in the NeurIPS context in particular. (I believe that NeurIPS is not part of ACM; in that case, the NeurIPS board could vote to adopt the ACM Code of Ethics just as the ACL Executive Committee has.)

4) Whether or not these documents are officially adopted, I think it would help to introduce them in the introduction to the present document and refer back to them (ideally in detail) as the motivation for each separate point in the document.

5) There seem to be two interrelated tasks here: setting standards and helping researchers (as authors and reviewers) meet/maintain the standards. It may help to address these tasks with separate documents.

From: ~Pedro_Domingos1
title: Hubris
comment: What hubris this is. A small group of people telling a very large, dynamic and diverse research community what it should and shouldn't work on, and how; dressing up their political preferences as ethical principles; oblivious to their overreach. NeurIPS Board, please put an end to this grotesque experiment before it does any more damage.

From: ~Martin_Hofmann2
title: Scientific Responsibility
comment: Why should Engineers decide on ethical topics? Let we Ethics scientists correct our equations? No, we don’t, and we should also be careful with ethical statements. There seems to be the opinion that it's decidable if an invention is harmful given that a world is complex (observations are not complete and local) and that the world changes; this claim needs proof. We need to open our discussion for scholars of ethics and to ease the understanding of our work for the ones that can make educated sense of what we do and how it will influence society. And this will be an ongoing discussion. As the world and society change, so the opinion and influence will. We should provide tools to track the ethical and societal and ai progress and what new questions arise. This is helpful. Not an overly blurry and squishy written ethics paragraph.

From: ~Martin_Hofmann2
title: Open ethical discussion. Not a overly broad statement.
comment: Please consider actual Scientific progress—a breakthrough in fundamental research- like the Theory of relativity. Einstein enabled the correct forecast of planets' orbits and described how stars’ projections are enlarged when we look into the sky.  He predicted black holes and gravity waves. He invented nuclear energy and weapons, global positioning, and precision bombs. He invented the internet's foundation and the most potent Propaganda instrument. 
Actual progress is measurable with its impact on society, which leads scientists to inflate the result of their work to a point where it is getting harder for reviewers to find the true advancement.  NeurIPS collects the highest scientific quality, and therefore, we can not assess pages of ethical statements lacking a particular matter. We can not wait for the next breakthrough to be ethically assessed in advance. Everything that is conceivable will one day be thought. If something is significant, we should give it an ethical discussion right away. Therefore we should help people find ethical statements on a certain research artifact but we should not blur the scientific matter. Therefore: Open ethical discussion but no overly broad ethical statement.

From: ~Yann_LeCun1
title: Moral trade-offs
comment: I don't think it is the place of NeurIPS to decide a priori what constitutes an acceptable trade-off when it comes to things like defense, military applications of AI, or even law enforcement applications.

These are complicated issues for society at large and international bodies to deal with through the normal democratic and diplomatic process.
We, scientists, have no particular legitimacy, and very little competence, to make decisions for others on such issues.

The the moral point of view, there is no simple answer to questions about the use of ML/AI in weapons.
For example, "smart" precision weapons can help reduce collateral damage. Is that good or bad?
If, say, Putin invades an EU country tomorrow. What would be our stance about the development of, say, autonomous anti-tank weapons? (Which, by the way already exist, without much AI).

Complex issues.
No simple answer.
NeurIPS should focus on science and leave that up to the  governments and international institutions to deal with.
Whether we like what they do or not, it's actually their job.

From: ~Pascale_Fung1
title: Needs to be more actionable for the average researcher
comment: Congratulations on this first draft of the much needed Code of Ethics! It is not an easy job and the authors have obviously done a lot of work to come up with this document. However, the document as it stands is way too detailed and overreaching. Some part of it very onerous and risks alienating researchers, especially those from less wealthy regions, without access to legal representation. People should not need legal representation to do research!  For ethical reviewers, the document is also too long and too vague. Code of Ethics should be devised with the objective of promoting more ethical research. So it needs to be readily comprehensible and actionable to the average ML researcher both as an author or a reviewer, without any background in philosophy, social science or law.  My suggestion is to make it more succinct, prioritise actionable items, and think of the average researcher, not someone who works in a rich country.

From: ~Kamalika_Chaudhuri1
title: Beyond the scope of program committee
comment: I too agree with Yann and Jitendra -- this is much too broad in scope. While I agree there needs to be a few (exceptional) topics -- such as directly developing autonomous weapons -- that ICML and NeuRIPS can agree not to accept papers on -- this kind of broad mandate is perhaps beyond the scope of a conference's program committee.

From: ~Kush_R._Varshney1
title: Data and model documentation
comment: I also agree with the spirit of that section, but find it not to be inclusive as written with 'datasheets' and 'model cards' explicitly called out.

From: ~Jitendra_Malik2
title: Leave this to IRBs
comment: I share some of Yann's concern about making this too grand in scope. Certainly there needs to be some monitoring of research ethics - is the work harmful, exploitative etc. Universities have dealt with this by setting up IRBs (a link to Berkeley's version https://cphs.berkeley.edu/ )  . Let's leave this job to them.

From: ~Angus_Galloway1
title: Re: Data and model documentation
comment: Thanks for kicking off a much needed discussion and for a thoughtful initial draft. 

Although I agree with the spirit of the section, I find Section 2.3.2 a bit prescriptive where it says "Data documentation requirements should adhere to templates". Could the implementation be left more open ended, while stating the kinds of information that should accompany new dataset/model releases, e.g., details regarding data provenance and curation as already mentioned. I can also envision scenarios where data/code/models are released as a helpful gesture mainly to facilitate the peer reviewing process, and not necessarily as a research contribution, in which case detailed documentation may be a bit of a burden. Perhaps the section could clarify the types of scenarios in which formal documentation requirements apply.

From: ~Yann_LeCun1
title: Much too broad in scope
comment: As much as I agree with many of the items in the document, and as much as I like the authors, I do not recognize their legitimacy to legislate on such a broad set of topics. Neither do I recognize the NeurIPS Foundations' legitimacy to do so.

An ethical guideline should be at most one page long and only contain items that are non controversial and obviously shared by every reasonable author.

The longer the document, the more topics it covers, the more likely it is to include *opinions* that are subject to debate and not shared by everyone.
Overreaching guidelines will, in fact, *exclude* many scientists from contributing to NeurIPS.

It is not NeurIPS's role nor place to legislate morality, politics, beliefs, and philosophy. 
It is not NeurIPS's place to legislate personal behavior other than to ensure the safety of the participants within the confines of the conference itself.