Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 
The authors go over trends in the research and use of machine learning
and in particular deep learning in the applied sciences, via analysis
of the open-access literature (bibliometrics and/or full-text analyses),
and highlight areas for concern. The findings are somewhat similar to
prior results of the analyses of recent trends in machine learning R&D 
and its related fields.

strength_and_weaknesses: Strengths:

-- A variety of useful insights and takeaways into how  ml (in particular large-scale deep ml) is impacting other
   scientific fields, in particular biology and chemistry, and several
   trends are estimated from analysis of published research  (societal/environmental/energy impact).  

Weaknesses:

-- No or very little technical contributions.  The ICLR venue may not to be
   relevant for this report.

clarity,_quality,_novelty_and_reproducibility: Clarity is good, although the introduction abruptly dives into the
history of hardware development.  The paper uses existing techniques
but the novelty mainly lies in applying the analyses to research in 
biology and chemistry.



summary_of_the_review: The paper reports on trends in ml and how it has been developing in
that past decade, specially in biology and chemistry.  Several of
these trends are certainly concerning (the de-democratizatin of ml
work).

However, I don't think that ICLR is the venue for this publication,
just as literature reviews are highly useful, but not relevant in this
conference. The paper does not have a technical contribution.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper draws attention to the societal and environmental impacts of machine learning research in chemistry and biology. The paper specifically makes a contribution in the following components such as: environmental considerations, citation inequality, academic brain-drain, and scientific considerations:

The study makes the following key assertions:

* During the past two decades, there has been an  unparalleled shift of professors from university to industry. 
* chemistry saw an increase in transitions of machine learning practitioners from academia to industry, while overall transitions in biology remain slightly skewed towards industry 
* Technology companies drive machine learning research. 
* More citations in chemistry and biology are generated from Pharm, tech, and “other” companies compared to academia. 
* In terms of environmental impact, the data was limited and as a result the  quantitative analysis of the environmental impact of  machine learning research in biology and chemistry was limited. 
strength_and_weaknesses: Strength: results reported were consistently compared with those in literature. 

Weakness 1: The sentences are just too long. Need to be broken down. For example, this sentence is too long : “Potential societal and environmental effects such as the rapidly increasing resource use and the associated environmental impact, reproducibility issues, and exclusivity, the privatization of ML research leading to a public research brain-drain, a narrowing of the research effort caused by a focus on deep learning, and the introduction of biases through a lack of sociodemographic diversity in data and personnel caused by recent developments in machine learning are a current topic of discussion and scientific publications.

Weakness 2: A simple figure highlighting all the steps/methods that were followed in generating the paper findings is missing.


clarity,_quality,_novelty_and_reproducibility: The paper presents novel knowledge as it specifically draws attention to the impact of machine learning research in biology and chemistry. The findings are also general in  that they apply to other fields as well. 
The results reported in the study may be reproduced as the authors have shared  data  and code used in this study. 
summary_of_the_review: 
The paper presents the critical interfaces of social and environmental impact of machine learning research specifically in chemistry and biology.  The findings reported in the paper are applicable to other fields as well.  This work provided a basis for discussion as key fields such as drug development, genetics, and cancer research can be affected as well. Overall, this overview provided essential knowledge for the advancement of machine learning research in biology and chemistry.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Paper studying social and environmental impact of recent developments in ML (with a focus on DL) on biology and chemistry research.
strength_and_weaknesses: Strengths:
Lots of information and very interesting topic
Weaknesses:
Minimally coherent structure. At some points it almost seems like a stream of thoughts dump.
Mixed topics
Unorthodox information source referencing
clarity,_quality,_novelty_and_reproducibility: Not very clear for the reasons stated above.
Definitely novel from the point of vew of the research and certainly reproducible.
summary_of_the_review: Interesting paper studying social and environmental impact of recent developments in ML (with a focus on DL) on biology and chemistry research.

It falls within the remit of the conference, as the call for papers includes work on "societal considerations of representation learning 
including fairness, safety, privacy, and interpretability, and explainability"

Even if interesting and certainly relevant, my main qualm with this study is that it lacks coherent structure. A lot of ideas and information
are thrown in the pot, but they are barely organized at all, jumping from a topic to the next, almost without the respite of a paragraph ending.
This is compounded by a rather unirthodox referencing system.

I definitely like the content, but it seems to me that it would require a different venue in which these ideas could breath, be it a jounal position paper or a book chapter.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 
In this paper, the authors perform several quantitive studies for understanding the social and environmental impact when applying machine learning methods for biology and chemistry research. Some of authors observations include a potential inequity increase in applied machine learning research in biology and chemistry, and big companies tend to be actively involved in machine learning based research.


strength_and_weaknesses: 
The topic is of importance and in general interesting.
clarity,_quality,_novelty_and_reproducibility: 
All experiments seem to be reproducible.
summary_of_the_review: 
This paper is not proposing some new tasks or introducing some new methods. Although the topic and the authors' findings are interesting, it seems this paper is not suitable for a conference like ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors went over trends in the research and use of machine learning and in particular deep learning in chemistry and biology, via analysis of the open-access literature (bibliometrics and/or full-text analyses), and highlight areas for concern. Some of the authors' observations include a potential inequity increase in applied machine learning research in biology and chemistry, and big companies tend to be actively involved in machine learning based research. The findings are somewhat similar to prior results of the analyses of recent trends in machine learning R&D and its related fields. The reviewers pointed out to very limited technical contributions and minimally coherent article structure. The authors also ignored one reviewer’s comments on the weaknesses. Even though that reviewer gave a high score, the pointed weaknesses still need to be addressed. Overall, there still seems to be a lack of enough enthusiasm among the reviewers after the author responses.
justification_for_why_not_higher_score: The reviewers pointed out to very limited technical contributions and minimally coherent article structure. The authors also ignored one reviewer’s comments on the weaknesses. Even though that reviewer gave a high score, the pointed weaknesses still need to be addressed. Overall, there still seems to be a lack of enough enthusiasm among the reviewers after the author responses.
Decision: Reject