Original: Teaching Machine Learning for the Physical Sciences: A summary of lessons learned and challenges
Revised: Teaching Machine Learning for the Physical Sciences: A summary of lessons learned and challenges

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: This paper summarizes some challenges encountered and best practices established in several years of teaching Machine Learning for the Physical Sciences at the undergraduate and graduate level.
Revised: This paper summarizes some challenges encountered and best practices established in several years of teaching Machine Learning for the Physical Sciences at the undergraduate and graduate level.

Original: I discuss motivations for teaching ML to Physicists, desirable properties of pedagogical materials such as accessibility, relevance, and likeness to real-world research problems and give examples of components of teaching units
Revised: I discuss motivations for teaching ML to physicists, desirable properties of pedagogical materials, such as accessibility, relevance, and likeness to real-world research problems, and give examples of components of teaching units.

Original: 1. ML x Physical Sciences
Revised: 1. ML x Physical Sciences

Original: Machine learning methods have become ubiquitous in many data-intensive disciplines, including, of course, Physics and
Revised: Machine learning methods have become ubiquitous in many data-intensive disciplines, including, of course, Physics and

Original: Astronomy.
Revised: Astronomy.

Original: The Physical sciences offer a rich landscape of observational and simulated data that are suitable to be analyzed using machine learning and deep learning tools.
Revised: The Physical sciences offer a rich landscape of observational and simulated data that are suitable to be analyzed using machine learning and deep learning tools.

Original: Identifying particles produced in collision events at the Large
Revised: Identifying particles produced in collision events at the Large

Original: Hadron Collider, processing astronomical images from large surveys, identifying transient phenomena in real time, creating fast approximated solutions for lattice theories, or building “emulators” for expensive cosmological simulations are just some of the uses that have become popular in the last decade (see e.g. Carleo et al.
Revised: Hadron Collider, processing astronomical images from large surveys, identifying transient phenomena in real time, creating fast approximated solutions for lattice theories, or building “emulators” for expensive cosmological simulations are just some of the uses that have become popular in the last decade (see e.g. Carleo et al.

Original: for a review).
Revised: for a review).

Original: It follows as a logical consequence that the foundations of machine learning methods should be taught as part of the standard Physics curriculum.
Revised: It follows as a logical consequence that the foundations of machine learning methods should be taught as part of the standard Physics curriculum.

Original: In part, this is because they are bound to become standard tools for Physics research.
Revised: In part, this is because they are bound to become standard tools for Physics research.

Original: But even more importantly, they are a great pedagogical tool to stimulate critical thinking and to build transferable skills that would create better job perspective for Physics graduates, by leveraging the enormous growth of jobs in the Data Science area that are accessible to those with a rigorous scientiﬁc background and strong computational skills.
Revised: Even more importantly, they are a great pedagogical tool to stimulate critical thinking and to build transferable skills that can create better job perspective for Physics graduates, by leveraging the enormous growth of jobs in the Data Science area that are accessible to those with a rigorous scientiﬁc background and strong computational skills.

Original: 2. Teaching ML to Physicists
Revised: 2. Teaching ML to physicists

Original: There are many great learning resources that either low-cost or free.
Revised: There are many great learning resources that either low-cost or free.

Original: These include excellent books with free online
Revised: These include excellent books with free online

Original: Jupyter notebooks (G´eron, 2019; VanderPlas, 2016), courses on online platforms like Coursera or Udemy, and chances to practice on fairly complex data sets such as those hosted by Kaggle.
Revised: Jupyter notebooks (G´eron, 2019; VanderPlas, 2016), courses on online platforms like Coursera or Udemy, and chances to practice on fairly complex data sets such as those hosted by Kaggle.

Original: However, the abundance of choices can be overwhelming for beginner practitioners, who would have to “mix-and-match” different resources to create a curriculum, and more importantly, resources that are tailored to the process of scientiﬁc research, and, in particular, to the physical sciences, are still scarce; (Ivezi´c et al., 2014) is a happy exception, focused on
Revised: However, the abundance of choices can be overwhelming for beginner practitioners, who would have to “mix-and-match” different resources to create a curriculum, and more importantly, resources that are tailored to the process of scientiﬁc research, and, in particular, to the physical sciences, are still scarce; (Ivezi´c et al., 2014) is a happy exception, focused on

Original: Astronomy.
Revised: Astronomy.

Original: Furthermore, if we want machine learning to become a standard part of the Physics curriculum, we need to provide resources for instructors: many of them won’t have been trained in this subject during their own course of study, so that lowering the barrier for teaching is as important than lowering the barrier for learning .
Revised: Furthermore, if we want machine learning to become a standard part of the Physics curriculum, we need to provide resources for instructors: many of them won’t have been trained in this subject during their own course of study, so that lowering the barrier for teaching is as important than lowering the barrier for learning .

Original: Five years ago, I started creating materials for a “ML for Physics and Astronomy” course, almost from scratch.
Revised: Five years ago, I started creating materials for a “ML for Physics and Astronomy” course, almost from scratch.

Original: Since than, I have taught this class several times, at the undergraduate level to STEM majors, and at the graduate level to Physicists and Astronomers, and I have written the ﬁrst draft of a textbook on the same subject.
Revised: Since than, I have taught this class several times, at the undergraduate level to STEM majors, and at the graduate level to Physicists and Astronomers, and I have written the ﬁrst draft of a textbook on the same subject.

Original: Here, I’d like to share some of the practices I have found to be useful and some of the challenges I have identiﬁed during this time.
Revised: Here, I’d like to share some of the practices I have found to be useful and some of the challenges I have identiﬁed during this time.

Original: 2.1. Needs
Revised: 2.1. Needs

Original: These are some desirable qualities of materials used to teach Machine Learning to Physics (or more in general,
Revised: These are some desirable qualities of materials used to teach Machine Learning to Physics (or more in general,

Original: STEM) students:
Revised: STEM) students:

Original: Accessibility: While most STEM majors are familiar with linear algebra, calculus, and statistics, there is great variability in the level of mathematics they are able or willing to handle.
Revised: Accessibility: While most STEM majors are familiar with linear algebra, calculus, and statistics, there is great variability in the level of mathematics they are able or willing to handle.

Original: At the undergraduate level, I advocate for keeping the complex mathematics at a minimum, and focus on the conceptual aspects of how different algorithms work.
Revised: At the undergraduate level, I advocate for keeping the complex mathematics at a minimum, and focus on the conceptual aspects of how different algorithms work.

Original: My experience is that this approach is the most inclusive and still provides a good foundation to those who would like to explore a topic in higher detail.
Revised: My experience is that this approach is the most inclusive and still provides a good foundation to those who would like to explore a topic in higher detail.

Original: Relevance: This is possibly the greatest challenge - ﬁnding problems and data sets that are relevant to scientiﬁc research is hard!
Revised: Relevance: This is possibly the greatest challenge - ﬁnding problems and data sets that are relevant to scientiﬁc research is hard!

Original: Most of the “introductory” data sets (MNIST,
Revised: Most of the “introductory” data sets (MNIST,

Original: Boston housing data, Iris...) are too simple, and working with those does not resemble the typical challenges of a research problem.
Revised: Boston housing data, Iris...) are too simple, and working with those does not resemble the typical challenges of a research problem.

Original: Many more advanced ones (e.g., those found on Kaggle, which include several Physics/Astronomy challenges) are very complex, are require a vast amount of background information.
Revised: Many more advanced ones (e.g., those found on Kaggle, which include several Physics/Astronomy challenges) are very complex, are require a vast amount of background information.

Original: Striking the right balance of ﬁnding data and problems that are beginner-friendly, while at the same time presenting some nontrivial features and help develop intuition, is difﬁcult.
Revised: Striking the right balance of ﬁnding data and problems that are beginner-friendly, while at the same time presenting some nontrivial features and help develop intuition, is difﬁcult.

Original: Real-world likeness:
Revised: Real-world likeness:

Original: Experienced machine learning practitioners would often say that “prepping” the data is one of the most time-consuming and important tasks in a ML project.
Revised: Experienced machine learning practitioners would often say that “prepping” the data is one of the most time-consuming and important tasks in a ML project.

Original: I wholeheartedly agree.
Revised: I wholeheartedly agree.

Original: Many existing data sets for beginner practitioners are very “curated”, and this limits their usefulness in demonstrating best practices in very important aspects such as data exploration, data cleaning, transformations, imputing strategies, and feature engineering.
Revised: Many existing data sets for beginner practitioners are very “curated”, and this limits their usefulness in demonstrating best practices in very important aspects such as data exploration, data cleaning, transformations, imputing strategies, and feature engineering.

Original: Additionally, themes that are of fundamental importance in research, such as choosing or building an appropriate evaluation metric or estimating uncertainties, are often absent from pedagogical material.
Revised: Additionally, themes that are of fundamental importance in research, such as choosing or building an appropriate evaluation metric or estimating uncertainties, are often absent from pedagogical material.

Original: How can we put together materials that have all these desirable properties?
Revised: How can we put together materials that have all these desirable properties?

Original: In short, it takes a lot of work, and it takes a village.
Revised: In short, it takes a lot of work, and it takes a village.

Original: As I was assembling resources, I was grateful to count on the help of a very supportive Physics/Astronomy community.
Revised: As I was assembling resources, I was grateful to count on the help of a very supportive Physics/Astronomy community.

Original: An approach that I have found to be promising is to start with a literature review, and ﬁnd papers that apply a certain method to data of interest.
Revised: An approach that I have found to be promising is to start with a literature review, and ﬁnd papers that apply a certain method to data of interest.

Original: Some of the most successful pedagogical examples I have used propose to match, or improve on, the performance of a published paper.
Revised: Some of the most successful pedagogical examples I have used propose to match, or improve on, the performance of a published paper.

Original: This is appealing to students because it communicates that they are doing quality work (or at least, that the ML aspect is at a publication-worthy level).
Revised: This is appealing to students because it communicates that they are doing quality work (or at least, that the ML aspect is at a publication-worthy level).

Original: For several exercises in my upcoming book, I have reached out to authors of a paper, and asked for access to data, if necessary, and for some introductory guidelines to create a problem-solving pipeline.
Revised: For several exercises in my upcoming book, I have reached out to authors of a paper, and asked for access to data, if necessary, and for some introductory guidelines to create a problem-solving pipeline.

Original: In class, I show students what our ﬁnal goal (performancewise) would be, and try to build the pipeline together with them.
Revised: In class, I show students what our ﬁnal goal (performancewise) would be, and try to build the pipeline together with them.

Original: I have found that if I propose a solution, it’s always helpful to riddle it with poor choices; for example, “forgetting” to normalize the data when needed, including noisy or very correlated features, using accuracy on imbalanced data sets...
Revised: I have found that if I propose a solution, it’s always helpful to riddle it with poor choices; for example, “forgetting” to normalize the data when needed, including noisy or very correlated features, using accuracy on imbalanced data sets...

Original: The process of improving the code helps students retain a stronger memory of these potential pitfalls.
Revised: The process of improving the code helps students retain a stronger memory of these potential pitfalls.

Original: Colleagues have indicated that they use a similar approach successfully (S. Caron, private comm.)
Revised: Colleagues have indicated that they use a similar approach successfully (S. Caron, private comm.)

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: • F IGURE OUT WHICH OF THE DATA CLEANING CUTS WE MADE WAS THE MOST SIGNIFICANT IN TERMS OF IMPROVING THE SCORES OF THE FINAL MODEL .

Original:  A TTRIBUTE EACH OF THESE STATEMENTS TO B AGGING M ETHODS OR B OOSTING M ETHODS :
Revised: N OW CHOOSE ONE REFERENCE ALGO RITHM AMONG THE ONES WE SAW (RF, A DA B OOST , GBM), AND REFER TO

Original: [Table]
Revised: THE OPTIMIZED MODEL ( NO NEED TO RE - RUN THE G RID S EARCH ). L ET ’ S CALL THIS M ODEL 1.

Original: H OW IS THE FEATURE IMPORTANCE CALCULATED IN TREE - BASED METHODS ?
Revised: U SE THE DATA SET SELECTION WITH 6,307 OBJECTS AND 6 FEATURES . • G ENERATE PREDICTIONS USING CV AND PLOT THEM IN A HISTOGRAM , TOGETHER WITH THE TRUE VALUES .

Original: [Table]
Revised: [Table]

Original: W HICH OF THESE PARAMETERS , IF DECREASED , IS MOST LIKELY TO REDUCE
Revised: HICH DISTRIBUTION IS NARROWER ? [Table] XPLAIN WHY . • O PTIMIZE ( USING A G RID S EARCH FOR THE PARAMETERS YOU DEEM TO BE MOST RELEVANT )

Original: THE 
Revised: THE E XTREMELY R ANDOM T REE ALGORITHM AND COMPUTE THE OUTLIER FRACTION AND NMAD. H OW DOES IT COMPARE TO M ODEL 1? C OMMENT NOT JUST ON THE SCORING PARAMETER ( S ), BUT ALSO ON VARIANCE / BIAS .

Original: GAP BETWEEN TRAINING AND TEST SCORES ?
Revised: W HICH ONE WOULD YOU PICK ?

Original: [Table]
Revised: • I N THE PAPER THAT WE USED AS A REFERENCE (Z HOU ET AL ., 2019), THE AUTHORS ACTUALLY USE COLORS , NOT MAGNITUDES , AS FEATURES . D

Original: W HICH OF THESE ESTIMATORS IS MOST
Revised: ISCUSS WHY THIS MIGHT BE A BETTER CHOICE . • F IND IN THE PAPER THE EXACT LIST OF FEATURES , AND GENERATE THEM . A RMED WITH YOUR NEW SET OF FEATURES , USE AN ALGORITHM OF YOUR CHOICE TO MATCH OR BEAT THE PERFORMANCE QUOTED IN THE PAPER (NMAD: 0.0174; OLF, 4.54%). • D O

Original: LIKELY TO WORK WELL EVEN WITH A VERY WEAK BASE LEARNER ?
Revised: YOU HAVE ANY IDEAS TO GENERATE FEATURES THAT COULD BE USEFUL ?

Original: [Table]
Revised: I F SO , FEEL FREE TO GO AHEAD AND REPORT ANY IMPROVEMENT ( OR LACK THEREOF ). Y OU SHOULD MOTIVATE YOUR CHOICE OF FEATURES !

Original: 3. Elements of a teaching unit
Revised: 3. Elements of a teaching unit

Original: The strategies I have developed are especially tailored to introductory Machine Learning courses at the advanced undergraduate/early graduate level in “ofﬁcial”, grade-bearing classes.
Revised: The strategies I have developed are especially tailored to introductory Machine Learning courses at the advanced undergraduate/early graduate level in “ofﬁcial”, grade-bearing classes.

Original: I ﬁnd it helpful to integrate some traditional coursework elements (for example, slides and review questions) with more hands-on content, such as lecture notebooks and programming worksheets assignments.
Revised: I ﬁnd it helpful to integrate some traditional coursework elements (for example, slides and review questions) with more hands-on content, such as lecture notebooks and programming worksheets assignments.

Original: Often, the two sets “talk” to each other; for example, homework might include completing parts of a lecture notebook.
Revised: Often, the two sets “talk” to each other; for example, homework might include completing parts of a lecture notebook.

Original: For more advanced practitioners or more informal settings, such as summer schools, I usually skip the assignments and try to make each unit self-contained.
Revised: For more advanced practitioners or more informal settings, such as summer schools, I usually skip the assignments and try to make each unit self-contained.

Original: My approach is to present each machine learning theme (typically, a new algorithm or a discipline-wide concept, such as cross-validation or hyperparameter optimization) in parallel with a Physics or Astronomy problem (examples I have used recently include identifying potentially habitable planets, classifying the products of collision events in particle physics data, or creating a model for the rise of water in different US stations).
Revised: My approach is to present each machine learning theme (typically, a new algorithm or a discipline-wide concept, such as cross-validation or hyperparameter optimization) in parallel with a Physics or Astronomy problem (examples I have used recently include identifying potentially habitable planets, classifying the products of collision events in particle physics data, or creating a model for the rise of water in different US stations).

Original: The example materials shown here are excerpts from a teaching unit where we discuss and use ensemble methods (in particular, bagging and boosting algorithms) to solve the problem of determining distance (parameterized by redshift, z ) to faraway galaxies from the shape of their spectral energy emission.
Revised: The example materials shown here are excerpts from a teaching unit where we discuss and use ensemble methods (in particular, bagging and boosting algorithms) to solve the problem of determining distance (parameterized by redshift, z ) to faraway galaxies from the shape of their spectral energy emission.

Original: The elements of each teaching unit are the following:
Revised: The elements of each teaching unit are the following:

Original: A Power Point (or equivalent) presentation , with some blackboard discussion, which introduces the ML theme from a theoretical perspective, as well as the data set used; A Jupyter notebook lecture , with some blank parts “to ﬁll”.
Revised: A Power Point (or equivalent) presentation , with some blackboard discussion, which introduces the ML theme from a theoretical perspective, as well as the data set used; A Jupyter notebook lecture , with some blank parts “to ﬁll”.

Original: The notebook shows the process of problem-solvingfor example, setting up and optimizing the new ML method, or establishing good practices.
Revised: The notebook shows the process of problem-solvingfor example, setting up and optimizing the new ML method, or establishing good practices.

Original: The extent of the “ﬁllable” parts varies according to the class and the time availability.
Revised: The extent of the “ﬁllable” parts varies according to the class and the time availability.

Original: When possible, I like to do mini assignments (

Original: - minutes),

Original: where

Original:  students are asked to complete a single task - for example, do some exploratory data analysis, ﬁnd a “bug” in my code, or improve on a benchmark performance.
Revised: When possible, I like to do mini assignments (10minutes), where students are asked to complete a single task - for example, do some exploratory data analysis, ﬁnd a “bug” in my code, or improve on a benchmark performance.

Original: An example, including the “target performance” from a published paper (Zhou et al., 2019) and a “mini-task”, is shown in Fig.
Revised: An example, including the “target performance” from a published paper (Zhou et al., 2019) and a “mini-task”, is shown in Fig.

Original: Quizzes/review questions ; these are ungraded and usually a students’ favorite.
Revised: Quizzes/review questions ; these are ungraded and usually a students’ favorite.

Original: Most of them are in multiple-choice form and are meant to reinforce the theoretical foundations, for example with questions on the suitability of a given method to a given problem, or the parameters of a speciﬁc method.
Revised: Most of them are in multiple-choice form and are meant to reinforce the theoretical foundations, for example with questions on the suitability of a given method to a given problem, or the parameters of a speciﬁc method.

Original: In a classroom setting, these usually work well as think/pair/share material; I have also organized them as group competitions for extra credit.
Revised: In a classroom setting, these usually work well as think/pair/share material; I have also organized them as group competitions for extra credit.

Original: They also lend themselves to be used in online settings with interactive lecture softwares like Sli.do, Mentimeter, or similar.
Revised: They also lend themselves to be used in online settings with interactive lecture softwares like Sli.do, Mentimeter, or similar.

Original: A small selection of review questions for this teaching unit is shown in Table 1.

Original: Reading material
Revised: Reading material

Original: ; I include traditional ( e.g. book chapters, [Table] T ASK S TART FROM THE FULL DATA SET .
Revised: ; I include traditional ( e.g. book chapters, journal articles) and less traditional ( e.g. blog posts,

Original: W E SAW IN

Original: THE LECTURE NOTEBOOK THAT THE PERFORMANCE CHANGES A

Original: CRITERIA

Original: ARE APPLIED .

Original: • F IGURE OUT WHICH OF THE DATA CLEANING CUTS WE MADE WAS THE MOST SIGNIFICANT IN TERMS OF

Original: IMPROVING

Original: THE SCORES OF THE

Original: FINAL MODEL .

Original: N OW CHOOSE ONE REFERENCE ALGORITHM AMONG

Original: THE ONES WE SAW (RF, A DA B OOST , GBM), AND

Original: REFER TO

Original: THE OPTIMIZED MODEL ( NO NEED TO

Original: RE -

Original: RUN THE G RID S EARCH ).

Original: L ET ’ S CALL THIS M ODEL 1.

Original: U SE

Original: THE DATA SET SELECTION WITH 6,307 OBJECTS AND 6 FEATURES .

Original: • G ENERATE PREDICTIONS USING CV AND PLOT THEM IN A HISTOGRAM , TOGETHER WITH THE TRUE VALUES .

Original: W HICH DISTRIBUTION IS NARROWER ?

Original: E XPLAIN WHY .

Original: • O PTIMIZE ( USING A G RID S EARCH FOR THE PARAMETERS YOU DEEM TO BE MOST RELEVANT )

Original: THE E XTREMELY R ANDOM T REE ALGORITHM AND COMPUTE THE OUTLIER FRACTION AND NMAD.

Original: H OW DOES IT COMPARE TO M ODEL 1?

Original: C OMMENT NOT JUST ON THE SCORING PARAMETER ( S ), BUT ALSO ON VARIANCE / BIAS .

Original: W HICH ONE WOULD YOU PICK ?

Original: • I

Original: N THE PAPER THAT WE USED

Original: AS A REFERENCE (Z HOU ET AL ., 2019), THE AUTHORS ACTUALLY USE COLORS , NOT MAGNITUDES , AS FEATURES .

Original: F IND IN THE PAPER

Original: THE EXACT LIST OF FEATURES , AND GENERATE THEM .

Original: • A RMED WITH YOUR NEW SET OF FEATURES , USE AN ALGORITHM OF YOUR CHOICE TO MATCH OR BEAT THE PERFORMANCE QUOTED IN THE PAPER (NMAD: 0.0174; OLF, 4.54%

Original: journal articles) and less traditional ( e.g. blog posts,

Original: YouTube videos) resources for every unit.
Revised: YouTube videos) resources for every unit.

Original: Learners have different learning preferences/styles and I feel that it is important for them to have several options.
Revised: Learners have different learning preferences/styles and I feel that it is important for them to have several options.

Original: Some of the sources I recommend for this unit are (Louppe, 2014) and the relevant part of Chapter 5 of (VanderPlas, 2016), but also this YouTube video, this blog post and this one, and this notebook.
Revised: Some of the sources I recommend for this unit are (Louppe, 2014) and the relevant part of Chapter 5 of (VanderPlas, 2016), but also this YouTube video, this blog post and this one, and this notebook.

Original: Homework assignments ; these vary according to the class, but usually they include programming exercises that complement the material in the lecture notebook, and some non-coding tasks.
Revised: Homework assignments ; these vary according to the class, but usually they include programming exercises that complement the material in the lecture notebook, and some non-coding tasks.

Original: The latter could be, e.g. , writing pseudocode for a given algorithm, commenting code line-by-line, or reﬂecting on how to approach a problematic issue such as a severe imbalance, or missing data.
Revised: The latter could be, e.g. , writing pseudocode for a given algorithm, commenting code line-by-line, or reﬂecting on how to approach a problematic issue such as a severe imbalance, or missing data.

Original: An example assignment for this unit is shown in Table 2.
Revised: An example assignment for this unit is shown in Table 1.

Original: In a traditional course, it is useful to have a ﬁnal open-ended project that resembles a real-world application; importantly, this helps students create a “portfolio” item that can be added to a resume or brought up in job interviews.
Revised: In a traditional course, it is useful to have a ﬁnal open-ended project that resembles a real-world application; importantly, this helps students create a “portfolio” item that can be added to a resume or brought up in job interviews.

Original: Students are encouraged to use Git and GitHub for their projects.
Revised: Students are encouraged to use Git and GitHub for their projects.

Original: In undergraduate settings, I have recently settled on a two-steps research problem.
Revised: In undergraduate settings, I have recently settled on a two-steps research problem.

Original: Step 1 sets a common goal for all students ( i.e. , exploring and cleaning data, training and optimizing a model using one of the algorithms already discussed).
Revised: Step 1 sets a common goal for all students ( i.e. , exploring and cleaning data, training and optimizing a model using one of the algorithms already discussed).

Original: In Step 2, I propose possible “tracks” that can be worked on by small group of students; students choose the “track”.
Revised: In Step 2, I propose possible “tracks” that can be worked on by small group of students; students choose the “track”.

Original: The “track” could consist in learning about and deploying a new algorithm, experimenting with feature engineering, analyzing the dependence on signal-to-noise ratio, and so on.
Revised: The “track” could consist in learning about and deploying a new algorithm, experimenting with feature engineering, analyzing the dependence on signal-to-noise ratio, and so on.

Original: In graduate-level classes, students are asked to come up with their own project and data, and the products include a project plan and a ﬁnal report, ideally in the form of a research paper. 
Revised: In graduate-level classes, students are asked to come up with their own project and data, and the products include a project plan and a ﬁnal report, ideally in the form of a research paper. 4. More about teaching to physicists

Revised: Many of the teaching tools highlighted in the previous sections are fairly general, and might be useful when teaching students from various backgrounds.

Revised: So what aspects can we emphasize in particular when teaching to physicists?

Original: 4. Conclusions
Revised: Here are some that come to mind: • Literature search! This may sound like a given, but by asking students to read papers where these techniques are applied, and/or to search for related

Revised: papers, we help them improve their literature searching skills, “skimmingthrough” skills, and of course we encourage building subject matter knowledge.

Revised: • Creating exercises that rely on domain knowledge is also important.

Revised: For example

Revised: , the last few exercises

Revised: in

Revised: Table 1 require an understanding of why photometric colors, which describe gradients in the spectrum, might be better features than measures of brightness, when the goal is to measure redshift.

Revised: In the physical sciences, understanding the data before attempting feature engineering is very important, and these applications can help reinforce why as well as foster discussion of speciﬁc data issues

Revised: • It is also useful to consider how machine learning approaches compare to other solutions.

Revised: In the present example of photometric redshifts, it is possible to use templates of galaxy spectra to obtain an estimate of the redshift via inference methods.

Revised: Physics and Astronomy students will be able to understand both approaches: a useful class discussion could be the comparison of pros and cons of inference versus machine learning.

Revised: • Finally, in Physics, we never want to quote a result without stating some level of conﬁdence in our estimate.

Revised: In this case, we could reﬂect on possible sources of uncertainties, from, e.g., the size of training set, the chosen architecture, and the available features, to the experimental uncertainties on the data; sometimes they are classiﬁed as epistemic and aleatoric uncertainties, see e.g. (Kendall & Gal, 2017).

Revised: The former refer to the data set as a whole, while the latter can also be characterized on an object-by-object basis.

Revised: Discussing how to estimate these uncertainties, and how to establish a fair comparison with existing results, is very important.

Revised: 5. Conclusions

Original: My conclusions are the following:
Revised: My conclusions are the following:

Original: • It is important to include ML techniques in Physics (and more generally, STEM) curricula, as they are useful for both academic and non-academic careers; • Using a mix of techniques, from traditional lectures to hands-on programming exercises, and recommending varied learning resources helps meet the needs of different learners; • Practical projects are important, because they teach students to be comfortable with open-ended questions, and help them build a portfolio; students can be encouraged to post them on platforms like GitHub; • Preparing good sets of materials is important not just for students, who tend to be resourceful and resilient, but also to widen the pool of instructors who can teach this subject; • Persisting challenges include: 1.
Revised: • It is important to include ML techniques in Physics (and more generally, STEM) curricula, as they are useful for both academic and non-academic careers; • Using a mix of techniques, from traditional lectures to hands-on programming exercises, and recommending varied learning resources helps meet the needs of different learners; • Practical projects are important, because they teach students to be comfortable with open-ended questions, and help them build a portfolio; students can be encouraged to post them on platforms like GitHub; • Preparing good sets of materials is important not just for students, who tend to be resourceful and resilient, but also to widen the pool of instructors who can teach this subject; • Persisting challenges include: 1.

Original: Improving access to data sets and problems at the right level of complexity and size; 2.
Revised: Improving access to data sets and problems at the right level of complexity and size; 2.

Original: Finding effective ways to teach across-discipline concepts, such as uncertainty estimation or interpretability, that are important but don’t ﬁt in the algorithm/problem mold, and 3.
Revised: Finding effective ways to teach across-discipline concepts, such as uncertainty estimation or interpretability, that are important but don’t ﬁt in the algorithm/problem mold, and 3.

Original: Integrating the conversation around ML topics with content taught e.g. in Statistics or Computational Methods courses.
Revised: Integrating the conversation around ML topics with content taught e.g. in Statistics or Computational Methods courses.

Revised: All the materials from the most recent iteration of this class are available here (just send a sharing request, so I can keep track of who has access).

Revised: Acknowledgements

Revised: A big big thank you goes to all my ML students throughout the years.

Revised: You have inspired me so much, put up with me trying out new things, and provided great feedback, with kindness and humour.

Revised: A special thanks to some of you from my ﬁrst cohort: Hashir Qureshi, Harpreet Gaur, George

Revised: Nwankwo, Kayla Ford and Charlie Meyers, and to Ashwin

Revised: Satyanarayana, who accepted to be my travel-mate in the crazy adventure of conjuring a course out of thin air and zero money as usual.

Revised: I am also grateful to Andy Lawler,

Revised: Elena Filatova and Johann Thiel, who have kindly guestlectured for my class at City Tech so many times, and have enriched the experience of students with their competence and passion.

Revised: Here is to many more years of learning and improving!

Revised: Some of the work described here was partially sponsored by a CUNY “Research in the Classroom” Award (RIC 451).

Revised: I am grateful to the Center for Computational Astrophysics of the Flatiron Institute for their continued hospitality and support.