Automating teaching about automation in Python
I heard you like automation, so I put some automation in your automation

Florian Bruhin

Swiss Python Summit 2022
September 22nd

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

About me

Florian Bruhin, @The-Compiler
2006 Started programming (QBasic, bash)
2011 Started using Python
2013 Started developing qutebrowser
2015 Switched to pytest, ended up as a maintainer
2016 – 2019 BSc. in Computer Science at HSR OST
40% employed (OST): Teaching Automation with Python to first-semester students
60% open-source and freelancing (Bruhin Software): Python, pytest, Qt

The problem• Pre-2021: Students learn Java as their primary programming language
at OST• Java can be a pain to deal with[citation needed]...especially if you want a tool to make your life easier rather than learn
the fundamentals of programming

The problem• Pre-2021: Students learn Java as their primary programming language
at OST• Java can be a pain to deal with[citation needed]...especially if you want a tool to make your life easier rather than learn
the fundamentals of programming• More and more places where Python is used as a tool
(to teach math, physics, AI, but also projects, final thesis, etc.)• Students demand learning Python in their studies...and lots of schools/universities have introduced/switched to it

The solution• Fall semester 2021: New course Automatisierung mit Python(Automation with Python) for all first-semester IT students• In addition to Java, but with a different goal: Solving real-life problems!
• “Students will be able to use the Python programming language [. . . ] for simple
and complex automation tasks.”

The solution• Fall semester 2021: New course Automatisierung mit Python(Automation with Python) for all first-semester IT students• In addition to Java, but with a different goal: Solving real-life problems!
• “Students will be able to use the Python programming language [. . . ] for simple
and complex automation tasks.”• Flipped classroom: No lectures, no paper exam.
Interactive graded labs and a small graded project!
• We have many newcomers studying IT, or people mostly doing
support/network/. . . , without much programming experience.
You need to get your hands dirty to learn programming.
It’s not just theory, but also a “craft”!
We want students to learn both: University of Applied sciences!

Interactive learning
Labs

Interactive learning
Exercises

Interactive learning
Tests with testbook

Interactive learning
Grading• 1 ungraded lab (setup and getting started)• 5 graded labs, 1/3 of final grade, automated tests• Final project, 2/3 of final grade, graded manually• Python basics, flow control, data structures, . . .
• Writing a CLI• Using web APIs

The... problem?
Over 120 students, a total of 9 slots (4 hours each) every 2 weeks.
Slightly less this year: 110 or so, and “only” 7 slots.

That’s a lot given that I’m doing this the first time!
Thanks, Stefan Richter, for trusting that we could pull it off.

In addition:• I love writing opensource (qutebrowser/pytest), andgiving company trainings• Thus, this needs to stay a 40% occupation(averaged over a year, I don’t teach in spring)• Other people are busy too! But I got some help.
Thanks, Marco, Méline and Urs!

Focusing my attention• With > 100 students, any kind of manual work with
O(n) is almost certainly worth automating!
• I teach students how to make their studies easier.
Might as well make my job easier!
• I want to focus on the interesting part:
Creating an environment to help people learn,
helping people who are stuck, the beauty of teaching.

Focusing my attention• With > 100 students, any kind of manual work with
O(n) is almost certainly worth automating!
• I teach students how to make their studies easier.
Might as well make my job easier!
• I want to focus on the interesting part:
Creating an environment to help people learn,
helping people who are stuck, the beauty of teaching.
• Let Python do the boring part. Bonus points:
It gets easier every year, because more is automated!
• A word of caution: Automation is not a substitute forteaching. Know where to stop!
• I’m not lazy (...sometimes) – but I want to focus myattention on things which benefit students most.

The danger of automation

XKCD 1319, Randall Munroe / xkcd.com

The real danger

Eh, I just need a handful of very simple scripts.
I won’t bother setting up...
• ...a proper Python package• ...type annotations• ...linters / formatters• ...tests (Yes, I’m a pytest maintainer. Yes, I’m ashamed.)

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

Getting progress about final project hand-ins
Grading final projects, sending feedback

350 lines
680 lines

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

Getting progress about final project hand-ins
Grading final projects, sending feedback

350 lines
680 lines

Checking / resetting to deadlines“pseudotaas”: Rerunning tests locallyCalculating final grades

180 lines
300 lines
550 lines

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

Getting progress about final project hand-ins
Grading final projects, sending feedback

350 lines
680 lines

Checking / resetting to deadlines“pseudotaas”: Rerunning tests locallyCalculating final grades

180 lines
300 lines
550 lines

45 Python files

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

Getting progress about final project hand-ins
Grading final projects, sending feedback

350 lines
680 lines

Checking / resetting to deadlines“pseudotaas”: Rerunning tests locallyCalculating final grades

180 lines
300 lines
550 lines

45 Python files
6500 lines
5000 LOC (tokei)
not including:
950 lines of solutions
3000 lines tests for labs

The real danger: Whooops
Development / Deployment scripts
Scraping participants
Sending welcome mails / other mail code

730 lines
600 lines
370 lines“taas”: Testing As A Service“AutPy Commander” GUI
Checking lab submission progress
Misc. utilities for handling data

430 lines
1300 lines
480 lines
390 lines

Getting progress about final project hand-ins
Grading final projects, sending feedback

350 lines
680 lines

Checking / resetting to deadlines“pseudotaas”: Rerunning tests locallyCalculating final grades

180 lines
300 lines
550 lines

45 Python files
6500 lines
5000 LOC (tokei)
not including:
950 lines of solutions
3000 lines tests for labs

Success!

1. I was there

8. Understandable

2. Contents important

9. Speed good

15. Link between
theory/practice

3. Contents interesting

10. Extra material / media

16. Didactics

4. Needed time is high

11. Link between lect./ex.

17. Lecturer engaged

5. Content matches desc.

12. Room for questions

18. Mutual respect

6. Useful material

13. Checking progress

19. Room / environment

7. Well structured

14. Lecturer competent

20. OverallSuccess!

• Concept of “Interactive learing” / “flipped classroom” as a whole.
Daniele Procida / EvilDMP of Diátaxis1 :“I hardly believe in teaching anymore. The best thing you can do is creating anenvironment where people learn.”• Using git as a “database” for student submissions, with one branch per student• Including test logs (HTML + JSON) in the commits• Having a custom GUI tool to view a student’s submission and test report• Project grading based on parsing Markdown checklists• ...all the other automation really, not regretting any of the time spent on it!

1

diataxis.fr, structuring docs into tutorial/how-to/explanation/referenceIssues• Students accidentally deleting Jupyter cell tags
⇒ Tooling to notify us, protecting cells• OST GitLab admins migrating storage without making it read-only⇒ Thankfully I had the lost commits locally... please don’t do that again!
• Various smaller issues with Jupyter cluster⇒ Ample deadline extension for affected students, we’re humans!

Issues• Students accidentally deleting Jupyter cell tags
⇒ Tooling to notify us, protecting cells• OST GitLab admins migrating storage without making it read-only⇒ Thankfully I had the lost commits locally... please don’t do that again!
• Various smaller issues with Jupyter cluster⇒ Ample deadline extension for affected students, we’re humans!
• Me forgetting to check some checkboxes in the project grading checklists⇒ Human mistakes bound to happen with so many students and days of grading.
Caught thanks to detailed feedback mails being sent, additional sanity checks• Grading system’s API is “Download a template .xlsx, add grades, upload”...but openpyxl somehow corrupts template.
⇒ Needs further debugging, until then, copy-paste all grades once in LibreofficeIssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

IssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

•

Ctrl

+ C , Ctrl + V

IssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

•

Ctrl

+ C , Ctrl + V• “Allgemeiner Verpeiltheitsfaktor”
Student was in military (“WK”) for weeks, “didn’t know” they had to hand in stuff

IssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

•

Ctrl

+ C , Ctrl + V• “Allgemeiner Verpeiltheitsfaktor”
Student was in military (“WK”) for weeks, “didn’t know” they had to hand in stuff• r = requests.get("https://random.dog/woof.json")data = eval(r.content)IssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

•

Ctrl

+ C , Ctrl + V• “Allgemeiner Verpeiltheitsfaktor”
Student was in military (“WK”) for weeks, “didn’t know” they had to hand in stuff• r = requests.get("https://random.dog/woof.json")data = eval(r.content)• git --config user.name "student.email@example.com"
git --config user.email "password-for-said-email"IssuesThings nobody can prepare you for...
Disclaimer: I don’t like calling students out for their mistakes, making mistakes is normal.
But those occurrences are just too strange to not tell you about...

•

Ctrl

+ C , Ctrl + V• “Allgemeiner Verpeiltheitsfaktor”
Student was in military (“WK”) for weeks, “didn’t know” they had to hand in stuff• r = requests.get("https://random.dog/woof.json")data = eval(r.content)• git --config user.name "student.email@example.com"
git --config user.email "password-for-said-email"⇒ With >100 students, prepare to see every corner case you can think of,and some you’d never think of. Automation won’t help you take difficult decisions.

Automation examples
Before the semester

Automation examples
Before the semester

XKCD 1205, Randall Munroe / xkcd.com

Automation examples
Before the semester

No automation needed?

XKCD 1205, Randall Munroe / xkcd.com

Automation examples
Before the semester

No automation needed?
• People get added last-minute• ...after preparing• ...even after semester started

XKCD 1205, Randall Munroe / xkcd.com

Automation examples
Before the semester

No automation needed?
• People get added last-minute• ...after preparing• ...even after semester started• People leave in the middle ofthe semester• ...and nobody tells you

XKCD 1205, Randall Munroe / xkcd.com

Automation examples
Before the semester• No API (as far as I know)• Lots of data (200 KB of JSON) in window.adunisModel = ...
• Not what we need, however...
• But HTML is structured enough. requests and bs4 to the rescue!

Automation examples
Before the semester• No API (as far as I know)• Lots of data (200 KB of JSON) in window.adunisModel = ...
• Not what we need, however...
• But HTML is structured enough. requests and bs4 to the rescue!
• Weird Microsoft-based login flow
• Couldn’t figure out how it works, libraries seem to be for APIs only

Automation examples
Before the semester• No API (as far as I know)• Lots of data (200 KB of JSON) in window.adunisModel = ...
• Not what we need, however...
• But HTML is structured enough. requests and bs4 to the rescue!
• Weird Microsoft-based login flow
• Couldn’t figure out how it works, libraries seem to be for APIs only
• Whatever, I write a browser since 2013, and I can access cookies• Login via QtWebEngine browser (injected JS to fill values)• Grab session cookie, feed it to requests

Automation examples
During the semester: Commander

Automation examples
During the semester: Overview

Automation examples
During the semester: Grep

Automation examples
Towards end of semester: Project overview

Automation examples
After end of semester: Project grading

Project grading workflow:• Pick random student• Get zip from submissions repository• Unpack zip in “grading-area” folder

Automation examples
After end of semester: Project grading

Project grading workflow:• Pick random student• Get zip from submissions repository• Unpack zip in “grading-area” folder• Show overview (file list, detected features)• Prepare checklist and open in editor• Wait until editor closed

Automation examples
After end of semester: Project grading

Project grading workflow:• Pick random student• Get zip from submissions repository• Unpack zip in “grading-area” folder• Show overview (file list, detected features)• Prepare checklist and open in editor• Wait until editor closed• Parse checklist• Show parsed points and grade, wait for confirmation• Commit grading file to submissions repository• ...and during the whole process, only show names as rot13

Automation examples
After end of semester: Project grading

Project grading workflow:• Pick random student• Get zip from submissions repository• Unpack zip in “grading-area” folder• Show overview (file list, detected features)• Prepare checklist and open in editor• Wait until editor closed• Parse checklist• Show parsed points and grade, wait for confirmation• Commit grading file to submissions repository• ...and during the whole process, only show names as rot13

ABCDEFGHIJKLMNOP. . .
↓↑NOPQRSTUVWXYZABC. . .

Automation examples
After end of semester: Project grading

Project grading workflow:• Pick random student• Get zip from submissions repository• Unpack zip in “grading-area” folder• Show overview (file list, detected features)

ABCDEFGHIJKLMNOP. . .
↓↑NOPQRSTUVWXYZABC. . .

• Prepare checklist and open in editor• Wait until editor closed• Parse checklist• Show parsed points and grade, wait for confirmation• Commit grading file to submissions repository• ...and during the whole process, only show names as rot13

florian.bruhin↓
sybevna.oehuva

Automation examples
After end of semester: Project grading# Functionality (24P)- Data download / reading (7P)
- [ ] Download URL is obtained via API (2P)
- [ ] Latest available data set used by default (1P)
- ...
- [ ] Searching for dogs (2P)
- Statistics (9P)
- [ ] Longest dog name is output correctly (0.5P)- [ ] Shortest dog name is output correctly (0.5P)- [ ] Top 10 is output correctly (1P)- ...

Automation examples
After end of semester: Project grading• Parse Markdown
checklist• Calculate points• Send HTML +plaintext mails

Automation examples
After end of semester: Final grade• Rerun all test-cases
local, parallelized, 1
Docker container per
student• Calculate final grade
Using fractions,no rounding!
• Send HTML +plaintext mails

Next steps• Teaching another ≈ 100 students, with many small improvements!
• Grading another ≈ 100 student projects...

Next steps• Teaching another ≈ 100 students, with many small improvements!
• Grading another ≈ 100 student projects...
• Turning this into a proper package (autpy? autpypy? pyautpy? autmetapy?)• Type annotations and autoformatters (done!)• Tests for all the automation logic...
• Using GitPython/pygitops/pygit2/Dulwich/Gittle/. . . instead of subprocess(nicer API and performance)Next steps• Teaching another ≈ 100 students, with many small improvements!
• Grading another ≈ 100 student projects...
• Turning this into a proper package (autpy? autpypy? pyautpy? autmetapy?)• Type annotations and autoformatters (done!)• Tests for all the automation logic...
• Using GitPython/pygitops/pygit2/Dulwich/Gittle/. . . instead of subprocess(nicer API and performance)• Maybe: Generalizing and publishing?

https://fstring.help
https://twitter.com/the_compiler
florian@bruhin.software