DevOps for research
Jun Kato
AIST, Arch Inc., Universite Paris-Saclay#non_research_tips #情報科学研究補助技法

Icons by FontAwesome, licensed under SIL Open Font License 1.1. For more information, visit: https://fontawesome.com/license/free2024/6/26#non_research_tips #情報科学研究補助技法

1Links to open-source code in this lecture
arcatdmz/non-research-tips-gitlab-ci••

GitLab CI/CD simple example
https://gitlab.com/arcatdmz/non-research-tips-gitlab-ci

arcatdmz/non-research-tips-devcontainer•••

VS Code DevContainer for consistent development experienceNode.js + MySQL + Prisma
https://github.com/arcatdmz/non-research-tips-devcontainer

arcatdmz/acm2024-primary-article-template•••

Editing, linting, and sharing papers
Overleaf + GitHub Actions + GitHub Pages + Slackhttps://github.com/arcatdmz/acm2024-primary-article-template

ChezTachibana/website••

Cloudflare Pages + Google Sheets
https://github.com/ChezTachibana/website

arcatdmz/arcatdmz.github.io•••

Single source of truth for multiple views
Gulp + Pug + BibTeX etc.
https://github.com/arcatdmz/arcatdmz.github.io

2024/6/26#non_research_tips #情報科学研究補助技法2

My research style: a toolsmith researcher
A researcher who builds computational tools for creative people(creativity support tools, toolkits, programming environments)© Crypton Future Media, INC. www.piapro.net / © SEGA
Graphics by SEGA / MARZA ANIMATION PLANET INC.
Organized by TOKYO MX / Crypton Future Media, INC.

Programmers

Musicians

Anime directors

DejaVu [UIST '12], Picode [CHI ‘13
Honorable Mention],
TouchDevelop [PLDI ‘13]

TextAlive [CHI ‘15 Honorable Mention],
Lyric App Framework[CHI ‘23 Honorable Mention]

Griffith [SAS '21 (non-archival),
ISID '21 & '22 Best Poster Awards,
CHI '24]More examples at https://junkato.jp/projects

2024/6/26#non_research_tips #情報科学研究補助技法

3

Last week: research is not a dead-end
Society
Academic society“HCI Research as Problem-Solving” Antti Oulasvirta [ACM CHI 2016]
2024/6/26#non_research_tips #情報科学研究補助技法

4

Let’s go against irony of expertise• 医者• Japan: 医者の不養生• 靴屋• France: Les cordonniers sont les plus mal chaussés• Germany: Der Schuster hat die schlechtesten Leisten• UK: The shoemaker's children go barefoot• 鍛冶屋• Spain: En casa de herrero, cuchillo de palo

2024/6/26#non_research_tips #情報科学研究補助技法

5

DevOps for research: agenda• Development environment setup• Continuous integration/delivery• Web technologies for development/public relations

2024/6/26#non_research_tips #情報科学研究補助技法

6

Development environment
setup
DevOps for research2024/6/26#non_research_tips #情報科学研究補助技法

7

Development environment setup• Package managers for Windows/macOS• Two-factor authentication for improved security• SSH config for custom networking• DevContainer for consistent development experience2024/6/26#non_research_tips #情報科学研究補助技法

8Package managers for Windows/macOS• Windows• winget [recommended]• scoop, Chocolatey• macOS• Homebrew [recommended]• macPorts

2024/6/26#non_research_tips #情報科学研究補助技法

9Two-factor authentication
for improved security• Microsoft Authenticator• Authy [recommended]• 1Password

2024/6/26#non_research_tips #情報科学研究補助技法

10

SSH config• Cf. Week 1: SSH + server coding• You can do a lot with SSH, probably more than you'd expect

2024/6/26#non_research_tips #情報科学研究補助技法

11

SSH config case studies 1:

ProxyJump for jumping hosts
SSH access• Suppose you have a server that can only be accessed fromwithin a restricted network that includes another server• You have SSH keys to access both the bastion host(intermediate server) and the final destination server
2024/6/26#non_research_tips #情報科学研究補助技法

12

SSH config case studies 1:

ProxyJump for jumping hosts
SSH access [1]2024/6/26#non_research_tips #情報科学研究補助技法

SSH access [2]13SSH config case studies 2:

LocalForward for remote service access
SSH access• Suppose you have a database server that can only beaccessed from within a restricted network that includes anapp server• You have SSH access to the app server, but not to thedatabase server
2024/6/26#non_research_tips #情報科学研究補助技法

14

SSH config case studies 2:

LocalForward for remote service accesslocalhost:13306
SSH access

2024/6/26#non_research_tips #情報科学研究補助技法

rds:3306

15

SSH config case studies 3:

DynamicForward for SOCKS v5 proxy
SSH access• Suppose you have a service that can only be accessed fromwithin a restricted network that includes an app server• You have SSH access to the app server, but the service runs
on multiple servers to be accessed via a web browser
2024/6/26#non_research_tips #情報科学研究補助技法

16

SSH config case studies 3:

DynamicForward for SOCKS v5 proxy
localhost:12345
SSH access

2024/6/26#non_research_tips #情報科学研究補助技法

17

SSH config case studies 2:

DynamicForward for SOCKS v5 proxyFirefox browser
config:Network Settings

2024/6/26#non_research_tips #情報科学研究補助技法

18

SSH config case studies 2:

DynamicForward for SOCKS v5 proxyFirefox browser
config:Network Settings >Manual proxyconfiguration >SOCKS Host, Port,
SOCKS v52024/6/26#non_research_tips #情報科学研究補助技法

19

DevContainer (VS Code feature)• Prerequisite: Docker Desktop (or Rancher Desktop)• Useful feature for Windows: WSL2
• DevContainer for consistent development experience

2024/6/26#non_research_tips #情報科学研究補助技法

20

DevContainer (VS Code feature):

Prerequisite
Docker Desktop

Rancher Desktop• Free for educational, noncommercial, personal, andsmall business use• Completely OSS, free foranyone who agrees theterms of use• docker and docker-compose command support• aaa2024/6/26

https://www.docker.com/wpcontent/uploads/2023/08/dockerdesktop-hero-v2.svg#non_research_tips #情報科学研究補助技法

https://suse-ranchermedia.s3.amazonaws.com/desktop/v
1.14/uimain/Windows_Containers.png
21

DevContainer (VS Code feature):

Useful Windows feature: WSL2
• Windows Subsystem for Linux 2• To install and start using bash on Ubuntu:• wsl --install in the administrator privileged console• Restart system• wsl --set-default-version 2 in the admin console• bash in console

2024/6/26#non_research_tips #情報科学研究補助技法

22

DevContainer (VS Code feature) for

consistent development experience• Use the same runtime (e.g., Node.js)• Use the same package manager (e.g., npm)
• Use the same backend server (e.g., MySQL)• Use the same VS Code extensions, editor config, etc.
…
Want a consistent development experience across machines?

DevContainer has you covered!
2024/6/26#non_research_tips #情報科学研究補助技法

23

DevContainer (VS Code feature) for

consistent development experience
Example devcontainer.json

2024/6/26#non_research_tips #情報科学研究補助技法

24

DevContainer (VS Code feature) for

consistent development experience
Example docker-compose.yml

2024/6/26#non_research_tips #情報科学研究補助技法

25

By the way...

Share beautiful images of your code
https://carbon.now.sh/2024/6/26#non_research_tips #情報科学研究補助技法

26

Continuous integration/continuous delivery
DevOps for research2024/6/26#non_research_tips #情報科学研究補助技法

27

Continuous integration/continuous delivery• GitHub or GitLab?
• Self-hosted runners• CI/CD use cases 1: linting papers• CI/CD use cases 2: running heavy workloads on clouds• CI/CD use cases 3: ChatGPT integration• CI/CD use cases 4: publishing websites

2024/6/26#non_research_tips #情報科学研究補助技法

28

GitHub or GitLab?
from CI/CD perspective
GitHub Actions

GitLab CI/CD• Not much difference for basic usage• writes config under• writes config in ~/.gitlab~/.github/workflows/
ci.yml• Pros: Has a vast library of• Pros: Can be hosted entirely
community-definedon your own servers without“Actions” useful for various
worrying much about
purposesresource limits• Cons: You’re vendor-locked! • Cons: Maintenance costs
2024/6/26#non_research_tips #情報科学研究補助技法

29GitLab CI/CD basic example
Example .gitlab-ci.yml

2024/6/26#non_research_tips #情報科学研究補助技法

30

Self-hosted runners
Basics and usages• GitHub Actions and GitLab CI/CD both support self-hosted
runners – CI/CD workers that run on machines under yourcontrol, such as modern NAS devices and various servers• These are especially useful if• you have a handy machine (or some special machines) and wantto avoid being charged for running an automated process• you have a deployment target on cloud computing platforms andwant to securely update running services on each push
GitHub self-hosted runnerGPU-enabled server / web app server / ...
2024/6/26#non_research_tips #情報科学研究補助技法

31CI/CD case studies 1:

Editing, linting, and sharing papers
Overleaf + GitHub Actions + GitHub Pages + Slack• Overleaf has a good GitHub integration by default• GitHub Actions can automatically process pushed TeX files• GitHub Actions can upload the end result to GitHub Pages• GitHub Actions can post a message to Slack upon completion
https://github.com/arcatdmz/acm2024-primary-article-templateLet's look at the actual code on GitHub:
2024/6/26#non_research_tips #情報科学研究補助技法

32CI/CD case studies 2:

Running heavy workloads on clouds
Amazon EC2 + GitHub + GitHub Actions• Automated self-hosted runner management with GitHub
personal access token of repo scope• Automated EC2 instance management with AWS access keys
with permissions for ec2:* actions• On-demand tests/experiments with EC2 instance!
https://github.com/machulav/ec2-github-runner

2024/6/26#non_research_tips #情報科学研究補助技法

33CI/CD case studies 3:

ChatGPT integration
GitHub + GitHub Actions + OpenAI Platforms API
For each GitHub comment,• look for harmful text (moderations endpoint)
https://github.com/takahiroanno2024/election2024/blob/ma
in/.github/workflows/comment-review.yml#L89-L96• look for URLs and check if they look harmful
https://github.com/takahiroanno2024/election2024/blob/ma
in/.github/workflows/comment-review.yml#L54-L87

2024/6/26#non_research_tips #情報科学研究補助技法

34CI/CD case studies 4:

Publishing web services

To be covered in the next section!
2024/6/26#non_research_tips #情報科学研究補助技法

35

Web technologies fordevelopment/public relations
DevOps for research2024/6/26#non_research_tips #情報科学研究補助技法

36

Web technologies
for development/public relations• Crowdsourcing• npx + Prettier for code beautification• Infrastructure for web deployments• Languages and frameworks• Tips for making informative portfolio pages

2024/6/26#non_research_tips #情報科学研究補助技法

37

Crowdsourcing
https://www.mturk.com/• Amazon Mechanical Turk, Yahoo! Crowdsourcing cf. acaric
website• Crowdsourcing and LLMs (search llm modeling human behaviors
to replace crowdsourcing or visit HCOMP website)2024/6/26#non_research_tips #情報科学研究補助技法

38npx + Prettier for code beautification• npx comes with Node.js by default and Prettier is a wellknown OSS code formatter• npx prettier --write {your files} is useful, even if you
don’t write JavaScript (e.g, HTML and Markdown files; morelanguages if combined with plugins)• npx http-server {your directory} is also useful• See also: live-server, localtunnel, cpx, cross-env, ...

Tell me your favorite npm package!
2024/6/26#non_research_tips #情報科学研究補助技法

39Infrastructure for web deployments• For static sites: GitHub Pages and Cloudflare Pages• For a bit more complex sites: Netlify and Vercel• For backends: PlanetScale and... Google Sheets!
• For professional-minded developers: AWS, GCP, Azure

2024/6/26#non_research_tips #情報科学研究補助技法

40

Web technologies case studies 1:

Cloudflare Pages + Google Sheets• Google Forms• Google Sheets

Google Sheets
Cloudflare Pages

https://github.com/ChezTachibana/website

Let's look at the actual code on GitHub:
2024/6/26#non_research_tips #情報科学研究補助技法

41

Languages and frameworks• HTML, Pug, Markdown• React (JSX/TSX), MDX, and more!
• Responsive design for free with CSS frameworks• Pretty icons with FontAwesome and alternatives

2024/6/26#non_research_tips #情報科学研究補助技法

42Languages and frameworks

HTML, simplified
PugMarkdown• html• body• h1#mytitle Main title• p This is the main paragraph• ol.myclass• # Main title {#mytitle}• This is the main paragraph• li This is the first item
• li second item
• li third item

• 1. This is the first item
• 2. second item
• 3. third item

• p You can add• a(href=“#mytitle") links• | pretty easily!
2024/6/26• You can add [links](#mytitle)
pretty easily!

#non_research_tips #情報科学研究補助技法

43

Languages and frameworks

HTML, evolved
React (*.jsx)

MDX// Every user interface component is a function

import { MyButton } from"../src/MyButton";export function MyButton = () => {const handleClick = useCallback(() => { … }, []);
return <Button onClick={handleClick}className=“my-button">My button!</Button>}

2024/6/26

This is the main paragraphwith a custom <MyButton />.

#non_research_tips #情報科学研究補助技法

44Languages and frameworks

CSS frameworks for responsive design• Bootstrap• Semantic UI / Fomantic UI• and more!

Search for “CSS frameworks”
2024/6/26#non_research_tips #情報科学研究補助技法

45

Tips for making informative portfolio pages• Stick to static sites, allow easy copy and paste• Show the story, not only the papers• Single source of truth, multiple views for multiple purposes

2024/6/26#non_research_tips #情報科学研究補助技法

46

Web technologies case studies 2:Portfolio web pagehttps://github.com/arcatdmz/arcatdmz.github.io

Let's look at the actual code on GitHub:
2024/6/26#non_research_tips #情報科学研究補助技法

47

DevOps for research: agenda• Development environment setup• Continuous integration/delivery• Web technologies for development/public relations

2024/6/26#non_research_tips #情報科学研究補助技法

48

DevOps for research2024/6/26#non_research_tips #情報科学研究補助技法

49