2.1 Day to day cluster work - Tuesday tools and techniques for HPC
Channel: CodeRefinery
Date: 23/04/2024

Today, we look at ways of working on the cluster, with a focus on interactive working.  We go over project arrangement, moving data around (one way transfer and two-way syncing), planning for these transfers, code syncing with git, graphical interfaces, working interactively (interactive jobs, screen/tmux), and VS Code for HPC.

https://coderefinery.github.io/TTT4HPC_Interactive/

00:00 Introduction
05:03 § Motivation
11:16 § Project arrangement
29:06 § Data sync: moving data around
33:24 Data sync: rsync (one-way)
43:33 Data sync: unison (two-way)
50:46 Data sync: git-annex
54:41 Data sync: sshfs (remote mount via ssh)
59:50 § Code sync
1:05:02 § Graphical interfaces
1:05:23 Graphical interfaces: X forwarding
1:11:49 Graphical interfaces: Open OnDemand
1:13:10 § Working interactively
1:13:50 Working interactively: Interactive jobs
1:20:03 Working interactively: screen/tmux
1:26:31 § VS Code on HPC
1:29:20 VS Code: Remote-SSH
1:34:59 VS Code: transferring files
1:37:31 VS Code: shell and Slurm
1:40:59 VS Code: debugging (debugpy)
1:49:10 Wrap up, what is next week

-----

Do you use supercomputers in your research work? Are you curious about making your computing faster and more efficient? Join us for TTT4HPC: four self-contained episodes on best practices in High Performance Computing. This is a great chance to enhance your computational skills. What you will learn is also used a lot outside academia whenever large scale computations are needed.

Improve your cluster workflows! You've had a basic course on working with a cluster - but what do people actually do? This course shows practical examples and tips which may help you, but aren't usually covered in either basic or advanced courses.

We won't go so deep into each topic that you know everything about it, but you learn what is possible, see some examples, and know where to follow up on the tools that are right for you. The format is designed to be "fireside chat with experts", a combination of demos, type-along, and independent exercises. You can engage at different levels, depending on your interest.

Links:

- Playlist: https://www.youtube.com/playlist?list=PLpLblYHCzJABy4epFn-rqsfDbUZ1ff5Pl

- Workshop webpage: https://scicomp.aalto.fi/training/scip/ttt4hpc-2024/

- CodeRefinery: https://coderefinery.org