# Presentation: 144281

## Lecture 1Basic Skills

- Presenter Name
- Presenter Institution
- Presenter email address
-                  Grid Summer Workshop
-                        June 21-25, 2004

- Lecture 1: Basic Skills

- June 21-25, 2004

## Welcome!

- You are at the Grid Summer School
- If you are looking for the underwater basket weaving classes, they are two block to the west.
- Sponsored by:
- Center for Gravitational Wave Astronomy
- GriPhyN
- iVDGL
- NMI GRIDS Center

- Lecture 1: Basic Skills

- June 21-25, 2004

## Goals for this week

- Learn about Grids
- What is a grid?
- How do you...
- Submit jobs?
- Transfer data?
- Cope with failure?
- Where are Grids going?
- Use grids
- Hands-on exercises
- Local and remote grids

- Format of the week
- Morning
- Lecture
- Exercises
- Lunch! Mmm...
- Afternoon
- Lecture
- Exercises

- Lecture 1: Basic Skills

- June 21-25, 2004

## Day 1

- Basic Skills
- What is a Grid?
- Basics of Using a Grid

- Lecture 1: Basic Skills

- June 21-25, 2004

## Day 2

- Running a real application on a grid
- What requirements do real applications have?
- How should you build a real application?
- Taking care of an application
- Condor-G
- DAGMan
- Staging your application

- Lecture 1: Basic Skills

- June 21-25, 2004

## Day 3

- Lecture 1: Basic Skills

- June 21-25, 2004

## Day 4

- Lecture 1: Basic Skills

- June 21-25, 2004

## Day 5

- Lecture 1: Basic Skills

- June 21-25, 2004

## Without further adieu... 

- Lecture 1: Basic Skills

- June 21-25, 2004

## Step 1: Basic Networking Skills

- We apologize in advance if you already know these concepts.
- We want to make sure that there is a certain basic level of understanding for all students.

- Lecture 1: Basic Skills

- June 21-25, 2004

## IP Addresses

- All computers on the Internet use TCP/IP.
- All computers have at least one IP address.
- 32-bit number
- Written as four numbers, like: 128.105.3.61
- An IP Address identifies a network interface, not a computer.
- A computer can have multiple IP addresses.

- Lecture 1: Basic Skills

- June 21-25, 2004

## DNS

- DNS maps IP addresses to names, and vice-versa
- www.amazon.com  207.171.163.30
- Discover this with “host” or “nslookup” or “dig”
- Try all three—how do they differ?

- Lecture 1: Basic Skills

- June 21-25, 2004

## Whois

- Who owns or runs a domain?
- Amazon.com:
-   Amazon.com, Inc.  (HOS276-ORG)          hostmaster@AMAZON.COM
-   PO BOX 81226
-   SEATTLE, WA 98108-1300
-   US
-   +1 206 266 4064 fax: +1 206 266 7010
- Discover this with “whois”.

- Lecture 1: Basic Skills

- June 21-25, 2004

## Ping! Are you awake?

- Is a computer on the network?
- Use ping to find out
- % ping www.cs.wisc.edu
- PING marzipan.cs.wisc.edu (128.105.7.11) 56(84) bytes of data.
- 64 bytes from marzipan.cs.wisc.edu (128.105.7.11): icmp_seq=1 ttl=63 time=0.327 ms
- 64 bytes from marzipan.cs.wisc.edu (128.105.7.11): icmp_seq=2 ttl=63 time=0.241 ms
- --- marzipan.cs.wisc.edu ping statistics ---
- 2 packets transmitted, 2 received, 0% packet loss, time 2611ms
- rtt min/avg/max/mdev = 0.241/0.276/0.327/0.036 ms

- Lecture 1: Basic Skills

- June 21-25, 2004

## Internet routes

- Between you and a computer on the network, there is an often complex route.
- % traceroute www.cs.uwm.edu
- traceroute to miller.cs.uwm.edu (129.89.143.24), 30 hops max, 40 byte packets

- 1  svi-121.cisco1.cs.wisc.edu.105.128.in-addr.arpa (128.105.121.248)  0.423 ms  0.242 ms  0.227 ms

- 2  rh-cssc-b280c-2-core-vlan-492.net.wisc.edu (144.92.128.186)  0.404 ms  4.985 ms  0.489 ms

- ... snip...

- 6  r-uwmilwaukee-isp-atm1-0-1.wiscnet.net (140.189.8.2)  2.730 ms  2.603 ms  2.689 ms

- 7  space-needle-mke.csd.uwm.edu (216.56.1.194)  2.836 ms  2.718 ms  2.748 ms

- 8  miller.cs.uwm.edu (129.89.38.24)  2.754 ms *  2.796 ms

- Lecture 1: Basic Skills

- June 21-25, 2004

## Port numbers

- A port number indicates which program to talk to on a computer.
- Some port numbers are standard:
- HTTP (web): port 80
- SMTP (mail): port 25
- Ping: port 7
- Some port numbers are assigned dynamically when you run a server.

- Lecture 1: Basic Skills

- June 21-25, 2004

## Netstat

- Netstat can answer the question: is a program running on a port on the local computer.
- 	netstat --protocol=inet –l
- 	tcp  0 0 *:finger  *:* LISTEN
- -l meant “listening for connections”. Look for active connections:
- 	netstat --protocol=inet | grep ssh
- 	% netstat --protocol=inet | grep ssh
- 	tcp  0 0 chopin.cs.wisc.edu:ssh  ppp-67-38-160-108:20715 ESTABLISHED
- 	tcp  0 0 chopin.cs.wisc.edu:ssh  68.185.181.47:1176      ESTABLISHED
- 	...

- Lecture 1: Basic Skills

- June 21-25, 2004

## Telnet

- Telnet isn’t just for remote access to a computer
- Telnet can tell you if some remote services are running correctly.
- Is ssh running?
- Find ssh port number in /etc/services. It’s 22.
- telnet <host> 22. Example:
- telnet beak.cs.wisc.edu 22
- Trying 128.105.146.14...
- Connected to beak.cs.wisc.edu (128.105.146.14).
- Escape character is '^]'.
- SSH-1.99-OpenSSH_3.6.1p2
- ^]    (That is control-right bracket)
- telnet> quit

- Lecture 1: Basic Skills

- June 21-25, 2004

## Telnet & HTTP (Just for fun)

- % **telnet www.cs.wisc.edu 80**
- Trying 128.105.7.31...
- Connected to www.cs.wisc.edu (128.105.7.31).
- Escape character is '^]'.
**GET http://www.cs.wisc.edu/~roy/index.html**
- <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
- <html>
- <head>
- <title> Alain Roy</title>
- ... snip ...
- I typed the lines in red, and got back a web page.

- Lecture 1: Basic Skills

- June 21-25, 2004

## Why are we telling you this?

- We have a shameful secret:
- Maintaining a grid is hard work. Things break. You will get involved in debugging them.
- You can use all of these commands to figure out where a problem lies.

- Lecture 1: Basic Skills

- June 21-25, 2004

## Example

- Bad complaint: Waaah!!! My grid server-thingy is down
- Good complaint: The Globus gatekeeper isn’t running, even though the computer is up.
- Is the computer up? Use ping
- Is the computer accessible on the network? Use traceroute
- Can you telnet to the gatekeeper?

- Lecture 1: Basic Skills

- June 21-25, 2004

## Grid Without a Grid (GWAG)

- Now we’re going to learn how to do grid work without using grid technology.
- What is a grid? We’ll tell you this afternoon. For now:
- Run jobs on other computers
- Transfer data
- Discover information
- Do it all securely
- Could we have thought of a worse acronym? GWAG?

- Lecture 1: Basic Skills

- June 21-25, 2004

## GWAG: Security

- Explain OpenSSL
- Explain OpenSSH
- Explain HTTPS
- Note password authentication
- Describe ssh keygen stuff, to eliminate passwords. Note how this has to be set up on a per-computer basis.
- This needs to be fleshed out.

- Lecture 1: Basic Skills

- June 21-25, 2004

## GWAG: Job submission

- ssh can run jobs remotely:
- ssh roy@beak.cs.wisc.edu ls
- This expects a few things:
- Your program already exists on the remote computer
- A shell is invoked to set up your environment, so you can find ls.

- Lecture 1: Basic Skills

- June 21-25, 2004

## GWAG: File Transfer	

- How do you transfer data to be executed?
- scp executable roy@beak.cs.wisc.edu:.
- Note the final period at the end of that line
- How do you transfer the output back?
- scp roy@beak.cs.wisc.edu:output .
- Submitting your job takes a minimum of three commands. If you don’t set up your keys, you’ll type your password three times.

- Lecture 1: Basic Skills

- June 21-25, 2004

## GWAG: Issues

- Issues you will find:
- How do you run three jobs sequentially that require data transfer? You just run each command in a row.
- How do you run three jobs in parallel that require data transfer?
- Make each transfer/job/transfer a script. Run the scripts in parallel from the command line
- How do you deal with failures?
- What if scp or ssh returns an error?
- What if scp or ssh never returns? (It hangs)
- How do you keep track of everything?
- In an ad-hoc way?

- Lecture 1: Basic Skills

- June 21-25, 2004

## GWAG: Dealing With the Issues

- We will talk about grid solutions to these problems.
- Keep an eye out for:
- GSI for security
- GRAM or Condor-G for transfer of input/output files
- Condor-G for job reliability
- DAGMan for running sets of jobs

- Lecture 1: Basic Skills

- June 21-25, 2004