# Presentation: 219032

## The State of the MCSSystems Group and Computing Environment1998

- systems@mcs.anl.gov
- http://www.mcs.anl.gov/systems/
- 630-252-6813

- Rémy Evard

## Talk Outline

- 1. Strategic Issues for the Pre-Retreat
- 2. Systems Group Directions
- 3. MCS Computing Environment Update & Plans

**You**
**Are**
**Here**

## Systems Group Members

**Intro**

- Rémy Evard - Mgmt

- John-Paul Navarro - SP

- Sandra Bittner - Unix

- Janet Sayre - User issues

- Karen Toonen - Docs

- Joe Bester - DSL

- Gene Rackow - Servers

- Michail Gomberg - NT

- Craig Stacey - Desktops

- Dan Nurmi - Student

- Bill Nickless - FL

- Chris Wesolowski

## MCS Computing Environment

**Intro**

- Network

- Home Machines

- Unix

- NT

- CCST

- Visualization

- Special Projects

## MCS Computing Environment

**Intro**

- Net: 5+

- Unix: 125 hosts

- Windows NT: 70
- Laptops: 40+

- Quad: 100+ hosts
- Tri: 70 hosts

- 1 CAVE, 2 Idesks
- Yukon, Alaska

- CP-6
- Alpha, Linux, ...

## Overview of the field.
Major issues in the national community.
Where ANL/MCS effort fits in:
Key problems.
Scale of effort.
Competitive analysis.
Relationship to other ANL/MCS efforts.
Future outlook for the field and ANL.
Needs from infrastructure and supporting program.

- Major issues in the national community.
- Where ANL/MCS effort fits in:
- Key problems.
- Scale of effort.
- Competitive analysis.
- Relationship to other ANL/MCS efforts.
- Future outlook for the field and ANL.
- Needs from infrastructure and supporting program.

**Questions for the Pre-Retreat**

- Tech

- Career

- Research

**Pre-Retreat**

## Overview of Systems Administration

- So, we:
- 	Design, build, and maintain the environment.
- 	Resolve problems.
- 	Educate.
- We must:
- 	Understand organizational and individual goals.
- 	Understand computing.

***Systems Administrators enable people to exploit computers***
***at a level which gains leverage for the entire organization. ***
***     *****-- Tina Darmohray, SAGE Publications**

**Pre-Retreat**

## Issues in the Sysadmin Community

- Generally recognized as an actual career                  in the early 90s.
- Far more jobs than people:
- 1996:  40,000 jobs - 20,000 administrators
- Fierce competition between companies.
- Growth of consulting companies.
- Administrators are specializing.

**Pre-Retreat**
**|**
**Career**

## Issues in the Sysadmin Community

- Growing complexity of computing.
- Tension between centralized resources and powerful desktops.
- General trend towards management via the web and databases.
- Decrease in the availability and use of free tools.

**Pre-Retreat**
**|**
**Tech**

## MCS effort: scale

- FTEs on Systems  (6.5 - 8.5 or so)
- 6 administrators: Unix, NT, network, SP, ...
- .5 documentation
- 1 manager
- 2 liaisons: DSL, FL
-                         Which is:
- Large    for a CS department our size.
- Normal    for a technology company our size.
- Small 	      for a company of our computing diversity.
- Very small     for a supercomputing facility.

**Pre-Retreat**

## MCS effort: sources of support

- People:
- 	Across all MICS program money.
- 	LDRD.
-    (No specific MICS core funding.)
- Equipment:
- 	DOE capital equipment.
- 	DOE HECRC (quad).
- 	2.5% of all incoming funding.

**Pre-Retreat**

## MCS effort: competition

- Industry is our main competition for good administrators.
- Otherwise, systems administration is non-competitive because it’s not externally funded.

**Pre-Retreat**

## MCS effort: comparisons & ranking

- ANL has long been well-known in the Unix administration community.
- MCS is better managed than 90% of the sites out there.
- Comparison to other ANL administrators is somewhat difficult.  We find that we have knowledge-level peers in APS and ECT networking.
- At least one grant with UofC has leveraged MCS infrastructure.

**Pre-Retreat**

## Relation to other ANL/MCS efforts

- MCS:
- 	The Systems Group’s primary goal is to support the MCS research efforts.
- ANL:
- 	We interact with ANL network administrators for coherent lab-wide networking.
- 	There has been no real effort to work with other ANL administrators.

**Pre-Retreat**

## Relation to other ANL/MCS efforts

- Worrisome issue:
- 	Systems members today understand less about what the division is doing than we did five years ago.

**Pre-Retreat**

## Future for the field and ANL

- If computing trends continue, administrators will remain in high demand.
- If MCS continues to be involved in many facets of computing and many projects, the MCS environment will continue to get more complex.

**Pre-Retreat**

## Future for the field and ANL

- Faced with growing requirements, we will constantly evaluate these choices:
-  - Improve the efficiency of the group.
-  - Hire additional people.
-  - Lower quality and/or quantity of services.

**Pre-Retreat**

## Infrastructure Needs

- A nap.

**Pre-Retreat**

## Support Needed to Correctly Evolve

- Support from MICS/DOE in the core budget.
- Continued support via other funding.
- Feedback to Systems on what we can do to help bring in that funding.
- Note: Whatever may change in the division, continue the model of a central systems group.   (With liaisons.)

**Pre-Retreat**

## Overview of the field.
Major issues in the national community.
Where ANL/MCS effort fits in:
Key problems.
Scale of effort.
Competitive analysis.
Relationship to other ANL/MCS efforts.
Future outlook for the field and ANL.
Needs from infrastructure and supporting program.

- Major issues in the national community.
- Where ANL/MCS effort fits in:
- Key problems.
- Scale of effort.
- Competitive analysis.
- Relationship to other ANL/MCS efforts.
- Future outlook for the field and ANL.
- Needs from infrastructure and supporting program.

**Pre-Retreat: Systems Research**

- Tech

- Career

- _Research_

**Pre-Retreat**

## Systems Research

- Not a conventional research area.
- Two groupings of research:
- 	1. Advanced Deployment:
- performance, feasibility, early testing
- 	2. Systems Admin Research:
- methodology, tools

**Pre-Retreat**
**|**
**Research**

## Advanced Deployment

- “How well does this work?”
- Constantly ongoing in MCS:
- Networking: ATM, VLANs, DHCP, ...
- CCST: HPSS, DMF, HPGN, Betas, scheduling, ...
- NT: SMS, Software install models, profiles, Collaboration tools, funky domains, ...
- UNIX: Linux, Database integration, ...

**Pre-Retreat**
**|**
**Research**

## Systems Administration Research

- Administration:
- The use of tools and methods to design, build, and maintain an environment.
- Development:
- The creation of tools and methods for administration.
- Research:
- The exploration of new models for tools and methods.

**Pre-Retreat**
**|**
**Research**

## Sysadmin Research - Major Issues

- Very uncommon.
- Growing need for administrators points out the need to understand administration better.
- Basic questions:
- How to manage very large numbers of machines with very few people.
- How to decrease need for user expertise.
- How to evaluate the quality of administration.

**Pre-Retreat**
**|**
**Research**

## Sysadmin Research - ANL Effort

- A paper at LISA 97 presenting an in-depth study of configuration files, proposing a new model for configuring machines.
- A partial implementation of that model.
- Scale: 0.02 FTE
- Competition is starting to form.

**Pre-Retreat**
**|**
**Research**

## Sysadmin Research Relations

- Becoming increasingly important for the Globus project.
- Could be a key factor in the PACI project.
- Petaflops is going to be difficult without petadmins.

**Pre-Retreat**
**|**
**Research**

## Sysadmin Research - Directions

- Small-scale prototype effort.
- Seed the funding community.
- Full-scale support in the next 3-5 years.
- 					   ?

**Pre-Retreat**
**|**
**Research**

## The Systems Group

- A dash of history, for context.
- Then an overview of where we are, where we’re going, ...
-    ...and how that affects you.

## Divisional Eras, Support-wise

- ACRF Era
- Medium Environment
- Simple  Requirements
- Little Funding
- 3 Support Staff
- Progress + Fires
- Keeping Up
- Clean, Tight Code

- SP Era
- Larger Environment
- Serious Requirements
- Little Funding
- 3 Support Staff
- Fires
- Barely Survived
- Spaghetti Code

- Quad Era
- Larger Environment
- Complex Requirements
- Better Funding
- 7.5 Support Staff
- Fires + Progress
- Catching Up
- New code is clean.
- Old code is scary.

**Systems**
**Group**

**Notes:**
Lack of documentaiton.
Interdependencies.
Inconsistent filesystems.
Insecurities.

## Impact of the Changing Times

- 1.  Systems is on a very slow course to fix the environment by:
- 		Installing new admin mechanisms.
- 		Replacing old ones.
-    The old mess affects nearly everything we do.
- 2. Systems is now beginning to deal with growing larger.

**Systems**
**Group**

## Systems Group Goals, 1998

- 1. Keep things working.
- 2. Make more information available.
- 3. Understand and meet new requirements.
- 4. Maintain the current quality of the group.
- 5. Continue to replace old system with new.

**Systems**
**Group**

## The Computing Environment

- Terminology
- Current State
- Plans

## Levels of Support

**Computing**
**Environment**

## ... which means:

- Environment

- Responsibility

- Relationship

- MCS Environment

- Systems

- Systems

- Local, and
- MCS Environment

- The local admin,
-  then Systems

- Local admin
- coordinates
- with Systems

- On your own

- On your own

- Cable and IP

- Green

- Yellow

- Red

**Computing**
**Environment**

## Unix Events, 1997

**/software tree created. ****	****	****Jan - Dec**
**Disks moved to /nfs mounts. ****	****March**
_**IRIX 6.x**_** supported. ****	****	****	****March - Sept**
**ADSM HSM available. ****	****	****June**
**Servers crash like mad. ****	****	****July - October**
**Cfg system created. ****	****	****	****July**
**Hostbase system installed.****	****	****July**
**Backup disaster.****	****	****	****August**
**ADSM used for backups.****	****	****September**
**.software v2.****	****	****	****	****September**
_**Solaris 2.5**_** supported. ****	****	****October**
**FreeBSD didn’t get upgraded.**
**Linux didn’t happen.**

**Computing**
**Environment**

## Unix Server Plans, 1998

- Critical tools:
- /software tools.
- New .software complete.
- Machine configuration tools.
- Account system.
- New platforms:
- Replace FreeBSD with Linux.
- Make Solaris == Irix == Linux == SunOS.
- Compute servers.
- Fix servers:
- Servers re-architected for security, reliability.
- Improve performance.
- Revise the account system.

**Computing**
**Environment**

## Desktop Events, 1997

- Installed ~36 PCs.
- Installed ~5 sparcs.
- Got rid of the last black & white machine.

**Computing**
**Environment**

## Desktop Plans, 1998

- SunOS will live forever, but grow no more.
- FreeBSD will be replaced with Linux.
- Start testing the “PC as desktop” strategy...

**Computing**
**Environment**

## PC Desktop Feasibility Testing

- PCs on desktops.
- Cheaper, equally powerful.
- Flexibility: NT or Linux.
- Added capability: Windows Apps, Multimedia.
- High-powered compute servers in machine room.
- Pooling resources results in higher-powered computers.
- Drastically improve compile time via local disk and fast network.
- At least one of each major platform type.

**Computing**
**Environment**

## Green Platforms, 1998

- SunOS 4.1.3+			no new machines
- Solaris 2.5+
- IRIX 6.x
- AIX 4.1+
- FreeBSD 2.1			phasing it out
- Linux				not yet
- Windows NT 4.0 		on Desktops
- Windows 95 			on Laptops

**Computing**
**Environment**

## NT Events, 1997

- CP-6 installed.				January
- Developed NT build method, v1.		May
- Converted all PCs to NT.			June
- Office97 deployed.				June
- Installed new server	.			June
- Created \\dfs	.				September
- Started using SMS.				December
- Worked on software inventory.		February - December

**Computing**
**Environment**

## NT Plans, 1998

- Create an actual server environment.
- Organize \\dfs .
- Sort out software licensing.
- Create standard NT build and methods.
- Install software remotely.
- Try our hardest to integrate with Unix.
- Start a “collaboration tools” test project.

**Computing**
**Environment**

## SP Events, 1997

- Old SP reconfigured, downsized.		January - February
- Quad installed.				January
- Quad has big problems.			January - April
- HSM comes up.				March
- Fought the scheduler.			March - June
- Fixed the scheduler.				July
- Usage reporting.				August - December
- Quad on ATM.  August
- Quad goes into SC97 mode.		October - November
- Old SP off.					October
- Rebirth of SP as Tri.				October
- GAO Audits.					November - December

**Computing**
**Environment**

## SP Plans, 1998

- Upgrade plans are unclear at this time, but we want to:
- Double the memory in all nodes.
- Go to 140 nodes.
- Test the High Performance Gateway Router.
- Install HPSS, GPFS, latest PSSP, etc.
- Production Mode around March.
- Users Group Meeting.

**Computing**
**Environment**

## Visualization Events, 1997

- DSL Idesk installed			June
- Yukon installed  			August
- CAVE on Yukon			October

**Computing**
**Environment**

## Visualization Plans, 1998

- CAVE going to 4 walls.
- Yukon driving the Idesks.
- New Idesk for Numerical Group.
- New Portadesk.
- Closer integration with Quad.

**Computing**
**Environment**

## Network Events, 1997

- A-wing gets real cabling.		January - July
- Routing moved off servers.		March - November
- Netadmins established.		June
- .3 net crashed a lot.			July - November
- Installed serious router.		August
- Installed ATM switches.		August
- Fixed .3 net.				December
- Overall:
- 	Moved seriously to ATM and switching during the year.

**Computing**
**Environment**

## Network Plans, 1998

- Wiring infrastructure:
- Complete wiring of downstairs wings.
- Slow migration of upstairs wiring.
- Services:
- Mbone.
- DHCP, ...
- Strategy:
- Continual move to flat, switched network.
- Improve manageability and monitoring.

**Computing**
**Environment**

## Database Plans, 1998

- Install production Oracle database.
- Make clients available to groups who are interested:
- FL
- Systems
- Allocations on Quad.
- Accounting on Yukon.
- Account management.
- Generating software web pages.
- If you’re interested, talk to JP.

**Computing**
**Environment**

## User Support Events, 1997

- 4 MCS Tech Updates.
- BMR open house.				June
- Whiteboard.					July
- Helpdesk.					August
- Announcement system.			September
- Started doing req buzzes.			September
- Cluesheets updated, then abandoned.
- Started computing web.			November

**Computing**
**Environment**

## User Support Plans, 1998

- As much documentation on the web as possible:
- 	http://www.mcs.anl.gov/computing/
- Use the unix-announce, win-announce and quad-announce mailing lists.
- Upcoming SP Users Group meeting, NT Users Group meeting.
- Reference manuals available for checkout at help desk.

**Computing**
**Environment**

## Help Desk Resources

- CD-ROM drives.
- Older laptops.
- Projector.
- Very-Short-Term Guest accounts.
- Ricochet modems.
- Janet’s wireless card.
- Friendly people.

**Computing**
**Environment**

## Computing Resources

- Quad
- 8-way SMP nodes:  oct01, oct02
- Alaska
- Yukon
- Tri
- Solaris machines: fire, caffeine, dew
- CP-6: the NT cluster
- NT machine in Graphics Lab with scanner &   CD writer

**Computing**
**Environment**

## Web Server Restructuring

- We’re putting in an entirely new web structure:
- Division web pages “look and feel” needs updating.
- Current web server is insecure, overloaded, and confusing.
- New requirements:  FrontPage, Virtual hosting, Intranets.

**Computing**
**Environment**

## New Division Web Look

- Early beta version:
- 	http://www.mcs.anl.gov/division/
- Will be official before the review committee arrival in March.

**Computing**
**Environment**

## New Web Server Infrastructure

- www.mcs.anl.gov

- www-fp

- www-unix

- Redirector only.
- Apache on Solaris.

- ...

- FrontPage access.
- IIS on NT.

- Unix file system access.
- Apache on Solaris.

- Add more
- as necessary.

**Computing**
**Environment**

## New Web URL styles

- http://www.mcs.anl.gov/*project*
- http://www.mcs.anl.gov/~*user*
- Why?
- 	Top-level naming is shorter and more prestigious.
- 	More organization is difficult and a bit arbitrary.

**Computing**
**Environment**

## Web Transition Plans

- New servers up.                               (Done)
- Insert them into the system.
- Detailed instructions on how to migrate.
- Move over a few early testers.
- Instruct everyone on how to move files over.
- Decommission the old web server.
- Timeframe:  Now - April

**Computing**
**Environment**

## Web Server - Longer Term Plans

- Virtual hosting.
- Pointing to servers on individual machines.
- Statistics gathering.
- Search capability.
- Intranet.

**Computing**
**Environment**

## Conclusion

**The Mission:**
***Systems Administrators enable people to exploit computers***
***at a level which gains leverage for the entire organization.****** ***
- There’s a lot going on.  Hopefully some part of it is good for you.
- Information from you helps us prioritize.
- Please come talk to me about any issues.

## The State of the MCSSystems Group and Computing Environment1998

- systems@mcs.anl.gov
- http://www.mcs.anl.gov/systems/
- 630-252-6813

- Rémy Evard