# Presentation: 174863

## Internet Technical Issues - Building High Performance Top Level Web Servers

- 1

*Internet Technical Issues - Building High Performance Top Level Web Servers*

- This talk will focus on the hardware and software designs necessary to build high performance, high availability, scalable Web servers with dynamically updated content at relatively low cost and over relatively low capacity network links.
**Jim Dumoulin -  NASA Payload Operations**
**Internet Systems Lab -  Kennedy Space Center FL**

**Notes:**
1
1
1
08/13/97
1

## Top Level Government Web Sites

- 2

- 08/13/97

- 2

*Top Level Government Web Sites*

- Typical Goals
- High Security
- High Availability
- High Capacity
- Up-to-Date Content

- Typical  Constraints
- No Direct Budget
- No Manpower
- Little Training
- Low available Network capacity

**Notes:**
2
2
2
08/13/97
2

## Design Considerations:

- 3

- 08/13/97

- 3

*Design Considerations:*

- Question:
- How do you build a Secure, High Availability, High Capacity Web site with timely content under typical government constraints**.**
- No magic Answer, but....
- Think of what can go wrong and how long you would be down if a bad event happens
- Think of your Web site as a cluster of machines instead of a single machine, even if it doesn’t start out that way.

**Notes:**
3
3
3
08/13/97
3

## WWW Security - what can go wrong

- 4

- 08/13/97

- 4

*WWW** **Security - **what can go wrong*

**Theft of Information**
- Someone can read info they shouldn’t.  Mail back password file, system information etc.
**Theft of resources**
- Server storage space or processing cycles.  Storage and distribution of illegal items such as porn or copyrighted software or denial of service  by filling up hard drive space.
**Changes to Information (Graphics or text)**
- (IE: “Dept of Justice” became “Department of Injustice”) Log files can get deleted to cover tracks, hosts or  users added.
**Complete loss of subnet security**
- Programs can be installed, spy programs on local net, telnet programs installed at higher port and bypass firewall

**Notes:**
4
4
4
08/13/97
4

## Considerations for Security

- 5

- 08/13/97

- 5

*Considerations for Security*

- Don’t allow login accounts on your server
- Load only services you absolutely need - “tcpwrapper”
- Stay on top of vendor patches
- Never run server child processes at “root”
- Use strong passwords/C2 Security/Shadow etc
- Be paranoid when running any server “CGI” scripts
- Log system logs to a separate inside firewall machine
- Run security checks & checksums from separate machine

- Security issues can be simplified if you can off load processes to one or more dedicated trusted machines inside your firewall.   (ie: Log Host, Forms Host, User Host, etc)

**Notes:**
5
5
5
08/13/97
5

## Availability - what can go wrong

- 6

- 08/13/97

- 6

*Availability - what can go wrong*

- Scheduled  Downtime
- Backups, System Upgrades, Maintenance
- Planned Network /Facility Outages
- Unscheduled Downtime
- Hardware failure, Software crashes,
- Inability to handle peak loads, operator error
- recovery due to poor security
- Perceived Downtime
- Problems with networks upstream, DNS failures,
- Cached copies of bad pages, congestion

**Notes:**
6
6
6
08/13/97
6

## Availability - What to do right

- 7

- 08/13/97

- 7

*Availability - What to do right*

- Use alias names (not the real name) of a web server host so you can move services to a backup host if you have a failure
- Run a hard (redundant copy) or soft (network mounted) mirror site (or both).  Better yet, have your prime visible server really be a mirror of the real files on an internal server.
- Have an internal development machine that can be used to load new software and test concepts before going operational.

**Notes:**
7
7
7
08/13/97
7

## Availability - What to do right

- 8

*Availability** - **What to do right*

- Use fault tolerant systems (RAID, Redundant power, dual network cards and network paths, UPS etc)
- Automatically monitor important machines and facility systems and tie them into your system managers alphanumeric beeper.
- If you need to recover after a security intrusion,  put new time stamps on all affected pages to force remote cached copies to update.
- Design your system backup policy so it can run without shutting down or disabling your server.

## Capacity - what can go wrong

- 9

- 08/13/97

*Capacity** - what can go wrong*

- Congested outside links can backup and slow down your internal Intranet.
- System resources can get exceeded (memory, disks, processes, network etc)
- Huge spikes due to planned or unplanned “events”.

**Notes:**
9
9
8
08/13/97
8

## KSC’s server on Launch Day

- 10

*KSC’s server on Launch Day*

- Saturates E-1 and Fractional T-3 Circuits into KSC
- Over 2 Million hits within a few hours around launch time.
- CPU capacity not the problem but network congestion requires constant tweaking to handle load.

- 08/13/97

- Photo courtesy  Silicon Graphics Inc

## Capacity - What to do right

- 11

- 08/13/97

*Capacity - What to do right*

- Line up mirror site agreements with other agencies in the event something goes really right (or really wrong) with your servers
- Have a backup server you can place online when necessary or load level across multiple servers.
- Get to know your local DNS people so quick changes can be made if you need to add more horsepower.
- If you can afford to do so, run redundant network connections to separate Internet providers
- Keep your top level Web server free to serve just web pages.  Run heavy computation and forms/database programs on separate hosts.

**Notes:**
11
11
9
08/13/97
9

## Content - what can go wrong 

- 12

- 08/13/97

*Content - what can go wrong *

- Information becomes outdated
- New Web developers may produce pages that do not conform to standards
- Network resources move and links break
- Style considerations require entire site to change look and feel
- New standards become available that you decide to use across your entire site.

**Notes:**
12
12
10
08/13/97
10

## Content - What to do right

- 13

- 08/13/97

*Content - What to do right*

- Stick to standards and good style practices
- Automate creation of pages wherever possible.
- Periodically run your pages thru an HTML validation Tool
- Crawl your own site and inform Web authors of broken links on their pages

**Notes:**
13
13
11
08/13/97
11

## NASA/KSC’s Server Design

- 14

*NASA/KSC’s Server Design*

- CISCO Router /
- External
- Firewall

- TITAN04
- (WWW)

- To MCI

- To NSI

- TITAN05
- (Soft Mirror)

- Login  Server
- (User Accounts
- BackEnd Computation)

- Security Server
- (Log Files, Tripwire,
- DNS Resolver,  Account
- Backup)

- Hard Mirror /
- System Test Server
- 6 hours behind prime

- Forms /CGI Server

- Internal Firewall Isolation Network

- Data Disk RAID 5
- System Disk RAID 1

- Data Disk RAID 5
- System Disk RAID 1

- Internal Network

- External  Network

- Pager
- Modem

- (Future)

## KSC’s WWW Toplevel Server

- 15

- 08/13/97

*KSC’s WWW Toplevel Server*

- Multiple DEC Alpha 2100/RM Servers
- Multiple DEC Alpha Processors
- 640Mbytes of RAM, 24Gbytes Disk (RAID 5 and RAID 1) on main Public server  - 160Gbytes total on cluster
- Redundant hot-swappable power supplies
- Multiple Networks (1FDDI, 3 Ethernets)
- Workload spread across 13 other systems
- Digital OSF/1 Unix, Apache HTTPD

**Notes:**
15
15
12
08/13/97
12

## WWW.KSC.NASA.GOV

- 16

- 08/13/97

*WWW.KSC.NASA.GOV*

- Cluster of 4 multi-processor and 2 uniprocessor DEC Alpha Servers on a 100Mb/s FDDI ring.  Current Disk Capacity 160Gbytes

**Notes:**
16
16
14
08/13/97
14

## Internet System Hardware

- 17

- 08/13/97

*Internet System Hardware*

- TITAN01
- VMS 7.1
- 128MB RAM
- (Users)

- 16G Raid 5

- TITAN03
- OSF/1 4.2
- 128MB RAM
- (News/Lists)

- 22G Raid 5

- TITAN02
- OSF/1 4.2
- 512MB RAM
- (Users/FTP)

- 9-Track Tape

- Exabyte 120
- 4 Drive Tape
- Jukebox System
- 115 8mmTapes
- 1 Robot

- ZENO OSF
- WWW Hard Mirror

- FDDI Concentrator

- CISCO 7507

- Internal Log and Security Server
- 512MB/RAM

- 18G Raid 5

- TITAN04
- OSF/1 4.2
- 640MB RAM
- (WWW)

- 24G Raid 5

- 0

- 1

- 2

- 3

- TITAN05
- OSF/1 4.2
- 256MB RAM
- (Soft Mirror)

**Notes:**
17
17
15
08/13/97
15

## WWW Page creation

- 18

*WWW Page creation*

- 70,000+ HTML pages on Server - most produced and hypertexed automatically thru a series of PERL scripts
- Interfaces to external systems for dynamic content thru Common Gateway Interface (CGI) Scripts
- Daily content and automatic backups done thru Unix Crontabs

## KSC’s Automatic Hypertexing

- 19

- 19

*KSC’s Automatic Hypertexing*

- Treat’s HTML as a binary Language
- Keep original material in text, template  or marked up text files
- Run all text thru specialized text to hypertext Perl Programs
- Programs search for Regular expressions in context and replaces with Hypertexed URL’s from dictionary

- 08/13/97

## Summary

- 20

- 08/13/97

*Summary*

- 1.  Think of your Web server as a cluster of machines
- even if it doesn't start out that way
- 2.  Automate your content whenever possible.
- treat HTML as a binary language written by programs not people
- 3.  Stick to standards.
- Don't always jump into the latest vendor extensions
- 4.  Design for High Availability and Upgradability.
- RAID, HotSwap Disks, Hard & Soft Mirrors, UPS, etc
- 5.  Automate and track your access statistics.
- Know your average load. Have plan for handling spikes.
- 6.  Be paranoid about system security.

**Notes:**
20
18
16
08/13/97
16

## Where to get more information

- 21

- 08/13/97

- 1

*Where to get more information*

- This Presentation
- “http://www.ksc.nasa.gov/docs/ppt/webtech/”
- Example Web Site using Techniques
- “http://www.ksc.nasa.gov/shuttle/countdown/”
- Library of Congress HTML Page
- “http://lcweb.loc.gov/global/internet/html.html”
- WWW Security Links
- “http://hoohoo.ncsa.uiuc.edu/cgi/security.html”
- “http://www-genome.wi.mit.edu/WWW/faqs/www-security-faq.html”

**Notes:**
21
19
17
08/13/97
17