# Presentation: 061186

## The Benefits of Dark Fiber Between LBNL and OSF

**Mike Bennett**

## Contributors

**John Shalf – NERSC**
**Ted G. Sopher Jr. - LBLnet**

## Agenda

**Why Dark Fiber?**
**How LBNL and NERSC benefit**
**What are our choices?**
**Why National LightRail?**
**What are the costs?**
**What are the issues?**
**conclusion**

## Why Dark Fiber?

**Virtually unlimited bandwidth**
**This is achieved through use of DWDM equipment**
**But depends on the quality of the fiber**
**Point to point 10 gigE over dark fiber is simple and low cost compared to managed service**
**Combined with a partnership with vendors, an opportunity to understand how applications perform over a 10 GbE MAN link without a large initial capital outlay**
**Since CENIC is offering to help with the connection by combining our request with many others we get a low cost lease on the fiber**

## Why Dark Fiber? (cont.)

**Cost of dark fiber is at an all time low:**
**Fiber owners are hungry for business**
**ILECs/CLECs are all but gone**
**This leaves companies with lots of fiber and no one to lease it**
**Example: MFN**
**MFN is struggling, but CENIC is coming to their rescue by purchasing a large lease of fiber**
**We can benefit from this!**

## Why Dark Fiber? (cont.)

**Dark Fiber enables cutting-edge networking research at LBNL **
**Can be used as leverage to get funding from MICS for such research**
**We need a HPN between NERSC and LBNL:**
**"Current and future cutting-edge science is characterized by large data volumes. In some fields, like high-energy physics, huge volumes of data have to be processed and filtered to find the small percentage that actually tells a story. Data will reach petabytes in the next five years“ (NSF report)**

## How LBNL and NERSC benefit

**A relatively low cost way to understand how real applications perform over 10 gig Ethernet link**
**Provides the foundation for a network testbed that many folks could use for research**
**If you choose to connect to the National Lightrail, many more research opportunities become available**
**This could bring research dollars to the lab!**
**This positions NERSC/LBNL better to participate in the Ultrascale Computing Initiative**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## How LBNL/NERSC benefit (cont.)

**Consider how others have benefited from such a connection:**
**ORNL - Got its own 10-Gbps connection to the Internet2 research network in Nov. **
**ORNL has a 10 Gbs link, LBNL/NERSC have  1 Gbs links.  Who is in a better position to get research funding for Ultrascale Computing?**
**“****Access points to the 10-Gbps link at the University of Tennessee in Chattanooga also will spur growth there. A number of companies in the area grew directly out of technology transfer from Oak Ridge,” Walli said.**
**LBNL/NERSC could also provide technology transfer to spur growth**
**ANL/Teragrid - The $53 million Distributed Terascale Facility, also called the teragrid, will link four giant computer clusters over a 40-Gbps optical network.**
**How are we able to connect to this 40 Gbs network in our current position?**

## How LBNL/NERSC benefit (cont.)

**Also worth noting:**
**UCSD/UIC gets $13.5M for “optiputer”**
**They were able to get this grant because they had invested $4M in dark fiber infrastructure**

## What are our choices?

**1 pair of SMF-28 fiber from LBNL to OSF**
**cheaper**
**we have control of the network**
**we get access to network elements **
**because we have to provide them**
**Managed service with CENIC to gain access to the National Lightrail**
**More expensive**
**We don’t get access to the network elements**
**We don’t have to maintain DWDM equipment**
**We don’t get the performance numbers from the network equipment**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## Why National LightRail?

**It gets us on a 10 Gigabit backbone connection us to many research institutions**
**This could lead to new research funding opportunities**
**NTON appears to be going away**
**Perhaps already gone?**
**Trade-off is that we can’t get access to network stats on CENIC managed equipment**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## What are the costs?

**Leasing 1 pair of fiber from Berkeley to Oakland:**
**$27.5K/year for 5 year lease**
**$35K to install a lateral from the OSF to the splice case on Lakeside st.**
**Should allocate funds to cover a fiber pull from the UCB Recreation Facility to Evans Hall, say a maximum of $20K**
**Lease includes maintenance of fiber**
**total cost  < $200K over 5 years is < $40K/yr!**

## What are the costs? (cont.)

**10 Gigabit Ethernet point-to-point**
**2 ports of 10GBASE-LR**
**$140K**
**Dark Fiber**
**200K**
**Total cost: $340K**
**Annual Cost: $68K**
**Monthly Cost:$5.7K**
**OC-12 (622 Mbs) Monthly cost: $35K**
**You get 6% of the BW for 6 times the price!**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## What are the costs? (cont.)

**Managed service costs not yet available**
**Two types of Ethernet connectivity offered**
**1 Gigabit connection**
**10 Gigabit connection**
**CENIC still working out price structure**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## What are the issues?

**Is the fiber capable of handling DWDM and 10 GbE?**
**SMF-28 is Corning fiber**
**Low loss at 1310 nm and 1550 nm**
**For link distances of less than 75 KM, chromatic dispersion is not an issue**
**Optimized for 1310 and 1550 nm**
**DWDM operates near the 1550 nm region**
**Who is responsible for repair if the fiber gets damaged?**
**MFN per the lease agreement**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## What are the issues? (cont.)

**Can this be used in production in the future?**
**Yes, in fact we could have research and production wavelengths over the same fiber**
**The risk would be running a production network over a single pair of fibers.  Could be mitigated by microwave radio as a back-up, with the obvious disadvantage of degraded service while the fiber was being repaired**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## What are the issues? (cont.)

**Will this fiber be ready for higher BW technology, i.e. 100 GbE or OC-768?**
**Good question, don’t know but that’s why the lease is only 5 years.  By then we will know and will be able to make an informed decision about renewal**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## Conclusion

**We have an opportunity to acquire *****leased***** dark fiber at a very reasonable cost**
**We have an opportunity to acquire a high capacity path *****that can be leveraged to bring in research funding***
**We should not let this opportunity pass us by**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## References

_**[http://mics-03.er.doe.gov/EarthSim/AGnetwork.html](http://mics-03.er.doe.gov/EarthSim/AGnetwork.html)**_
_**[http://www.npaci.edu/online/v6.20/optiputer.html](http://www.npaci.edu/online/v6.20/optiputer.html)**_

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**

## Additional Information

**About the “Optiputer”**
**The researchers in southern California and Chicago will focus on new network-control and traffic-engineering techniques to optimize data transmission; new middleware to bandwidth-match distributed resources; and new collaboration and visualization to enable real-time interaction with high-definition imagery.**
**"Think of the OptIPuter as a giant graphics card, connected to a giant disk system, *****via a system bus that happens to be an extremely high-speed optical network*****," said DeFanti, a distinguished professor of computer science at UIC and co-director of the university's Electronic Visualization Laboratory.**

**Delete sample document****icons and replace with working document icons as follows:**
**From Insert Menu, select Object...**
**Click “Create from File”**
**Locate File name in “File” box**
**Make sure “Display as Icon” is checked**
**Click OK**
**Select icon**
**From Slide Show Menu, Select “Action Settings”**
**Click “Object Action” and select “Edit”**
**Click OK**