WhitePapers Worth Reading
Tags: softwareengineering, whitepapers, google, cloudcomputing
Published: 2023-11-26 11:18:01

Here are some awesome white papers from Google, Amazon, Meta, and Apache. These white papers are worth your time if you are a software enthusiast or a tech geek.

## Google

1. [Google File System](https://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf)
2. [Map Reduce Big Data Algorithm](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/16cb30b4b92fd4989b8619a61752a2387c6dd474.pdf)
3. [BigTable NoSQL Document Store](https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf)
4. [Colossus Next Gen File Store](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system)
5. [Megastore Large Object Store](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/36971.pdf)
6. [Monarch Time Series DB](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/d84ab6c93881af998de877d0070a706de7bec6d8.pdf)
7. [Chubby Distributed Lock Management](https://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf)
8. [Spanner Distributed Database](https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf)
9. [Spanner - CAP theorem considerations](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45855.pdf)
10. [Dapper Tracing System](https://static.googleusercontent.com/media/research.google.com/en//archive/papers/dapper-2010-1.pdf)
11. [Borg Cluster Management](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43438.pdf)
12. [Zanzibar Authentication System](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/10683a8987dbf0c6d4edcafb9b4f05cc9de5974a.pdf)
13. [Pregel Graph Processing](https://15799.courses.cs.cmu.edu/fall2013/static/papers/p135-malewicz.pdf)
14. [Napa - Data Warehousing](https://www.vldb.org/pvldb/vol14/p2986-sankaranarayanan.pdf)
15. [Napa - Partitioning Algorithm](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/af91a9db5437586192a1ca87685e5b7d937a4e1a.pdf)
16. [TensorFlow - Machine Learning at Scale](https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf)
17. [F1 - Fast Analytics](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/19fab8e81c9583f3ce67a8a137d1655970eb3efe.pdf)
18. [HALP - YouTube Content Delivery Network](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/c9ebaf640152028a6cdedb684577a7c9e52b6f10.pdf)
19. [Mesa - Data Warehousing](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/42851.pdf)
20. [Firestore](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/d647cb73166040a82b7e5569574451be517f5c59.pdf)

## Amazon
1. [Amazon Aurora DB Architecture](https://assets.amazon.science/dc/2b/4ef2b89649f9a393d37d3e042f4e/amazon-aurora-design-considerations-for-high-throughput-cloud-native-relational-databases.pdf)
2. [Dynamo DB NoSQL Database](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf)
	
## Misc
1. [Foundation DB NewSQL database](https://www.foundationdb.org/files/fdb-paper.pdf)
2. [Monolith Embedding in real time](https://arxiv.org/pdf/2209.07663.pdf)
3. [Scalability at what COST](https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf)
	
## Meta 

1. [Gorilla Time series DB](https://www.vldb.org/pvldb/vol8/p1816-teller.pdf)
2. [Cassandra NoSQL DB](https://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf)
3. [FlexiRaft Distributed Concensus Tradeoffs](https://www.cidrdb.org/cidr2023/papers/p83-yadav.pdf)
4. [Memcache In-memory Cache](https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf)
5. [Millisampler Network Sampling](https://dl.acm.org/doi/pdf/10.1145/3517745.3561430)
6. [TAO Graph Database](https://www.usenix.org/system/files/conference/atc13/atc13-bronson.pdf)
7. [MineSweeper - Root Cause Analysis](https://arxiv.org/pdf/2010.09974.pdf)
8. [Prophet - Forecasting at Scale](https://peerj.com/preprints/3190.pdf)
9. [ShardManager](https://research.facebook.com/file/245575980870853/Shard-Manager-A-Generic-Shard-Management-Framework-for-Geo-distributed-Applications.pdf)
10. [Hive - Map Reduce Jobs](https://research.facebook.com/file/2435031296630175/hive-a-warehousing-solution-over-a-map-reduce-framework.pdf)
11. [Thrift - Definition Language](https://thrift.apache.org/static/files/thrift-20070401.pdf)
12. [Twine - Unified Cluster Management System](https://research.facebook.com/file/214174457337055/Twine-A-Unified-Cluster-Management-System-for-Shared-Infrastructure-v2.pdf)
13. [ServiceRouter - Hyperscale and Servicemesh from Meta](https://www.usenix.org/system/files/osdi23-saokar.pdf)
	
## Apache
1. [Hadoop File System](https://storageconference.us/2010/Papers/MSST/Shvachko.pdf)
2. [Kafka Event Bus](https://notes.stephenholiday.com/Kafka.pdf)
3. [Flink](https://asterios.katsifodimos.com/assets/publications/flink-deb.pdf)

---------------------

Whitepapers have the benefit of discussing practical implementation details and the related tradeoffs to engineering products. 

The best engineers worldwide have written them after years of work, so don't miss any of them!