Source: https://patents.com/us-10394789.html
Timestamp: 2019-10-20 21:17:28
Document Index: 108602338

Matched Legal Cases: ['art 1', 'art 2', 'art 3', 'art 4', 'art 1', 'art 1', 'art 2', 'art 2', 'art 3', 'art 3', 'Application No. 16739357']

US Patent # 1,039,4789. Techniques and systems for scalable request handling in data processing systems - Patents.com
United States Patent 10,394,789
Animesh , et al. August 27, 2019
Techniques and systems for scalable request handling in data processing systems
Animesh; Rishabh (Seattle, WA), Doddameti; Sandesh (Seattle, WA)
14/961,749
Current International Class: G06F 16/00 (20190101); G06F 16/22 (20190101); H04L 29/08 (20060101); G06F 16/27 (20190101)
5729671 March 1998 Peterson et al.
6249836 June 2001 Downs et al.
6513099 January 2003 Smith et al.
6779150 August 2004 Walton et al.
6862362 March 2005 Gangadhar
6922700 July 2005 Aggarwal et al.
7117294 October 2006 Mi et al.
7142150 November 2006 Thackray
7380129 May 2008 Keohane et al.
7490013 February 2009 Wells
7783600 August 2010 Spertus et al.
8370579 February 2013 Strumpen
8386841 February 2013 Renade
8413187 April 2013 Del Sesto et al.
8479078 July 2013 Resch et al.
8504535 August 2013 He et al.
8566525 October 2013 Ash et al.
8612219 December 2013 Tsuchinaga et al.
8706980 April 2014 Dhuse et al.
8769049 July 2014 Murphy et al.
8788855 July 2014 Cong et al.
8806296 August 2014 Lazier
8850288 September 2014 Lazier et al.
8868825 October 2014 Hayes et al.
8869001 October 2014 Lazier
8935221 January 2015 Lazier et al.
8935761 January 2015 Gladwin et al.
8938591 January 2015 Mark et al.
8959067 February 2015 Patiejunas et al.
8984363 March 2015 Juels et al.
8984384 March 2015 Juels et al.
9002805 April 2015 Barber et al.
9003144 April 2015 Hayes et al.
9009491 April 2015 Resch
9021297 April 2015 Hayes et al.
9047214 June 2015 Northcott
9052942 June 2015 Barber et al.
9092441 July 2015 Patiejunas et al.
9110797 August 2015 Lazier
9165002 October 2015 Lazier
9213485 December 2015 Hayes et al.
9213709 December 2015 Patiejunas et al.
9218244 December 2015 Hayes et al.
9223789 December 2015 Seigle et al.
9225675 December 2015 Patiejunas et al.
9250811 February 2016 Patiejunas
9251097 February 2016 Kumar et al.
9256467 February 2016 Singh et al.
9256761 February 2016 Sahu et al.
9274956 March 2016 Salyers et al.
9281845 March 2016 Lazier
9298760 March 2016 Li et al.
9354683 May 2016 Patiejunas et al.
9378084 June 2016 Calder et al.
9384147 July 2016 Morshed et al.
9405333 August 2016 Pine
9448614 September 2016 Slik
9459959 October 2016 Franklin et al.
9495249 November 2016 Franklin et al.
9495255 November 2016 Davis et al.
9559889 January 2017 Vincent et al.
9563681 February 2017 Patiejunas et al.
9672110 June 2017 Patel
9753669 September 2017 Ben-Shaul et al.
9785495 October 2017 Lazier et al.
9792179 October 2017 Lazier
9825625 November 2017 Thalheim
9825652 November 2017 Lazier
9838041 December 2017 Lazier
9838042 December 2017 Lazier
9853662 December 2017 Lazier et al.
9866242 January 2018 Lazier
9870211 January 2018 Risbood et al.
9904589 February 2018 Donlan et al.
9923966 March 2018 Franklin et al.
9940244 April 2018 Quimbey et al.
9998539 June 2018 Brock et al.
2002/0152104 October 2002 Ojha
2003/0131198 July 2003 Wolrich et al.
2003/0172325 September 2003 Wyatt et al.
2003/0236961 December 2003 Qiu et al.
2004/0040025 February 2004 Lehtinen
2004/0054997 March 2004 Katragadda et al.
2004/0230764 November 2004 Merchant et al.
2006/0004675 January 2006 Bennett et al.
2006/0074954 April 2006 Hartline et al.
2006/0117217 June 2006 Chien et al.
2006/0136928 June 2006 Crawford et al.
2006/0168575 July 2006 Bhatt et al.
2006/0212873 September 2006 Takahisa
2006/0224431 October 2006 von Helmolt
2007/0124020 May 2007 Staples
2007/0179995 August 2007 Prahlad et al.
2007/0180294 August 2007 Kameyama et al.
2007/0245331 October 2007 Daynes et al.
2008/0033914 February 2008 Cherniack et al.
2008/0189705 August 2008 Weinert et al.
2008/0222214 September 2008 Tokuda
2009/0094250 April 2009 Dhuse et al.
2010/0122033 May 2010 Gebara et al.
2010/0131792 May 2010 Herrod
2010/0153941 June 2010 Borissov et al.
2010/0306267 December 2010 Zamkoff et al.
2010/0328528 December 2010 Eggert
2011/0022633 January 2011 Bernosky et al.
2011/0055661 March 2011 Grube et al.
2011/0078277 March 2011 Baptist
2011/0202929 August 2011 Schleimer et al.
2011/0225209 September 2011 Volvovski et al.
2011/0225426 September 2011 Agarwal et al.
2011/0264717 October 2011 Grube et al.
2011/0296195 December 2011 Nakagawa et al.
2011/0307514 December 2011 Goodman
2012/0017096 January 2012 Snider
2012/0079189 March 2012 Colgrove et al.
2012/0079190 March 2012 Colgrove et al.
2012/0131257 May 2012 Rudosky et al.
2012/0243687 September 2012 Li et al.
2012/0254089 October 2012 Alba et al.
2012/0254690 October 2012 Resch et al.
2012/0290539 November 2012 Bryant et al.
2012/0310878 December 2012 Vuksan et al.
2012/0322422 December 2012 Frecks, Jr. et al.
2013/0073600 March 2013 Jenkins et al.
2013/0151653 June 2013 Sawicki et al.
2013/0191527 July 2013 Ashok et al.
2013/0238932 September 2013 Resch
2013/0275776 October 2013 Baptist et al.
2013/0304711 November 2013 Resch
2014/0006458 January 2014 Hsieh et al.
2014/0006850 January 2014 Alley et al.
2014/0046906 February 2014 Patiejunas et al.
2014/0046908 February 2014 Patiejunas et al.
2014/0046909 February 2014 Patiejunas et al.
2014/0047040 February 2014 Patiejunas et al.
2014/0047181 February 2014 Peterson et al.
2014/0047261 February 2014 Patiejunas et al.
2014/0108421 April 2014 Isaacson et al.
2014/0122572 May 2014 Finkelstein et al.
2014/0149794 May 2014 Shetty et al.
2014/0149986 May 2014 S M et al.
2014/0153481 June 2014 Draznin et al.
2014/0156632 June 2014 Yu et al.
2014/0173058 June 2014 Twitchell, Jr.
2014/0189388 July 2014 Lynar et al.
2014/0222778 August 2014 Nie et al.
2014/0269484 September 2014 Dankberg et al.
2014/0298134 October 2014 Grube et al.
2014/0304356 October 2014 Allen, Sr. et al.
2014/0310462 October 2014 Waldspurger et al.
2014/0310571 October 2014 Fetterly et al.
2014/0344446 November 2014 Rjeili et al.
2014/0351632 November 2014 Grube et al.
2014/0372383 December 2014 Sipek
2014/0372693 December 2014 Yochai
2014/0380126 December 2014 Yekhanin et al.
2015/0149870 May 2015 Kozat
2015/0154111 June 2015 D'Abreu et al.
2015/0169716 June 2015 Franklin et al.
2015/0268866 September 2015 Nakagawa
2015/0278324 October 2015 Wong et al.
2015/0324745 November 2015 Goodall et al.
2015/0331635 November 2015 Ben-Shaul et al.
2015/0355974 December 2015 Hayes et al.
2015/0356005 December 2015 Hayes et al.
2016/0011816 January 2016 Aizman
2016/0034295 February 2016 Cochran
2016/0041868 February 2016 Davis et al.
2016/0041869 February 2016 Davis et al.
2016/0041878 February 2016 Davis et al.
2016/0041887 February 2016 Davis et al.
2016/0048399 February 2016 Shaw
2016/0085797 March 2016 Patiejunas et al.
2016/0092248 March 2016 Shani et al.
2016/0179824 June 2016 Donlan et al.
2016/0216991 July 2016 Ansari et al.
2016/0286289 September 2016 Long et al.
2016/0335310 November 2016 Lahiri et al.
2017/0024281 January 2017 Franklin et al.
2017/0060687 March 2017 Franklin et al.
2017/0222814 August 2017 Oberhauser et al.
2017/0250801 August 2017 Chen et al.
2017/0331896 November 2017 Holloway et al.
2004531923 Oct 2004 JP
20130107383 Oct 2013 KR
02071382 Sep 2002 WO
2014047073 Mar 2014 WO
Amazon, "Batch Cloud Data Transfer Services--Amazon Import/Export Snowball Appliance," Jun. 17, 2016, retrieved Oct. 8, 2016, https://web.archive.org/web/20160617044144/http://aws.amazon.com/importex- port/, 6 pages. cited by applicant .
Barr, "AWS Import/Export: Ship us That Disk!," Amazon Web Services Blog, May 21, 2009, retrieved Mar. 14, 2017, https://aws.amazon.com/blogs/aws/send-us-that-data/, 7 pages. cited by applicant .
International Search Report and Written Opinion dated Aug. 25, 2016, International Patent Application No. PCT/US2016/040510, filed Jun. 30, 2016. cited by applicant .
Storer et al., "POTSHARDS--A Secure, Recoverable, Long-Term Archival Storage System," ACM Transactions on Storage, Published Jun. 2009, vol. 5, No. 2, Article 5, pp. 5:1 to 5:35. cited by applicant .
Zyga, "Light-up Cereal Boxes Powered by Shelvers on Display at CES," Phys.org, Jan. 11, 2011, retrieved May 19, 2015, http://phys.org/news/201101lightupcerealpoweredshelvesces.html, 13 pages. cited by applicant .
"New! xTablet T7000 Rugged Mini Tablet PC," MobileDemand, copyright 2012 [web archive Mar. 12, 2012], https://web.archive.org/web/20120312010139/http://www.ruggedtabletpc.com/- products/xtablet-t7000-rugged-mini-tablet-pc/, 3 pages. cited by applicant .
Binns, "Elasticsearch Failure and Recovery," TechRabbit, Oct. 31, 2014 [retrieved Nov. 17, 2017], http://tech.taskrabbit.com/blog/2014/10/31/es-failure-recovery/, four pages. cited by applicant .
Franco, "Understanding Bitcoin: Cryptography, Engineering and Economics," Wiley, Nov. 24, 2014, 167 pages. cited by applicant .
He et al., "Elastic Application Container: A Lightweight Approach for Cloud Resource Provisioning," 26th IEEE International Conference on Advanced Information Networking and Applications, Mar. 26, 2012, pp. 15-22. cited by applicant .
International Organization for Standardization/ International Electrotechnical Commission, "Information technology--Trusted Platform Module--Part 1: Overview," International Standard, ISO/IEC 11889-1(E), May 15, 2009, 20 pages. cited by applicant .
International Organization for Standardization/International Electrotechnical Commission, "Information technology Trusted Platform Module--Part 2: Design principles," International Standard, ISO/IEC 11889-2(E), May 15, 2009, 152 pages. cited by applicant .
International Organization for Standardization/International Electrotechnical Commission, "Information technology--Trusted Platform Module--Part 3: Structures," International Standard, ISO/IEC 11889-3:2009(E), 204 pages. cited by applicant .
International Organization for Standardization/International Electrotechnical Commission, "Information technology--Trusted Platform Module--Part 4: Commands," International Standard, ISO/IEC 11889-4:2009(E), 254 pages. cited by applicant .
International Search Report and Written Opinion in International Patent Application No. PCT/US2015/050513, dated Feb. 16, 2016, 22 pages. cited by applicant .
International Search Report and Written Opinion dated Feb. 4, 2016, International Patent Application No. PCT/US2015/059983, 12 pages. cited by applicant .
International Search Report and Written Opinion dated Nov. 22, 2017, International Patent Application No. PCT/US2017/054319, filed Sep. 29, 2017, 14 pages. cited by applicant .
Kim, "How Sharding Works," Medium, Dec. 5, 2014 [retrieved Nov. 17, 2017], https://medium.com/@jeeyoungk/how-sharding-works-b4dec46b3f6, 12 pages. cited by applicant .
MacCarthaigh, "Shuffle Sharding: Massive and Magical Fault Isolation," AWS Architecture Blog, Apr. 14, 2014 [retrieved Nov. 27, 2017], https://aws.amazon.com/blogs/architecture/shuffle-sharding-massive-and-ma- gical-fault-isolation/, six pages. cited by applicant .
Soltesz et al., "Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors," ACM SIGOPS Operating Systems Review 41(3):275-287, Mar. 2007. cited by applicant .
Swan, "Blockchain: Blueprint for a New Economy," O'Reilly Media, Inc., Jan. 22, 2015, 144 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 1 Design Principles," Specification Version 1.2, Level 2 Revision 103, Jul. 9, 2007, 182 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 1 Design Principles," Specification Version 1.2, Revision 116, Mar. 1, 2011, 184 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 2 TPM Structures," Specification Version 1.2, Level 2 Revision 103, Jul. 9, 2007, 198 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 2 TPM Structures," Specification Version 1.2, Revision 116, Mar. 1, 2011, 201 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 3 Commands," Specification Version 1.2, Level 2 Revision 103, Jul. 9, 2007, 330 pages. cited by applicant .
Trusted Computing Group, "TPM Main, Part 3 Commands," Specification Version 1.2, Revision 116, Mar. 1, 2011, 339 pages. cited by applicant .
Van et al., "SLA-aware Virtual Resource Management for Cloud Infrastructures," IEEE Ninth International Conference on Computer and Information Technology, Oct. 11, 2009, pp. 357-362. cited by applicant .
Wikipedia, "IEEE 802.21," Wikipedia, the Free Encyclopedia, page last modified Aug. 4, 2016, retrieved Feb. 13, 2017, https://en.wikipedia.org/wiki/IEEE_802.21, 3 pages. cited by applicant .
Xavier et al., "Performance evaluation of container-based virtualization for high performance computing environments," Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference, Feb. 2013, pp. 233-240. cited by applicant .
Zhao et al., "Experimental study of virtual machine migration in support of reservation of cluster resources," Proceedings of the 2nd international workshop on Virtualization technology in distributed computing, Nov. 2007, pp. 1-8. cited by applicant .
Zheng et al., "Grid-partition index: a hybrid method for nearest-neighbor queries in wireless location-based services," The VLDB Journal--The International Journal on Very Large Data Bases 15(1):21-39, online publication Jul. 22, 2005, print publication Jan. 1, 2006. cited by applicant .
Pikkarainen et al., "The impact of agile practices on communication in software development," Empirical Software Engineering 13(3):303-37, Jun. 1, 2008. cited by applicant .
Ramamritham, "Allocation and scheduling of precedence-related periodic tasks," IEEE Transactions on Parallel and Distributed Systems 6(4):412-420, Apr. 1995. cited by applicant .
Thiele et al., "Embedded Software in Network Processors--Models and Algorithms," Lecture Notes in Computer Science 2211:416-34, Oct. 8, 2001. cited by applicant .
IEEE 100, "The Authoritative Dictionary of IEEE Standards Terms", Seventh Edition, IEEE Standards Information Network, IEEE Press, Dec. 2000, 5 pages (pertinent pages 1, 2, 155, 207, 1112). cited by applicant .
Mickens et al., "Blizzard: Fast, Cloud-scale Block Storage for Cloud-oblivious Applications," Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation, Apr. 2, 2014, 18 pages. cited by applicant .
Third-Party Submission Under 37 CFR 1.290 dated Apr. 24, 2018, U.S. Appl. 15/283,017, filed Sep. 30, 2016, 10 pages. cited by applicant .
European Office Action dated Nov. 6, 2018, Patent Application No. 16739357.8-1222, published May 9, 2018, 7 pages. cited by applicant .
PC Plus, "How to turn an old netbook into a NAS drive," TechRadar, Mar. 1, 2010 [retreived Feb. 5, 2019], https://www.techradar.com/news/networking/routers-storage/how-to-turn-an-- old-netbook-into-a-nas-drive-670757, 12 pages. cited by applicant.
1. A computer-implemented method, comprising: in response to receiving a plurality of data storage requests for data to be stored in a plurality of volumes of durable storage of a data storage system, processing the data storage requests by at least: obtaining events associated with each of the plurality of data storage requests; assigning the events to one or more specified database tables, wherein the one or more database tables are rotated for assignment of the events, the database tables being configured so as to include an entry for each of the assigned events in both: a primary index table that utilizes a pseudorandomly generated primary key for each of the assigned events; and a secondary index table that uses information in the primary index table as its primary key, the information being different from the pseudorandomly generated primary key; storing, asynchronous from receiving the plurality of the data storage requests, the data into the plurality of volumes in accordance with the events assigned to the one or more database tables, the data stored by at least: applying a redundancy code to the data associated with the data storage requests to generate a set of shards representative of the data; storing the shards on the plurality of volumes; and updating entries corresponding to the events assigned to the one or more database tables; and responding, synchronous from receiving the plurality of the data storage requests, to status requests regarding pendency of the events associated with each of the plurality of data storage requests by at least using the primary key of the secondary index table.
4. A system, comprising: at least one computing device configured to implement one or more services, wherein the one or more services are configured to: in response to receiving a plurality of data processing requests, process the plurality of requests by at least: assigning events associated with each request of the plurality of requests to a subset of a series of database tables, wherein the subset of the series of database tables is cycled for assignment of the events, each of the assigned events being associated with a respective identifier, the identifier generated stochastically, and the assigning of events causing each event of the assigned event to be allocated to a processing partition based on a value of the respective identifier; and in response to an update related to an allocated event, using a respective processing partition to execute the update.
8. The system of claim 7, wherein the one or more services are further configured to, in response to the data storage requests, asynchronously service the data storage requests by at least: encoding data associated with the data storage requests using a redundancy code, so as to generate a set of shards, a quorum quantity of the set usable to reconstruct original data associated with the set of shards; storing the shards on one or more data storage devices; and generating updates for a respective allocated event in accordance with storing the shards.
12. A non-transitory computer-readable storage medium storing thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least: in response to receiving a data processing request, populate a database table with information relating to a plurality of events associated with the data processing request, the information including an identifier for each event of the plurality of events, the database table being configured to use the identifier as its primary key, the identifier being generated so as to decorrelate the events from other events when the events are assigned; assign each event of the plurality of events to one of a plurality of processing partitions based at least in part on a value of the identifier, each processing partition of the plurality of processing partitions having a portion of overall processing capacity of a database service configured to service the plurality of events; and cause the database service to service the event using the processing partition.
This application incorporates by reference for all purposes the full disclosure of co-pending U.S. patent application Ser. No. 14/578,230, filed Dec. 19, 2014, entitled "TECHNIQUES FOR ADAPTING DATA STORAGE SYSTEMS FOR PEAKY USAGE MODELS," co-pending U.S. patent application Ser. No. 14/741,409, filed Jun. 16, 2015, entitled "ADAPTIVE DATA LOSS MITIGATION FOR REDUNDANCY CODING SYSTEMS," and co-pending U.S. patent application Ser. No. 14/789,783, filed Jul. 1, 2015, entitled "GRID ENCODED DATA STORAGE SYSTEMS FOR EFFICIENT DATA REPAIR."
The data processing requests 104 may be produced by a program, process, application, module, service, or system associated with a computing resource service provider as described herein. The data processing requests 104 may also be produced by a user or customer of the computing resource service provider, and submitted to the computing resource service provider via a customer device and transmitted via a network. The data processing request may include volatile data, which may be added to, changed, and/or deleted from in response to, for example, one or more requests (e.g., application programming interface requests or "API requests") made by the user or customer of the computer system. The data processing request may also include non-volatile data (also referred to herein as "static data"), which may be at least partially unchanging as the one or more requests are received.
In examples where the data processing system 116 is a data storage service, the data stored across associated durable storage volumes, such as in a bundle of shards, may have an associated durability that may be based on, for example, an annual failure rate ("AFR") of the data storage volume or the mapped data storage volume. For a given AFR, it may be assumed that the daily failure rate ("DFR") for the data storage volume or the mapped data storage volume is the AFR divided by three-hundred and sixty-five (i.e., the number of days in a year) and the hourly failure rate ("HFR") of the data storage volume or the mapped data storage volume is the DFR divided by twenty-four (i.e., the number of hours in a day). For example, if a data storage volume or the mapped data storage volume has an AFR of 2 percent, the data storage volume or the mapped data storage volume has a DFR of 0.0055 percent and an HFR of 0.00023 percent.
If, for example, the illustrated bundle of shards has a minimum quorum quantity of two shards out of the three illustrated, any two of the bundle of shards--regardless of whether the shard is an identity shard or a derived shard, may be processed using the redundancy code so as to regenerate the data, e.g., encrypted data. Additionally, the original data may be regenerated by directly reading the identity shards.
It should be noted that, as used herein, the durability of data and/or data storage may be separate from the redundancy of the data in the data storage. For example, data stored in preliminary storage (as described in connection with FIG. 2) may be highly durable (i.e., have a very low failure rate) but may not be redundant if, for example, it is stored as a single copy. Conversely, data stored using one or more redundancy encoding techniques such as those described herein and while such data may be less durably stored (i.e., may have fewer "9's" of durability), it may be highly redundant. For example, data stored in a grid may have no fewer than four separate copies of the data (one of the data shard, one from the horizontally-derived shards, one from the vertically-derived shards, and one from the remaining shards). If the grid is geographically distributed into, for example, multiple datacenters in multiple geographic regions, the data may have greater redundancy due to the added tolerance for loss of a complete datacenter.
The database service 110 may be any service or system that provides for organization, collection, and retrieval of data (e.g., metadata), which may include schemas, tables, queries, events, and the like. The database service may be connected to other components via a network. In some embodiments, the database service 110 communicates with other components in programmatic fashion, such as by the use of application programming interfaces (APIs), web services, and the like. The database service 110 may be capable of scaling for demand within a given level of provisioned or overall capacity (such as the capacity allocated by a requesting customer or other entity) by the use of processing partitions, each of which provides a portion of the provisioned or overall capacity. For example, at a nominal level of activity, the database service 110 may allocate events, requests, etc. to a single processing partition that provides the full provisioned capacity of the database service. However, over a given threshold of activity, the database service 110 may "split" the processing partition in two, three, or more partitions, and may continue to do so as activity increases. Similarly, partitions may be removed as activity decreases. The overall keyspace available for allocation, as may be contemplated, may also be partitioned accordingly. In some embodiments, the database service allocates incoming events and other entries according to the value of an identifier for each entry, which, in some embodiments, is the primary key of the primary index to which the items are allocated (e.g., a primary database table). The activity, as well as the capacity, may be on any relevant axis. Examples include transactional capacity (e.g., the number of input/output operations per unit time, e.g., IOPS), payload capacity (e.g., in bytes), request capacity (e.g., in bytes or in unit quantity (number of requests)), and the like.
The set of data may be stored in the preliminary storage 212 in an unaltered form (e.g., not processed, compressed, indexed, or altered prior to storage). The set of data may also be stored in the preliminary storage 212 as, for example, original data (also referred to herein as an "identity shard") such as the original data shards described herein. In an embodiment, the set of data stored in the preliminary storage 212 is stored without indexing and without any redundancy encoding. In another embodiment, the set of data stored in the preliminary storage 212 is stored with null redundancy encoding (i.e., a redundancy encoding that maps the data to itself). The data in preliminary storage may be stored as raw data, or may be bundle-encoded, or may be grid-encoded, or may be stored using some other method.
As used herein, the durability of a data object may be understood to be an estimate of the probability that the data object will not unintentionally become permanently irretrievable (also referred to herein as "unavailable"). This durability is an estimated probability and is generally expressed as a percentage (e.g., 99.9999 percent). This durability is based on assumptions of probabilities of certain failures (e.g., the AFR of drives used to store the data) and may be based on an average failure rate, a maximum failure rate, a minimum failure rate, a mean failure rate, or some other such failure rate. The durability may be based on a statistical average of the failure over a collection of drives when there are many different drives and/or when there are many different types of drives. The durability may also be based on historical measurements of the failure of drives and/or statistical sampling of the historical measurements of the failure of drives. The durability may also be correlated with the probability that a data object will not unintentionally become unavailable such as, for example, basing the durability on the probability that a data object will unintentionally become unavailable. As may be contemplated, the methods of determining durability of data described herein are merely illustrative examples and other such methods of determining durability of data may be considered as within the scope of the present disclosure.
The database service 208 may, as previously discussed, programmatically interact with other entities (e.g., other services 206, such as a front end for the database service 208 and the data storage service 214) and store events and other entries in one or more database tables according to one or more schemas. In some embodiments, also as previously discussed, the database service 208 may be capable of scaling for demand within a given level of provisioned or overall capacity (such as the capacity allocated by a requesting customer or other entity) by the use of processing partitions, each of which provides a portion of the provisioned or overall capacity. For example, at a nominal level of activity, the database service 208 may allocate events, requests, etc. to a single processing partition that provides the full provisioned capacity of the database service. However, over a given threshold of activity, the database service 208 may "split" the processing partition in two, three, or more partitions, and may continue to do so as activity increases. Similarly, partitions may be removed as activity decreases. The overall keyspace available for allocation, as may be contemplated, may also be partitioned accordingly. In some embodiments, the database service allocates incoming events and other entries according to the value of an identifier for each entry, which, in some embodiments, is the primary key of the primary index to which the items are allocated (e.g., a primary database table). The activity, as well as the capacity, may be on any relevant axis. Examples include transactional capacity (e.g., the number of input/output operations per unit time, e.g., IOPS), payload capacity (e.g., in bytes), request capacity (e.g., in bytes or in unit quantity (number of requests)), and the like.
As previously mentioned in connection with FIGS. 1 and 2, a database service 302 (depicted to schematically show a relative level of processing capacity) may be implemented to "split" its capacity into one or more processing partitions 308 (four are exemplarily shown) as demand on the database service 302 increases and decreases. Also as previously described, the database service 302 may allocate a given entry (e.g., a given event, which may map many-to-one to incoming requests) to the overall keyspace (as shown in FIG. 3 as the cylindrical object) based on a value of the column selected as the primary key for the primary index. As may be contemplated, if the selected primary key causes significant correlation between an assigned partition within the keyspace and bursty behavior (e.g., a given account generating a disproportionately large number of entries/events), a large number of entries may be allocated to the same processing partition 308, thereby potentially overloading the capacity allocated thereto while the actual full capacity of the database service 302 remains underutilized.
For implementations where database operations (e.g., status requests, event updates, etc.) benefit from database lookups based on or sorted by a primary key (e.g., "provide all pending events for AccountID n") the database service 302 may further implement a secondary minimal index 306 that uses a different column or set of values (e.g., attributes) across the entries of the primary index 304 as its primary key. For example, as illustrated, the minimal secondary index 306 may use the AccountID field of the primary index as its primary key. So as to further aid lookups, the partition to which the entry/event is allocated may also be encapsulated in the value within that field (also as illustrated).
FIG. 6 illustrates an example environment 600 where a redundancy encoding technique is applied to data stored in durable storage as described in connection with FIG. 1 and in accordance with an embodiment. The redundancy encoding technique illustrated in FIG. 6 is an example of a grid encoding technique wherein each identity shard is part of a first set of one or more identity shards which may be bundled with one or more derived shards in a first group or bundle (i.e., in one dimension or direction) and each identity shard is also part of at least a second set of one or more identity shards which may be bundled with one or more other derived shards in a second bundle or group (i.e., in a second dimension or direction). As is illustrated in FIG. 6, a grid encoding technique is often implemented as a two-dimensional grid, with each shard being part of two bundles (i.e., both "horizontal" and "vertical" bundles). However, a grid encoding technique may also be implemented as a three-dimensional grid, with each shard being part of three bundles, or a four-dimensional grid, with each shard being part of four bundles, or as a larger-dimensional grid. Additional details of grid encoding techniques are described in U.S. patent application Ser. No. 14/789,783, filed Jul. 1, 2015, entitled "GRID ENCODED DATA STORAGE SYSTEMS FOR EFFICIENT DATA REPAIR," which is incorporated by reference herein.
In the example illustrated in FIG. 6, data 602 from preliminary storage is provided for storage in durable storage using a redundancy encoding technique with both horizontal derived shards and vertical derived shards. In the example illustrated in FIG. 6, a first datacenter 612 may contain data shards (denoted as a square shard with the letter "I"), horizontal derived shards (denoted as a triangular shard with the Greek letter "8" or delta), and vertical derived shards (denoted as an inverted triangle with the Greek letter "8") all of which may be stored on durable storage volumes within the first datacenter 612. A second datacenter 614, which may be geographically and/or logically separate from the first datacenter 612, may also contain data shards, horizontal derived shards, and/or vertical derived shards. A third datacenter 616, which may be geographically and/or logically separate from the first datacenter 612 and from the second datacenter 614, may also contain data shards, horizontal derived shards, and/or vertical derived shards. As illustrated in FIG. 6, each of the three datacenters may be a single vertical bundle. In an embodiment, each of the datacenters can include multiple vertical bundles. As may be contemplated, the number of datacenters illustrated in FIG. 6 and/or the composition of the datacenters illustrated in FIG. 6 are merely illustrative examples and other numbers and/or compositions of datacenters may be considered as within the scope of the present disclosure. The datacenters may be co-located or may be located in one or more separate datacenter locations.
FIG. 7 illustrates an example environment 700 where a redundancy encoding technique is applied to data stored in durable storage as described herein and in accordance with at least one embodiment. The redundancy encoding technique illustrated in FIG. 7 is an example of a bundle encoding technique wherein one or more identity shards (also referred to herein as "data shards") may be bundled with one or more derived shards in a single group or dimension. Additional details of bundle encoding techniques are described in U.S. patent application Ser. No. 14/741,409, filed Jun. 16, 2015, entitled "ADAPTIVE DATA LOSS MITIGATION FOR REDUNDANCY CODING SYSTEMS," which is incorporated by reference herein.
The archives may be stored, bit for bit (e.g., the "original data" of the archives), on a subset of the plurality of volumes 706. Also as mentioned, appropriate indices may also be stored on the applicable subset of the plurality of volumes 706. The original data of the archives is stored as a plurality of shards across a plurality of volumes, the quantity of which (either shards or volumes, which in some cases may have a one to one relationship) may be predetermined according to various factors, including the number of total shards that may be used to reconstruct the original data using a redundancy encode. In some embodiments, the number of volumes used to store the original data of the archives is the quantity of shards that may be used to reconstruct the original data from a plurality of shards generated by a redundancy code from the original data. As an example, FIG. 7 illustrates five volumes, three of which contain original data archives 708 and two of which contain derived data 710, such as redundancy encoded data. In the illustrated example, the redundancy code used may require any three shards to regenerate original data, and therefore, a quantity of three volumes may be used to write the original data (even prior to any application of the redundancy code).
Such encoded information may be any mathematically computed information derived from the original data, and depends on the specific redundancy code applied. As mentioned, the redundancy code may include erasure codes (such as online codes, Luby transform codes, raptor codes, parity codes, Reed-Solomon codes, Cauchy codes, Erasure Resilient Systematic Codes, regenerating codes, or maximum distance separable codes) or other forward error correction codes. In some embodiments, the redundancy code may implement a generator matrix that implements mathematical functions to generate multiple encoded objects correlated with the original data to which the redundancy code is applied. In some of such embodiments, an identity matrix is used, wherein no mathematical functions are applied and the original data (and, if applicable, the indices) are allowed to pass straight through. In such embodiments, it may be therefore contemplated that the volumes bearing the original data (and the indices) may correspond to objects encoded from that original data by the identity matrix rows of the generator matrix of the applied redundancy code, while volumes bearing derived data correspond to other rows of the generator matrix. In the example illustrated in FIG. 7, the five volumes 706 include three volumes that have shards (e.g., identity shards) corresponding to the original data of the original data archives 708, while two have encoded shards corresponding to the derived data 710 (also referred to herein as "derived shards"). As illustrated in FIG. 7, the three original data archives 708, and the two encoded shards corresponding to the derived data 710 form a bundle 718 (denoted by the dashed line). In this example, the applied redundancy code may result in the data being stored in a "3:5" scheme, wherein any three shards of the five stored shards are required to regenerate the original data, regardless of whether the selected three shards contain the original data or the derived data.
The illustrative environment includes at least one application server 908 and a data store 910. It should be understood that there can be several application servers, layers or other elements, processes or components, which may be chained or otherwise configured, which can interact to perform tasks such as obtaining data from an appropriate data store. Servers, as used herein, may be implemented in various ways, such as hardware devices or virtual computer systems. In some contexts, servers may refer to a programming module being executed on a computer system. As used herein, unless otherwise stated or clear from context, the term "data store" refers to any device or combination of devices capable of storing, accessing and retrieving data, which may include any combination and number of data servers, databases, data storage devices and data storage media, in any standard, distributed, virtual or clustered environment. The application server can include any appropriate hardware, software, and firmware for integrating with the data store as needed to execute aspects of one or more applications for the client device, handling some or all of the data access and business logic for an application. The application server may provide access control services in cooperation with the data store and is able to generate content including, but not limited to, text, graphics, audio, video, and/or other content usable to be provided to the user, which may be served to the user by the web server in the form of HyperText Markup Language ("HTML"), Extensible Markup Language ("XML"), JavaScript, Cascading Style Sheets ("CSS"), or another appropriate client-side structured language. Content transferred to a client device may be processed by the client device to provide the content in one or more forms including, but not limited to, forms that are perceptible to the user audibly, visually, and/or through other senses. The handling of all requests and responses, as well as the delivery of content between the client device 902 and the application server 908, can be handled by the web server using PHP: Hypertext Preprocessor ("PHP"), Python, Ruby, Perl, Java, HTML, XML, or another appropriate server-side structured language in this example. Further, operations described herein as being performed by a single device may, unless otherwise clear from context, be performed collectively by multiple devices, which may form a distributed and/or virtual system.
The data store 910 can include several separate data tables, databases, data documents, dynamic data storage schemes and/or other data storage mechanisms and media for storing data relating to a particular aspect of the present disclosure. For example, the data store illustrated may include mechanisms for storing production data 912 and user information 916, which can be used to serve content for the production side. The data store also is shown to include a mechanism for storing log data 914, which can be used for reporting, analysis or other such purposes. It should be understood that there can be many other aspects that may need to be stored in the data store, such as page image information and access rights information, which can be stored in any of the above listed mechanisms as appropriate or in additional mechanisms in the data store 910. The data store 910 is operable, through logic associated therewith, to receive instructions from the application server 908 and obtain, update or otherwise process data in response thereto. The application server 908 may provide static, dynamic, or a combination of static and dynamic data in response to the received instructions. Dynamic data, such as data used in web logs (blogs), shopping applications, news services and other such applications may be generated by server-side structured languages as described herein or may be provided by a content management system ("CMS") operating on, or under the control of, the application server. In one example, a user, through a device operated by the user, might submit a search request for a certain type of item. In this case, the data store might access the user information to verify the identity of the user and can access the catalog detail information to obtain information about items of that type. The information then can be returned to the user, such as in a results listing on a web page that the user is able to view via a browser on the user device 902. Information for a particular item of interest can be viewed in a dedicated page or window of the browser. It should be noted, however, that embodiments of the present disclosure are not necessarily limited to the context of web pages, but may be more generally applicable to processing requests in general, where the requests are not necessarily requests for content.
Previous Patent US 10,394,788 | Next Patent US 10,394,790