Big Data Specific Challenges (Gaps) High throughput of data for reduction into higher levels. Discovery of meaningful insights from low-value-density data needs new approaches to the deep, complex analysis e.g., using machine learning, statistical modelling, graph algorithms etc. which go beyond traditional approaches to the space physics. Big Data Specific Challenges in Mobility Is not likely in mobile platforms Security & Privacy Requirements Lower level of data has restrictions for 1 year within the associate countries. All data open after 3 years. Highlight issues for generalizing this use case (e.g. for ref. architecture) EISCAT 3D data e-Infrastructure shares similar architectural characteristics with other ISR radars, and many existing big data systems, such as LOFAR, LHC, and SKA More Information (URLs) Note: <additional comments> NBD(NIST Big Data) Requirements WG Use Case Template Use Case Title Big Data Archival: Census 2010 and 2000 – Title 13 Big Data Vertical (area) Digital Archives Author/Company/Email Vivek Navale & Quyen Nguyen (NARA) Actors/Stakeholders and their roles and responsibilities NARA’s Archivists Public users (after 75 years) Goals Preserve data for a long term in order to provide access and perform analytics after 75 years. Use Case Description Maintain data “as-is”. No access and no data analytics for 75 years. Preserve the data at the bit-level. Perform curation, which includes format transformation if necessary. Provide access and analytics after nearly 75 years. Current Solutions Compute(System) Linux servers Storage NetApps, Magnetic tapes. Networking Software Big Data Characteristics Data Source (distributed/centralized) Centralized storage. Volume (size) 380 Terabytes. Velocity (e.g. real time) Static. Variety (multiple datasets, mashup) Scanned documents
Requirements Highlight issues for generalizing this use case (e.g. for ref. architecture) . More Information (URLs) Note: <additional comments> NBD(NIST Big Data) Requirements WG Use Case Template Aug 11 2013 Use Case Title LifeWatch – E-Science European Infrastructure for Biodiversity and Ecosystem Research Vertical (area) Scientific Research: Life Science Author/Company/Email Wouter Los, Yuri Demchenko (y.demchenko@uva.nl), University of Amsterdam Actors/Stakeholders and their roles and responsibilities End-users (biologists, ecologists, field researchers) Data analysts, data archive managers, e-Science Infrastructure managers, EU states national representatives Goals Research and monitor different ecosystems, biological species, their dynamics and migration. Use Case Description LifeWatch project and initiative intends to provide integrated access to a variety of data, analytical and modeling tools as served by a variety of collaborating initiatives. Another service is offered with data and tools in selected workflows for specific scientific communities. In addition, LifeWatch will provide opportunities to construct personalized ‘virtual labs', also allowing to enter new data and analytical tools. New data will be shared with the data facilities cooperating with LifeWatch. Particular case studies: Monitoring alien species, monitoring migrating birds, wetlands LifeWatch operates Global Biodiversity Information facility and Biodiversity Catalogue that is Biodiversity Science Web Services Catalogue Current Solutions Compute(System) Field facilities TBD Datacenter: General Grid and cloud based resources provided by national e-Science centers Storage Distributed, historical and trends data archiving Networking May require special dedicated or overlay sensor network. Software Web Services based, Grid based services, relational databases Big Data Characteristics Data Source (distributed/centralized) Ecological information from numerous observation and monitoring facilities and sensor network, satellite images/information, climate and weather, all recorded information. Information from field researchers Volume (size) Involves many existing data sets/sources Collected amount of data TBD Velocity (e.g. real time) Data analysed incrementally, processes dynamics corresponds to dynamics of biological and ecological processes.