Francesc Alted - New Trends In Storing Large Data Silos With Python
Channel: EuroPython Conference
Date: 05/08/2015

Francesc Alted - New Trends In Storing Large Data Silos With Python
[EuroPython 2015]
[20 July 2015]
[Bilbao, Euskadi, Spain]

My talk is meant to provide an overview of our current set of tools
for storing data and how we arrived to these.  Then, in the light of
the current bottlenecks, and how hardware and software are evolving,
provide a brief overview of the emerging technologies that will be
important for handling Big Data within Python.  Although I expect my
talk to be a bit prospective, I won't certainly be trying to predict
the future, but rather showing a glimpse on what I expect we would be
doing in the next couple of years for properly leveraging modern
architectures (bar unexpected revolutions ;).

As an example of library adapting to recent trends in hardware, I will
be showing bcolz (https://github.com/Blosc/bcolz), which implements a
couple of data containers (and specially a chunked, columnar 'ctable')
meant for storing large datasets efficiently.