Package: pyckup
pyckupTools to provide easy access to prepared data to data scientists that can't
be asked.They just want to get on with the fun --notget stuck in data access and
data preparation concerns. And they should want that!Of course, someone needed to do the work of getting the data from where and how it is,
to where and how it needs to be (for a particular problem and context).What we believe is that this work should not only be less tedious and less
time-consuming (see py2store and related for that!), but also, once it's done,
it shouldn't have to be re-done every time someone wants to kick the data
around.So we madepyckup.We hope it helps.installpip install pyckupUsageA protocol is what's to the left of://in a url.
You've seen'em. For example,httpandhttps.
Well, if you try grabbing a valid url, you'll getbytesout of it,
without having to make an actual http request yourself.>>> b = grab('https://raw.githubusercontent.com/i2mint/pyckup/master/LICENSE')
>>> isinstance(b, bytes)
True

Now let's put those bytes in a file:

>>> from tempfile import mktemp
>>> filepath = mktemp()
>>> with open(filepath, 'wb') as fp:
...     n_bytes_written = fp.write(b);
>>> assert n_bytes_written == len(b)A filepath is a url. So you should be able to grab it's contents too.>>> file_bytes = grab(filepath)
>>> assert file_bytes == bThere wasn't really a protocol there, but whengrabsees that the url
you're passing it starts with a slash, it automatically prepends afile://to it, like most browswers do.
You can also specify thefile://protocol explicitly (again, like in
most browsers):>>> file_bytes_2 = grab('file://' + filepath)
>>> assert file_bytes_2 == file_bytes == bIf your filepath points to a folder, you'll get back adol.Filesobject,
which is aMappingwhose keys are file paths (relative to the folder) and
the values are the bytes (acquired lazily) of the corresponding file's contents.>>> import os, typing
>>> folder, filename = os.path.dirname(filepath), os.path.basename(filepath)
>>> f = grab(folder)
>>> isinstance(f, typing.Mapping)
True
>>> filename in f
True
>>> file_bytes_3 = f[filename]
>>> assert file_bytes_3 == file_bytes_2 == file_bytes == bgrabcan handle various protocols, according to what packages it finds
on your system. For example, if you havehaggle(https://pypi.org/project/haggle/) installed, you'll find thatkaggleis also a valid protocol.>>> sorted(grab.dflt_protocols) # doctest: +SKIP
['file', 'http', 'https', 'kaggle']Notice thatgrabhas an argument calledprotocols.
Yes, this means you have control.
You just need to specify a mapping between protocol strings and the "url_to_content"
function that should be used to get the content from that url.In the following we'll add a fakefooprotocol that doesn't really fetch any
data, but appliesstr.upperto the url, but you get the point.>>> from functools import partial
>>> from pyckup.base import dflt_protocols
>>> mygrab = partial(grab, protocols=dict(dflt_protocols, foo=str.upper))
>>> mygrab('foo://a_fake_url')
'FOO://A_FAKE_URL'ExamplesfrompyckupimportgrabSee what (default) protocols you have access to.frompyckupimportgrabgrab.procotols# ['file', 'kaggle', 'http', 'https']Grab file contentsSpecifying a "file" protocol (i.e. prefixing your string with
"file://" -- followed by a full path) will give you the contents of the file
in bytes.frompyckupimportgrabb=grab('file:///Users/Thor.Whalen/Dropbox/dev/p3/proj/i/pyckup/pyckup/__init__.py')assertisinstance(b,bytes)print(b.decode())# from pyckup.base import grab, protocolsBut you can also use a full path, or other natural means of specifying files.
In that case though,grabwill try to give you the contents in a convenient type
(e.g. adictfor.json, a python object of.pickle, string for.txt...).
This is convenient, but don't depend on the type to strongly
since it depends on whatpy2store.miscsets it to be.frompyckupimportgrabgrab('/Users/Thor.Whalen/Dropbox/dev/p3/proj/i/pyckup/pyckup/__init__.py')# b'from pyckup.base import grab, protocols\n\n\n'grab('~/Dropbox/dev/p3/proj/i/pyckup/data/example.json')# {'hello': 'world', 'abc': [1, 2, 3]}grab('~/Dropbox/dev/p3/proj/i/pyckup/data/example.pickle')# [1, 2, 3]print(grab('~/Dropbox/dev/p3/proj/i/pyckup/data/example.txt'))# This# is# textGrab the contents of a urlfrompyckupimportgrabb=grab('https://raw.githubusercontent.com/i2mint/pyckup/master/LICENSE')type(b),len(b)# (bytes, 11357)print(b[:100].decode())#                                  Apache License#                            Version 2.0, January 2004Grab stuff from kaggleNote: If you want to access kaggle datasets withpyckup,
you'll need to get an account.
Seehagglefor more information.frompyckupimportgrabz=grab('kaggle://drgilermo/face-images-with-marked-landmark-points')list(z)# ['face_images.npz', 'facial_keypoints.csv']print(z['facial_keypoints.csv'][:100].decode())# left_eye_center_x,left_eye_center_y,right_eye_center_x,right_eye_center_y,left_eye_inner_corner_x,le