Package: kodiak
Kodiak======.. image:: https://img.shields.io/github/license/mashape/apistatus.svg.. image:: https://readthedocs.org/projects/kodiak/badge/?version=latest:target: http://kodiak.readthedocs.io/en/latest/?badge=latestOverview--------**Kodiak** enhances your feature engineering workflow extracting commonpatterns so you can create more features faster.Ex: You have the ``writers`` dataframe, where ``born`` is a datetime+-----------------------+--------------+| name                  | born         |+=======================+==============+| Miguel de Cervantes   | 09-29-1547   |+-----------------------+--------------+| William Shakespeare   | 04-23-1617   |+-----------------------+--------------+and you want to extract from the ``born`` column: ``day``, ``month`` and``year`` and create 3 new columns+-----------------------+--------------+---------------+-------------+--------------+| name                  | born         | born\_month   | born\_day   | born\_year   |+=======================+==============+===============+=============+==============+| Miguel de Cervantes   | 09-29-1547   | 9             | 29          | 1547         |+-----------------------+--------------+---------------+-------------+--------------+| William Shakespeare   | 04-23-1617   | 4             | 23          | 1617         |+-----------------------+--------------+---------------+-------------+--------------+The simplest thing you could do in Pandas is:.. code:: pythonwriters["born_month"] = writers.born.map(lambda x: x.month)writers["born_day"]   = writers.born.map(lambda x: x.day)writers["born_year"]  = writers.born.map(lambda x: x.year)With Kodiak you could get the same result in one line:.. code:: pythonwriters.gencol("born_{month,day,year}", "born", lambda x, y: getattr(x, y))# or more succinctlywriters.gencol("born_{.month,.day,.year}", "born")But, how does it work? Kodiak uses ``"born_{month,day,year}"`` as atemplate for the columns: ``born_month``, ``born_day`` and ``born_year``and passes ``month``,\ ``day`` and ``year`` as arguments to a providedfunction that also has the current 'born' as an an argument, so you'rebasically doing:.. code:: pythonfor y in ['month', 'day', 'year']:writers["born_{}".format(y)] = writers.born.map(lambda x: getattr(x, y))Kodiak does a lot of other things to boost your workflow, for that, seethe *Basic Usage* and *Advanced Usage* sectionsInstallation------------To install Kodiak, ``cd`` to the Kodiak folder and run the installcommand:.. code:: shsudo python setup.py installYou can also install Kodiak from PyPI:.. code:: shsudo pip install kodiakBasic Usage-----------**Kodiak** main object is ``KodiakDataFrame`` an extension of``pandas.Dataframe`` that provides the instance method ``colgen`` tocreate one or more columns. You create ``KodiakDataFrames`` the same wayyou create a ``pandas.DataFrame````colgen`` signature is:``colgen(newcols, col, colbuilder=None, enum=False, config=None)``newcolshas a double function, it works as a specification of the columns youwant to create, and it also contains the values passed to ``colbuilder``.. code:: python# If you want to create the columns `first_name`, `last_name`# and pass `first` and `last` as arguments to `colbuilder` you write>>> newcols = "{first,last}_name"# More complex patterns allowed>>> groups = "col_{a,b}_{c,d}"# Will create the columns: `col_a_c`, `col_b_d`# The way `a,b` and `c,d` is combined can be configuredcolis the `KodiakDataframe` column from where you'll extract informationto create your new column/scolbuilderis a function or lambda used to extract info from ``col``and create the columns specified in ``newcols`` with thecorresponding ``col`` instance and the ``newcols`` values.The signature of `colbuilder` is `colbuilder(x, y)` or`colbuilder(i, x, y)` `x` is an instance of the column passedin `col` and `y` is an argument extracted from `newcols`. Theextra argument `i` is an index of the arguments.configtweak Kodiak inner workings with your own config, see thededicated section for more infoAdvanced Usage--------------In this section we're going to describe the main components and conceptsthat are essential to KodiakTemplating~~~~~~~~~~The template language is minimal but has some extensions to help you:Ranges^^^^^^The range notation is ``start:end:step``. Reverse ranges are permittedsetting ``end`` bigger than ``start``. ``step`` default value is ``1``, and``start`` is ``0``, finally if ``end`` is absent, it'll be setted to ``0`` andyou'll have a reversed range. Ranges are inclusive... code:: pythonsimple_range = "col_{1:3}" # -> col_1, col_2, col_3step_range = "col_{:3:2}" # -> col_0, col_3inverse_range = "col_{3:1}" # -> col_3,col_2,col_1no_end = "2:" # -> col_2,col_1,col_0Key-Value^^^^^^^^^If you want the column name and argument passed to the ``colbuilder`` todiffer you can use key-values... code:: pythondataframe.gencol("{short=very_long_name}_col", "col", alambda)# In this case the column name will be ``short_col`` but you'll pass# ``very_long_name`` to ``alambda``# key-value notation can be extended to more arguments:dataframe.gencol("{k1=v1,k2=v2,k3=v3}_col", "col", alambda).. WARNING::values are always interpreted as *strings* so in:``col_{k=1:5}`` the value ``1:5`` is interpreted as ``"1:5"`` and not asa range, the same for ``col_{k=[1,2,3]}`` and any other object, also ifyou pass a number it will also be interpreted as string so you will needto convert it if you intend to use it as an ``int``.Transforms~~~~~~~~~~Under the hood when you pass ``newcols`` to ``gencol``, Kodiak builds an``OrderedDict`` where it's keys are column names and it's values aretuples of ``Match`` objects -even if it's just one Match it's wrappedinside a tuple-.. code:: pythonnewcol = "{first,last}_name"# will buildargs_dict = {'first_name': (Match('first'),), 'last_name': (Match('last'),)}``Transforms`` are a way to pre-process the values and change themenriching the ``Match`` object with a payload as you will see in the``Default colbuilder`` section.So, if the values are ``Match`` objects, how is that when you write your``colbuilder`` you deal with ``strings``? Kodiak understands that if the``Match`` object doesn't have a payload it's better to pass stringsarguments to ``colbuilder``, this behaviour can be controlled.What's the use of ``Match`` objects and their ``payload``? What're someexamples of ``Transforms``? The next section will answer this questionsDefault colbuilder~~~~~~~~~~~~~~~~~~As you can see in the ``colgen`` signature, ``colbuilder`` defaultargument is ``None``, in special cases Kodiak can infer the``colbuilder`` method, let's revisit the opening example... code:: pythonwriters.gencol("born_{.month,.day,.year}", "born")The ``colbuilder`` in this case is inferred from the hint you gaveKodiak in the template: ``.month``, prefixing ``month`` with a ``.``indicates that you're referring to an attribute of ``born``, sointernally Kodiak builds a ``colbuider`` that extracts the ``month``from a ``born`` instance. Another way of omitting the ``colbuilder`` iswhen you have an instance method:.. code:: python# Notice the `!` after weekdaywriters.gencol("born_{weekday!}", "born").. WARNING::This hint only works for methods with no arguments, passinga method with one or more arguments will raise an errorHow is that Kodiak infers the ``colbuilder``? When the ``newcols`` areprocessed they go through a pipeline of ``Transforms``, one of them:``PropertyTransform`` detects that ``.month`` refers to an attribute andenriches de ``Match`` object hinting in the payload the corresponding``colbuilder``, that's why you don't need to pass the ``colbuilder``argument. But what happens if you give a ``colbuilder``? In this case,as the ``Match`` object has a ``payload`` instead of working with plainstrings you will work with tuples of ``Match`` objects.. Note::Kodiak will raise an exception when it can't figure out adefault colbuilderEnumerations~~~~~~~~~~~~Sometimes you care about the position of the arguments not the exactvalue, in that case you can use the ``enum`` param or the implicit``enum`` with a function or lambda of arity 3, the first argument willbe the index starting at 0... code:: pythonwriters.gencol("{first=0,last=1}_name", "name", lambda x,y: x.split(" ")[int(y)])# Another way with enum=Truewriters.gencol("{first,last}_name", "name", lambda i,x,y: x.split(" ")[i], enum=True)# Without enum=True but with a colbuilder with arity 3writers.gencol("{first,last}_name", "name", lambda i,x,y: x.split(" ")[i])Configuration-------------Almost everything is configurable in Kodiak, you could have a per-methodconfiguration or system-wide config.The ``Config`` object has the following customizable params:parserKodiak by default uses the ``ArgsParser`` class to parse ``newcols``match\_transformdata passed to the ``colbuilder`` could betransformed first, by default we use the ``default_transform`` pipeline,you could replace it with an array of ``Transforms`` objects.new\_col\_combinerin the newcols template if you have``"col_{a,b}_{c,d}"``, this results in the columns: ``"col_a_c"`` and``"col_b,d"``, how the different groups ``['a','b']`` and ``['c', 'd']``are combined is controlled with this param, currently we use the ``zip``function, and you could replace it with a function with similarsignature.unpackBoolean Default True, when ``newcols`` is simple, ``foo_{a,b}``instead of ``foo_{.a,b!}`` instead of passing to ``colbuilder``tuples of ``Match`` objects we just pass individual items,``a``, ``b``, so it's easier to build a ``colbuilder`` withouthaving to unwrap the ``Match`` tuplescol\_pair\_combinerOnce you have the arguments that you're going topass to the ``colbuilder`` you can combine them in different ways, currentlywe use ``product`` from itertools, ie: the cartesian product between anelement, ex: ``event``, and the other n-columns, creating the followingtuples:.. code:: python[('event', 'day') , ('event', 'month'), ('event', 'year')]You can replace this method with another with the same signature as ``product``Config can be accessed, modified and restored with:.. code:: python>> import config>> from config import cfg>> config.options# Global change on config>> config.options["unpack"] = False>> config.options["col_pair_combiner"] = zip# Restoring one or more fields of the configuration>> config.restore_default_config("col_pair_combiner")# Restoring all the options>> config.restore_default_config()# With `base_config` or it's alias `cfg` you can create modified versions# of the default config>> dataframe.gencol("col_{a!,b!}","col", func, config=cfg(unpack=False))=======History=======0.1.0 (2017-09-19)------------------* First release on PyPI.