Module Expr Str ImportFrom alias ImportFrom alias Import alias Import alias ImportFrom alias ImportFrom alias identifier alias identifier Import alias Import alias ImportFrom alias alias alias alias Import alias identifier Assign NameStore Dict Str Str Str ListLoad Dict Str Str Str Str Dict Str Str Str Str Str Call NameLoad keyword Str Assign NameStore Dict Call NameLoad Str Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str ListLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Num Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Num Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Num Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Num Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad ListLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str ListLoad Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str NameLoad NameLoad NameLoad Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str NameLoad NameLoad NameLoad Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str NameLoad NameLoad NameLoad Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str NameLoad NameLoad NameLoad Call NameLoad Str Dict Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str Call NameLoad Str NameLoad Call NameLoad Str NameLoad NameLoad NameLoad Call NameLoad Str Num FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore ListLoad Assign NameStore ListLoad For NameStore NameLoad body Assign NameStore Call NameLoad NameLoad NameLoad If UnaryOpNot NameLoad body Assign NameStore ListLoad NameLoad If Call NameLoad SubscriptLoad NameLoad Index Num Call NameLoad Dict body Expr Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad SubscriptLoad NameLoad Index Num attr orelse Expr Call AttributeLoad NameLoad attr ListLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad Call NameLoad Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad For NameStore NameLoad body Assign NameStore Call NameLoad For TupleStore NameStore NameStore Call NameLoad NameLoad body If Call NameLoad NameLoad Call NameLoad Dict body Expr Call AttributeLoad NameLoad attr NameLoad orelse Assign SubscriptStore NameLoad Index SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num NameLoad Expr Yield NameLoad decorator_list Return Call NameLoad Call NameLoad ListComp Call NameLoad NameLoad NameLoad comprehension NameStore NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad FunctionDef arguments args NameParam defaults body TryExcept body Assign NameStore Call NameLoad Call NameLoad NameLoad If CompareLt NameLoad Num body Assign NameStore ListLoad NameLoad Return NameLoad handlers ExceptHandler type NameLoad body Return ListComp Call AttributeLoad NameLoad attr NameLoad NameLoad comprehension NameStore NameLoad decorator_list Assign AttributeStore NameLoad attr BinOpMod Str TupleLoad NameLoad NameLoad Assign AttributeStore NameLoad attr Str Return NameLoad decorator_list FunctionDef arguments args defaults vararg body Expr Str Assign NameStore Call NameLoad NameLoad While NameLoad body TryExcept body Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Num handlers ExceptHandler type NameLoad body Expr Call AttributeLoad NameLoad attr Num Continue If BoolOpAnd Call NameLoad NameLoad Str Call NameLoad Call NameLoad NameLoad Str body Expr Call AttributeLoad NameLoad attr Num NameLoad orelse Expr Yield NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad NameLoad If Call NameLoad Dict NameLoad body For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad body If CompareEq NameLoad NameLoad body If Call NameLoad NameLoad Call NameLoad ListLoad body Expr Yield Call NameLoad NameLoad orelse Expr Yield NameLoad If CompareIn Call NameLoad NameLoad TupleLoad Call NameLoad Dict Call NameLoad ListLoad body Expr Yield Call NameLoad NameLoad NameLoad orelse If Call NameLoad ListLoad NameLoad body For NameStore NameLoad body Expr Yield Call NameLoad NameLoad NameLoad orelse Pass decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If Call NameLoad NameLoad Call NameLoad ListLoad body Return Call NameLoad ListComp Call NameLoad NameLoad NameLoad comprehension NameStore NameLoad orelse If Call NameLoad NameLoad Call NameLoad Dict body Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad NameLoad attr For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad NameLoad NameLoad Return NameLoad orelse If CompareIn Str NameLoad body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Assign NameStore NameLoad For NameStore NameLoad body Assign NameStore Call NameLoad NameLoad NameLoad Assign NameStore NameLoad Return NameLoad orelse Return Call NameLoad Call NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad NameLoad NameLoad If Call NameLoad NameLoad Call NameLoad Dict body For NameStore NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call NameLoad SubscriptLoad NameLoad Index NameLoad Return NameLoad orelse Return Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call NameLoad Assign NameStore Call NameLoad Str NameLoad Assign NameStore Call NameLoad keyword NameLoad NameLoad Return Call NameLoad GeneratorExp Call NameLoad Call NameLoad NameLoad comprehension NameStore NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad AttributeLoad NameLoad attr kwarg body Expr Str Assign NameStore Call NameLoad NameLoad NameLoad keyword NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr GeneratorExp Call AttributeLoad NameLoad attr comprehension NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad NameLoad Expr Call AttributeLoad NameLoad attr For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args defaults body Expr Call NameLoad Str Expr Call NameLoad Call NameLoad Call NameLoad Str NameLoad Assert CompareEq Call NameLoad Call NameLoad Str NameLoad ListLoad Str Str Str Assert CompareEq Call NameLoad Str NameLoad ListLoad Str Str Str Expr Call NameLoad Str Expr Call NameLoad Call NameLoad Call NameLoad ListLoad Str Str NameLoad Assert CompareEq Call NameLoad ListLoad Str Str NameLoad ListLoad Str Str Str Expr Call NameLoad Str Expr Call NameLoad Call NameLoad Str NameLoad Expr Call NameLoad Str Expr Call NameLoad Call NameLoad Call NameLoad Str NameLoad Expr Call NameLoad Str Expr Call NameLoad Call NameLoad Str NameLoad Expr Call NameLoad Str Expr Call NameLoad ListLoad Str Str ListLoad NameLoad keyword Str Expr Call NameLoad Str ImportFrom alias identifier Expr Call NameLoad Str Expr Call NameLoad SubscriptLoad Call NameLoad ListLoad Str Str Str Str ListLoad NameLoad keyword Str Slice Num decorator_list If CompareEq NameLoad Str body Pass
N/A N/A 

Tasks
-------

Search and transform jsonable structures, specifically to make it 'easy' to make tabular/csv output for other consumers.

Example
~~~~~~~~~~~~~

    *give me a list of all the fields called 'id' in this stupid, gnarly
    thing*

    >>> Q('id',gnarly_data)
    ['id1','id2','id3']


Observations:
---------------------

1) 'simple data structures' exist and are common.  They are tedious
   to search.

2)  The DOM is another nested / treeish structure, and jQuery selector is
    a good tool for that.

3a) R, Numpy, Excel and other analysis tools want 'tabular' data.  These
    analyses are valuable and worth doing.

3b) Dot/Graphviz, NetworkX, and some other analyses *like* treeish/dicty
    things, and those analyses are also worth doing!

3c) Some analyses are best done using 'one-off' and custom code in C, Python,
    or another 'real' programming language.

4)  Arbitrary transforms are tedious and error prone.  SQL is one solution,
    XSLT is another,

5)  the XPATH/XML/XSLT family is.... not universally loved :)  They are
    very complete, and the completeness can make simple cases... gross.

6)  For really complicated data structures, we can write one-off code.  Getting
    80% of the way is mostly okay.  There will always have to be programmers
    in the loop.

7)  Re-inventing SQL is probably a failure mode.  So is reinventing XPATH, XSLT
    and the like.  Be wary of mission creep!  Re-use when possible (e.g., can
    we put the thing into a DOM using

8)  If the interface is good, people can improve performance later.


Simplifying
---------------


1)  Assuming 'jsonable' structures

2)  keys are strings or stringlike.  Python allows any hashable to be a key.
    for now, we pretend that doesn't happen.

3)  assumes most dicts are 'well behaved'.  DAG, no cycles!

4)  assume that if people want really specialized transforms, they can do it
    themselves.

 __future__ print_function collections namedtuple N/A csv N/A itertools itertools product operator attrgetter aget itemgetter iget N/A operator N/A sys pandas.compat map u callable Counter N/A pandas.compat compat N/A ex1 N/A name extensions Gregg N/A N/A id url hello url1 N/A id url more gbye url2 N/A dict url url3 N/A ex2 N/A N/A u metadata N/A u survey_data N/A N/A u accessibilities N/A u extensions N/A u fxVersion N/A u location N/A u operatingSystem N/A u surveyAnswers N/A u task_guid N/A u tpVersion N/A u updateChannel N/A N/A N/A u name N/A u value N/A u accessibility.tabfocus 7 N/A N/A u name N/A u value N/A u accessibility.mouse_focuses_formcontrol False N/A N/A u name N/A u value N/A u accessibility.browsewithcaret False N/A N/A u name N/A u value N/A u accessibility.win32.force_disabled False N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.startlinksonly False N/A N/A u name N/A u value N/A u accessibility.usebrailledisplay N/A u N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.timeout 5000 N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.enabletimeout True N/A N/A u name N/A u value N/A u accessibility.tabfocus_applies_to_xul False N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.flashBar 1 N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.autostart True N/A N/A u name N/A u value N/A u accessibility.blockautorefresh False N/A N/A u name N/A u value N/A u accessibility.browsewithcaret_shortcut.enabled True N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.enablesound True N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.prefillwithselection True N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.soundURL N/A u beep N/A N/A u name N/A u value N/A u accessibility.typeaheadfind False N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.casesensitive 0 N/A N/A u name N/A u value N/A u accessibility.warn_on_browsewithcaret True N/A N/A u name N/A u value N/A u accessibility.usetexttospeech N/A u N/A N/A u name N/A u value N/A u accessibility.accesskeycausesactivation True N/A N/A u name N/A u value N/A u accessibility.typeaheadfind.linksonly False N/A N/A u name N/A u value N/A u isInstantiated True N/A N/A N/A u id N/A u isEnabled N/A u 216ee7f7f4a5b8175374cd62150664efe2433a31 True N/A N/A u id N/A u isEnabled N/A u 1aa53d3b720800c43c4ced5740a6e82bb0b3813e False N/A N/A u id N/A u isEnabled N/A u 01ecfac5a7bd8c9e27b7c5499e71c2d285084b37 True N/A N/A u id N/A u isEnabled N/A u 1c01f5b22371b70b312ace94785f7b0b87c3dfb2 True N/A N/A u id N/A u isEnabled N/A u fb723781a2385055f7d024788b75e959ad8ea8c3 True N/A u 9.0 N/A u zh-CN N/A u WINNT Windows NT 5.1 N/A u N/A u d69fbd15-2517-45b5-8a17-bb7354122a75 N/A u 1.2 N/A u beta N/A N/A u extensions N/A u version_number N/A N/A N/A u appDisabled N/A u id N/A u isCompatible N/A u isEnabled N/A u isPlatformCompatible N/A u name False N/A u testpilot?labs.mozilla.com True True True N/A u Test Pilot N/A N/A u appDisabled N/A u id N/A u isCompatible N/A u isEnabled N/A u isPlatformCompatible N/A u name True N/A u dict?www.youdao.com False False True N/A u Youdao Word Capturer N/A N/A u appDisabled N/A u id N/A u isCompatible N/A u isEnabled N/A u isPlatformCompatible N/A u name False N/A u jqs?sun.com True True True N/A u Java Quick Starter N/A N/A u appDisabled N/A u id N/A u isCompatible N/A u isEnabled N/A u isPlatformCompatible N/A u name False N/A u ?20a82645-c095-46ed-80e3-08825760534b? True True True N/A u Microsoft .NET Framework Assistant N/A N/A u appDisabled N/A u id N/A u isCompatible N/A u isEnabled N/A u isPlatformCompatible N/A u name False N/A u ?a0d7ccb3-214d-498b-b4aa-0e8fda9a7bf7? True True True N/A u WOT 1 denorm N/A N/A queries iterable_of_things default N/A None N/A N/A 
    'repeat', or 'stutter' to 'tableize' for downstream.
    (I have no idea what a good word for this is!)

    Think ``kronecker`` products, or:

    ``SELECT single,multiple FROM table;``

    single   multiple
    -------  ---------
    id1      val1
    id1      val2


    Args:

        queries:  iterable of ``Q`` queries.
        iterable_of_things:  to be queried.

    Returns:

        list of 'stuttered' output, where if a query returns
        a 'single', it gets repeated appropriately.


     _denorm N/A N/A queries thing N/A N/A N/A fields N/A N/A results N/A N/A q queries N/A N/A r N/A Ql q thing N/A N/A r N/A N/A r N/A default N/A N/A isinstance N/A r N/A 0 N/A type N/A N/A N/A N/A N/A fields append N/A sorted N/A N/A N/A r N/A 0 keys N/A N/A N/A N/A fields append N/A q N/A N/A N/A results append r N/A flist N/A list N/A flatten N/A map iter fields N/A prod N/A N/A itertools product results N/A p prod N/A N/A U N/A dict N/A N/A ii thing N/A enumerate p N/A N/A N/A isinstance thing N/A type N/A N/A N/A N/A N/A U update thing N/A N/A N/A U N/A N/A N/A fields N/A ii N/A 0 thing N/A N/A U N/A N/A N/A list N/A flatten N/A N/A _denorm queries thing N/A thing iterable_of_things N/A default_iget N/A N/A fields default N/A None N/A N/A  itemgetter with 'default' handling, that *always* returns lists

    API CHANGES from ``operator.itemgetter``

    Note: Sorry to break the iget api... (fields vs *fields)
    Note: *always* returns a list... unlike itemgetter,
        which can return tuples or 'singles'
     N/A myiget N/A N/A operator itemgetter fields N/A L N/A len fields f N/A N/A thing N/A N/A N/A N/A N/A ans N/A list N/A myiget thing N/A N/A L 2 N/A N/A ans N/A ans N/A ans N/A N/A N/A KeyError N/A N/A N/A N/A N/A thing get x default N/A x fields N/A N/A N/A f __doc__ N/A itemgetter with default %r for fields %r N/A default fields N/A N/A f __name__ default_itemgetter N/A f N/A flatten N/A N/A N/A stack N/A N/A 
    helper function for flattening iterables of generators in a
    sensible way.
     N/A stack N/A list stack N/A stack N/A N/A N/A N/A x N/A next N/A stack N/A 0 N/A N/A N/A StopIteration N/A N/A N/A N/A stack pop 0 N/A N/A N/A N/A hasattr x next N/A callable N/A getattr x next N/A N/A N/A N/A stack insert 0 x N/A N/A N/A x N/A _Q N/A N/A filter_ thing N/A N/A N/A  underlying machinery for Q function recursion  N/A T N/A type thing N/A N/A isinstance N/A T N/A N/A N/A k v N/A N/A compat iteritems thing N/A N/A N/A filter_ k N/A N/A N/A isinstance v N/A type N/A N/A N/A N/A N/A iter v N/A N/A N/A v N/A N/A N/A type v N/A N/A type N/A N/A type N/A N/A N/A N/A N/A Q filter_ v N/A N/A N/A isinstance N/A T N/A N/A k thing N/A N/A N/A N/A Q filter_ k N/A N/A N/A Q N/A N/A filter_ thing N/A N/A N/A 
    type(filter):
    - list:  a flattened list of all searches (one list)
    - dict:  dict with vals each of which is that search

    Notes:

    [1] 'parent thing', with space, will do a descendent
    [2] this will come back 'flattened' jQuery style
    [3] returns a generator.  Use ``Ql`` if you want a list.

     N/A N/A isinstance filter_ N/A type N/A N/A N/A N/A flatten N/A N/A _Q x thing N/A x filter_ N/A N/A N/A isinstance filter_ N/A type N/A N/A N/A d N/A N/A dict fromkeys N/A list N/A N/A filter_ keys N/A k d N/A N/A N/A d N/A k N/A Q k thing N/A d N/A N/A N/A filter_ N/A N/A parts N/A N/A N/A N/A filter_ strip split N/A r None N/A p parts N/A N/A r N/A Ql p thing N/A thing r N/A r N/A N/A N/A flatten N/A _Q filter_ thing N/A Ql N/A N/A filter_ thing N/A N/A N/A  same as Q, but returns a list, not a generator  N/A res N/A Q filter_ thing N/A N/A isinstance filter_ N/A type N/A N/A N/A k res N/A N/A N/A res N/A k N/A list N/A res N/A k N/A res N/A N/A N/A list res N/A countit N/A N/A fields iter_of_iter default N/A None N/A N/A 
    note: robust to fields not being in i_of_i, using ``default``
     N/A C N/A Counter N/A T N/A namedtuple Thing fields N/A get N/A default_iget default default fields N/A N/A Counter N/A N/A T N/A get thing N/A thing iter_of_iter N/A printout N/A N/A queries things default f N/A None N/A sys stdout kwargs N/A N/A  will print header and objects

    **kwargs go to csv.DictWriter

    help(csv.DictWriter) for more.
     N/A results N/A denorm queries things default None N/A fields N/A set N/A N/A itertools chain N/A N/A N/A x keys N/A x results N/A W N/A N/A csv DictWriter f f fieldnames fields kwargs N/A N/A N/A W writeheader N/A r results N/A N/A N/A N/A W writerow r N/A test_run N/A N/A N/A N/A N/A N/A print 
>>> print(list(Q('url',ex1))) N/A N/A print N/A list N/A Q url ex1 N/A N/A N/A list N/A Q url ex1 N/A url1 url2 url3 N/A N/A N/A Ql url ex1 N/A url1 url2 url3 N/A N/A print 
>>>  print(list(Q(['name','id'],ex1))) N/A N/A print N/A list N/A Q N/A name id ex1 N/A N/A N/A Ql N/A name id ex1 N/A Gregg hello gbye N/A N/A print 
>>> print(Ql('more url',ex1)) N/A N/A print N/A Ql more url ex1 N/A N/A print 
>>> list(Q('extensions',ex1)) N/A N/A print N/A list N/A Q extensions ex1 N/A N/A print 
>>> print(Ql('extensions',ex1)) N/A N/A print N/A Ql extensions ex1 N/A N/A print 
>>> printout(['name','extensions'],[ex1,], extrasaction='ignore') N/A N/A printout N/A name extensions N/A ex1 extrasaction ignore N/A N/A print pprint pprint pp N/A N/A print -- note that the extension fields are also flattened!  (and N/A) --  N/A N/A pp N/A N/A denorm N/A location fxVersion notthere survey_data extensions N/A ex2 default N/A N/A 2 N/A N/A N/A __name__ __main__ N/A N/A