Module Expr Str Import alias Import alias identifier Import alias alias Import alias Import alias identifier Import alias Expr Call AttributeLoad AttributeLoad NameLoad attr attr Str Import alias identifier Import alias Import alias Import alias identifier ClassDef bases body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Dict Dict NameLoad body Expr Str If CompareIs NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad orelse If CompareLt Call AttributeLoad NameLoad attr Str Num body AugAssignAdd NameStore Str With Call AttributeLoad NameLoad attr NameLoad NameStore body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Return IfExp CompareIsNot NameLoad NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr Num Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListComp AttributeLoad NameLoad attr comprehension TupleStore NameStore NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr ListComp Call AttributeLoad NameLoad attr ListLoad NameLoad comprehension TupleStore NameStore NameStore NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpMult ListLoad NameLoad Call NameLoad NameLoad For NameStore Call NameLoad Call NameLoad NameLoad body Assign NameStore UnaryOpInvert SubscriptLoad Call AttributeLoad NameLoad attr NameLoad Index NameLoad If Call AttributeLoad NameLoad attr NameLoad body Assign SubscriptStore NameLoad Index NameLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index TupleLoad NameLoad NameLoad orelse Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign NameStore ListComp TupleLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num SubscriptLoad NameLoad Index NameLoad comprehension NameStore Call NameLoad Call NameLoad NameLoad Assign TupleStore NameStore NameStore Call NameLoad NameLoad keyword Lambda arguments args NameParam defaults SubscriptLoad NameLoad Index Num Return SubscriptLoad SubscriptLoad SubscriptLoad Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr Index NameLoad Index NameLoad Index Num decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareIs AttributeLoad NameLoad attr NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareLt Call AttributeLoad NameLoad attr Str Num body AugAssignAdd NameStore Str With Call AttributeLoad NameLoad attr NameLoad Str NameStore body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore NameLoad For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Yield TupleLoad NameLoad SubscriptLoad SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr Index NameLoad Index NameLoad Index Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore NameLoad For TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Yield TupleLoad NameLoad SubscriptLoad SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr Index NameLoad Index NameLoad Index Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore ListComp Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad Return BinOpDiv Call NameLoad NameLoad SubscriptLoad NameLoad Index Num decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults BinOpPow Num Num body Expr Str Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad Assign NameStore ListLoad For TupleStore NameStore NameStore NameLoad body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr ExtSlice Slice Index Num Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad SubscriptLoad AttributeLoad NameLoad attr ExtSlice Slice Slice Num keyword Num NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr ListLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign NameStore SubscriptLoad NameLoad ExtSlice Slice Index Num Assign NameStore Call AttributeLoad SubscriptLoad NameLoad ExtSlice Slice Slice Num attr Assign NameStore BinOpMult Call AttributeLoad NameLoad attr NameLoad BinOpAdd Call AttributeLoad NameLoad attr NameLoad Num For NameStore Call NameLoad NameLoad body TryExcept body Assign SubscriptStore NameLoad Index NameLoad SubscriptLoad SubscriptLoad Call AttributeLoad NameLoad attr CompareEq SubscriptLoad NameLoad ExtSlice Slice Index NameLoad NameLoad Index Num Index Num handlers ExceptHandler type NameLoad body Continue Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad For NameStore Call NameLoad NameLoad body Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign SubscriptStore NameLoad ExtSlice Slice SubscriptLoad NameLoad Index NameLoad Index NameLoad BinOpSub NameLoad BinOpMult BinOpSub Num BinOpDiv NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword Num Return Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr ListLoad NameLoad AttributeLoad NameLoad attr decorator_list decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareGt Call AttributeLoad NameLoad attr Str Num body Return ListComp Call NameLoad NameLoad comprehension NameStore Call AttributeLoad NameLoad attr Str TryExcept body Return Call NameLoad NameLoad handlers ExceptHandler type NameLoad body Pass Assign NameStore Call NameLoad keyword Call NameLoad Call NameLoad Num Num keyword Call NameLoad ListLoad Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num Num Num Num Num Num Num keyword Call NameLoad ListLoad Num Num Num Num Num Num Num Num Num Num Num Num keyword Call NameLoad Call NameLoad Num Num keyword Call NameLoad Call NameLoad Num Num keyword Call NameLoad Call NameLoad Num Num keyword Call NameLoad Call NameLoad Num Num keyword Call NameLoad Call NameLoad Num Num Assign NameStore Call NameLoad ListLoad For NameStore Call AttributeLoad NameLoad attr Str NameLoad body If CompareIs Call AttributeLoad NameLoad attr Num NameLoad body Assign NameStore SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num orelse If CompareEq Call AttributeLoad NameLoad attr Num Str body AugAssignBitOr NameStore SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num orelse If CompareEq Call AttributeLoad NameLoad attr Num Str body AugAssignBitAnd NameStore SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num orelse If CompareEq Call AttributeLoad NameLoad attr Num Str body AugAssignSub NameStore SubscriptLoad NameLoad Index Call AttributeLoad NameLoad attr Num orelse Raise Call NameLoad BinOpAdd Str NameLoad Return Call NameLoad NameLoad decorator_list
N/A N/A 
Common routines providing post-processing utility.  They serve mainly
as data-feeding backends for cocopf.pplot routines.  The goal is to
have all the number crunching in this module (also usable for tables
generation, machine learning etc.) while keeping all presentation
in pplot.

Their counterpart
are several scripts in the pptools/ directory, the goal here is to
have these scripts as thin as possible to enable code reuse.

We use some COCO post-processing modules, but our focus here is shifted
from raw performance (in terms of FEVs needed for convergence to
a given target) to relative performance - it is not so important for
us to see how a portfolio converges for a particular (instance,target)
pair, but rather how the convergence speed changes relative to individual
algorithms (and some reference strategies).  Also, uncertainities are
compounded with portfolios and we thus further de-emphasize statistical
estimates of unobserved ERTs.
 N/A itertools N/A numpy np N/A pickle gzip N/A re N/A scipy.stats ss N/A sys N/A N/A N/A N/A sys path append . N/A bbob_pproc bb N/A bbob_pproc.algportfolio N/A bbob_pproc.bestalg N/A bbob_pproc.readalign ra PortfolioDataSets N/A N/A N/A 
    This container class holds data sets of a portfolio in terms
    of both individual algorithms and a variety of strategies,
    both preloaded and generated.

    Public class attributes:

      - *algds* -- dict of per-algorithm DataSetLists
      - *stratds* -- dict of per-strategy DataSetLists

    Use *algds* and *stratds* only read-only or things may get out of sync.

    Example:

        algorithms = {}
        for a in ['CMA', 'Powell', 'SLSQP']:
            algorithms[a] = bb.load(glob.glob('ppdata/'+a+'/bbobexp_f*.info'))
        strategies = {}
        for s in ['mUNIF', 'mEG50']:
            strategies[s] = bb.load(glob.glob('ppdata/'+s+'/bbobexp_f*.info'))
        pds = PortfolioDataSets(algorithms, strategies)

    (...continued in the cocopf.pplot examples.)
     __init__ N/A N/A self algorithms strategies pickleFile N/A N/A N/A None N/A N/A 
        Initialize the portfolio dataset container; pass dicts
        of DataSetLists (e.g. returned by bb.load()). Alternatively,
        we can unpickle an existing portfolio.
         N/A N/A pickleFile None N/A N/A N/A self algds algorithms N/A N/A self stratds strategies N/A N/A self _bestalg None N/A N/A self _unifpf None N/A N/A N/A N/A N/A pickleFile find .gz 0 N/A N/A pickleFile .gz N/A N/A N/A gzip open pickleFile f N/A N/A entry N/A N/A pickle load f N/A N/A self algds N/A entry algds N/A N/A self stratds N/A entry stratds N/A N/A self _bestalg N/A entry _bestalg N/A N/A self _unifpf N/A entry _unifpf N/A add_algorithm N/A N/A self name ds N/A N/A N/A 
        Add another algorithm.
         N/A N/A N/A self algds N/A name ds N/A N/A self _bestalg None N/A N/A self _unfipf None N/A add_strategy N/A N/A self name ds N/A N/A N/A 
        Add another strategy.
         N/A N/A N/A self stratds N/A name ds N/A bestalg N/A N/A self dimfun N/A N/A N/A 
        A BestAlgSet from all algorithms (like a DataSetList).
        Contains the best performance for each (instance,target) pair
        for a given dimfun=(dim,fun) tuple.

        The BestAlgSet represents a "left envelope" of lowest ERTs over
        all (function, target) pairs.  C.f. the oracle() method.

        Also, avoid depending on algbestfinalfunvals and similar attributes,
        which take only reached targets but not required budgets into account.
         N/A N/A N/A self _bestalg None N/A N/A N/A self _bestalg N/A N/A N/A bb bestalg generate N/A self algds N/A N/A N/A dimfun None N/A N/A self _bestalg N/A dimfun N/A self _bestalg N/A oracle N/A N/A self dimfun N/A N/A N/A 
        An oracle strategy is such that knows "in advance" which algorithm
        performs best on a given function and runs that one. This method
        returns a DataSetList of such a strategy - simply a DataSetList
        of the algorithm that reaches the best target fastest for the given
        dimfun=(dim,fun) tuple.

        N.B. the *bestalg* attribute is _better_ than even an oracle strategy,
        since it can switch between algorithms "for free" while the target
        moves ahead.
         N/A N/A dim funcId dimfun N/A bestfinalfunval N/A max N/A N/A np median N/A N/A N/A self bestalg dimfun bestfinalfunvals 1e-08 N/A algs N/A list N/A N/A self algds_dimfunc dimfun N/A maxevals N/A N/A np max N/A N/A ds maxevals N/A N/A name ds algs N/A evals N/A N/A np array N/A N/A N/A ds detEvals N/A bestfinalfunval N/A N/A name ds algs N/A nanmask N/A N/A np isnan evals N/A medevals N/A N/A maxevals N/A len algs N/A i N/A range N/A len algs N/A N/A algnanmask N/A N/A N/A N/A np isnan evals N/A i N/A N/A N/A np any algnanmask N/A N/A N/A medevals N/A i N/A N/A np median N/A evals N/A N/A i algnanmask N/A N/A N/A medevals N/A i maxevals N/A nametarget N/A N/A N/A N/A algs N/A i N/A 0 N/A medevals N/A i N/A i N/A range N/A len algs N/A N/A name target N/A min nametarget key N/A N/A N/A k N/A N/A k N/A 1 N/A N/A N/A N/A N/A N/A N/A N/A self algds N/A name dictByDimFunc N/A dim N/A funcId N/A 0 N/A unifpf N/A N/A self N/A N/A N/A 
        A "uniform portfolio" strategy is generated using the COCO's
        bb.algportfolio machinery, which represents a retroactively computed
        uniform strategy on the function evaluation level; i.e. it "stops"
        each algorithm after each function evaluation.
         N/A N/A N/A self _unifpf None N/A N/A N/A self _unifpf N/A N/A N/A bb algportfolio build N/A self algds N/A N/A self _unifpf N/A pickle N/A N/A self pickleFile N/A N/A N/A 
        Pickle the current portfolio dataset.  The file is automatically
        gzipped.

        Currently, this is aimed at fastest load times (to play with plotting
        stuff in different ways), therefore even redundant information like
        best and eUNIF are stored.  An alternative mode pickling only bare
        minimum (aimed at redistribution) will come in the future.
         N/A N/A N/A N/A pickleFile find .gz 0 N/A N/A pickleFile .gz N/A N/A N/A gzip open pickleFile w f N/A N/A N/A N/A pickle dump self f N/A algds_dimfunc N/A N/A self dimfun N/A N/A N/A 
        Return an iterable of (name, DataSet) tuples corresponding
        to the given dimfun.
         N/A N/A dim funcId dimfun N/A N/A algname dset N/A N/A N/A self algds iteritems N/A N/A N/A N/A algname N/A N/A N/A N/A N/A dset dictByDimFunc N/A dim N/A funcId N/A 0 N/A stratds_dimfunc N/A N/A self dimfun N/A N/A N/A 
        Return an iterable of (name, DataSet) tuples corresponding
        to the given dimfun.
         N/A N/A dim funcId dimfun N/A N/A stratname dset N/A N/A N/A self stratds iteritems N/A N/A N/A N/A stratname N/A N/A N/A N/A N/A dset dictByDimFunc N/A dim N/A funcId N/A 0 N/A maxevals N/A N/A self dimfun N/A N/A N/A 
        Return the maximum nominal budget across all algorithms; typically,
        something around 1e5 if you run your experiments with budget 100000.
         N/A evals N/A N/A N/A np median N/A ds maxevals N/A N/A name ds N/A N/A self algds_dimfunc dimfun N/A N/A N/A max evals N/A dimfun N/A 0 N/A ranking N/A N/A self dimfun groupby ftarget N/A N/A 10 -8 N/A N/A 
        Produce a set of function evaluation ranks over all algorithms
        and strategies.

        Returns a set of rows where each row contains a budget as first
        element and ranks for individual algorithms and strategies
        as the second element (in the order of the strategies in the
        output of algds_dimfunc(), and stratds_dimfunc() respectively).

        The ranks are always computed based on function values after
        a particular budget.  If multiple algorithms reach ftarget,
        they are ranked by the order in which they did.
         N/A nameds N/A list N/A N/A itertools chain N/A N/A self algds_dimfunc dimfun N/A N/A self stratds_dimfunc dimfun N/A count N/A len nameds N/A fvset N/A N/A N/A name ds nameds N/A N/A budgets N/A N/A ds funvals N/A N/A N/A 0 N/A f1vals N/A N/A np maximum N/A groupby N/A N/A ds funvals N/A N/A N/A 1 axis 1 ftarget N/A fv N/A N/A np transpose N/A N/A np vstack N/A budgets f1vals N/A N/A N/A fvset append fv N/A fva N/A N/A ra alignArrayData N/A N/A ra VArrayMultiReader fvset N/A budgets N/A fva N/A N/A N/A 0 N/A values N/A N/A N/A fva N/A N/A N/A 1 copy N/A firstconv N/A N/A N/A np ones count N/A N/A N/A np size budgets 1 N/A i N/A range count N/A N/A N/A N/A N/A firstconv N/A i N/A N/A N/A N/A np nonzero N/A N/A values N/A N/A N/A i ftarget N/A 0 N/A 0 N/A N/A N/A IndexError N/A N/A N/A firstconvranks N/A N/A N/A ss mstats rankdata firstconv N/A i N/A range count N/A N/A r N/A firstconvranks N/A i N/A N/A values N/A N/A N/A firstconv N/A i N/A i N/A ftarget N/A N/A 1 N/A r count ftarget N/A ranks N/A N/A N/A ss mstats rankdata values axis 1 N/A N/A N/A np transpose N/A N/A np vstack N/A budgets N/A ranks T N/A N/A resolve_fid N/A N/A fid N/A N/A N/A 
    Convert a given "function id" string to a number of list of numbers,
    with the ability to resolve symbolic names for a variety of function
    classes.

    XXX: So far, the classes are fixed as determined on the scipy+CMA
    portfolio. Instead, determine them dynamically.
     N/A N/A N/A N/A fid count , 0 N/A N/A N/A N/A int i N/A i N/A N/A fid split , N/A N/A N/A N/A int fid N/A N/A N/A ValueError N/A N/A N/A symbols N/A dict all N/A set N/A range 1 25 q N/A set N/A 1 2 5 single N/A set N/A 2 3 4 6 7 10 11 12 13 15 16 17 18 19 20 21 22 23 many N/A set N/A 1 5 8 9 14 24 volatile N/A set N/A 6 8 9 10 11 12 13 14 15 19 20 21 22 23 24 steady N/A set N/A 6 12 17 18 19 24 sudden N/A set N/A 1 2 3 4 5 7 8 9 10 11 13 14 15 16 20 21 22 23 CMAgood N/A set N/A 6 7 10 11 12 13 16 17 18 21 22 23 CMAbad N/A set N/A 1 2 3 4 5 8 9 14 15 19 20 24 separ N/A set N/A range 1 6 lcond N/A set N/A range 6 10 hcond N/A set N/A range 10 15 multi N/A set N/A range 15 20 mult2 N/A set N/A range 20 25 N/A fidset N/A set N/A N/A m N/A N/A re finditer ([+:-])?([^+:-]+) fid N/A N/A N/A N/A N/A m group 1 None N/A N/A fidset N/A symbols N/A N/A N/A m group 2 N/A N/A N/A N/A N/A m group 1 + N/A N/A fidset N/A symbols N/A N/A N/A m group 2 N/A N/A N/A N/A N/A m group 1 : N/A N/A fidset N/A symbols N/A N/A N/A m group 2 N/A N/A N/A N/A N/A m group 1 - N/A N/A fidset N/A symbols N/A N/A N/A m group 2 N/A N/A N/A ValueError N/A bad fid syntax  fid N/A N/A list fidset N/A