Module Expr Str Import alias identifier ImportFrom alias Import alias Import alias Import alias identifier Import alias Import alias ImportFrom alias Expr Str FunctionDef arguments args NameParam defaults body If CompareNotEq AttributeLoad NameLoad attr Num body Raise Call NameLoad Str Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Num Expr Str Assign NameStore BinOpSub SubscriptLoad AttributeLoad NameLoad attr Index Num Num Expr Str Assign NameStore BinOpMult ListLoad Num NameLoad Assign NameStore BinOpMult ListLoad Num NameLoad Assign NameStore Num Expr Str For NameStore Call NameLoad NameLoad body For NameStore Call NameLoad BinOpAdd NameLoad Num NameLoad body Assign NameStore Num For NameStore Call NameLoad Num NameLoad body AugAssignAdd NameStore BinOpPow BinOpSub SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index NameLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Str If CompareLt NameLoad SubscriptLoad NameLoad Index NameLoad body Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad If CompareLt NameLoad SubscriptLoad NameLoad Index NameLoad body Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Assign NameStore Num Expr Str For NameStore Call NameLoad NameLoad body If CompareEq SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num SubscriptLoad SubscriptLoad NameLoad Index SubscriptLoad NameLoad Index NameLoad Index Num body Assign NameStore BinOpAdd NameLoad Num Return BinOpDiv Call NameLoad NameLoad NameLoad decorator_list Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num keyword Str Expr Call AttributeLoad Call NameLoad attr Str keyword TupleLoad Num Num keyword TupleLoad Num Num keyword Call NameLoad keyword Str keyword Num Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr Call NameLoad Call NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr Call NameLoad Call NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr decorator_list Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index Num SubscriptLoad NameLoad ExtSlice Slice Index Num keyword Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr decorator_list Expr Str FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore BinOpMult SubscriptLoad AttributeLoad NameLoad attr Index Num Num Assign NameStore SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice Assign NameStore SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Num Expr Str Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Slice Num Num Str Expr Str Assign NameStore Num For TupleStore NameStore NameStore Call NameLoad NameLoad body If CompareEq NameLoad SubscriptLoad NameLoad Index TupleLoad NameLoad Num body Assign NameStore BinOpAdd NameLoad Num Assign NameStore BinOpDiv Call NameLoad NameLoad AttributeLoad NameLoad attr Return NameLoad decorator_list Expr Str FunctionDef arguments args defaults body Expr Str For NameStore Call NameLoad Num body Expr Str If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num orelse If CompareEq NameLoad Num body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num orelse Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword ListLoad Num Num keyword Num keyword ListLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Call AttributeLoad Call AttributeLoad NameLoad attr Num attr Num Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Expr Str Assign NameStore Call NameLoad Assign NameStore BinOpSub NameLoad NameLoad Assign NameStore BinOpSub NameLoad NameLoad Assign NameStore ListLoad Assign NameStore ListLoad For NameStore Call NameLoad Num body Assign NameStore Call NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad Assign NameStore BinOpAdd NameLoad BinOpSub Call NameLoad NameLoad Assign NameStore Call NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore BinOpAdd NameLoad BinOpSub Call NameLoad NameLoad Assign SubscriptStore NameLoad ExtSlice Slice Num Index Num BinOpSub SubscriptLoad NameLoad ExtSlice Slice Num Index Num Num Print Str NameLoad Print Str NameLoad Expr Call NameLoad NameLoad NameLoad NameLoad NameLoad decorator_list Expr Str FunctionDef arguments args defaults body Expr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr Num Expr Str Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Num attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Num attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Num attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Num attr Num Num Assign NameStore Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Num Num Num attr Num Num Assign NameStore ListLoad NameLoad NameLoad NameLoad NameLoad NameLoad Expr Str Assign NameStore BinOpAdd BinOpAdd BinOpAdd Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Num Num For TupleStore NameStore NameStore Call NameLoad NameLoad body If CompareGtE NameLoad Num body Assign SubscriptStore NameLoad Index NameLoad Num orelse Assign SubscriptStore NameLoad Index NameLoad Num For NameStore Call NameLoad Num Num body Assign NameStore ListLoad Assign NameStore ListLoad Assign NameStore ListLoad For NameStore Call AttributeLoad NameLoad attr Str NameLoad body Expr Str Assign NameStore SubscriptLoad NameLoad Index BinOpSub Call NameLoad SubscriptLoad NameLoad Index Num Num Assign NameStore SubscriptLoad NameLoad Index Num Expr Str For NameStore Call NameLoad Num Call NameLoad NameLoad body Assign NameStore BinOpAdd NameLoad Call NameLoad SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad SubscriptLoad NameLoad Index BinOpSub Call NameLoad SubscriptLoad NameLoad Index NameLoad Num Expr Str Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Str Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call NameLoad NameLoad NameLoad Num keyword Str Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr Call NameLoad NameLoad Num NameLoad Num keyword Str Expr Call AttributeLoad NameLoad attr TupleLoad SubscriptLoad NameLoad Index Num SubscriptLoad NameLoad Index Num TupleLoad Str Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr BinOpAdd BinOpAdd Str Call NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr TupleLoad Num Num If CompareLt Call NameLoad NameLoad Num body Expr Call AttributeLoad NameLoad attr TupleLoad Num Num Expr Call AttributeLoad AttributeLoad Call NameLoad attr attr BinOpAdd Call AttributeLoad NameLoad attr Call NameLoad NameLoad Num Expr Call AttributeLoad AttributeLoad Call NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list If CompareEq NameLoad Str body Expr Call NameLoad
N/A N/A 
(c) 2011, 2012 Georgia Tech Research Corporation
This source code is released under the New BSD license.  Please see
http://wiki.quantsoftware.org/index.php?title=QSTK_License
for license details.

Created on Feb 20, 2011
@author: John Cornwell
@organization: Georgia Institute of Technology
@contact: JohnWCornwellV@gmail.com
@summary: This is an implementation of the 1-KNN algorithm for ranking features quickly.
          It uses the knn implementation.
@status: oneKNN functions correctly, optimized to use n^2/2 algorithm.
 N/A matplotlib.pyplot plt pylab gca N/A itertools N/A string N/A numpy np N/A math N/A knn time clock N/A 
@summary: Query function for 1KNN, return value is a double between 0 and 1.

@param naData: A 2D numpy array. Each row is a data point with the final column containing the classification.
 oneKnn N/A N/A naData N/A N/A N/A N/A N/A naData ndim 2 N/A N/A N/A Exception Data should have two dimensions N/A lLen N/A N/A naData shape N/A 0 N/A  # of dimensions, subtract one for classification  N/A lDim N/A N/A N/A naData shape N/A 1 1 N/A  Start best distances as very large  N/A ldDistances N/A N/A 1e+300 lLen N/A llIndexes N/A N/A -1 lLen N/A dDistance 0.0 N/A  Loop through finding closest neighbors  N/A i N/A range lLen N/A N/A j N/A range N/A i 1 lLen N/A N/A dDistance 0.0 N/A k N/A range 0 lDim N/A N/A dDistance N/A N/A N/A N/A naData N/A i N/A k N/A N/A naData N/A j N/A k 2 N/A dDistance N/A N/A math sqrt dDistance N/A  Two distances to check, for i's best, and j's best  N/A N/A dDistance N/A ldDistances N/A i N/A N/A N/A ldDistances N/A i dDistance N/A N/A llIndexes N/A i j N/A N/A dDistance N/A ldDistances N/A j N/A N/A N/A ldDistances N/A j dDistance N/A N/A llIndexes N/A j i N/A lCount 0 N/A  Now count # of matching pairs  N/A i N/A range lLen N/A N/A N/A N/A N/A naData N/A i N/A -1 N/A N/A naData N/A N/A llIndexes N/A i N/A -1 N/A N/A lCount N/A lCount 1 N/A N/A N/A float lCount lLen N/A N/A  Test function to plot  results  _plotResults N/A N/A naDist1 naDist2 lfOneKnn lf5Knn N/A N/A N/A N/A N/A plt clf N/A N/A N/A plt subplot 311 N/A N/A N/A plt scatter N/A naDist1 N/A N/A N/A 0 N/A naDist1 N/A N/A N/A 1 N/A N/A N/A plt scatter N/A naDist2 N/A N/A N/A 0 N/A naDist2 N/A N/A N/A 1 color r N/A N/A N/A N/A gca annotate xy N/A 0.7 0 xytext N/A 1.5 0 arrowprops N/A dict facecolor black shrink 0.05 N/A N/A N/A plt title Data Distribution N/A N/A N/A plt subplot 312 N/A N/A N/A plt plot N/A range N/A len lfOneKnn lfOneKnn N/A N/A N/A plt ylabel 1-KNN Value N/A N/A N/A plt title 1-KNN Performance N/A N/A N/A plt subplot 313 N/A N/A N/A plt plot N/A range N/A len lf5Knn lf5Knn N/A N/A N/A plt ylabel % Correct Classification N/A N/A N/A plt title 5-KNN Performance N/A N/A N/A plt subplots_adjust N/A N/A N/A plt show N/A N/A  Function to plot 2 distributions  _plotDist N/A N/A naDist1 naDist2 i N/A N/A N/A N/A N/A plt clf N/A N/A N/A plt scatter N/A naDist1 N/A N/A N/A 0 N/A naDist1 N/A N/A N/A 1 N/A N/A N/A plt scatter N/A naDist2 N/A N/A N/A 0 N/A naDist2 N/A N/A N/A 1 color r N/A N/A N/A plt ylabel Feature 2 N/A N/A N/A plt xlabel Feature 1 N/A N/A N/A plt title N/A Iteration  N/A str i N/A N/A N/A plt show N/A N/A  Function to test KNN performance  _knnResult N/A N/A naData N/A N/A N/A  Split up data into training/testing  N/A lSplit N/A N/A N/A naData shape N/A 0 0.7 N/A naTrain N/A naData N/A N/A lSplit N/A N/A naTest N/A naData N/A N/A lSplit N/A N/A N/A N/A knn addEvidence N/A N/A naTrain astype float 1 N/A  Query with last column omitted and 5 nearest neighbors  N/A naResults N/A N/A knn query N/A naTest N/A N/A N/A -1 5 mode N/A  Count returns which are correct  N/A lCount 0 N/A N/A i dVal N/A enumerate naResults N/A N/A N/A dVal N/A naTest N/A N/A i -1 N/A N/A lCount N/A lCount 1 N/A dResult N/A N/A float lCount N/A naResults size N/A dResult N/A N/A  Tests performance of 1-KNN  _test1 N/A N/A N/A N/A N/A  Generate three random samples to show the value of 1-KNN compared to 5KNN learner performance  N/A i N/A range 3 N/A N/A  Select one of three distributions  N/A N/A i 0 N/A N/A naTest1 N/A N/A N/A np random normal loc N/A 0 0 scale 0.25 size N/A 500 2 N/A naTest1 N/A N/A np hstack N/A naTest1 N/A N/A N/A N/A np zeros 500 reshape -1 1 N/A naTest2 N/A N/A N/A np random normal loc N/A 1.5 0 scale 0.25 size N/A 500 2 N/A naTest2 N/A N/A np hstack N/A naTest2 N/A N/A N/A N/A np ones 500 reshape -1 1 N/A N/A N/A i 1 N/A N/A naTest1 N/A N/A N/A np random normal loc N/A 0 0 scale 0.25 size N/A 500 2 N/A naTest1 N/A N/A np hstack N/A naTest1 N/A N/A N/A N/A np zeros 500 reshape -1 1 N/A naTest2 N/A N/A N/A np random normal loc N/A 1.5 0 scale 0.1 size N/A 500 2 N/A naTest2 N/A N/A np hstack N/A naTest2 N/A N/A N/A N/A np ones 500 reshape -1 1 N/A N/A naTest1 N/A N/A N/A np random normal loc N/A 0 0 scale 0.25 size N/A 500 2 N/A naTest1 N/A N/A np hstack N/A naTest1 N/A N/A N/A N/A np zeros 500 reshape -1 1 N/A naTest2 N/A N/A N/A np random normal loc N/A 1.5 0 scale 0.25 size N/A 250 2 N/A naTest2 N/A N/A np hstack N/A naTest2 N/A N/A N/A N/A np ones 250 reshape -1 1 N/A naOrig N/A N/A np vstack N/A naTest1 naTest2 N/A naBoth N/A N/A np vstack N/A naTest1 naTest2 N/A  Keep track of runtimes  N/A t N/A clock N/A cOneRuntime N/A t t N/A cKnnRuntime N/A t t N/A lfResults N/A N/A lfKnnResults N/A N/A i N/A range 15 N/A N/A t N/A clock N/A N/A N/A lfResults append N/A oneKnn naBoth N/A cOneRuntime N/A cOneRuntime N/A N/A clock t N/A t N/A clock N/A N/A N/A lfKnnResults append N/A _knnResult N/A N/A N/A np random permutation naBoth N/A cKnnRuntime N/A cKnnRuntime N/A N/A clock t N/A N/A naBoth N/A N/A 500 N/A 0 N/A N/A naBoth N/A N/A 500 N/A 0 0.1 N/A Runtime OneKnn: cOneRuntime N/A Runtime 5-KNN: cKnnRuntime N/A N/A _plotResults naTest1 naTest2 lfResults lfKnnResults N/A N/A  Tests performance of 1-KNN  _test2 N/A N/A N/A N/A N/A  Generate three random samples to show the value of 1-KNN compared to 5KNN learner performance  N/A N/A N/A N/A np random seed 12345 N/A  Create 5 distributions for each of the 5 attributes  N/A dist1 N/A N/A N/A N/A N/A np random uniform -1 1 1000 reshape -1 1 N/A dist2 N/A N/A N/A N/A N/A np random uniform -1 1 1000 reshape -1 1 N/A dist3 N/A N/A N/A N/A N/A np random uniform -1 1 1000 reshape -1 1 N/A dist4 N/A N/A N/A N/A N/A np random uniform -1 1 1000 reshape -1 1 N/A dist5 N/A N/A N/A N/A N/A np random uniform -1 1 1000 reshape -1 1 N/A lDists N/A dist1 dist2 dist3 dist4 dist5 N/A  All features used except for distribution 4  N/A distY N/A N/A N/A N/A N/A np sin dist1 N/A N/A np sin dist2 N/A N/A np sin dist3 N/A N/A np sin dist5 N/A distY N/A N/A distY reshape -1 1 N/A N/A i fVal N/A enumerate distY N/A N/A N/A fVal 0 N/A N/A N/A distY N/A i 1 N/A N/A N/A distY N/A i 0 N/A i N/A range 1 6 N/A N/A lsNames N/A N/A lf1Vals N/A N/A lfVals N/A N/A perm N/A N/A itertools combinations 12345 i N/A N/A  set test distribution to first element  N/A naTest N/A lDists N/A N/A N/A int N/A perm N/A 0 1 N/A sPerm N/A perm N/A 0 N/A  stack other distributions on  N/A j N/A range 1 N/A len perm N/A N/A sPerm N/A sPerm N/A str N/A perm N/A j N/A naTest N/A N/A np hstack N/A naTest N/A lDists N/A N/A N/A int N/A perm N/A j 1 N/A  finally stack y values  N/A naTest N/A N/A np hstack N/A naTest distY N/A N/A N/A lf1Vals append N/A oneKnn naTest N/A N/A N/A lfVals append N/A _knnResult N/A N/A N/A np random permutation naTest N/A N/A N/A lsNames append sPerm N/A  Plot results  N/A plt1 N/A N/A plt bar N/A N/A np arange N/A len lf1Vals lf1Vals 0.2 color r N/A plt2 N/A N/A plt bar N/A N/A N/A np arange N/A len lfVals 0.2 lfVals 0.2 color b N/A N/A N/A plt legend N/A N/A plt1 N/A 0 N/A plt2 N/A 0 N/A 1-KNN KNN, K=5 N/A N/A N/A plt ylabel 1-KNN Value/KNN Classification N/A N/A N/A plt xlabel Feature Set N/A N/A N/A plt title N/A N/A Combinations of  N/A str i  Features N/A N/A N/A plt ylim N/A 0 1 N/A N/A N/A len lf1Vals 2 N/A N/A N/A N/A plt xlim N/A -1 1 N/A N/A N/A N/A N/A gca xaxis set_ticks N/A N/A N/A np arange N/A len lf1Vals 0.2 N/A N/A N/A N/A N/A gca xaxis set_ticklabels lsNames N/A N/A N/A plt show N/A N/A N/A __name__ __main__ N/A N/A N/A _test1