Package: yo-fluq-ds
yo_fluq_dsThis package is an data science-specific update foryo_fluqthat introduces:querying and output forpandasdata structures and files inQueryablehandyfeed-based extension methods.The main reason for separatingyo_fluq_dsfromyo_fluqis that data science functionality requires huge packages likepandasandmatplotlib, which I didn't want to include in a basic package.Small useful classesObjis an ordered dict with a member-like access:obj.a=12works exactly asobj['a']=12OrderedEnumisEnumwith ordering, it's useful when using enums inpandas, because the basic enumeration cannot be used as keys forgroup_byPull-queries updatesCombinatoricsQuery.combinatoricshas some useful method to create lazy combinatorics enumerations:cartesian(en1,en2,...)will create a cartesian product of enumerations inen1,en2, etc.grid(field1=en1,field2=en2)will create an enumeration ofObjwith fieldsfield1,field2that runs over cartesian product ofen1,en2, etc.triangleis query-like replacement for loopsi=0..N,j=0..ipowersetproduces all the subsets of a given setFile systemAdds several aggregators/query sources to work with files.to_text_file/Query.file.text: text file, its lines are interpreted as enumeration's objectsto_zip_file/Query.file.zipped_text: zipped text fileto_pickle_file/Query.file.pickle: a internal format, lazily writes a sequence of objects in pickle format in one file.to_zip_folder/Query.to_zipped_folder: representation forKeyValuePair: filenames are keys, its concent is valuesAddsFileIOclass with one-line instruction to read text, json, pickle, jsonpickle, yaml files.AddsQuery.foldermethod to create enumeration ofPathobjects from folderpandasAddsto_series,to_dataframeandto_ndarrayaggregatorsAddsQuery.seriesto convert series inKeyValuePairenumerationAddsQuery.dfto convert dataframe inObj(ordict) enumerationAddsfeedmethod toDataFrame,Series,DataFrameGroupByandSeriesGroupByby monkey-patching. It is now possible to write something like:(df.loc[df.status=='shipped'].feed(lambdaz:groupby(z.date.dt.to_period('M'))).size())When calling lambda insidefeed,zwill be assigned to the dataframe after filtering out.This technique allows longer fluent instructions forpandas, which is otherwise impossible due to filtering.feed-extension methods.Some methods fromyo_fluq_dsare not incorporated intoQueryables, because they are used not that often and I want to avoid overloadingQueryablewith such methods. So, they are accessible only viafeedmethod.All of them are insidefluqmodule.ForQueryablefluq.with_progress_baris a Queryable-friendly wrapping overtqdm. It automatically detects notebooks/console environments. Thetotal(length of enumerable) in most cases is known fromQueryable.lengthfield, but sometimes needs to be provided.fluq.with_plots(columns,figsize)will create plots for each of elements in enumerable and return the enumerable ofItemWithAx. Very handy to draw several plots at once, e.g. for different columns in dataframefluq.pairwiseconverts enumerable to the enumerable of pair of neighbouring elementsForpandasfluq.fractionscan be used where size is normally used to determine the relative size of the groupsfluq.trimmercan be used to trim too high/too low values from the series, thus facilitating histograms' creation.