Package: features-factory
features_factoryA small python package that helps dealing with Machine Learning features (usingpandas).Installing the packageLatest available code:pipinstallfeatures_factorySpecific version:pipinstallfeatures_factory==X.Y.ZQuickstartVerify the input columncountry_codein a pandas DataFramedf:feat=CountryCodeInputFeature('Country Code')input_error=feat.verify_input(df)input_error.has_missing_columns(),input_error.has_columns_with_nan(),input_error.has_columns_with_wrong_format()Map a column into another through a lambda function:original_feat=StringInputFeature('Full Name')# declare a feature corresponding to the columnfeat=OneComponentFeature('First Name',original_feat,lambdax:x.split(' ')[0])enriched_df=feat.insert_into(df)Map two columns into one using a lambda function:original_feat1=StringInputFeature('First Name')original_feat2=StringInputFeature('Second Name')feat=TwoComponentFeature('Full Name',original_feat1,original_feat2,lambdar:r[0]+' '+r[1])new_df=feat.insert_into(df)Create a stack of features:original_feat1=StringInputFeature('First Name')original_feat2=StringInputFeature('Second Name')feat=TwoComponentFeature('Full Name',original_feat1,original_feat2,lambdar:r[0]+' '+r[1])stack=Stack([original_feat1,original_feat2,feat])Use a stack like a list:stack.add(feat)stack.remove(feat)print('Number of features in this stack = ',len(stack))forfeatinstack:print(feat.name())feat=stack[1]stack=stack1+stack2stack+=stack3A stack automatically ignores duplicates:stack=Stack([feat1,feat2])stack.add([feat1])len(stack)==2Handy stack functionalities:original_feat1=StringInputFeature('First Name')original_feat2=StringInputFeature('Second Name')feat=TwoComponentFeature('Full Name',original_feat1,original_feat2,lambdar:r[0]+' '+r[1])stack=Stack([feat])stack=stack.with_dependencies()stack.names()==['Full Name','First Name','Second Name']original_feat1=StringInputFeature('First Name')original_feat2=StringInputFeature('Second Name')feat=TwoComponentFeature('Full Name',original_feat1,original_feat2,lambdar:r[0]+' '+r[1])stack=Stack([feat1,feat2,feat])stack=stack.only_inputs()stack.names()==['First Name','Second Name']Verify multiple input columns:stack=Stack([feat1,feat2,feat3])input_error=stack.verify(df).get_input_data_error()Create a stack of features, verify the input data, the feature dependencies, and insert the feature in the df:# input featuresdistance=FloatInputFeature('Distance [m]')duration=IntInputFeature('Duration [s]')runner_first_name=StringInputFeature('Runner First Name')runner_last_name=StringInputFeature('Runner Last Name')runner_age=IntInputFeature('Runner Age')# derived featuresspeed=TwoComponentFeature('Average Speed [km/h]',distance,duration,lambdar:3.6*r[0]/r[1])full_name=TwoComponentFeature('Full Name',runner_first_name,runner_last_name,lambdar:r[0]+' '+r[1])full_name_with_age=TwoComponentFeature('Full Name With Age',full_name,runner_age,lambdar:r[0]+' (age{})'.format(r[1]))# final featuresummary=TwoComponentFeature('Summary',full_name_with_age,speed,lambdar:'The runner{}run with and average speed of{}km/h'.format(r[0],r[1]))# create a stackstack=Stack([summary]).with_dependencies()# look for errorsstack_error=stack.verify(df)# populate the df with all the featuresifstack_error.is_empty():new_df=stack.insert_into(df)Are you working with a moltitude of features and you need to apply the same operation
to them? Check out theStackFactoryclass. E.g.int1=IntInputFeature('int1')int2=IntInputFeature('int2')float1=FloatInputFeature('float1')names=['2 x int1','2 x int2','2 x float1']dependencies=[int1,int2,float1]args=[{'name':name,'dependency':feat,'map_function':lambdax:2*x}forname,featinzip(names,dependencies)]stack=StackFactory.clones(OneComponentFeature,args)df=pd.DataFrame({int1.name():[3,5,7],int2.name():[15,20,50],float1.name():[2.2,0.1,5.5]})df=stack.with_dependencies().insert_into(df)print(df)#    int1  int2  float1  2 x float1  2 x int1  2 x int2# 0     3    15     2.2         4.4         6        30# 1     5    20     0.1         0.2        10        40# 2     7    50     5.5        11.0        14       100Pre-Built FeaturesInput Features:BoolInputFeature: booleanIntInputFeature: integerFloatInputFeature: floating pointDateTimeInputFeature: datetimeDateInputFeature: dateStringInputFeature: stringStringTimestampInputFeature: string encoding a timestamp readable via pandas.to_datetime, or according a specificformatCountryCodeInputFeature: two-digit country code (e.g. DE, IT, FR, ES)One-Component FeaturesOneComponentFeature: define a new feature starting from another one, simply by specifying a lambda functionRenamedFeature: rename a feature columnDateTimeFromStringFeature: extract the datetime from a string which encodes a timestampDateFromStringFeature: extract the date from a string which encodes a timestampMonthFromDateFeature: extract the month from a date-like objectWeekdayFromDateFeature: extract the weekday from a date-like object (0=Monday, 6=Sunday)Two-Component FeaturesTwoComponentFeature: define a new feature starting from two others, simply by specifying a lambda functionDurationFeature: given a start datetime and an end datetime, compute the durationMulti-Component Features:MultiComponentFeature: define a new feature starting from multiple other ones, simply by specifying a lambda functionComposed Features:MeanValueForKeyFeature: given a column with keys and one with values, aggregate the values according to the keys and compute their averages. Finally assign the averages to the new column, according to the keys.External Data Source Features:ValueFromExternalDfFeature: merge with an external dataframe on some columns and one resulting column.Why You Should Use This Librarydata verification is a rather painful and tricky task. This library can help in many ways:make you think about itlet you use some checks that others already used that can help you identify issues, like missing columns, presence of NaN, and wrong data formathow many times did it happen that you check the data and they seem ok, but then you modify them somehow, don't check them again (because what should have changed?) but something goes wrong? With this library you build a stack that let you make this verification in a very simple manner, avoiding these situations.oftenfeatures are built one on top of another creating a rather complicated tree of dependenciesthat can be annoying to manage manually. This library lets you define the features structure, and then take care of everything for you.think for a moment about how many timespeople wrote again and again the same verification codefor a feature, or the code to generate one. And how many times stupid mistakes led to a big waste of time? The idea of this open source library is to avoid this.using this library will force you to aseparation of concepts. Using it, your code will look cleaner.Developers should knowCreate a virtual environment and activate itpython-mvenvvenvsourcevenv/bin/activateInstall the development packagespipinstall-e.[dev]and use pre-commit to make sure that your code is blackified automatically (used theblackpackage):pre-commitinstallRun tests:pipinstall-e.[test]coveragerun-munittestdiscovertests
coveragereportBuild documentation (see more detailshere):pipinstall-e.[doc]mkdocsbuildor usemkdocsserveif you prefer a live, self-refreshing, documentation.