Package: paiargparse
PAI Argument ParserThe PAI Argument Parser extends the common python argument parser by allowing to automatically parse and add arguments
based on dataclasses.See the following example for usage:fromtypingimportListfrompaiargparseimportpai_dataclass,pai_meta,PAIArgumentParserfromdataclassesimportdataclass,field@pai_dataclass@dataclassclassSubmoduleParams:required_float_arg:floatlist_arg:List[int]=field(default_factory=lambda:[1,2])arg_with_custom_help:str=field(default="You can specify a custom help string",metadata=pai_meta(help="Custom help string."))@pai_dataclass@dataclassclassMyArguments:required_int_arg:intoptional_str_arg:str="This is cool stuff"sub_params:SubmoduleParams=field(default_factory=lambda:SubmoduleParams(required_float_arg=2))if__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.required_int_arg1--myArgs.sub_params.required_float_arg0.1Since the parsing is performed dynamically, the help (-h) always shows the current state of parsing.SetupTo setup install the paiargparse pip package:pipinstallpaiargparseSupported TypesThe following is a list of supported types, and how to access/write their values via the command lineType(example) NameCmd Lineint,str,float,boolmy_primitive--my_primitive 4Enum(str)my_enum--my_enum valueList[int/str/float/bool/Enum(str)]orSet[...]my_primitive_list--my_primitive_list 4 2 1Dict[str/int, int/str/float/bool/Enum]my_primitive_dict--my_primitive_dict first=1 second=2dataclassmy_sub_class--my_subclass ClassName --my_subclass.PARAMETERList[dataclass]my_list--my_list ClassName ClassName --my_list.0.PARAMETER --my_list.1.PARAMETERDict[str, dataclass]my_dict--my_dict first=ClassName second=ClassName --my_dict.first.PARAMETER --my_dict.second.PARAMETERExamplesThe following shows some examples for the different typesPrimitive Types@pai_dataclass@dataclassclassMyArguments:required_int_arg:intoptional_str_arg:str="This is cool stuff"if__name__=="__main__":parser=PAIArgumentParser()# If passing flat, the root arg name is not passed in the shell (see below)parser.add_root_argument("myArgs",MyArguments,flat=True)args=parser.parse_args()Call withpythonmy_program.py--required_int_arg1EnumerationsclassMyEnum(Enum(str)):Tree='tree'Leaf='leaf'@pai_dataclass@dataclassclassMyArguments:enum:MyEnum=MyEnum.Treeenum2:MyEnum=MyEnum.Treeif__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.enumleafLists and Sets of Primitives@pai_dataclass@dataclassclassMyArguments:int_list:List[int]str_set:Set[str]if__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.int_list1234--myArgs.str_setBertSusiDonaldDictionaries of Primitives@pai_dataclass@dataclassclassMyArguments:dict1:Dict[int,str]dict2:Dict[str,str]if__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.dict11=one2=two3=three--myArgs.dict2one=onetwo=twothree=threeChild dataclass@pai_dataclass@dataclassclassMySubArgs:any_arg:int=0@pai_dataclass@dataclassclassMySubArgsAlt(MySubArgs):any_other_arg:int=0@pai_dataclass@dataclassclassMyArguments:sub:MySubArgsif__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.sub.any_arg2pythonmy_program.py--myArgs.subMySubArgsAlt--myArgs.sub.any_other_arg2Lists of dataclasses@pai_dataclass@dataclassclassMySubArgs:any_arg:int=0@pai_dataclass@dataclassclassMySubArgsAlt(MySubArgs):any_other_arg:int=0@pai_dataclass@dataclassclassMyArguments:sub:List[MySubArgs]if__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.subMySubArgsMySubArgsMySubArgsAlt--myArgs.sub.0.any_arg1--myArgs.sub.2.any_other_arg3Dicts of dataclasses@pai_dataclass@dataclassclassMySubArgs:any_arg:int=0@pai_dataclass@dataclassclassMySubArgsAlt(MySubArgs):any_other_arg:int=0@pai_dataclass@dataclassclassMyArguments:sub:dict[str,MySubArgs]if__name__=="__main__":parser=PAIArgumentParser()parser.add_root_argument("myArgs",MyArguments)args=parser.parse_args()Call withpythonmy_program.py--myArgs.subone=MySubArgstwo=MySubArgsthree=MySubArgsAlt--myArgs.sub.one.any_arg1--myArgs.sub.three.any_other_arg3Further examplesHave a look at the varioustestsfor additional examples.Exporting/Importing to dict/jsonSince apai_dataclassinheritsdataclass_json, a dataclass can be writting into a dict and json and read back while preserving the actual types of dataclasses.
This is achieved by an additional__cls__field which is added to eachpai_dataclass. For example:@pai_dataclass@dataclassclassSubmoduleParams:required_float_arg:floatlist_arg:List[int]=field(default_factory=lambda:[1,2])arg_with_custom_help:str=field(default="You can specify a custom help string",metadata=pai_meta(help="Custom help string."))@pai_dataclass@dataclassclassMyArguments:required_int_arg:intoptional_str_arg:str="This is cool stuff"sub_params:SubmoduleParams=field(default_factory=lambda:SubmoduleParams(required_float_arg=2))args=MyArguments(required_int_arg=0,sub_params=SubmoduleParams(required_float_arg=1))print(args.to_dict())# {'required_int_arg': 0, 'optional_str_arg': 'This is cool stuff', 'sub_params': {'required_float_arg': 1, 'list_arg': [1, 2], 'arg_with_custom_help': 'You can specify a custom help string', '__cls__': '__main__:SubmoduleParams'}, '__cls__': '__main__:MyArguments'}assert(MyArguments.from_dict(args.to_dict())==args)# Trueassert(MyArguments.from_json(args.to_json())==args)# TrueMeta-DataSet themetadata-argument offieldtopai_metato enrich the information for the argument parser:argnamedefaultdescriptionexamplehelpNoneThe help string to print when calling -hpai_meta(help="Show the help")separator"."The separator for concatenating hierarchical args, e.g./to useparent/subpai_meta(separator="/")mode"snake"Use"ignore"to ignore this field from the command line, use"flat"to add the argument as a new root argument, i.e. no prefixes will be added, use"snake"for the default mode of snaking all arguments with the separator, use"ssnake"to drop only the parent prefix of one hierachy`pai_meta(mode="ignore")requiredNoneSet toTrueorFalseto force if this parameter must be set from the command line even though a default value is givenpai_meta(required=True)nargs*Override the defaultnargsfield for list, set, or dict fields. The alternative+forces to add at least one element to the list.pai_meta(nargs="+")choicesNoneA list of choices the select from when using lists, sets or subdataclasses. Only the class name must be set in the command line if the class is present in choices (instead of the full path)pai_meta(choices=[SubClass1, SubClass2])disable_subclass_checkFalseBy default changing the type of a class via the command line requires that the new class is a subclass of the type of the dataclass field. Use this flag to disable this check.pai_meta(disable_subclass_check=Trueenforce_choicesNoneOverride the default choices checking. For dataclasses, it is permitted to select dataclasses that are not in choices, for primitive types they must be within choicespai_meta(enforce_choices=True)fix_dcFalseSet to true to forbid overriding of a dataclass via the command line.pai_meta(fix_dc=True)tuple_likeFalseThis enables also to set values similar to tuples by passing a list to the dataclass argument instead of accessing all child arguments. Automatically sets "fix_dc"pai_meta(tuple_like=True)Developmentpaiargparseusesblackcode style.
Install thedevelopment_requirements.txtand runpre-commit installonce to automatically run black on commits.
To upgrade thepre-commitpackages callpre-commit autoupdate.