Package: parser-binding
DocumentationTranslated from中文文档byGPT 3.5A tool for quickly binding dataclass classes to ArgumentParser to achieve rapid customization of command-line parameters. This binding supports most parameters of ArgumentParser, such as help documentation, whether it is a required parameter, multiple values, etc., as detailed below. At the same time, this tool supports Python's type hints as much as possible.Quick StartPython version:>=3.7pip install parser-bindingdemo.py：fromdataclassesimportdataclass,fieldfromenumimportEnumfromparser_bindingimportBindingParser,FieldclassLogLevel(Enum):DEBUG='debug'INFO='info'WARNING='warning'ERROR='error'@dataclassclassTestOptions:input_file:str=Field(default=None,aliases=['i'],help='The input file to read.')workers:int=1logging_level:LogLevel=LogLevel.WARNINGverbose:bool=Falseif__name__=='__main__':parser=BindingParser(TestOptions)options=parser.parse_into_dataclasses((TestOptions,))print(options)Executepython demo.py -h, you will get the following output:usage:demo.py[-h][-iINPUT_FILE][--workersWORKERS][--logging-level{debug,info,warning,error}][--with-verbose]optionalarguments:-h,--helpshowthishelpmessageandexit-iINPUT_FILE,--input-fileINPUT_FILE,--input_fileINPUT_FILETheinputfiletoread.Optional.Default`None`.--workersWORKERSOptional.Default`1`.--logging-level{debug,info,warning,error},--logging_level{debug,info,warning,error}Optional.Default`warning`.--with-verbose,--with_verboseOptional.Default`False`asverbosedisabled.Executepython demo.py -i ./test.txt --workers 2 --logging-level debug --with-verbose, and you will automatically get an instance of the dataclass with corresponding parameters read from the command-line:_MergedDataClass@1704877855657(input_file='./test.txt',workers=2,logging_level=<LogLevel.DEBUG:'debug'>,verbose=True)Formatting GuideCommand-line parameter names will be formatted into three types:--xxx-yyy, when the class property name contains_, it will be converted to a command-line parameter name starting with--and separated by-.--xxx_yyy, when the class property name contains_, it will be converted to a command-line parameter name starting with--and separated by_as an alternative.-x, shortcut option support, when the property name is a single character or contains a single character in the specified alias, it will use the shortcut option-as one of the parameters.Currently, it supports most types in Python:Basic Typesint: Integer type, automatically converts command-line parameters to integers.float: Floating-point type, automatically converts command-line parameters to floating-point numbers.str: String type, automatically converts command-line parameters to strings.bytes: Byte type, supports direct conversion of strings in UTF-8 encoding, does not support other complex data, if other conversions are needed, it can be implemented by specifying the type.bool: Boolean type, automatically adds a switch option to the command-line, the switch name is influenced by the default value; when the default value isTrue, the switch name is formatted as--without-xxx-yyyand--without_xxx_yyy, when the default value isFalse, the switch name is formatted as starting with--with.Note: Boolean value types will only be formatted as switches and do not need to pass parameters on the command-line. Pay attention to the change in parameter names caused by changes in default values.Types with Generic Annotations: When using generic annotations, please explicitly specify the type inside the generic. Complex types such asUnion[str, int]are not directly supported：Optional，Used to define optional properties, provide a default value when using this annotation.For all properties with default values, the corresponding parameters on the command-line will be treated as optional parameters and provide default values.List/list: When using theListannotation, please specify the type of the list elements explicitly, such asList[int]. When using this type, the command-line parameter corresponding to it will be set as a multi-value type, and spaces will be used as the default separator for each value, such as--multiple 1 2 3. The explicit element type will format each value passed in as the target type, that is, each element in theList[int]property will be of typeint, and not the result ofList[str].
If using thelistannotation, there is no additional type mapping for the passed elements by default. If additional type mapping is needed, please specify the type mapping using the type parameter (as shown in the example later).Tuple/tuple: Same asListorlist, the difference lies in the final collection class output.Set/set: Same as above.Deueue/dequeue: Same as above.Queue: Same as above. These collection classes default to using a space as the separator between multiple values. If a custom separator is needed, it should be specified usingsep(as shown in the subsequent example).Dict/dict: When usingDictas the property type, please explicitly specify the key and value types. If usingdictas an annotation, no mapping will be done for the key/value types. By default, the command-line parameters passed in will be treated as a JSON-string for parsing. If it cannot be parsed, it will try to parse it as a JSON file (in this case, pass in a JSON file directory). If both methods fail, it will result in failure.Enum: When using an enumeration class, the command-line will construct an optional value parameter, and the input will be limited to a fixed list of values. The command-line will pass in a string, but during parsing, it will get a corresponding enumeration and provide it to the dataclass instance initialization.Literal: Requirespython >= 3.8support, equivalent toEnum, but does not produce an enumeration.Complex TypesFile type: For ease of reading from the command-line or setting output devices, you can quickly open files using this method to reduce the overhead of callingopenin the code.Nested or unknown types: To further expand the scenarios of complex types, support custom type conversion for command-line passing values through theFieldandtypeparameters.Usage ExamplesBasic Type Scenariofromdataclassesimportdataclassfromparser_bindingimportBindingParser@dataclassclassTestOption():required_float:floata:int=0string_type:str=Noneswitch:bool=Falsebytes_data:bytes=Noneparser=BindingParser(TestOption)options=parser.parse_into_dataclasses((TestOption,))print(options)python3 basic-demo.py --required-float 0.1 -a 10 --string-type 'Hello Parser!' --with-switch --bytes-data 'Hello Parser!'Result:_MergedDataClass@1704880154551(required_float=0.1, a=10, string_type='Hello Parser!', switch=True, bytes_data=b'Hello Parser!')You can observe that--required-floatis a required parameter; otherwise, parsing will fail.For parameters without default values,parser-bindingwill consider them as required parameters; otherwise, provide default values.typingAnnotation TypesfromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportBindingParser@dataclassclassTestOption:data:List[int]=Noneitems:list=Noneparser=BindingParser(TestOption)opt=parser.parse_into_dataclasses((TestOption,))print(opt)Executepython list-demo.py --data 1 2 3 --items 1 2 3to parse and get the result_MergedDataClass@1704880514613(data=[1, 2, 3], items=['1', '2', '3'])。From the above difference, if using theList[int]annotation, each element in the list will be converted to an integer, otherwise, no conversion will be performed.
If conversion is required, thetypeshould be specified, as shown below, changing theitemsannotation to:items:list=Field(default=None,type=int)Executepython list-demo.py --data 1 2 3 --items 1 2 3, and the parsed result will be_MergedDataClass@1704881428343(data=[1, 2, 3], items=[1, 2, 3]), where the type is now correctly converted.Forset,tuple,deque,queue, and other collection types, the effect is the same.Note: If the property is annotated withlist,tuple,set,queue, etc., the command-line will automatically convert the corresponding parameter to a multi-value parameter separated by spaces. At this time, if you specify type usingField, type should correspond to the class of the elements, not the final property collection type. For example, in the aboveField(default=None, type=int),typespecifies the elementtypeasint, i.e., the property type isList[int].Given the default behavior of command-line multi-value parameters, which default to being separated by spaces, further support for more separators can be achieved throughFiledby specifyingsep.
For example:items: tuple = Field(sep=',', default=None, type=int), will result in aTuple[int]parsing result, and the input parameters will be separated by,. In this case, you should run the following command:python list-demo.py --data 1 2 3 --items 1,2,3, and the parsing result will be_MergedDataClass@1704893904876(data=(1, 2, 3), items=(1, 2, 3)).。parser-bindingalso supports dictionary types, used to support JSON-like formats. When usingDictas the type annotation, you need to specify the key and value types so that they can be correctly converted. If the JSON string contains key-value pairs of multiple types, you can directly usedictfor annotation, as shown in the following example:fromdataclassesimportdataclassfromtypingimportDictfromparser_bindingimportBindingParser,Field@dataclassclassTestOption:data:Dict[str,int]=Noneitems:dict=Noneparser=BindingParser(TestOption)opt=parser.parse_into_dataclasses((TestOption,),)print(opt)Executepython dict-demo.py --data '{"1": "1", "2": "2"}' --items '{"1": "1", "2": "2"}', and the parsing result will be_MergedDataClass@1704944849798(data={'1': 1, '2': 2}, items={'1': '1', '2': '2'})。From the above parsing result, when usingdictfor annotation, the key-type and value-type will remain consistent with the original JSON, but when usingDictand specifying types, the key/value will be converted to the corresponding types.Enum TypesEnum types allow the command-line to define an optional value parameter, restricting the input to a fixed list of values, making the selection from the command-line effective.As shown in the example below:fromdataclassesimportdataclassfromenumimportEnumfromparser_bindingimportparse_argsclassMode(Enum):train:str='train'eval:str='eval'@dataclassclassTestOption:mode:Mode=Mode.trainargs=parse_args((TestOption,))print(args.mode)At this point, check the usage with usage:enum-demo.py [-h] [--mode {train,eval}], indicating that only the valuestrainandevalare supported.Enum classes are equivalent in effect to annotations likeLiteral。Complex TypeTo further support more scenarios, such as nested collections, multi-type inference, file reading and writing, etc.,parser-bindingsupports implementation throughFieldcombined withtype.We allow the use ofOptionalorUnion[x, None]to declare optional properties, but we do not yet support multi-type inference, such asUnion[str, int], or nested types likeList[List[int]].
These types are considered complex types and require additional type configuration!By default, for complex types when notypeconfiguration is specified, we won't perform any processing. Consequently, the parameter values read will be of typestr. This may lead to unexpected results, so please use caution.Recommended: Provide thehelpargument for complex types to help the command-line understand how to pass values.Nested TypesIn complex scenarios, we useList[List[int]]as an example for parsing, as shown in the following code:fromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportparse_args@dataclassclassTestOption:data:List[List[int]]=Noneargs=parse_args((TestOption,))print(args)At this time, thedataproperty is not given atypeargument and is treated as a complex type, resulting in the following warning:UserWarning: The filed "data" is complex but there is no type specified, this could make an error.At this time, directly passing values from the command-line will result in the result:_MergedDataClass@1704955154188(data='1,2,3'), indicating thatdatadoes not match the expected typeList[List[int]].Therefore, a reasonable practice should be:fromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportField,parse_args@dataclassclassTestOption:data:List[List[int]]=Field(default=None,type=lambdax:list(map(lambdax:list(map(int,x.split(':'))),x.split(','))))args=parse_args((TestOption,))print(args)Executingpython complex-demo.py --data 1:4:5,2,3will result in_MergedDataClass@1704955285471(data=[[1, 4, 5], [2], [3]]), which aligns with the expected type annotations.File TypeTo facilitate the input of files from the command line,parser-bindingsupports simple text reading and writing methods, making it easy to obtain file instances directly from a data class without the need foropenmethod calls. Here is an example:fromdataclassesimportdataclassfromtypingimportIO,Iterable,TextIOfromparser_bindingimportField,parse_args@dataclassclassTestOption:in_file:Iterable[str]=Field(default=None,file=True,file_mode='r',file_encoding='utf-8')# in_file: IO[str] = Field(default=None, file='r', file_encoding='utf-8')out_file:IO[str]=Field(default='-',file_mode='w',file_encoding='utf-8')args=parse_args((TestOption,))forlinargs.in_file:print('o',l.strip(),sep='\t',file=args.out_file)For file handling, there are two annotation methods, as seen in the example above with thein_fileproperty. The difference lies in whether thefileargument needs to be specified asTruewhen using theIOannotation. Otherwise, it must be specified asTrue, or else it will be treated as a complex type.Suppose the content of the existingtest-file.txtis:a
b
cBy executingpython file-demo.py --in-file test-file.txt, the above code will produce the output:o       a
o       b
o       cFor file types, when the default value is'-', it is automatically set tostdinorstdout, depending on the file's read/write opening mode.File types support plain text files. If the file name ends with.gzand the property type annotation is a genericstr, such asIO[str]/TextIO, it will open the file usinggzip.open()+rtmode.使用文档EN-DOC用于快速将dataclass类绑定到ArgumentParser，以实现快速定制化command-line参数。该绑定支持argument-parser大部分参数，如help文档，是否必须参数、多选值等，具体见下。同时，该工具尽可能地支持了Python内置的类型提示。快速使用Python版本：>=3.7pip install parser-bindingdemo.py：fromdataclassesimportdataclass,fieldfromenumimportEnumfromparser_bindingimportBindingParser,FieldclassLogLevel(Enum):DEBUG='debug'INFO='info'WARNING='warning'ERROR='error'@dataclassclassTestOptions:input_file:str=Field(default=None,aliases=['i'],help='The input file to read.')workers:int=1logging_level:LogLevel=LogLevel.WARNINGverbose:bool=Falseif__name__=='__main__':parser=BindingParser(TestOptions)options=parser.parse_into_dataclasses((TestOptions,))print(options)执行python demo.py -h，得到如下输出：usage:demo.py[-h][-iINPUT_FILE][--workersWORKERS][--logging-level{debug,info,warning,error}][--with-verbose]optionalarguments:-h,--helpshowthishelpmessageandexit-iINPUT_FILE,--input-fileINPUT_FILE,--input_fileINPUT_FILETheinputfiletoread.Optional.Default`None`.--workersWORKERSOptional.Default`1`.--logging-level{debug,info,warning,error},--logging_level{debug,info,warning,error}Optional.Default`warning`.--with-verbose,--with_verboseOptional.Default`False`asverbosedisabled.执行python demo.py -i ./test.txt --workers 2 --logging-level debug --with-verbose，将自动得到一个从command-line读取对应参数的dataclass实例：_MergedDataClass@1704877855657(input_file='./test.txt',workers=2,logging_level=<LogLevel.DEBUG:'debug'>,verbose=True)格式化说明command-line参数名将统一格式化为三种类型：--xxx-yyy，当类属性名带有_时，将转换为使用--开头，-分割的command-line参数名；--xxx_yyy, 当类属性名带有_时，将转换为使用--开头，_分割的command-line参数名作为备选；-x，快捷选项支持，当属性名为单字符或指定的别名中含有单字符名称时，将使用快捷选项-开头，作为参数之一；当前支持python中大部分类型：基本类型int，整数类型，将自动将command-line的参数转化为整数；float，浮点数类型，自动将command-line参数转化为浮点数；str，字符串类型，自动将command-line参数转化为字符串；bytes，字节类型，默认支持UTF8编码方式的字符串直接转换，其他复杂数据暂不支持，若需其他转换，可通过指定type实现；bool，布尔类型，自动为command-line添加开关选项，开关名称受默认值影响；当默认值为True时，开关名称为--without-xxx-yyy或--without_xxx_yyy，当默认值为False时，则开关名称被格式化为以--with开头。注意：布尔值类型仅会被格式化为开关，无需在command-line中传入参数，且注意默认值变化所带来的参数名变化。带有范型注释的类型，在使用范型注释时，请明确指出范型内的类型，暂不直接支持Union[str, int]此类复杂类型：Optional，用于定义可选属性，使用该注释时，请为属性提供默认值。对于一切含有默认值的属性，command-line对应的参数将被视为可选参数，并提供默认值。List/list，在使用List注释时，请明确指明列表元素的类型，如List[int]，使用该类型时，自动将command-line对应的参数设为多值类型，并默认使用空格表示每个值，如--multiple 1 2 3。明确的元素类型，将使得传入的每个值进行目标类型的格式化，即List[int]属性中的每个元素为int，而不会得到List[str]的结果；
若使用list注释，默认不对传入的元素进行额外的类型映射，若需要额外的类型映射，请指明type映射方法（使用可见后续介绍）；Tuple/tuple，同List或list，区别在于输出的最终集合类不同；Set/set，同上；Deueue/dequeue，同上；Queue，同上；上述集合类，均默认以空格作为多个值之间的分割，若需自定义分割符，需要使用sep指定，可参考后续实例使用方式。Dict/dict，当使用Dict作为属性类型时，请明确指出key与value类型；若使用dict作为注释，则不对key/value类型做任何映射；默认情况下，command-line传入的参数，将被视为一个JSON-string解析，若无法解析，则尝试以JSON文件方式解析（此时传入一个JSON文件目录）；若两种方式均无法解析，则会导致失败；Enum枚举，使用枚举类时，将使得command-line构造一个可选值的参数，并且将输入限定在固定的值列表内；command-line将传入字符串，但解析时将得到一个对应的枚举并提供给dataclass实例初始化；Literal，需要python >= 3.8版本支持，与Enum等效，但不产出枚举；复杂类型文件类型，为便于从command-line中读取或设置输出设备，可以通过该方式快速打开文件，减少编码open的开销；嵌套或未知类型，为进一步扩展复杂类型的场景，支持自定义type对command-line传值进行转换；使用实例基本类型场景fromdataclassesimportdataclassfromparser_bindingimportBindingParser@dataclassclassTestOption():required_float:floata:int=0string_type:str=Noneswitch:bool=Falsebytes_data:bytes=Noneparser=BindingParser(TestOption)options=parser.parse_into_dataclasses((TestOption,))print(options)python3 basic-demo.py --required-float 0.1 -a 10 --string-type 'Hello Parser!' --with-switch --bytes-data 'Hello Parser!'结果：_MergedDataClass@1704880154551(required_float=0.1, a=10, string_type='Hello Parser!', switch=True, bytes_data=b'Hello Parser!')通过-h可观察到，此时--required-float为必传参数，否则将解析失败；对不不提供默认值的参数，parser-binding将视为必传参数处理，否则，请提供默认值。typing注释类型fromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportBindingParser@dataclassclassTestOption:data:List[int]=Noneitems:list=Noneparser=BindingParser(TestOption)opt=parser.parse_into_dataclasses((TestOption,))print(opt)执行python list-demo.py --data 1 2 3 --items 1 2 3将解析得到_MergedDataClass@1704880514613(data=[1, 2, 3], items=['1', '2', '3'])。从上述区别可以看出，若使用List[int]注释，则列表中的每个元素将被转换为整数，否则将不进行任何转换。
若需要进行转换，需要指定type，如下，将items注释更换为：items:list=Field(default=None,type=int)再次执行python list-demo.py --data 1 2 3 --items 1 2 3，解析得到结果MergedDataClass@1704881428343(data=[1, 2, 3], items=[1, 2, 3])，此时类型将进行对应的转换。对于set、tuple、deque、queue等集合类型，具有相同效果。注意事项：若属性被list、tuple、set、queue等集合注释时，command-line将自动将对应的参数转化为以空格分割的多值参数。此时，若通过Field重新指定type时，type应该对应为元素的类，而非最终属性的集合类型。如上述Field(default=None, type=int)中，type指定元素类型为int，即属性类型为List[int]。鉴于默认command-line多值参数都默认以空格区分，为进一步扩展支持更多分割符，可通过Filed指定sep来定义。
如：items: tuple = Field(sep=',', default=None, type=int)，将会得到Tuple[int]的解析结果，并且传入参数使用,分割。此时，应当执行以下命令：python list-demo.py --data 1 2 3 --items 1,2,3，解析结果为：_MergedDataClass@1704893904876(data=(1, 2, 3), items=(1, 2, 3))。parser-binding额外支持字典类型，用于支持类似JSON的格式。当使用Dict作为类型注释时，需要为其分配Key-Type与Value-Type，以便于能够正确的转换。若JSON字符串中，存在多样类型的key-value，可直接使用dict进行注释，如下实例：fromdataclassesimportdataclassfromtypingimportDictfromparser_bindingimportBindingParser,Field@dataclassclassTestOption:data:Dict[str,int]=Noneitems:dict=Noneparser=BindingParser(TestOption)opt=parser.parse_into_dataclasses((TestOption,),)print(opt)执行python dict-demo.py --data '{"1": "1", "2": "2"}' --items '{"1": "1", "2": "2"}'，得到解析结果为：_MergedDataClass@1704944849798(data={'1': 1, '2': 2}, items={'1': '1', '2': '2'})。从上述解析结果可以看出，当使用dict进行注释时，key-type与value-type将与原JSON保持一致，但当使用Dict并指明类型时，key/value将进行对应的类型转换。枚举类型枚举类型将使得command-line定义一个可选值的参数，使得输入固定在某个集合内选择方为有效参数。如以下示例：fromdataclassesimportdataclassfromenumimportEnumfromparser_bindingimportparse_argsclassMode(Enum):train:str='train'eval:str='eval'@dataclassclassTestOption:mode:Mode=Mode.trainargs=parse_args((TestOption,))print(args.mode)此时，查看usage为usage: enum-demo.py [-h] [--mode {train,eval}]，即只支持train、eval二者之一的值；枚举类在效果上与Literal注释等效。复杂类型场景为进一步支持更多的场景，如嵌套集合，多类型推断，文件读写等场景，parser-binding支持通过Field结合type的方式实现。我们允许使用Optional或Union[x, None]的方式，用于声明可选属性，但尚不支持多类型推断，如Union[str, int]，或嵌套类型List[List[int]]，这些类型都将被视为复杂类型，且需要额外提供type属性，用于类型转换。默认情况下，对于复杂类型且未指定type配置时，我们不会进行任何处理，因此读取的参数值是str类型，这可能会导致非预期的结果，请慎重！推荐：推荐为复杂类型提供help属性，以帮助command-line理解应该如何传值。嵌套类型复杂场景下，我们以List[List[int]]为示例进行解析，如下代码：fromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportparse_args@dataclassclassTestOption:data:List[List[int]]=Noneargs=parse_args((TestOption,))print(args)此时，属性data并未给定type属性，并且被视为复杂类型，将得到如下警告：UserWarning: The filed "data" is complex but there is no type specified, this could make an error.此时，直接通过command-line传值，将会得到结果：_MergedDataClass@1704955154188(data='1,2,3')，可见data与预期的List[List[int]]类型不符。因此，合理的实践应为：fromdataclassesimportdataclassfromtypingimportListfromparser_bindingimportField,parse_args@dataclassclassTestOption:data:List[List[int]]=Field(default=None,type=lambdax:list(map(lambdax:list(map(int,x.split(':'))),x.split(','))))args=parse_args((TestOption,))print(args)执行python complex-demo.py --data 1:4:5,2,3，将会得到_MergedDataClass@1704955285471(data=[[1, 4, 5], [2], [3]])的结果，符合类型注释预期。文件类型为便于command-line传入文件，parser-binding支持了简单的文本读、写方式，便于从dataclass中直接拿到文件实例，而避免open方法调用，示例如下：fromdataclassesimportdataclassfromtypingimportIO,Iterable,TextIOfromparser_bindingimportField,parse_args@dataclassclassTestOption:in_file:Iterable[str]=Field(default=None,file=True,file_mode='r',file_encoding='utf-8')# in_file: IO[str] = Field(default=None, file='r', file_encoding='utf-8')out_file:IO[str]=Field(default='-',file_mode='w',file_encoding='utf-8')args=parse_args((TestOption,))forlinargs.in_file:print('o',l.strip(),sep='\t',file=args.out_file)对于文件，存在两种注释方式，如上述示例中的in_file属性定义，区别在于，若通过IO注释属性，则可以不指明file为True，否则必须指定file为True，
不然则视为复杂类型处理。现有test-file.txt内容为：a
b
c通过执行python file-demo.py --in-file test-file.txt，上述代码得到输出为：o       a
o       b
o       c对于文件类型，当默认值为'-'时，则自动设默认值为stdin或stdout，这取决于文件的读/写的打开方式。文件类型支持纯文本文件，若文件名以.gz结尾，且属性类型注释为str的范型，如IO[str]/TextIO，则会以gzip.open()+rt的模式打开文件。