Package: python-benedict
python-benedictpython-benedict is a dict subclass withkeylist/keypath/keyattrsupport,I/Oshortcuts (base64,cli,csv,html,ini,json,pickle,plist,query-string,toml,xls,xml,yaml) and manyutilities... for humans, obviously.Features100%backward-compatible, you can safely wrap existing dictionaries.NEWKeyattrsupport for get/set items usingkeys as attributes.Keylistsupport usinglist of keysas key.Keypathsupport usingkeypath-separator(dot syntax by default).Keypathlist-indexsupport(also negative)using the standard[n]suffix.NormalizedI/O operationswith most common formats:base64,cli,csv,html,ini,json,pickle,plist,query-string,toml,xls,xml,yaml.MultipleI/O operationsbackends:file-system(read/write),url(read-only),s3(read/write).Manyutilityandparse methodsto retrieve data as needed(check theAPIsection).Welltested. ;)IndexInstallationOptional RequirementsUsageBasicsKeyattrmy_dict.x.y.zKeylistmy_dict["x", "y", "z"]Keypathmy_dict["x.y.z"]Custom keypath separatorChange keypath separatorDisable keypath functionalityList index supportI/OAPIUtility methodsI/O methodsParse methodsTestingLicenseInstallationIf you want to installeverything:Runpip install "python-benedict[all]"alternatively you can install the main package:Runpip install python-benedict, then install only theoptional requirementsyou need.Optional RequirementsHere the hierarchy of possible installation targets available when runningpip install "python-benedict[...]"(each target installs all its sub-targets):[all][io][html][toml][xls][xml][yaml][parse][s3]UsageBasicsbenedictis adictsubclass, so it is possible to use it as a normal dictionary(you can just cast an existing dict).frombenedictimportbenedict# create a new empty instanced=benedict()# or cast an existing dictd=benedict(existing_dict)# or create from data source (filepath, url or data-string) in a supported format:# Base64, CSV, JSON, TOML, XML, YAML, query-stringd=benedict("https://localhost:8000/data.json",format="json")# or in a Django viewparams=benedict(request.GET.items())page=params.get_int("page",1)KeyattrIt is possible to get/set items usingkeys as attributes(dotted notation).d=benedict(keyattr_dynamic=True)# default Falsed.profile.firstname="Fabio"d.profile.lastname="Caccamo"print(d)# -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }By default, if thekeyattr_dynamicis not explicitly set toTrue, this functionality works for get/set only already existing items.Disable keyattr functionalityYou can disable the keyattr functionality passingkeyattr_enabled=Falseoption in the constructor.d=benedict(existing_dict,keyattr_enabled=False)# default Trueor using thegetter/setterproperty.d.keyattr_enabled=FalseDynamic keyattr functionalityYou can enable the dynamic attributes access functionality passingkeyattr_dynamic=Truein the constructor.d=benedict(existing_dict,keyattr_dynamic=True)# default Falseor using thegetter/setterproperty.d.keyattr_dynamic=TrueWarning- even if this feature is very useful, it has some obvious limitations: it works only for string keys that areunprotected(not starting with an_) and that don't clash with the currently supported methods names.KeylistWherever akeyis used, it is possible to use also alist (or a tuple) of keys.d=benedict()# set values by keys listd["profile","firstname"]="Fabio"d["profile","lastname"]="Caccamo"print(d)# -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }print(d["profile"])# -> { "firstname":"Fabio", "lastname":"Caccamo" }# check if keypath exists in dictprint(["profile","lastname"]ind)# -> True# delete value by keys listdeld["profile","lastname"]print(d["profile"])# -> { "firstname":"Fabio" }Keypath.is the default keypath separator.If you cast an existing dict and its keys contain the keypath separator aValueErrorwill be raised.In this case you should use acustom keypath separatorordisable keypath functionality.d=benedict()# set values by keypathd["profile.firstname"]="Fabio"d["profile.lastname"]="Caccamo"print(d)# -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }print(d["profile"])# -> { "firstname":"Fabio", "lastname":"Caccamo" }# check if keypath exists in dictprint("profile.lastname"ind)# -> True# delete value by keypathdeld["profile.lastname"]Custom keypath separatorYou can customize the keypath separator passing thekeypath_separatorargument in the constructor.If you pass an existing dict to the constructor and its keys contain the keypath separator anExceptionwill be raised.d=benedict(existing_dict,keypath_separator="/")Change keypath separatorYou can change thekeypath_separatorat any time using thegetter/setterproperty.If any existing key contains the newkeypath_separatoranExceptionwill be raised.d.keypath_separator="/"Disable keypath functionalityYou can disable the keypath functionality passingkeypath_separator=Noneoption in the constructor.d=benedict(existing_dict,keypath_separator=None)or using thegetter/setterproperty.d.keypath_separator=NoneList index supportList index are supported, keypaths can include indexes(also negative)using[n], to perform any operation very fast:# Eg. get last location cordinates of the first result:loc=d["results[0].locations[-1].coordinates"]lat=loc.get_decimal("latitude")lng=loc.get_decimal("longitude")I/OFor simplifying I/O operations,benedictsupports a variety of input/output methods with most common formats:base64,cli,csv,html,ini,json,pickle,plist,query-string,toml,xls,xml,yaml.Input via constructorIt is possible to create abenedictinstance directly from data-source (filepath,url,s3ordatastring) by passing the data source and the data format (optional, default "json") in the constructor.# filepathd=benedict("/root/data.yml",format="yaml")# urld=benedict("https://localhost:8000/data.xml",format="xml")# s3d=benedict("s3://my-bucket/data.xml",s3_options={"aws_access_key_id":"...","aws_secret_access_key":"..."})# datad=benedict('{"a": 1, "b": 2, "c": 3, "x": 7, "y": 8, "z": 9}')Input methodsAllinputmethods can be accessed as class methods and are prefixed byfrom_*followed by the format name.In allinputmethods, the first argument can represent a source:filepath,url,s3url, ordatastring.Input sourcesAll supported sources (file,url,s3,data) are allowed by default, but in certains situations when the input data comes fromuntrusted sourcesit may be useful to restrict the allowed sources using thesourcesargument:# urld=benedict("https://localhost:8000/data.json",sources=["url"])# -> okd=benedict.from_json("https://localhost:8000/data.json",sources=["url"])# -> ok# s3d=benedict("s3://my-bucket/data.json",sources=["url"])# -> raise ValueErrord=benedict.from_json("s3://my-bucket/data.json",sources=["url"])# -> raise ValueErrorOutput methodsAlloutputmethods can be accessed as instance methods and are prefixed byto_*followed by the format name.In alloutputmethods, iffilepath="..."kwarg is specified, the output will be alsosavedat the specified filepath.Supported formatsHere are the details of the supported formats, operations and extra options docs.formatinputoutputextra options docsbase64:white_check_mark::white_check_mark:-cli:white_check_mark::x:argparsecsv:white_check_mark::white_check_mark:csvhtml:white_check_mark::x:bs4(Beautiful Soup 4)ini:white_check_mark::white_check_mark:configparserjson:white_check_mark::white_check_mark:jsonpickle:white_check_mark::white_check_mark:pickleplist:white_check_mark::white_check_mark:plistlibquery-string:white_check_mark::white_check_mark:-toml:white_check_mark::white_check_mark:tomlxls:white_check_mark::x:openpyxl-xlrdxml:white_check_mark::white_check_mark:xmltodictyaml:white_check_mark::white_check_mark:PyYAMLAPIUtility methodscleanclonedumpfilterfindflattengroupbyinvertitems_sorted_by_keysitems_sorted_by_valueskeypathsmatchmergemovenestremoverenamesearchstandardizesubsetswaptraverseunflattenuniqueI/O methodsfrom_base64from_clifrom_csvfrom_inifrom_htmlfrom_jsonfrom_picklefrom_plistfrom_query_stringfrom_tomlfrom_xlsfrom_xmlfrom_yamlto_base64to_csvto_inito_jsonto_pickleto_plistto_query_stringto_tomlto_xmlto_yamlParse methodsget_boolget_bool_listget_dateget_date_listget_datetimeget_datetime_listget_decimalget_decimal_listget_dictget_emailget_floatget_float_listget_intget_int_listget_listget_list_itemget_phonenumberget_slugget_slug_listget_strget_str_listget_uuidget_uuid_listUtility methodsThese methods are common utilities that will speed up your everyday work.Utilities that accept key argument(s) also support keypath(s).Utilities that return a dictionary always return a newbenedictinstance.clean# Clean the current dict instance removing all empty values: None, "", {}, [], ().# If strings or collections (dict, list, set, tuple) flags are False,# related empty values will not be deleted.d.clean(strings=True,collections=True)clone# Return a clone (deepcopy) of the dict.c=d.clone()dump# Return a readable representation of any dict/list.# This method can be used both as static method or instance method.s=benedict.dump(d.keypaths())print(s)# ord=benedict()print(d.dump())filter# Return a filtered dict using the given predicate function.# Predicate function receives key, value arguments and should return a bool value.predicate=lambdak,v:visnotNonef=d.filter(predicate)find# Return the first match searching for the given keys/keypaths.# If no result found, default value is returned.keys=["a.b.c","m.n.o","x.y.z"]f=d.find(keys,default=0)flatten# Return a new flattened dict using the given separator to join nested dict keys to flatten keypaths.f=d.flatten(separator="_")groupby# Group a list of dicts at key by the value of the given by_key and return a new dict.g=d.groupby("cities",by_key="country_code")invert# Return an inverted dict where values become keys and keys become values.# Since multiple keys could have the same value, each value will be a list of keys.# If flat is True each value will be a single value (use this only if values are unique).i=d.invert(flat=False)items_sorted_by_keys# Return items (key/value list) sorted by keys.# If reverse is True, the list will be reversed.items=d.items_sorted_by_keys(reverse=False)items_sorted_by_values# Return items (key/value list) sorted by values.# If reverse is True, the list will be reversed.items=d.items_sorted_by_values(reverse=False)keypaths# Return a list of all keypaths in the dict.# If indexes is True, the output will include list values indexes.k=d.keypaths(indexes=False)match# Return a list of all values whose keypath matches the given pattern (a regex or string).# If pattern is string, wildcard can be used (eg. [*] can be used to match all list indexes).# If indexes is True, the pattern will be matched also against list values.m=d.match(pattern,indexes=True)merge# Merge one or more dictionary objects into current instance (deepupdate).# Sub-dictionaries keys will be merged together.# If overwrite is False, existing values will not be overwritten.# If concat is True, list values will be concatenated together.d.merge(a,b,c,overwrite=True,concat=False)move# Move an item from key_src to key_dst.# It can be used to rename a key.# If key_dst exists, its value will be overwritten.d.move("a","b",overwrite=True)nest# Nest a list of dicts at the given key and return a new nested list# using the specified keys to establish the correct items hierarchy.d.nest("values",id_key="id",parent_id_key="parent_id",children_key="children")remove# Remove multiple keys from the dict.# It is possible to pass a single key or more keys (as list or *args).d.remove(["firstname","lastname","email"])rename# Rename a dict item key from "key" to "key_new".# If key_new exists, a KeyError will be raised.d.rename("first_name","firstname")search# Search and return a list of items (dict, key, value, ) matching the given query.r=d.search("hello",in_keys=True,in_values=True,exact=False,case_sensitive=False)standardize# Standardize all dict keys, e.g. "Location Latitude" -> "location_latitude".d.standardize()subset# Return a dict subset for the given keys.# It is possible to pass a single key or more keys (as list or *args).s=d.subset(["firstname","lastname","email"])swap# Swap items values at the given keys.d.swap("firstname","lastname")traverse# Traverse a dict passing each item (dict, key, value) to the given callback function.deff(d,key,value):print(f"dict:{d}- key:{key}- value:{value}")d.traverse(f)unflatten# Return a new unflattened dict using the given separator to split dict keys to nested keypaths.u=d.unflatten(separator="_")unique# Remove duplicated values from the dict.d.unique()I/O methodsThese methods are available for input/output operations.from_base64# Try to load/decode a base64 encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to choose the subformat used under the hood:# ('csv', 'json', 'query-string', 'toml', 'xml', 'yaml'), default: 'json'.# It's possible to choose the encoding, default 'utf-8'.# A ValueError is raised in case of failure.d=benedict.from_base64(s,subformat="json",encoding="utf-8",**kwargs)from_cli# Load and decode data from a string of CLI arguments.# ArgumentParser specific options can be passed using kwargs:# https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser# Return a new dict instance. A ValueError is raised in case of failure.d=benedict.from_cli(s,**kwargs)from_csv# Try to load/decode a csv encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to specify the columns list, default: None (in this case the first row values will be used as keys).# It's possible to pass decoder specific options using kwargs:# https://docs.python.org/3/library/csv.html# A ValueError is raised in case of failure.d=benedict.from_csv(s,columns=None,columns_row=True,**kwargs)from_html# Try to load/decode a html data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://beautiful-soup-4.readthedocs.io/# A ValueError is raised in case of failure.d=benedict.from_html(s,**kwargs)from_ini# Try to load/decode a ini encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://docs.python.org/3/library/configparser.html# A ValueError is raised in case of failure.d=benedict.from_ini(s,**kwargs)from_json# Try to load/decode a json encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://docs.python.org/3/library/json.html# A ValueError is raised in case of failure.d=benedict.from_json(s,**kwargs)from_pickle# Try to load/decode a pickle encoded in Base64 format and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://docs.python.org/3/library/pickle.html# A ValueError is raised in case of failure.d=benedict.from_pickle(s,**kwargs)from_plist# Try to load/decode a p-list encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://docs.python.org/3/library/plistlib.html# A ValueError is raised in case of failure.d=benedict.from_plist(s,**kwargs)from_query_string# Try to load/decode a query-string and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# A ValueError is raised in case of failure.d=benedict.from_query_string(s,**kwargs)from_toml# Try to load/decode a toml encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://pypi.org/project/toml/# A ValueError is raised in case of failure.d=benedict.from_toml(s,**kwargs)from_xls# Try to load/decode a xls file (".xls", ".xlsx", ".xlsm") from url, filepath or data-string.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# - https://openpyxl.readthedocs.io/ (for .xlsx and .xlsm files)# - https://pypi.org/project/xlrd/ (for .xls files)# A ValueError is raised in case of failure.d=benedict.from_xls(s,sheet=0,columns=None,columns_row=True,**kwargs)from_xml# Try to load/decode a xml encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://github.com/martinblech/xmltodict# A ValueError is raised in case of failure.d=benedict.from_xml(s,**kwargs)from_yaml# Try to load/decode a yaml encoded data and return it as benedict instance.# Accept as first argument: url, filepath or data-string.# It's possible to pass decoder specific options using kwargs:# https://pyyaml.org/wiki/PyYAMLDocumentation# A ValueError is raised in case of failure.d=benedict.from_yaml(s,**kwargs)to_base64# Return the dict instance encoded in base64 format and optionally save it at the specified 'filepath'.# It's possible to choose the subformat used under the hood:# ('csv', json', 'query-string', 'toml', 'xml', 'yaml'), default: 'json'.# It's possible to choose the encoding, default 'utf-8'.# It's possible to pass decoder specific options using kwargs.# A ValueError is raised in case of failure.s=d.to_base64(subformat="json",encoding="utf-8",**kwargs)to_csv# Return a list of dicts in the current dict encoded in csv format and optionally save it at the specified filepath.# It's possible to specify the key of the item (list of dicts) to encode, default: 'values'.# It's possible to specify the columns list, default: None (in this case the keys of the first item will be used).# A ValueError is raised in case of failure.s=d.to_csv(key="values",columns=None,columns_row=True,**kwargs)to_ini# Return the dict instance encoded in ini format and optionally save it at the specified filepath.# It's possible to pass encoder specific options using kwargs:# https://docs.python.org/3/library/configparser.html# A ValueError is raised in case of failure.s=d.to_ini(**kwargs)to_json# Return the dict instance encoded in json format and optionally save it at the specified filepath.# It's possible to pass encoder specific options using kwargs:# https://docs.python.org/3/library/json.html# A ValueError is raised in case of failure.s=d.to_json(**kwargs)to_pickle# Return the dict instance as pickle encoded in Base64 format and optionally save it at the specified filepath.# The pickle protocol used by default is 2.# It's possible to pass encoder specific options using kwargs:# https://docs.python.org/3/library/pickle.html# A ValueError is raised in case of failure.s=d.to_pickle(**kwargs)to_plist# Return the dict instance encoded in p-list format and optionally save it at the specified filepath.# It's possible to pass encoder specific options using kwargs:# https://docs.python.org/3/library/plistlib.html# A ValueError is raised in case of failure.s=d.to_plist(**kwargs)to_query_string# Return the dict instance as query-string and optionally save it at the specified filepath.# A ValueError is raised in case of failure.s=d.to_query_string(**kwargs)to_toml# Return the dict instance encoded in toml format and optionally save it at the specified filepath.# It's possible to pass encoder specific options using kwargs:# https://pypi.org/project/toml/# A ValueError is raised in case of failure.s=d.to_toml(**kwargs)to_xml# Return the dict instance encoded in xml format and optionally save it at the specified filepath.# It's possible to pass encoder specific options using kwargs:# https://github.com/martinblech/xmltodict# A ValueError is raised in case of failure.s=d.to_xml(**kwargs)to_yaml# Return the dict instance encoded in yaml format.# If filepath option is passed the output will be saved ath# It's possible to pass encoder specific options using kwargs:# https://pyyaml.org/wiki/PyYAMLDocumentation# A ValueError is raised in case of failure.s=d.to_yaml(**kwargs)Parse methodsThese methods are wrappers of thegetmethod, they parse data trying to return it in the expected type.get_bool# Get value by key or keypath trying to return it as bool.# Values like `1`, `true`, `yes`, `on`, `ok` will be returned as `True`.d.get_bool(key,default=False)get_bool_list# Get value by key or keypath trying to return it as list of bool values.# If separator is specified and value is a string it will be splitted.d.get_bool_list(key,default=[],separator=",")get_date# Get value by key or keypath trying to return it as date.# If format is not specified it will be autodetected.# If choices and value is in choices return value otherwise default.d.get_date(key,default=None,format=None,choices=[])get_date_list# Get value by key or keypath trying to return it as list of date values.# If separator is specified and value is a string it will be splitted.d.get_date_list(key,default=[],format=None,separator=",")get_datetime# Get value by key or keypath trying to return it as datetime.# If format is not specified it will be autodetected.# If choices and value is in choices return value otherwise default.d.get_datetime(key,default=None,format=None,choices=[])get_datetime_list# Get value by key or keypath trying to return it as list of datetime values.# If separator is specified and value is a string it will be splitted.d.get_datetime_list(key,default=[],format=None,separator=",")get_decimal# Get value by key or keypath trying to return it as Decimal.# If choices and value is in choices return value otherwise default.d.get_decimal(key,default=Decimal("0.0"),choices=[])get_decimal_list# Get value by key or keypath trying to return it as list of Decimal values.# If separator is specified and value is a string it will be splitted.d.get_decimal_list(key,default=[],separator=",")get_dict# Get value by key or keypath trying to return it as dict.# If value is a json string it will be automatically decoded.d.get_dict(key,default={})get_email# Get email by key or keypath and return it.# If value is blacklisted it will be automatically ignored.# If check_blacklist is False, it will be not ignored even if blacklisted.d.get_email(key,default="",choices=None,check_blacklist=True)get_float# Get value by key or keypath trying to return it as float.# If choices and value is in choices return value otherwise default.d.get_float(key,default=0.0,choices=[])get_float_list# Get value by key or keypath trying to return it as list of float values.# If separator is specified and value is a string it will be splitted.d.get_float_list(key,default=[],separator=",")get_int# Get value by key or keypath trying to return it as int.# If choices and value is in choices return value otherwise default.d.get_int(key,default=0,choices=[])get_int_list# Get value by key or keypath trying to return it as list of int values.# If separator is specified and value is a string it will be splitted.d.get_int_list(key,default=[],separator=",")get_list# Get value by key or keypath trying to return it as list.# If separator is specified and value is a string it will be splitted.d.get_list(key,default=[],separator=",")get_list_item# Get list by key or keypath and return value at the specified index.# If separator is specified and list value is a string it will be splitted.d.get_list_item(key,index=0,default=None,separator=",")get_phonenumber# Get phone number by key or keypath and return a dict with different formats (e164, international, national).# If country code is specified (alpha 2 code), it will be used to parse phone number correctly.d.get_phonenumber(key,country_code=None,default=None)get_slug# Get value by key or keypath trying to return it as slug.# If choices and value is in choices return value otherwise default.d.get_slug(key,default="",choices=[])get_slug_list# Get value by key or keypath trying to return it as list of slug values.# If separator is specified and value is a string it will be splitted.d.get_slug_list(key,default=[],separator=",")get_str# Get value by key or keypath trying to return it as string.# Encoding issues will be automatically fixed.# If choices and value is in choices return value otherwise default.d.get_str(key,default="",choices=[])get_str_list# Get value by key or keypath trying to return it as list of str values.# If separator is specified and value is a string it will be splitted.d.get_str_list(key,default=[],separator=",")get_uuid# Get value by key or keypath trying to return it as valid uuid.# If choices and value is in choices return value otherwise default.d.get_uuid(key,default="",choices=[])get_uuid_list# Get value by key or keypath trying to return it as list of valid uuid values.# If separator is specified and value is a string it will be splitted.d.get_uuid_list(key,default=[],separator=",")Testing# clone repositorygitclonehttps://github.com/fabiocaccamo/python-benedict.git&&cdpython-benedict# create virtualenv and activate itpython-mvenvvenv&&.venv/bin/activate# upgrade pippython-mpipinstall--upgradepip# install requirementspipinstall-rrequirements.txt-rrequirements-test.txt# install pre-commit to run formatters and linterspre-commitinstall--install-hooks# run tests using toxtox# or run tests using unittestpython-munittestLicenseReleased underMIT License.Supporting:star: Star this project onGitHub:octocat: Follow me onGitHub:blue_heart: Follow me onTwitter:moneybag: Sponsor me onGithubSee alsopython-fontbro- 🧢 friendly font operations on top offontTools.python-fsutil- 🧟‍♂️ high-level file-system operations for lazy devs.