Package: typedpath
typedpathUse typed Python objects to represent files and directories.If you have a project that reads or writes non-trivial structures of directories and files it can be
hard to keep track of which structure they should have.typedpathallow you to declare the
structure using Python objects, and access the data with object methods.Example:importtypedpathastpclassPerson(tp.StructDir):name:tp.TextFileconfig:tp.JSONFileclassDatabase(tp.StructDir):people:tp.DictDir[str,Person]d=Database("database")d.people["alice"].name.write("Alice")d.people["alice"].config.write({"require_authentication":True})d.people["bob"].name.write("Bob")d.people["bob"].config.write({"require_authentication":False})>treedatabase/
database/
└──people├──alice│├──config.json│└──name.txt└──bob├──config.json└──name.txtBuilt-in classestypedpathcomes with a built-in collection of classes for representing directories and files, and
you can create your own to support any additional types you need.TextFileandBytesFileThe two most basic classes included intypedpathareTextFileandBytesFile, which allows you
to read and write basicstrs andbytess. Both come withreadandwritemethods for accessing
data:tf=tp.TextFile("my_text.txt")tf.write("Hello, world!")print(tf.read())bf=tp.BytesFile("my_bytes.bin")bf.write(b"Hello, world!")print(bf.read())StructDirand passing argumentsThe first class provided for composition isStructDir. AStructDirhas a fixed number of
members that may have different types. The members are declared using Python type hints:classPerson(tp.StructDir):name:tp.TextFileconfig:tp.JSONFileclassDatabase(tp.StructDir):people:tp.DictDir[str,Person]All members should be a subclass oftp.TypedPath, and the name of the member becomes the name of
the file on the filesystem.Some members may require configuration that is not (easily) expressible in the type system. For
exampleTextFilecan take anencodingargument. To pass such arguments to the members you can
use thewithargsfunction:classPerson(tp.StructDir):name:tp.TextFile=tp.withargs(encoding="ascii")config:tp.JSONFilep=Person("person")p.name.write("Eve")DictDirand key encodingThe other class provided for composition isDictDir. TheDictDirhas a variable number of
members, but they must all have the same type. As the name implies theDictDirmimics a Pythondict, mapping filenames totypedpathobjects.If aDictDiris created as part of aStructDirthe types of the keys and values are determined
from the type annotations in theStructDir. If you create a free-standingDictDiryou must pass
the type of the keys and values to__init__:people=tp.DictDir("people",str,Person)You can use thevalue_argskeyword-argument to pass arguments to the children:configs=tp.DictDir("configs",str,tp.TextFile,value_args=tp.withargs(encoding="ascii"))By defaultDictDirusesstrto convert the keys into a filename, and the key types__init__to
convert filenames back into key objects. If that does not work for the type you would like to use
for keys, you can implement aKeyCodecfor converting between your keys and strings:fromtypingimportTypeclassBoolKeyCodec(tp.KeyCodec[bool]):defencode(self,key:bool)->str:return"True"ifkeyelse"False"defdecode(self,key_str:str,key_type:Type[bool])->bool:assertissubclass(key_type,bool),key_typematchkey_str:case"True":returnTruecase"False":returnFalseraiseAssertionError(f"Don't know how to interpret{key_str}as a bool")Then register your codec for default use in allDictDirs:tp.add_key_codec(bool,BoolKeyCodec())Or you can set whichKeyCodecto use in just one specificDictDir:bools=tp.DictDir("bools",bool,tp.TextFile,key_codec=BoolKeyCodec())JSON supporttypedpathprovidesJSONFilefor reading and writing using Python's built-injsonmodule:json=tp.JSONFile("example.json")json.write({"is_example":True,"example_names":["alice","bob","eve"],})print(json.read())Pickle supportFor pickling objectstypedpathprovides thePickleFileclass. It takes a parameter for which
type of object to (de)serialize:classA:def__init__(self,value:int)->None:self.value=valuedeftalk(self)->None:print(self.value)pf=tp.PickleFile("a.pickle",A)pf.write(A(13))pf.read().talk()If used with aStructDirthe type hint defines the kind of object to (de)serialize:classMyDir(tp.StructDir):a:tp.PickleFile[A]b:tp.TextFilemd=MyDir("my_dir")md.a.write(A(42))md.a.read().talk()NumPy supporttypedpathalso provides (admittedly limited) classes for reading and writing NumPy
arrays.NpyFileallows you to store a single array in a single file, andNpzFiledoes the same,
but with compression:importnumpyasnpnpy=tp.NpyFile("array.npy")npy.write(np.array([1,2,3]))print(npy.read())npz=tp.NpzFile("array.npz")npz.write(np.array([1,2,3]))print(npz.read())Pandas supporttypedpathhas a couple of classes for reading and writing Pandas data frames, supporting.csv,.featherand.parquetfiles:importpandasaspddf=pd.DataFrame({"a":[1,2,3],"b":[True,False,True],})csv=tp.PandasCsvFile("df.csv")csv.write(df)print(csv.read())feather=tp.PandasFeatherFile("df.feather")feather.write(df)print(feather.read())parquet=tp.PandasParquetFile("df.parquet")parquet.write(df)print(parquet.read())Declaring your own classesObviouslytypeddictonly provides a very small subset of the file types you may want to read and
write. It is expected you will need to write you own classes to support further file types. To
integrate with thetypedpathframework your classes must:If it is a file it should extendtp.TypedFile. If it is a directory it should extendtp.TypedDir.You class should have a static member variable calleddefault_suffixdefining
what the suffix of these objects normally is. It can be empty ("").In simple cases do not define__init__. If you need to define__init__it must have:self; the filesystem path this object represents, with typetp.PathLikeLike; then any generic type arguments this class may need; and finally any keyword arguments your class needs for configuration.To write to a file useself.write_path()to access the path. This method ensures any parent directories are created.To read from a file useself.read_path()to access the path. This method ensures the path already exists.To do anything else with the path, useself.pretty_path().Other than that, add any methods you feel you need to read/write data.Generally the template is:class<YourClassName>(TypedFile):default_suffix="<your suffix>"def__init__(self,path:PathLikeLike,<anygenerictypeargumentsgohere>,*,<kwargsforconfiguration>)->None:super().__init__(path)<initializestuffhere>defwrite(self,...)->None:<writetoself.write_path()here>defread(self)->...:<readfromself.read_path()here>For example, here's the implementation ofTextFile:classTextFile(TypedFile):default_suffix=".txt"def__init__(self,path:PathLikeLike,*,encoding:str="utf-8")->None:super().__init__(path)self._encoding=encodingdefwrite(self,data:str,*,errors:str|None=None,newline:str|None=None,)->int:returnself.write_path().write_text(data,encoding=self._encoding,errors=errors,newline=newline)defread(self,errors:str|None=None)->str:returnself.read_path().read_text(encoding=self._encoding,errors=errors)AndPickleFile(which is a generic class):classPickleFile(TypedFile,Generic[T]):default_suffix=".pickle"def__init__(self,path:PathLikeLike,value_type:Type[T])->None:super().__init__(path)self._value_type=value_typedefwrite(self,data:T,**kwargs:Any)->None:withopen(self.write_path(),"wb")asfp:pickle.dump(data,fp,**kwargs)defread(self,**kwargs:Any)->T:withopen(self.read_path(),"rb")asfp:result:T=pickle.load(fp,**kwargs)origin=get_origin(self._value_type)iforiginisnotNone:assertisinstance(result,origin)returnresult