Package: iterfiles
iterfilesFind and process files in a Pythonic way, without boilerplate code. Implementsfor_each_fileand other common scenarios.>>>fromiterfilesimportfor_each_file>>>for_each_file('example',print,pattern='*/*.txt')This will print all*.txtfile names in all first-level subdirectories ofexample.Let’s say we have following directory structure:example/
    shapes.txt
    aa/
        colors.dat    # not a txt!
        numbers.txt
        pets.txt
    bb/
        names.txt
        cc/
            cars.txtThe output will be:example/aa/numbers.txt
example/aa/pets.txt
example/bb/names.txtFilter directories and files via glob()All syntax ofpathlib.Path.globis supported.Print all*.txtfiles inallsubdirectories:>>>for_each_file('example',print,pattern='**/*.txt')example/shapes.txtexample/aa/numbers.txtexample/aa/pets.txtexample/bb/names.txtexample/bb/cc/cars.txtPrint all*.txtfiles only in a top-level directory:>>>for_each_file('example',print,pattern='*.txt')example/shapes.txtFiles as an iterableIterate overpathlib.Pathobjects:>>>fromiterfilesimportiter_files>>>[x.nameforxiniter_files('example','**/*.txt')]['shapes.txt','numbers.txt','pets.txt','names.txt','cars.txt']…or over text file contents directly, for example combine the first words from each file:>>>fromiterfilesimportiter_texts>>>', '.join(x.split(' ')[0]forxiniter_texts('example',pattern='**/*.txt'))'Square, One, Cat, Alice, Toyota'Pasting all files together into corpusUsefor_each_textto work with plain text contents directly.>>>withopen('corpus.txt','w')ascorpus:...for_each_text('example',corpus.write,pattern='**/*.txt')Convert files from one directory to another directoryLet’s say you want to extract OCR text from a large collection of*.pdfinto*.txtfiles.You have a wonderful functionpdftotext(pdf_filename, txt_filename)from another package,
it does the job well, but how to apply it to a nested directory tree?>>>fromiterfilesimportconvert_files>>>convert_files('input_pdfs','output_txts',pdftotext,pattern='**/*.pdf',rename=lambdap:p.with_suffix('.txt'))That’s all. You’ll have the same directory structure in output, and same file names, but with*.txtsuffix instead of*.pdf.Of course,convert_filescan be used for any kind of conversion.Convert text filesIf both input and output is plain text, useconvert_textsand forget about reading and writing files.
For example, here’s a snippet which transforms all files into uppercase:>>>fromiterfilesimportconvert_texts>>>convert_texts('example','output',str.upper,pattern='**/*.txt')Gotchas and LimitationsAny unhandled exception raised from your function will break the loop.
Make sure to suppress exceptions which are tolerable.
Error handling (such as logging) is out of scope of this package.Collecting list of files according to glob happens (almost) instantly before any processing takes place.
If you add files to directory during long processing, these new files will not be detected on the fly.
If you remove files during processing and before they had a chance to be processed, you will see an error.Only files are considered. Directories are traversed in a search for files; and during conversion,
directories are created when necessary; but that’s it. You can’t do anything custom with directories.Package was not tested with symlinks, and behavior with symlinks is undefined.RequirementsPython 3.6+No dependenciesHistory0.1.0 (2021-02-02)First release on PyPI.