Package: vistir
🐉 InstallationInstall fromPyPI:$ pipenv install vistirInstall fromGithub:$ pipenv install -e git+https://github.com/sarugaku/vistir.git#egg=vistir🐉 Summaryvistiris a library full of utility functions designed to make life easier. Here are
some of the places where these functions are used:pipenvrequirementslibpip-toolspassapythonfinder🐉 UsageImporting a utilityYou can import utilities directly fromvistir:fromvistirimportcdwithcd('/path/to/somedir'):do_stuff_in('somedir')🐉 Functionalityvistirprovides several categories of functionality, including:Compatibility ShimsContext ManagersMiscellaneous UtilitiesPath Utilities🐉 Context Managersvistirprovides the following context managers as utility contexts:vistir.contextmanagers.atomic_open_for_writevistir.contextmanagers.cdvistir.contextmanagers.open_filevistir.contextmanagers.replaced_streamvistir.contextmanagers.replaced_streamsvistir.contextmanagers.temp_environvistir.contextmanagers.temp_pathatomic_open_for_writeThis context manager ensures that a file only gets overwritten if the contents can be
successfully written in its place.  If you open a file for writing and then fail in the
middle under normal circumstances, your original file is already gone.>>>fn="test_file.txt">>>withopen(fn,"w")asfh:fh.write("this is some test text")>>>read_test_file()thisissometesttext>>>defraise_exception_while_writing(filename):withvistir.contextmanagers.atomic_open_for_write(filename)asfh:fh.write("Overwriting all the text from before with even newer text")raiseRuntimeError("But did it get overwritten now?")>>>raise_exception_while_writing(fn)Traceback(mostrecentcalllast):...RuntimeError:Butdiditgetoverwrittennow?>>>read_test_file()thisissometesttextcdA context manager for temporarily changing the working directory.>>>os.path.abspath(os.curdir)'/tmp/test'>>>withvistir.contextmanagers.cd('/tmp/vistir_test'):print(os.path.abspath(os.curdir))/tmp/vistir_testopen_fileA context manager for streaming file contents, either local or remote. It is recommended
to pair this with an iterator which employs a sensible chunk size.>>>filecontents=b""withvistir.contextmanagers.open_file("https://norvig.com/big.txt")asfp:forchunkiniter(lambda:fp.read(16384),b""):filecontents.append(chunk)>>>importio>>>importshutil>>>filecontents=io.BytesIO(b"")>>>withvistir.contextmanagers.open_file("https://norvig.com/big.txt")asfp:shutil.copyfileobj(fp,filecontents)replaced_streamA context manager to temporarily swap outstream_namewith a stream wrapper.  This will
capture the stream output and prevent it from being written as normal.>>>orig_stdout=sys.stdout>>>withreplaced_stream("stdout")asstdout:...sys.stdout.write("hello")...assertstdout.getvalue()=="hello"...assertorig_stdout.getvalue()!="hello">>>sys.stdout.write("hello")'hello'replaced_streamsTemporarily replaces bothsys.stdoutandsys.stderrand captures anything written
to these respective targets.>>>importsys>>>withvistir.contextmanagers.replaced_streams()asstreams:>>>stdout,stderr=streams>>>sys.stderr.write("test")>>>sys.stdout.write("hello")>>>assertstdout.getvalue()=="hello">>>assertstderr.getvalue()=="test">>>stdout.getvalue()'hello'>>>stderr.getvalue()'test'spinnerA context manager for wrapping some actions with a threaded, interrupt-safe spinner. The
spinner is fully compatible with all terminals (you can usebouncingBaron non-utf8
terminals) and will allow you to update the text of the spinner itself by simply settingspinner.textor write lines to the screen above the spinner by usingspinner.write(line). Success text can be indicated usingspinner.ok("Text")and
failure text can be indicated withspinner.fail("Failtext").>>>lines=["a","b"]>>>withvistir.contextmanagers.spinner(spinner_name="dots",text="Running...",handler_map={},nospin=False)assp:forlineinlines:sp.write(line+"\n")whilesome_variable=some_queue.pop():sp.text="Consuming item:%s"%some_variableifsuccess_condition:sp.ok("Succeeded!")else:sp.fail("Failed!")temp_environSets a temporary environment context to freely manipulateos.environwhich will
be reset upon exiting the context.>>>os.environ['MY_KEY']="test">>>os.environ['MY_KEY']'test'>>>withvistir.contextmanagers.temp_environ():os.environ['MY_KEY']="another thing"print("New key:%s"%os.environ['MY_KEY'])Newkey:anotherthing>>>os.environ['MY_KEY']'test'temp_pathSets a temporary environment context to freely manipulatesys.pathwhich will
be reset upon exiting the context.>>>path_from_virtualenv=load_path("/path/to/venv/bin/python")>>>print(sys.path)['/home/user/.pyenv/versions/3.7.0/bin','/home/user/.pyenv/versions/3.7.0/lib/python37.zip','/home/user/.pyenv/versions/3.7.0/lib/python3.7','/home/user/.pyenv/versions/3.7.0/lib/python3.7/lib-dynload','/home/user/.pyenv/versions/3.7.0/lib/python3.7/site-packages']>>>withtemp_path():sys.path=path_from_virtualenv# Running in the context of the path aboverun(["pip","install","stuff"])>>>print(sys.path)['/home/user/.pyenv/versions/3.7.0/bin','/home/user/.pyenv/versions/3.7.0/lib/python37.zip','/home/user/.pyenv/versions/3.7.0/lib/python3.7','/home/user/.pyenv/versions/3.7.0/lib/python3.7/lib-dynload','/home/user/.pyenv/versions/3.7.0/lib/python3.7/site-packages']🐉 Cursor UtilitiesThe following Cursor utilities are available to manipulate the console cursor:vistir.cursor.hide_cursorvistir.cursor.show_cursorhide_cursorHide the console cursor in the given stream.>>>vistir.cursor.hide_cursor(stream=sys.stdout)show_cursorShow the console cursor in the given stream.>>>vistir.cursor.show_cursor(stream=sys.stdout)🐉 Miscellaneous UtilitiesThe following Miscellaneous utilities are available as helper methods:vistir.misc.shell_escapevistir.misc.unnestvistir.misc.runvistir.misc.load_pathvistir.misc.partialclassvistir.misc.to_textvistir.misc.to_bytesvistir.misc.takevistir.misc.decode_for_outputvistir.misc.get_canonical_encoding_namevistir.misc.get_wrapped_streamvistir.misc.StreamWrappervistir.misc.get_text_streamvistir.misc.replace_with_text_streamvistir.misc.get_text_stdinvistir.misc.get_text_stdoutvistir.misc.get_text_stderrvistir.misc.echoshell_escapeEscapes a string for use as shell input when passingshell=Truetoos.Popen.>>>vistir.misc.shell_escape("/tmp/test/test script.py hello")'/tmp/test/test script.py hello'unnestUnnests nested iterables into a flattened one.>>>nested_iterable=(1234,(3456,4398345,(234234)),(2396,(23895750,9283798,29384,(289375983275,293759,2347,(2098,7987,27599)))))>>>list(vistir.misc.unnest(nested_iterable))[1234,3456,4398345,234234,2396,23895750,9283798,29384,289375983275,293759,2347,2098,7987,27599]dedupDeduplicates an iterable (like aset, but preserving order).>>>iterable=["repeatedval","uniqueval","repeatedval","anotherval","somethingelse"]>>>list(vistir.misc.dedup(iterable))['repeatedval','uniqueval','anotherval','somethingelse']runRuns the given command usingsubprocess.Popenand passing sane defaults.>>>out,err=vistir.run(["cat","/proc/version"])>>>out'Linux version 4.15.0-27-generic (buildd@lgw01-amd64-044) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #29-Ubuntu SMP Wed Jul 11 08:21:57 UTC 2018'load_pathLoad thesys.pathfrom the given python executable’s environment as json.>>>load_path("/home/user/.virtualenvs/requirementslib-5MhGuG3C/bin/python")['','/home/user/.virtualenvs/requirementslib-5MhGuG3C/lib/python37.zip','/home/user/.virtualenvs/requirementslib-5MhGuG3C/lib/python3.7','/home/user/.virtualenvs/requirementslib-5MhGuG3C/lib/python3.7/lib-dynload','/home/user/.pyenv/versions/3.7.0/lib/python3.7','/home/user/.virtualenvs/requirementslib-5MhGuG3C/lib/python3.7/site-packages','/home/user/git/requirementslib/src']partialclassCreate a partially instantiated class.>>>source=partialclass(Source,url="https://pypi.org/simple")>>>new_source=source(name="pypi")>>>new_source<__main__.Sourceobjectat0x7f23af189b38>>>>new_source.__dict__{'url':'https://pypi.org/simple','verify_ssl':True,'name':'pypi'}to_textConvert arbitrary text-formattable input to text while handling errors.>>>vistir.misc.to_text(b"these are bytes")'these are bytes'to_bytesConverts arbitrary byte-convertable input to bytes while handling errors.>>>vistir.misc.to_bytes("this is some text")b'this is some text'>>>vistir.misc.to_bytes(u"this is some text")b'this is some text'chunkedSplits an iterable up into groupsof the specified length, permore itertools.  Returns an iterable.This example will create groups of chunk size5, which means there will be6 groups.>>>chunked_iterable=vistir.misc.chunked(5,range(30))>>>forchunkinchunked_iterable:...add_to_some_queue(chunk)takeTake elements from the supplied iterable without consuming it.>>>iterable=range(30)>>>first_10=take(10,iterable)>>>[iforiinfirst_10][0,1,2,3,4,5,6,7,8,9]>>>[iforiiniterable][0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29]divideSplits an iterable up into thespecified number of groups, permore itertools.  Returns an iterable.>>>iterable=range(30)>>>groups=[]>>>forgrpinvistir.misc.divide(3,iterable):...groups.append(grp)>>>groups[<tuple_iteratorobjectat0x7fb7966006a0>,<tuple_iteratorobjectat0x7fb796652780>,<tuple_iteratorobjectat0x7fb79650a2b0>]decode_for_outputConverts an arbitrary text input to output which is encoded for printing to terminal
outputs using the system preferred locale usinglocale.getpreferredencoding(False)with some additional hackery on linux systems.get_canonical_encoding_nameGiven an encoding name, get the canonical name from a codec lookup.>>>vistir.misc.get_canonical_encoding_name("utf8")"utf-8"get_wrapped_streamGiven a stream, wrap it in aStreamWrapperinstance and return the wrapped stream.>>>stream=sys.stdout>>>wrapped_stream=vistir.misc.get_wrapped_stream(sys.stdout)>>>wrapped_stream.write("unicode\u0141")>>>wrapped_stream.seek(0)>>>wrapped_stream.read()"unicode\u0141"StreamWrapperA stream wrapper and compatibility class for handling wrapping file-like stream objects
which may be used in place ofsys.stdoutand other streams.>>>wrapped_stream=vistir.misc.StreamWrapper(sys.stdout,encoding="utf-8",errors="replace",line_buffering=True)>>>wrapped_stream=vistir.misc.StreamWrapper(io.StringIO(),encoding="utf-8",errors="replace",line_buffering=True)get_text_streamAn implementation of theStreamWrapperfor the purpose of wrappingsys.stdinorsys.stdout.On Windows, this returns the appropriate handle to the requested output stream.>>>text_stream=vistir.misc.get_text_stream("stdout")>>>sys.stdout=text_stream>>>sys.stdin=vistir.misc.get_text_stream("stdin")>>>vistir.misc.echo(u"\0499",fg="green")ҙreplace_with_text_streamGiven a text stream name, replaces the text stream with aStreamWrapperinstance.>>>vistir.misc.replace_with_text_stream("stdout")Once invoked, the standard stream in question is replaced with the required wrapper,
turning it into aTextIOWrappercompatible stream (which ensures that unicode
characters can be written to it).get_text_stdinA helper function for callingget_text_stream(“stdin”).get_text_stdoutA helper function for callingget_text_stream(“stdout”).get_text_stderrA helper function for callingget_text_stream(“stderr”).echoWrites colored, stream-compatible output to the desired handle (sys.stdoutby default).>>>vistir.misc.echo("some text",fg="green",bg="black",style="bold",err=True)# write to stderrsometext>>>vistir.misc.echo("some other text",fg="cyan",bg="white",style="underline")# write to stdoutsomeothertext🐉 Path Utilitiesvistirprovides utilities for interacting with filesystem paths:vistir.path.get_converted_relative_pathvistir.path.normalize_pathvistir.path.is_in_pathvistir.path.handle_remove_readonlyvistir.path.is_file_urlvistir.path.is_readonly_pathvistir.path.is_valid_urlvistir.path.mkdir_pvistir.path.ensure_mkdir_pvistir.path.create_tracked_tempdirvistir.path.create_tracked_tempfilevistir.path.path_to_urlvistir.path.rmtreevistir.path.safe_expandvarsvistir.path.set_write_bitvistir.path.url_to_pathvistir.path.walk_upnormalize_pathReturn a case-normalized absolute variable-expanded path.>>>vistir.path.normalize_path("~/${USER}")/home/user/useris_in_pathDetermine if the provided full path is in the given parent root.>>>vistir.path.is_in_path("~/.pyenv/versions/3.7.1/bin/python","${PYENV_ROOT}/versions")Trueget_converted_relative_pathConvert the supplied path to a relative path (relative toos.curdir)>>>os.chdir('/home/user/code/myrepo/myfolder')>>>vistir.path.get_converted_relative_path('/home/user/code/file.zip')'./../../file.zip'>>>vistir.path.get_converted_relative_path('/home/user/code/myrepo/myfolder/mysubfolder')'./mysubfolder'>>>vistir.path.get_converted_relative_path('/home/user/code/myrepo/myfolder')'.'handle_remove_readonlyError handler for shutil.rmtree.Windows source repo folders are read-only by default, so this error handler attempts to
set them as writeable and then proceed with deletion.This function will call checkvistir.path.is_readonly_pathbefore attempting to
callvistir.path.set_write_biton the target path and try again.is_file_urlChecks whether the given url is a properly formattedfile://uri.>>>vistir.path.is_file_url('file:///home/user/somefile.zip')True>>>vistir.path.is_file_url('/home/user/somefile.zip')Falseis_readonly_pathCheck if a provided path exists and is readonly by checking forbool(path.stat & stat.S_IREAD) and not os.access(path, os.W_OK)>>>vistir.path.is_readonly_path('/etc/passwd')True>>>vistir.path.is_readonly_path('/home/user/.bashrc')Falseis_valid_urlChecks whether a URL is valid and parseable by checking for the presence of a scheme and
a netloc.>>>vistir.path.is_valid_url("https://google.com")True>>>vistir.path.is_valid_url("/home/user/somefile")Falsemkdir_pRecursively creates the target directory and all of its parents if they do not
already exist.  Fails silently if they do.>>>os.mkdir('/tmp/test_dir')>>>os.listdir('/tmp/test_dir')[]>>>vistir.path.mkdir_p('/tmp/test_dir/child/subchild/subsubchild')>>>os.listdir('/tmp/test_dir/child/subchild')['subsubchild']ensure_mkdir_pA decorator which ensures that the caller function’s return value is created as a
directory on the filesystem.>>>@ensure_mkdir_pdefreturn_fake_value(path):returnpath>>>return_fake_value('/tmp/test_dir')>>>os.listdir('/tmp/test_dir')[]>>>return_fake_value('/tmp/test_dir/child/subchild/subsubchild')>>>os.listdir('/tmp/test_dir/child/subchild')['subsubchild']create_tracked_tempdirCreates a tracked temporary directory usingvistir.path.TemporaryDirectory, but does
not remove the directory when the return value goes out of scope, instead registers a
handler to cleanup on program exit.>>>temp_dir=vistir.path.create_tracked_tempdir(prefix="test_dir")>>>asserttemp_dir.startswith("test_dir")True>>>withvistir.path.create_tracked_tempdir(prefix="test_dir")astemp_dir:withio.open(os.path.join(temp_dir,"test_file.txt"),"w")asfh:fh.write("this is a test")>>>os.listdir(temp_dir)create_tracked_tempfileCreates a tracked temporary file usingvistir.compat.NamedTemporaryFile, but creates
aweakref.finalizecall which will detach on garbage collection to close and delete
the file.>>>temp_file=vistir.path.create_tracked_tempfile(prefix="requirements",suffix="txt")>>>temp_file.write("some\nstuff")>>>exit()path_to_urlConvert the supplied local path to a file uri.>>>path_to_url("/home/user/code/myrepo/myfile.zip")'file:///home/user/code/myrepo/myfile.zip'rmtreeStand-in forshutil.rmtreewith additional error-handling.This version ofrmtreehandles read-only paths, especially in the case of index files
written by certain source control systems.>>>vistir.path.rmtree('/tmp/test_dir')>>>[dfordinos.listdir('/tmp')if'test_dir'ind][]NoteSettingignore_errors=Truemay cause this to silently fail to delete the pathsafe_expandvarsCallos.path.expandvarsif value is a string, otherwise do nothing.>>>os.environ['TEST_VAR']="MY_TEST_VALUE">>>vistir.path.safe_expandvars("https://myuser:${TEST_VAR}@myfakewebsite.com")'https://myuser:MY_TEST_VALUE@myfakewebsite.com'set_write_bitSet read-write permissions for the current user on the target path.  Fail silently
if the path doesn’t exist.>>>vistir.path.set_write_bit('/path/to/some/file')>>>withopen('/path/to/some/file','w')asfh:fh.write("test text!")url_to_pathConvert a valid file url to a local filesystem path. Follows logic taken from pip.>>>vistir.path.url_to_path("file:///home/user/somefile.zip")'/home/user/somefile.zip'