Package: shoosh
shooshWrapper for sh to run Shell commands inside Docker container and handle volume mappings seamlessly.How to1: Instantiate a containerFirst step is to instantiate the (Docker) container we are going to use next.
(You'll use whatever method/interface you're used to do it; the lib isnotmeant to manage the containers lifecycle.)For example, let's run a simplealpinecontainer mapping localtest/directory to/some/pathinside the container:$dockerrun-dt\--namesome_container\-v/tmp/test:/some/path\debian2: Create a handle to containerThen, we create a handle to the container whit the volume(s) set:>>>importshoosh>>>sh=shoosh.init('some_container')3: And execute a command through the handleBy default, whenever you cite the (host) volume, it translate to the internal path:>>>echo=sh.wrap('echo')>>>echo("Internal path:",'/tmp/test')Internalpath:/some/pathRationaleThe typical use of (Docker) containers is to spin-up a container containing
the software packages we need to accomplish a given task, process the data
we aim to, and eventually get the results back to the host system.When we run (Docker) containers, though, the filesystem structure inside the
container is completely detached from that of the host system.
The way to exchange data files between host and container is through container mounting points -- orvolumes.In practice, it means that thepathswe see inside a container -- where our
data is sitting -- is different than that of the host system -- where the very
same data files are stored, and being shared through volumes to the container.Consider the following scenario:For some reason we use a [osgeo/gdal] container to process geographical data;The data in our host system is stored under/home/user/data/some_project;When we instantiate the [osgeo/gdal] image assome_gdalcontainer, we mount-bind that path to container's/data/geo;Suppose all we want to do it to [convert our data files from GeoTiff to Cloud Optimized GeoTiff(https://gdal.org/drivers/raster/cog.html):$gdal_translate/data/geo/raster.tif/data/geo/raster_cog.tif-ofCOGEverything works very well if the interaction with the container is decoupled
from the host system,i.e.if we areexecutingthe commandsinsidethe
container. But if there is a need forrequestingthe execution of a commandfromthe host systeminsidethe container, one should be aware of the
different paths resulting from a host-container volumes binding.The demand forrequestingprocessing tasks (from the host system) to execute
inside a container have different motivations, clearly; It can be a personal
demand of having the host system software set clean and the convenience of not
getting inside the container every time a simple, atomic task has to be performed.
But it can also be part of a more complex software application running on the
host system -- like a data processing pipeline -- where different components
demand different third-party software tools (available through containers).MotivationIn my case, the reason I wrote this software, is I use Python to transform
planetary (Mars) geo-located data in different ways using either OSGEO's GDAL
as well as USGS's ISIS software packages in a couple of cloud computing projects (GMAP, NEANIAS).
I had to find a way, for my own convenience, to merge data visualization, metadata
selection, download, cleaning, map-projection, formatting, etc., from the host
system and on containers seamlessly through Python.Application (value)This (Python) libraryprimary featureis the execution of Shell commands inside
(Docker) containersandthe correctre-mapping of paths on the fly.
It doesalsoprovides a seamless interface to execute Shell commands in the
current environment, re-mapping of paths is not necessary in this case but it
allow you to move your code from a "host-container" scenario to "container-only"
-- in a cloud infrastructure -- seamlessly.All thisseamlesslyfeature is possible because of theshpackage.