Package: rmount
RMount - Robust Remote MountThis is a robust remote mount wrapper around the mount utilityrclone. The python-side implementation makes it possible to integrate rclone into your python application with pre-packaged rclone binaries (version v1.62.2). This is a ready-to-go solution without external dependencies for mounting a local directory to a remote storage provider such as AWS S3. RMount is robust to time-outs, connection drops, while it abstracts the details of integrating directly with the remote provider.NOTECurrently only supports Linux.Philosophy of the library:MonitorRestart when possibleExit GracefullyFail LoudlyStorage Systemscurrently supported:Google Cloud Storagevia S3.AWSvia S3.Remote SSH, i.e. for your own private server with SSH accessS3 remote file-systems are supported by several cloud providerslisted below.System Requirementsmountpointcommand should be in PATHfusermountcommand should be in PATHSystem support for FUSE file systemThe above requirements are by default met on most Linux distributions such as Ubuntu.Installpip install rmountmountpointcommand must be accessible and in Path. e.g. runningmountpoint .should return. is not a mountpointor. is a mountpointUsageYou will first need to define your configuration object and then you can useRemoteMountwith a context manager i.e.withor simply by calling.mount()and.unmount(). See below ands3 example,ssh example.AWS S3 ConfigSet-up your access keysfromrmountimportS3config=S3(provider="AWS",region="us-east-1",secret_access_key="xxx",access_key_id="xxx",)GCS S3 ConfigSet-up your access keysconfig = S3(
    provider="GCS",
    secret_access_key="xxx",
    access_key_id="xxx",
    endpoint="https://storage.googleapis.com",
)SSH Remote ConfigSet-up your access keysfrompathlibimportPathfromrmountimportRemoteconfig=Remote(host="localhost",user="root",port=22,key_file=Path.home()/".ssh"/"id_rsa",)Writing a file# local directorylocal_path=Path("/tmp/s3")# remote directory e.g. s3://rmount gs://rmount or /rmountremote_path="rmount"mount=RemoteMount(config,remote_path,local_path)withmount:local_path.joinpath("foo").write_text("bar")Advanced UsageYou might prefer to not use S3 storage, for several reasons, such as privacy, costs and more. In those cases we support the use of a personalRemoteServer.pip install rmount[server]You will also need Docker installed with root access. Please seeDEVELOPER GUIDERemoteServerRemoteServer is a docker container running an SFTP server. The advantage of using aRemoteServeris that you can isolate access between the storage of the mount process and access to more sensitive files. This allows you fine-grained control of file-system access using SSH.Use-case exampleAssume you usermountfor the experiment You might want to provide access to the storage of the experiment data to several people you trust enough to get access to experiment data, e.g.ABLATORdashboard, but at the same time you might not want them to be able tosshinto your main machine to have access to your personal files.fromrmount.serverimportRemoteServerfrompathlibimportPathpublic_key=Path.home()/".ssh"/"id_rsa.pub"private_key=Path.home()/".ssh"/"id_rsa"local_path=Path("/tmp/")server=RemoteServer(local_path=local_path,remote_path=remote_path,public_key=public_key,)server.start()config=Remote(host=server.ip_address,user=server.user,port=server.port,key_file=private_key,)mount=RemoteMount(config,remote_path,local_path)withmount:local_path.joinpath("foo").write_text("bar")print(server.ssh_command)A full example can be foundHEREKnown IssueBecause the monitoring of the mount process happens in the background and is monitored via threads and processes. The background threads / processes are run async to the main process and are used to restart the mount process and eventually gracefully exit. When an application dies or is killed for x,y,z reason the graceful clean-up, such as unmounting might not take place. As such you can have a mount process running on the background. The problem can not be addressed from within the library as this depends on many OS related factors. For example, a new process is spawned and ifSIG_KILLis received for one process it is not possible to detect from within python or attempt a clean-up as such all other background processes will remain alive.In summaryif youSIG_KILLa python application usingrmountyoumightalso need to manually clean-up after yourself any remaining background processes e.g.rcloneor they will use memory and compute.Developer guideFull detailsHERE$pipinstall-e.[dev]$maketestCurrently there is no support for Mac or Windows as they require multiple additional steps to install mount. It is not viable for themain developer of this projectto support these systems. If you are interested in writing code to support additional OS. You should find a way to replace the command line utility depedencies listed above.mountpointcommand checks whether a directory is a mount point andfusermountcommand unmounts a directory. As long as robust alternatives can be found and packaged in this repo or documentation provided for them it should be sufficient for the same library to work on any OS. Additionally, the OS must support FUSE filesystem, e.g.WinFspormacFUSE.S3 Remote Storage ProvidersIn theory, RMount support allthe providers supported by rclone, but you will need to implement your own configuration object. We have only tested with AWS S3 but intheoryit should work with all providers:AWS S3Alibaba Cloud (Aliyun) Object Storage System (OSS)CephChina Mobile Ecloud Elastic Object Storage (EOS)Cloudflare R2Arvan Cloud Object Storage (AOS)DigitalOcean SpacesDreamhostGCSHuawei OBSIBM COS S3IDrive e2IONOS CloudLiara Object StorageMinioPetaboxQiniu Cloud Object Storage (Kodo)RackCorp Object StorageScalewaySeagate Lyve CloudSeaweedFSStackPathStorjTencent Cloud Object Storage (COS)Wasabi