Package: pfstorage
Table of ContentsOverviewpfstorageInstallationPython Virtual EnvironmentUsing thefnndsc/pfstoragedocker containerUsageCommand line argumentsEXAMPLESscript modeserver modeOverviewThis repository providespfstorage– a library / module that speaks to an object storage backend (such asswift) and also provides logic for handling input/output data locations for the ChRIS system.pfstorageMost simply,pfstorageis a module that offers a regularized interface to some other backend object storage. While currently supportingswift, the long term idea is to support a multitude of backends. By providing its own interface to several storage backends, this module removes the need for client code to change when a different object storage backend is used.While at its core a module/library,pfstoragealso provides two modes of stand-alone access: (1) a command line script interface mode to the library, and (2) a persistent http server mode. In the command line mode, the main module functions are exposed to appropriate CLI. In the http server mode, a client can use curl-type http calls to call the underlying library functions.InstallationInstallation is relatively straightforward, and we recommend using either python virtual environments or docker.Python Virtual EnvironmentOn Ubuntu, install the Python virtual environment creatorsudoaptinstallvirtualenvThen, create a directory for your virtual environments e.g.:mkdir~/python-envsYou might want to add to your .bashrc file these two lines:exportWORKON_HOME=~/python-envssource/usr/local/bin/virtualenvwrapper.shNote that depending on distro, the virtualenvwrapper.sh path might be/usr/share/virtualenvwrapper/virtualenvwrapper.shSubsequently, you can source your.bashrcand create a new Python3 virtual environment:source.bashrcmkvirtualenv--python=python3python_envTo activate or “enter” the virtual env:workonpython_envTo deactivate virtual env:deactivateUsing thefnndsc/pfstoragedocker containerThe easiest option however, is to just use thefnndsc/pfstoragedock.dockerpullfnndsc/pfstorageand then run (for example in http server mode access to the library):dockerrun--namepfstorage-v/home:/Users--rm-ti\fnndsc/pfstorage\--ipSwiftlocalhost\--portSwift8080\--forever\--httpResponse\--serveror in CLI mode:dockerrun--namepfstorage-v/home:/Users--rm-ti\fnndsc/pfstorage\--ipSwiftlocalhost\--portSwift8080\--msg'
        { "action": "ls",
          "meta": {
                        "path":         "",
                        "retSpec":      ["name", "bytes"]
                  }
        }'UsageFor usage ofpfstorage, consult the relevant wiki pages  <https://github.com/FNNDSC/pfstorage/wiki/pfcon-overview>`.Command line arguments--msg '<JSON_formatted>'
The action to perform. This can be one of:

    * objPull -- pull data from storage to file system
    * objPush -- push data from file system to storage
    * ls      -- listing of data within storage

with a JSON formatted string similar to:

    * ls:
    { "action": "ls",
      "meta": {
                    "path":         "",
                    "retSpec":      ["name", "bytes"]
              }
    }

    * objPut:
    {  "action": "objPut",
        "meta": {
                    "putSpec":              "./data",
                    "inLocation":           "storage",
                    "mapLocationOver":      "./data"
                }
    }

    * objPull:
    {  "action": "objPull",
        "meta": {
                    "path":                 "chris",
                    "substr":               "/018",
                    "fromLocation":         "chris/uploads/DICOM",
                    "mapLocationOver":      "./data"
                }
    }

[--type<storageBackendType>]
The type of object storage. Currently this is 'swift'.

[--ipSwift<swiftIP>]
The IP interface of the object storage service. Default %s.

[--portSwift<swiftPort>]
The port of the object storage service. Defaults to '8080'.

[--ipSelf<selfIP>]
The IP interface of the pfstorage service for server mode. Default %s.

[--portSelf<selfPort>]
The port of the pfstorage service for server mode. Defaults to '4055'.

[--httpResponse]
In servier mode, send return strings as HTTP formatted replies
with content-type html.

[--configFileLoad<file>]
Load configuration information from the JSON formatted<file>.

[--configFileSave<file>]
Save configuration information to the JSON formatted<file>.

[-x|--desc]
Provide an overview help page.

[-y|--synopsis]
Provide a synopsis help summary.

[--version]
Print internal version number and exit.

[--debugToDir<dir>]
A directory to contain various debugging output -- these are typically
JSON object strings capturing internal state. If empty string (default)
then no debugging outputs are captured/generated. If specified, then
``pfcon`` will check for dir existence and attempt to create if
needed.

[-v|--verbosity<level>]
Set the verbosity level. "0" typically means no/minimal output. Allows for
more fine tuned output control as opposed to '--quiet' that effectively
silences everything.EXAMPLESscript modepfstorage\--ipSwiftlocalhost\--portSwift8080\--verbosity1\--debugToDir/tmp\--typeswift\--msg'
    {
        "action":   "ls",
        "meta": {
            "path":         "",
            "retSpec":      ["name", "bytes"]
        }
    }
    'server modestart server:pfstorage\--ipSwiftlocalhost\--portSwift8080\--ipSelflocalhost\--portSelf4055\--httpResponse\--verbosity1\--debugToDir/tmp\--typeswift\--server\--foreverquery server:pfurl--verbPOST--raw\--httplocalhost:4055/api/v1/cmd\--httpResponseBodyParse\--jsonwrapper'payload'\--msg'
            {
                "action":   "ls",
                "meta": {
                    "path":         "",
                    "retSpec":      ["name", "bytes"]
                }
            }
    '