Package: backend.ai-storage-proxy
Backend.AI Storage ProxyBackend.AI Storage Proxy is an RPC daemon to manage vfolders used in Backend.AI agent, with quota and
storage-specific optimization support.Package Structureai.backend.storageserver: The agent daemon which communicates between Backend.AI Managerapi.client: The client-facing API to handle tus.io server-side protocol for uploads and ranged HTTP
queries for downloads.api.manager: The manager-facing (internal) API to provide abstraction of volumes and separation of
the hardware resources for volume and file operations.vfsThe minimal fallback backend which only uses the standard Linux filesystem interfacesxfsXFS-optimized backend with a small daemon to manage XFS project IDs for quota limitsagent: Implementation ofAbstractVolumeAgentwith XFS supportpurestoragePureStorage's FlashBlade-optimized backend with RapidFile Toolkit (formerly PureTools)netappNetApp QTree integration backend based on the NetApp ONTAP REST APIwekaWeka.IO integration backend with Weka.IO V2 REST APIcephfs(TODO)CephFS-optimized backend with quota limit supportInstallationPrerequisitesPython 3.8 or higher withpyenvandpyenv-virtualenv(optional but recommended)Installation ProcessFirst, prepare the source clone of this agent:#gitclonehttps://github.com/lablup/backend.ai-storage-proxyFrom now on, let's assume all shell commands are executed inside the virtualenv.Now install dependencies:#pipinstall-U-rrequirements/dist.txt# for deployment#pipinstall-U-rrequirements/dev.txt# for developmentThen, copy halfstack.toml to root of the project folder and edit to match your machine:#cpconfig/sample.tomlstorage-proxy.tomlWhen done, start storage server:#python-mai.backend.storage.serverIt will start Storage Proxy daemon bound at127.0.0.1:6021(client API) and127.0.0.1:6022(manager API).NOTE: Depending on the backend, the server may require to be run as root.Production DeploymentTo get performance boosts by using OS-providedsendfile()syscall
for file transfers, SSL termination should be handled by reverse-proxies
such as nginx and the storage proxy daemon itself should be run without SSL.Filesystem BackendsVFSPrerequisitesUser account permission to access for the given directoryMake sure a directory such as/vfroot/vfsa directory or you want to mount existsXFSPrerequisitesLocal device mounted under/vfrootNative support for XFS filesystemMounting XFS volume with an option-o pquotato enable project quotaTo turn on quotas on the root filesystem, the quota mount flags must be
set with therootflags=boot parameter. Usually, this is not recommended.Access to root privilegeExecution ofxfs_quota, which performs quota-related commands, requires
therootprivilege.Thus, you need to start the Storage-Proxy service by arootuser or a
user with passwordless sudo access.If the root user starts the Storage-Proxy, the owner of every file created
is also root. In some situations, this would not be the desired setting.
In that case, it might be better to start the service with a regular user
with passwordless sudo privilege.Creating virtual XFS device for testingCreate a virtual block device mounted tolo(loopback) if you are the only one
to use the storage for testing:Create file with your desired size#ddif=/dev/zeroof=xfs_test.imgbs=1Gcount=100Make file as XFS partition#mkfs.xfsxfs_test.imgMount it to loopback#exportLODEVICE=$(losetup-f)#losetup$LODEVICExfs_test.imgCreate mount point and mount loopback device, with pquota option#mkdir-p/vfroot/xfs#mount-oloop-opquota$LODEVICE/vfroot/xfsNote on operationXFS keeps quota mapping information on two files:/etc/projectsand/etc/projid. If they are deleted or damaged in any way, per-directory quota
information will also be lost. So, it is crucial not to delete them
accidentally. If possible, it is a good idea to backup them to a different disk
or NFS.PureStorage FlashBladePrerequisitesNFSv3 export mounted under/vfrootPurity API accessCephFSPrerequisitesFUSE export mounted under/vfrootNetApp ONTAPPrerequisitesNFSv3 export mounted under/vfrootNetApp ONTAP API accessnative NetApp XCP or Dockerized NetApp XCP containerTo install NetApp XCP, please referNetApp XCP install guideCreate Qtree in Volume explicitly using NetApp ONTAP Sysmgr GUINote on operationThe volume host of Backend.AI Storage proxy corresponds to Qtree of NetApp ONTAP, not NetApp ONTAP Volume.Please DO NOT remove Backend.AI mapped qtree in NetApp ONTAP Sysmgr GUI. If not, you cannot access to NetApp ONTAP Volume through Backend.AI.NOTE:Qtree name in configuration file(storage-proxy.toml) must have the same name created in NetApp ONTAP Sysmgr.Weka.IOPrerequisitesWeka.IO agent installed and runningWeka.IO filesystem mounted under local machine, with permission set to somewhat storage-proxy process can read and writeWeka.IO REST API access (username/password/organization)