Repository: minio/minio (language: go)

Issue:
Add option to disable free inodes check for Minio
We are trying out Minio on our Openshift clusters. We are using IBM Spectrum Scale (GPFS) file system, via standard CSI interface.

We ran into the issue, where S3 bucket will not accept new files and reports no free disk space, even though there is still plenty space available.

Analyzing the issue and looking into Minio source code, we found out that Minio checks that it still has at least 1000 free inodes after write on all volumes. In case of GPFS (and possibly other file systems), there is some number of inodes preallocated based on volume size, but GPFS is able to dynamically allocate additional inode space on demand, once preallocated inodes are starved.

Alas, Minio never tries to write, so dynamic allocation never happens.

It would be great to have an option in Minio to disable the 1000 inodes check, so that we are able to make use of the GPFS functionality.

Other option for us is to manually preallocate few millions inodes upfront, but this requires manual intervention and is generally a waste of space in case the inodes are not actually needed.



Discussion / hints:
@bukovjanmic are you using this with a single drive setup?
On Openshift, the test tenant has two pods with two volumes on each.

If I understand the minio code correctly, it checks each of the 4 volumes and refuses to accept new content if any of the volumes has <1000 free inodes.
We do not support network FS on a distributed system, you can either use GPFS on a single drive setup or use JBODs and disks for distributed setup.

I will send a fix to relax this check for a single drive setup. 
The GPFS filesystem under Minio (GPFS is distributed network storage cluster) works very well with Minio so far, with reasonable performance.

The 4 volumes that we use in our testing are backed by the same GPFS cluster, but are each independent from each other (all are separate RWO volumes).

Since the GPFS cluster handles duplication/triplication of the data, single volume for each pool should be enough, two volumes per pool will not bring any advantage, I guess.
It will work @bukovjanmic but that doesn't mean that we will support it. Creating erasure coding on top of another NAS that also does replication/protection is a wrong architecture and design that will eventually fail. 




Fix patch:
diff --git a/cmd/object-api-utils.go b/cmd/object-api-utils.go
index b7135ed3fd5b5..043c3b76a4f23 100644
--- a/cmd/object-api-utils.go
+++ b/cmd/object-api-utils.go
@@ -1198,7 +1198,7 @@ func hasSpaceFor(di []*DiskInfo, size int64) (bool, error) {
 		if disk == nil || disk.Total == 0 {
 			continue
 		}
-		if disk.FreeInodes < diskMinInodes && disk.UsedInodes > 0 {
+		if !globalIsErasureSD && disk.FreeInodes < diskMinInodes && disk.UsedInodes > 0 {
 			// We have an inode count, but not enough inodes.
 			return false, nil
 		}


Test patch: