JuiceFS 1.2 Beta 1: Gateway Upgrade, Enhanced Multi-User Permission Management
Published: 2024-05-06 02:38:27

[JuiceFS v1.2-beta1](https://github.com/juicedata/juicefs/releases/tag/v1.2.0-beta1) is released, featuring significant improvements in user experience, bug fixes, and three new features:

- **Enhanced gateway capabilities**: Introduces Identity and Access Management (IAM) and event notifications to provide users with more secure, flexible, and automated data management and monitoring capabilities, suitable for multi-user environments and complex application scenarios.

- **Support for POSIX access control lists (ACLs)**: Enables more powerful user permission management using Linux's built-in ACL tools (`setfacl`/`getfacl`).

- **Support for smooth upgrades**: Allows seamless upgrades without disrupting operations by remounting JuiceFS on the same mount point. This also facilitates online adjustment of mount parameters.

This release saw contributions from 41 contributors with 354 code merges. We appreciate the efforts of each contributor!
In this post, we’ll introduce JuiceFS Gateway’s implementation and new features. In upcoming blog posts, we’ll dive into other features’ implementation principles and applications. Stay tuned.

## About JuiceFS Gateway

JuiceFS stores files in chunks in the underlying [object storage](https://juicefs.com/docs/community/architecture/) system. It provides you with POSIX interface access to files in JuiceFS. When you need to access files using the S3-compatible API, JuiceFS Gateway comes into play.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tzjux1gfi7dn55x25nmq.png)



In JuiceFS v1.2-beta1, we’ve added two highly anticipated modules to the gateway:

- **IAM**: Supports multi-user management and access control as well as anonymous access control.
- **Event notifications**: Allows monitoring events happening on objects within buckets using bucket event notifications.

## Implementation
The core service of JuiceFS Gateway is to expose the POSIX file system via the S3 API. Before v1.2, we integrated the MinIO Gateway module into our code. This module was based on the Apache license. This integration made JuiceFS operations similar to MinIO's native command `minio gateway NAS`. For users, JuiceFS could be seen as an implementation of [MinIO Gateway](https://github.com/minio/minio/tree/ea1803417f80a743fc6c7bb261d864c38628cf8d/docs/gateway). MinIO's other backend implementations include NAS and Hadoop.

In version v1.2, we’ve directly implemented the [MinIO object](https://github.com/minio/minio/blob/d46386246fb6db5f823df54d932b6f7274d46059/cmd/object-api-interface.go#L88) interface and used JuiceFS as its backend storage. This approach provides a use experience close to native MinIO, inheriting many advanced features of MinIO. In this setup, JuiceFS acts as a local disk for MinIO server commands, similar to `minio server /data1` in principle.

You can manage JuiceFS Gateway directly using MinIO's `mc` tool. Since MinIO is currently released under the Affero General Public License (AGPL) license, while JuiceFS uses its [Apache license version](https://github.com/minio/minio/commit/e0d3a8c1f4e52bb4a7d82f7f369b6796103740b3), we recommend using the corresponding Apache licensed version of the `mc` tool to avoid potential compatibility issues. You can download it [here](https://dl.min.io/client/mc/release/linux-amd64/archive/mc.RELEASE.2021-04-22T17-40-00Z).

The usage of the newly added modules is similar to the corresponding modules in MinIO. For details, see MinIO's [document](https://github.com/minio/minio/tree/e0d3a8c1f4e52bb4a7d82f7f369b6796103740b3/docs) or the [JuiceFS document](https://juicefs.com/docs/community/guide/gateway/). Next, we’ll briefly introduce these new features and their usage.

## New features
### IAM
#### User management
`juicefs gateway` allows adding new users using `mc admin user add`. You can use `mc admin user` to manage added users. It supports operations such as add, disable, enable, delete, list all users, display user information, and view user policies.

#### Service accounts
We allow adding service accounts for users. Each service account is associated with the user identity and inherits policies attached to its parent user or parent user's group. Each access key supports optional inline policies that can further restrict access to operations and resource subsets available to the parent user. You can use the `mc admin user svcacct` command to manage service accounts.

#### AssumeRole security token service
`AssumeRole` returns a set of temporary security credentials that the user can use to access gateway resources. `AssumeRole` requires the authorization credentials of an existing gateway user and returns temporary security credentials including access keys, secret keys, and security tokens. Applications can use these temporary security credentials to make signed calls to gateway API operations. The policy applied to these temporary credentials is inherited from the gateway user credentials. For usage methods, see [AssumeRole security token service](https://juicefs.com/docs/community/guide/gateway/#assumerole-security-token-service).

#### Permission management
Newly created users have no permissions by default and require assignment using `mc admin policy` to grant access. This command supports adding, deleting, updating, and viewing permissions as well as adding, deleting, and updating permissions for users. Additionally, the gateway has four built-in common policies:

- `readonly`: Read-only users.
- `readwrite`: Read-write users.
- `writeonly`: Write-only users.
- `consoleAdmin`: Read-write-admin users, where "admin" means the ability to use management APIs such as creating users.

You can use `mc admin policy add` to add set custom policies.

#### User group management
JuiceFS Gateway supports creating user groups, similar to Linux user groups, and uses `mc admin group` for management. You can set one or more users to a group and grant permissions uniformly to the group.

In addition to user-specific permissions, anonymous access management is also possible. This allows specific objects or buckets to be accessible to anyone. You can use the `mc policy` command to manage this functionality.

By default, the gateway has built-in support for four types of anonymous permissions:

- `none`: Disallows anonymous access (typically used to clear existing permissions).
- `download`: Allows anyone to read.
- `upload`: Allows anyone to write.
- `public`: Allows anyone to read and write.

### Bucket event notifications
If you want to trigger certain actions based on events occurring on a bucket, you need the bucket event notification feature. You can use this feature to monitor events happening on objects within a storage bucket.

Currently supported object event types include:

- `s3:ObjectCreated:Put`
- `s3:ObjectCreated:CompleteMultipartUpload`
- `s3:ObjectAccessed:Head`
- `s3:ObjectCreated:Post`
- `s3:ObjectRemoved:Delete`
- `s3:ObjectCreated:Copy`
- `s3:ObjectAccessed:Get`

Supported global events include:

- `s3:BucketCreated`
- `s3:BucketRemoved`

You can use the `mc` client tool to set up and listen for event notifications through the `event` subcommand. The notification message sent by MinIO for publishing events is in JSON format. See the [JSON structure](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html).

Currently, storage bucket events can be published to the following destinations:

- Redis
- MySQL
- PostgreSQL
- Webhooks

For details, see [Bucket event notifications](https://juicefs.com/docs/community/guide/gateway/#bucket-event-notifications).

### Other features
This release has also introduced these gateway features:

- **Object tagging support**: Allows setting custom tags for objects. You can use `mc tag` to manage it.
- **Background running**: Supports running the gateway in the background. You can add the `-d` parameter during startup to enable this feature.

## Give it a try!
Feel free to download and try JuiceFS 1.2-beta1 [here](https://github.com/juicedata/juicefs/releases/tag/v1.2.0-beta1). If you have any questions, join [JuiceFS discussions on GitHub](https://github.com/juicedata/juicefs/discussions) and our [community on Slack](https://juicefs.slack.com/ssb/redirect).