Repository: rclone/rclone (language: go)

Issue:
oracleobjectstorage  oos- archive storage - should allow to do a "backend restore"
#### The associated forum post URL from `https://forum.rclone.org`
https://forum.rclone.org/t/rclone-backed-restore-oracle-oci/32571


#### What is your current rclone version (output from `rclone version`)?
pi@raspberrypi:~ $ rclone version
rclone v1.64.0
- os/version: raspbian 11.8 (64 bit)
- os/kernel: 6.1.21-v8+ (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.21.1
- go/linking: static
- go/tags: none



#### What problem are you are trying to solve?
Using the Oracle Cloud Infrastructure (OCI) Object storage I'd like to backup and restore to archive storage ().
I can setup a backup to Archive storage (as also described here: https://rclone.org/oracleobjectstorage/)

#### --oos-storage-tier

The storage class to use when storing new objects in storage. https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/understandingstoragetiers.htm

Properties:

- Config:      storage_tier
- Env Var:     RCLONE_OOS_STORAGE_TIER
- Type:        string
- Default:     "Standard"
- Examples:
    - "Standard"
        - Standard storage tier, this is the default tier
    - "InfrequentAccess"
        - InfrequentAccess storage tier
    - "Archive"
        - Archive storage tier



Creating backups is working fine, but restore fails:
 rclone sync -v jmichlerace_archive:archive-rclone-tests sync-tests2

 Error returned by ObjectStorage Service. Http Status Code: 409. Error Code: NotRestored. Opc request id: fra-1:0TggNoT3SZI2uWyfZEvUNIr8J72Hy0TrKAixJtFpCpQqQWY7pU-48llRfdmAfscy. Message: The object is in Archive tier and needs to be restored.

I could probably over the OCI UI "restore" every single file, but this is highly inconvenient.

I tried as in the forum post to run a backend restore; but that gives me:
pi@raspberrypi:~ $ rclone backend restore jmichlerace_archive:archive-rclone-tests/ -n
2023/10/14 23:29:46 Failed to backend: "restore" command not found


#### How do you think rclone should be changed to solve that?

dynamically restore on a failed access or provide the "rclone backend restore" command for oos as well that is available for s3.

Documentation from Oracle side is over there: https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/managingobjects_topic-To_restore_objects_from_Archive_Storage.htm


<!--- Please keep the note below for others who read your feature request. -->

#### How to use GitHub

* Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue.
* Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
* Subscribe to receive notifications on status change and new comments.



Discussion / hints:
This needs to be ported over from the s3 backend. 

@msays2000 do you want to have a go at this? Or maybe you'd like to @jmichler ?
@ncw yes I can provide support for it, i will try to find another collaborator on oracle backend support. If I cannot find some collaborators in couple of weeks, I can provide the code. 
@msays2000 great. Should you need any support/testing with a Oracle Backend let me know. I know a bunch of things on OCI but nothing on the implementation of rclone itself unfortunately
@jmichler FYI, I assume you want rclone to just trigger the restore of a object if not already restoring and not wait for it as it will be useless to wait for 1 hour. 

```
Restore an object from Archive Storage to Object Storage.

Depending on the size of the object, it can take at most an hour to restore an object from Archive Storage. You can’t download an item object until the item is fully restored.

By default, you have 24 hours to download an object after restoration. Alternatively, you can specify a download time from 1 to 240 hours. You can find out how much download time is remaining by looking at the Available for Download value in the object’s details. 
```
@msays2000 the question is if a sync would trigger it implicitly or if we have a dedicated "restore" command. I think for s3 the approach is the latter one, then probably the user would need to issue the restore first and then a sync would fail if documents are not (yet) available in non-archive-storage.
How is this solved for AWS glacier?
yes its the later one, where one needs to execute restore command first and then a sync would fail if documents are not (yet) available in non-archive-storage.

behavior is same as aws glacier. 
@msays2000 is this still looking for a contributor?
> @msays2000 is this still looking for a contributor?

Yes, we are still looking for contributors. I can contribute, but trying to get some more contributors. 

Fix patch:
diff --git a/backend/oracleobjectstorage/command.go b/backend/oracleobjectstorage/command.go
index 4be61e3e5f9a6..e6b079231a0b8 100644
--- a/backend/oracleobjectstorage/command.go
+++ b/backend/oracleobjectstorage/command.go
@@ -7,12 +7,15 @@ import (
 	"context"
 	"fmt"
 	"sort"
+	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"github.com/oracle/oci-go-sdk/v65/common"
 	"github.com/oracle/oci-go-sdk/v65/objectstorage"
 	"github.com/rclone/rclone/fs"
+	"github.com/rclone/rclone/fs/operations"
 )
 
 // ------------------------------------------------------------
@@ -23,6 +26,7 @@ const (
 	operationRename        = "rename"
 	operationListMultiPart = "list-multipart-uploads"
 	operationCleanup       = "cleanup"
+	operationRestore       = "restore"
 )
 
 var commandHelp = []fs.CommandHelp{{
@@ -77,6 +81,42 @@ Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
 	Opts: map[string]string{
 		"max-age": "Max age of upload to delete",
 	},
+}, {
+	Name:  operationRestore,
+	Short: "Restore objects from Archive to Standard storage",
+	Long: `This command can be used to restore one or more objects from Archive to Standard storage.
+
+	Usage Examples:
+
+    rclone backend restore oos:bucket/path/to/directory -o hours=HOURS
+    rclone backend restore oos:bucket -o hours=HOURS
+
+This flag also obeys the filters. Test first with --interactive/-i or --dry-run flags
+
+	rclone --interactive backend restore --include "*.txt" oos:bucket/path -o hours=72
+
+All the objects shown will be marked for restore, then
+
+	rclone backend restore --include "*.txt" oos:bucket/path -o hours=72
+
+	It returns a list of status dictionaries with Object Name and Status
+	keys. The Status will be "RESTORED"" if it was successful or an error message
+	if not.
+
+	[
+		{
+			"Object": "test.txt"
+			"Status": "RESTORED",
+		},
+		{
+			"Object": "test/file4.txt"
+			"Status": "RESTORED",
+		}
+	]
+`,
+	Opts: map[string]string{
+		"hours": "The number of hours for which this object will be restored. Default is 24 hrs.",
+	},
 },
 }
 
@@ -113,6 +153,8 @@ func (f *Fs) Command(ctx context.Context, commandName string, args []string,
 			}
 		}
 		return nil, f.cleanUp(ctx, maxAge)
+	case operationRestore:
+		return f.restore(ctx, opt)
 	default:
 		return nil, fs.ErrorCommandNotFound
 	}
@@ -290,3 +332,63 @@ func (f *Fs) listMultipartUploadParts(ctx context.Context, bucketName, bucketPat
 	}
 	return uploadedParts, nil
 }
+
+func (f *Fs) restore(ctx context.Context, opt map[string]string) (interface{}, error) {
+	req := objectstorage.RestoreObjectsRequest{
+		NamespaceName:         common.String(f.opt.Namespace),
+		RestoreObjectsDetails: objectstorage.RestoreObjectsDetails{},
+	}
+	if hours := opt["hours"]; hours != "" {
+		ihours, err := strconv.Atoi(hours)
+		if err != nil {
+			return nil, fmt.Errorf("bad value for hours: %w", err)
+		}
+		req.RestoreObjectsDetails.Hours = &ihours
+	}
+	type status struct {
+		Object string
+		Status string
+	}
+	var (
+		outMu sync.Mutex
+		out   = []status{}
+		err   error
+	)
+	err = operations.ListFn(ctx, f, func(obj fs.Object) {
+		// Remember this is run --checkers times concurrently
+		o, ok := obj.(*Object)
+		st := status{Object: obj.Remote(), Status: "RESTORED"}
+		defer func() {
+			outMu.Lock()
+			out = append(out, st)
+			outMu.Unlock()
+		}()
+		if !ok {
+			st.Status = "Not an OCI Object Storage object"
+			return
+		}
+		if o.storageTier == nil || (*o.storageTier != "archive") {
+			st.Status = "Object not in Archive storage tier"
+			return
+		}
+		if operations.SkipDestructive(ctx, obj, "restore") {
+			return
+		}
+		bucket, bucketPath := o.split()
+		reqCopy := req
+		reqCopy.BucketName = &bucket
+		reqCopy.ObjectName = &bucketPath
+		var response objectstorage.RestoreObjectsResponse
+		err = f.pacer.Call(func() (bool, error) {
+			response, err = f.srv.RestoreObjects(ctx, reqCopy)
+			return shouldRetry(ctx, response.HTTPResponse(), err)
+		})
+		if err != nil {
+			st.Status = err.Error()
+		}
+	})
+	if err != nil {
+		return out, err
+	}
+	return out, nil
+}
diff --git a/docs/content/oracleobjectstorage.md b/docs/content/oracleobjectstorage.md
index a418ad8b30621..8ac76dc27bf46 100644
--- a/docs/content/oracleobjectstorage.md
+++ b/docs/content/oracleobjectstorage.md
@@ -771,6 +771,47 @@ Options:
 
 - "max-age": Max age of upload to delete
 
+### restore
+
+Restore objects from Archive to Standard storage
+
+    rclone backend restore remote: [options] [<arguments>+]
+
+This command can be used to restore one or more objects from Archive to Standard storage.
+
+	Usage Examples:
+
+    rclone backend restore oos:bucket/path/to/directory -o hours=HOURS
+    rclone backend restore oos:bucket -o hours=HOURS
+
+This flag also obeys the filters. Test first with --interactive/-i or --dry-run flags
+
+	rclone --interactive backend restore --include "*.txt" oos:bucket/path -o hours=72
+
+All the objects shown will be marked for restore, then
+
+	rclone backend restore --include "*.txt" oos:bucket/path -o hours=72
+
+	It returns a list of status dictionaries with Object Name and Status
+	keys. The Status will be "RESTORED"" if it was successful or an error message
+	if not.
+
+	[
+		{
+			"Object": "test.txt"
+			"Status": "RESTORED",
+		},
+		{
+			"Object": "test/file4.txt"
+			"Status": "RESTORED",
+		}
+	]
+
+
+Options:
+
+- "hours": The number of hours for which this object will be restored. Default is 24 hrs.
+
 {{< rem autogenerated options stop >}}
 
 ## Tutorials


Test patch: