Repository: rclone/rclone (language: go)

Issue:
No access to Linkbox files and dirs
#### The associated forum post URL from `https://forum.rclone.org`
n/a


#### What is the problem you are having with rclone?
Rclone successfully encodes and copies dirs and files to Linkbox, but then no way to sync or list or download it.

The problem is that Linkbox allows files and folders with names of up to 100 characters. But the search in their API works with names of up to 50 ASCII characters and 25 UTF-16 characters (for example, Cyrillic) only. Thus, normal operation is possible only with very short names. For example, using base32 encoding for names, the maximum name length can be only 8 UTF-16 characters, and for base32768 - 16 UTF-16 characters. Which is, of course, very little in real life.


#### What is your rclone version (output from `rclone version`)
rclone v1.65.0 and rclone v1.66.0-beta.7594.5fa13e3e3


#### Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
Arch Linux 64 bit


#### Which cloud storage system are you using? (e.g. Google Drive)
Linkbox


#### The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`)

`./rclone ls -vv "telebox-encoded:photos from London"`


#### A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`)

```
<7>DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "ls" "-vv" "telebox-encoded:photos from London"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "telebox-encoded:photos from London"
<7>DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
<7>DEBUG : Creating backend with remote "telebox:files/encoded/347ftlbc73000l927l7hbud1i77qle7j8pfi9j01c5lu0bve5k8g"
<3>ERROR : : error listing: directory not found
<7>DEBUG : 5 go routines active
Failed to ls with 2 errors: last error was: directory not found
```



<!--- Please keep the note below for others who read your bug report. -->

#### How to use GitHub

* Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue.
* Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
* Subscribe to receive notifications on status change and new comments.



Discussion / hints:
And what you want rclone to do about it? 
> And what you want rclone to do about it?

Any possible solution would be great, I think. For example, just some ideas:

1. Limit the allowed name length and throw an error if it is exceeded. At least this will prevent the user from falling into the trap, as it is now: the files are uploaded, but there is actually no access to them.
2. Add a new encoder for file names with a function to shorten the name length, even if it is not crypto-resistant.
3. Automatically trim the names to the allowed length (warning the user about this).
4. Store the IDs of directories and files locally instead of constantly using the `api/open/file_search` request (perhaps such  option is already in rclone somewhere?)
5. Somehow, the UI Explorer (http://127.0.0.1:5572/#/remoteExplorer) can show a list of files in directories with long names, but `rclone ls` cannot. (However, the explorer fails to download these files). Perhaps it's possible somehow to use some functionality from there.

It just turns out that Linkbox is now on the list, but in fact it is not working / not usable.
Indeed Linkbox remote limitations seems to be a bit extreme - whoever implemented this remote should document it clearly in docs.

I do not think personally that rclone should include any of your suggestions - it is a lot of extra complexity with very limited gains.

It  should be addressed by Linkbox - as their existing API limitations make this remote barely useful for anybody.

Fix patch:
diff --git a/backend/linkbox/linkbox.go b/backend/linkbox/linkbox.go
index 75445ea72820d..cc2ac3d1693b5 100644
--- a/backend/linkbox/linkbox.go
+++ b/backend/linkbox/linkbox.go
@@ -36,7 +36,7 @@ import (
 )
 
 const (
-	maxEntitiesPerPage = 1024
+	maxEntitiesPerPage = 1000
 	minSleep           = 200 * time.Millisecond
 	maxSleep           = 2 * time.Second
 	pacerBurst         = 1
@@ -219,7 +219,8 @@ type listAllFn func(*entity) bool
 // Search is a bit fussy about which characters match
 //
 // If the name doesn't match this then do an dir list instead
-var searchOK = regexp.MustCompile(`^[a-zA-Z0-9_ .]+$`)
+// N.B.: Linkbox doesn't support search by name that is longer than 50 chars
+var searchOK = regexp.MustCompile(`^[a-zA-Z0-9_ -.]{1,50}$`)
 
 // Lists the directory required calling the user function on each item found
 //
@@ -238,6 +239,7 @@ func (f *Fs) listAll(ctx context.Context, dirID string, name string, fn listAllF
 		// If name isn't good then do an unbounded search
 		name = ""
 	}
+
 OUTER:
 	for numberOfEntities == maxEntitiesPerPage {
 		pageNumber++
@@ -258,7 +260,6 @@ OUTER:
 		err = getUnmarshaledResponse(ctx, f, opts, &responseResult)
 		if err != nil {
 			return false, fmt.Errorf("getting files failed: %w", err)
-
 		}
 
 		numberOfEntities = len(responseResult.SearchData.Entities)


Test patch:
diff --git a/backend/linkbox/linkbox_test.go b/backend/linkbox/linkbox_test.go
index aa03c79ca763b..d9b448c0c76a2 100644
--- a/backend/linkbox/linkbox_test.go
+++ b/backend/linkbox/linkbox_test.go
@@ -13,5 +13,7 @@ func TestIntegration(t *testing.T) {
 	fstests.Run(t, &fstests.Opt{
 		RemoteName: "TestLinkbox:",
 		NilObject:  (*linkbox.Object)(nil),
+		// Linkbox doesn't support leading dots for files
+		SkipLeadingDot: true,
 	})
 }
diff --git a/fstest/fstests/fstests.go b/fstest/fstests/fstests.go
index 58420991f99c7..3e3237f1a1d74 100644
--- a/fstest/fstests/fstests.go
+++ b/fstest/fstests/fstests.go
@@ -303,6 +303,7 @@ type Opt struct {
 	SkipObjectCheckWrap             bool     // if set skip ObjectCheckWrap
 	SkipDirectoryCheckWrap          bool     // if set skip DirectoryCheckWrap
 	SkipInvalidUTF8                 bool     // if set skip invalid UTF-8 checks
+	SkipLeadingDot                  bool     // if set skip leading dot checks
 	QuickTestOK                     bool     // if set, run this test with make quicktest
 }
 
@@ -690,6 +691,9 @@ func Run(t *testing.T, opt *Opt) {
 					if opt.SkipInvalidUTF8 && test.name == "invalid UTF-8" {
 						t.Skip("Skipping " + test.name)
 					}
+					if opt.SkipLeadingDot && test.name == "leading dot" {
+						t.Skip("Skipping " + test.name)
+					}
 					// turn raw strings into Standard encoding
 					fileName := encoder.Standard.Encode(test.path)
 					dirName := fileName