Repository: syncthing/syncthing (language: go)

Issue:
Sync status stays at 'Syncing xx%" when remote device pauses the folder.
### What happened?

Example: Just 2 syncthing devices sharing a single large folder.

Hi all,

Been seeing an issue recently whereby if, with large folders (5000+ files), syncthing is in the process of syncing the folder and then the folder is paused on the remote device, the local device will continue trying to pull the out of sync files and stay in "Syncing xx%" status instead of going to "Out of Sync". It does eventually get there, but on lower powered devices it takes a very long time.

I've worked around this in:
https://github.com/hireworksltd/syncthing/tree/remotefolderpaused-test
but I'm not completely familiar with the codebase and this seems like it could be a pretty expensive check to make if there are a lot of devices connected.

Any advice welcome!

Thanks so much,
Jon

### Syncthing version

v.1.27.9

### Platform & operating system

Linux amd64

### Browser version

_No response_

### Relevant log output

_No response_


Discussion / hints:
Related: https://forum.syncthing.net/t/folders-paused-but-uploading-still-going-on-and-stalled-download/17172.
Ah yeah either I am missing something, or our puller is indeed completely ignorant regarding remote folders being paused. Thanks for sending the patch, just the description would have just left me confused (and admittedly disinterested, being slow on slow devices isn't something I care about overly strongly - this is worse than that imo). The puller is "so ignorant", that it shouldn't even matter if the pausing happens during a sync. As far as I see even if the remote is paused from the get-go, we will still attempt to sync all outstanding files from it. We simply don't care if the remote is actually ready to sync, we only check that the remote has the file according to our database and that we have a connection to it. And as such do a truckload of BEP back and forth, all resulting in an error response from the remote. 

I don't have time right now to check this (test), but given you solved your problem with that patch I think we can consider it confirmed. There's no performance issue with your patch, just a few mechanics/structure issues (could be simpler). If you want to contribute the patch and work those issues out, please file a PR. And if you'd prefer not to (though you'd be very welcome to :) ), please say so too that someone else could pick it up. 

> Related: https://forum.syncthing.net/t/folders-paused-but-uploading-still-going-on-and-stalled-download/17172.

I'd say probably unrelated. The forum topic describes how actual progress on file/network transfer still happens after pausing. While I believe the ticket here describes that nothing like this is happening, it's purely doing busywork without progress until finally being idle again (and out-of-sync). @hireworksltd  Could you please confirm/deny that's the case for you?

Many Thanks Simon. Happy to open a PR and work on this.

Can confirm that is the case RE: doing busywork without progress.

Best,
Jon

Fix patch:
diff --git a/lib/model/folder_sendrecv.go b/lib/model/folder_sendrecv.go
index 6ff984f23b3..7834ee7c744 100644
--- a/lib/model/folder_sendrecv.go
+++ b/lib/model/folder_sendrecv.go
@@ -505,13 +505,10 @@ nextFile:
 			continue nextFile
 		}
 
-		devices := snap.Availability(fileName)
-		for _, dev := range devices {
-			if f.model.ConnectedTo(dev) {
-				// Handle the file normally, by copying and pulling, etc.
-				f.handleFile(fi, snap, copyChan)
-				continue nextFile
-			}
+		devices := f.model.fileAvailability(f.FolderConfiguration, snap, fi)
+		if len(devices) > 0 {
+			f.handleFile(fi, snap, copyChan)
+			continue
 		}
 		f.newPullError(fileName, errNotAvailable)
 		f.queue.Done(fileName)
@@ -1547,7 +1544,7 @@ func (f *sendReceiveFolder) pullBlock(state pullBlockState, snap *db.Snapshot, o
 	}
 
 	var lastError error
-	candidates := f.model.availabilityInSnapshot(f.FolderConfiguration, snap, state.file, state.block)
+	candidates := f.model.blockAvailability(f.FolderConfiguration, snap, state.file, state.block)
 loop:
 	for {
 		select {
diff --git a/lib/model/model.go b/lib/model/model.go
index b4ee3925e52..3615eca1653 100644
--- a/lib/model/model.go
+++ b/lib/model/model.go
@@ -2882,16 +2882,31 @@ func (m *model) Availability(folder string, file protocol.FileInfo, block protoc
 	}
 	defer snap.Release()
 
-	return m.availabilityInSnapshotRLocked(cfg, snap, file, block), nil
+	return m.blockAvailabilityRLocked(cfg, snap, file, block), nil
 }
 
-func (m *model) availabilityInSnapshot(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo, block protocol.BlockInfo) []Availability {
+func (m *model) blockAvailability(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo, block protocol.BlockInfo) []Availability {
 	m.mut.RLock()
 	defer m.mut.RUnlock()
-	return m.availabilityInSnapshotRLocked(cfg, snap, file, block)
+	return m.blockAvailabilityRLocked(cfg, snap, file, block)
 }
 
-func (m *model) availabilityInSnapshotRLocked(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo, block protocol.BlockInfo) []Availability {
+func (m *model) blockAvailabilityRLocked(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo, block protocol.BlockInfo) []Availability {
+	var candidates []Availability
+
+	candidates = append(candidates, m.fileAvailabilityRLocked(cfg, snap, file)...)
+	candidates = append(candidates, m.blockAvailabilityFromTemporaryRLocked(cfg, file, block)...)
+
+	return candidates
+}
+
+func (m *model) fileAvailability(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo) []Availability {
+	m.mut.RLock()
+	defer m.mut.RUnlock()
+	return m.fileAvailabilityRLocked(cfg, snap, file)
+}
+
+func (m *model) fileAvailabilityRLocked(cfg config.FolderConfiguration, snap *db.Snapshot, file protocol.FileInfo) []Availability {
 	var availabilities []Availability
 	for _, device := range snap.Availability(file.Name) {
 		if _, ok := m.remoteFolderStates[device]; !ok {
@@ -2905,13 +2920,16 @@ func (m *model) availabilityInSnapshotRLocked(cfg config.FolderConfiguration, sn
 			availabilities = append(availabilities, Availability{ID: device, FromTemporary: false})
 		}
 	}
+	return availabilities
+}
 
+func (m *model) blockAvailabilityFromTemporaryRLocked(cfg config.FolderConfiguration, file protocol.FileInfo, block protocol.BlockInfo) []Availability {
+	var availabilities []Availability
 	for _, device := range cfg.Devices {
 		if m.deviceDownloads[device.DeviceID].Has(cfg.ID, file.Name, file.Version, int(block.Offset/int64(file.BlockSize()))) {
 			availabilities = append(availabilities, Availability{ID: device.DeviceID, FromTemporary: true})
 		}
 	}
-
 	return availabilities
 }
 


Test patch: