Solution is: Add a management layer above the copy queue:
The layer gets all file copy requests
The layer has waiting list of files to be copied. If more files are requested, they are merged into the waiting list.
The layer manages the files copy so that at any given time only 'max_copy_streams' are open.
The layer has a thread which moves files from waiting list to start copy if streams are availalbe
Solution is: Add a management layer above the copy queue:
The layer gets all file copy requests
The layer has waiting list of files to be copied. If more files are requested, they are merged into the waiting list.
The layer manages the files copy so that at any given time only 'max_copy_streams' are open.
The layer has a thread which moves files from waiting list to start copy if streams are availalbe
Solution is: Add a management layer above the copy queue:
The layer gets all file copy requests
The layer has waiting list of files to be copied. If more files are requested, they are merged into the waiting list.
The layer manages the files copy so that at any given time only 'max_copy_streams' are open.
The layer has a thread which moves files from waiting list to start copy if streams are availalbe
Fixed issue where stream was not deleted correctly and caused a gem in the files under copy data base.

when file recieved on backup we used to check the cynamic content data. Now we check directly in the destination location. This will pass an issue where the monitoring on backup is slow and caused files already saved to destination be requested again.
New design which uses a queue to start copy files per demend (when there are waiting files to be copied and when there is an open place in streams
remove the use if sleep (in start copy) and moved to queue only
Merge pull request #193 from bbfsdev/copy_files

Copy files