Repository: gradio-app/gradio
Pull request: Allow supporting >1000 files in `gr.File()` and `gr.UploadButton()`
A few months ago, `starlette` decided to limit the number of files that could be uploaded via a single request to [1,000 by default (though configurable)](https://github.com/encode/starlette/commit/8c74c2c8dba7030154f8af18e016136bea1938fa). FastAPI does not support configuring this yet, [though there is an open PR](https://github.com/tiangolo/fastapi/pull/9640). 

This has prevented Gradio users from uploading more than 1,000 files through our `UploadButton` and `File` components. This PR modifies the `upload_files()` method used in the frontend to upload files in batches of 1,000 to get around this limitation. 

Fixes: https://github.com/gradio-app/gradio/issues/4361

To test:

1. First create >1000 files in a directory:

```py
import os

def create_tiny_files(folder_name, num_files):
    if not os.path.exists(folder_name):
        os.makedirs(folder_name)
    
    for i in range(num_files):
        file_name = f"{folder_name}/file_{i + 1}.txt"
        with open(file_name, "w") as file:
            file.write(str(i + 1))

if __name__ == "__main__":
    folder_name = "tiny_files2"
    num_files = 100
    create_tiny_files(folder_name, num_files)
    print(f"{num_files} tiny files have been created in the '{folder_name}' folder.")
```

2. Then try uploading them:

```py
import gradio as gr

def upload_file(files):
    file_paths = [file.name for file in files]
    return file_paths

with gr.Blocks() as demo:
    full_button = gr.Files(file_count="directory")
    textbox = gr.Textbox()
    button = gr.Button()
    
    button.click(upload_file, full_button, textbox)
    
demo.launch()
```

Problem statement:
An error happened when upload a directory more than 1000 files
### Describe the bug

it receive an error when I upload a directory with more 1000 pictures.

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Reproduction

import gradio as gr

def upload_file(files):
    file_paths = [file.name for file in files]
    return file_paths


with gr.Blocks() as demo:
    full_images = gr.Gallery(
                    label="full images", show_label=True, elem_id="full_images",info="上传全图测试目录"
                        ).style(columns=[10])
    full_button = gr.UploadButton("Upload", file_count="directory")
    full_button.upload(upload_file, full_button, full_images, scroll_to_output=True)

if __name__ == "__main__":
    ip = "0.0.0.0"
    port = 18091
    demo.queue()
    demo.launch(server_name=ip, server_port=port)

### Screenshot

_No response_

### Logs

```shell
file without key: name
```


### System Info

```shell
gradio: 3.30, linux
```


### Severity

annoying

Hints from issue discussion:
Same problem. Suggest to support batched input.

Commit message: add changeset

Patch to review:
diff --git a/.changeset/late-shrimps-tease.md b/.changeset/late-shrimps-tease.md
new file mode 100644
index 0000000000..dd94699d25
--- /dev/null
+++ b/.changeset/late-shrimps-tease.md
@@ -0,0 +1,6 @@
+---
+"@gradio/client": patch
+"gradio": patch
+---
+
+fix:Allow supporting >1000 files in `gr.File()` and `gr.UploadButton()`
diff --git a/client/js/src/client.ts b/client/js/src/client.ts
index c4ac27ef5d..a57022b01c 100644
--- a/client/js/src/client.ts
+++ b/client/js/src/client.ts
@@ -185,22 +185,28 @@ export function api_factory(fetch_implementation: typeof fetch) {
 		if (token) {
 			headers.Authorization = `Bearer ${token}`;
 		}
-
-		const formData = new FormData();
-		files.forEach((file) => {
-			formData.append("files", file);
-		});
-		try {
-			var response = await fetch_implementation(`${root}/upload`, {
-				method: "POST",
-				body: formData,
-				headers
+		const chunkSize = 1000;
+		const uploadResponses = [];
+		for (let i = 0; i < files.length; i += chunkSize) {
+			const chunk = files.slice(i, i + chunkSize);		
+			const formData = new FormData();
+			chunk.forEach((file) => {
+				formData.append("files", file);
 			});
-		} catch (e) {
-			return { error: BROKEN_CONNECTION_MSG };
+			try {
+				var response = await fetch_implementation(`${root}/upload`, {
+					method: "POST",
+					body: formData,
+					headers
+				});
+			} catch (e) {
+				return { error: BROKEN_CONNECTION_MSG };
+			}
+			const output: UploadResponse["files"] = await response.json();
+            uploadResponses.push(...output);
 		}
-		const output: UploadResponse["files"] = await response.json();
-		return { files: output };
+		console.log("uploadResponses", uploadResponses)
+		return { files: uploadResponses };
 	}
 
 	async function client(


Code review comments:
diff_hunk: @@ -185,22 +185,28 @@ export function api_factory(fetch_implementation: typeof fetch) {
 		if (token) {
 			headers.Authorization = `Bearer ${token}`;
 		}
-
-		const formData = new FormData();
-		files.forEach((file) => {
-			formData.append("files", file);
-		});
-		try {
-			var response = await fetch_implementation(`${root}/upload`, {
-				method: "POST",
-				body: formData,
-				headers
+		const chunkSize = 1000;
+		const uploadResponses = [];
+		for (let i = 0; i < files.length; i += chunkSize) {
+			const chunk = files.slice(i, i + chunkSize);		
+			const formData = new FormData();
+			chunk.forEach((file) => {
+				formData.append("files", file);
 			});
-		} catch (e) {
-			return { error: BROKEN_CONNECTION_MSG };
+			try {
+				var response = await fetch_implementation(`${root}/upload`, {
+					method: "POST",
+					body: formData,
+					headers
+				});
+			} catch (e) {
+				return { error: BROKEN_CONNECTION_MSG };
+			}
+			const output: UploadResponse["files"] = await response.json();
+            uploadResponses.push(...output);
 		}
-		const output: UploadResponse["files"] = await response.json();
-		return { files: output };
+		console.log("uploadResponses", uploadResponses)
path: client/js/src/client.ts
text: @author:
```suggestion
```

diff_hunk: @@ -185,22 +185,28 @@ export function api_factory(fetch_implementation: typeof fetch) {
 		if (token) {
 			headers.Authorization = `Bearer ${token}`;
 		}
-
-		const formData = new FormData();
-		files.forEach((file) => {
-			formData.append("files", file);
-		});
-		try {
-			var response = await fetch_implementation(`${root}/upload`, {
-				method: "POST",
-				body: formData,
-				headers
+		const chunkSize = 1000;
+		const uploadResponses = [];
+		for (let i = 0; i < files.length; i += chunkSize) {
+			const chunk = files.slice(i, i + chunkSize);		
+			const formData = new FormData();
+			chunk.forEach((file) => {
+				formData.append("files", file);
 			});
-		} catch (e) {
-			return { error: BROKEN_CONNECTION_MSG };
+			try {
+				var response = await fetch_implementation(`${root}/upload`, {
+					method: "POST",
+					body: formData,
+					headers
+				});
+			} catch (e) {
+				return { error: BROKEN_CONNECTION_MSG };
+			}
+			const output: UploadResponse["files"] = await response.json();
+            uploadResponses.push(...output);
 		}
-		const output: UploadResponse["files"] = await response.json();
-		return { files: output };
+		console.log("uploadResponses", uploadResponses)
path: client/js/src/client.ts
text: @user1:
remove console log

@author:
Thx