Repository: xtekky/gpt4free
Pull request #2964: feat: Improve handling of media and API response structures
Author: login: hlohaus
Created: 2025-04-26T23:10:31Z | Merged: 2025-04-30T17:32:57Z

- Added validation for `media` parameter in `MarkItDown` class to raise `ValueError` if `media` is not provided.
- Included a check for `has_markitdown` in `MarkItDown` class to raise `ImportError` if `markitdown` is not installed.
- Refactored API response structure by defining `responses` as a reusable dictionary in `g4f/api/__init__.py`.
- Updated `/v1/chat/completions` endpoint to include a new parameter `conversation_id` in the `chat_completions` function.
- Enhanced logic in `chat_completions` to set `conversation_id` from configuration or function input.
- Added a new endpoint `/api/{provider}/{conversation_id}/chat/completions` in `g4f/api/__init__.py` to handle chat completions with `conversation_id`.
- Replaced duplicate response dictionary definitions with the reusable `responses` dictionary in the `/api/{provider}/chat/completions` and `/v1/chat/completions` endpoints.


Diff:
diff --git a/README.md b/README.md
index b366581bf84..b4d21f10815 100644
--- a/README.md
+++ b/README.md
@@ -94,8 +94,8 @@ Is your site on this repository and you want to take it down? Send an email to t
 1. **Install Docker:** [Download and install Docker](https://docs.docker.com/get-docker/).
 2. **Set Up Directories:** Before running the container, make sure the necessary data directories exist or can be created. For example, you can create and set ownership on these directories by running: 
 ```bash
-mkdir -p ${PWD}/har_and_cookies ${PWD}/generated_images
-sudo chown -R 1200:1201 ${PWD}/har_and_cookies ${PWD}/generated_images
+mkdir -p ${PWD}/har_and_cookies ${PWD}/generated_media
+sudo chown -R 1200:1201 ${PWD}/har_and_cookies ${PWD}/generated_media
 ```
 3. **Run the Docker Container:** Use the following commands to pull the latest image and start the container (Only x64):
 ```bash
@@ -103,18 +103,18 @@ docker pull hlohaus789/g4f
 docker run -p 8080:8080 -p 7900:7900 \
   --shm-size="2g" \
   -v ${PWD}/har_and_cookies:/app/har_and_cookies \
-  -v ${PWD}/generated_images:/app/generated_images \
+  -v ${PWD}/generated_media:/app/generated_media \
   hlohaus789/g4f:latest
 ```
 
 4. **Running the Slim Docker Image:** And use the following commands to run the Slim Docker image. This command also updates the `g4f` package at startup and installs any additional dependencies: (x64 and arm64)
 ```bash
-mkdir -p ${PWD}/har_and_cookies ${PWD}/generated_images
-chown -R 1000:1000 ${PWD}/har_and_cookies ${PWD}/generated_images
+mkdir -p ${PWD}/har_and_cookies ${PWD}/generated_media
+chown -R 1000:1000 ${PWD}/har_and_cookies ${PWD}/generated_media
 docker run \
   -p 1337:1337 \
   -v ${PWD}/har_and_cookies:/app/har_and_cookies \
-  -v ${PWD}/generated_images:/app/generated_images \
+  -v ${PWD}/generated_media:/app/generated_media \
   hlohaus789/g4f:latest-slim \
   /bin/sh -c 'rm -rf /app/g4f && pip install -U g4f[slim] && python -m g4f --debug'
 ```
diff --git a/g4f/Provider/Cloudflare.py b/g4f/Provider/Cloudflare.py
index 1ce22d33f9e..ea83510c2ee 100644
--- a/g4f/Provider/Cloudflare.py
+++ b/g4f/Provider/Cloudflare.py
@@ -76,14 +76,15 @@ def clean_name(name: str) -> str:
                 read_models()
             except ResponseStatusError as f:
                 if has_nodriver:
+                    async def nodriver_read_models():
+                        try:
+                            cls._args = await get_args_from_nodriver(cls.url)
+                            read_models()
+                        except (RuntimeError, FileNotFoundError) as e:
+                            debug.log(f"Nodriver is not available: {type(e).__name__}: {e}")
+                            cls.models = cls.fallback_models
                     get_running_loop(check_nested=True)
-                    try:
-                        args = get_args_from_nodriver(cls.url)
-                        cls._args = asyncio.run(args)
-                        read_models()
-                    except (RuntimeError, FileNotFoundError) as e:
-                        cls.models = cls.fallback_models
-                        debug.log(f"Nodriver is not available: {type(e).__name__}: {e}")
+                    asyncio.run(nodriver_read_models())
                 else:
                     cls.models = cls.fallback_models
                     debug.log(f"Nodriver is not installed: {type(f).__name__}: {f}")
diff --git a/g4f/Provider/audio/MarkItDown.py b/g4f/Provider/audio/MarkItDown.py
index b05707d7a05..cb937d11851 100644
--- a/g4f/Provider/audio/MarkItDown.py
+++ b/g4f/Provider/audio/MarkItDown.py
@@ -23,6 +23,10 @@ async def create_async_generator(
         media: MediaListType = None,
         **kwargs
     ) -> AsyncResult:
+        if media is None:
+            raise ValueError("MarkItDown requires media to be provided.")
+        if not has_markitdown:
+            raise ImportError("MarkItDown is not installed. Please install it with `pip install markitdown`.")
         md = MaItDo()
         for file, filename in media:
             text = None
diff --git a/g4f/Provider/needs_auth/Grok.py b/g4f/Provider/needs_auth/Grok.py
index d2731d89ee6..14c6f06d5a9 100644
--- a/g4f/Provider/needs_auth/Grok.py
+++ b/g4f/Provider/needs_auth/Grok.py
@@ -83,7 +83,7 @@ async def create_authed(
         model: str,
         messages: Messages,
         auth_result: AuthResult,
-        return_conversation: bool = False,
+        return_conversation: bool = True,
         conversation: Conversation = None,
         **kwargs
     ) -> AsyncResult:
diff --git a/g4f/api/__init__.py b/g4f/api/__init__.py
index 48d5aa8329c..7fceb041326 100644
--- a/g4f/api/__init__.py
+++ b/g4f/api/__init__.py
@@ -301,21 +301,25 @@ async def model_info(model_name: str) -> ModelResponseModel:
                 })
             return ErrorResponse.from_message("The model does not exist.", HTTP_404_NOT_FOUND)
 
-        @self.app.post("/v1/chat/completions", responses={
+        responses = {
             HTTP_200_OK: {"model": ChatCompletion},
             HTTP_401_UNAUTHORIZED: {"model": ErrorResponseModel},
             HTTP_404_NOT_FOUND: {"model": ErrorResponseModel},
             HTTP_422_UNPROCESSABLE_ENTITY: {"model": ErrorResponseModel},
             HTTP_500_INTERNAL_SERVER_ERROR: {"model": ErrorResponseModel},
-        })
+        }
+        @self.app.post("/v1/chat/completions", responses=responses)
         async def chat_completions(
             config: ChatCompletionsConfig,
             credentials: Annotated[HTTPAuthorizationCredentials, Depends(Api.security)] = None,
-            provider: str = None
+            provider: str = None,
+            conversation_id: str = None,
         ):
             try:
                 if config.provider is None:
                     config.provider = AppConfig.provider if provider is None else provider
+                if config.conversation_id is None:
+                    config.conversation_id = conversation_id
                 if credentials is not None and credentials.credentials != "secret":
                     config.api_key = credentials.credentials
 
@@ -395,13 +399,7 @@ async def streaming():
                 logger.exception(e)
                 return ErrorResponse.from_exception(e, config, HTTP_500_INTERNAL_SERVER_ERROR)
 
-        @self.app.post("/api/{provider}/chat/completions", responses={
-            HTTP_200_OK: {"model": ChatCompletion},
-            HTTP_401_UNAUTHORIZED: {"model": ErrorResponseModel},
-            HTTP_404_NOT_FOUND: {"model": ErrorResponseModel},
-            HTTP_422_UNPROCESSABLE_ENTITY: {"model": ErrorResponseModel},
-            HTTP_500_INTERNAL_SERVER_ERROR: {"model": ErrorResponseModel},
-        })
+        @self.app.post("/api/{provider}/chat/completions", responses=responses)
         async def provider_chat_completions(
             provider: str,
             config: ChatCompletionsConfig,
@@ -409,6 +407,15 @@ async def provider_chat_completions(
         ):
             return await chat_completions(config, credentials, provider)
 
+        @self.app.post("/api/{provider}/{conversation_id}/chat/completions", responses=responses)
+        async def provider_chat_completions(
+            provider: str,
+            conversation_id: str,
+            config: ChatCompletionsConfig,
+            credentials: Annotated[HTTPAuthorizationCredentials, Depends(Api.security)] = None,
+        ):
+            return await chat_completions(config, credentials, provider, conversation_id)
+
         responses = {
             HTTP_200_OK: {"model": ImagesResponse},
             HTTP_401_UNAUTHORIZED: {"model": ErrorResponseModel},
@@ -527,7 +534,7 @@ async def markitdown(
             return await convert(file, "MarkItDown")
 
         responses = {
-            HTTP_200_OK: {"class": FileResponse},
+            HTTP_200_OK: {"content": {"audio/*": {}}},
             HTTP_401_UNAUTHORIZED: {"model": ErrorResponseModel},
             HTTP_404_NOT_FOUND: {"model": ErrorResponseModel},
             HTTP_500_INTERNAL_SERVER_ERROR: {"model": ErrorResponseModel},


Reviews:
user: login: github-actions[bot]
body: # Pull Request Review for "feat: Improve handling of media and API response structures"

Hi H Lohaus,

Thank you for your contribution to the project! This PR brings several valuable improvements around media validation and API response management.

### Highlights
- Strict validation for the `media` parameter in `MarkItDown` streamlines error handling upfront.
- Clear and proactive import check for `markitdown` prevents obscure runtime issues.
- DRY refactor of response dictionaries into a reusable `responses` variable simplifies maintenance and reduces duplication.
- Adding `conversation_id` support in `/v1/chat/completions` and new dedicated endpoint `/api/{provider}/{conversation_id}/chat/completions` greatly enhances conversational context handling.
- Consistent use of `responses` dict across endpoints ensures uniform API documentation and behavior.

### Suggestions
- The two `provider_chat_completions` endpoint functions share the same name but have different signatures. Renaming one (e.g., `provider_chat_conversation_completions`) would improve clarity and avoid possible confusion.
- It may be helpful to add tests validating the new error cases raised in `MarkItDown` (`ValueError` and `ImportError`) to ensure these guardrails remain effective.
- In the async generator inside `MarkItDown.py`, could you confirm the `MaItDo()` class/function is correctly named? It stands out since the class is `MarkItDown`, but the instance is `MaItDo()`—just want to ensure that is intentional.

Overall, this is a solid step toward cleaner API design and improved media handling. Looking forward to the updates!

Thanks again for enhancing the codebase.

Best regards,  
g4f copilot
state: COMMENTED
author_association: NONE
submitted_at: 2025-04-26T23:13:28Z

Review comments:
diff_hunk: @@ -23,6 +23,10 @@ async def create_async_generator(
         media: MediaListType = None,
path: g4f/Provider/audio/MarkItDown.py
user: login: github-actions[bot]
body: Consider providing a default value other than `None` for `media` to avoid the need for validation in line 27.
created_at: 2025-04-26T23:13:28Z
author_association: NONE
diff_hunk: @@ -23,6 +23,10 @@ async def create_async_generator(
         media: MediaListType = None,
         **kwargs
     ) -> AsyncResult:
+        if media is None:
+            raise ValueError("MarkItDown requires media to be provided.")
+        if not has_markitdown:
path: g4f/Provider/audio/MarkItDown.py
user: login: github-actions[bot]
body: The check for `has_markitdown` might not be necessary if the package is imported at the module level, as `ImportError` would naturally be raised if the module isn't found.
created_at: 2025-04-26T23:13:28Z
author_association: NONE
diff_hunk: @@ -23,6 +23,10 @@ async def create_async_generator(
         media: MediaListType = None,
         **kwargs
     ) -> AsyncResult:
+        if media is None:
+            raise ValueError("MarkItDown requires media to be provided.")
+        if not has_markitdown:
+            raise ImportError("MarkItDown is not installed. Please install it with `pip install markitdown`.")
         md = MaItDo()
path: g4f/Provider/audio/MarkItDown.py
user: login: github-actions[bot]
body: The instantiation of `MaItDo()` appears to be a typo. Verify if this is the intended class name, or if it should be `MarkItDo`.
created_at: 2025-04-26T23:13:28Z
author_association: NONE

PR comments:
user: login: github-actions[bot]
created_at: 2025-04-30T15:34:30Z
body: ## Review of Pull Request by H Lohaus

### Summary
Thanks for the contribution! This pull request adds several useful features, including improved error handling, API structure refactoring, and new endpoints for handling `conversation_id`. It looks like a solid enhancement to the codebase.

### Key Changes
1. **Validation in `MarkItDown`**:
   - You've added validation for the `media` parameter in the `MarkItDown` class. This is a good way to ensure proper data is passed to the class, improving robustness.
   - The check for `has_markitdown` to raise an `ImportError` if `markitdown` isn't installed is a great way to handle missing dependencies.

2. **API Response Refactoring**:
   - Centralizing the response dictionary (`responses`) in `g4f/api/__init__.py` is a good move for cleaner and more maintainable code.
   - The addition of `conversation_id` as a new parameter in the `chat_completions` function and the updated `/api/{provider}/{conversation_id}/chat/completions` endpoint are smart ways to enhance the API.

3. **Duplication Removed**:
   - The move to use a reusable `responses` dictionary instead of defining it repeatedly in each endpoint improves maintainability and reduces code duplication.

### Minor Suggestions:
- The `conversation_id` is a new parameter for the `chat_completions` function, which could potentially cause issues for clients relying on older versions of the API. It might be helpful to document this change in the release notes and ensure backward compatibility.
- The new `/api/{provider}/{conversation_id}/chat/completions` endpoint could benefit from a short description in the documentation to clarify its purpose and usage.

### Overall
Great work on improving both the functionality and structure of the codebase. The refactoring and added validations will certainly make the project more robust and easier to maintain. Thanks again for contributing!

#### Approve ✅
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-30T15:46:39Z
body: # Pull Request Review

## Summary
Thank you for your contributions, H Lohaus! This pull request introduces several important improvements to the handling of media and API response structures, including better validation, error handling, and API enhancements.

## Changes Overview
- **Validation for `media` in `MarkItDown`**: The addition of a `ValueError` if `media` is not provided in `MarkItDown` is a great step in ensuring data integrity.
- **Check for `has_markitdown`**: The `ImportError` is useful for guiding users to install the necessary dependencies.
- **Refactoring API response structures**: The creation of a reusable `responses` dictionary enhances code maintainability and reduces redundancy.
- **New `/api/{provider}/{conversation_id}/chat/completions` endpoint**: This addition to handle chat completions based on the `conversation_id` will help in managing conversations more effectively.

## Code Review
### Positive Points
- The overall structure and organization of the changes are clear and concise.
- Replacing duplicate response definitions with a reusable dictionary (`responses`) improves maintainability and reduces potential for errors in future changes.
- The `conversation_id` handling in the `/v1/chat/completions` and `/api/{provider}/{conversation_id}/chat/completions` endpoints is a helpful enhancement to support tracking of conversations.

### Suggestions
- **Error Handling in `MarkItDown`**: The additional validation checks for `media` and `has_markitdown` are well placed. However, it would be useful to add a bit more context to the error messages, such as suggesting how the user can provide or install the necessary dependencies. 
- **Testing**: Make sure that these changes are covered by unit tests, particularly for the new endpoint and the added validation logic in `MarkItDown`.

### Minor Observations
- In the diff for `README.md`, the directory name change from `generated_images` to `generated_media` is clear and should be reflected in any relevant documentation. Ensure all references to media directories are consistent across the project.

## Conclusion
Overall, this pull request brings valuable enhancements and improvements to both the API and media handling in the project. Thank you for your work on this, H Lohaus! 

Looking forward to seeing these changes merged!
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-30T16:15:38Z
body: ### Pull Request Review: feat: Improve handling of media and API response structures

#### Review Summary
Thank you, H Lohaus, for contributing to the project and enhancing its functionality. Your pull request provides thoughtful improvements across several areas, including media validation, API response structure refactoring, and the introduction of a new endpoint for handling `conversation_id` in chat completions. Below are my comments and observations:

---

#### Strengths
1. **Media Validation Enhancements:**
   - The addition of a `ValueError` for missing `media` in the `MarkItDown` class provides robustness to the validation process, ensuring proper usage and preventing unexpected errors.
   - Raising an `ImportError` when `markitdown` is not installed adds clarity and actionable error handling for users.

2. **API Response Structure Refactoring:**
   - Consolidating duplicate response dictionary definitions into a reusable `responses` dictionary simplifies the codebase, enhances maintainability, and reduces redundancy.

3. **Endpoint Improvements:**
   - Introducing `conversation_id` in `/v1/chat/completions` ensures better configurability and user experience.
   - The new endpoint `/api/{provider}/{conversation_id}/chat/completions` expands API functionality to support contextual conversations, which is a significant enhancement.

4. **Docker Configuration Update:**
   - Updating directories from `generated_images` to `generated_media` reflects consistency with the new media handling logic. The documentation changes in the README are helpful for guiding users.

5. **Improved Logic in `Cloudflare` Provider:**
   - Asynchronous handling with `nodriver_read_models` improves performance and readability, making the code more robust and scalable.

---

#### Suggestions for Improvement
1. **Backward Compatibility:**
   - Consider whether the refactored API response structure and the new endpoints maintain backward compatibility with existing integrations. If not, providing a migration guide would be beneficial.

2. **Error Messages:**
   - While the enhanced error handling in `MarkItDown` is useful, providing additional context or remediation steps in the error messages could make them even more user-friendly.

3. **Testing and Validation:**
   - Ensure extensive test coverage for the changes to prevent regressions, especially for the new endpoint `/api/{provider}/{conversation_id}/chat/completions` and the refactored `responses` dictionary.

---

Overall, this pull request significantly improves the project's functionality, usability, and maintainability. Thank you for your valuable contributions. Once the above considerations are addressed, I recommend merging this pull request.

Great work, H Lohaus!

### Action Items
- Review backward compatibility and create a migration guide if needed.
- Enhance error messages with additional context.
- Add thorough testing for new changes and endpoints.

---
author_association: NONE