Repository: oumi-ai/oumi
File: src/oumi/inference/remote_inference_engine.py (Python)

Code before:
        semaphore: asyncio.Semaphore,
        session: aiohttp.ClientSession,
        inference_config: Optional[InferenceConfig] = None,
    ) -> Conversation:
        """Queries the API with the provided input.

        Args:
            conversation: The conversations to run inference on.
            semaphore: Semaphore to limit concurrent requests.
            session: The aiohttp session to use for the request.
            inference_config: Parameters for inference.

        Returns:
            Conversation: Inference output.
        """
        if inference_config is None:
            remote_params = self._remote_params
            generation_params = self._generation_params
            output_path = None
        else:
            remote_params = inference_config.remote_params or self._remote_params
            generation_params = inference_config.generation or self._generation_params
            output_path = inference_config.output_path

        if not remote_params.api_url:
            remote_params.api_url = self.base_url

        assert remote_params.api_url
        async with semaphore:
            api_input = self._convert_conversation_to_api_input(
                conversation, generation_params
            )
            headers = self._get_request_headers(remote_params)
            retries = 0
            failure_reason = None
            # Retry the request if it fails.
            for _ in range(remote_params.max_retries + 1):
                async with session.post(
                    remote_params.api_url,
                    json=api_input,
                    headers=headers,
                    timeout=remote_params.connection_timeout,
                ) as response:
                    response_json = await response.json()
                    if response.status == 200:
                        result = self._convert_api_output_to_conversation(
                            response_json, conversation
                        )
                        if output_path:
                            # Write what we have so far to our scratch directory.
                            self._save_conversation(

Review comment (suggestion):
I think we should use `self._remote_params.api_url` first, and if not set base_url. If none are set, raise a value error





Code after:
        semaphore: asyncio.Semaphore,
        session: aiohttp.ClientSession,
        inference_config: Optional[InferenceConfig] = None,
    ) -> Conversation:
        """Queries the API with the provided input.

        Args:
            conversation: The conversations to run inference on.
            semaphore: Semaphore to limit concurrent requests.
            session: The aiohttp session to use for the request.
            inference_config: Parameters for inference.

        Returns:
            Conversation: Inference output.
        """
        if inference_config is None:
            remote_params = self._remote_params
            generation_params = self._generation_params
            output_path = None
        else:
            remote_params = inference_config.remote_params or self._remote_params
            generation_params = inference_config.generation or self._generation_params
            output_path = inference_config.output_path

        if not remote_params.api_url:
            remote_params.api_url = self._remote_params.api_url or self.base_url

        # Validate API URL one final time.
        if not remote_params.api_url:
            raise ValueError("API URL is required for remote inference.")
        async with semaphore:
            api_input = self._convert_conversation_to_api_input(
                conversation, generation_params
            )
            headers = self._get_request_headers(remote_params)
            retries = 0
            failure_reason = None
            # Retry the request if it fails.
            for _ in range(remote_params.max_retries + 1):
                async with session.post(
                    remote_params.api_url,
                    json=api_input,
                    headers=headers,
                    timeout=remote_params.connection_timeout,
                ) as response:
                    response_json = await response.json()
                    if response.status == 200:
                        result = self._convert_api_output_to_conversation(
                            response_json, conversation
                        )
                        if output_path: