Repository: harbor-framework/harbor
Issue: Feature Request: Add docker-compose.override.yml support for custom configurations
Root cause: Documentation Deficiency

## Feature Request: docker-compose.override.yml Support

### Background
After migrating from terminal bench to Harbor, I discovered that while environment variables were successfully migrated to Dockerfiles, other critical docker-compose configurations like `extra_hosts` and custom `volumes` are not easily configurable in Harbor's current architecture.

### Current Situation
- Harbor generates `docker-compose.yml` from a Jinja2 template during the `prepare` phase
- The generated file at `/compose_location/docker-compose.yml` should not be edited directly
- There's no official way to add custom volumes or extra_hosts without modifying the template
- Some use cases require mounting commercial software as volumes and adding extra_hosts for license servers

### Problem
Users who need to:
- Mount proprietary/commercial software volumes that can't be included in the image
- Add custom host entries for license servers or internal services
- Add other docker-compose configurations not covered by harbor.yml

Currently must either:
1. Fork and modify the Jinja2 template (requires maintenance)
2. Manually edit the generated docker-compose.yml after each `prepare` (error-prone)
3. Use undocumented docker-compose.override.yml (works but unsupported)

### Proposed Solution
Officially support `docker-compose.override.yml` for user customizations:

1. **Document the feature**: Add documentation explaining that users can create a `docker-compose.override.yml` for custom configurations
2. **Add to .gitignore**: Include `docker-compose.override.yml` in `.gitignore` to prevent accidental commits
3. **Add example**: Provide a `docker-compose.override.yml.example` showing common use cases
4. **Preserve during updates**: Ensure Harbor's install/prepare scripts don't interfere with user's override file

### Example Use Case
```yaml
# docker-compose.override.yml
version: '2.4'
services:
  core:
    extra_hosts:
      - "license-server.internal:192.168.1.100"
    volumes:
      - /opt/proprietary/software:/mnt/software:ro
      
  jobservice:
    extra_hosts:
      - "license-server.internal:192.168.1.100"
    volumes:
      - /opt/proprietary/tools:/mnt/tools:ro
```

### Benefits
- **Non-invasive**: Leverages Docker Compose's built-in functionality
- **Backward compatible**: Doesn't affect existing installations
- **User-friendly**: Provides a clean separation between Harbor's configuration and user customizations
- **Maintainable**: Survives Harbor updates without manual intervention
- **Standard practice**: Follows Docker Compose conventions that users already know

### Implementation Considerations
- Docker Compose automatically merges override files (no code changes needed)
- Only documentation and .gitignore updates required
- Could optionally add a check in `prepare` script to warn if override file exists (for transparency)

### Alternative Consideration
If full override support is not desired, consider adding `extra_hosts` and `custom_volumes` as configurable options in `harbor.yml` that get rendered into the template.

### References
- Docker Compose override documentation: https://docs.docker.com/compose/multiple-compose-files/merge/
- Current template location: `/make/photon/prepare/templates/docker_compose/docker-compose.yml.jinja`
- Template rendering: `/make/photon/prepare/utils/docker_compose.py`

This feature would significantly improve Harbor's flexibility for enterprise deployments with specific infrastructure requirements.

This is very much missing for multi-container custom agents

If you put a `docker-compose.yaml` file in the `environment` directory of your task and use the `DockerEnvironment` (`--env docker`) then Harbor will use that instead of the default docker compose.

I'll add this information to the docs to make it clearer. Let me know if that solves your problem!

Btw, the task migrator should keep your docker compose if its custom.

Currently, we haven't found a cloud sandbox provider that works for multi-container envs yet btw, so you'll be restricted to local docker.

@alexgshaw I think the pipeline for local agents running on multi-container env should be iterated e2e if possible as it has many use cases. Many agent provider would like to test it locally without remote repo/dist/packaging