zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: 🧭 Fix Left-Hand Navigation Bar in EvalAI Docs

**Description:**

The left-hand sidebar in the EvalAI documentation appears cluttered and unorganized. This affects usability and makes it harder for users to locate relevant sections quickly.

📸 **Screenshot for reference:**

![Sidebar Screenshot](https://github.com/user-attachments/assets/7bb2f17c-7ada-4eb1-bb7c-442b45094c0e)

### 🛠 Issues Identified:

* No clear grouping or hierarchy between related topics (e.g., challenge hosting vs participation vs development).
* Inconsistent naming conventions (e.g., some start with verbs, others are nouns).
* Redundant or scattered structure (e.g., "Submission" vs. "Make Submission Public/Private").
* Lack of collapsible sections for improved readability.

### ✅ Suggested Fix:

Refactor the sidebar navigation in the docs site with the following improvements:

* Group content under logical categories like:

  * 🧪 Getting Started (Introduction, Installation)
  * 🎯 Hosting a Challenge (Host Challenge, Challenge Configuration, Writing Evaluation Script, Approve Challenge)
  * 🚀 Participating in Challenges (Participate, Make Submission Public/Private)
  * ⚙️ Developer Guide (Pull Request, Contributing Guidelines, Directory Structure, Migrations, Architecture)
  * 📚 Reference (Cite, FAQ, Glossary)

* Ensure consistent naming (e.g., use "Submit a Challenge", "Join a Challenge", etc.).

* Use collapsible/expandable headings (if the documentation theme supports it).

### 📍References:

* \[Current Docs Sidebar Screenshot attached above]

### 💡 Benefit:

Improving the navigation structure will significantly enhance documentation usability for new users, challenge hosts, and contributors.


Got it. Thanks!

Hey @aditi-dsi , This is what I see on the UI. Can you please check:

https://evalai.readthedocs.io/en/latest/

![Image](https://github.com/user-attachments/assets/f8ed721b-692e-4d02-ba44-80e35d15d2d1)

Hi! @RishabhJain2018 I'm Interested in working on this sidebar refactoring issue. Could you please assign it to me?

Hello @RishabhJain2018 I would like to contribute to Cloud-CV, could you please assign me any of issues available
Repository: Cloud-CV/EvalAI
Issue: Team name should be unique
GitHub labels: bug

GSOC-2019

backend

critical

The participant team name should be unique ( or a primary key)! Right now two different users can create a team with the same name!! 

We need to fix this in both participant teams and challenge teams. 

I want to work on this,could you assign me

Thanks! This is a super critical issue so expecting rapid response on this. 
You can work on this. 

@Rishav09 Let me know if you are stuck somewhere with this issue. 

@Rishav09 are you still working on this? 

Yes ,I had some issues with my database.I tried to add primary key to the team-name of `models.py-host` and `models.py-participants` but it is throwing an error `Unable to create or select a team.
{{perm.emailError}}`.Could you suggest me where to head now?

you can just make it `unique=True` and remain the `id` attribute as the primary key. 

Fixed in #543 and #544
Repository: Cloud-CV/EvalAI
Issue: Broken website link ( http://cloudcv.org )on LinkedIn and GitHub profiles

Hi CloudCV team,
I noticed that the website link (http://cloudcv.org/) provided on both your LinkedIn page and GitHub profile is currently not functioning or leads to a broken page.

This might make it difficult for new users and contributors to find official resources or get involved.

Could you please update or fix the website URL?

Thanks for your great work!

I face same issues


@Jayrajsinh01 please assign me.


Hi @Jayrajsinh01 !

I've created PR #4835 to fix this issue. The PR replaces all deprecated `cloudcv.org` references with the active `eval.ai` domain across the entire codebase.

**Changes include:**
- Updated website links in all frontend components (both v2 and legacy)
- Fixed email addresses from `@cloudcv.org` to `@eval.ai`
- Updated documentation and configuration files
- Modified backend settings (`DEFAULT_FROM_EMAIL` and `ADMIN_EMAIL`)

**Total files updated:** 19 files across frontend, backend, and documentation

The changes maintain backward compatibility where needed and only affect user-facing URLs and contact information. All broken links should now point to the correct `eval.ai` domain.

Thanks for reporting this issue!

Hi @RishabhJain2018, @Kajol-Kumari, @spyshiv, @aka-jain, @Ram81, @deshraj and other Maintainers

Kindly Review this minor change and merge it
Repository: Cloud-CV/EvalAI
Issue: Can't build on CentOS Stream release 9 because of self-signed certificate
Root cause: Environment Incompatibility

```
git clone https://github.com/Cloud-CV/EvalAI
cd EvalAI
docker compose up --build 
```

```
[…]
 > [nodejs base  8/14] RUN npm install -g bower gulp gulp-cli:
2.020 npm ERR! code SELF_SIGNED_CERT_IN_CHAIN
2.020 npm ERR! errno SELF_SIGNED_CERT_IN_CHAIN
2.026 npm ERR! request to https://registry.npmjs.org/bower failed, reason: self signed certificate in certificate chain
2.036 
2.036 npm ERR! A complete log of this run can be found in:
2.036 npm ERR!     /root/.npm/_logs/2025-07-03T10_00_50_586Z-debug.log
------
Dockerfile:13

--------------------

  11 |     

  12 |     # Install Prerequisites

  13 | >>> RUN npm install -g bower gulp gulp-cli

  14 |     RUN npm link gulp

  15 |     RUN npm cache clean -f

--------------------

target nodejs: failed to solve: process "/bin/sh -c npm install -g bower gulp gulp-cli" did not complete successfully: exit code: 1
```

Commit: `bfd54ac8564155972e942e479b530ca5777faece`

It does work on Debian, though.

npm install -g bower gulp gulp-cli  use this code sir using : at end make that error if you working on old project use npm install -g gulp-cli
npm install

if u getting issue again then the issue in docker . instead use
RUN npm config set strict-ssl false
RUN npm install -g bower gulp gulp-cli
before the npm install

Thanks @uday-codes69, I found a solution, it was due to our network setup. Sorry for the noise.

Glad to hear you found the solution, @jeromebelleman! No worries at all — these things happen. Appreciate the follow-up and happy to have been a part of the thread. Let me know if I can help further anytime
Repository: Cloud-CV/EvalAI
Issue: Create a notification if user enters incorrect email address while inviting participants
Root cause: Unimplemented Feature Gap

If user A enters incorrect email address while inviting users to his / her participant team, then the user should be notified if the request failed. 

The response is a 400, with the json as: {"email":["Enter a valid email address."]}
We should show a notification regarding this

The current behavior is that the add email address pops up goes away which means that the request was successful. Instead, we should create a toast for both success / failure. 

Closing this issue since it's being handled in #482
Repository: Cloud-CV/EvalAI
Issue: Add Payment Options and Compute Tier Details to Documentation

<p><strong>Description:</strong><br>
We should update the documentation to clearly describe the available <strong>payment options</strong> and <strong>compute tier configurations</strong> so that users can easily compare plans.</p>
<hr>
<p><strong>Proposed Content:</strong></p>
<h3>💳 <strong>Payment Options and Compute Tiers</strong></h3>

Feature | Essentials ($125) | Core ($250) | Advanced ($850)
-- | -- | -- | --
Compute Tier | 0.5 vCPU, 1–4 GB RAM, Linux | 1 vCPU, 2–4 GB RAM, Linux | 1 vCPU, 4–16 GB RAM, Linux
Task Runtime | 24/7 for 30 days | 24/7 for 30 days | 24/7 for 30 days
Ephemeral Storage | 20 GB (expandable) | 20 GB (expandable) | 20 GB (expandable)
Concurrent Evaluations | 1 container | 1 container | 1 container
Custom Dependencies (init.py) | ✔️ | ✔️ | ✔️


<hr>
<p><strong>Tasks:</strong></p>
<ul class="contains-task-list">
<li class="task-list-item">
<p><input type="checkbox" disabled=""> Add this table to the <strong>Pricing</strong> section of the documentation.</p>
</li>
</ul>
<hr>
Repository: Cloud-CV/EvalAI
Issue: 🐛 Feature Request: Replace Worker Note Block with Worker Configuration Details
Root cause: Unimplemented Feature Gap

**Description:**
Currently, the *Worker logs* section includes this informational block:

> **Note:** We have auto-scaling and auto-allocation features in place which assess the resource requirement for your challenge based on previous submissions. Ideally, you should not need any manual intervention for scaling the worker. However, in case your evaluation requires additional resources or you face any issues, please email us at [[admin@cloudcv.org](mailto:admin@cloudcv.org)](mailto:admin@cloudcv.org).

This should be replaced with a dynamic display of the **current worker configuration**, so challenge hosts have immediate visibility into how their evaluation workers are provisioned.

**Proposed Change:**
Remove the existing note block and instead render the following **current worker configuration** properties retrieved from the backend:

* CPU and memory allocation (vCPUs, RAM)
* Number of worker processes
* Queue name
* Add this line:  If you need assistance, please contact [[team@eval.ai](mailto:team@eval.ai)](mailto:team@eval.ai).

This information should be displayed in a clean, readable format (e.g., a table or card).

![Image](https://github.com/user-attachments/assets/8a2558f7-5a02-4a0d-9f8c-507b4fc16f87)
Repository: Cloud-CV/EvalAI
Issue: Logout sometime fails due to invalid token
Root cause: Validation Gap

Likely an issue with the authorization token expiring.

@dexter1691 @deshraj Can I work on this issue?

@RishabhJain2018, @spyshiv is already working on it. Can you pick any other issue?

@deshraj yeah sure.

This issue has been resolved. PR #519. 
@dexter1691 please check once and close this.

Working properly on staging. Closing it.
Repository: Cloud-CV/EvalAI
Issue: Improve Test Coverage for challengeHostTeamsCtrl.js (Current: 70.51%)

**Description:**

The file EvalAI/frontend/src/js/controllers/challengeHostTeamsCtrl.js.js currently has 70.51% test coverage (as per Codecov). To improve this, comprehensive unit tests need to be added targeting untested and partially covered functionality

hey, I am interested in picking up this issue, Please assign this to me.

Hi @Akshat453,  
I’d like to work on this test-coverage issue.  
I have the Docker env running and can see the current gaps in challengeHostTeamsCtrl.js.  
Could you assign me? I’ll open a PR once coverage > 90 %. Thanks!
Repository: Cloud-CV/EvalAI
Issue: Fix the read the docs UI

<img width="1215" height="1323" alt="Image" src="https://github.com/user-attachments/assets/e7cf27b1-a7a8-44d6-b7fa-f2d51a96eb2a" />

Hey @aditi-dsi , Can you please fix this UI and, if possible, make it centered?
Repository: Cloud-CV/EvalAI
Issue: Adding same participant multiple times sends a 200 response
Root cause: Validation Gap

Sending same participant multiple times should not send a response. Instead it should send an error response saying that the participant has already been added. 

@dexter1691 : inviting a user to a team involves sending an email, this is the reason why it is giving 202 status code (if email is not delivered). But on the db part only one row will be created.


Interesting. But the status of the participant is set as accepted. It should probably be set as pending, no?

see: https://github.com/Cloud-CV/EvalAI/blob/master/apps/participants/serializers.py#L48

@dexter1691 : yeah , I and @deshraj decided that for v1, status will be `accepted` only.

Closing this since according to version 1, this is handled correctly.
Repository: Cloud-CV/EvalAI
Issue: Add documentation on how to make a submission baseline
Root cause: Documentation Deficiency

**## 🚀 Feature Request**

**### Description**
The “Baseline Submissions” section of the EvalAI docs currently exists as an empty stub. We need to fill it out with clear, step‐by‐step guidance so challenge hosts know how to create, register, and upload a baseline submission.

**### Motivation**
Baseline submissions are critical for challenge hosts and participants to understand expected performance and get started quickly.
Repository: Cloud-CV/EvalAI
Issue: Add documentation to remove the ability for participants to hide their submissions
Root cause: Documentation Deficiency

🚀 Feature Request

**Description**

We currently allow participants to toggle visibility of their own submissions (“hide” or “show”), but challenge organizers may want to enforce full transparency. We need to document how to disable or remove the “hide submission” feature both in the EvalAI web UI and CLI/API.

**Motivation**
Without clear docs, challenge hosts won’t know how to lock down submission visibility. Proper documentation will help:

* Prevent participants from concealing their performance
* Ensure all baselines and results remain visible to organizers and other participants.
Repository: Cloud-CV/EvalAI
Issue: Link CLI documentation in the EvalAI docs
GitHub labels: enhancement

docs

### Description

The EvalAI documentation currently references CLI usage in various sections but does not include a direct link to the official CLI documentation site. We should add a prominent “CLI Documentation” link ([https://cli.eval.ai/](https://cli.eval.ai/)) in the relevant docs sections to help users quickly find detailed CLI reference and installation instructions.

HI @RishabhJain2018 
I have figured out what needs to be changed and would like to work on this issue. Is there by any chance that I can get assigned ? 

I see @aditi-dsi has just started working on this. I will remove my branch to avoid conflicts.
Repository: Cloud-CV/EvalAI
Issue: Add documentation for adding/removing participants and challenge hosts to the team
GitHub labels: enhancement

docs

Description
We need comprehensive documentation on how to add/remove both participants and challenge hosts to the team.



Hey @RishabhJain2018, I'd like to make the document for this issue.



Hey @RishabhJain2018 👋
If this issue is still open, I’d like to work on it.

Approach

Participants

Create team → Dashboard > Participant Teams > Create
Add → Manage Team > Invite via username/email
Remove → Manage Team > Remove (only creator can)

Hosts
Create host team → Dashboard > Host Teams > Create
Add → Manage Team > Add via username/email
Remove → Manage Team > Remove (only primary host can)

I’ll add these details in:
doc/source/02-challenge-host
doc/source/03-participate
ℹ️ Although I am new here, I’m excited to contribute and learn.

Hey @RishabhJain2018, 
I’m a beginner and I’d like to work on this. 
I will create a documentation page describing the UI steps and the API endpoints for adding/removing participant team members and challenge hosts, and open a PR soon. 
Please let me know if there’s any preferred location or format.
Repository: Cloud-CV/EvalAI
Issue: Rework and finalize #2786: Unique phase‑specific URLs for “My Submissions,” “All Submissions,” and Leaderboard
Root cause: Unimplemented Feature Gap

### Description

Pull Request #2786 implements slug‑based URLs so each challenge phase under “My Submissions” and “All Submissions” has its own unique, shareable link. We also need to extend this to the Leaderboard view. Before merging, we must address outstanding review comments, fix build errors, update tests, and ensure API consistency (including slug support in analytics and leaderboard endpoints).


Hi @RishabhJain2018,  
I’d like to take over #4759.  
I have the Docker dev-env running and can start immediately — could you assign me?  
Thanks!
Repository: Cloud-CV/EvalAI
Issue: Create API endpoint to get all participants for a team
GitHub labels: enhancement

GSOC-2019

medium-difficulty

backend

critical


Thanks for opening this issue @jayantsa. I will wait for sometime and if someone doesn't pick this issue, then I will work on it myself. 

@deshraj can I work on this?

Oh great, that will be awesome. Assigning it to you. :-) 

@jayantsa Any update on this? Do you need some help? I am just pushing since we need this feature  ASAP. 

@deshraj  I am working on this. I have created a pull request but some checks are not successful .Can you help?

Fixed in #493
Repository: Cloud-CV/EvalAI
Issue: Rework and clarify bi‑directional GitHub sync feature (#3565)
GitHub labels: enhancement

🚀 Enhancement Request
**Description**

Pull request #3565 introduced a bi‑directional synchronization between EvalAI challenge UI and corresponding GitHub repositories, but it needs rework to fix outstanding issues, and finalize its implementation. We need to test  the intended behavior end‑to‑end and add documentation for the feature.

if the issue is still open can i work on this
Repository: Cloud-CV/EvalAI
Issue: Add functionality of inviting users to Host Team
Root cause: Unimplemented Feature Gap

A Host Team member can invite others in the same host team. A similar implementation can be found in the participant team view and the controller will be almost the same as of Participant Team View. 

Let me know if you have doubts/concerns. 

@deshraj  can you assign this to me?

Sure. 

@Ram81 if you need help, then you can refer to https://github.com/Cloud-CV/EvalAI/pull/351/commits/786edf14afa092341c52ac5b46ff409c978699cd

 Okay @deshraj :+1: 

@deshraj  is the api for inviting user to host team ready? because I can't find it in /docs

@Ram81 Sorry for replying late. I missed this thread somehow. Yes, the API is ready. Please check `/docs` again. 

@deshraj I've already added the URL for the API you are mentioning, in PR #512. There is no need to change right!

@Ram81 Yeah there is no need to change the URL. Again, sorry I missed this thread. :) Please ping me on gitter if I miss something. 


@deshraj You can close this one too.
Repository: Cloud-CV/EvalAI
Issue: Submission status stuck at "submitted"
Root cause: Unimplemented Feature Gap

<img width="926" height="118" alt="Image" src="https://github.com/user-attachments/assets/bcb82058-829a-4699-8bc7-cf41d19f9882" />

nuscenes detect task





> <img alt="Image" width="926" height="118" src="https://private-user-images.githubusercontent.com/89957109/481363246-bcb82058-829a-4699-8bc7-cf41d19f9882.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTczMTY2NDYsIm5iZiI6MTc1NzMxNjM0NiwicGF0aCI6Ii84OTk1NzEwOS80ODEzNjMyNDYtYmNiODIwNTgtODI5YS00Njk5LThiYzctY2Y0MWQxOWY5ODgyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA5MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwOTA4VDA3MjU0NlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFkOTExZjY3MGRkYWZjYjJjMGU3MTRmMjJhZGNjOTJhNTJhNjMwMDAyZTcxYWJjOWFmY2M0ZWRhYzBiMWJhZjUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.p-B4_GLXH1SlNle27sS2gT_jdQYF7Lt3TJCJcsbfkLE">
> nuscenes detect task

Hello, have you solved this problem now?

Hello, have you solved this problem now? I uploaded a zip package for the 3D Detection challenge five days ago, but the submission status remains "Submitted"  and tried the Prediction challenge and encountered the same issue— stuck in the "Submitted". However, submissions to the Tracking challenge work fine and typically complete within a few hours.  （ Q _ Q ）555555～
Repository: Cloud-CV/EvalAI
Issue: Add functionality to remove yourself from the Host Team
GitHub labels: enhancement

GSOC-2019

easy_to_fix

backend

The view and controller will be almost similar to that of Participant Team View. 
Please see #348 for code. 

@deshraj as Im not working on any other issue right now. Can I pick this up ? 

Sure. :-) 

fixed via #574 ?

@anweshknayak I don't think so. Since you are already working on other issue, so I am assigning @daas-ankur-shukla on this. 

@deshraj no actually there is a confusion. I already had a PR ready. But closed that since some PR for the same issue has already been merged. https://github.com/Cloud-CV/EvalAI/pull/504 Please take a look and let me know. 
And please take a look a similar functionality has been added in https://github.com/Cloud-CV/EvalAI/pull/574. Shall I take it up ? 

@anweshknayak I get what you are saying. But looks like there is some issue at the backend. That's why @aka-jain removed the API support from the frontend. Can you please cross check the API on your own machine? 

@deshraj i have got the environment ready on my machine but i am not able to recreate the issue. i am getting the following error while signing up

OPTIONS http://localhost:8000/api/auth/registration/ net::ERR_CONNECTION_REFUSED
(anonymous) @ vendor.js:112
p @ vendor.js:107
(anonymous) @ vendor.js:105
(anonymous) @ vendor.js:138
$eval @ vendor.js:152
$digest @ vendor.js:149
$apply @ vendor.js:153
(anonymous) @ vendor.js:287
dispatch @ vendor.js:3
q.handle @ vendor.js:3

Some guidance on recreating the issue will be helpful

sorry for the trivia. got it sorted

@aka-jain close this ? fixed by #574 

so is it closed or still open?

No @anweshknayak, this hasn't solved yet. Code is there for client side but there is some error from backened side. Are you working on any other issue?
And I had closed your PR because of code replication but issue is still open.

@aka-jain related to #607 ? @gauthamzz is working on it.

Yep, let solve him first. Then we will integrate the Client side code.

@anweshknayak Are you working on any issue?

@aka-jain No. Are there any new issues to work ?

What about #575 ?

@daas-ankur-shukla Please check other issues to work on :)

@aka-jain sure, but i am curious, weather #478 has been solved or not? and if yes, how? :)

@aka-jain @deshraj close this ? 

@anweshknayak Sorry, I missed this one. closing it now.