zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: ✨ Feature: Allow Challenge Hosts to Purge Submissions Queue from Manage Tab
GitHub labels: frontend

backend

**Description:**
Currently, challenge hosts do not have the ability to purge the submissions queue from the **Manage** tab in EvalAI. This feature would allow hosts to clear out pending/running submissions for their challenge when necessary (e.g., stuck jobs, debugging, challenge resets).

**Proposed Solution:**

* Add a **“Purge Queue”** button to the Manage tab UI for challenge hosts.
* Clicking this button should:

  * Prompt for confirmation (with clear warning: “This action will delete all pending submissions in the queue. This cannot be undone.”).
  * On confirmation, call a backend API endpoint that purges the submissions queue for that challenge.
* Backend:

  * Implement a new endpoint (accessible only by challenge hosts/admins) that clears all pending/running submissions for the given challenge.
  * Ensure proper authorization checks to prevent misuse.

* UI/UX:

  * Place the button in the Submissions/Queue management section.
  * Provide success/error messages after action.
  *   * Shows the current number of submissions in the queue

**Benefits:**

* Helps challenge hosts quickly resolve issues like stuck queues.
* Reduces dependency on super-admins for manual purges.
* Improves host autonomy in managing their challenges.

Hi 👋 I noticed this issue was assigned a few days ago. I’m also very interested in working on it. @Zahed-Riyaz , are you already making progress? If not, I’d be happy to collaborate or pick it up.

Hi is this issue still open ?can i work on this 

Hi @RishabhJain2018,

    This issue has been inactive for over two months. I would like to pick it up, specifically focusing on the backend API endpoint implementation.

    I can start working on this immediately. Could you please assign it to me?

Hello @RishabhJain2018  , I’m interested in tackling this issue. Is it available to pick up? Please assign it to me if it’s free; otherwise, could you point me to other good starter issues? Thank you!

hey @vennelavarshini18 i am currently working on this issue

Hi! @RishabhJain2018 , I’m interested in working on this issue and have started reviewing the EvalAI submission queue and authorization flow.

Before proceeding, I wanted to confirm a couple of things:

1--> Should the purge action target only PENDING and RUNNING submissions, excluding completed/failed ones?

2--> Is the queue currently managed by background task workers, and are there any existing admin utilities for purging that could be reused?

Once clarified, I’ll start with a safe backend implementation and then connect it to the UI.

Thanks!

Hi @kushxagra , are you still working on this? I have my local environment set up and would love to pick this up if you are busy. Let me know!
Repository: Cloud-CV/EvalAI
Issue: Hide or Remove Duplicate Submissions on the Leaderboard
Root cause: Unimplemented Feature Gap

Hi EvalAI Team,

@RishabhJain2018 We need assistance with managing duplicate submissions in our approved challenge.

### Challenge Information:
- Challenge Title: RoboSense Social Navigation Challenge  
- Challenge ID: 2557
- EvalAI Page: https://eval.ai/web/challenges/challenge-page/2557/overview

### Issue Description:
Few participants have created multiple teams and submitted identical results to our leaderboard. As the challenge organizer, we would like to hide or remove the duplicate entries to maintain leaderboard integrity.

Could you please advise on:
1. How to remove/hide duplicate submissions on the leaderboard as a challenge organizer
2. If this requires admin assistance from your team

Thank you for your support!

Fixed by rerunning the process and manually canceling it.
Repository: Cloud-CV/EvalAI
Issue: Submission status stuck at "submitted"
Root cause: External Dependency Breakage

I uploaded a zip package for the 3D Detection challenge five days ago, but the submission status remains "Submitted" and has not progressed. I also tried the Prediction challenge and encountered the same issue— stuck in the "Submitted" state. However, submissions to the Tracking challenge work fine and typically complete within a few hours.
Could you please check if there is an issue with the evaluation server for the 3D Detection and Prediction test sets?      ( QAQ )
Repository: Cloud-CV/EvalAI
Issue: Challenge Approval Request

Hi EvalAI Team,

We have submitted an approval request for our challenge via the EvalAI dashboard. Could you please help review and approve it?

Challenge Information:
Challenge ID: 2650
EvalAI Page: https://eval.ai/web/challenges/challenge-page/2650/manage

We would really appreciate your help reviewing and approving this challenge. Please let us know if any changes or additional steps are required. Thank you for your support!
Repository: Cloud-CV/EvalAI
Issue: No error message when downloading > 5k results from All submissions page

Hi,
I am trying to download all submissions as a CSV from `all_submissions` page but I just get a small red banner with no error message. I think It also looks like there is no way to export submissions for a specific phase, which might help avoid this issue in some cases.

Is there a workaround?

Not sure if this is same as #3341 @RishabhJain2018?

<img width="724" height="166" alt="Image" src="https://github.com/user-attachments/assets/a61db528-af71-4f7f-94a1-1e0600328182" />

Thanks @imprs for opening this. I think someone from the team might have reached out to you. Let's resolve it over the email. Closing this.
Repository: Cloud-CV/EvalAI
Issue: Eval AI submission stuck at “Submitted”  on the NuScenes detection task
Root cause: Resource Mishandling

I uploaded a zip package for the nuScenes 3D Detection challenge ten days ago, but the submission status remains "Submitted" and has not progressed. I also tried the Prediction challenge and encountered the same issue— stuck in the "Submitted" state. However, submissions to the Tracking challenge work fine and typically complete within a few hours. And I've been failing to upload the new results today as well.
Could you please check if there is an issue with the evaluation server for the 3D Detection and Prediction test sets?

Hit @mei0114  I’d like to work on this, but I’m not sure how to access the EvalAI worker logs for nuScenes locally. Could you point me to the exact command or documentation I should use to run the worker and see logs?
Repository: Cloud-CV/EvalAI
Issue: Add number of concurrent request metric in metrics middleware
GitHub labels: backend

We will need to use "Gauge" metric type: http://docs.datadoghq.com/guides/dogstatsd/#metrics

This will require increasing a shared counter "atomically". Check this for reference: http://stackoverflow.com/questions/2080660/python-multiprocessing-and-a-shared-counter

@dexter1691 @deshraj can I work on this issue?

@RishabhJain2018 : Sorry, but this will require access to Datadog credentials which only the core team members have for now. Please choose a different issue. 

@dexter1691 ok sure.

@dexter1691 @deshraj : is this done ?

No, not yet. But this is not the priority as of now. 

Closing this since not a priority as of now.
Repository: Cloud-CV/EvalAI
Issue: Evaluations stuck for VQA Challenge 2021 phase 2
Root cause: Others

Hello, @gchhablani @RishabhJain2018  sorry for bothering you. My submission files to the test dev phase 2 of the VQA Challenge 2021 have been stuck at the "Submitted" state for a long time. Since my current work heavily depends on this benchmark, please help me fix this situation. Thank you

Since the issue still persists, I would like to alert @gchhablani  @RishabhJain2018 again.
Sorry if I bother you.

There is still an issue regarding that. My exact same submission file was evaluated before and now it doesn't via either the command line upload or the manual upload. They are stuck in the `Submitted` status. @gchhablani @RishabhJain2018
Repository: Cloud-CV/EvalAI
Issue: Create notification system for EvalAI
GitHub labels: enhancement

GSOC-2019

medium-difficulty

frontend

Notificatins for all warnings, alert and messages.

@aka-jain shall i try this?......notification as pop up or just error message is to be shown?

@parismita This one is critical task, let us make the global module for the notification first then will break this system into smaller chunks. Then you can work on it.


yooo....pl notify me when done

@aka-jain : What is the current status of this issue?
Are you using any kind of lib. for showing notification?? or will it be done from scratch?

@spyshiv  We are going to build this from Scratch. Just make a directive for this which take messages as a value and return notification. Thoughts?
Any other suggestions from your side?

@aka-jain: Doing from scratch will be very comfortable for everyone.  :+1: 
and I think directive is the best solution. I am creating one for this purpose. 

Awesome @spyshiv  :)

@spyshiv lets get this done ASAP since most of the features are dependent on this. 

Work is in progress...will do it ASAP.

@spyshiv can you please complete is ASAP since we are really close to the deadline. :) 

@deshraj : Sorry for the delay. I was busy in other works.
It's almost done.(branch `ui_notification`) 
Just brief a list of events when these notification will be shown.

Closing this issue, fixed with #684
Repository: Cloud-CV/EvalAI
Issue: Docker build fails intermittently due to network errors during apt-get install
Root cause: Validation Gap

### Problem
The Docker build process fails intermittently when downloading packages during `apt-get install`, specifically when installing Chrome dependencies in the AMD64 stage. The error occurs when the network connection drops during package downloads:



### Impact
- Builds fail randomly, requiring manual retries
- CI/CD pipelines become unreliable
- Developer experience is degraded due to inconsistent builds

### Expected Behavior
Docker builds should handle transient network failures gracefully and retry failed package downloads automatically.

### Environment
- Base Image: `node:18.19.0-bullseye`
- Architecture: AMD64 and ARM64
- Affected Files: `Dockerfile.dev`, `Dockerfile`

### Proposed Solution
Add retry logic with `--fix-missing` flag to `apt-get install` commands to handle transient network failures.

<img width="1909" height="1079" alt="Image" src="https://github.com/user-attachments/assets/99f0eb1c-d24b-49aa-9139-ad12e7ae47b4" />

solution proposed at #4821 

i can work on this please assign


Hey @balaraj74,
I am already working on it.
Repository: Cloud-CV/EvalAI
Issue: Add close button to announcement banner

## Description
The announcement banner on the homepage is fixed at the top of the page and remains visible even when scrolling. Currently, there is no way for users to dismiss this banner, which can be intrusive for users who have already read the announcement.

## Proposed Solution
Add a close button (×) to the top-right corner of the announcement banner that allows users to dismiss it.

## Expected Behavior
- A close button should be visible in the top-right corner of the announcement banner
- Clicking the close button should hide the banner
- The main navigation should adjust to the top position when the banner is dismissed
- The close button should have hover effects for better UX

## Current Behavior
The announcement banner is permanently fixed at the top with no option to close it.

## Screenshots

<img width="1598" height="735" alt="Image" src="https://github.com/user-attachments/assets/1f0cbbfa-2b45-4781-96b0-7d3dc9ca6377" />

Hi! I’d love to work on this issue. Could you please assign it to me?

Hi, I’m exploring this issue and would like to work on implementing the dismissible close button for the announcement banner.
I’m reviewing the announcement banner component and checking how the navigation adjusts when the banner is removed.
My approach will be to add a close (×) button, implement the hide-on-click logic, and ensure the layout shifts correctly afterward.
I’ll share updates soon.

Hey @Impromptu55,
I am already working on it.

Hey @Musa-Qureshi-01,
I am already working on it.
Repository: Cloud-CV/EvalAI
Issue: Documentation pages for Setup (Linux, Windows, macOS) are empty on ReadTheDocs
Root cause: Documentation Deficiency

Hi team,
I noticed that several documentation pages in the EvalAI ReadTheDocs site appear to be incomplete or completely empty.

### Affected Pages

These pages currently show only a title but no actual content:

* **Docker Setup**
* **Linux Setup**
* **Windows Setup**
* **macOS Setup**
* Possibly other pages under **Getting Started → Setup**

###  Issue Summary

The pages are visible in the sidebar, but when opened they appear to contain only:

* Page title
* Empty body
* Navigation buttons ("Previous" / "Next")

This seems like either:

* The `.rst` files exist but are empty
* The content was not added to the Sphinx build
* The pages are placeholders that were never completed
* They were mistakenly excluded from the documentation build

### Impact

* New contributors cannot properly set up EvalAI on their systems.
* The hosts and developers following the docs may face confusion.
* This reduces usability and onboarding for the project.

### Suggestion

* Verify if the corresponding `.rst` files exist under `docs/source/`
* Add missing content or hide these pages until they are properly documented
* (If helpful) I’m willing to contribute and help improve the documentation once the expected structure or missing details are clarified.
* And finally I can solve this issue early.

---

If you want, I can also help you **create a PR after this issue is opened**.

<img width="1919" height="1011" alt="Image" src="https://github.com/user-attachments/assets/281e46eb-fbcc-4f2e-9ee5-656210e2722e" />

<img width="1919" height="969" alt="Image" src="https://github.com/user-attachments/assets/db29c2d9-0aea-467c-a1b4-2459ea34e921" />


So, jagriti is this issue solved?
Repository: Cloud-CV/EvalAI
Issue: Frontend(service): Add a service for basic operations like password verification etc.
GitHub labels: frontend

Need to pass password matching feature as service since we have to use it in different controllers
as for now we have to use it in - 
1. Signup (AuthCtrl)
2. Reset password (AuthCtrl)
3. Change password (ChangePwdCtrl)
 
@aka-jain : any suggestions ??

this is done.
closing this
Repository: Cloud-CV/EvalAI
Issue: Bug: Race Condition in Public Submission Toggling Allows Multiple Public Submissions
Root cause: Resource Mishandling

## Description 
When is_restricted_to_select_one_submission is enabled for a challenge phase, multiple submissions can become public simultaneously due to a race condition. This violates the business rule that only one submission should be public at a time.

## Location 
**File:** apps/jobs/views.py 
**Lines:**
 - 342-349 (in challenge_submission function) 
 - 475-485 (in change_submission_data_and_visibility function)
    


## Reproduction 
1. Save the attached test script as test_bug_race_condition.py 
2. Run: python3 test_bug_race_condition.py 
3. Observe the output showing 2 public submissions instead of 1

**Test Output:** 
Initial: 1 public submission(s) Final: 2 public submission(s) - IDs: [2, 3]


## Expected Behavior 
When a challenge phase has is_restricted_to_select_one_submission=True: 
- Only 1 submission should be public at any given time
- When making a new submission public, the existing public submission should be made private first 
- This should work correctly even with concurrent requests
              

## Actual Behavior 
- Multiple submissions can become public simultaneously 
- The race condition occurs because there is no database-level locking 
- Both concurrent requests pass the count() == 1 check at the same time 
- Both proceed to make their submissions public


![Image](https://github.com/user-attachments/assets/48c0b968-7bfd-43c3-acad-6c723494d722)

i would like to work on this issue please

@THE-Amrit-mahto-05  if u dont mind can u check the pull request i cant travis ci is again failing test but there is none problem shown for backend apart from npm cache error and i would also like the owner and creators to check once @taranjeet @deshraj 

@Aryanpagaria  The Travis CI failure is unrelated to the backend changes.
I checked the logs, and there are no backend/Django test failures the only issues are related to the CI environment itself
These failures happen before running any of our backend code, so they are not caused by the race condition fix


If possible @taranjeet  @deshraj  can rerun CI internally or review the patch manually, since the backend logic, tests, and the atomic locking fix are working correctly


and actully Travis blocks encrypted Docker credentials for PRs from forks 

@THE-Amrit-mahto-05  Thanks for your just wanna ask is mainly due to npm cache which is not able to import as it has failed more than 47 times to get diff cache of npm as in my notice if u don't mind on checking this also thank you

Hi,
The Travis CI failure appears to be unrelated to the backend changes in this PR.
From the logs, the build fails in the CI environment itself (before any Django/backend tests or code-quality checks are run), and I don’t see any errors tied to the submission visibility logic or the new transaction.atomic() block.

Whenever convenient, could someone please retrigger the CI job or review the patch manually?

/cc @taranjeet @deshraj@THE-Amrit-mahto-05
Repository: Cloud-CV/EvalAI
Issue: Change logo of twitter (old bird) to X

@RishabhJain2018  Sir
In login and signup page ,website uses old twitter logo

<img width="487" height="244" alt="Image" src="https://github.com/user-attachments/assets/7e88866b-c087-4117-af96-3182458134f7" />

you can find issue on [Login](https://eval.ai/auth/login) and [Signup](https://eval.ai/auth/signup)

Hi @RishabhJain2018 Sir, I would like to work on this issue. Could you please assign it to me?

Hello, @RishabhJain2018 Sir, I’d like to work on this issue and fix the outdated Twitter logo on the Login and Signup pages. 

I’ve located the exact place where the icon is used. The old Twitter bird icon appears in the auth footer section, rendered from the following file:
`frontend_v2/src/app/components/auth/auth.component.html`

Specifically, the icon is currently implemented using the Font Awesome class:  
`<i class="fa fa-twitter"></i>`

**Proposed fix**
1. Replace the old Font Awesome Twitter icon with the new X logo from official X pack.
2.  Use an image-based icon (SVG/PNG) placed under. `frontend_v2/src/assets/images/`
3.  Update only the Twitter icon markup, without touching: GitHub icon, Auth layout, Animations or existing styles.

Once assigned, I’ll verify the change locally and submit a focused PR.

Thanks!

Hi @RishabhJain2018 sir, I would like to work on this issue.

### Problem Identified
1. `frontend_v2/src/index.html` is using **Font Awesome 4.7**, which does not include
   the new Twitter (X) logo.
2. `frontend_v2/src/app/components/auth/auth.component.html` uses:
   `<i class="fa fa-twitter"></i>`, which renders the old Twitter icon.

### Solution
1. Upgrade Font Awesome from **v4.7 to v6.x**
2. Replace the Twitter icon with:
   `<i class="fa-brands fa-x-twitter"></i>`

Note: After upgrading, other icons may require minor class updates
(e.g. `fa fa-github` → `fa-brands fa-github`).

### ETA
1 day


Hi @RishabhJain2018

Thanks for reporting this.

I can confirm the issue — the login and signup pages are still using the old Twitter (bird) icon instead of the updated X branding.

I’d like to work on this if it’s not already assigned.

 Proposed approach
- Identify where the social login icons are defined for login/signup pages
- Replace the old Twitter logo with the updated X icon
- Ensure consistency across all authentication-related pages
- Verify that the change does not affect existing social login functionality

I’ll keep the fix minimal and aligned with the project’s UI guidelines.

Please let me know if I can take this up. Happy to proceed as per your guidance 😊  
Thanks!


hi @RishabhJain2018 sir i would like to solve this problem ,

By this approach

Replace `<i class="fa fa-twitter"></i>
` 
by 

`<i class="fa-brands fa-x-twitter"></i>`

Please assign me this issue
Repository: Cloud-CV/EvalAI
Issue: Add a check in Endpoint to check if challenge is active or not
Root cause: Validation Gap

See this endpoint: https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L197
We need to add a check that is the challenge is active or not otherwise return a 404 response. 

@deshraj Can I work on this?

Sure. 

Fixed in #486
Repository: Cloud-CV/EvalAI
Issue: Security: Replace inline superuser check with permission-based access control in bulk email endpoint

The **notify_users_about_challenge** endpoint currently relies on inline **request.user.is_superuser** checks for authorization.

This approach is fragile and risks accidental misuse if the view is reused or middleware behavior changes.

Proposed solution:

Use DRF permission classes (e.g. IsAdminUser or a custom permission)

Add endpoint-specific throttling

Add audit logging for bulk email triggers

If this approach is acceptable, I’d be happy to open a PR.
Repository: Cloud-CV/EvalAI
Issue: Challenge creation checks only host team membership, not status or permissions
Root cause: Validation Gap

Hi maintainers,

I noticed that challenge creation currently checks only host team membership,
not host status or permissions.
While tracing this flow, I also observed that the host team listing endpoint
returns all teams associated with a user without filtering by status or
permissions. Because of this, it seems possible for a user with a non-ACCEPTED
status or non-ADMIN/WRITE permission to access a host team ID and pass the
membership check.
Is this behavior intentional, or should permission checks (e.g. status=ACCEPTED
and permissions in [ADMIN, WRITE]) be enforced at the API level? I might be
missing some context, but happy to work on this if a backend-side check is
desired.
Repository: Cloud-CV/EvalAI
Issue: Convert start_date and end_date of Challenge model to human readable format
GitHub labels: enhancement

easy_to_fix

backend


hey @virajprabhu, are you working on this issue ?
if not, I'd like to take this one up. :smile: 

@manu-chroma I'm nearly done with this. Can you pick up another one?

sure thanks!

Fixed in #499
Repository: Cloud-CV/EvalAI
Issue: Add is_active field to ChallengePhase serializer
GitHub labels: enhancement

GSOC-2019

backend

Please see #484 for reference. 

@deshraj Can I work on this issue?

Sure. 

No, I don't think so. `is_active` in ChallengePhaseSerializer is just to give data to the client side so that it becomes easy to figure out if the challenge phase is active or not. 

@deshraj I think we should also check [here](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L271) , if the challenge phase `is_active` or not ?

@deshraj Please help with [this](https://travis-ci.org/Cloud-CV/EvalAI/builds/192783985#L953) error in tests. Also take a look at [this](https://travis-ci.org/Cloud-CV/EvalAI/builds/192783985#L887) link.

It looks like yo forgot to set the value of start_date or end_date. 

@deshraj  The value of `start_date` and `end_date` has already been set in the base class. Please refer to this [link](https://github.com/Cloud-CV/EvalAI/blob/master/tests/unit/challenges/test_views.py#L850).

Fixed in #502
Repository: Cloud-CV/EvalAI
Issue: Problem with Auth messages
Root cause: Validation Gap

Clicking on login button without filling any field is not showing any messages. 
I think a message like "All fields are required" should also be shown there.


@spyshiv @RishabhJain2018 : Can you look into this?

@aka-jain yeah sure.

@RishabhJain2018 : assigning this to you.

Fixed in #510