zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: Add ability to add or delete challenge phase and splits
Root cause: Unimplemented Feature Gap

Currently its not possible to add or delete challenge phases and splits. The host should be able to delete phases and splits by removing them from the yaml file. This was previously implemented in #4157. But it was revert due to a bug that the phases were deleted and recreated everytime the yaml file is used to update the challlenge.

Hi, I am working on this issue. Could you please tell me how you test this feature? To the best of my knowledge, the yaml file is located in the github repo created by the user, how to update the yaml file in the dev envrionment?

> Hi, I am working on this issue. Could you please tell me how you test this feature? To the best of my knowledge, the yaml file is located in the github repo created by the user, how to update the yaml file in the dev envrionment?

Hello @poju3185 I use the EvalAI-Starter modification that was build by Ayush to upload challenge and test these features. https://github.com/Cloud-CV/EvalAI-Starters/pull/71

Could you please tell me how did you find out that every phases are deleted an re-created when deleting some phases?
When I am testing, I think the old phases are not been deleted and re-created.
Here's what I did

1. Create a new challenge with two phases.
2. Update the challenge with a yaml file with only one phase.
3. Observe that the id for the first phases did not change.
Repository: Cloud-CV/EvalAI
Issue: Add info about clearing browser cache or incognito in email verification
Root cause: Documentation Deficiency

Along with email verification issue (verify your email), we should also show a warning that this does not work sometimes due to old browser cache and ask users to try incognito.

I am interested please assign me this issue

@Akshat453 Please create a PR and let me know.

> .
hey there i was working on it 
just wanted to clarify if it is about the message that appears when you signup and its on the home screen?

@gchhablani @Suryansh5545 
I added the information that send a message to clear cache, use incognito or refresh the page on not recieving the email.
Repository: Cloud-CV/EvalAI
Issue: Unable to create new challenge: Unauthorized for url
Root cause: Validation Gap

I am following the instructions from https://eval.ai/web/challenge-create and https://evalai.readthedocs.io/en/latest/host_challenge.html#host-challenge-using-github to host a challenge on https://eval.ai, but I am getting the following error during the Actions in the build step during 'Validate challenge': 
```
Traceback (most recent call last):
  File "github/challenge_processing_script.py", line 74, in <module>
    response.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://eval.ai/api/challenges/challenge/challenge_host_team/3109/validate_challenge_config/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "github/challenge_processing_script.py", line 81, in <module>
    error = response.json()["error"]
KeyError: 'error'
Error: Process completed with exit code 1.
```


Have you solved the issue ? because I am having the same problem 

I ran into this issue and realised that my EvalAI token had expired. You can check if your token is still valid in _Profile_ -> _Auth Token_ and if it is expired, try generating a new one with _Refresh Token_.

The error message could have been informative.

Hi @imprs , Thanks for the fix and the feedback. We will fix the error message. I am closing this one since I opened another issue for it.
Repository: Cloud-CV/EvalAI
Issue: Forbidden for url error when creating challenge
Root cause: Validation Gap

Following errors occurred while validating the challenge config: 403 Client Error: Forbidden for url: https://eval.ai/api/challenges/challenge/challenge_host_team/xxxx/create_or_update_github_challenge/
There was an error while creating an issue: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest/repos/repos#get-a-repository"}

What should I do to solve this issue?

Closing this since it is a very old issue.
Repository: Cloud-CV/EvalAI
Issue: Instance is None while saving the file in Submission Model
Root cause: Algorithmic Error

In Submission model, when the model is saved then the path where the `input_file` is saved is `submission_files/submission_None/` but ideally the `None` should be replaced with the id of the submission.

@TroJan can you please prioritize this over all other issues assigned to you? It seems like this the super critical thing for us for now. 

Fixed via #681
Repository: Cloud-CV/EvalAI
Issue: Django Crashes Due to  "silk_request" does not exist

### Description
After deploying our Django application using Docker, the admin page crashes after some period of normal operation. 

### Error Message
The exact error message encountered is as follows:
```bash
relation "silk_request" does not exist

LINE 1: INSERT INTO "silk_request" ("id", "path", "query_params", "r...
```

### Steps to Reproduce

- Deploy the using Docker container.
- Use the application normally
- After some time, attempt to access the admin page again.
- The page fails to load, and the above error message is logged.

### Environment
Ubuntu 22.04.2 LTS

Thank you in advance for your time and help in addressing this issue.

Closing this as it is not an issue.
Repository: Cloud-CV/EvalAI
Issue: Update Twitter (X) Logo to New one.
GitHub labels: enhancement

frontend

good first issue

GSOC-2025

The website still has the old logo of Twitter, now known as, X.
It should be updated with the new one.

Thanks!
![image](https://github.com/Cloud-CV/EvalAI/assets/67970947/a13fa749-5ee6-4b42-91a4-ddb6efcc57f3)


Hey @Gautam413, may I work on this issue?

Helle @Gautam413 , assign this issue to me, will fix it

Hello @RishabhJain2018, I’d like to work on this issue. Could you please assign it to me?

hey @Gautam413 , I got it this one, pls assign it to me!!


@Gautam413 @RishabhJain2018 if this issue is open I would love to contribute! I am new to OSS but I am proficient in python, CSS, JS and C++. I have experience in training and fine tuning ML Models as well was frontend and backend using React and Django. If there are any beginner friendly issues I can work on kindly let me know.

Hello mentors @Gautam413 @RishabhJain2018 ,  

I would like to work on issue #4289 as part of my contributions towards GSoC.  
The issue is about updating the outdated "Twitter bird logo" to the new "X logo" in the project footer. 

🔧 My Plan  
- Locate the current Twitter logo reference in the codebase (`<i class="fa-brands fa-twitter"></i>`).  
- Replace it with the updated Font Awesome **X logo** (`<i class="fa-brands fa-x-twitter"></i>`).  
- Test the fix locally and verify the updated UI.  
- Attach before/after screenshots in the PR for clarity.  

Screenshots  


Current Footer with Old Twitter Logo (Before):

<img width="1910" height="359" alt="Image" src="https://github.com/user-attachments/assets/a21fce75-0de6-4cbe-b6de-33f1de2024d6" />


How i Locally create footer just like the website and my approach to solve this issue: 

![Image](https://github.com/user-attachments/assets/b2e24603-f0ee-4ce0-9b36-921b8c33208a)


Updated Footer with New X Logo (After):

<img width="1914" height="423" alt="Image" src="https://github.com/user-attachments/assets/32dfa166-01ae-4a8b-8e42-136507c16e7f" />


Zoomed in view of Final Result:

![Image](https://github.com/user-attachments/assets/b9f69583-eee6-4b48-bbb0-cb5f37840881)

I’ve already tested this locally (see screenshots above).  
Kindly assign this issue to me so I can start working on the implementation.  


Hey, is this issue still open. I would like to contribute.

@RishabhJain2018 @Gautam413 please assign me.

Hi 👋

I’d like to work on this issue and add the MongoDB version check instructions to the README for Ubuntu users.
I have 2+ years of industry experience in a product-based company, have contributed to multiple open-source projects, and have hands-on experience working with MongoDB.

Please assign this issue to me so I can start working on it and submit a PR.

Thanks!


Hi! I’m a GSoC aspirant and would love to work on this issue.
Could you please assign it to me? Thank you!
Repository: Cloud-CV/EvalAI
Issue: Enhancement [Frontend] : Improve Accessibility of "Show Analysis" Button in Challenge Analytics

When looking at the challenge analytics tab, especially for people new to the platform, the current design doesn't make it easy to find the 'Show Analysis' button. As more challenges get added, the button gets harder to see, which makes the experience for users not as good as it could be.

So to make it user-friendly adding a scrollable container here will look good here.

# Before:
![Screenshot 2024-03-12 213247](https://github.com/Cloud-CV/EvalAI/assets/48647625/cd2b55ae-5ff5-419b-a880-b2810eef8cf0)

# After:
![Screenshot 2024-03-12 214003](https://github.com/Cloud-CV/EvalAI/assets/48647625/567b7119-62a1-45ad-94a4-bfe6cd157207)
Repository: Cloud-CV/EvalAI
Issue: Enhancement [Frontend]: Add Logout Button for Mobile and Other Viewer Compatibility
Root cause: Unimplemented Feature Gap

I have observed that when viewing the site from mobile, tablet viewports, the logout button is not visible due to the responsive web design. This creates an issue where users on these devices do not have a way to log out. To improve the user experience a dedicated logout button that is accessible and visible on mobile and tablet viewports is necessary


https://github.com/Cloud-CV/EvalAI/assets/48647625/26dbb803-d7d4-4a6b-b56c-5fab8d5c59d2
Repository: Cloud-CV/EvalAI
Issue: Uneven width of Partner organization logos across responsive size below 993px.

The Partner organization logos has uneven widths across diiferent size of responsive layout below 993px.

## Proof of  Issue

https://github.com/Cloud-CV/EvalAI/assets/67970947/d30e519a-6c5a-42f1-b635-73de4f85a08b
Repository: Cloud-CV/EvalAI
Issue: LVIS Evaluation Server Stuck
Root cause: Others

I've recently tried to submit to the [LVIS v1.0 Evaluation Server](https://eval.ai/web/challenges/challenge-page/1114/overview) specifically the Test-Dev Phase v1.0 with some dummy answers, and it's been over 24 hours and the submission is still running. Is there any update as to what is happening?

Hi, Closing this since this isn't the issue anymore. Please feel free to open a new issue if it persists.
Repository: Cloud-CV/EvalAI
Issue: Bug: Showing irrelavant warning whenever wrong details are shared

when we put wrong entries on github url / linkedin url/ email in the update-profile section, it triggered two warnings, one of which was empty and is distracting.


![Screenshot from 2024-03-20 14-13-01](https://github.com/Cloud-CV/EvalAI/assets/118094105/d62e7e57-1bd0-425f-8f0e-9c8cf766fbe7)
Repository: Cloud-CV/EvalAI
Issue: Bad Request for Host a remote evaluation challenge
Root cause: Validation Gap

I am holding a remote evaluation challenge with EvalAI. 

I can succcessfully download the submissions from the website. 

However, when I need to update the leaderboard with `update_finished`, I meet error in 'update_submission_data'. I found that when goes through it goes wrong in function `make_request` and `response.raise_for_status()`. Specifically:
```
response = requests.request(
                method=method, url=url, headers=headers, data=data
            )
```

It reports that `requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://eval.ai/api/jobs/challenge/2253/update_submission/` and the content in the website is `{"detail":"Authentication credentials were not provided."}`

How can I fix it?

Did you find a fix for this?

Closing this since this is an old issue. Please feel free to re-open.
Repository: Cloud-CV/EvalAI
Issue: add documentation on docker m1 setup
Root cause: Documentation Deficiency

add docker installation documentation on a separate new `.md` file and link it on the readme

PS: there is already a PR (https://github.com/Cloud-CV/EvalAI/pull/4450) opened. Please add a video for installation as well so that contributors can verify

I am interested please assign me this issue

Hi, @gautamjajoo I can do this please assign this token to me. 
Thank you!


hi @gautamjajoo , can i take up this one ?


Closing this since #4498 solves this.
Repository: Cloud-CV/EvalAI
Issue: New Design Buttons Elements
GitHub labels: enhancement

GSOC-2019

medium-difficulty

frontend

Please check attached file for new design button (host challenge button in design). Need to design a global component for button which can be use anywhere with common class name.
Points need to be focus:
1. Button color,
2. Button Border-radius,
3. Button back shadow and shadow color,
4. Button text color and font-weight.
5. Class name should be unique,
6. Write a sass in sepearate file. Don't put code in older files.

**Note: Please make PR to **new-ui** branch instead of **master** branch. The changes will be later merged into master.**

Let us know in case of query.
![relax](https://cloud.githubusercontent.com/assets/11537940/21943766/c36ef5ea-d9f8-11e6-895d-786adfb552f3.png)


@aka-jain Can I work on this?

@RishabhJain2018 : Do #433  first, it will be helpful to set colors easily here.

Okk sure.

@RishabhJain2018 : Now you can work on this.

Fixed with #437
Repository: Cloud-CV/EvalAI
Issue: send request to create a new challenge failed
Root cause: Configuration Error

Hello, I am new to EvalAI. I run a local EvalAI server in my local computer, listen on 127.0.0.1:8888。I can open the web page in chrome. I can login with default account. 

And now, I want to create a new challenge with script `challenge_processing_script.py`, I use `EvalAI-Starters` as my template. But when I run the script, It failed with error:

```text
e = 405 Client Error: Method Not Allowed for url: http://127.0.0.1:8888/api/challenges/challenge/challenge_host_team/4/validate_challenge_config/

e = 405 Client Error: Method Not Allowed for url: http://127.0.0.1:8888/api/challenges/challenge/challenge_host_team/4/create_or_update_github_challenge/
```

But when I use the script to send request to `https://eval.ai/`, It can success create a new challenge. I don't kown how to fix this. I just run local server with `docker-compose up --build`.

Thanks.

I would like to work on this issue

Closing this since it is an old issue.
Repository: Cloud-CV/EvalAI
Issue: Doc: Duplicated Paragraphs in the Host Challenge section of the doc

There are duplicated paragraphs in the Host challenge section of the doc:
![image](https://github.com/Cloud-CV/EvalAI/assets/93506927/32a2fc74-cdf0-4350-8767-307bd9fb5291)

If you'd like to report a bug, please include the following information:
- the expected behaviour
There should be only a single occurence of this text: 
```
We support two types of code-upload based challenges -

Code-Upload Based Challenge (without Static Dataset): These are usually reinforcement learning challenges which involve uploading a trained model in form of docker images and the environment is also saved in form of a docker image.
Static Code-Upload Based Challenge: These are challenges where the host might want the participants to upload models and they have static dataset on which they want to run the models and perform evaluations. This kind of challenge is especially useful in case of data privacy concerns.
```
- the observed behaviour
There are duplicated Paragraphs
- steps to reproduce the problem
- which operating system and/or browser you're using, if relevant
- (if possible) copies of the console/server logs

On the other hand, if you'd like to propose a new feature:
- please describe, as clearly as possible, what you'd like to see
- (if possible) include mockups/screenshots, if these would help convey the meaning better

Thanks!


Please assign me this issue
Repository: Cloud-CV/EvalAI
Issue: There is no result after nuScenes tracking challenge submission.
Root cause: Others

![image](https://github.com/Cloud-CV/EvalAI/assets/54844191/b6f6da9b-d1de-4d76-8e82-91be35e0370e)



Have you solved your problem? I got the same issue recently.

This issue is resolved. Please re-open if it happens again.
Repository: Cloud-CV/EvalAI
Issue: Issue with AWS service on worker stop/restart

- There are start/stop/restart worker buttons on manage tab for challenge hosts.
- When they stop/restart workers or when we auto-scale worker, something on the backend becomes inconsistent and workers attribute becomes empty and task_def_arn is non-empty. After this, it is not possible to access the worker and I have to manually clear the task_def_arn for that worker.
- We need to explore when this happens and find a good solution for it.
Repository: Cloud-CV/EvalAI
Issue: Improve error handling by providing specific error messages
Root cause: Validation Gap

![image](https://github.com/Cloud-CV/EvalAI/assets/48647625/72d0488a-c556-4ad5-8b98-294b8eb75f6e)
![image](https://github.com/Cloud-CV/EvalAI/assets/48647625/13ac1db0-b9d8-4a35-a4d3-e8f11627ceab)
![Screenshot 2024-04-03 190707](https://github.com/Cloud-CV/EvalAI/assets/48647625/cc5f20ba-86d8-4393-9071-4cd29ad7354f)
Repository: Cloud-CV/EvalAI
Issue: Update variable.scss file for new UI
GitHub labels: enhancement

GSOC-2019

frontend

Change the variables file of scss with new color codes as mentioned below:
1. Gradient : blue= #323d8a, purple= #af34d3
2. Dark blue (Primary color): dark blue = #382373
3. Light gray (secondary colors) : light gray = #9d9d9d
4. dark gray (secondary colors): dark gray = #4d4d4d

**Note: Please make PR to new-ui branch instead of master branch. The changes will be later merged into master.**
Please see attached design for reference.
![relax](https://cloud.githubusercontent.com/assets/11537940/21944741/bbbef318-d9fd-11e6-84fe-b4a59dfa54fa.png)




Fixed in #435. Closing it.
Repository: Cloud-CV/EvalAI
Issue: Frontend: Auth Token doesn't get updated on UI if the user logs on after a long time.
Root cause: Configuration Error

**Issue:**

If a user logs in after a very long time, the UI still shows the old auth token. See the screenshot -
![Screenshot 2024-04-06 at 12 25 28 PM](https://github.com/Cloud-CV/EvalAI/assets/12206047/553b52ad-68a1-426b-903e-68c0a028b874)

**Expected Behaviour:**

The updated token should be shown to the user on the UI.

cc: @gchhablani @gautamjajoo 

Hi @RishabhJain2018, I have made small changes to fix the UI, let me know if it is wrong and I have to make changes elsewhere.


Hi @Yashovardhan08 , I would love to take this issue

Hey @Yashovardhan08 , Thanks for the quick PR. I have added @gautamjajoo  to review the PR.

> Hi @Yashovardhan08 , I would love to take this issue

Hey @ranjanmangla1 , This issue is already taken. Please be on lookout for new issues.

Hi, I noticed this issue hasn't been solved yet. I would love to work on this if it's available or if I can assist the current assignee


Can you please assign this issue to me

Hey @RishabhJain2018 , is this still open? if yes can you assign this issue to me.

Hello @RishabhJain2018 is the issue still open?If yes could you please assign it to me 

Hey @Imkkrish , Sure, please take this issue.

Hey @Akshat453 , Please feel free to pick any other issue.

> Hey [@Imkkrish](https://github.com/Imkkrish) , Sure, please take this issue.
> 
> Hey [@Akshat453](https://github.com/Akshat453) , Please feel free to pick any other issue.

Sure! I'll check it out!


Hey @Imkkrish , Any updates on this?

i want to work on it.


Hello @RishabhJain2018 , I’m interested in tackling this issue. Is it available to pick up? Please assign it to me if it’s free; otherwise, could you point me to other good starter issues? Thank you!



Hello @RishabhJain2018 is the issue still open?If yes could you please assign it to me

cc: @gchhablani @gautamjajoo
Repository: Cloud-CV/EvalAI
Issue: submission blocked on Argoverse 2
Root cause: Others

Hello. I have a submission blocked problem on Argoverse 2: Motion Forecasting Competition's Multi-world prediction channel. Thank you for your help. https://eval.ai/web/challenges/challenge-page/1719/my-submission


The status always is 'submitted'.

Thanks @kingwmk for reaching out.

@gchhablani Can you please look into it?


I have the same question, I have a submission blocked problem on Argoverse 2: Motion Forecasting Competition's single agent forecasting. It always in "submitted" status.

Sorry. The problem come out again. @RishabhJain2018 @gchhablani 

Hey Everyone, Closing this since this isn't the issue anymore. Please feel free to open a new issue if it persists.
Repository: Cloud-CV/EvalAI
Issue: Add a flag to submissions for CLI vs UI
GitHub labels: enhancement

easy_to_fix

backend

GSOC-2025

Add flag to know if submission was made through CLI or UI.

Please pull the PR and check if it works correctly.

@gchhablani I got this one, Assign it to me.



@RishabhJain2018 hello is this issue  open to work on??

@RishabhJain2018 
I'd love to work on this issue and submit a PR soon! Can I take this up?
Repository: Cloud-CV/EvalAI
Issue: Add a total submissions (including failed/cancelled) limit for phases.
Root cause: Unimplemented Feature Gap

People can submit a file as many times as they want if it always keeps getting failed. We should have a daily submission limit per challenge phase. We should use `max(total_limit, finished_limit)` as total allowed submissions in a day, while also checking the current `finished_limit`.
Repository: Cloud-CV/EvalAI
Issue: Evaluation stuck at "Submitted" on VQAv2 2021
Root cause: External Dependency Breakage

I’m sorry to bother you. My submissions for the [VQAv2](https://eval.ai/web/challenges/challenge-page/830/submission) Phase: Test-Dev Phase keep getting stuck at 'submitted' status.

team name is : wair
user name is : yxh
thanks
Repository: Cloud-CV/EvalAI
Issue: BugFix: Remove Submission Type Marker on UYI
Root cause: Algorithmic Error

**Issue:** After making the submissions, the submission type maker stays on the UI while the phase maker gets cleared. 

**Ideal Behaviour:** Both the UI makers should be cleared.

![image](https://github.com/Cloud-CV/EvalAI/assets/12206047/6381d63f-476b-4b77-aad1-c984624c60e5)



@RishabhJain2018 I got this one, assign it to me

I think we should clear submission visibility marker as well ?

![image](https://github.com/Cloud-CV/EvalAI/assets/48647625/6746966d-20ff-4645-b163-766ac19031eb)


Yes, please do that too!

@RishabhJain2018 this will need some changes in the challengesubmit.component.ts file , i figured it out and will make the changes, can you please assign it to me


@RishabhJain2018  Hey, I'm new to open source contribution. Can you help me get started with contributing.

hello, Is it possible being assigned to this issue?

Heya, any update on this issue?

Hey @RishabhJain2018,
Has the issue been closed? 

Hello @RishabhJain2018 , please let me know is this issue need to fix?

@RishabhJain2018  hello I want to work on this issue

Hi @SHRISHTISHUKLA-0 , There is already a PR for this. Please pick any other issue :)

Hi, @RishabhJain2018  
I find this issue is still open
Let me know if I can contribute if the issue is still there