zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: [Backend] Send email with challenge pk and queue name for a remote evaluation challenge 
Root cause: Unimplemented Feature Gap

### Description

For remote evaluation challenge, challenge hosts need `challenge_pk` and challenge `queue_name` to start a remote evaluation worker. When a challenge host creates a new challenge we should send an email with `challenge_pk` and `queue_name`

@Ram81 could I please take up this issue and make a PR for the same?

@burnerlee sure, go ahead

@Ram81 how do I create a new email template and get an ID for the same?

![Screenshot from 2021-02-22 18-15-06](https://user-images.githubusercontent.com/55936223/108710334-eb86d580-7539-11eb-8f36-d1e11f9e1055.png)


@burnerlee for now you can make changes using a dummy id. You can add a template in text format in your PR and we can iterate over it. We store emails in sendgrid template, once we converge on content I'll create a template and share the id with you.

HI @Ram81  Can I take this issue?


@Ram81 Can I please take up this issue ?

@shashikant231 @rawat-divyanshu this issue is already fixed in #3283. Please pick up another issue. We will also open a list of new issues in the next few days.

> @shashikant231 @rawat-divyanshu this issue is already fixed in #3283. Please pick up another issue. We will also open a list of new issues in the next few days.

@Ram81 Thanks for the update.
Repository: Cloud-CV/EvalAI
Issue: Django admin given link is not working in docs

In docs under the ' Approve a challenge(for forked version) '. In step 1, The link given for ' Django admin panel ' is not working.

Link given here -
![docs](https://user-images.githubusercontent.com/57143358/108632313-39e19900-7494-11eb-8c15-145ac352a990.png)

result -
![admin](https://user-images.githubusercontent.com/57143358/108632321-40701080-7494-11eb-8c49-ed8459e216fe.png)


@Ram81 @RishabhJain2018 , I want to work on this issue.
Repository: Cloud-CV/EvalAI
Issue: Changing Font type and size of Tagline

We can choose a different font for mentioning the tagline "**Evaluating state of the art in AI!**" with increasing the size of the font in Login/SignUp page. As the tagline must act like an enhancement factor, it's important to not apply standard font and size in it.

![Screenshot (375)](https://user-images.githubusercontent.com/67580203/108912836-2af31600-764f-11eb-92a9-aa15d47d76d8.png)

Should I do it?

@RishabhJain2018 @Ram81 I would love to work on this if this seems useful.

I would like to work with this :) 

I would like to work on this

Working on this

:) Closing this since it is not needed.
Repository: Cloud-CV/EvalAI
Issue: Getting error while Build and run the Docker containers. (docker-compose up --build) 
Root cause: Environment Incompatibility


![Screenshot (62)](https://user-images.githubusercontent.com/67184718/109263954-32bae200-782a-11eb-868d-5871aa3292d3.png)
_Thanks for taking the time to report a bug in the EvalAI project, or propose a new feature! Before creating a new issue, please do a quick search to check that it hasn't already been filed on the [issue tracker](https://github.com/Cloud-CV/EvalAI/issues)._

If you'd like to report a bug, please include the following information:
- the expected behaviour
- the observed behaviour
- steps to reproduce the problem
- which operating system and/or browser you're using, if relevant
- (if possible) copies of the console/server logs

On the other hand, if you'd like to propose a new feature:
- please describe, as clearly as possible, what you'd like to see
- (if possible) include mockups/screenshots, if these would help convey the meaning better

Thanks!


Are you sure you have started the docker service?

> Are you sure you have started the docker service?

Yes I have started docker service

@AbhisekOmkar our docker setup is tested and verified on Linux and OSX. On windows there are multiple issues. It would be easier if you could set up EvalAI on Linux machine. If you face any issues after switching OS let us know we will be happy to help.

Yeah sure I have tested it on Linux its working fine thnx for your support
Repository: Cloud-CV/EvalAI
Issue: Add Docstring to all the API Endpoints
GitHub labels: enhancement

easy_to_fix

backend


@deshraj I would like to work on the issue.

@souravsingh Sorry, this one is already assigned to @TroJan. Feel free to pick any unassigned issue. :-) 

@souravsingh : you can work on this, I am little busy with other issues.
@deshraj : please re assign it.

Oh Ok. @souravsingh If you are interested then you can work on this issue. :-) 

@souravsingh: Status update? Are you still working on this issue? 

@dexter1691 I am working on it

@souravsingh any updates on this? I think it has been really long time since you have been working on this. If you are not working on this, then please let me know so that I can assign it to someone else. 

@deshraj I got caught up with some commitments. I will finish this up right away.

@souravsingh looks like you are busy and hence I am unassigning this issue. 

@deshraj I almost forgot the issue. I will send a WIP PR right away.

Oh Ok. Thanks for letting me know. :-) 

I apologize for keeping this pending

Looks like @souravsingh is not able to work on this. Anyone interested in working on this issue is welcome. :) 

@deshraj Can I work on this?

Sure. Assigning it to you. 

@RishabhJain2018  are you working on this ?

@gauthamzz No, I am not working on this but @souravsingh is working on this.

Closing this for now. Will reopen it once there comes a requirement.
Repository: Cloud-CV/EvalAI
Issue: Error while building and run the Docker Container (docker-compose up --build)
Root cause: Environment Incompatibility

Ended up with this error while building Eval in my local ubuntu.
![2021-03-01_11-11](https://user-images.githubusercontent.com/50979887/109456458-e5837e00-7a7e-11eb-8ea9-0869329a99fa.png)


@piyushsingariya try  using the command `docker-compose build` instead.

@burnerlee I am ending up with the same error. :sweat_smile: 

did you try `apt-get update`?

Yes I tried that but this didn't solve the concerned error

Delete the whole repo and clone it once again it might work

@AbhisekOmkar Thanks! It actually worked...

Encountered a new error. I killed the service running on the port just to be sure.
![2021-03-03_13-24](https://user-images.githubusercontent.com/50979887/109772349-e4d81c80-7c23-11eb-9f2e-3da1a813e42d.png)
Could you help me with this?

@piyushsingariya This will help
 https://evalai.readthedocs.io/en/latest/faq(developers).html#q-getting-the-following-error

To solve, execute the following commands:

npm config set registry=registry.npmjs.org
If the above does not work, try deleting them by following commands:

npm config delete proxy
npm config delete https-proxy

Thanks @AbhisekOmkar @gautamjajoo It worked finally. Thanks a lot.
Repository: Cloud-CV/EvalAI
Issue: Improvements in Auth Token Card
Root cause: Unimplemented Feature Gap

- [x]  
 Cancel/Download not working
 (https://github.com/Cloud-CV/EvalAI/pull/3201 solved the issue)
- [ ] Remove the Auth Token Label 
- [ ] Align copy to clipboard with the auth token

@Ram81 @RishabhJain2018 


![Screenshot from 2021-03-02 23-12-55](https://user-images.githubusercontent.com/24366008/109691030-021dd400-7bad-11eb-98ab-a9440e6e706c.png)
 

@gautamjajoo , I want to work on this : )

Sure @Alabhya268 you can pick it up.
Repository: Cloud-CV/EvalAI
Issue: Allow show and hide password 

Add support for showing and hiding password similar to login page in the update password card as well.

@Ram81 @RishabhJain2018 

![Screenshot from 2021-03-02 23-26-46](https://user-images.githubusercontent.com/24366008/109692975-1e227500-7baf-11eb-8333-31cf2b7b5679.png)


@gautamjajoo , I want to work on this.

@gautamjajoo , I want to work on this.
Repository: Cloud-CV/EvalAI
Issue: Profile completed percentage changing on refreshing
GitHub labels: bug

easy_to_fix

frontend

GSOC-2025

The profile completed percentage is changing sometimes 40% or 50% or 100% on refreshing the page.

@Ram81  @RishabhJain2018 

![Screenshot from 2021-03-02 23-34-56](https://user-images.githubusercontent.com/24366008/109693860-26c77b00-7bb0-11eb-958b-108f317f731d.png)


@gautamjajoo can you share a gif of the incorrect behavior or steps to reproduce it?

@Ram81 
![profile](https://user-images.githubusercontent.com/24366008/110359658-c333c680-8063-11eb-9689-cc9bacc6d0e9.gif)


Hi @gautamjajoo @Ram81,

I was able to reproduce this issue locally. The fluctuation occurs because the current logic calculates the percentage by iterating over *all* properties returned by the user API. Since the backend response structure can vary (metadata fields, order, etc.), the total count fluctuates on refresh, causing the percentage to jump between values like 40% and 100%.

I have fixed this by calculating the completion percentage based on a fixed list of relevant profile fields (`first_name`, `last_name`, `affiliation`, etc.) instead of the dynamic response object.

I have verified the fix locally, and the percentage is now stable. I am raising a PR for this shortly.
Repository: Cloud-CV/EvalAI
Issue: Hosting a private competition

Thanks for the great initiative!

I am currently reading through the EvalAI documentation and I am missing an important aspect in the documentation: Is it possible to host a private competition on EvalAI, i.e. select the participants (e.g. by giving them a password, or an email invite or similar)? Many thanks in advance for getting back.


Hi @juliuskittler , No, it is currently not possible. Thanks!
Repository: Cloud-CV/EvalAI
Issue: Adding GUIs in the home page

We can introduce some basic user-friendly GUIs for "Hosted AI Challenges", "Users", "Submissions" and "Organizations" in the home page, as it will enhance the first look of the website in front of users.
Currently the home page has a lot of empty space above the mentioned sections.
![Screenshot (24)](https://user-images.githubusercontent.com/67580203/110244889-dd4ba700-7f86-11eb-8a5c-0ccd75791791.png)



@devanshraj300 can i work on this? 

Closing this since this isn't the priority for now.
Repository: Cloud-CV/EvalAI
Issue: Host can't see private submissions toggle not available when no public submissions are available on leaderboard
Root cause: Validation Gap

When the host goes to the leaderboard, he is not able to see the private submissions at first. And if no public submissions are available, then it directly shows that no results are available. The toggle to include/exclude private submissions does not appear in this case. Therefore, if for a particular challenge all the submissions are private, then the host would see no submissions in the leaderboard, and nor would he get a toggle to include private submissions in his view.
Therefore, the toggle to include/exclude private submissions also be available if `leaderboard.length === 0`


![Screenshot from 2021-03-09 17-19-59](https://user-images.githubusercontent.com/55936223/110467038-8536b100-80fc-11eb-9f39-40e0abbaa8dc.png)


@Ram81

@burnerlee thanks for pointing out the issue. Would you be interested in working on this issue?

Yeah sure, should the labels of private and baseline submission appear here? Or only the Toggle option?@Ram81
Repository: Cloud-CV/EvalAI
Issue: I can't login rencently
Root cause: Environment Incompatibility


![error](https://user-images.githubusercontent.com/32010097/110633136-5d6e4880-81e3-11eb-9e0c-85a240b65897.png)
I want to evalue the model on trackingnet and before I can login and submit the result.


@wsumel are you using some proxy server to connect to EvalAI?

@wsumel Could you please check/share the terminal logs of the `Django` container?

Thank you so much, I can do it recently

hi, I m very new in this project but i m very interested to learn. I tried to set up the EvalAl on my system but i cant login. The Django container does not run . Can some one please help me with it?
<img width="957" alt="Screenshot 2021-09-12 135018" src="https://user-images.githubusercontent.com/87220986/132979386-27dbc676-30c9-46ea-8d26-f5c19786dd9e.png">



Closing this since it is a very old issue.
Repository: Cloud-CV/EvalAI
Issue: [Frontendv2]Add testimonials, statistics on the home page

Currently there is no testimonial stats on homepage.

@Ram81, @RishabhJain2018  

Frontend module for testimonials already exists but It only shows hard-coded data.
I could not find data to fetch with it.
Is there an api or model exist to fetch the data or do we need to make one ?

I want to work on this.

@Alabhya268 i am guessing we'll have to make an api to show some live statistics like number of submissions, 
number of challenges hosted ,etc etc. 

As for testimonials i'm not sure what needs to go there.


Closing this since we are not working on Frontend_v2
Repository: Cloud-CV/EvalAI
Issue: Feature: Public Profile User + Follow Users + LeaderBoard Custom Sorting + Editorial on Past Challenges
Root cause: Unimplemented Feature Gap

Hey developers,

In the current scenario, the public profile of the user or the team does not exist. The idea is inspired by the public profile on codeforces.com (https://codeforces.com/profile/tourist) as an example.

**Public Profile User**
The public profile could contain the basic information with the past contest history, some of the best ranks in the past competition. The overall activity by the user. The public profile can be linked to https://evalai-forum.cloudcv.org/ profile and the blogs/discussion on approaches could be linked with the profile for better community bonding.

**Follow Users**
By introducing a public profile the concept of making friends can be introduced to follow the ranking of the followed users on the leaderboard. Users can follow other users.

**LeaderBoard Custom Sorting**
This also introduces an option to view only friends or their respective teams on the leaderboard. or Sorting by affiliation.

**Editorial on Past Challenges**
By the addition of Editorials by the winner or the host users on the challenge page of past contests would increase the engagement of the community and thereby linking of the editorial with a blog post on the forum page where all users can discuss their approaches to the contest.

**Rating**
Rating of Users based on their performance can be introduced not necessarily numeric but the badges like Consistent User, Top performer badges could be introduced.

@Ram81 @RishabhJain2018 I would like to work on the above features and was looking for some suggestions and a heads up to work on the above features. 

These can be good features, but I wonder if they align with the overall web vision. If they allow you to work on these proposed features, I can help in building them.
Repository: Cloud-CV/EvalAI
Issue: [Backend]Add boolean check in github challenge update API to avoid overwriting annotations uploaded using cli
Root cause: Unimplemented Feature Gap

### Description

Currently, when challenge hosts update the challenge using github based challenge workflow the test annotations get overwrite after each update. We also allow hosts to update annotations using EvalAI cli. When a challenge host updates the annotation using CLI we should not overwrite the annotations passed from github based challenge update API.

### Deliverable

- [ ] Add `annotations_uploaded_using_cli` field in challenge phase model
- [ ] Set `annotations_uploaded_using_cli` to `true` when annotations are uploaded using CLI
- [ ] Add check in update github based challenge API to look up whether annotations were uploaded using CLI before overwriting annotations
 

@Ram81 can I please work on this issue?

@Ram81 is this issue still free for picking?


@Ram81 @RishabhJain2018  ping, may I start working on the issue if required

@Ram81 Can I work on this issue?
Repository: Cloud-CV/EvalAI
Issue: [Backend] Add start evaluation worker function call in challenge creation using github API
Root cause: Unimplemented Feature Gap

### Description

When a challenge is created using github based challenge creation pipeline, follow the steps mentioned [here](https://github.com/Cloud-CV/EvalAI-starters#create-challenge-using-github) to use the set up, we don't automatically start evaluation workers for the challenge. We should make a `start_worker` celery task call when a new challenge is created using github based set up. [Here's](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L1472) the reference to how we are doing it for challenge creation using zip file.

### Deliverable

- [x] Backend API changes to trigger `start_workers` celery task when challenge is created using github based setup.

@Ram81, Can I work on this issue?

Hey @afif-fahim, Sure, you can pick this issue.

@Ram81 @afif-fahim can i also try out this issue

@shashwatdh PR #3316 is opened for this issue. You can try any other issue from the issue tab.

PR Raised #3316 

@Ram81, Can I work on this issue?



@Ram81 are you still working on this project


@Ram81 
Is this up for Grab??


Hey @Ram81 can I work on this?
Repository: Cloud-CV/EvalAI
Issue: Add execution time of submissions on the leaderboard
Root cause: Unimplemented Feature Gap

**Issue:**
We want to display the execution time of the submissions for a specific phase split leaderboard.

**Tasks:**
- [x] Add a boolean field `show_execution_time` with `default=False` in the ChallengePhaseSplit model.
- [x] Add the field in the `ChallengePhaseSplitSerializer`.
- [x] While filtering the query set in the `calculate_distinct_sorted_leaderboard_data` function, use the above field to annotate the query set with the `execution_time` field.
- [x] On the UI in the `leaderboard.html`, add a new column with name `Execution time (sec.)` for the execution time before `Submitted at` and use `challenge.selectedPhaseSplit.show_execution_time` to show and hide that column with the values.

Please feel free to ask if you have any doubts/questions.

**Note:** This is a high _very priority issue_, please pick it up only when a PR can be opened within 24 hours.

@burnerlee Would you like to work on this?

@RishabhJain2018 yeah sure

I'll start right away

@RishabhJain2018 @Ram81 could you please tell me how to run makemigrations using docker-compose?

figured it out

@RishabhJain2018 @Ram81 made a PR for the above changes. Do we also need to allow user to sort the leaderboard table according to the execution time? Currently not implemented this sorting feature, as was not mentioned in the issue

>Do we also need to allow user to sort the leaderboard table according to the execution time? Currently not implemented this sorting feature, as was not mentioned in the issue

@burnerlee Yes, please add that as well.

> figured it out

@burnerlee could you also tell me how to do it in Linux?

Hey @Alabhya268 sure, actually i executed the command by entering into the docker container using exec... Start evalai using docker-compose up. In another terminal, run `docker ps` to see all containers. Copy the id of the django docker container. then use the command `docker exec -i -t <container-id>` and would be inside the container. Then simply run `python manage.py makemigrations`. Migrations are applied automatically when we run evalai, so restart it once.

https://stackoverflow.com/questions/33992867/how-do-you-perform-django-database-migrations-when-using-docker-compose see this for reference
Repository: Cloud-CV/EvalAI
Issue: Make the execution time human readable on UI.
Root cause: Unimplemented Feature Gap

**Issue**:
We now show the `Execution time` on the leaderboard but it is in seconds. We would like to make it more human-readable on UI like for `3600` seconds, it should display 1 hour.
For e.g. in the below screenshot, `134983.881963 seconds` isn't readable.

![image](https://user-images.githubusercontent.com/12206047/111252407-701fbd80-85e7-11eb-9476-03d3e5193019.png)

**Tasks:**
- [ ] Convert the seconds from the API to human-readable format in the controller.

Please feel free to ask if you have any doubts/questions.

Note: This is a high very priority issue, please pick it up only when a PR can be opened within 24 hours.

@Alabhya268 Would you like to work on this?

Yes @RishabhJain2018  I want to work on this.

@Alabhya268 There is one more issue, the sorting on the UI is off. Can you please check that as well.



> @Alabhya268 There is one more issue, the sorting on the UI is off. Can you please check that as well.

@RishabhJain2018 , I will look into it.

> @Alabhya268 There is one more issue, the sorting on the UI is off. Can you please check that as well.

@RishabhJain2018 isn't the sorting with execution time not working? I tested it in my PR actually 😅

@RishabhJain2018 @Ram81 , execution can be converted in day, hour, min and sec but what should I about time after decimal ? How should it be represented ?

> @RishabhJain2018 isn't the sorting with execution time not working? I tested it in my PR actually 

Hey @burnerlee, No it isn't working correctly. Check here -- https://eval.ai/web/challenges/challenge-page/580/leaderboard/1631

> execution can be converted in a day, hour, min, and sec but what should I about time after decimal? How should it be represented?

You can drop the time after decimals.

> Hey @burnerlee, No it isn't working correctly. Check here -- https://eval.ai/web/challenges/challenge-page/580/leaderboard/1631

oh sorry, i checked it with 2 values and it seemed to work correctly to me that time 😅
Repository: Cloud-CV/EvalAI
Issue: Add tests for testing urls in accounts app.
GitHub labels: enhancement

GSOC-2019

easy_to_fix

backend

The tests file must be located at `tests/unit/accounts/test_urls.py`. Create one, if it doesn't exists.
For reference please use http://stackoverflow.com/questions/18987051/how-do-i-unit-test-django-urls

@deshraj @RishabhJain2018 anyone working on this ?

@anweshknayak No, I don't think so. If you want to work then please ask **deshraj** or **TroJan** to assign it to you.

@anweshknayak You can work on it. Assigning it to you. :-) 

@RishabhJain2018 as you said django-allauth is already a tested. close this ? 

@anweshknayak there is one url in accounts app that needs to be tested. Please see this https://github.com/Cloud-CV/EvalAI/blob/master/apps/accounts/urls.py#L7