zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: couldn't approve a challenge 
Root cause: Documentation Deficiency


Hello I am runny EvalAI locally using docker. I created a challenge , but I couldn't approve it using the superuser django admin interface.  I checked the official documentation
https://evalai.readthedocs.io/en/latest/approve_challenge.html

However I couldn't approve the challenge since I couldn't access it (The title of the challenge is not a link in my Django admin interface)


- the expected behaviour : click on the challenge title and approve the challenge
- the observed behaviour: the challenge title is not a link so I couldn't click on it

<img width="1201" alt="django admin" src="https://user-images.githubusercontent.com/33626402/53823314-d17d1280-3f71-11e9-8a74-a7b7e36a3d5a.png">

Thanks!


Hi! @nadzma .To approve a challenge, you can click the hyperlink in the 'ID' column, scroll down and check the '`Approved by Admin`' checkbox. 

@KhalidRmb  Thank you so much for your answer

@nadzma Was your issue resolved?

@gautamjajoo  Yes it was resolved

Closing this since it is resolved. @nadzma Please free to ping the gitter channel for any other issue you face during installation.
Repository: Cloud-CV/EvalAI
Issue: Misplaced logout button in mobile view

# Logout button falls out of the menu bar in mobile view

### Observed behaviour : 
![screencapture-evalai-cloudcv-org-web-host-analytics-2019-03-08-04_19_57 1](https://user-images.githubusercontent.com/20144724/53995095-0481fa80-415a-11e9-8ad1-82013b5bf335.png)

### Expected behaviour : 
![4](https://user-images.githubusercontent.com/20144724/53995587-7e66b380-415b-11e9-809d-f30aab5c792c.png)

OS: Android Pie ( tested on Motorola G5 Plus )

P.S. If this is by design, it is a bit inconvenient design, I guess.

@cafeshoes I would like to work on this, can you tell me the width, height of the mobile that you tried it on, so that I can inspect the page.

After inspecting the source code I came to the conclusion that in the file.EvalAI\frontend\src\views\web\partials\sim-header.html

![Capture](https://user-images.githubusercontent.com/41429261/54074266-766f5680-42b6-11e9-8595-f9514ebadde7.PNG)


this is the reason that logout button is misplaced. <a> contains the logo and <i> contains the hamburger button for menu.

Right now I am thinking of updating css in such a way that the media screen won't show the logo while it will show only the hamburger button.


@cafeshoes @GearGit #2162 A similar issue already exists.

Thanks, @GearGit for reporting it but I'm closing it since it is the duplicate issue of #2162 .
Repository: Cloud-CV/EvalAI
Issue: Fix the feature of editing challenge details
Root cause: Validation Gap

## Current Behaviour

When a challenge host tries to edit the challenge details after creating a challenge then the details aren't edited.

## Expected Behaviour
The challenge should be able to edit the challenge details in the challenge.

## Hint: Please use the package used in this PR: https://github.com/Cloud-CV/EvalAI/pull/2144




@RishabhJain2018 I would like to work on this :).

Sure! It would be really nice if you can fix this asap.

Noted. 
Just a clarification: What all details exactly do you mean? _(Also, how is the package to be used here? Should I add end/start dates edit functionality in this as well?)_
1. Currently, only the host team creator can edit anything in the challenge (_We thereby need to add a permission class for host team members as well)_.
2. Also, the edit is successfully made only upon going through `All Challenges` page and displays an error through `Hosted Challenges` page :(**Reason identified** - because the `localStorage` is not updated in the `hostedChallengectrl.js`.)

It seems that issue #2088 and #2025 are related to this. @RishabhJain2018 

If everything's alright I can go ahead and make a PR to satisfy both 1 & 2 above.
Repository: Cloud-CV/EvalAI
Issue: Align EvalAI in SideNav in mobile view

- the expected behaviour

The header of SideNav ( i.e., EvalAI ) should be vertically centred.

- the observed behaviour

![Screen Shot 2019-03-10 at 18 48 00](https://user-images.githubusercontent.com/22499864/54085576-b0e3fc80-4365-11e9-9cad-d1733b10df0f.png)

I would like take this issue up.
Repository: Cloud-CV/EvalAI
Issue: Active states of links in SideBar
Root cause: Algorithmic Error

The active states in SideBar are not properly set. The issue exists for almost every link.

- the expected behaviour
Example: The dashboard link should be in active state ( bold font ) when on dashboard as shown in the screenshot below.

- the observed behaviour
![Screen Shot 2019-03-12 at 01 44 58](https://user-images.githubusercontent.com/22499864/54154699-8a5aba00-4468-11e9-84ee-02dd218cb092.png)


@RishabhJain2018 @geekayush Can I work on this issue?

@Sanji515 Let @RishabhJain2018 confirm the issue then you can surely work on it.

Hi, @geekayush Thanks for reporting this issue. Since you've reported the issue would you like to work on it?

@geekayush @RishabhJain2018 I would like to work on this if no one is assigned.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Add status in contact messages
GitHub labels: enhancement

GSOC-2019

easy_to_fix

backend

good first issue

## Current Behaviour

If there are multiple EvalAI admins then it is not clear in contact messages if one admin has resolved it or not.

## Expected Behaviour
Add a choice field in the contact model with status as `pending` and `resolved` having default as `pending`.

## Deliverables
- [ ] Add the fields in the model.
- [ ] Display it in Contact admin.

@RishabhJain2018 Can I work on this issue?

Sure!

can i work on this issue if it's not solved?
Repository: Cloud-CV/EvalAI
Issue: Remove ParticipantTeamMember and add team as Foreign key in Participant Model

__ParticipantTeamMember__ contains participant and team as foreign key to respective models. But this model is not required. Foreign key to team can be kept in __Participant__ model and then it will map a team member to a team. This model is increasing data redundancy and hence a overhead of maintaining duplicated data.

Also while considering the new design, we considered some scenarios(queries actually) like
* get all the members of a team
* get all the team in which a user is participating or has participated
* get all teams participating in a challenge

All these cases are handled in the new design. So this issue involves

* removing __ParticipantTeamMember__ and its reference in the project(models, imports)
* adding team as foreign key in __Participant__
* migration for the above


Merge in #220
Repository: Cloud-CV/EvalAI
Issue: couldn't add a new field  in challenge model
Root cause: Validation Gap

Hello  I am trying to add a new field in the challenge configuration file . I added a a "category" field in the yaml file. Then I added the field "category" in the challenge model.py and admin.py . However I have an error message "Error in creating challenge. Please check the yaml configuration!" when creating a challenge and I have this error when accessing the challenge from the admin:
ProgrammingError at /admin/challenges/challenge/
column challenge.category does not exist
LINE 1: ..., "challenge"."modified_at", "challenge"."title", "challenge...

I think that it's a migration problem but I couldn't solve it since I am running evalAi with docker. Could anyone help me ?
Thank you

<img width="707" alt="Capture d’écran 2019-03-14 à 10 31 29" src="https://user-images.githubusercontent.com/33626402/54345975-74a1ec00-4644-11e9-98f8-dfaebf53f84e.png">
Repository: Cloud-CV/EvalAI
Issue: Display GitHub stats in footer

 **The observed behaviour**
![Screenshot from 2019-03-13 18-00-49](https://user-images.githubusercontent.com/40494926/54291736-08d36b00-45d3-11e9-910e-9ce64ab986b3.png)
 
**The expected behaviour**
Count of stars and forks displayed properly.

@RishabhJain2018  can you help me with this?

@C-Anirudh Can you verify once more? It is working on my end! 

@gautamjajoo the stats aren't visible to me locally or in the [website](https://evalai.cloudcv.org/).

@C-Anirudh Which browser are you using?

@gautamjajoo I am using Google Chrome 72

@C-Anirudh It is working for me!
![Capture](https://user-images.githubusercontent.com/24366008/54607427-51c46d00-4a74-11e9-954a-97f74ec4bfd0.JPG)


@gautamjajoo It most probably is an error from my side, I will look into it again, thanks for looking into it!

@C-Anirudh Feel free to open if you still face the issue 

@gautamjajoo issue was caused by AdBlocker , sorry for the trouble.
Repository: Cloud-CV/EvalAI
Issue: Add unit tests for create challenge API
GitHub labels: enhancement

GSOC-2019

hard-to-fix

backend

inactivity

## Current Scenario
There are no unit tests for create challenge using zip [API](https://github.com/Cloud-CV/EvalAI/blob/c156f3d54c95861c4ac4eaef1d2e3419bab83a7a/apps/challenges/views.py#L488) 

## Deliverables
- [ ] Add unit tests for the API

I'd like to do this. @RishabhJain2018 May I?

sure @KhalidRmb . Please feel free to ask doubts if you have any.

Surely @RishabhJain2018 . Just wanted to remind you to take a look at #2172 comments when you can. Thanks!
 

yep, I've it on my list. I'll review it soon. Thanks for the reminder.

@RishabhJain2018 It seems that unit tests for this already exist:
https://github.com/Cloud-CV/EvalAI/blob/c156f3d54c95861c4ac4eaef1d2e3419bab83a7a/tests/unit/challenges/test_views.py#L2101
The coverage of the challenge/views.py file is also 100%. 
May I get another issue meanwhile? 

So, this issue to be closed right? @RishabhJain2018

Hey @KhalidRmb, I missed this -- the test cases are there but all the cases aren't covered in it.

Can I take this issue?


@sanchit-ahuja I'm working on this one. Will make a PR soon.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Display error on UI while creating challenge
Root cause: Validation Gap

## Current Behaviour
When there are syntax errors in the `yaml` challenge configs then the challenge creation using zip [API](https://github.com/Cloud-CV/EvalAI/blob/c156f3d54c95861c4ac4eaef1d2e3419bab83a7a/apps/challenges/views.py#L575) fails with 500 Internal Server error which should not be the case.


## Expected Behaviour
The API should give a proper error with 406 error code. The error should be displayed on the UI.

## Resources
For creating a challenge zip, please use [EvalAI-Starters](https://github.com/Cloud-CV/EvalAI-Starters)


@RishabhJain2018 Can I work on this issue?

Sure!

@RishabhJain2018 Should I show all the syntax errors of the yaml file on the UI (over toast) or just show the message saying `There is a syntax error on the yaml file, Please correct them before creating a challenge!`

@RishabhJain2018 Can you please confirm my comment?

Hi @Sanji515, It'd nice to show all the specific errors.
Repository: Cloud-CV/EvalAI
Issue: Create Participant object when saving Participant Team object
Root cause: Unimplemented Feature Gap

In flow of creating a ParticipantTeam, when a use enters a team name its __ParticipantTeam__ and __Participant__ object should be created

Presently the api to create a new participant team only supports creation of ParticipantTeam. 

So add a block in the api which creates a new __Participant__ object also. 

merged in #221
Repository: Cloud-CV/EvalAI
Issue: Display total number of participated participant teams in a challenge
Root cause: Unimplemented Feature Gap

# Current Scenario
We don't display the total number of participant teams in a challenge.
The API is already written in `analytics` app.

## Expected scenario
The total number of participant teams in a challenge should be displayed here:
![Screenshot 2019-03-14 at 5 10 11 PM](https://user-images.githubusercontent.com/12206047/54391764-3e805e80-467c-11e9-8483-c8923d8be420.png)


Can I work on this issue?

Sure!
Repository: Cloud-CV/EvalAI
Issue: Challenges not displayed on the challenge page if > 10
Root cause: Unimplemented Feature Gap

## Current Behaviour:
If there are more than 10 challenges on the challenge page then those challenges aren't displayed on the page.

## Expected Behaviour:
All the challenges should be displayed on the challenge page.

## Deliverables - 
- [ ] The API already returns the challenge data along with a URL for the next set of challenges.
- [ ] Make an API call to the next URL and append the data into the existing list in the controller.
- [ ] Terminate the appending once the `next` field is null in the API's response.

Can I work on this issue?

Sure!
Repository: Cloud-CV/EvalAI
Issue: Error while executing evaluation script 
Root cause: Validation Gap

I have a bug after submitting a solution

- the expected behaviour : After submitting a solution , I access "my submissions" and I can see my score , Execution Time , Result file , STDout file , STDERR file
- the observed behaviour : All these value are "None" , Show on Leaderboard: N/A 
<img width="878" alt="Capture d’écran 2019-03-15 à 16 49 16" src="https://user-images.githubusercontent.com/33626402/54524489-628cba00-4972-11e9-91bb-b8abdbedc683.png">
<img width="854" alt="Capture d’écran 2019-03-15 à 17 04 18" src="https://user-images.githubusercontent.com/33626402/54524490-628cba00-4972-11e9-9286-31e0d15b46bd.png">

When I run : docker-compose up --build , I got this message error
worker_1  | ModuleNotFoundError: No module named 'challenge_data.challenge_2'

When I submit a solution I got this error: 
worker_1  | [2019-03-18 12:45:37] ERROR Exception while receiving message from submission queue with error 'NoneType' object has no attribute 'get'
Thanks!


Hey @nadzma, Please restart the worker after approving the challenge from the backend. It'll work fine. I'm closing this issue, for now, please feel free to reopen it if you still face some problem.

Hey @RishabhJain2018    Unfortunately I still having the same issue by restarting the python worker:

<img width="1013" alt="Capture d’écran 2019-03-19 à 11 27 17" src="https://user-images.githubusercontent.com/33626402/54599178-61bf5b00-4a3a-11e9-989b-cd83e9ca0364.png">
<img width="605" alt="Capture d’écran 2019-03-19 à 11 27 35" src="https://user-images.githubusercontent.com/33626402/54599179-6257f180-4a3a-11e9-897c-7a132c93da75.png">
<img width="1010" alt="Capture d’écran 2019-03-19 à 11 28 03" src="https://user-images.githubusercontent.com/33626402/54599180-6257f180-4a3a-11e9-9df5-c5b6c2af14cf.png">
<img width="1018" alt="Capture d’écran 2019-03-19 à 11 28 16" src="https://user-images.githubusercontent.com/33626402/54599181-6257f180-4a3a-11e9-9461-1a02063a5d03.png">
Repository: Cloud-CV/EvalAI
Issue: Outdated Python 2 code while creating challenge using zip file.
Root cause: Validation Gap

**Observed behavior:** When the serializer is not valid while creating a challenge, the error message never gets sent due to incompatible python 2 code which raises an error. Instead, the control is redirected to the except block, sending a generic message.

**Expected behavior:** The serializer error should be sent in the response when appropriate.

**Reason:** This happens because because of this line https://github.com/Cloud-CV/EvalAI/blob/1dde5467e6cfc4c7338f0107757025f5f80d1fed/apps/challenges/views.py#L940
In Python 3, dictionaries  return "view" objects from the `values()` which are iterable but don't support indexing. In python 2, they returned lists.
@RishabhJain2018 @deshraj 

I'm working on this.
Repository: Cloud-CV/EvalAI
Issue: Display error message when Leaderboard schema is missing while creating challenge.
Root cause: Validation Gap

**Expected behavior**: A message like the following `"There is no leaderboard schema in the YAML configuration file. Please add it and then try again!"` with 406 status code should be returned when the schema is missing from the YAML file. This provides information to the user as to what needs to be done.

**Observed behavior:** A 500 internal server error.

**Deliverables:**
- [ ] Change the API to return appropriate response when the schema is not present.

@RishabhJain2018 @deshraj 



Thanks, @KhalidRmb for reporting it. Since you're working on another issue. This issue is open for all to work on.

@RishabhJain2018 can I take this up??

Sure @shiv6146 

Meanwhile @RishabhJain2018 please go through #2205 whenever you find time :+1: Thanks!
Repository: Cloud-CV/EvalAI
Issue: Add a feature to send emails to all the participants of a challenge
GitHub labels: GSOC-2019

hard-to-fix

frontend

backend

new-feature

inactivity

## Current Behaviour
If a `challenge host` wants to send an email to all the participants of a challenge then they have to download the `csv` file for all the submissions and send the email manually.

## Expected Behaviour
We want to automate this process in which the challenge host should have to fill the subject, body and any attachments and the email is sent automatically.

## Deliverables
- [ ] Add the required API's.
- [ ] Add the required frontend code. 
- [ ] Make sure that the feature is only accessible to the challenge host of that challenge.

@RishabhJain2018 Can I take this up ?


@RishabhJain2018 Can I work on this?

@RishabhJain2018 Can I work on this?

@Sanji515 You can take this up.

Hey @RishabhJain2018, this issue hasn't been active since half a month and no PRs have been made. I wrote some sample code for this, can I open a PR?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Add support for installing requirements from the requirements.txt file in the challenge config.
Root cause: Unimplemented Feature Gap

## Current Scenario
Since we're having challenge specific workers paired with challenge specific queues, suppose if a challenge host wants to install some custom requirements for a particular challenge, then we have to install it for all the challenge workers which should not be the case.


## Expected Scenario
There should be a file in the evaluation script `requirements.txt` which contains these requirements and the worker while starting up should check for this file and if this file is present then it should install the dependencies first and then load the challenge evaluation script.


Please feel free to ask if you've doubts.
For creating sample challenge use `EvalAI-Starters` repository.

@rishabhjain2018 can I work on this issue?

I would like to work on this. @RishabhJain2018

@RishabhJain2018 I'd like to work on this

@KhalidRmb You can take this up.

Sure! @RishabhJain2018 could you also please review #2172  and corresponding PR #2174 ? 

@KhalidRmb Since you are working on #2188, I'm sorry you can't work on this. As a piece of advice please do make sure that you've opened up the PR for the issue you were working on and then pick another issue.

This issue is open for all to work on. :)

Can I work on this issue?

Sure!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Add a feature to change the status of a submission by challenge host
GitHub labels: GSOC-2019

hard-to-fix

frontend

backend

new-feature

inactivity

## Current Scenario
If a challenge host wants to cancel the submission status of submission, then they can't do it. They have to approach EvalAI admin in order to do it.

## Expected Scenario
The challenge host should be able to change the submission status using UI.

@rishabhjain2018 can I work on this? 

Sure, I was about to mention you :)

Hey @RishabhJain2018 , I was going through the repo trying to figure out how it works and the changes I need to make. So as I've understood, I need to 
* give a provision for the "host" of the challenge in the UI to change the status of any submission (to *any* of the possible `STATUS_OPTIONS`?). 
* provide an API endpoint in django to change the status and save it. 

Did I miss anything? I've started working on the backend and will send a WIP PR soon.

yeah, this seems to be correct. For the API part you might want to check the existing API's before creating a new one :)

@RishabhJain2018 So the closest API is `change_submission_data_and_visibility` which changes the submission data. But this API needs the user that requested it to be a participant of the team that made the submission. This is not the case in this feature/issue. So I've created a new API named `change_submission_status`. Is that fine?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Command not copied properly
Root cause: Algorithmic Error

When a challenge is docker based then the command `evalai set_token <auth_token>` isn't copied properly. Basically, the token isn't copied.

 
<img width="1161" alt="Screenshot 2019-03-20 at 1 44 50 PM" src="https://user-images.githubusercontent.com/12206047/54706996-8ab51e00-4b16-11e9-9652-128e44c9789b.png">


Can I take this?

Sure! please fix this asap.
Repository: Cloud-CV/EvalAI
Issue: Add filtering package for and unit tests
Root cause: Unimplemented Feature Gap

## Current Scenario
Since we require #2129 urgently, hence the unit tests are missing in it. Also, the standard way to do filtering is using [django-filters](https://github.com/carltongibson/django-filter).

## Expected Scenario

Modify the API to use django filters and write the unit tests.

## Deliverables
- [ ] Add django-filters package in requirements.
- [ ] Modify the API to use django-filters
- [ ] Add unit tests for the API.

Hey! can I work on this issue?

Hi @sudharsana-kjl Welcome to CloudCV! Sure, you can work on this.

Can I be the next person on waiting list to work on this @RishabhJain2018 ?

Hey @RishabhJain2018! Do you want to use django-filter to ```get_submissions_for_challenge``` API as discussed in #2129 or do you want to add this to other APIs as well?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.