zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: Feature Request: Show accuracy/submission stats on the challenge page
Root cause: Unimplemented Feature Gap

People who visit a challenge page often want to know how active it is and what the rate of progress is (when did state of art last improve). 

Example on Codalabs: https://competitions.codalab.org/competitions/20687#learn_the_details


can i take this?


Can i work on this one?

@RishabhJain2018 can you give some details for this issue?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Inaccuracy in page number in Existing Teams
GitHub labels: enhancement

GSOC-2019

easy_to_fix

frontend

good first issue

### Observed Behaviour
When we have less than 10 teams on existing teams page, the intial page count is shown as 0. (Refer to the screenshot) 

### Expected Behaviour
The initial count should be 1.

### Steps to Reproduce
Go to any challenge in which you have not particpated. Select the `Participate` tab.

### Deliverables
- [ ] Update the intial page number on the participant side.
- [ ] Check for any other pagination issue and fix that too.

@deshraj @RishabhJain2018 

![capture](https://user-images.githubusercontent.com/24366008/52141011-74442780-267a-11e9-991e-d08d086ae9aa.JPG)


@RishabhJain2018 @gautamjajoo  I want to work on this issue.

@Sanji515 Sure! You can work on this issue.

Since there is no progress, Can I take this now?

@anuj-rathore It's already under review.
Repository: Cloud-CV/EvalAI
Issue: Rewrite Participant Team api to remove dependency of challenge
Root cause: Architectural Constraint

As of [b02d43](https://github.com/Cloud-CV/EvalAI/tree/b02d433b21e34a4c39985c6e490614d02435cfce), Participant model has foreign key to challenge. This means that to create entry for participant a challenge should exists. This could have been handled by making challenge a nullable foreign key, but this is not the way to do it.

Proposed solution involves a many to many key in challenge to participant team. Flow will be something like 

* Create a participant entry(without challenge dependency)
* When user clicks on `Participate in Challenge`, then pass challenge id and participant id to do entry in many to many mapping.

So this issue will involve updating participant team api to remove its dependency on challenge model. From now, participant will be independent of challenge. Mapping participant with challenge will be done in later release.

This also involves changing the url patters and tests. Also migration will be created for the same.

Fixed in #207
Repository: Cloud-CV/EvalAI
Issue: Unnecessary notification for unverified users
GitHub labels: bug

GSOC-2019

frontend

good first issue

### Observed Behaviour
When an un-verified user tries to see his profile, he is shown a blanck notification.

### Expected Behaviour
Remove the notification.

### Steps to reproduce
Create a user and go to the profile section. You could see a black notification popping out

@deshraj @RishabhJain2018 

![capture](https://user-images.githubusercontent.com/24366008/52142675-54fbc900-267f-11e9-8c29-4a9328d2cb44.JPG)


I'd like to solve this bug @gautamjajoo 

@yashdusing Sure! You can work on this.

I believe @yashdusing you are still on an issue. Hence this is still unassigned :)

Can I work on this?

Sure! You can work on this.
Repository: Cloud-CV/EvalAI
Issue: Update the Documentation for Monthly Submission Limit Per Team for Challenge Creation

### *This is Issue is only for Newbies i.e. who haven't been assigned any issues yet*
Deliverables:
- [ ] Add field for *Monthly Submission Limit Per Team* to [`challenge_config.yaml` in EvalAI-Starters](https://github.com/Cloud-CV/EvalAI-Starters/blob/master/challenge_config.yaml)
- [ ] Update the [Challenge Creation Documentation](https://github.com/Cloud-CV/EvalAI/blob/master/docs/source/challenge_creation.md) for field added in [challenge_config.yaml](https://github.com/Cloud-CV/EvalAI-Starters/blob/master/challenge_config.yaml)

Important Links:
- EvalAI Website: https://evalai.cloudcv.org
- EvalAI Github repository: https://github.com/Cloud-CV/EvalAI
- EvalAI Docs: http://evalai.readthedocs.io/en/latest/
- Gitter Channel: https://gitter.im/Cloud-CV/

Hi, I'm new to CloudCV. can I work on this issue?

@Mike-Dai Sure! you can work on this issue.

I am a newbie can i work on these issue please
Repository: Cloud-CV/EvalAI
Issue: Disable user to participate in a challenge when one of the user is unverified
Root cause: Validation Gap

### Observed Behaviour
When a verified user adds an unverified user to his team and then tries to participate in a challenge, he is successfully able to submit his entry to the challenge.
Please see the [screencast](https://drive.google.com/file/d/0B1bLU2zTKVNmRUhRSkVpSGRheEFQSmVHWTdUMnhaWnNXd2I4/view?usp=sharing) for reference.

### Expected Behaviour
If the user hasn’t verified and is invited to a team, then he/she cannot participate in the challenge. They have to first verify their email address.

@deshraj @RishabhJain2018 

@gautamjajoo Can I take this one?

Good Catch @gautamjajoo .
@KhalidRmb You can work on it.

@RishabhJain2018 @gautamjajoo We can consider being a verified user as a pre-requisite for being invited into any team. If not verified, we can display an appropriate message (at the invitation side).
Either that or to disable the team name in  the participation section until all team members are verified. What do you think?

@KhalidRmb We should allow all the users to be added in the team irrespective of the fact that the email address is verified or not. If the user hasn’t verified and is invited to a team, then he/she cannot participate in the challenge. They have to verify their email address.
cc: @RishabhJain2018 

I see. In that case, we can throw an appropriate warning while inviting (i.e, you can't participate until the user is verified) and follow through by not displaying/throwing an appropriate error when a user tries to participate through the team under question. 
**P.S:** I might be missing something, but isn't the whole purpose of creating a team to participate in a challenge? So, I thought a user is not to be added unless they're verified. The above step seems unnecessary. (As in, why at all add them to the team if they can't participate?).
@gautamjajoo  @RishabhJain2018 

Is this issue still open. Can i work on it?

@akshay0505 I'm already doing this one. Take up other open issues :)

@gautamjajoo Hi, I'm new to the CloudCV community and would love to get started with this issue. Has this been resolved already or is there room for improvement? :)

This is already fixed, hence closing.
Repository: Cloud-CV/EvalAI
Issue: Bug while editing evaluation script
Root cause: Algorithmic Error

@RishabhJain2018 

![out](https://user-images.githubusercontent.com/19167324/52169141-e8a1c800-2759-11e9-8cb9-abfb685faf30.gif)

Text under Evaluation Criteria disappears.

@RishabhJain2018 I want to work on this issue.

@Sanji515  I have solved the issue and opened a PR for it.

Hi @Ayukha 
Thanks for opening an Issue and Making a PR for this!
Can you please correct the title `evaluation script` -> `evaluation criteria`, as `evaluation script` is used for Submission Evaluation. Please See [docs/source/challenge_creation.md](https://github.com/Cloud-CV/EvalAI/blob/master/docs/source/challenge_creation.md) and [docs/source/submission.md](https://github.com/Cloud-CV/EvalAI/blob/master/docs/source/submission.md)

Hi @vkartik97 
I think the title is correct as the bug was observed while trying to edit the evaluation script.
Please let me know if I'm missing something :)

@Ayukha 
The Title is correct!
Apologies :)
Repository: Cloud-CV/EvalAI
Issue: Updating numpy in requirements/worker.txt
Root cause: External Dependency Breakage

>An issue was discovered in NumPy 1.16.0 and earlier. It uses the pickle Python module unsafely, which allows remote attackers to execute arbitrary code via a crafted serialized object, as demonstrated by a numpy.load call

Reference: 
- [Bug 1122208 - (CVE-2019-6446) VUL-0: CVE-2019-6446: python-numpy: NumPy uses the pickle Python module unsafely, which allows remote attackers to execute arbitrary code](https://bugzilla.suse.com/show_bug.cgi?id=1122208)
- [numpy load function with evil data will cause command execution #12759](https://github.com/numpy/numpy/issues/12759)
- [https://www.securityfocus.com/bid/106670](https://www.securityfocus.com/bid/106670)

Deliverable:
- [ ] Update numpy to 1.16.1 in [requirements/worker.txt](https://github.com/Cloud-CV/EvalAI/blob/d5588343d7f17cfe5ea787f8abb716b79ce70037/requirements/worker.txt#L3)

Please feel free to work on this issue and let me know if you have questions. :)

@vkartik97 Can I work on this issue?


@namish800 Sure! you can work on this issue.

If I am right I just have to change the version of numpy in worker.txt from 1.14 to 1.16.1

@namish800 
Yes
Repository: Cloud-CV/EvalAI
Issue: After deleting a challenge, redirect to Dashboard.
Root cause: Unimplemented Feature Gap

**Current Behaviour:**
After deleting a challenge, the page does not change and the user can still modify the contents of the page which obviously won't persist. 
**Expected:**
As the challenge has been deleted, it makes more sense to redirect the user to some relevant page like the dashboard or the Hosted Challenges page.
@RishabhJain2018 

**Deliverables:** 

- [x] Add necessary frontend code to redirect the user to the aforementioned page after confirming the deletion.

@KhalidRmb @RishabhJain2018 I want to work on this issue.

@KhalidRmb @RishabhJain2018 can i work on this issue?

@KhalidRmb @RishabhJain2018 can i work on this issue? 
 

I have written the code for this, if you want i can raise PR for this. This is my first time contributing and i don't know the procedure.

Hi! @ArshdeepSingh98 Sure, since you've already worked on this make a PR, but from next time it would be better to wait until it's assigned. :) Here's the process to make a PR: https://github.com/Cloud-CV/EvalAI/blob/master/.github/CONTRIBUTING.md.

#2106 https://github.com/Cloud-CV/EvalAI/pull/2106/commits/b37b6c96185d41ee14e4f0574b5b54f9cb556ef1 Deleting challenge redirects to dashboard
Please review this

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Requirement Incompatibility
Root cause: Configuration Error


Following warnings are show during `docker-compose build` when I was doing a fresh install. Certain packages in requirements are incompatible to certain other and would require changes in version no for certain package. I fell a long term solution other than using requirements.txt can also be thought about

```
docker-compose 1.21.0 has requirement PyYAML<4,>=3.10, but you'll have pyyaml 4.2b1 which is incompatible.
djangorestframework-expiring-authtoken 0.1.4 has requirement djangorestframework<=3.4.6,>=3.2.3, 
but you'll have djangorestframework 3.7.7 which is incompatible.
boto3 1.7.31 has requirement botocore<1.11.0,>=1.10.31, but you'll have botocore 1.10.12 which is incompatible.
```

#### Would you like to work on this issue?
Yes

> I fell a long term solution other than using requirements.txt can also be thought about

Can you please share your thoughts about it?

The python requirements management is currently being criticised by a no of people like Armin Ronacher(creator of flask). The issue is being tried to fix by PSF also. What I feel is a better solution than current requirements.txt is `pipenv`  [website](https://pipenv.readthedocs.io/en/latest/). It has features like check to detect old packages which are incompatible. 

I am not sure how suitable `pipenv` is for EvalAI project and may need setup.py also to fix requriements. Also now pipenv is not yet mature and is now just about 2 years into development.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Add permission for all the members of host teams
Root cause: Unimplemented Feature Gap

### Observed Behaviour
If a user makes a host team with additional members in it, and then hosts a challenge. It is observed that the user who made the challenge is only able to delete the challenge and any other member of the team can't do it.

### Expected Behaviour
Add support for giving permissions to other team members of the host team .

### Deliverables
- [ ] Allow other members of the host team to delete the challenge.
- [ ] Check whether the other host members can't perform any other task and fix that too.

@deshraj @RishabhJain2018 

@RishabhJain2018 @deshraj @gautamjajoo I’d like to take this up 

@gautamjajoo But I think that anyone besides the team lead shouldn't be allowed to make drastic changes to the host team. Instead of deleting, we could think of other privileges which are exclusive to the team lead (i.e. challenge creator/host)

Can I work on this issue @gautamjajoo @RishabhJain2018 @Ram81 

Can I work on this Issue, also how to add additional members in host team ?

@yashdusing You can work on this! 

@yashdusing Are you working on the issue?

@gautamjajoo I have my exams coming up so I won’t be able to push out any commits for now. Is it alright if I solve the issue after 2 weeks?

@yashdusing Maybe there would be new issues till then and you can take them up! Till then this issue is open!!

@gautamjajoo  I would like to take this on. I've finished my prev assignments and just waiting for a reply.

I am more than happy to take on this issue. I haven't have any issue assigned and always looking for an open issue for me to work on! :) @gautamjajoo 

So, right now only the creator can disable the challenge, delete the challenge or make any edits to the phase details or to the challenge details. So, I think we can make all the above available to the host team members as well. (But this will remove any exclusivity the creator has over the challenge.) 

@gautamjajoo Maybe disabling the challenge is to be kept exclusive to the creator as suggested by @yashdusing since it's rather drastic. We could add permissions for other hosts to edit challenge details & phase details, including changing the visibility of the challenge.
@RishabhJain2018 Any suggestions?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Create api to map a participant team with challenge.
Root cause: Unimplemented Feature Gap

This is a big change. This will involve the following

* create a many to many mapping between __Challenge__ and __ParticipantTeam__. key can exist in either model, but we will be keeping it in Challenge model, because queries like _list all teams that have participated or are participating in a challenge_ will be more common and frequent than _get all challenges in which a team has participated or participating_. Both are easily retrievable but to facilitate convenience, __Challenge__ will have many to many key to __ParticipantTeam__ model
* migrations for the above
* an api which takes challenge id and participant team id in url and maps that challenge with a participant team. API is _POST_ only
* tests for the api


Fixed in #210
Repository: Cloud-CV/EvalAI
Issue: Disable submissions from UI if the challenge is Docker based
Root cause: Unimplemented Feature Gap

## Current Scenario

Currently, a user can submit to both docker based and non-docker based challenges using the UI. We'd like to restrict the user to submit to docker based challenges using the CLI.

## Expected Behaviour

The user can't submit to a challenge using the UI.

## Deliverables

- [ ] Add a check in the `challenge_submission` API to check if the challenge is docker based or not. If yes, then don't create a submission instead return a response saying `The challenge {} requires uploading code in the form of docker images. Please use EvalAI-CLI to make a submission.format(challenge.title)`. The response should be returned for both challenge host as well as participants.

- [ ] Add the test cases
- [ ] Add a check on UI using the field `is_docker_based` from the challenge model to disable all the form attributes in the submission page if `is_docker_based == true`.

Please feel free to ask in case of doubts.

Can I work on this issue?

sure!

@RishabhJain2018 
>Add a check on UI using the field `is_docker_based` from the challenge model to disable all the form attributes in the submission page if `is_docker_based == true`

Instead of disabling Form, can I just display `The challenge *title* requires uploading code in the form of docker images. Please use EvalAI-CLI to make a submission` below `Submission Guidelines` and hide the Form?

Example:
![screenshot from 2019-02-09 03-04-15](https://user-images.githubusercontent.com/3920286/52507423-7c223f80-2c17-11e9-8e34-a37f76a89ccb.png)


Yeah, this looks good to me! Once you open the PR then we might need to iterate on the text :)
Repository: Cloud-CV/EvalAI
Issue: Outdated zip_challenge.yml leads to creation of challenge that is already expired
Root cause: Configuration Error

EvalAI takes metadata from zip_challenge.yaml file inside the test_zip_folder to create a new challenge. The start and end dates in the .yaml file are outdated. So if a new user when tries to create a new challenge, the challenge is already over when created. This creates a lot of issues, particularly while submission.

Deliverables:-
- Update the dates in zip_challenge.yml

Duplicate of #2151 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Add leaderboard sorting for featured challenges
Root cause: Unimplemented Feature Gap

## Current Scenario

Currently, the leaderboard fields can be sorted using the arrows under the `All Challenges Tab` as shown here https://evalai.cloudcv.org/web/challenges/challenge-page/161/leaderboard/483 but when the same challenge is in `Featured challenges` then the arrows don't appear as shown here: https://evalai.cloudcv.org/featured-challenges/161/leaderboard/483 

## Expected Scenario

The arrows should appear on both the challenge pages and the entries should be sorted using these arrows.

Hint:
https://github.com/Cloud-CV/EvalAI/blob/master/frontend/src/views/web/challenge/leaderboard.html#L42

@RishabhJain2018 I'd like to take this up

Sure! you can work on it.

This issue is still open :)

> This issue is still open :)

@RishabhJain2018 Can I work on this issue?

Sure! you can work on this.

@RishabhJain2018 , can I work this issue. I will put a PR now itself


@kurianbenoy since @sanji515 is already assigned for the PR I think you should ask to be assigned first before working on a PR

@yashdusing @kurianbenoy  ok, that's fine, waiting for another issue to work on :)

Sorry @Sanji515. I thought the issue was of high priority that's why I worked on it immediately. Also seeing 2 people being assigned to work on the issue, I thought maybe I can also come in and start working :smiley: 

Closing this one because we are not redirecting users to the featured challenges page now.
Repository: Cloud-CV/EvalAI
Issue: Add unit tests for leaderboard API
GitHub labels: enhancement

GSOC-2019

easy_to_fix

backend

good first issue

inactivity

## Deliverable

- [ ] Add unit tests for the corner case mentioned in PR #2100 in the leaderboard API.

I’d like to work on this issue

Sure! you can work on this issue.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.
Repository: Cloud-CV/EvalAI
Issue: Redirect users to Profile section on less than medium screen

### Observed Behaviour
When a user visits any page on less than medium screen and tries to see `Profile` section, he is redirected to Home page.  You could see the Profile section from homepage only.

### Expected Behaviour
The user should be directed to the `Profile` page. 

@deshraj @RishabhJain2018 

@deshraj @RishabhJain2018 @gautamjajoo Can I work on this issue?

@deshraj @RishabhJain2018 @gautamjajoo I would like to work on this issue ? 

@Sanji515 You can work on it!
Repository: Cloud-CV/EvalAI
Issue: Add Updates to docs
Root cause: Documentation Deficiency

- In [submission page](https://evalai.readthedocs.io/en/latest/submission.html) the `url namespace` link points to the django 1.10 docs
- In [Approve submission page](https://evalai.readthedocs.io/en/latest/approve_challenge.html) Reloading Submission worker is done by command `docker-compose reload worker `. Current the command reload is replaced by [restart](https://docs.docker.com/compose/reference/restart/)

@RishabhJain2018 @deshraj is this issue necessary?

Hi, @kurianbenoy Thanks for updating the documentation. Can you create two separate PR's for docs and leaderboard fix.
Repository: Cloud-CV/EvalAI
Issue: Incorrect implementation of encode_data() and decode_data()
Root cause: Validation Gap

I come across this issue when I am writing the unit tests for `encode_data()` and `decode_data()` functions in `apps/base/utils.py` due to the decrease in coverage. The error in implementation is not found because there are no unit tests and the functions are not used in other files.

First, the functions `base64.encodestring` and `base64.decodestring` have deprecated.
Second, without `.decode()`, the function `base64.encodestring(i).split("=")[0]` would raise the `TypeError: a bytes-like object is required, not 'str'`.
Third, without `.encode()`, the function `base64.decodestring(i+"==")` would raise `TypeError: expected bytes-like object, not str`.

The correct implementation should be 
`encoded.append(base64.encodebytes(i).decode().split("=")[0])` and `s = i+"=="; decoded.append(base64.decodebytes(s.encode()))`.

## Deliverables
- [ ] Correct implementations of `encode_data(data)` and `decode_data(data)`.
- [ ] Write unit tests for these two functions.



I will be working on this since I run into the coverage issue with PR #2054 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically closed as there is no further activity. Thank you for your contributions.