zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: Mention the version of python used in the project in the README file.
Root cause: Documentation Deficiency

Mention the version of python used in the project in the README file.

Good catch @HargovindArora :+1: .If you are interested in working on this issue then let me know, I will assign it to you. :)
Moreover, the instructions above are for opening the issue, you can remove them after reading and then open a issue by following these instructions.

@RishabhJain2018 Yes please, I would like to do it.

@HargovindArora Go ahead :)
Repository: Cloud-CV/EvalAI
Issue: Add API for showing the last submission time.
Root cause: Unimplemented Feature Gap

Expected:

- [x] To get the last submission time in a challenge.

- [x] To get the last submission time in a challenge phase.

- [x] To get the last submission time by a participant team in a challenge phase.

- [x] To get the last submission time by a user in a challenge phase.

@deshraj @taranjeet Please add if I have missed anything.
Repository: Cloud-CV/EvalAI
Issue: Remove pagination from challenge phase split list API
Root cause: Unimplemented Feature Gap

## Observed behavior

The Challenge Phase Split list API endpoint currently returns the challenge phase splits in a paginated manner. But while showing the leaderboard, we need the list of all the challenge phase splits requesting once. 

## Objective

Remove the pagination from this API (https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L357).

@deshraj : Can we instead of removing pagination, take a query parameter like noPage or some better name, which if presents, causes the api to return complete result set instead of pagination?
Repository: Cloud-CV/EvalAI
Issue: Change the email confirmation template according to current UI.
GitHub labels: enhancement

easy_to_fix

frontend

critical

## Current Behaviour
![email](https://user-images.githubusercontent.com/12206047/29171837-7cab1c2c-7dfb-11e7-9778-93876e6ae87b.jpg)
The Email confirmation template is according to old UI.

## Expected Behaviour
The email confirmation template should be according to the current UI.



I want to work on this.

Sure, go ahead!

Where can I refer the current UI?


Try signing up and clicking the email you receive in your inbox.
That should take you to the UI.

Also please set up these values in your dev.py file
`EMAIL_HOST`
`EMAIL_HOST_PASSWORD`
`EMAIL_HOST_USER`
`EMAIL_PORT`
`EMAIL_USE_TLS`

@RishabhJain2018 can you help with this https://github.com/Cloud-CV/EvalAI/issues/1131

@aayusharora Yes sure. I will fix that by tonight. :+1: 

@deshraj can I take this ?

@Ram81: looks like no one is working on this. Feel free to work on this.
Repository: Cloud-CV/EvalAI
Issue: Add approved_by_admin check in submission worker script.
Root cause: Validation Gap

**Need:** Since we need to only publish those challenges that are approved by admin, so it is also necessary that `submission_worker` should load only admin approved challenges.

**Solution:** Make the query parameters [here](https://github.com/Cloud-CV/EvalAI/blob/master/scripts/workers/submission_worker.py#L215) as
`q_params = {'published': True, 'approved_by_admin': True}`
Repository: Cloud-CV/EvalAI
Issue: Provided samples not working
Root cause: Documentation Deficiency

On creating a challenge using the provided [test_zip_file.zip](https://github.com/Cloud-CV/EvalAI/blob/master/examples/example1/test_zip_file.zip) , the following error is thrown.

<img width="344" alt="screen shot 2017-08-11 at 12 45 16 pm" src="https://user-images.githubusercontent.com/6565270/29204149-f7399332-7e92-11e7-8e9b-1099f020ca48.png">

Looks like the samples are outdated and need to be updated.

@the-dagger Thanks for pointing this issue. 
If you want to test this feature then you can use https://github.com/Cloud-CV/EvalAI-Examples repository.
Hence,I am closing this issue. :)

Sure, thanks! 
So should we link this repo in our Readme and delete the existing samples from the project folder? 

Yes, Please refer to issue #1146 and I am creating a new issue for deleting the outdated samples.

👍
Repository: Cloud-CV/EvalAI
Issue: Remove unnecessary files from examples folder.
GitHub labels: enhancement

easy_to_fix

Remove the folder `test_zip_file` and `test_zip_file.zip` from the `EvalAI/examples/example1` folder.

@RishabhJain2018 I would like to work on it.


@HargovindArora sure. assigning it to you.

closed #1165 via #1169 .
Repository: Cloud-CV/EvalAI
Issue: Add models to star a challenge
Root cause: Unimplemented Feature Gap

**Deliverable**:

- [x] Add a model in `apps/challenges/model.py` for starring a challenge.

**Attributes:** 
1. `user` - Foreign key to `User` model
2. `challenge` - Foreign key to `Challenge` model.

- [x] Add the migration file.

- [x] Register the model in django-admin.
Repository: Cloud-CV/EvalAI
Issue: Add feature to update submission metadata.
GitHub labels: frontend

backend

new-feature

priority-high

**Current Behaviour:** There is no feature to update the submission meta once a submission is done by the participant.
**Expected Behaviour:**
The Participant should be able to update:

- [x] Method Name

- [x] Method Description

- [x] Project Url

- [x] Publication Url

@RishabhJain2018 is it possible to add submission metadata that is viewable to the public? For example, if I make a submission to the Argoverse 3d Tracking leaderboard, can I add a reference to our corresponding publication?

The [KITTI leaderboard](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) has this functionality, for example, and also the [nuScenes object detection leaderboard](https://www.nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any).

Ideally there would be an "Expand" button and then more metadata about the publication would be visible.
Repository: Cloud-CV/EvalAI
Issue: Add Forgot password feature
GitHub labels: enhancement

GSOC-2019

medium-difficulty

frontend

Need to add a Password Change (forgotten password) feature. Refer API docs for change password endpoints

Can I try? Can you tell me how to start

@gauthamzz Sure, go ahead. Let me know if you get any problem.

@gauthamzz just to make you hands on, can you try issue #140? This is just to make you familiar with the client side code

@gauthamzz : Let me know if you are still working on it?

sorry i had a quick hectic week in college. i wont be able to do anything for next four days

Sure, no problem.

is anybody doing this ?

@anweshknayak I don't think so. You go ahead and try it.

can you provide some hint ?

@anweshknayak : you have to make a nested view for Forgotten Password in auth view (like we have login and signup). Refer http://localhost:8000/docs/ on your local machine for API endpoints. Use design same as of Login page. Steps are:
1. Make an email input field to send password reset link.
2. After confirmation reset password with new password, confirm password, token, and Unique ID.

@deshraj : do yo want to add something?


@anweshknayak : Please don't try this issue, we have an urgent requirement of this feature that's why I'm assigning this to myself.

@aka-jain sure. anyway I have semester exams going on too :P 

@deshraj : You can work on this, let me know if you need any help :)

Sure, will do. 

@spyshiv : assigning this to you. Let me know if you need any help.

@aka-jain : Cool. I am working on it.

Fixed in #446. Closing it.
Repository: Cloud-CV/EvalAI
Issue: Update when made public timestamp.
Root cause: Unimplemented Feature Gap

**Current behaviour:** Currently we are not updating `when_made_public` timestamp when a user makes his submission public.
**Expected behaviour:** When a user makes his submission public then update this timestamp.
The timestamp must be in` UTC`.

**Dependency:** Dependent on the PR #1179 

@deshraj I think we should update `when_made_public` field only when `is_public` field in `Submissions` model is `true`. OR should we update it every time when a user toggles the `is_public` field. What do you say?

Yeah, everytime when the participant makes it true, we should update the datetime. Does that make sense? 

yes, it does.
Repository: Cloud-CV/EvalAI
Issue: Change the message on Challenge Creation
GitHub labels: enhancement

easy_to_fix

backend

docs

priority-high

Currently, the error message on missing test annotation file for a phase shows the following error:
<img width="1440" alt="screen shot 2017-08-17 at 4 55 26 pm" src="https://user-images.githubusercontent.com/2945708/29433427-107e80a8-836d-11e7-9511-d98fc2e11c1c.png">

We want to change it to 

```
No test annotation file is present in the zip file for challenge phase 'test-dev2017'. Please try uploading again after adding test annotation file!
```
Repository: Cloud-CV/EvalAI
Issue: Console error after seeing the time remaining for the next submission
Root cause: Validation Gap

## Observed behaviour

Currently, if one has exhausted the limit to do submissions to a particular phase and he clicks on the phase radio button, then he can see the following console error:

<img width="1440" alt="screen shot 2017-08-17 at 5 13 17 pm" src="https://user-images.githubusercontent.com/2945708/29434064-8bd61f0c-836f-11e7-9e22-d3966d2a6694.png">
Repository: Cloud-CV/EvalAI
Issue: Evaluation Script extraction not working properly for Zip Challenge Creation
Root cause: Validation Gap

## Current Behaviour

When someone tries to upload a challenge on EvalAI, the evaluation script extraction code is buggy and whenever someone tries to make submission, then it shows the following error for each of the submission:

```
Traceback (most recent call last):
  File "scripts/workers/submission_worker.py", line 296, in run_submission
    submission_output = EVALUATION_SCRIPTS[challenge_id].evaluate(annotation_file_path,
AttributeError: 'module' object has no attribute 'evaluate'
```

## What is the error??

From what I experimented, it seems like when the `evaluation_script.zip` extraction is done at the backend, then while unzipping, there is a directory created `evaluation_script` and then while making submissions, the submission worker looks for `evaluate()` method which it cannot find and hence throws the error.

## How to fix

Ideally, one might want to extract the contents of the `evaluation_script` directory and not the directory itself. It should fix the error. 

I think you are uploading a zip file which has no `__init__.py` file (it is not a valid python module).

Can you check ?

@deshraj 

@taranjeet: I checked and verified that it is not the case. I checked the `/tmp` directory created and it is unzipping the `evaluation_script` in the directory which it should not do ideally. 

@deshraj Can I look into this?

@isht3 Yes sure. Go ahead!

@deshraj @RishabhJain2018 Hi! Was this fixed? I tried to reproduce the error, but everything seemed to work cool. Bottom right window.
![img](https://image.prntscr.com/image/kaatkH62SmqpVugLrrmcRw.png)

@isht3 No, this isn't fixed. Let me check once.

@RishabhJain2018 Thanks!

@RishabhJain2018 Any updates?

@isht3 ohh..It got slipped from my mind. I will update soon :)

@RishabhJain2018 Cool! :+1: 

@RishabhJain2018 Should I start working on this? :smile: 

Just checked and this is working properly. Sorry for bothering. :)
Repository: Cloud-CV/EvalAI
Issue: Add docs precompiled libraries in challenge evaluation scripts
Root cause: Documentation Deficiency

Some of the challenge evaluation scripts use precompiled libraries such as MSCOCO (https://github.com/pdollar/coco/). We want to add a section in the documentation where the challenge hosts should be guided to compile those libraries against `Ubuntu 14.04` ONLY since all the EvalAI servers use `Ubuntu 14.04` so it might not work with libraries that are compiled against OSx or Windows.
Repository: Cloud-CV/EvalAI
Issue: Add Mapillary logo in the section Who uses EvalAI.
GitHub labels: enhancement

easy_to_fix

frontend

priority-high

**Expected:** The mapillary [logo](https://d3rqg5ndc8pbid.cloudfront.net/original/1X/77b14c662df42f71f8a7f0775d1873f686a58788.png) must be added with this [link](https://www.mapillary.com/).
Repository: Cloud-CV/EvalAI
Issue: Unable to run rabbitmq worker
Root cause: Environment Incompatibility

Upon trying to run the rabbitmq worker, an error is thrown.

```
python scripts/workers/submission_worker.py
[2017-08-22 07:50:50] INFO Using /var/folders/km/885chd6n6gn3wfs1cwzcv0yh0000gn/T/tmpU_HP_Y as temp directory to store data
[2017-08-22 07:50:50] INFO Connecting to ::1:5672
[2017-08-22 07:50:50] WARNING Connection to ::1:5672 failed: [Errno 61] Connection refused
[2017-08-22 07:50:50] INFO Connecting to 127.0.0.1:5672
[2017-08-22 07:50:50] WARNING Connection to 127.0.0.1:5672 failed: [Errno 61] Connection refused
[2017-08-22 07:50:50] WARNING Could not connect, 0 attempts left
Traceback (most recent call last):
  File "scripts/workers/submission_worker.py", line 519, in <module>
    main()
  File "scripts/workers/submission_worker.py", line 476, in main
    host=settings.RABBITMQ_PARAMETERS['HOST'], heartbeat_interval=0))
  File "/Users/harshit/Desktop/OSProjects/EvalAI/venv/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 339, in __init__
    self._process_io_for_connection_setup()
  File "/Users/harshit/Desktop/OSProjects/EvalAI/venv/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 374, in _process_io_for_connection_setup
    self._open_error_result.is_ready)
  File "/Users/harshit/Desktop/OSProjects/EvalAI/venv/lib/python2.7/site-packages/pika/adapters/blocking_connection.py", line 395, in _flush_output
    raise exceptions.ConnectionClosed()
pika.exceptions.ConnectionClosed
```

The reason for this error is that the dev server is running on port  `8000` whereas rabbitmq is listening for changes on `5672`.

Rabbit mq runs listens on port 5672. Worker is a normal python script which does not runs or binds on any port.

You are actually a bit confused here.Worker when starts tries to make connection with the app(running on port 8000 generally) and rabbit mq (runs on port 5672). I guess you need to install rabbit mq.

@the-dagger 

@taranjeet Oh I see.
Thanks for the detailed explanation, I'll try to read up a bit on rabbitmq and retry this. 👍 

This can be closed I guess :) @the-dagger please reopen if it does not work for you.
Repository: Cloud-CV/EvalAI
Issue: No email recieved on forget password
Root cause: Documentation Deficiency

After clicking forget password button, no email is received to reset the password.

It's working

Yes sure @xamfy  :) 

@aayusharora During the `Debug=True` mode in django, the email is printed on console. Hence, the feature is working in production.
So, I am closing this. :)
Repository: Cloud-CV/EvalAI
Issue: Writing Proper SCSS files

The current ` .SCSS ` is not modular, and contain a lot of redundant code.
We can remove all the redundancy by designing some mixins

- Mixins for 
     - Margin, padding, position, display, keyframes
- Gradients for Extending the classes
- Breaking the code in different files to introduce Modularity

@aayusharora we are already breaking the files into modular files, do you want it to be more modular or something else?
Repository: Cloud-CV/EvalAI
Issue: Add ^ & $ in regex of urls for proper matching
Root cause: Validation Gap

' `^` ' sign should be introduced in the regular expressions of the url's, mentioned in the url files so that our search is always matched up correctly.

For example:  
If we have not added the initial caret character (i.e., 'hello/$'), Django would match any URL that ends with hello/, such as /my/site/hello/.
Also it will be more syntactically correct.


@HargovindArora Can I take this up?

Hey, @isht3 I was interested to do this on my own and also it will give me some experience of the code base. Would you mind if I do this? :)

@HargovindArora Haha, cool! I'll close my PR. :)
Repository: Cloud-CV/EvalAI
Issue: Add new requirements for AI challenges
Root cause: Unimplemented Feature Gap

We would like to add some of the new requirements for the upcoming challenges in the requirements/common.txt

```
pickleshare==0.7.4
proc==0.10.1
rstr==2.2.5
```

Please assign me

@xamfy I think this has already been assigned to @deshraj . Please take up any other issue!

Can i contribute to this?

@lenixlobo I have opened the PR for the same. Please feel free to take up any other issue which is not assigned. :)

I have made a pull request #1236 which solves the current issue,please check it out . @RishabhJain2018 

@lenixlobo Okay no problem . I will close mine :)
Repository: Cloud-CV/EvalAI
Issue: Change message after the challenge creation
GitHub labels: enhancement

easy_to_fix

frontend

Currently, when a challenge is being created using the zip file feature, then the user is shown something like this:

<img width="1440" alt="screen shot 2017-09-06 at 6 10 21 pm" src="https://user-images.githubusercontent.com/2945708/30137037-8cb1bb64-932f-11e7-89dd-7cde192b5baf.png">

We would like to change the message to "Challenge <challenge_name> has been created successfully and sent for review to EvalAI Admin."

@deshraj can I take this?


@Ram81 Yes Sure , go ahead.

@RishabhJain2018  Can I get a sample zip file for challenge?

Please take a look at the repo https://github.com/Cloud-CV/EvalAI-Examples for sample zip file.
Repository: Cloud-CV/EvalAI
Issue: Change the font everywhere to Roboto
GitHub labels: easy_to_fix

frontend

priority-high

We are planning to switch the font to Roboto for the whole web application. Please let me know if there are concerns/issues. 

Please assign me

@xamfy Yes sure. Go ahead.

@xamfy Any update on this? Please let me know if you need any help.

Couldn't get time, will do today.

@RishabhJain2018 I have sent a message please see.

@xamfy Please discuss here as it will be available for everyone. :)

Should I remove the mons-font classes from the templates?

Done

Please review
Repository: Cloud-CV/EvalAI
Issue: Text not being displayed for the 'star challenge feature'
Root cause: Validation Gap

Please see the below image for the illustration:

<img width="1438" alt="screen shot 2017-09-06 at 6 09 47 pm" src="https://user-images.githubusercontent.com/2945708/30137179-2a443974-9330-11e7-82c2-63658825255c.png">


@deshraj  can I take this?

@Ram81 yes sure. go ahead

@RishabhJain2018 you haven't created an entry for each new challenge's Star Challenge model. That's why when we try to fetch initially we can't get any message on the button.

For now I have done changes in controller & whenever we get an error fetching star challenge for any challenge I've set the message as 'Star'

@Ram81 I didn't get your point of creating the entry. Can you please explain a bit further ?

@RishabhJain2018 whenever you create a challenge there is no record entry made in 'Star Challenge' model that's why whenever the user opens the challenge page before being starred by anyone we can't fetch any entry for the particular challenge from 'Star Challenge'.

Did you get my point?

I suggest for each challenge you create an entry in 'Star Challenge' with initial values as 0 stars. 

@Ram81 Just initialise star count with 0 at the start of controller and it will solve the problem. 
@RishabhJain2018 please change the star count variable name from 'count' to 'starCount'.

@aka-jain count variable is being fetched from response that's why initially when there is no record of star challenge it fails. And this bug occurs only once, suppose if a person stars a challenge then he "unstar's" it then controller returns 0 as expected

Got your point @Ram81 , @RishabhJain2018 please do the require changes.
Also @Ram81 it will be good if we initialise the count variable with 0 value. Otherwise it might create problem somewhere.

@aka-jain Sure I will do it.

@Ram81 I think we can handle it using the back-end case.Please refer PR #1239 for further details.

@aka-jain We can easily handle it using the back-end case. Please check PR #1239 for further details. 

@RishabhJain2018 should I close this one #1235 then? & actually I wanted to complete this one!
But I will take another issue.

@Ram81 Yes you can take up any other issue :). I will take care of this issue.
Moreover, we are glad to see your contributions and dedication!
Repository: Cloud-CV/EvalAI
Issue: "Setting up your details" shows up in white color 
Root cause: Algorithmic Error

While signing up the text "Setting up your details" shows up in white color which is not visible.

I think nobody is on this, can I take this issue @RishabhJain2018 

@deshraj @RishabhJain2018 The text is visible while signing up, Can we close this issue??

@gautamjajoo Thanks for pointing this. Yes, I am closing it as it is already fixed.
Repository: Cloud-CV/EvalAI
Issue: Splitting the tests with too many lines to smaller test files (150 lines per file).

Hi, I was in the process for adding some tests when I noticed the `challenges/test_views.py` was over 2000 lines in a single file. Is there any improvements to performance? Could it be better if it was smaller chunks of code?

@isht3: the way you can improve is the code quality. You might want to use inheritance in the tests to decrease the number of lines but this is not on the top priority so I would recommend you building new features instead of working on this. Please close this if you are not planning to work on this. :) 

@deshraj Okay, thanks for the heads up!
Repository: Cloud-CV/EvalAI
Issue: Add `^` symbol before url paths in url.py files.


Hey , I was just reading the documentation and code of EvalAI and found missing '^' on urls.py files

I want to fix this can I do that???
I am new to GitHub so please help me :D

It has been raised earlier, see https://github.com/Cloud-CV/EvalAI/issues/1216

But I think the changes are still not accepted.

Because they are still missing from some files.

It's currently being worked upon so look for a different issue.

Can you help me with that because I am new to GitHub.

Join the gitter channel

@modestlearner Please join https://gitter.im/Cloud-CV/EvalAI for more details and discussion about the project.
I am closing this issue as #1223 already points to it.

@RishabhJain2018 Cool!!!! Thanks!!!
Repository: Cloud-CV/EvalAI
Issue: Create endpoint to get details about a particular Challenge

Details about the endpoint:

Url: /api/challenges/challenge_host_team/<challenge_host_team_id>/challenge/<challenge_id>
Header: Authorization: Token
Method: GET
Response:
status on success: HTTP_200_OK
status on exception: HTTP_406_NOT_ACCEPTABLE
status on Unauthorized access: HTTP_203_NON_AUTHORITATIVE_INFORMATION

Merged in #125
Repository: Cloud-CV/EvalAI
Issue: Unnecessary r' ' in the regular expressions in urls.py file


I am not too sure about the issue but I think there's unnecessary r ' ' in the regular expression in urls.py file in analytics..

If this is a valid issue I will be glad to fix this.

@modestlearner No, since we are following pep8 guidelines for writing the code. Please refer https://stackoverflow.com/questions/22775697/how-to-specify-long-url-patterns-using-regex-so-that-they-follow-pep8-guidelines for more details.
Also let me know if you have further doubts.

Okay thanks!!!!