zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: Add support for Python3.X
Root cause: Unimplemented Feature Gap

## Description
 Considering that Python 2 will no longer be maintained past 2020.
Also, Python 3.6 has native support for asynchronous programming, which can be very useful in the case of EvalAI.
## Deliverables
Check for Python 3 support for EvalAI and discuss the roadmap of supporting both the versions of python


@dibyadas: Yes, we have planned to add support for Python 3.x. Please feel free to work on this if you are interested. 

Sure. Will do!  Will work on this after my tests are over. 

Sounds good. Thanks for letting us know. 

I would like to work on this. Is there any branch for this @deshraj

@Yatish04 Yes you can work on it. No there isn't any branch for it, you can fork the repo and then start working on a new branch. Please feel free to ping us if you have doubts or queries.

@dibyadas @Yatish04  are you still working on this or I can take this up ? 
If you are working on this , can we collaborate ? Please let me know :+1: 

Hey @ParthS007 !  Unfortunately, my exams are going on, so I was unable to work on this. Great that you are interested. Please go ahead on this one. So sorry that I did not inform earlier. It seems @Yatish04 is working on this. Just ask him before proceeding. :smile: 

@dibyadas I have updated my previous comment :+1: , Thanks for responding so quickly. @Yatish04  waiting for your response :smiley: 

Hey I also started working on this. I made some progress - https://github.com/siddharth185/EvalAI/tree/port_python3 using Futurize. The dependencies blocking python3 are - `asgiref (which is blocking asgi-redis) django-rest-auth recommonmark uwsgi` Can collaborate if anyone wants :)

django-rest-auth asgiref and uwsgi are actually compatible with python3, it seems like a glitch with PyPi not listing their compatible versions. pip will still install them correctly.
I'm not entirely sure about recommonmark, but it looks like its also compatible.

@ParthS007 feel  Free to work on this. We both can work on this as i am also working on this.

I'm very much interested in this, but I need someone to guide me, what should I do ?

Similar to #1681. Closing it.
Repository: Cloud-CV/EvalAI
Issue: Updating Team in README

## Observed Behaviour 

![capture2](https://user-images.githubusercontent.com/24366008/35666940-e45a91b0-0751-11e8-876f-7e6d6402fc16.JPG)

## Expected Behaviour - Better Representation

![capture](https://user-images.githubusercontent.com/24366008/35666966-f74e978a-0751-11e8-9d44-7adf7958ad01.JPG)

@aka-jain @deshraj @RishabhJain2018  @spyshiv 

@gautamjajoo This looks good. This should be done. What do you say @RishabhJain2018?
Repository: Cloud-CV/EvalAI
Issue: Link not Working

## Observed Behaviour- Link not working

![link](https://user-images.githubusercontent.com/24366008/35667327-3589a7fa-0753-11e8-9628-4f01bdd9a1f2.JPG)

## Expected Behaviour

Link Working

## Steps to Reproduce

Remove the https:/ from the link.

@RishabhJain2018 @aka-jain 

I think this is due to SSL error. @deshraj are we done with the certificates?

@aka-jain So, closing the issue.

Don't close this please. Let's wait for the @deshraj confirmation.

It's not fixed yet. Needs some time. @gautamjajoo: can you please change it to `http://cloudcv.org` for now?
Repository: Cloud-CV/EvalAI
Issue: Featured challenge on home page
GitHub labels: bug

easy_to_fix

backend

good first issue

## Deliverables

- [ ] On the home page, the featured challenge is not selected on the basis of featured flag, but on the basis of approved_by_admin and published flag.


@deshraj please correct this if this is not intentional.

@viditjain08: Good catch. Please feel free to work on it. Assigning this to you. 

Since @viditjain08 cannot work on this issue, so this is open now for anyone willing to take. You will have to work on top of #1512. There aren't lot of changes to be done in this feature.

This issue is still unassigned. Please feel free to take this if someone is interested. 

@deshraj I can work on it.

Sure! you can. Please feel free to ask if you have doubts.

Hey I got error when I tried to push my code up to my branch. Is there any steps that I am missing?
```
$git push --set-upstream origin feature-challenge
remote: Permission to Cloud-CV/EvalAI.git denied to JaneLiuUSA.
fatal: unable to access 'https://github.com/Cloud-CV/EvalAI.git/': The requested URL returned error: 403
```

Hi @JaneLiuUSA. Thanks for fixing the issue.
You can't push directly to CloudCV EvalAI repository since you are not a member of the organization.
Ideally, the flow should be:
1. Fork the EvalAI repository into your account.
2. Create a new branch and then make the required changes in the code.
3. Open a PR from your fork to EvalAI.

Please feel free to ask if something is not clear. 

Got it. Will do so. Thanks!
Repository: Cloud-CV/EvalAI
Issue: Add support in the back-end for having a user display picture.

I think this would be a neat feature to have in which the user would be having more customising options for their own profile. We could also display the display picture in the leaderboards so that it looks neat and easily recognisable. 

Let me know what you think about this!

Thanks for the suggestion @isht3. Yes, it is already in our Todo list. But not implementing this now.
Repository: Cloud-CV/EvalAI
Issue: My Submissions Page
Root cause: Algorithmic Error

1. For any challenge, the participant is unable to download all submissions by selecting csv and clicking download button.

2. Also, the submission count remains 0 even if a submission is present.
![screenshot from 2018-02-06 09-38-33](https://user-images.githubusercontent.com/5248993/35841428-f5c11ba8-0b21-11e8-8d65-7712808f28c5.png)
 I would like to take this issue

Thanks for pointing out this bug @viditjain08.
You can work on this.

This issue is already fixed in #1582
Repository: Cloud-CV/EvalAI
Issue: See-more button not working in mobile/tablet view

- the expected behaviour
 scroll button should work

- the observed behaviour
  scroll button is not working

- steps to reproduce the problem
 check see-more button in landing page

- which operating system and/or browser you're using, if relevant
  Firefox Quantum
  Ubuntu 16.04

![button](https://user-images.githubusercontent.com/8139783/35885376-9fa8221a-0bb3-11e8-9438-85fdc61e3a22.gif)
Repository: Cloud-CV/EvalAI
Issue: Images should not be draggable

- the expected behaviour
  Images of cloud,rocket should not be draggable

- the observed behaviour
  Certain images are draggable

- steps to reproduce the problem
 Drag the clouds off-screen. 

- which operating system and/or browser you're using, if relevant
 Firefox Quantum
 Ubuntu 16.04

![cloud](https://user-images.githubusercontent.com/8139783/35886580-893c45f2-0bb7-11e8-988d-5caef2952192.gif)


@aka-jain @spyshiv is this legit issue?
`<img draggable="false" img>` happens to solve this.

This issue is not valid now since we are moving to Cloud-CV/evalai-ngx
Repository: Cloud-CV/EvalAI
Issue: Sphinx doc building fails
Root cause: Environment Incompatibility

## Current Scenario

When someone tries to build sphinx doc using the following commands:

```
cd docs
make html
```

You get an error: 

```
# Sphinx version: 1.4.8
# Python version: 2.7.10 (CPython)
# Docutils version: 0.14 
# Jinja2 version: 2.10
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/Users/deshraj/Documents/evalai/env/lib/python2.7/site-packages/sphinx/cmdline.py", line 243, in main
    opts.warningiserror, opts.tags, opts.verbosity, opts.jobs)
  File "/Users/deshraj/Documents/evalai/env/lib/python2.7/site-packages/sphinx/application.py", line 137, in __init__
    confoverrides or {}, self.tags)
  File "/Users/deshraj/Documents/evalai/env/lib/python2.7/site-packages/sphinx/config.py", line 287, in __init__
    execfile_(filename, config)
  File "/Users/deshraj/Documents/evalai/env/lib/python2.7/site-packages/sphinx/util/pycompat.py", line 130, in execfile_
    exec_(code, _globals)
  File "/Users/deshraj/Documents/evalai/env/lib/python2.7/site-packages/six.py", line 709, in exec_
    exec("""exec _code_ in _globs_, _locs_""")
  File "<string>", line 1, in <module>
  File "conf.py", line 35, in <module>
ImportError: No module named recommonmark.parser
```

## How to solve this?

Add recommonmark==0.4.0 in the requirements/dev.txt
Repository: Cloud-CV/EvalAI
Issue: UX Improvements for mobile/tablet view
Root cause: Unimplemented Feature Gap

![mobile](https://user-images.githubusercontent.com/8139783/35913004-52d9e204-0c24-11e8-8c4b-c8babc1231f5.gif)

- For Landing Page 
Okay we can make just one button like **Register and Log In**  as in above. And rest as shown we can show as a menu or dropdown. 

Current view : 
![current](https://user-images.githubusercontent.com/8139783/35913254-12889406-0c25-11e8-8f14-3bba98a574b0.gif)

takes too much space 


Okay, we can do this.
You working on this? @anweshknayak

Sure @aka-jain 

Seems like there are more bugs
![bug_responsive](https://user-images.githubusercontent.com/8139783/35924855-3648546a-0c4a-11e8-9625-1eb3e7f8d686.gif)


@aka-jain Would it be good if I include this in GSOC proposal? I will have the mock-ups ready before the proposal period. There are certain bugs. I plan to revamp it completely. Sounds good?

@anweshknayak: this doesn't seem like a lot of work. I would recommend you to fix the bugs beforehand and mention the Pull Requests link in your proposal instead of just listing the idea. Also, since revamping to Angular 5 is not a trivial task and will require a lot of time, so it's better to fix the existing bugs. 

@anweshknayak Hey!! Are you on it? If you are facing any problem ,please ask !!

No. You can do this if you want to

This issue is not valid now since we are moving to Cloud-CV/EvalAI-ngx. Thanks for opening this @anweshknayak though :)
Repository: Cloud-CV/EvalAI
Issue: Scroll-bar overlapping with footer in firefox


If you'd like to report a bug, please include the following information:
- the expected behaviour
  Scroll bar should not overlap

- the observed behaviour
 Scroll overlapping with footer

- steps to reproduce the problem
 Log in -> Dashboard -> Scroll

- which operating system and/or browser you're using, if relevant
  Firefox Quantum
  Ubuntu 16.04

![scrollbar_overlap](https://user-images.githubusercontent.com/8139783/35925065-c4012ab6-0c4a-11e8-8b79-0a19145d5a2a.gif)



Thanks @anweshknayak for opening this issue. Please feel free to work on this issue. 

@anweshknayak Any Update on this? Let me know if you need any help

@spyshiv I am going to start on this :smile:. Will let you know if I need any help. Thanks :tada: 

This issue is already fixed. Closing it.
Repository: Cloud-CV/EvalAI
Issue: clicking on `view more`of VQA challenge leads to permission denied page 
Root cause: Validation Gap

_Thanks for taking the time to report a bug in the EvalAI project, or propose a new feature! Before creating a new issue, please do a quick search to check that it hasn't already been filed on the [issue tracker](https://github.com/Cloud-CV/EvalAI/issues)._

If you'd like to report a bug, please include the following information:
- the expected behaviour
Reason for permission denied if any
- the observed behaviour
![screenshot 2018-02-09 11 27 30](https://user-images.githubusercontent.com/20624380/36013989-7b8943a6-0d8d-11e8-9a1b-48ce44222122.png)
- steps to reproduce the problem
go to https://evalai.cloudcv.org/, then scroll down to bottom and click `view more` of VQA challenge
- which operating system and/or browser you're using, if relevant
windows 10, firefox, I am logged in while experiencing this behaviour

Thanks!


@obtin although page denied is more or like the same , But I have verified my account and then I am getting the error. Please see steps to reproduce the problem I mentioned.
Repository: Cloud-CV/EvalAI
Issue: Permission denied: '/tmp/logfile'
GitHub labels: bug

GSOC-2019

medium-difficulty

backend

priority-high

The submission worker currently faces the problem of permission denied due to the dependency on `/tmp/logfile`. Here is the error log:

```
(EvalAI) 137 ubuntu@staging-evalai:~/Projects/EvalAI⟫ python scripts/workers/submission_worker.py settings.prod
Traceback (most recent call last):
  File "scripts/workers/submission_worker.py", line 44, in <module>
    django.setup()
  File "/home/ubuntu/.virtualenvs/EvalAI/local/lib/python2.7/site-packages/django/__init__.py", line 22, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/home/ubuntu/.virtualenvs/EvalAI/local/lib/python2.7/site-packages/django/utils/log.py", line 75, in configure_logging
    logging_config_func(logging_settings)
  File "/usr/lib/python2.7/logging/config.py", line 794, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/lib/python2.7/logging/config.py", line 576, in configure
    '%r: %s' % (name, e))
ValueError: Unable to configure handler 'logfile': [Errno 13] Permission denied: '/tmp/logfile'
```

@deshraj Shall I look into it?

@isht3: sure, go ahead. 

@deshraj  How do I reproduce the error?

@isht3` python scripts/workers/submission_worker.py settings.prod `
check the first line @deshraj has posted

@anweshknayak I missed the settings.prod part :rofl:, thanks for pointing it out.  

@deshraj @anweshknayak Is there any prerequisite installation required before running the production server? I'm getting a `ImportError: No module named raven` error.

@isht3 yep `pip install raven` would do. Check https://github.com/Cloud-CV/EvalAI/pull/517

@anweshknayak All the dependencies are taken care off but when I run `python scripts/workers/submission_worker.py settings.prod` it's just stuck, no progress, no errors, no anything. Any ideas?
Repository: Cloud-CV/EvalAI
Issue: Leaderboard in past challenges is not visible without login
Root cause: Validation Gap

**Steps to reproduce:**

1. Go to `All Challenges` page
2. Select one of the `past challenges`
3. Go to `leaderboard` tab
4. On selecting one of the `challenge phase` it redirects to auth wall

@RishabhJain2018 Assigning this to you since you were working on similar issue
Repository: Cloud-CV/EvalAI
Issue: Add support for changing Participant Team Name
GitHub labels: enhancement

GSOC-2019

easy_to_fix

frontend

new-feature

priority-high

good first issue

Currently, we don't provide support for editing the participant team name on UI. We would like to add a support for this. 

Can I take this up? 😄 

@isht3 sure. This needs to be done quickly. Please take it if you are sure that you can get it done quickly. 

Also, FYI: I have a deadline this friday so I will be able to look at the pending Pull requests after that only.  

@deshraj Okay, that sounds good! I'll do it in 24 hours.
Repository: Cloud-CV/EvalAI
Issue: Create MNIST Digit Classification Challenge
Root cause: Unimplemented Feature Gap

## Why are we asking you to create a challenge?

Challenge creation is the main functionality of EvalAI. If you are able to create a challenge on EvalAI, then you have a better idea of how things are working in challenge creation and will be able to figure out errors/bugs etc. This also helps us to make the challenge creation process even better. 

## Who can work on this issue?

Don't worry if someone is already working on this issue. **You can start working on this issue by commenting on this thread**. Feel free to ask questions if you have any. We would be happy to help.

## Deliverables

- [ ] Create one/two challenge phase Dataset (http://yann.lecun.com/exdb/mnist/)
- [ ] Create one/two/three Dataset Split
- [ ] Leaderboard with one column named `score`
- [ ] Add sample challenge details and html
- [ ] Write a tutorial on how to create a MNIST Digit Classification challenge on EvalAI
- [ ] Put the challenge evaluation script in `examples/` directory

## How to get started? 

- How to create a challenge on EvalAI: http://evalai.readthedocs.io/en/latest/challenge_creation.html
- How to write a custom evaluation metric/script on EvalAI: http://evalai.readthedocs.io/en/latest/evaluation_scripts.html
- MNIST Digits Dataset: http://yann.lecun.com/exdb/mnist/

Please let us know if you have questions/doubts. 

Hello Sir, I would like to work on this 

@Yashs744: since multiple people can work on this issue, so you can go ahead and work on this. 

@deshraj I would also love to work on this

@S2606 sure, go ahead. 

@deshraj I would like to work on this

@vishalpolley sure. Go ahead. 👍 


@deshraj I'd like to work on this as well 🌮 


@live-wire sure. Feel free to comment if you have questions. 

@deshraj Would it be useful to create some other examples? 
Is there an UI for creating challenge configuration files?

Hi ,I'd like to work on this 


@MohamedElshazly sure, go ahead. 

@vladan-jovicic yeah, feel free to work on some other example. Just let us know before you start working on the other example. 

You can find the documentation for creating the challenge here: http://evalai.readthedocs.io/en/latest/challenge_creation.html

@deshraj could you please show me some love and have a look at [this](https://github.com/Cloud-CV/EvalAI-Examples/pull/3) ? Thanks 😄 

It was a GSOC 2018 related issue and is no longer valid. Closing it.
Repository: Cloud-CV/EvalAI
Issue: Add support for sorting leaderboard column wise
Root cause: Unimplemented Feature Gap

## Current Scenario

One cannot sort the fields based on different columns in the leaderboard for a challenge.

## Deliverables

- [ ] Add support for sorting leaderboard based on different columns
- [ ] Make sure to add support for sorting based on the submission date. For example, see VQA 2017 Challenge leaderboard: https://evalai.cloudcv.org/web/challenges/challenge-page/1/leaderboard/
- [ ] Add proper comments in the codebase
- [ ] Write tests if possible to make sure that you are covering all the corner cases such as when the data is empty etc.
 

i will like to work on this issue..


@xender98 sure, go ahead. 

Hi @deshraj and @xender98 

I almost finished this issue. The only remaining thing is testing.
Here is a description of my approach:
- when user selects the challange phase, a new row is shown containing sort_by form
- the form contains names of labels of columns in results table
- when an option is selected, the index of selected label is saved
- a corresponding url is defined, so that the index is an optional parameter in leaderboard view
- this is taken into account when sorted leaderboard data is constructed

Currently, I am trying to insert some random results to see if everything works correctly. @xender98 do you want to help me with this?


@vladan-jovicic: in future, please comment on an issue if you are interested in working on it. 

You can follow a different flow than creating a form as you have suggested above. See the table '2012 Gold Medal Olympic Divers' given in this link: http://www.bu.edu/tech/services/cccs/websites/www/wordpress/how-to/sortable-searchable-tables/

@deshraj so should i continue or move on another issue

@deshraj I followed your suggestion and before I make a pull request, I would like to ask for a comment on styling.
Here are a few examples:

Without any sorting:
![0](https://user-images.githubusercontent.com/8584694/37244981-b222e960-2491-11e8-85c3-b8c6cd9193df.png)

Sort by "score" from largest to lowest
![1](https://user-images.githubusercontent.com/8584694/37244986-ccb4fc82-2491-11e8-9c45-908601947f7a.png)

Sort by "score2" from largest to lowest
![3](https://user-images.githubusercontent.com/8584694/37245032-87f03282-2492-11e8-8a20-13ea44177154.png)

There is an arrow next to column name indicating ascending or descending sort.


@vladan-jovicic this looks good. You can create a pull request now. I will review it once you have created the pull request.

@xender98: please pick some other issue. 

@deshraj I tried to make a PR. However, it fails travis checks. I don't see a reason since I changed only 2 files (challengeCtrl.js and leaderboard.html). Can you have a look at it and suggest how I can solve it?
Repository: Cloud-CV/EvalAI
Issue: Create a endpoint for challenge host and team creation
Root cause: Unimplemented Feature Gap

When creating a challenge, a challenge host teams needs to exist.  This route will do the following
* Create a new challenge host team 
* use this challenge host team to create a new challenge host with status as `Self` and permissions as `Admin`.

This will return `HTTP_201_CREATED` if everything is successfully created, else `HTTP_400_BAD_REQUEST` 

Fixed in #156
Repository: Cloud-CV/EvalAI
Issue: UI Improvements in Leaderboard
Root cause: Unimplemented Feature Gap

- [ ] Make the team name as **Bold** on leaderboard.

- [ ] Make `Rank` and `Participant Team` field as left aligned including the title.

- [ ] Make all the other fields as right aligned including the title.

- [ ] In the heading `submission time`, change the text to `Last Submission at` & display the delta between the current time and the last submission time. For Example: If the current time is 10 am, & last submission was made at 6 am, then the field should display `4h` (10am - 6am).

- [ ] Add a border line after every entry in the leaderboard.

Please feel free to ask if you have further doubts or queries.

I want to work on this @RishabhJain2018 

Sure go ahead @ParthS007 .

@RishabhJain2018 Now the leaderboard looks like this.
![image](https://user-images.githubusercontent.com/24358501/37252355-c3e3be86-2545-11e8-9d7a-97fd20c18f0f.png)
Can you please help me with the 4th point?

@ParthS007 Here are few comments: 
- There are other fields `(yes/no, number, other, overall)` too in the right side of the leaderboard. Please don't remove them. Just make them right aligned. 
- The border line after every entry is very dark. Make it similar to the one just after the heading.
 
Moreover, in the fourth point we need to display the time difference between now and when the last submission was made. I think this can be easily handled client side and we don't need to modify the API. Please feel free to ask if you have further doubts.  

@ParthS007 Since we need this feature early, I am taking this up. Please take up any other issue from the list. :-)
Repository: Cloud-CV/EvalAI
Issue: UI Improvements in Leaderboard Entry
Root cause: Unimplemented Feature Gap

- [ ] Show the meta data fields (**Team Members** and **Method Description**) in each leaderboard entry. 
Hint: Please refer the screenshot.
![6](https://user-images.githubusercontent.com/12206047/37241823-a1524bf0-2485-11e8-88a0-1b7c99ccfa9a.jpg)

- [ ] The user should be able to edit the **Method Description** field if the entry is created by the same user.

Please feel free to ask if you have doubts or queries.

@RishabhJain2018 can I give a try to this one?

Is it ok if I add a details button to each leaderboard entry, so that when user clicks on it, a popup (see the image below) is shown containing the method description and other details about the method or I should add it next to the team name?

Example of popup
![00](https://user-images.githubusercontent.com/8584694/37298026-8a3ce664-261f-11e8-879e-0225d1f5dd7f.png)



@vladan-jovicic I am sorry, I somehow missed this thread. Did you start working on it?

> Is it ok if I add a details button to each leaderboard entry, so that when user clicks on it, a popup (see the image below) is shown containing the method description and other details about the method or I should add it next to the team name?

No, we don't need a button in leaderboard entry. It should be a collapsable as shown in the screenshot above.

@RishabhJain2018 I have not started yet. I am struggling with another issue. I will come back to this one next week (if nobody solves it until then).

@vladan-jovicic Okay!

It seems like @vladan-jovicic has been inactive for a long time.
@live-wire Do you want to take this up ?

@live-wire I am closing this issue here. Can you please open a similar issue for EvalAI-ngx? We should totally add this feature.