zhiminy/EvalEng huggingface dataset:
Repository: Cloud-CV/EvalAI
Issue: Login through internet returns error: Access has been blocked
Root cause: Documentation Deficiency

I checked [issue 2434](https://github.com/Cloud-CV/EvalAI/issues/2434), it looks like something different.
- **Expected behaviour**
Login through internet works
- **Observed behaviour**
Error `Access to XMLHttpRequest ... has been blocked by CORS policy`
- **Steps to reproduce the problem**
Open http://competition.pozdn.keenetic.pro/auth/login and try to log in by "admin", "password".
- **Operating system**
Server: Windows 10 Pro 1909
Client: macOS Mojave 10.14.6 (18G4032), Google Chrome 81.0.4044.129
- **Copies of the console/server logs**
I created the following list of domain names on my router (I don't understand which components should be visible on frontend):
![image](https://user-images.githubusercontent.com/12996098/81481599-06ea5380-923a-11ea-8b3f-f9455665c082.png)

I changed the API path in `/frontend/src/js/config.json`:
![image](https://user-images.githubusercontent.com/12996098/81480872-14e9a580-9235-11ea-81ad-420dc870d6e5.png)

I try to log in and get this error:
![image](https://user-images.githubusercontent.com/12996098/81481017-e02a1e00-9235-11ea-95dc-b19bc9404b52.png)



Hi @vpozdnyakov, thanks for opening the issue. Since you are trying to run EvalAI on your own private machine, you will have to enable cors. Follow these steps:

1. https://github.com/Cloud-CV/EvalAI/blob/master/settings/prod.py#L17 (set this to `True`)
2. Comment / remove these lines: https://github.com/Cloud-CV/EvalAI/blob/master/settings/prod.py#L19-L23

This should fix your issue. Closing this issue since not related to the active development of EvalAI. Please consider asking questions on our Gitter channel.

Hi @deshraj, thanks for your answer. Unfortunately your advice didn't help. Here is absolutely the same problem. I don't exactly understand how these settings work, but I guess my local machine uses the `dev.py` settings where CORS are already enabled (loaded from `common.py`). Could you suggest other options to solve this problem? Or tools/approaches which can help me detect the cause of the problem? Unfortunately I don't have enough experience in WEB and HTTP to solve this problem on my own.

Also I suggest to give an exhaustive description of how to configure EvalAI on the local machine in the documentation.

@deshraj sorry, this problem is related to my DNS service settings. thank you!

Sorry @vpozdnyakov I was occupied with other stuff. Glad that the issue is resolved on your side.
Repository: Cloud-CV/EvalAI
Issue: Show participating challenges in the dashboard
Root cause: Unimplemented Feature Gap

As a participant, we are not shown challenges we are participating in / have participated in the past in a dedicated view. 

Can we add a view similar to "All Challenges" / "Hosted Challenges" which shows all the challenges a user has participated in the past or is actively participating sorted either based on last activity or "challenge creation date". 

Thanks!


Duplicate of #2698
Repository: Cloud-CV/EvalAI
Issue: Setup logging in django
GitHub labels: enhancement

GSOC-2019

medium-difficulty

For more information, see this https://docs.djangoproject.com/en/1.10/topics/logging/

I can work on this


Great. :-)


is someone working on this ?

@anweshknayak : the work is already done, just forget to get it into master.

where is this stuck? @deshraj 

Fixed in #33
Repository: Cloud-CV/EvalAI
Issue: Create a custom 500 and 404 error page in for django
GitHub labels: enhancement

GSOC-2019

easy_to_fix

backend

For reference, see this http://stackoverflow.com/questions/17662928/django-creating-a-custom-500-404-error-page

@deshraj Im working on this.

Sure, go ahead. Please ask before working on any issue otherwise it would be a duplicate effort. 

@deshraj sure :)

@deshraj this custom error page for every app?

No, there will be a single 404 and 500 error page. Please see the reference link that I shared. 

@deshraj Is someone still working on this issue??

Yes. Code is almost done. Tests needs to be added 

@RishabhJain2018 let me know if you can help with the tests. you can see the closed PR. @deshraj can we split this ?

Yeah sure. That sounds like a collaborative effort. :-) 

@anweshknayak yeah sure, I will help you with the tests. :+1: 

@aka-jain #290 related to this ?

No @anweshnayak , It's on backend side.

@deshraj For adding tests, Can I create a file on this path `tests/unit/evalai/test_urls.py`.

@deshraj I think we should also remove [this](https://github.com/Cloud-CV/EvalAI/blob/master/evalai/urls.py#L29) url from `evalai/urls.py` as we are not using `django-allauth`.

@RishabhJain2018 please use this file `tests/unit/evalai/test_views.py` for test or create one if does not exist. 

@RishabhJain2018 We will remove the URL later. Not related to this PR. Also, we might use it somewhere in future. 

@deshraj ok.

@deshraj I think we should close this issue.

Looks like this is causing issue. @jainarunarunjain is working on it. Let me know @jainarunarunjain if you want some context about the bug. 

Forgot to close. Fixed now. 

Reopening this since now the 404 and 500 template would be on backend and won't be dependent on the frontend. @jainarunarunjain let me know if you are interested in working on this issue. I can guide you. :-) 

@deshraj Sure, I want to work on this.

Awesome. Let me know if you need any help. :-) 

@deshraj please give me some details.

So, in `apps/web/templates` directory create two templates named `500.html` and `404.html` in which you can add basic html content that can be modified later. Also, you would need to create a custom 404 and 500 handler for the same as it was before. 

fixed in #631
Repository: Cloud-CV/EvalAI
Issue: Reuse common code from leaderboard API

As in [this](https://github.com/Cloud-CV/EvalAI/pull/2798) PR we have created an API for not only giving public submissions but also private submissions to display on the leaderboard. So we have used most the things same which are present in the [leaderboard API](https://github.com/Cloud-CV/EvalAI/blob/master/apps/jobs/views.py#L467).

**How to fix this issue:**

In order to fix this issue one need to move that reusable code from both the APIs to some common place and then use it from there in both the APIs.
Repository: Cloud-CV/EvalAI
Issue: Add documentation for challenge hosts on how to make a submission a baseline submission
Root cause: Documentation Deficiency

**Description**
The submissions of the challenge hosts don't appear until they are made as a baseline but we are missing on the documentation for it.

**Deliverables**
Step by step guide on how to make a submission a baseline submission.

Hi. I would be interested in working on this.

Sure. Please let us know if something is unclear.

@RishabhJain2018  is any one working on the issue?
If no i would like to take up the issue

Hi @shraddhavp, Sure you can take this up. 

can you please tell more about baseline submission or any document that i can refer to start of @RishabhJain2018 ?

Hi @RishabhJain2018  is the issue still open. I was thinking of contribuitng. It would be  great help if you can help me in getting started.
 

hi @RishabhJain2018 if this issue is still open i would love to give it a try , i am a beginner in open source projects so i think i can learn something valuable from it.

Hi, is anyone working on this issue? if not, I would like to submit a PR.


Hey @Mukul2000, sure you can work on this.

Hi @RishabhJain2018, I have submitted PR regarding this issue, please review. Thanks.

@RishabhJain2018 is any one working on the issue?
If no i would like to take up the issue


Hi @RishabhJain2018 ,
Is the issue still open, I would Like to work on it??

Hi @RishabhJain2018 ,
I would like to work on it, if the issue is still open


Any updates on how it works?

Hi @RishabhJain2018 ,

I'm Shiva Ganesh Kasani, a GSoC 2025 aspirant. I’d love to work on this documentation issue and contribute by writing a clear, step-by-step guide on how to mark a submission as a baseline.

I've gone through the EvalAI documentation and will ensure the guide is:
- User-friendly
- Includes screenshots or code snippets (if needed)
- Suitable for both new and experienced challenge hosts

Could you please assign this issue to me?

Thank you!


Hi @RishabhJain2018 ,

I have worked on this issue and added a detailed guide on how Challenge Hosts can mark submissions as Baseline in the documentation.

Please find the PR here: https://github.com/Cloud-CV/EvalAI/pull/4761
Or Checkout with the GitHub CLI: 
`gh pr checkout 4761`
This should resolve Issue #2804.

Looking forward to your review! 🚀

Thank you.

Hi @RishabhJain2018 ,

 have worked on this issue and added a detailed guide on how Challenge Hosts can mark submissions as Baseline in the documentation.

Please find the PR here: #4843

This should resolve Issue #2804.

Eagerly waiting for your response!

Thank you.
Repository: Cloud-CV/EvalAI
Issue: Bug in compressing files for production
GitHub labels: bug

easy_to_fix

frontend

priority-high

**Description**: While compressing files for pushing to production server, there is a bug that is mentioned here: https://travis-ci.org/github/Cloud-CV/EvalAI/builds/689910381#L5593.

The potential line which is causing this issue is: https://github.com/Cloud-CV/EvalAI/commit/5a78b2048461e95e7cc325fad264ce0c2055a879#diff-14a845646ed48757b5c7798b7ccdddb6R679

Steps to reproduce:
1. Pull the latest master branch
2. Run `docker-compose up`
3. Go to nodejs container by running - `docker-compose exec nodejs bash`
4. Run `gulp staging` in the container

cc: @Kajol-Kumari
Repository: Cloud-CV/EvalAI
Issue: Fix the card UI for challenges
GitHub labels: enhancement

frontend_v2

The challenge logo is required to be a square
![Screen Shot 2020-05-23 at 2 00 15 PM](https://user-images.githubusercontent.com/2039548/82738532-d0164200-9cfd-11ea-9721-7566c13ed406.png)

however, the card display for each challenge uses a rectangular preview which results in the logo getting cropped from top and bottom. 
![Screen Shot 2020-05-23 at 2 00 10 PM](https://user-images.githubusercontent.com/2039548/82738531-d0164200-9cfd-11ea-95b6-47ac235abb42.png)

It'd be nice to update the card UI so that it shows the full logo instead of a cropped version.
Repository: Cloud-CV/EvalAI
Issue: Evalai UI leaving White Space in both sides

**The expected behaviour:-**
The UI should make use of all the space provided leaving no white space in left and right sides of the screen.
**The observed behaviour:-**
The Screen leaves a lot of white space in both the left and right. i tried debugging it but there is no paramenter which might make UI leave these types of margins. I also tried running the evalai in VM and other devices and the results are same.
**Screenshot:-**
![Selection_001](https://user-images.githubusercontent.com/34577232/83385230-7bae4900-a406-11ea-9f42-fe7abd3959c2.png)
OS:- Ubuntu 20.04


@Kajol-Kumari is this a new update or a bug . because it wasn't like that do you have any idea what has happened to UI?

> @Kajol-Kumari is this a new update or a bug . because it wasn't like that do you have any idea what has happened to UI?

Hey @Suryansh5545 please check you browser's view once as I don't see any problem in the UI on my PC.

@Kajol-Kumari what is your screen resolution? . i just checked it in a small laptop with 1366^768 resolution and it was ok . but on big screens bigger than 1366^768 or 1440^900 it is not expending well and leaving white space. I previously have ran evalai main website and the github repo on monitor bigger than 1366^768 but problem never occured.

See even the evalai.cloudcv.org is having the same problem
![Capture](https://user-images.githubusercontent.com/34577232/83515152-834b1c00-a4f2-11ea-843b-8e41e5817029.PNG)


can someone other than me confirm that whether they are having problem on bigger resolution displays?

@Kajol-Kumari I have tested this on Ubuntu 20.04 , 18.04 and windows 10 and one other laptop of 1366^768. and on different browsers(Chrome and Firefox). So this definitely isn't something related to browser i think. maybe the UI is no longer scaling for bigger displays.

Also i would like to add that this problem is only in the main repository. i have tested the ngx one and it works fine. 

i found the #2723 PR explains the reason for this issue so i am closing it. Also btw i think the older version was actually much better. Also thx @Kajol-Kumari  for looking into this.
Repository: Cloud-CV/EvalAI
Issue: Unverified Users are able to view all the functionalities in ngx

**Bug Report**

- Observed Behaviour
Newly signed up users are able to view all the challenge details along with the functionalities like `participant_team`, `host_challenge` and all other functionalities without showing any proper message for verifying the email.

- Expected Behaviour
Unverified users should be given proper message when they try to access the functionalities just like we do in EvalAI.

- Enhancement
We can allow limited privileges to the unverified users like allowing them to access their dashboard along with viewing the challenge list with proper message telling them to verify their email in order to get the full access.

- Steps to reproduce the problem
Signup as a new user
login to ngx without verifying the email
Navigate between different functionalities.

- Browser 
Google Chrome

- OS 
 Ubuntu 18.04

GIF verifying the Bug:
![ezgif com-video-to-gif (1)](https://user-images.githubusercontent.com/44888949/84299276-14169d00-ab6e-11ea-8b0b-3b4db849a296.gif)


Closing this one as this has been resolved through #2866 .
Repository: Cloud-CV/EvalAI
Issue: Login fails for dafault user 
Root cause: Environment Incompatibility

_
- the expected behaviour
Login of default user should be successful
- the observed behaviour
LOgin window keeps mum, doing nothing
- steps to reproduce the problem
Installed docker and docker-compose utility. and git clone evalAI from master branch. 
cd evalAI
docker-compose up --build 

It will start service on localhost:8888. click on login button use user admin and password as password.


- which operating system and/or browser you're using, if relevant
I have tried it on Windows 10 and CentOS 7 with latest chrome browser.

Thanks!


@samcom12 I am not able to reproduce this issue. Can you please try a fresh install? Please make sure to remove all the EvalAI related docker containers first. 

Closing this for now. Please feel free to reopen if you still see the issue after retrying based on the above instructions.
Repository: Cloud-CV/EvalAI
Issue: Cluster and node group creation should be a background process
Root cause: Architectural Constraint

After approving the challenge we get 504 Timeout error from nginx

<img width="1440" alt="Screenshot 2020-06-27 at 2 35 47 PM" src="https://user-images.githubusercontent.com/12206047/85929523-98159800-b883-11ea-9431-b619627605ac.png">
Repository: Cloud-CV/EvalAI
Issue: Create Apis for Test Environment model
Root cause: Unimplemented Feature Gap

Following apis should be created
* get list of all test environments for a particular challenge
* create a new test environment
* get details about a particular test environment
* update(partial also) a particular test environment
* delete specific test environment

@TroJan - This issue is resolved completely? 
Please feel free to close this issue if it's complete. 

For reference, Test Environment model is now ChallengePhase
Repository: Cloud-CV/EvalAI
Issue: Add meta attribute validation to challenge_config_validation API.
Root cause: Unimplemented Feature Gap

To Do:

- [ ] Add code to validate the submission meta attributes in a challenge, to the `challenge_config_validation` API in `apps/challenges/views.py`

Done in this PR: https://github.com/Cloud-CV/EvalAI/pull/2965
Repository: Cloud-CV/EvalAI
Issue: Change the code to handle the simultaneous change in eval_script and test_annotation at the same time.
Root cause: Unimplemented Feature Gap

ToDo: (Minor change.)

- [ ] When challenge host uploads a new eval_script and annotation file together, a mail with the appropriate message should be sent.
- [ ] Worker(s) should be restarted similar to that in the individual case.


@KhalidRmb I would like to work on this issue, but I am unable to find the piece of code where it sends an email after a `PUT` or `PATCH` request in [challenge_detail](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L200) function ([Line 222](https://github.com/Cloud-CV/EvalAI/blob/master/apps/challenges/views.py#L222) to be more specific)
Repository: Cloud-CV/EvalAI
Issue: Poll to see if the worker container has started while fetching logs.
Root cause: Unimplemented Feature Gap

**Why?**
It takes a while to start the worker containers on AWS, and the host may try to submit or expect logs without knowing this.

**ToDo:**

- [ ] Have a way to indicate on the challenge page/ Manage tab if the worker is running/transitioning/stopped/Non existent.


@KhalidRmb This might be helpful here https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/ecs.html#ECS.Client.get_waiter
Repository: Cloud-CV/EvalAI
Issue: Handle the case where log group hasn't been created yet when displaying logs.
Root cause: Validation Gap

It takes about 1.5 minutes for the logs group to get created on Cloudwatch once the workers are started for the first time. And if the host tries to view the logs in this period, they'll get an error message.

**ToDo:**

- [x] Change the message to something more appropriate, or wait for the creation of log group before sending any message back to the frontend.
Repository: Cloud-CV/EvalAI
Issue: Fixes in challenge approval email

![Screenshot 2020-07-02 at 2 26 27 PM](https://user-images.githubusercontent.com/12206047/86396512-09808c80-bc70-11ea-8650-e1cb44f97cba.png)
The issues in the above email are -
- [ ] Demo2 button CSS is not showing up.
- [ ] Demo2 button hyperlink is wrong.

cc: @Ram81 

This can be closed since I fixed the template with inline CSS, and the link seems to be working fine with all kinds of env variables while I tested the method.

<img width="1368" alt="Screenshot 2020-07-18 at 1 38 02 AM" src="https://user-images.githubusercontent.com/31621523/87826893-559a1680-c897-11ea-8552-ba06077a947e.png">

Earlier problem could have risen out of improper environment variable while testing on staging?

Closing this since it is a very old issue.
Repository: Cloud-CV/EvalAI
Issue: Link doesn't work for email notifying about about worker restart.

<img width="1549" alt="Screenshot 2020-07-03 at 3 42 23 AM" src="https://user-images.githubusercontent.com/31621523/86413611-3e77f800-bcdf-11ea-9534-9cff891cea75.png">

CSS is not rendered in the frontend.

Possible solution for CSS: Change to inline in the Email template.

Closing this since I fixed the template with inline CSS, and the link seems to be working fine with all kinds of env variables when testing.
Earlier problem could have risen out of improper environment variable while testing on staging?
Repository: Cloud-CV/EvalAI
Issue: Send email to challenge hosts only when the worker reload is complete and successful
GitHub labels: bug

backend

**Current Scenario**: Whenever a challenge hosts updates an evaluation script or the test annotation file then the email for reloading is sent to them immediately after the upload of annotations file or the evaluation script file which gives a wrong sense to challenge hosts that the worker is reloaded but actually it is not because it takes 2 mins on AWS to reload it successfully.

**Fix**:
1. Send the email to challenge hosts immediately after upload saying that worker reload is started and it will take 2 mins to complete and they will be notified again when it is successfully reloaded.
2. Send another email to challenge hosts when the worker reload is successful on AWS.

Hey, Is the issue still open?
Repository: Cloud-CV/EvalAI
Issue: Create 404 page for Client side.
GitHub labels: enhancement

GSOC-2019

easy_to_fix

frontend

Just add a content for 404 and a link for home page of Evalai.

@aka-jain I will fix it.

@RishabhJain2018 : Please ask before started working on any issue, it'll be god for managing tasks :)

@aka-jain sure.

@RishabhJain2018 : are you still working on it? Let me know please, so that others can work on this.

@aka-jain Yeah I am working on the issue. I apologize for the delay.

Done with PR #314 .
Closing it.
Repository: Cloud-CV/EvalAI
Issue: Stop and delete the challenge workers on AWS automatically when the challenge is un-approved by admin
Root cause: Unimplemented Feature Gap

**Description**
The challenge worker on Fargate keeps on running even when the challenge is unapproved by the EvalAI Admin.

**Fix**
Ideally, when the challenge is unapproved by the admin then the workers on Fargate should be stopped and deleted.
Repository: Cloud-CV/EvalAI
Issue: Refactor aws_keys initialization.
Root cause: Architectural Constraint

Wherever aws keys are used in the codebase, change it for the keys to be loaded from a method in `base/utils.py`

I would like to work on it but can you guide me a little more?

@KhalidRmb is this issue still open ?


Hey @codeboy5 , Yes this is still open!

Then i would like to take it up ! 
Thanks

Also can you point to place where aws keys are currently loaded ?

Here is the method which should be used -- `https://github.com/Cloud-CV/EvalAI/blob/9d522f86dead523180de6e9efaf9dce2bc9d29b1/apps/challenges/utils.py#L102`
Repository: Cloud-CV/EvalAI
Issue: [Frontend] Improvements for new frontend deployment

### Note: This list is incomplete. Adding small issues here as going through the new version. Please add your list of issues in the comment rather than editing the description of the issue.

- [ ] Home page should not be centrally aligned (it looks really bad on large screens)
- [ ] Remove `console.log()` statements
- [ ] Github stats (number of forks and stars) not showing up in the footer
- [ ] Remove the note -- `Unlicensed copy of the Froala Editor. Use it legally by purchasing a license.` from the editor or use a different editor. (Fig 1.)
- [ ] Calendar UI doesn't look right -- (Fig 2.)
- [ ] Inconsistency in the Font -- (Fig. 3)
- Figure 1
<img width="867" alt="Screenshot 2020-08-08 at 1 45 03 AM" src="https://user-images.githubusercontent.com/12206047/89703447-ca052a00-d918-11ea-84f4-54c03dad365d.png">

- Figure 2
<img width="303" alt="Screenshot 2020-08-08 at 1 48 20 AM" src="https://user-images.githubusercontent.com/12206047/89703488-53b4f780-d919-11ea-8a40-3c41f5a9da7d.png">

- Figure 3
<img width="241" alt="Screenshot 2020-08-08 at 1 51 40 AM" src="https://user-images.githubusercontent.com/12206047/89703545-c7ef9b00-d919-11ea-9fa5-c18c408b4d1a.png">
Repository: Cloud-CV/EvalAI
Issue: Enhancements to v1 deployment script in EvalAI

List of small scale changes to make to v1 deployment script (#2853):
- Get staging flag manually instead of hardcoding
- Get SSL certs manually by making a new nginx config
- Add better docs & comments
- Refactor nodejs to use prod version and make SED changes via script

Hey @RishabhJain2018 ! 
I've been away for some time now. Most of the work on deployment is done with some tweaks remaining. Is this still something that is still in the roadmap for EvalAI ?

Closing this since it is very old and deployment setup is changed.