Repository: Cloud-CV/EvalAI
Issue: Add a more detailed guide for new contributors

Currently, there is no specific style guide defined for either of our codebases (EvalAI, EvalAI-ngx or EvalAI-CLI). There are some instructions within the [EvalAI documentation](https://evalai.readthedocs.io/en/latest/contribution.html#instructions-to-submit-code) about using PEP8 or airbnb style guide. But I think these are very preliminary and a more detailed guide can be written. 
This can include conventions that we follow by choice in our codebase. For example, using double quotes or single quotes, or any naming conventions for tests. This has several upsides:
* Maintaining consistency throughout the codebase.
* Working on similar files is easier as a new contributor.
* Saves a new contributor from a small confusion or having to look around in the codebase for conventions used.
* Saves the reviewers' time who have to give similar comments on multiple PRs.

Additionally, I think having documentation for what is expected in a certain place would be very helpful to new contributors.
For example, for the process of writing tests, we can tell where the different types of tests are located in the project and what kind of tests these are. Adding examples to support them would be cherry on the cake. As in, for writing tests for django views, we usually write [functional tests](https://www.guru99.com/functional-testing.html). For submission workers, we have [unit tests](http://softwaretestingfundamentals.com/unit-testing/) which use mock extensively. Workers also have [integration tests](http://softwaretestingfundamentals.com/integration-testing/), which test a combination of multiple units.

Note: I understand this might be a bit unnecessary so please let me know your feedback anyway :)

@nikochiko I think instead of writing we should use the standard guidelines which are followed by most. What you say?

@Suryansh5545 Yes, you are right. We should use the standard general guidelines (such as PEP8, airbnb etc.). But, as I have stated above,
> There are some instructions within the EvalAI documentation about using PEP8 or airbnb style guide. But I think these are very preliminary and a more detailed guide can be written.

There are some things on which the [PEP8 guide](https://www.python.org/dev/peps/pep-0008/) and corresponding standards for other languages do not say anything. In such a case, it is a choice of the developers, but this could differ from person to person (for example, indentation used -- tabs or spaces or single or double quotes). A common guide can help us keep these choices consistent throughout the codebase. 
Other than that, there are the decisions we have made ourselves. As in the example stated above, the types of tests written: 
> As in, for writing tests for django views, we usually write functional tests. For submission workers, we have unit tests which use mock extensively. Workers also have integration tests, which test a combination of multiple units.

In such cases, for a new contributor to know about the type of tests they should write they would either have to ask other members (which takes time, and a response could be delayed) or they will have to figure it out themselves. This is not a very good practice as:
1. This process is repetitive i.e. every new contributor will have similar questions
2. The new contributor might not be familiar with the concepts used. In such a case, resources linked in the contributing guide can help.
For example. if I need to write tests for django apps and I am not familiar with functional testing, then my tests wouldn't be consistent. It will take more time on my side as well as on the reviewer's side. Plus, the quality of the new code may also not be as good as if I had a strong knowledge of the concepts involved.

@nikochiko then first we have to see what is missing in pep8 and airbnb then make a report and submit it to core developers??

@Suryansh5545 It'd be great to have PEP8 detail everything for us. But PEP8 is only a styling guide. And even in styling, it cannot have specific opinions on some topics (for example, there is no reason why 4 spaces indentation is better than 2 spaces indentation). So it comes down to the decision of us as contributors. And as we all are working on a common project, it makes sense to follow one protocol. We do follow a protocol at a certain level, but it is very implicit and the reviewers/contributors already refer other files and units to make sure everything remains consistent. For example, we use long names for tests which describe each testcase in detail. (e.g. `test_extract_submission_data_when_submission_does_not_exist`).
What we could do with a guide is just state exactly that, so we have a definite decision to follow when there is a conflict.
And, this already is more than just `styling` details, these are decisions we have made to make sure our codebase is *readable*. So as I already said with the example of tests, I am talking about a "more detailed contributing guide" and not just a styling guide.

@nikochiko Thats what i am trying to say . We should create a documentation with both the style we use and the things that are missing like the naming conversion that you stated. But first we might need a report on what is to be included. What you say??

Similar to #2578 

Closing this since it is a very old issue.