Repository: Cloud-CV/EvalAI
Issue: Log change in Challenge and Challenge Phase model when file fields are updated
GitHub labels: enhancement

GSOC-2019

medium-difficulty

backend

__Problem__

We need a way to figure out if the `evaluation_script`field value is changed for Challenge Model or not. Same goes for `test_annotation` field in ChallengePhase Model.

__Please be more clear__ ?

When ever a model is saved, its saved function is called. So this task involves figuring out a way in save model to detect if the model is updated or added. If the model is updated, then further it should be figured out whether the `evaluation_script` is changed for Challenge model or not.

If `evaluation_script` of Challenge model is changed, then simply print a message stating the change. This print statement will later on act as a logging statement in case we need to debug something.

The above requested thing goes same for `test_annotation` field of ChallengePhase model.

__Why__ ?

Submission worker when starts loads all `evaluation_script` and `test_annotation` file in advance. In case if the Challenge Host updates any of the file, there is no way to communicate this to the worker

__Impact__ ?

It actually guarantees that whenever any submission is evaluated, it is evaluated against the write set of parameters.

__How__ ?

As of now, I think it can be done via following ways

* Either overriding the save method of each model and adding a hook over there to check if the instance is being updated or added. May be there is a flag or something which tells whether the instance is added or updated

* Either by [signals](https://docs.djangoproject.com/en/1.10/topics/signals/). Reason I am asking to use signal because we need to carry out the same set of action for more than one models(presently _Challenge_ and _ChallengePhase_). Doing it by signal will help us achieve more modularized code.

* Your suggestion ??

__TakeAways__

About how django model is saved, how signal works, and a brief intro about RabbitMQ and submission worker

Also I will prefer that __before coding__, its better to __discuss__ the approach over here only, so that we can get an idea and update if something is not right.

@TroJan Can I work on this?

@RishabhJain2018 : sure

@TroJan I researched about the above task and found that [signals](https://docs.djangoproject.com/en/1.10/topics/signals/) are a suitable way to implement it. Please see these links ([link1](http://stackoverflow.com/a/1197708) [link2](http://stackoverflow.com/questions/8119570/a-correct-case-to-use-django-signals)) link for reference.
Moreover, I also found a django-app [django-model-changes](https://github.com/kajic/django-model-changes) to track the state and changes of django model instances.
What are your views about it?

@RishabhJain2018 : yeah we will be using signals since we need this functionality for multiple models(presently _Challenge_ and _ChallengePhase_). I will prefer that instead of using `django-model-changes` we write our own code, because we do not need to track the history of change(as of now).

@TroJan so, I am proceeding with signals.
I have a doubt that where we have to print the message? with reference to your statement **If evaluation_script of Challenge model is changed, then simply print a message stating the change. This print statement will later on act as a logging statement in case we need to debug something.**

  

@RishabhJain2018 : the message will be printed in the catched signal. This message is kind of a logging statement telling what was the change. So it will be placed where the change is actually being detected. 

@RishabhJain2018 : any updates here?

@TroJan I apologise for the delay..I will finish this asap.