Repository: Cloud-CV/EvalAI
Issue: Publish message whenever file field of Challenge and Challenge Phase changes
Root cause: Unimplemented Feature Gap

__Problem__

We need to propagate changes related to `test_annotation` and `evaluation_script` to the submission worker. The support for handling the change already in the worker is being taken care of in [Issue 840](https://github.com/Cloud-CV/EvalAI/issues/840). We need to add support for publishing message when ever any such change happens.

__Please be more clear__ ?

When ever `evaluation_script` or `test_annotation` file of Challenge or ChallengePhase changes, we need a way to tell this to submission worker that the files have changed and you need to update the files.

The best way can be by using a Queue based architecture and publishing a message to the exchange.

__Why__ ?

Submission worker when starts loads all `evaluation_script` and `test_annotation` file in advance. In case if the Challenge Host updates any of the file, there is no way to communicate this to the worker.

If the worker is not communicated about the changes, then submissions may be evaluated with the wrong set of parameters.

__Impact__ ?

It actually guarantees that whenever any submission is evaluated, it is evaluated against the write set of parameters.

__How__ ?

It can be done by capturing the update of concerned models(here _Challenge_ and _ChallengePhase_).
[Issue 839](https://github.com/Cloud-CV/EvalAI/issues/839) deals with this. The print statement in that issue can now be modified and there a new message can be published.

The __content__ of the message still needs to be figured out by the __contributor__. I will prefer that you publish only primary keys in the message but this is still open to discussion.

Regarding how to publish message, you can take help from function `publish_submission_message` of `jobs/sender.py`. The published message will have a routing key of `challenge.edit.*`.

__TakeAways__

How Exchanges work in RabbitMQ, how message are routed to the exchange ?

Also I will prefer that before coding, its better to discuss the approach over here only, so that we can get an idea and update if something is not right.

can i try ?


@gauthamzz : sure :)

@gauthamzz : Let me know if you are struck somewhere. We can discuss it over here. :)

I'm still reading, new to this, will get this done fast .I hope it's fine. 😊

@gauthamzz : Any update on this, You can actually ask here if you are stuck somewhere.

@gauthamzz : any progress on this?

@TroJan i been trying, but this is kinda very new . Could you help out?

@gauthamzz : yeah tell me where are you struck?

@gauthamzz : any update on this?

sorry ,i kind of got busy in exams and stuff ,hope it was okay 

@taranjeet Can I take over?

@isht3 : yeah please. :)

@taranjeet I've started working on this. I will soon let you know about the progress.

@taranjeet how do I confirm my email in development? I was trying to mimic the process of submitting a task to produce the issue. Do I have to do it through the API's?

@isht3 You can confirm it using django admin.

@RishabhJain2018 Okay, cool! 

@deshraj Can somebody assign me? :smile: 

@RishabhJain2018 The print statement mentioned in this issue is the model field logger you wrote in https://github.com/Cloud-CV/EvalAI/blob/57e5fb3d2fcdc8e6a7699638a320bdaade817101/apps/base/models.py#L40 Yes?  

@deshraj @RishabhJain2018 : Is this issue still required? If not can we please close it?

Similar to #842. Closing it. Thanks for pointing out @taranjeet :)