Repository: fossasia/open-event-server

Issue:
After-Event Email: Several identical mails sent out, if user has several sessions/roles
If a user has several sessions and roles, e.g. speaker and event organizer, several after-event emails are sent out. Expected: Only send one email per email address.

![Screenshot from 2020-01-08 11-02-59](https://user-images.githubusercontent.com/1583873/71952996-c8e70f80-3206-11ea-916f-f9e332d70faf.png)



Discussion:
@iamareebjamal server issue
Bug in APS, to be replaced by celery beat. Every gunicorn worker including celery runs a copy of APS, and hence everything is repeated 5 times (4 workers + 1 celery)
APS: Python Scheduler ? I will be looking into it.
Taking this up! 👍 

> APS: Python Scheduler ? I will be looking into it.

Also there are already 3-4 PR opened by you which are not merged. Please work on them before making more new one's.
> Also there are already 3-4 PR opened by you which are not merged. Please work on them before making more new one's.

It has been 3 days and no one was working and the priority is HIGH also.
@kushthedude If you are working on this, then no problem. Also yes they are in the pending state due to the solution being searched simultaneously as I m stuck in both and in sync with Areeb (You can see the conversation). And if you are not working on this, then I can send the patch, why not move the project forward simultaneously. Please tell what should be done?

Thanks.
There are a lot of issues already present,needless to say you should give
atleast a week to other dev before claiming the issue if they are working
on it. If you dont hear from them a week or dont see a patch you should
definitely go ahead after commenting.
Each contributor should get an equal opportunity for contribution whether
it be me, you or any new comer.
Anyway, If you don't have issue to work on, you can go ahead with it. No
issues 😄

On Fri, 10 Jan, 2020, 22:20 Suneet Srivastava, <notifications@github.com>
wrote:

> Also there are already 3-4 PR opened by you which are not merged. Please
> work on them before making more new one's.
>
> It has been 3 days and no one was working and the priority is HIGH also.
> @kushthedude <https://github.com/kushthedude> If you are working on this,
> then no problem. Also yes they are in the pending state due to the solution
> being searched simultaneously as I m stuck in both and in sync with Areeb
> (You can see the conversation). And if you are not working on this, then I
> can send the patch, why not move the project forward simultaneously. Please
> tell what should be done?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/fossasia/open-event-server/issues/6731?email_source=notifications&email_token=AKQMTLSD5G3ZLZAAWPSSZT3Q5CRMPA5CNFSM4KEER33KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIUQSKQ#issuecomment-573114666>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKQMTLTXERWZOAYMVUWWLXDQ5CRMPANCNFSM4KEER33A>
> .
>

@codedsun Are you working on it?
No, @vishakhanihore You may ask @kushthedude if he is working on it.
Sure go ahead.

On Sat, 11 Jan, 2020, 09:48 Suneet Srivastava, <notifications@github.com>
wrote:

> No, @vishakhanihore <https://github.com/vishakhanihore> You may ask
> @kushthedude <https://github.com/kushthedude> if he is working on it.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/fossasia/open-event-server/issues/6731?email_source=notifications&email_token=AKQMTLXVPYDKWTVLRBZFE4DQ5FB77A5CNFSM4KEER33KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIVY2RY#issuecomment-573279559>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKQMTLVZKMQUEXAME3RPH6TQ5FB77ANCNFSM4KEER33A>
> .
>

@vishakhanihore - Are you still working on this?
Didnt get time to finish it. You may go ahead if you want.

On Sun, 19 Jan, 2020, 18:35 Suneet Srivastava, <notifications@github.com>
wrote:

> @vishakhanihore <https://github.com/vishakhanihore> - Are you still
> working on this?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/fossasia/open-event-server/issues/6731?email_source=notifications&email_token=AM6PS4WDCABD4SW6FFVBRP3Q6RFZBA5CNFSM4KEER33KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJKRUZQ#issuecomment-576002662>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AM6PS4WMNTIMMBWNXQPQI5DQ6RFZBANCNFSM4KEER33A>
> .
>

@iamareebjamal - In this, all the scheduled jobs will become a celery task and then be scheduled by celery beat. correct?
yes

Resolving patch:
diff --git a/app/api/helpers/scheduled_jobs.py b/app/api/helpers/scheduled_jobs.py
--- a/app/api/helpers/scheduled_jobs.py
+++ b/app/api/helpers/scheduled_jobs.py
@@ -3,6 +3,8 @@
 import pytz
 from dateutil.relativedelta import relativedelta
 from flask import render_template
+from flask_celeryext import RequestContextTask
+from app.instance import celery
 
 from app.api.helpers.db import safe_query, save_to_db
 from app.api.helpers.mail import send_email_after_event, send_email_for_monthly_fee_payment, \
@@ -26,6 +28,7 @@
 from app.settings import get_settings
 
 
+@celery.task(base=RequestContextTask, name='send.after.event.mail')
 def send_after_event_mail():
     from app.instance import current_app as app
     with app.app_context():
@@ -52,6 +55,7 @@ def send_after_event_mail():
                     send_notif_after_event(owner.user, event.name)
 
 
+@celery.task(base=RequestContextTask, name='change.session.state.on.event.completion')
 def change_session_state_on_event_completion():
     from app.instance import current_app as app
     with app.app_context():
@@ -62,6 +66,7 @@ def change_session_state_on_event_completion():
             save_to_db(session, 'Changed {} session state to rejected'.format(session.title))
 
 
+@celery.task(base=RequestContextTask, name='send.event.fee.notification')
 def send_event_fee_notification():
     from app.instance import current_app as app
     with app.app_context():
@@ -84,7 +89,7 @@ def send_event_fee_notification():
                 for order_ticket in order.tickets:
                     ticket = safe_query(db, Ticket, 'id', order_ticket.ticket_id, 'ticket_id')
                     if order.paid_via != 'free' and order.amount > 0 and ticket.price > 0:
-                        fee = ticket.price * (get_fee(order.event.payment_currency) / 100.0)
+                        fee = ticket.price * (get_fee(event.payment_country, order.event.payment_currency) / 100.0)
                         fee_total += fee
 
             if fee_total > 0:
@@ -93,7 +98,7 @@ def send_event_fee_notification():
                     amount=fee_total, event_id=event.id, user_id=owner.user.id)
 
                 if event.discount_code_id and event.discount_code:
-                    r = relativedelta(datetime.utcnow(), event.created_at)
+                    r = relativedelta(datetime.datetime.utcnow(), event.created_at)
                     if r <= event.discount_code.valid_till:
                         new_invoice.amount = fee_total - \
                             (fee_total * (event.discount_code.value / 100.0))
@@ -120,6 +125,7 @@ def send_event_fee_notification():
                                                new_invoice.event_id)
 
 
+@celery.task(base=RequestContextTask, name='send.event.fee.notification.followup')
 def send_event_fee_notification_followup():
     from app.instance import current_app as app
     with app.app_context():
@@ -147,6 +153,7 @@ def send_event_fee_notification_followup():
                                                         incomplete_invoice.event.id)
 
 
+@celery.task(base=RequestContextTask, name='expire.pending.tickets')
 def expire_pending_tickets():
     from app.instance import current_app as app
     with app.app_context():
@@ -156,6 +163,7 @@ def expire_pending_tickets():
         db.session.commit()
 
 
+@celery.task(base=RequestContextTask, name='delete.ticket.holders.no.order.id')
 def delete_ticket_holders_no_order_id():
     from app.instance import current_app as app
     with app.app_context():
@@ -166,6 +174,7 @@ def delete_ticket_holders_no_order_id():
         db.session.commit()
 
 
+@celery.task(base=RequestContextTask, name='event.invoices.mark.due')
 def event_invoices_mark_due():
     from app.instance import current_app as app
     with app.app_context():
@@ -177,6 +186,7 @@ def event_invoices_mark_due():
         ).update({EventInvoice.status: 'due'}, synchronize_session=False)
 
 
+@celery.task(base=RequestContextTask, name='send.monthly.event.invoice')
 def send_monthly_event_invoice():
     from app.instance import current_app as app
     with app.app_context():
@@ -211,3 +221,16 @@ def send_monthly_event_invoice():
 
             event_invoice = EventInvoice(amount=net_revenue, invoice_pdf_url=pdf, event_id=event.id)
             save_to_db(event_invoice)
+
+
+@celery.on_after_configure.connect
+def setup_scheduled_task(sender, **kwargs):
+    from celery.schedules import crontab
+    sender.add_periodic_task(crontab(hour='*/5', minute=30), send_after_event_mail)
+    sender.add_periodic_task(crontab(day_of_week='0-6'), send_event_fee_notification)
+    sender.add_periodic_task(crontab(minute=0, hour=0, day_of_month=1), send_event_fee_notification_followup)
+    sender.add_periodic_task(crontab(hour='*/5', minute=30), change_session_state_on_event_completion)
+    sender.add_periodic_task(crontab(minute='*/45'), expire_pending_tickets)
+    sender.add_periodic_task(crontab(minute=0, hour=0, day_of_month=1), send_monthly_event_invoice)
+    sender.add_periodic_task(crontab(minute=0, hour='*/5'), event_invoices_mark_due)
+    sender.add_periodic_task(crontab(minute='*/5'), delete_ticket_holders_no_order_id)
diff --git a/app/instance.py b/app/instance.py
--- a/app/instance.py
+++ b/app/instance.py
@@ -37,10 +37,6 @@
 from app.views.blueprints import BlueprintsManager
 from app.api import routes
 from app.api.helpers.auth import AuthManager, is_token_blacklisted
-from app.api.helpers.scheduled_jobs import send_after_event_mail, send_event_fee_notification, \
-    send_event_fee_notification_followup, change_session_state_on_event_completion, \
-    expire_pending_tickets, send_monthly_event_invoice, event_invoices_mark_due, \
-    delete_ticket_holders_no_order_id
 from app.models.event import Event
 from app.models.role_invite import RoleInvite
 from app.views.healthcheck import health_check_celery, health_check_db, health_check_migrations, check_migrations
@@ -234,8 +230,11 @@ def track_user():
 # it is important to register them after celery is defined to resolve circular imports
 
 from .api.helpers import tasks
-
 celery = tasks.celery
+# register scheduled jobs
+from app.api.helpers.scheduled_jobs import setup_scheduled_task
+setup_scheduled_task(celery)
+
 
 # http://stackoverflow.com/questions/9824172/find-out-whether-celery-task-exists
 @after_task_publish.connect
@@ -248,20 +247,6 @@ def update_sent_state(sender=None, headers=None, **kwargs):
     backend.store_result(headers['id'], None, 'WAITING')
 
 
-scheduler = BackgroundScheduler(timezone=utc)
-# scheduler.add_job(send_mail_to_expired_orders, 'interval', hours=5)
-# scheduler.add_job(empty_trash, 'cron', hour=5, minute=30)
-scheduler.add_job(send_after_event_mail, 'cron', hour=5, minute=30)
-scheduler.add_job(send_event_fee_notification, 'cron', day=1)
-scheduler.add_job(send_event_fee_notification_followup, 'cron', day=1, month='1-12')
-scheduler.add_job(change_session_state_on_event_completion, 'cron', hour=5, minute=30)
-scheduler.add_job(expire_pending_tickets, 'cron', minute=45)
-scheduler.add_job(send_monthly_event_invoice, 'cron', day=1, month='1-12')
-scheduler.add_job(event_invoices_mark_due, 'cron', hour=5)
-scheduler.add_job(delete_ticket_holders_no_order_id, 'cron', minute=5)
-scheduler.start()
-
-
 @app.errorhandler(500)
 def internal_server_error(error):
     if current_app.config['PROPOGATE_ERROR'] is True:


Test patch: