Repository: fossasia/open-event-server

Issue:
fix: count query of sold tickets
<!--
(Thanks for sending a pull request! Please make sure you click the link above to view the contribution guidelines, then fill out the blanks below.)
-->
<!-- Add the issue number that is fixed by this PR (In the form Fixes #123) -->

Fixes #6653 

https://github.com/fossasia/open-event-server/issues/6653#issuecomment-565030517


#### Short description of what this resolves:

Count query of sold tickets. Solution: Calculate only placed orders + initializing orders which were created under order expiry time

#### Checklist

- [x] I have read the [Contribution & Best practices Guide](https://blog.fossasia.org/open-source-developer-guide-and-best-practices-at-fossasia) and my PR follows them.
- [x] My branch is up-to-date with the Upstream `development` branch.
- [x] The unit tests pass locally with my changes <!-- use `nosetests tests/all` to run all the tests -->
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] I have added necessary documentation (if appropriate)
<!-- If an existing function does not have a docstring, please add one -->
- [x] All the functions created/modified in this PR contain relevant docstrings.



Discussion:
# [Codecov](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=h1) Report
> Merging [#6693](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=desc) into [development](https://codecov.io/gh/fossasia/open-event-server/commit/f27e08d744c139182c651085e53d20067970d9a0?src=pr&el=desc) will **increase** coverage by `0.06%`.
> The diff coverage is `97.22%`.

[![Impacted file tree graph](https://codecov.io/gh/fossasia/open-event-server/pull/6693/graphs/tree.svg?width=650&token=HNBiyBrcBU&height=150&src=pr)](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=tree)

```diff
@@              Coverage Diff               @@
##           development   #6693      +/-   ##
==============================================
+ Coverage        65.44%   65.5%   +0.06%     
==============================================
  Files              300     300              
  Lines            15297   15324      +27     
==============================================
+ Hits             10011   10038      +27     
  Misses            5286    5286
```


| [Impacted Files](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [app/models/order.py](https://codecov.io/gh/fossasia/open-event-server/pull/6693/diff?src=pr&el=tree#diff-YXBwL21vZGVscy9vcmRlci5weQ==) | `90% <100%> (ø)` | :arrow_up: |
| [app/factories/attendee.py](https://codecov.io/gh/fossasia/open-event-server/pull/6693/diff?src=pr&el=tree#diff-YXBwL2ZhY3Rvcmllcy9hdHRlbmRlZS5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [tests/all/integration/api/helpers/test\_order.py](https://codecov.io/gh/fossasia/open-event-server/pull/6693/diff?src=pr&el=tree#diff-dGVzdHMvYWxsL2ludGVncmF0aW9uL2FwaS9oZWxwZXJzL3Rlc3Rfb3JkZXIucHk=) | `98.41% <100%> (+0.79%)` | :arrow_up: |
| [app/api/attendees.py](https://codecov.io/gh/fossasia/open-event-server/pull/6693/diff?src=pr&el=tree#diff-YXBwL2FwaS9hdHRlbmRlZXMucHk=) | `40% <85.71%> (+2.22%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=footer). Last update [f27e08d...bee05bf](https://codecov.io/gh/fossasia/open-event-server/pull/6693?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@iamareebjamal What command should  I run locally to get the hound violations here on my local?
Please add tests as well
> What command should I run locally to get the hound violations here only?

Not sure
I am away from keyboard for today, will update the PR as soon as I get back in the evening
@iamareebjamal Why not to count the placed orders and tickets in here https://github.com/fossasia/open-event-server/blob/f27e08d744c139182c651085e53d20067970d9a0/app/api/order_statistics/events.py#L37
. here we can event correct the sales statistics all over the web app. As of now we are not showing the sales data and ticket count of Offline Tickets in the dashboard? 
We are showing it, but separately. And yes, this is where it will be placed finally. This PR is work in progress
@iamareebjamal For the test, shall I write functions with the multiple status of orders? or seperate.

Also while creating attendee I can't change the order status, so shall I run the only create the order or shall I create the attendee.?
> For the test, shall I write functions with the multiple status of orders? or seperate.

What?

> Also while creating attendee I can't change the order status, so shall I run the only create the order or shall I create the attendee.?

You need to create both
@iamareebjamal - While creating both, 2 orders and 1 attendee is created as in attendee factory an order factory reference is also their.

>What?

For the query to test, I am creating a single function with multiple attendees and I am running this query to check that count is valid or not.
@iamareebjamal - Guide me
> While creating both, 2 orders and 1 attendee is created as in attendee factory an order factory reference is also their.

Split it and use base

https://github.com/fossasia/open-event-server/blob/development/app/factories/session.py
> For the query to test, I am creating a single function with multiple attendees and I am running this query to check that count is valid or not.

Create 2 attendees with no order ID, 6 with completed order, 1 with placed, 4 with initializing/initialized and not expired (created_at within order expiry time), 3 with initializing but expired created_at, 5 order status == expired

The count should be 11

> > While creating both, 2 orders and 1 attendee is created as in attendee factory an order factory reference is also their.
> 
> Split it and use base
> 
> https://github.com/fossasia/open-event-server/blob/development/app/factories/session.py

Shall this be done in another PR? as it requires changes everywhere it's used
It requires changes in just one file. attendee factory
Ok got that
@iamareebjamal Review.
@iamareebjamal - When we have same order_id then the count will not be `11` as the query runs on order ids not attendees.
Then what's the need for creating multiple attendees?

> Create 2 attendees with no order ID, 6 with completed order, 1 with placed, 4 with initializing/initialized and not expired (created_at within order expiry time), 3 with initializing but expired created_at, 5 order status == expired
The count should be 11
`query runs on order ids not attendees`

Then the query is wrong. Great, testing caught us a bug
@iamareebjamal - `Count query of sold tickets. Solution: Calculate only placed orders + initializing orders which were created under order expiry time`

Then tell me what shall be counted?

Also it's possible in the system that an order is created but the attendee is not or vice versa?
I think it is pretty clear. What was being counted before?
> I think it is pretty clear. What was being counted before?

Attendees. Okay so you want me to run a join query on the attendees and the orders ? Right
Yes
@iamareebjamal Review now!
@iamareebjamal Check now!
Great work. Rename the function to `get_sold_and_reserved_tickets_count` and add comments in test about which attendees should be counted
@codedsun Need to handle `initialized` orders created in less than 30 minutes as well

@kushthedude This will handle them, right?
@iamareebjamal So we have to take account of `initialized` orders under 30 minutes and not under order-expiry-time, but what about the `pending` state orders? Also, the system from the frontend should not tell the backend about the state of order, it's a bit weird here.
> @codedsun Need to handle `initialized` orders created in less than 30 minutes as well
> 
> @kushthedude This will handle them, right?

Initialised order created in 30+order_expiry_time minutes
There is no pending state. And yes, new Order API will change things so frontend doesn't control the order state
@iamareebjamal What i see here is something else.

https://github.com/fossasia/open-event-server/blob/f27e08d744c139182c651085e53d20067970d9a0/app/api/schema/orders.py#L72-L75

That's wrong. There is no pending state in production DB
> That's wrong. There is no pending state in production DB

Then what is the status getting assigned to pending orders or orders with online payment due?
@iamareebjamal Check, I have created also 2 attendees with the same `initialized` state in the test and checked for them also. The test is passing 👍 
> Then what is the status getting assigned to pending orders or orders with online payment due

Initialized AFAIK
Waiting for verification of a few things. Will merge then
@iamareebjamal Sure, thanks a lot. It was fun solving this. I am taking other issues now.
They are assigned pending,Also the reason for pending not be present in db
is because it either gets converted into completed after successful payment
or to expired after failing to pay within 30 min of creation.
I dont know what is the purpose of initialized status, as FE is not
labeling any order with initialized.

On Sat, 28 Dec, 2019, 16:04 Suneet Srivastava, <notifications@github.com>
wrote:

> @iamareebjamal <https://github.com/iamareebjamal> Sure, thanks a lot. It
> was fun solving this. I am taking other issues now.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/fossasia/open-event-server/pull/6693?email_source=notifications&email_token=AKQMTLWFSJDQSGRVGBJ3ZSTQ24TRTA5CNFSM4J6XTT3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYG6OI#issuecomment-569405241>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKQMTLUQK6XLZXCN7AOYO3TQ24TRTANCNFSM4J6XTT3A>
> .
>

Alright, please change initialized to pending @codedsun 

I just confirmed.

Don't know how but there are 257 orders in initialized state in production
@iamareebjamal Check now!

Resolving patch:
diff --git a/app/api/attendees.py b/app/api/attendees.py
--- a/app/api/attendees.py
+++ b/app/api/attendees.py
@@ -24,14 +24,17 @@
 from app.settings import get_settings
 
 
-def get_sold_and_reserved_tickets_count(event_id):
+def get_sold_and_reserved_tickets_count(ticket_id):
     order_expiry_time = get_settings()['order_expiry_time']
     return (
         db.session.query(TicketHolder.id)
         .join(Order)
-        .filter(TicketHolder.order_id == Order.id)
         .filter(
-            Order.event_id == int(event_id),
+            TicketHolder.ticket_id == ticket_id,
+            TicketHolder.order_id == Order.id,
+            TicketHolder.deleted_at.is_(None),
+        )
+        .filter(
             Order.deleted_at.is_(None),
             or_(
                 Order.status == 'placed',
@@ -82,7 +85,7 @@ def before_post(self, args, kwargs, data):
                 "Ticket belongs to a different Event",
             )
         # Check if the ticket is already sold out or not.
-        if get_sold_and_reserved_tickets_count(ticket.event_id) >= ticket.quantity:
+        if get_sold_and_reserved_tickets_count(ticket.id) >= ticket.quantity:
             raise ConflictException(
                 {'pointer': '/data/attributes/ticket_id'}, "Ticket already sold out"
             )


Test patch:
diff --git a/tests/all/integration/api/helpers/test_order.py b/tests/all/integration/api/helpers/test_order.py
--- a/tests/all/integration/api/helpers/test_order.py
+++ b/tests/all/integration/api/helpers/test_order.py
@@ -62,6 +62,7 @@ def test_count_sold_and_reserved_tickets(self):
 
         with self.app.test_request_context():
             ticket = TicketFactory()
+            other_ticket = TicketFactory()
 
             completed_order = OrderFactory(status='completed')
             placed_order = OrderFactory(status='placed')
@@ -79,24 +80,48 @@ def test_count_sold_and_reserved_tickets(self):
             db.session.commit()
 
             # will not be counted as they have no order_id
-            AttendeeFactoryBase.create_batch(2)
+            AttendeeFactoryBase.create_batch(2, ticket_id=ticket.id)
             # will be counted as attendee have valid orders
-            AttendeeFactoryBase.create_batch(6, order_id=completed_order.id)
+            AttendeeFactoryBase.create_batch(
+                6, order_id=completed_order.id, ticket_id=ticket.id
+            )
             # will be counted as attendee has valid placed order
-            AttendeeFactoryBase(order_id=placed_order.id)
+            AttendeeFactoryBase(order_id=placed_order.id, ticket_id=ticket.id)
+            # will not be counted as they are deleted
+            AttendeeFactoryBase.create_batch(
+                3,
+                order_id=placed_order.id,
+                ticket_id=ticket.id,
+                deleted_at=datetime.utcnow(),
+            )
             # will be counted as attendee has initializing order under order expiry time
-            AttendeeFactoryBase.create_batch(4, order_id=initializing_order.id)
+            AttendeeFactoryBase.create_batch(
+                4, order_id=initializing_order.id, ticket_id=ticket.id
+            )
             # will be counted as attendee has pending order under 30+order expiry time
-            AttendeeFactoryBase.create_batch(2, order_id=pending_order.id)
+            AttendeeFactoryBase.create_batch(
+                2, order_id=pending_order.id, ticket_id=ticket.id
+            )
             # will not be counted as the order is not under order expiry time
-            AttendeeFactoryBase.create_batch(3, order_id=expired_time_order.id)
+            AttendeeFactoryBase.create_batch(
+                3, order_id=expired_time_order.id, ticket_id=ticket.id
+            )
             # will not be counted as the order has an expired state
-            AttendeeFactoryBase.create_batch(5, order_id=expired_order.id)
+            AttendeeFactoryBase.create_batch(
+                5, order_id=expired_order.id, ticket_id=ticket.id
+            )
+            # will not be counted as the attendees have different ticket ID
+            AttendeeFactoryBase.create_batch(
+                2, order_id=completed_order.id, ticket_id=other_ticket.id
+            )
 
-            count = get_sold_and_reserved_tickets_count(ticket.event_id)
+            count = get_sold_and_reserved_tickets_count(ticket.id)
 
             self.assertEqual(count, 13)
 
+            # Last 2 attendees belong to other ticket
+            self.assertEqual(get_sold_and_reserved_tickets_count(other_ticket.id), 2)
+
 
 if __name__ == '__main__':
     unittest.main()