giseldo/neodataset huggingface dataset:
CI Minutes provisioning - New mechanism for storing CI Minutes purchases on GitLab

### Establish CI Minute Purchase Identifiers on GitLab

The following outlines requirements from the architecture proposal as defined in !3142, we've split up the tasks into separate issues along system boundaries. 

**Base components**

- [x] Add a table in the GitLab DB to store the array of purchase identifier data structures from CustomersDot.

**API (re-)Sync components**

- [x] Create a new endpoint that accepts a POST/PATCH of CI minute purchases from CustomersDot following the unique Purchase Identifier
- [x] Reuse of the same endpoint for re-syncing purchases from CustomersDot is preferable as the sync action should be idempotent

**Support Namespace Changes**

- [x] Trigger an update of the namespace ID in the Purchase Identifier table from the current to the new namespace ID. 



<details>
<summary markdown="span">Previous Summary</summary>
## Summary

Create a mechanism for receiving and storing Additional CI Minutes purchases in GitLab core. Possibly in a `new table that tracks purchased buckets of minutes` the details of which are unknown at this time. Possibly include the expiry date (other payload ideas are above)
</details>

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

See https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2925

Story points: 5.0
Remove zuora_callback_async_processing feature flag

### Background

The [`zuora_callback_async_processing` feature flag](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/70/edit) has been on in staging and production for quite some time.  It was introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2078 about 6 months ago.

### Proposal

Update the `ZuoraCallbacksController` to use the `zuora_callback_async_processing` feature flagged code and remove the legacy code.  Once merged and deployed to production, delete the [feature flag from Unleash](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/70/edit).

Story points: 1.0
Documentation: SaaS Deprovisioning Processes

## Summary

In https://gitlab.com/groups/gitlab-org/-/epics/4627 there was quite a bit of investigation and work done to ensure we are properly moving SaaS subscriptions to the Free plan after the expiration date of the subscription (plus 14 day buffer) is reached.

The outcome of this issue is to document the technical processes and knowledge gained related to this topic in the CustomersDot repo for future reference.

Story points: 1.0
Use new Trial plan in reactivate trial service

Address the following non-blocking suggestions from https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3034

- Update [`Gitlab::HostedPlans::ReactivateTrialService#update_order_attrs`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/services/gitlab/hosted_plans/reactivate_trial_service.rb#L32) to use the new trial plan `::Plan::ULTIMATE_SAAS_TRIAL_PLAN`
- As noted in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3034#note_544333441, add a comment to explain why `Plan::ULTIMATE_SAAS_1_YEAR_PLAN` is used for checking [`TrialController#require_gl_com_authentication`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/controllers/trials_controller.rb#L96)

The following discussion from !3034 should be addressed:

- [ ] @asubramanian1 started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3034#note_543952701):  (+2 comments)

    > **question:**  Should we also update [`TrialController#require_gl_com_authentication`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/controllers/trials_controller.rb#L96) and in [`Gitlab::HostedPlans::ReactivateTrialService#update_order_attrs`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/services/gitlab/hosted_plans/reactivate_trial_service.rb#L32)?

Story points: 2.0
Unable to Purchase

## Summary

Users are reporting an error, during attempting self-service purchases. This has been seen with both attempting to purchase CI minutes and paid plan subscriptions. No commonality has been found yet amongst the reported examples.  

Looking at the browser breakdown in [the sentry error](https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2485235/events/540e9ec18f3945d49ad1ee257ae1f80f/) we've got the majority on FF but chrome is 40% of the reports.

```
Request failed with status code 401
```

Assumption: The problem seems to have started on April 1 (see https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2909#note_550142037)

https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2485235/?referrer=gitlab_plugin

![Screenshot_2021-03-31_Buy_more_GitLab_CI_minutes](/uploads/b229dac3ac5e841e2f32f7a250602aee/Screenshot_2021-03-31_Buy_more_GitLab_CI_minutes.png)

### Reported Examples

1. https://gitlab.zendesk.com/agent/tickets/202870
1. https://gitlab.zendesk.com/agent/tickets/203323
1. https://gitlab.zendesk.com/agent/tickets/202265
1. https://gitlab.zendesk.com/agent/tickets/205085
1. https://gitlab.zendesk.com/agent/tickets/204891
1. https://gitlab.zendesk.com/agent/tickets/205470 (switch from Firefox to Chrome worked)
1. https://gitlab.zendesk.com/agent/tickets/206922 (switch from FF to Chromium worked)
1. https://gitlab.zendesk.com/agent/tickets/203510 ("It perfectly worked using chrome")
1. https://gitlab.zendesk.com/agent/tickets/207326
1. https://gitlab.zendesk.com/agent/tickets/207330
1. https://gitlab.zendesk.com/agent/tickets/207940
1. https://gitlab.zendesk.com/agent/tickets/208544
1. https://gitlab.zendesk.com/agent/tickets/209573
1. https://gitlab.zendesk.com/agent/tickets/208479
1. https://gitlab.zendesk.com/agent/tickets/210765

Story points: 2.0
Keep the gl_namespace_name in sync with the namespace name in GitLab.com

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

When a user changes the name of their namespace in GitLab.com, the subscription object in CustomersDot gets out of sync because there's no notice of that change. In CustomersDot the customer still sees the outdated name of the namespace in their subscriptions page.

It's only after a change in the linked namespace for the subscription that the subscription object gets the updated `gl_namespace_name`.

### Proposal

Make a call to CustomersDot when the namespace name changes in GitLab.com to update the `gl_namespace_name` of the subscription.

### Result

Both `gl_namespace_name` and `gl_namespace_id` are kept up to date in GitLab.com, CustomersDot and Zuora (through the functionality implemented in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1989)

## Delivery plan

This logic is part of a bigger picture :frame_photo: 
- [x] CustomersDot: Add endpoint that receives a namespace(id, name) and updates the namespace name for all the orders associated to this namespace: https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3797
- [x] GitLab: Add sync capabilities every time the name for a namespace with an associated subscription is updated **behind a feature flag**: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69896
- [x] Create the enable feature flag issue: https://gitlab.com/gitlab-org/gitlab/-/issues/341056

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

If possible, we should create a test to mock the related APIs and test the data sent to the APIs.  Let's test the synchronicity between these fields and ensure that they stay in sync.

Story points: 3.0
Remove mimemagic shim

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Due to the mimemagic gem being yanked, we had to shim a fake version into CustomersDot in order to prevent any issues  (https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2994)

With https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2890, the Rails version is being upgraded which includes the change to remove the dependency on mimemagic, which means we can remove the shim :tada: 

### Proposal

Remove the vendor shim once the upgrade has been merged

### Result

The shim which is no longer neccessary, is removed from the codebase

Story points: 1.0
Update Rails to the latest version (6.1.3.1)

Update Rails to the latest version. This will update Rails' (or more Active Storage's) dependency to `Marcel` to version `10.0` ([release notes](https://weblog.rubyonrails.org/2021/3/26/marcel-upgrade-releases/)):

> Before 1.0.0, Marcel—which is distributed under the terms of the MIT License, like Rails—indirectly depended on MIME type data released under the incompatible GNU General Public License. Marcel 1.0.0 instead directly packages MIME type data adapted from Apache Tika, released under the permissive and compatible Apache License 2.0.

**Note:** With this upgrade we should be able to remove the created shim for mimemagic again.

Story points: 1.0
Follow-up from "Add initial params to pass to frontend"

The following discussion from !2936 should be addressed:

- [x] @jhyson started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2936#note_535900220): 

    > **suggestion:** would it be better to actually call this service here? I wonder if accessing this constant like this is leaking domain logic that should stay in this service? WDYT?

### SUMMARY

We want to validate this and replace it with an actual call to the Service.

We want to also account for a similar usage here: https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/0eff285f7cc84361d092c7134937ad090c5a681f/app/services/amendment_base_service.rb#L165

Story points: 2.0
Investigate Salesforce Account inconsistencies in CustomersDot, Salesforce and Zuora

### Problem
In https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2671 it was noticed that while the CustomersDot's customer stores a Salesforce Account Id its associated record in Salesforce could not be found (more info [here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2671#note_532386353)). This resulted in a raised error due to relying on always finding a record which was fixed in the mentioned issue.

Now the issue is that the a [presence validation exception](https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2542134/?referrer=slack) will be raised when the Salesforce Account is empty and an Opportunity is supposed to be created. This is because since it needs the information of an `OwnerId`. That field belongs to the Salesforce Account. More info regarding this can be found in [this issue](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2883).

### Proposal
Investigate the inconsistency around the Salesforce Account in CustomersDot, Salesforce and Zuora. As suggested by @jbrennan1 in this [comment](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2671#note_537873407) a walkthrough/demo with him through the standard Account/Contact creation flow is the first step of this investigation.

The actual fix for the exception will be handled in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2883.

### Result
Finding a solution on how to address a fix for the presence validation exception.

#### Recap of errors investigated as part of this issue

:mag: [Owner ID: owner cannot be blank](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2884#note_810168556)

:mag: [Required fields are missing: [ContactId]](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2884#note_810172823)

### Next steps (if any)
Coordinate a call with @jbrennan1 to do a walkthrough/demo.

Story points: 3.0
Salesforce Account can't be blank or it results in a validation error

### Problem
Original mention of this issue: https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2671#note_534741081

While working on https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2671 a fix was implemented to not raise an exception when the Salesforce Account can't be found and instead proceed with the remaining logic. While the remaining code accounted for the Salesforce Account to be blank all along, it never reached that code with an empty Salesforce Account before due to the mentioned exception. Which is why we are now seeing a [presence validation exception](https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2542134/?referrer=slack) that the `OwnerId` for a new `Opportunity` can't be blank. That field comes from the Salesforce Account ([code](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/a888a4e0d82447d17986bcdf46d6c15104de59ea/app/services/salesforce/create_opportunity_service.rb#L70)).

### Proposal

After the investigation in [this issue](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2884#note_813196718), we want to:

* Revisit change attempted in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4074, if possible
* Unify the methods used to get a Salesforce account id in `CreateSalesforceAccountService` and `Salesforce::BaseOpportunityService`
* [Create an Account if absent during Opportunity creation](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4793#note_955228314)

### Result
Resolve exception and create Opportunities for these cases again.

### Next steps (if any)

### How will we measure success?

Story points: 5.0
Avoid using customer relationship for subscription methods on the Order model

## Purpose

There are many cases where Orders in CDot production have a `zuora_account_id` value but the Customer of the Order does not have a `zuora_account_id`.  This can be a problem when getting the [`subscription`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/models/order.rb#L117) or [`current_subscription`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/1b7fff46aa9ef908e33311e26999c8fb41d6c873/app/models/order.rb#L137) for an order.  These methods use the `order.customer.zuora_account_id` instead of `order.zuora_account_id`.  The subscription won't be found for those orders when it actually exists in Zuora.  

We should consider updating these subscription methods or if there are any negative consequences.  Some changes were made in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4551, as part of https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3837.  

There may be other changes necessary to make as we are still seeing reports where provisioning fails due to the relationship of the Customer on the Order. As recently as April 28 2022, [this was reported as a problem](https://gitlab.zendesk.com/agent/tickets/286138) (originally raised in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2118#note_928465165).

## Follow-up

The following discussion is from !2985.

Story points: 2.0
Taxes are displayed incorrectly and line items don't add up to order total

<!---
Please read this!

Before opening a new issue, make sure to search for keywords in the issues
filtered by the "bug" label in relevant trackers:

- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
- https://gitlab.com/gitlab-org/customers-gitlab-com/issues?label_name%5B%5D=bug

and verify the issue you're about to submit isn't a duplicate.

Do not include confidential information!
--->

### Summary

When purchasing a product that is taxed, the "Price" line item number incorrectly includes the tax.

### Steps to reproduce

In production, begin a purchase of CI Minutes with your address set to Denver, CO 80223.

### What is the current *bug* behavior?

When purchasing a product that is taxed, the "Price" line item number incorrectly includes the tax.

Note that the order total is correct and the customer is not getting overcharged. This is just an issue with how the numbers are displaying.

### What is the expected *correct* behavior?

The "Price" line item should only be the price of the product itself without any taxes. The taxes should only appear in the "Estimated taxes" line item. In the example below, "Price" should be $40.

### Relevant logs and/or screenshots

![Screen_Shot_2021-03-24_at_4.23.18_PM](/uploads/a45f9b5b3f763c017ee5d971904de4ba/Screen_Shot_2021-03-24_at_4.23.18_PM.png)


<!---
Please add both severity and priority level according to: https://docs.gitlab.com/ee/development/contributing/issue_workflow.html#severity-labels
--->

Story points: 1.0
Update webpacker to 6 after Rails 6 update

The following discussion from !2890 should be addressed:

- [ ] @asubramanian1 started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2890#note_535311824):  (+1 comment)

    > **question:** Should we update `webpacker` to  `5.0` based on http://railsdiff.org/6.0.3.5/6.1.3?

Webpacker 6 will be released soon so we could try to update to that version instead of 5.

Story points: 1.0
Graduate feature flag for `create_opportunity_objects`

### Problem
The feature flag for [create_opportunity_objects](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/42/edit) was introduced with https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1379 to add additional object creations along with an Salesforce Opportunity. The development issue (https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1379) was closed in August 2020 but the parent issue (https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/911) from the Sales team is still open.

### Proposal
- Check with the Sales team when the feature flag can be removed
- Remove the feature flag from the code

### Result
- Cleaned up code that was needed for the feature flag
- One less feature flag to maintain.

### Next steps (if any)
Check with Sales team if the feature flag can be removed.

Story points: 1.0
Reduce repetition of similar parts in subs components

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

At the moment the New Subscription Components for GitLab and EE are sharing quite a good amount of logic, therefore demanding every update to be ported in many places. Increasing the risk of inconsistencies and the time for maintenance.

### Proposal

Consider:

- `app/frontend/javascripts/subscription/components/new_gitlab_subscription_form.vue`
- `app/frontend/javascripts/subscription/components/new_ee_subscription_form.vue`

We have 2 viable strategies (not necessarily mutually excluding):
1. individuate the parts that can be easily extracted into new/shared components and move them there OR
1. generalize the component and add props customize the parts that are different
1. moving the duplicate values into constants and referencing those instead

Finally, update the tests where applicable.

### Result

Reduction of code and decreased maintenance time in the future.

### How will we measure success?

Removed code should be more than added one (excluding additional tests)

Story points: 2.0
Allow renewals of subscriptions for less quantity than the previous term

This issue is the remaining work on https://gitlab.com/gitlab-org/growth/product/-/issues/1577

## Summary

When a subscription renews, and the active seat count is less than the original purchased amount, it tries to renew at the active (lower) seat count than the previous term, which produces an error and subscription doesn't renew.  This applies to both Self-Managed and SaaS.

## Steps to reproduce

1. Create a SaaS subscription which will expire next date for 3 users
1. Set the subscription to autorenew (utilizing custom_autorenew setting in Zuora or by selecting from frontend Customers Portal)
1. Associate the subscription with a GitLab group
1. Ensure the GitLab group has 2 or less users
1. When the subscription attempts to renew, it will error

![Screen_Shot_2021-03-12_at_10.02.56_a.m.](/uploads/70d0bf8a01c0a8f6454dfe81bf0052e4/Screen_Shot_2021-03-12_at_10.02.56_a.m..png)

Story points: 5.0