giseldo/neodataset huggingface dataset:
Keeping amendment logic in sync for Orders API

The following discussion from !5458 should be addressed:

- [ ] @tvellishetty started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5458#note_1107066330):  (+1 comment)

    > thought (non-blocking): As we're duplicating the existing logic, it could happen that the logic in `AmendSubscriptionService` is updated and we miss out on updating the logic here. What do you think about moving these to a helper class and use them both here and in existing services? If the chances of this happening is low then it might not be required.

Story points: 2.0
Cancel subscription button should not be shown for expired subscriptions

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem
As raised in this [comment](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5390#note_1105664081), `Cancel subscription` button is shown on expired subscriptions.

### Proposal
Add a condition to hide `Cancel subscription` button on expired subscriptions. We could add a similar condition like the one for `Resume subscription` in `eligible_for_resumption?` in `subscription.rb`. Since the condition is common for `Cancel` and `Resume` logic, we could move it to the `recurrent_subscription?` method in `subscription.rb`.

<!-- Consult with `@gitlab-com/business-technology/enterprise-apps/zuora-architects` on the ~"Enterprise Applications" team if Zuora business logic is involved. -->

### Result
`Cancel subscription` button will not be shown on expired subscriptions.

### Next steps (if any)

### How will we measure success?

Story points: 2.0
Update general emails layout

We are moving from old purple header/footer layout to the black one. This issue is to update the general template, check all the emails affected and change it for all of them at the same time.

### Design draft:

| Old layout | New layout |
| ------ | ------ |
| ![layout_old](/uploads/0e726b07802d75e0ce82d468cdee8e10/layout_old.png) | <img src="/uploads/4df1a8413ad827aed8057539b42ba2b9/layout_new.png" width="450px"/> |

Figma: https://www.figma.com/file/RwDEORCySqiDhagP7OJSde/Clean-up-SaaS-Privisioning-Email?node-id=1%3A60


### Links that we use:

| Linked Email Text | URL |
| ------ | ------ |
| header: logo | https://about.gitlab.com/ |
| footer: logo | https://about.gitlab.com/ |
| footer: Support | https://about.gitlab.com/support/ |
| footer: Customers Portal | https://customers.gitlab.com/customers/sign_in |
| footer: Blog | https://about.gitlab.com/blog/ |
| footer: Contact Sales | https://about.gitlab.com/sales/ |

### Technical considerations:

We should make sure that email is responsive on smaller screens

### Test plan

https://gitlab.com/gitlab-org/fulfillment-meta/-/issues/786+

Story points: 2.0
Improvements to custom action mapping

The following discussion from !5439 should be addressed:

- [ ] @vshumilo started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5439#note_1102613144):  (+5 comments)

    > What do you think about mapping to the actual class?
    > 
    > That way if the class name changes or is removed we will immediately get an error here?

- [ ] @tvellishetty started a [discussion]https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5439#note_1103708391)

    > thought (non-blocking): similar to [using constants for amendment types](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3778), we could also use constants order action types

Story points: 1.0
[Spike] Replace Customer references to salesforce_account_id

### Problem

There are places in the application where `salesforce_account_id` is referenced from `Customer` (to be renamed `User`).  Here are a few examples:

- [Salesforce::CreateAccountWorker](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/a9a5f019735cfacfca05992367c369f1bc95ef27/app/workers/salesforce/create_account_worker.rb#L42)
- [Salesforce::FindAccountService](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/43b599e2dc1d52d074d97da549167c2b44a8bf69/app/services/salesforce/find_account_service.rb#L16)
- [Api::CustomersController](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/5c81f897836d36021d982a6c4f5b2645e946f5a6/app/controllers/api/customers_controller.rb#L35)

Note: This is not an exhaustive list.  As we start working on this issue, we could promote this to a sub-epic and create issues for smaller pieces.

### Proposal

Let's review the application code to find all the places where `salesforce_account_id` is referenced from a `Customer`.  When the feature flag (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4794) is enabled, the `Customer`'s `BillingAccountMembership` should be used instead.  In many instances, the logged in Customer would have selected the `BillingAccount` they wish to administer, so this scope should be set and used to fetch `salesforce_account_id` instead of using the `salesforce_account_id` directly on `Customer`.

The legacy code will be removed in a later issue once the feature flag has been vetted and promoted in production.

Story points: 1.0
Ability to add an Order Action type more than once in a transaction

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

[`ParamConstructor`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/tree/main/app/services/zuora/orders/actions/param_constructor/) (`Zuora::Orders::Actions::ParamConstructor`) is a set of middleware services that sanitizes and formulates params required by an order action service (`Zuora::Orders::Actions`).

Currently, `ParamConstructor` returns a single object, assuming that a transaction can contain an Order action type only once. While this is true for most order actions in the existing business driven actions, there are a few exceptions such as:

  - `UpdateProduct`: a single transaction can update [main product](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/d2ccaa0f4eab6ff879d4b00a8e5a9ac77db9a9f3/app/services/amend_subscription_service.rb#L83) as well as [options product](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/d2ccaa0f4eab6ff879d4b00a8e5a9ac77db9a9f3/app/services/amend_subscription_service.rb#L94)

### Proposal

<!-- Consult with `@gitlab-com/business-technology/enterprise-apps/zuora-architects` on the ~"Enterprise Applications" team if Zuora business logic is involved. -->

- Update `params` in `Zuora::Orders::Actions::BaseService` to be an Array of Objects instead of a single Object
- Update `.execute` in `Zuora::Orders::Actions::BaseService` to loop through `params` 
- Individual `Zuora::Orders::Actions::ParamConstructor` returns an array instead of an object

### Result

- Ability to add more than one instance of an OrderAction type in a transaction 
- Signature of `ParamConstructor` is consistent for all order actions

### Next steps (if any)

### How will we measure success?

Story points: 2.0
Replace Customer references to zuora_account_id in Rails Admin and Reconciliation services

### Problem

There are many places in the application where `zuora_account_id` is referenced from `Customer` (to be renamed `User`).  Here are a few examples:

- [Setting zuora_account_id on an Order](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/65d6481e7edac53a5839973c5f7aa6d916a10c75/app/models/order.rb#L308)
- [FindOrCreateCustomerService](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/a370c2b4b1a4fec50a6be2291c7b0ec7c73e0cd7/app/services/find_or_create_customer_service.rb#L23)
- [InvoicesFinder](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/ccc5b4d03aaa3929ab72b3a36fdc7b6eb14d672c/app/finders/invoices_finder.rb#L19)

Note: This is not an exhaustive list.  As we start working on this issue, we could promote this to a sub-epic and create issues for smaller pieces.

### Proposal

Let's review the application code to find all the places where `zuora_account_id` is referenced from a Customer.  When the feature flag (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4794) is enabled, the `Customer`'s `BillingAccountMembership` should be used instead.  In many instances, the logged in `Customer` would have selected the `BillingAccount` they wish to administer, so this scope should be set and used instead of using the `zuora_account_id` directly on `Customer`.

The legacy code will be removed in a later issue once the feature flag has been vetted and promoted in production.

**2022-12-16 UPDATE:** [We've broken this issue](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4807#note_1211450977) into an epic with many issues.  I moved this issue into the [newly created epic](https://gitlab.com/groups/gitlab-org/-/epics/9515) and I've narrowed the focus of this issue to just those items that have been delivered to this point (admin changes and updates to the reconciliation services).

Story points: 5.0
Create Admin tools for Account and AccountMembership

### Problem

Once Account and AccountMembership models are added, Support and other GitLab members will need a way to manage these records via the CustomersDot Admin interface.

### Proposal

Let's sections to the Admin interface (http://customers.gitlab.com/admin) for the new models `Account` and `AccountMembership`.  This should allow admins to:

- View all Accounts
- View AccountMemberships for an Account
- View AccountMemberships for a Customer
- Create an AccountMembership
- Delete an AccountMembership

Other functionality like creating or editing an Account, might be useful, but let's check with the team on this before adding, or it could always be added later.

Story points: 3.0
Add billing_account_id to Order

### Problem

The `Order` model currently has an association with an individual `Customer` (to be renamed `User`) via foreign key `customer_id`.  This relationship is problematic for many reasons and should be associated with a `BillingAccount` instead.

### Proposal

With the new data structure for `BillingAccount` and `BillingAccountMembership`, the `Order` model should be updated to be associated with a `BillingAccount` instead of a `Customer`.  The full migration will need to be done in steps, spread across multiple issues and MRs.  

For this issue, we will focus on getting to compatibility mode where `customer_id` continues to be set, but the `billing_account_id` will also be set.  In this issue, the following should be covered:

- [x] Add `billing_account_id` column - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5750
- [x] Start setting `billing_account_id` in application logic (e.g. when creating/updating a `Order`) - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5770
- [x] Use new `billing_account` relationship instead of `customer` when `account_data_structure` [feature flag](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4867) is enabled - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5874
- [x] Introduce rake task to backfill `billing_account_id` for existing `Order` records - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5967
   - ~~If it's easier, this could be added to the rake task from https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4798.  Or it could be a standalone task.~~

### Result

`billing_account_id` will be populated similar to how `customer_id` is set currently.  `customer_id` will continue to be populated to ensure compatibility and allow us to revert if needed.  When the feature flag is enabled, the new `billing_account_id` will be used instead of `customer_id`.

Story points: 3.0
Add billing_account_id to License

### Problem

The `License` model currently has an association with an individual `Customer` (to be renamed `User`) via foreign key `customer_id`.  This model really should be associated with a `BillingAccount` instead of an individual `Customer`.  Most data that references `Customer` should be pulled directly from the `BillingAccount`, or from the Sold To Contact associated with the `BillingAccount`.

### Proposal

With the new data structure for `BillingAccount` and `BillingAccountMembership`, the `License` model should be updated to be associated with an `BillingAccount` instead of a `Customer`.  The full migration will need to be done in steps, spread across multiple issues and MRs.  

For this issue, we will focus on getting to compatibility mode where `customer_id` continues to be set, but the `billing_account_id` will also be set.  In this issue, the following should be covered:

- [x] Add `billing_account_id` column: https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5732
- [x] Start setting `billing_account_id` in application logic (e.g. when creating a `License`) https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5747
- [x] Use new `billing_account` relationship instead of `customer` when `account_data_structure` [feature flag](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4867) is enabled https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5798
- [x] Introduce rake task to backfill `billing_account_id` for existing `License` records https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5967
   - ~~If it's easier, this could be added to the rake task from https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4798.  Or it could be a standalone task.~~

### Result

`billing_account_id` will be populated similar to how `customer_id` is set currently.  `customer_id` will continue to be populated to ensure compatibility and allow us to revert if needed.  When the feature flag is enabled, the new `billing_account_id` will be used instead of `customer_id`.

Story points: 3.0
Add billing_account_id to CloudActivation

### Problem

The `CloudActivation` model currently has an association with an individual `Customer` (to be renamed `User`) via foreign key `customer_id`.  This relationship is problematic for many reasons and should be associated with a `BillingAccount` instead.

### Proposal

With the new data structure for `BillingAccount` and `BillingAccountMembership`, the `CloudActivation` model should be updated to be associated with an `BillingAccount` instead of a `Customer`.  The full migration will need to be done in steps, spread across multiple issues and MRs.  

For this issue, we will focus on getting to a compatibility mode where `customer_id` continues to be set, but the `billing_account_id` should also be set.  In this issue, the following should be covered:

- [x] Add `billing_account_id` column
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5717
- [x] Start setting `billing_account_id` in application logic (e.g. when creating a `CloudActivation`)
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5738
- [x] Use new `billing_account` relationship instead of `customer` when `account_data_structure` [feature flag](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4867) is enabled
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5754
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5809
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5836
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5858
- [x] Introduce rake task to backfill `billing_account_id` for existing `CloudActivation` records
   - If it's easier, this could be added to the rake task from https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4798.  Or it could be a standalone task.
   - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5890

### Result

`billing_account_id` will be populated similar to how `customer_id` is set currently.  `customer_id` will continue to be populated to ensure compatibility and allow us to revert if needed.  When the feature flag is enabled, the new `billing_account_id` will be used instead of `customer_id`.

Story points: 3.0
Backfill BillingAccounts and BillingAccountMemberships for existing Customers

### Problem

Once `BillingAccount` and `BillingAccountMembership` records are being created (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4795 and https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4796) and kept in sync (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4797), we should be in a position to confidently backfill this data for existing `Customer` (to be renamed `User`) records.

### Proposal

Create a rake task to backfill `BillingAccount` and `BillingAccountMembership` records for existing `Customer` (to be renamed `User`) records.  We should consider an idempotent rake task that can be run multiple times.  We may need to run this several times as we test out the issues for creating `BillingAccount`s and keeping the information in sync.  

This task will need to be run at least once before [the feature flag](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/160/edit), `account_data_structure`, is enabled to ensure we can confidently start using the new data architecture in production.  Steps have been added to the feature flag rollout issue, https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4867+, to run the rake task introduced in this issue in non-production, then production environments.  As part of this issue, we should verify the rake task works by running it in staging and production to test, but we will run it again as part of the feature flag rollout.  Please update [the rollout issue](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4867) with the exact rake task command to run in staging and production.

Question to consider:
- What happens when the `zuora_account_id` and `salesforce_account_id` aren't consistent between `Customer` (to be renamed `User`) records associated to the same `BillingAccount`?  We should probably run a query to see how prevalent this is.  If this does happen, how prevalent is it and can we account for these discrepancies programmatically?

Story points: 3.0
Mechanism to keep BillingAccount data in sync with Zuora

### Problem

With `BillingAccount` records being created and stored locally in CustomersDot, it will be important to make sure the data stored locally is kept in sync with the Single Source of Truth, Zuora.  The main concern at this point might be the `BillingAccount` `name` as we may not be caching much else locally.  We can reference the issue where the `BillingAccount` model was introduced for more information, https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4792.

### Proposal

In this issue, we need to work with @gitlab-com/business-technology/enterprise-apps/zuora-architects to build the mechanism for ensuring changes to Zuora Account (mainly `name` and `salesforce_account_id`) are synced to CustomersDot.  

We've engaged the Finsys team via https://gitlab.com/gitlab-com/business-technology/enterprise-apps/financeops/finance-systems/-/issues/886.  In that issue, the Finsys team will create a Custom Event for Zuora Account which will send a callout notification to CustomersDot.

In this issue, we will be responsible for creating the API endpoint for updating the `BillingAccount` record based on this callout notification.

##### Checklist

- [x] Add new API endpoint to enqueue sync account background job
  - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5995
- [x] Add sync account worker to update BillingAccount with Zuora Account data
  - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5995
- [x] Update documentation in [doc/zuora/zuora_callouts.md](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/1c704fd9e437fd519a4b989884192eece9d080eb/doc/zuora/zuora_callouts.md)
  - https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/6031

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

When we arrive at a solution, we should add tests to ensure that this _mechanism_ indeed passes and receives correct data with Zuora.  This will likely take the form of integration tests, but contract tests would also work.

Story points: 5.0