giseldo/neodataset huggingface dataset:
Update billing account on customer salesforce_account_id update

The following discussion from !5660 should be addressed:

- [ ] @vshumilo started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5660#note_1153053368):  (+4 comments)

    > **context:** I will layer in the `salesforce_account_id` in a follow up.
    > 
    > I know there will be some feedback on this one given I think I want to propose to favor Zuora's value given the changes in CDot reflect in Zuora but manual SFDC account merges that are synced to Zuora currently do not sync back to CDot.
    > 
    > On the other hand we might want to create the Account with the exact information that the user has and I want to gather some thoughts around what is the best approach therefore don't want to delay this MR with this:
    > 
    > ```ruby
    > # option 1
    > salesforce_account_id: zuora_account.crm_id || customer.salesforce_account_id
    > 
    > # option 2
    > salesforce_account_id: customer.salesforce_account_id
    > ```
    > 
    > Plus the billing account too needs to be updated every time the `salesforce_account_id` for the customer changes -> I think we might use a separate issue to handle this SFDC value in the `BillingAccount`. Do you think another issue will better separate this update @tyleramos

### Acceptance criteria

- [x] Agree on a strategy to populate the billing account's `salesforce_account_id`. 
- [ ] ~~Update the code so every time the `salesforce_account_id` is updated for the `customer`, its associated billing account is updated as well.~~
- [x] Hide the `salesforce_account_id` from the Customer's Admin show and edit views when the `:account_data_structure` feature flag is enabled: https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/6134

As part of this we want to also question if with the current setup we are willing to trust the `customer.salesforce_account_id` or we want to always fetch it from the zuora account associated to the `customer`. Note we don't have a callback that will sync back to CDot changes on the Zuora account `crm_id` that result from merging SFDC accounts and manually updated the references to the surviving account. [Opional context here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4822#note_1166606063)

Story points: 2.0
A couple of follow-ups from "Set urgency class attribute"

The discussions from !5979 should be addressed. Two points:
- Rename `FeatureCategorizable` as its scope now broadened with the introduction of the `urgency` class attributes.
- The body of `Metrics::RequestCollector#controller` is the same as another method in `Metrics::RequestEndpoints`. Extract that shared logic.

cc @qzhaogitlab

Story points: 1.0
CloudLicenseMailer notifications do not send a voucher

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

The recently created CloudLicense mailers for [`renewal_notification`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/3cbef4c4b31ea7553f33165166e5dcef3f09e7d6/app/mailers/cloud_license_mailer.rb#L19) (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4542) and [`subscription_update_notification`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/3cbef4c4b31ea7553f33165166e5dcef3f09e7d6/app/mailers/cloud_license_mailer.rb#L12) (introduced in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4543) do not implement a voucher mailer, so no notification to the SFDC BCC or the reseller email is sent.

Both mailers send a [BCC](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/3cbef4c4b31ea7553f33165166e5dcef3f09e7d6/app/mailers/cloud_license_mailer.rb#L10) to the revenue team email address (licenses@gitlab.com). 

Since we are trying to improve our Proof of License Delivery mechanisms for resellers, ensuring that Cloud Licensing renewals and subscription updates are available for them seems like a logic next step.

### Proposal

Introduce a new `CloudLicenseVoucherMailer` with two methods (one for renewals and another for subscription updates) and call those methods from the corresponding notification.

### Result

* The vouchers provide better visibility for the Sales and Order Management teams by storing them in SFDC
* The vouchers provide better visibility for resellers by sending them to the `Invoice Owner` of the partner

### Reported Examples
1. https://gitlab.zendesk.com/agent/tickets/351040 - OM team requesting copy of renewal on CL
2.

Story points: 3.0
Document interactions between GitLab User / CustomersDot User / Zuora Contact / SFDC Contact

### Proposal

As mentioned in the [Architecture Review: Fulfillment Platform meeting on 2022-11-16](https://docs.google.com/document/d/1_YbxNCo3KXK1-KdIZTgSvmU9kIslYjQnFZAmmu1DSqQ/edit#heading=h.1nmnr3rsvzrk), we want to document the workflows involving interactions between GitLab Users, CustomersDot Users, Zuora Contacts, and SFDC Contacts.

This is important for the following reasons:
- Help to understand how to keep all these related “User/Contact" objects in sync 
- Help to understand what we should do with the CustomersDot `customers` (to be renamed `users`) table
- Help to understand how to tie the work from https://gitlab.com/groups/gitlab-org/-/epics/8331+ in with https://gitlab.com/groups/gitlab-org/-/epics/8905+

Story points: 1.0
🎄 Suspend deployments during December 2022 holidays

We will suspend all production deployments during the holidays.  Here's a link to the handbook page describing Production Change Locks (PCL) and the timing of the upcoming lock.  Please raise any concerns or comments on this issue.

There will be limited coverage as most people will be away. This should effectively limit the need to have someone on call to resolve any development-related issues in production during the holidays period.

~~We'll need 2 volunteers from the teams to add/remove the label and notify Slack.~~

## Process

- [x] @tyleramos On **2022-12-23 23:00 UTC** add the ~"production::blocker" label to this issue.
- [x] @tyleramos Then, notify #s_fulfillment, #s_fulfillment_engineering, #s_fulfillment_fyi, #support_licensing-subscription, and #production about the freeze
- [x] @cwiesner On **2023-01-03 09:00 UTC** remove the ~"production::blocker" label from this issue.
- [x] @cwiesner Then, notify #s_fulfillment, #s_fulfillment_engineering, #s_fulfillment_fyi, #support_licensing-subscription, and #production about the freeze

Please keep this issue public.

cc @fulfillment-group

Story points: 0.0
Implement Auto renewal payment method changes in Orders API

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/5034 implemented a few changes in the update payment method workflow during auto renewal. This issue is to apply in Orders API workflow as well.

Related MRs:
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5748
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5806

### Proposal

<!-- Consult with `@gitlab-com/business-technology/enterprise-apps/zuora-architects` on the ~"Enterprise Applications" team if Zuora business logic is involved. -->

### Result

### Next steps (if any)

### How will we measure success?

Story points: 2.0
Refactor feature specs for rails admin

### Problem

In https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5886#note_1172520728, it was brought up that having 2 actions with 2 assertions in one example isn't usually best practice. The example in the merge request uses the search on an index page twice, the first assertion is for a result while the second is for no results. This is a pattern that is used throughout other feature specs for rails admin too. 

### Proposal

Evaluate all feature specs for rails admin and split up examples that cover more than one action. Some specs could even use `RSpec::Parameterized::TableSyntax`.

If needed this issue could be promoted to an epic.

### Result

Following best practices for RSpec.

Story points: 3.0
Use presenter for community renewals

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem
As suggested in this [comment](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4372#note_865226482), it would be a good idea to move the logic block in `subscriptions/community_program/renew_subscription.html.haml` into a helper. With this we would get the additional advantage of being tested. The same helper could be used for `subscriptions/community_program/renew_subscription`(https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/602a40bbc66d0c353787fea9f56b9f8b6efea4e3/app/views/subscriptions/community_program/renew_subscription.html.haml#L1) too as it has similar logic.
 
### Proposal
Create a presenter with logic that can be shared with the `renew_subscription` file.

Story points: 2.0
Investigate high quantity for existing_renewal_error (auto-renew) SaaS

### Problem

While reviewing [Subscription Management Performance dashboard](https://app.periscopedata.com/app/gitlab:safe-dashboard/1092507/WIP:-Fulfillment:-Subscription-Management-Performance-dashboard), I noticed that [Number of Auto-renew errors by Product type](https://app.periscopedata.com/app/gitlab:safe-dashboard/1092507/WIP:-Fulfillment:-Subscription-Management-Performance-dashboard?widget=15776195&udv=1803088) and [% split of  SaaS AutoRenew errors by error type](https://app.periscopedata.com/app/gitlab:safe-dashboard/1092507/WIP:-Fulfillment:-Subscription-Management-Performance-dashboard?widget=15776187&udv=1803088) look off.

![CleanShot_2022-11-22_at_16.22.55](/uploads/483be4a44888217b7648cbee8348b98b/CleanShot_2022-11-22_at_16.22.55.png)

![CleanShot_2022-11-22_at_16.23.28](/uploads/f83934192cb0823943f8aad7286ba0ba/CleanShot_2022-11-22_at_16.23.28.png)

We only had this many SaaS and SM subscriptions up for auto-renewal:

| Aug-2022 | Sep-2022 | Oct-2022 |
|----------|----------|----------|
| 585      | 755      | 881      |


It seems that `existing_renewal_error` is reporting way too many times. 

### Proposal

Investigate whether our tracking is working as expected.

### Result

### Next steps (if any)

### How will we measure success?

Story points: 1.0
Fulfillment Platform PI instrumentalization: Developer Productivity

## Overview

We want to measure `MTTR` (mean time to recovery) as one of the ~"group::fulfillment platform" performance indicators. This will help us to capture the state and progress of the developer productivity within the fulfillment infrastructure we provide.

## Details

The proposal and progress of the PI is captured in the [Fulfillment PI internal handbook section](https://internal-handbook.gitlab.io/handbook/company/performance-indicators/product/fulfillment-section/#fulfillment-platform---mean-time-to-recovery-mttr).

**Definition of MTTR: Measure time (in minutes) between interruption of service(s) due to deployment/system failure and full recovery**

After a [first feedback loop](https://gitlab.com/gitlab-org/fulfillment-meta/-/issues/876#note_1165028722), and a sync discussion we identified a way how to instrument this metric:

We capture events whenever CustomersDot enters and leaves maintenance mode. This is visible when the `HealthCheck::Zuora` and `HealthCheck::Gitlab` jobs are triggered and when they catch changes of availability of these third-party services.

Additionally, triggering the maintenance mode manually (through the `customers:maintenance_mode` rake task) should also capture such events for the creation of this metric.


## Outcome

We can measure and see MTTR as a first iteration in the ~"group::fulfillment platform" Grafana Dashboard.


## High-level implementation

Below is an attempt to come-up with a plan to achieve the outcome highlighted above. This is likely to be refined before we start work.

This plan also takes into consideration requirements outlined in [this other issue](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/5579), as there are overlaps between both issues:

1. Update our Prometheus collector
   - Our metrics collector is currently suited for Sidekiq jobs and app/api controllers. We'll have to update it so that it can also capture data when entering / leaving maintenance mode (and when raising Zuora error, wrt to this other issue)
2. Implement new Prometheus collection 
   - Apply actual collection of data in related Health cron jobs and the maintenance rake task.
3. Update [the Runbooks project](https://gitlab.com/gitlab-com/runbooks)
   - We need to do that so that we can bridge data from Prometheus to new graphs in Grafana

### Issue breakdown

The plan above led to the creation of the following issue breakdown:

1. [ ] [Allow Prometheus collection of maintenance events](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/5718)
2. [ ] [Add Grafana dashboard to show new MTTR indicator](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/5725)

Story points: 8.0
Rename ZuoraAccountMixin#sold_to_contact (Follow-up from "License model - use of account_data_structure feature flag")

The following discussion from !5798 should be addressed:

- [ ] @tyleramos started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5798#note_1171316453):  (+1 comment)

    > **Question:** With this MR, we have a new abstraction called `BillingAccountContact` and this method called `contact`, in addition to the existing method `sold_to_contact` in `ZuoraAccountMixin` [here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/b834e92525d563dde9bd1fbb7175a11c4c5907a8/app/models/concerns/zuora_account_mixin.rb#L80).  I wonder if this will be a little confusing knowing what kind of object you get from these methods.  Perhaps, the `sold_to_contact` could be clarified as `sold_to_zuora_contact`?  That could be done as a follow-up to this to keep this change smaller.
    > 
    > Thoughts?

Story points: 1.0
Consolidate the openAPI doc into a single file

Due to a security fix, the openAPI doc ref links are not supported by Gitlab. This might be fixed, but we do not know when. The tracking issue is gitlab-org/gitlab#381047+.

This impact [CDot openAPI doc](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/main/doc/api/openapi.yaml) rendering.

Since we do not have lots of endpoints in the api doc, I suggest we consolidate it into one single file.

This seems to be align with the new version of the [Optic cli tool](https://github.com/opticdev/optic) as well. ATM we are using the previous Optic version's `api` command, which seems to be deprecated by the latest Optic version.


### Problem

### Proposal

<!-- Consult with `@gitlab-com/business-technology/enterprise-apps/zuora-architects` on the ~"Enterprise Applications" team if Zuora business logic is involved. -->

### Result

### Next steps (if any)

### How will we measure success?

Story points: 2.0
[Spike] Investigate and break down work for Customer-Dot ruby version >=3.0

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

In order to plan the upgrade to Ruby version >=3.0 for `customers-gitlab-com` , we want to understand the work that needs to be done to do so. Therefore, we would like to get that understanding during this spike.

### Proposal

Let's break down the work in this spike, and add issues to https://gitlab.com/groups/gitlab-org/-/epics/5149.

Crafting a list of all blockers for the Ruby 3.0 upgrade will help a lot. This list could consist of changes in our code, gems we have to update or drop (or replace), etc.

We need to decide if we upgrade to Ruby version 3.0 and afterwards to a higher version or directly to a higher version.

### Result

As a result/outcome we will have:

- Decide for a ruby version to upgrade (3.0 or higher?)
- List for all blockers/dependencies for the upgrade
- Break down the work to upgrade the ruby version for `customers-gitlab-com` and add issues to https://gitlab.com/groups/gitlab-org/-/epics/5149

Story points: 2.0
[Spike] Investigate Upgrade to Rails 7.0

### Problem

In order to [Upgrade rails to version 7.0](https://gitlab.com/groups/gitlab-org/-/epics/9235), we need to identify the work, places and steps where we need to take action to facilitate a smooth upgrade.

### Proposal

The outcome of this spike issue should be a plan, broken down in issues attached to the [epic](https://gitlab.com/groups/gitlab-org/-/epics/9235) that informs the person(s) handling the upgrade what steps need to be done and we can schedule the work in to corresponding milestone(s).

Story points: 2.0
Dos via lack of input validation ( customers.gitlab.com )

**[HackerOne report #1765576](https://hackerone.com/reports/1765576)** by `mrhacker404` on 2022-11-08:

[Report](#report) | [Attachments](#attachments) | [How To Reproduce](#how-to-reproduce)

## Report

#####  Summary

(there is an lack of input validation via email parameter. giving high lenghts will prompt 500 Internal error. making fast requests to the internal server error will make the server down for everyone ! )

#####  Steps to reproduce

(Step-by-step guide to reproduce the issue, including:)

1. Register like any new user. use random details.  
1. The next page should be ( /email_confirmations )   
1. So now upload my payload from the attachement and use it on the email input.   
1. Make sure to send the request in repeater and you receive 500 internal error.  
1. Run turbo intruder or intruder. ( i used turbo-intruder ) good threads then start the attack. 

#####  Impact

(Summarize the impact on users)

No one will be able to login or navigate into Gitlab Customers . the server has been highly affected with only an test attack ! server has been down for long time ! and i believe the same issue will happen on gitlab.com ( official server ) i tried just to get the 500 internal error on the main website and yeah i got it !

####  Impact

A Denial-of-Service (DoS) attack is an attack meant to shut down a machine or network, making it inaccessible to its intended users. DoS attacks accomplish this by flooding the target with traffic, or sending it information that triggers a crash.

## Attachments

**Warning:** Attachments received through HackerOne, please exercise caution!

* [publicdown2.png](https://h1.sec.gitlab.net/a/30c9b9c4-4ea1-4f24-afbe-d8245063798f/publicdown2.png)
* [publiccheck1.png](https://h1.sec.gitlab.net/a/560bd0a6-d78b-48d5-8edc-48664cb78f02/publiccheck1.png)
* [DOWN.png](https://h1.sec.gitlab.net/a/fdb5a210-a03f-4fce-b48b-994ca130209d/DOWN.png)
* [500forall.png](https://h1.sec.gitlab.net/a/1d4cece0-ddf0-4d96-b59b-0673cf36f26e/500forall.png)
* [payload.txt](https://h1.sec.gitlab.net/a/21d12a27-9c3d-4f54-888f-4d142f06c8f6/payload.txt)
* [17min-idk.png](https://h1.sec.gitlab.net/a/ee1c1301-a8a4-4f86-866d-fdc72419654a/17min-idk.png)
* [2022-11-08_04-13-47.mkv](https://h1.sec.gitlab.net/a/535a4ab8-452a-4d1a-af07-5e551b0894a6/2022-11-08_04-13-47.mkv)

## How To Reproduce

Please add [reproducibility information] to this section:

1.
1.
1.

[reproducibility information]: https://about.gitlab.com/handbook/engineering/security/#reproducibility-on-security-issues

Story points: 1.0
Remove breadcrumbs

### Problem

Long time ago we introduced breadcrumbs for CustomersDot but the application has changed a lot since then and I suggest we remove breadcrumbs from the application because of following reasons:

1. We don't have more than 2 level of nesting pages anywhere in the application. That means breadcrumbs always render two levels: `Manage purchases -> New Subscription`
2. We use them only on `Manage purchases` pages.
3. And they do not match our Pajamas Design https://design.gitlab.com/components/breadcrumb/ :smile: 

![image](/uploads/a64ec7590f1c4f6d94e6bf46a1819b3b/image.png)

### Proposal

Let's just remove them?

### Result

### Next steps (if any)

### How will we measure success?

Story points: 1.0
Port GL code for urgency setting for Rails controllers

### Problem

CDot lacks ability to set `urgency` on controller. `urgency` is available in [gitlab repo](https://docs.gitlab.com/ee/development/application_slis/rails_request_apdex.html#how-to-adjust-the-urgency) to allow devs to specify a fairer expectation of acceptable request duration. This means our APDEX scores are less useful, treating low and hight priorities as the same.

Currently on CDot, I think [any request taking more than 0.4 seconds](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/e7dd35ba9aec7a01e4dc89dd4b15c0ed61dc826b/lib/metrics/request_collector.rb#L31-L31) are counted as failure.

This will help https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4884

We ported [feature categorization earlier](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5287), and this will be build on top of that.

### Proposal

We need to port [GitLab's `urgency` setting over](https://gitlab.com/gitlab-org/gitlab/blob/61750d46bc057c63b072f34b885679091ba1a35c/lib/gitlab/endpoint_attributes.rb#L20), so we can set different `urgency` on different controllers.

### Result

More useful APDEX score

### Next steps (if any)

### How will we measure success?

Story points: 2.0
Update billing account when setting customer zuora_account_id

### Problem

As part of https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4795 the `UpdateBillingAccountService` was introduced to find or create a billing account for the customer based on its `zuora_account_id`.

Basically every time the `zuora_account_id` is set for the customer we want to also call the `UpdateBillingAccountService` to keep the customer's associated billing account up to date. This includes when the `zuora_account_id` is set to `nil`.

Reference MRs that do this:
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5810+
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5808+
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5811+

### Proposal

Update the code within `qa` / `quality` that sets customer's `zuora_account_id` to also update the billing account in a similar way.

Some places that do this (might not be the entire set):
* `PostZuoraRefreshTask`
* `OrderApiClient`
* `QA::Resource::Subscription`

Story points: 2.0