giseldo/neodataset huggingface dataset:
Ramps Discovery: Review Ramp subscription data in Zuora to inform provisioning plan

### Problem

The overall goal of this issue is to learn how CustomersDot supports ramps (multi-year subscriptions) once Orders Harmonization is live in Zuora Central Sandbox and the `orders_harmonization` feature is enabled in customers.staging.gitlab.com.  Orders Harmonization will be tested as part of https://gitlab.com/gitlab-org/gitlab/-/issues/343474 so that issue is a blocker for this.

~"Enterprise Applications" will be creating use cases as part of an investigative spike issue, https://gitlab.com/gitlab-com/business-technology/enterprise-apps/intake/-/issues/385#investigative-spike.  

### Proposal

Once [Orders Harmonization](https://gitlab.com/groups/gitlab-org/-/epics/6641) dev work is complete and the feature has been enabled in CustomersDot staging, ~"Enterprise Applications" will create Ramp use cases in Zuora API Sandbox and Zuora Central Sandbox as documented in [this issue](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/intake/-/issues/129).  This is estimated to start 2021-11-15, and completed by 2021-11-19.

The Fulfillment team will observe and document behavior in CustomersDot.  We should be able to start this effort by 2021-11-22.

Here's a list of this we should check:
- Were callouts received and processed as expected for ramp subscriptions?
  - Emails delivered?
  - Offline license generated for 1st year of subscription?
- Are ramp subscriptions displayed correctly in the "Manage Subscriptions" page for customers?
  - Does the download/copy License function as expected?  It should have created the license for the first year of the subscription's multi-year term.
- Is support able to manually create licenses effectively for the 2nd year or later of the multi year subscription?

If anything is found that requires Fulfillment work, we should note it in this issue and notify @teresatison as it could impact the delivery date for Ramps.

### Test Data
Ramps data has been generated in the dev environment (via Zuora API SB 1) for Fulfillment review. Test scenarios are listed out [here](https://docs.google.com/spreadsheets/d/1iCOjbZySCKxtLpjab8pBAP_1w440AyiaS30nAiLSifM/edit#gid=1534882502).

Story points: 5.0
Update usages of /plans page on GitLab

We are migrating from https://customers.gitlab.com/plans to https://about.gitlab.com/pricing/ and need to update links to /plans page that present in GitLab.

| Case | Screenshot | Should lead to |
| ------ | ------ | ------ |
| For free accounts we have a link in the navigation | ![Screenshot_2021-11-05_at_17.09.42](/uploads/aeb608177e528ef59254745390e99ae7/Screenshot_2021-11-05_at_17.09.42.png) | https://about.gitlab.com/pricing/ |
| For feature promotions we have a link in the sidebar | ![Screenshot_2021-11-05_at_17.24.41](/uploads/4a1a66b670e8d291110948e7834e94c9/Screenshot_2021-11-05_at_17.24.41.png) | https://about.gitlab.com/pricing/ |
| For feature promotions if the license has expired we have a link in the sidebar| ![Screenshot_2021-11-05_at_17.28.44](/uploads/165657bc8c825af02f434cafe0c6ec83/Screenshot_2021-11-05_at_17.28.44.png) | https://about.gitlab.com/pricing/ |
| On admin subscription page when no license applied | ![Screenshot_2021-11-05_at_17.33.57](/uploads/b58aa0eb240ffbc51aff3dd02f2150a6/Screenshot_2021-11-05_at_17.33.57.png) | https://about.gitlab.com/pricing/ |

Developers note: To release a feature simultaneously we could use a feature flag to set condition for these links. Lag between two releases is tolerable as we will have /plans redirect to root in place, the flow won't be completely broken - means no 404s.

Story points: 2.0
Follow-up from "Add Feature Flag to removing CDot Plans page"

The following discussion from !3967 should be addressed:

- [ ] @vshumilo started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3967#note_721389363): 

    > **suggestion:** We can create a small follow up ticket to address the remaining case when `:zuora_application_down` is enabled.

Story points: 1.0
Missing :access_token authentication option

Errors::PlatypusAppAuthSidekiq/Platypus::CreateLeadWorker
Missing :access_token authentication option

We get the following errors today on Sentry.
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2916270/events/


### Steps
- [x] Identify the root cause of the error
- [x] Check if the leads got transferred thanks to 3 retries.



### Corrective actions
- [x] Improve the error handling
- [x] 56 leads are stuck at CustomersDot. Build the job to fix it.
- [x] Log the sending issues in the leads table

Story points: 3.0
Licensing Email to Partner

## Summary

Today, licensing emails are sent to the End User: Sold To contact. Given the 3-tier model of these orders, the Partner Contact facilitating the purchase of GitLab needs to receive the licensing email in addition to the End User.  For detailed requirements about licensing email updates, see [this issue](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/132).

## Requirements

Customize the existing license email for e-marketplace orders to send a courtesy cop (cc) the Reseller Contact.

## Scope

- https://gitlab.com/gitlab-com/business-technology/enterprise-apps/intake/-/issues/445

## Proposal

- CustomersDot does have some logic already for resellers.
  - If a subscription is considered to be managed by a reseller (e.g. `invoice_owner_id != account_id`), then the customer account associated with the subscription will not be allowed to login to CustomersDot.
  - When the license email is sent to the customer and the subscription is managed by a reseller, a reseller notification email is sent to the reseller. The email used for the reseller is pulled from the work_email of the subscription invoice_owner contact account.
- The [issue description](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/132#details) states the "contact is different from the Sell To, Invoice Owner, or Bill To contacts and will most likely not be stored in Zuora". Given that, it sounds like we would have to fetch this information from SFDC which would be new work. However, the step to create data in SFDC is after this step, so I'm not sure where we should get this information.
- I'm curious what this means for the existing reseller logic. Would this need to be modified or disabled for E-Disty subscriptions?
- Would the Partner Contact email address be added to the existing email, or would it require a new email with specialized copy? The former would likely be less effort.
- Weight: 3-5 (This one is difficult to say with certainty based on the questions above)

_* proposal and weight extracted from Tyler's analysis [here](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/132#note_713339231)_

Story points: 5.0
[Purchase] SubscribeService: CustomersDot Data

## Summary

After Zuora has received all the necessary information, the SubscribeService API should create the customer account with all related info in CustomersDot. The customer then needs to be provisioned a license(s).  For detailed requirements for what records and data need to be created in Customers Dot, see [this issue](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/134).

## Requirements

Ensure all customer, subscription and license records are created in CustomersDot.

## Proposal

- The [issue description here](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/129#cdot-data-creation) mentions "After Zuora has received all necessary information, the SubscribeService API should create the customer account". I'm wondering if this is a hard requirement for the customer account to be created after. The `SubscribeService` in CDot currently requires a customer object so it would likely be easier to reuse this service if we created the customer before sending the information to Zuora.
- I have some questions about the [list of fields here](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/134#customersdot-data-requirements), but I don't think we have to work through these now as they probably wouldn't impact the LOE significantly.
- Weight: 3

_* proposal and weight extracted from Tyler's analysis [here](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/134#note_713337543)_

Story points: 3.0
[Purchase] SubscribeService: Zuora Data

## Summary

Once compliance has passed, the SubscribeService API should create the customer account and related subscription information in Zuora. For detailed requirements for what records and data need to be created in Zuora, see [this issue](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/127).

## Requirement

Ensure all customer and subscription records are created in Zuora.

## Proposal

- Based on the current requirements in the [issue description](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/127#zuora-requirements), I think Fulfillment engineers would have a difficult task knowing what's expected of the new subscribe API endpoint right now. This might be my confusion, but I thought the purpose of this issue was to define the transaction from CustomersDot into Zuora. However, the description includes information about Quote-related objects, which I thought were SFDC-related, and that this information would be fed into SFDC after successful entry into Zuora (as part of https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/128).
- What would be helpful to assess LOE for this issue might be to understand what exactly needs to be sent in the subscribe API request to Zuora. Or another way to think about it might be to understand how this the Zuora subscribe request differs from the current request from a web direct purchase.
- Just to confirm, the new CustomersDot API will only handling new subscription purchases, correct? Or does it also need to handle subscription updates from Arrow.
- Weight:8 (based on the unknowns so far)

_* proposal and weight extracted from Tyler's analysis [here](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/bsa/business-systems-analysts/-/issues/127#note_713335446)_

Story points: 8.0
Backend: Update orders table to set product_rate_plan_id of new OSS plan (for existing orders)

### Problem to solve

We need to update GL.com database to set GitLab subscription hosted_plan_id of `opensource` after the CustomersDot is able to sync `opensource` for Order having a subscription `Plan::ULTIMATE_SAAS_OSS_1_YEAR`.

### Proposal

- [x] Write a rake task to do the following 
     1. Find the Orders within CustomersDot having product_rate_plan_id = Plan::ULTIMATE_SAAS_OSS_1_YEAR
     1. Sync changes to the gl_namespace_id within order to GL.com to update gitlab_subscriptions as `opensource`
- [x] Get the MR merged into main branch.
- [x] Verify the rake command on staging.  
- [x] Turn on the flag `opensource_plans` for production. 
- [x] Run the rake task and verify that the data is updated correctly

In case there is an unexpected error

1. Turn off the [flag `opensource_plan`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags)
2. Invoke OnCall duty person to run the following on GL.com console. 

```ruby
GitlabSubscription.where(hosted_plan_id: 135).update_all(hosted_plan_id: 101)
```

Story points: 3.0
Sync opensource plan code to GL when order is created

### Description

Zuora issues a callback when a Zuora admin/sales creates a `opensource` subscription for a customer. 

The customer receives an email about the new subscription and steps to assign a namespace to the new subscription. 

The CustomersDot need to [sync `opensource` plan id](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/app/services/gitlab/namespaces/update_plan_info_service.rb#L124) to GL.com. This would notify the GL.com to use the `opensource` plan for the namespace.  

### Proposal

Update method [PlansFinder find](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/61b2dead60696ac0a9bddbbbc69890f217f2ea75/app/models/plans_finder.rb#L53) method to return plan_id as opensource for the OSS product_rate_plan_i

### Result

The new plan synced to GL.com is not `ultimate` but `opensource`

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

The unit test for `find` should be created/updated to accommodate the behavior.

Story points: 3.0
Validate presence of zuora_subscription_id in LicenseSeatLink model

The following discussion from !3949 should be addressed:

- [ ] @ebaque started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3949#note_703493550):  (+1 comment)

    > **Nitpick**: A bit unrelated to this MR: I just noticed that while we have `validates: zuora_subscription_name, presence: true` in `LicenseSeatLink`, we don't have the same for `zuora_subscription_id`. And there's no `null: false` for the related column at the DB level.
    > 
    > Not a big deal, as `LicenseSeatLink.where(zuora_subscription_id: nil).count` returns `0` in prod. But maybe worth having a follow-up MR to add a validation for `zuora_subscription_id`? WDYT?

Story points: 1.0
Instrument Customer Portal New Subscription Purchase flow (Self-Managed)

## Summary

As we begin to improve the measurement of our full e-commerce funnel we need to ensure we have proper instrumentation across all purchase flows so we can measure each step of the funnel. 

* Collection tool: Snowplow
* Release behind the feature flag ([`tracking_new_ee_subscription`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/113/edit))

## Information to track

| Type | Description | Location | Event Category | Event Action | Event Label |
| ------ | ------ | ----- | ----- | ----- | ------ |
| Pageview | Records pageviews for each page in the purchasing flow | */subscriptions/new* |
| Event | User changes active users count | */subscriptions/new* | Webstore | Click | Active users | 
| Event | User adds new credit card | */subscriptions/new* | Webstore | Click | Add new card |
| Event | User clicks "buy subscription" | */subscriptions/new* | Webstore | Click | Purchase subscriptoin |
| Event | User clicks on "activate your license" | */subscriptions/new* | Webstore | Click | Activate your license | 
| Event | User views their subscription | */subscriptions/new* | Webstore | Click | View your subscription | 
| Event | User copies cloud license activation code | */subscriptions/A-* | Webstore | Click | Copy cloud license code |


Any other information we should be tracking as part of the new subscription purchase flow?

### Implementation plan

|||
|-|-|
| Add [`tracking_new_ee_subscription`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/feature_flags/113/edit) Feature Flag | gitlab-org/customers-gitlab-com!4114 |
| Add Tracking helper fro FE tracking |https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4446|
| Add tracking events for Self-Managed New Subscription flow | https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4448  |
| Add `google_analytics_id` and `customer_id` to snowplow events | https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4438  |
| Add backend events at the start and end of the subscription flow | https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4466  |
| Add new customerDot standard schema as default | https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4505 |
| Add a new iglu schema for customerDot | https://gitlab.com/gitlab-org/iglu/-/merge_requests/76 |

## Dashboards

Please note these funnels track first-time (net new) purchases for SaaS and SM:

* [SaaS Purchase Funnel (via /billings page)](https://app.periscopedata.com/app/gitlab:safe-dashboard/985319/SaaS-Purchase-Flow-Funnel)
* [SaaS Purchase Funnel (All views to checkout page)](https://app.periscopedata.com/app/gitlab:safe-dashboard/1022191/Saas-purchase-flow-funnel-(all-views-to-checkout-page))
* [WIP: Self-Managed Purchase Flow Funnel (via Customer Portal)](https://app.periscopedata.com/app/gitlab:safe-dashboard/1019118/WIP:Subscription-purchase-flow-for-Customer-Portal)

Story points: 5.0
Sync 50_000 minutes for OpenSource plans

### Description

A customer when buys a paid subscription via CustomerDot, [CI minutes are synced](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/lib/namespace_params_generator.rb#L50) from CustomersDot to GitLab.com.

As per the new requirements, the CI minutes for new opesource program should be 50_000 as [noted here](https://gitlab.com/groups/gitlab-org/-/epics/6895#note_706068096). 

### Proposal

Update [method `gitlab_pipeline_minutes`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/app/models/plan.rb#L463) to sync 50_000 minutes for an opensource plan. 

### Result

Customer when subscribes to opensource plans have 50_000 CI minutes.

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

Unit tests should be updated to ensure the new minute count is reflected.

We should also consider an integration test to validate the behavior between CustomersDot -> GitLab

Story points: 2.0
Add opensource plan information in Plans API

### Description

On the billings page of a namespace, [plans information](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/controllers/groups/billings_controller.rb#L20) is fetched via API by making call to CustomersDot. 

The information received via API is then used by the billings page to display what an admin can upgrade their plans to. For example - If the namespace is on Premium plan then they can upgrade to Ultimate plan and disable the upgrade option for Premium plan. 

### Proposal

1. Create new OSS plan id `opensource` within the plans model in CustomersDot
1. Modify the plans controller to send opensource plan to GL.com.
1. Create a new feature flag `opensource_plan` within CustomersDot.
1. Display `[OSS] Ultimate SaaS Plan` on the billings page of GitLab.com

Story points: 3.0
Check display of credit card errors

### Problem

When entering an invalid credit card number, the error message is not correctly display (noticed in this [merge request](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3953#note_704042847)):

| System | Screenshot |
| ------ | ------ |
| GitLab | ![Screen_Shot_2021-10-14_at_5.43.52_PM](/uploads/0235d2d4f203a3637b8ef69d1c9fb9b4/Screen_Shot_2021-10-14_at_5.43.52_PM.png) |
| CustomersDot | ![Screen_Shot_2021-10-14_at_6.12.34_PM](/uploads/6611812234f4e82f0330ebcda5d05a45/Screen_Shot_2021-10-14_at_6.12.34_PM.png) |

The GitLab screenshot was taken in a project's `setting` -> `CI/CD` under the `Runners` section, e.g.: http://localhost:3000/foo1/bar1/-/settings/ci_cd#js-runners-settings

For comparison, when entering an invalid expiration date the error message is displayed differently:

![Screen_Shot_2021-10-14_at_5.47.43_PM](/uploads/e59a1eb650ece1a880ff4b6af00d9d19/Screen_Shot_2021-10-14_at_5.47.43_PM.png)

### Proposal

Check if this styling can be changed to display errors better.

### Result

Errors are properly displayed.

Story points: 2.0
Improve performance of `/gitlab_plans` endpoint

### Problem
When investigating Purchase error spend (https://gitlab.com/gitlab-org/gitlab/-/issues/342844), it was observed that the `/gitlab_plans` endpoint is slow to respond. It seems to average locally around 2 seconds with spikes to 5 or more seconds. The latency in this endpoint causes the Billing page in GitLab to frequently take over 5 seconds to load.

### Proposal

Since this endpoint is static and appears to be CPU bound, we should target 500ms or less response time. We should investigate the speed of JSON serialization, etc.

Based on Tyler's assessment [here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3677#note_702863713), with him having [fixed](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3677#note_705168537) one of the use cases, we just have this one remaining to address:

> If `namespace_id` is provided, it will be used to lookup the related order for that namespace and find the associated subscription in Zuora ([code reference](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/2c3cd06d36ec0a1535fca0ee96b7bfee2b36b052/app/controllers/plans_controller.rb#L34)). Later, the products are also fetched from Zuora and this is a slow process (\~1.3s on my local machine). This is necessary to determine which plan the namespace is already using, price for upgrades, etc. We will likely need to focus our efforts on ways to improve this lookup.

### Next steps (if any)


### How will we measure success?
Endpoint responds in 500ms or less

Story points: 3.0
Write docs for /subscription routes

Write yaml files to describe endpoints under /subscriptions

In the best case you should be able to send/receive requests in the already setup docs

#### Implementation plan
| Step | Issue |
| ------ | ------ |
| Setup documentation | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3509 |
| Write docs for /subscription | 👉 you are here |

Story points: 3.0
[UX] User flows for CustomersDot Support Admin Tool

This issue is to track efforts documenting top-level user flows across the CustomersDot Support Admin Tool. The goal is to use the defined JTBD in the parent epic and map user flows across the pages required to achieve the job.

_An example of previous user flow documentation work can be found in the [[UX] Document all flows between GitLab.com <> Customers, License, Zuora](https://gitlab.com/groups/gitlab-org/-/epics/3603) epic_

## :dart: Tasks (WIP)
- [x] Document user flows 
  - [x] Generate an Offline license
  - [x] Resend a license
  - [x] Find customers by email

## :art: Documentation

**[Mural Board](https://app.mural.co/t/gitlab2474/m/gitlab2474/1636995607862/095e896771fc71dad28d7ec5d185537cb5c12f0c?sender=tnoah8818)**

Story points: 3.0