giseldo/neodataset huggingface dataset:
QTC Systems Current State Field Mapping

## Proposal

Build a field mapping of the current state for the 3 Quote-to-Cash data models (CustomersDot, GitLab and Zuora).

## Spreadsheet

Given the limitations of MermaidJS in GitLab and the sheer size of the combined diagram, we decided to document this in a spreadsheet as a field mapping between the 3 systems.  When it's finalized, we will memorialize it in the handbook.

[QTC System Field Mapping Spreadsheet](https://docs.google.com/spreadsheets/d/1D159Osv6JvQtlHm2Bpekn8ADOWUfgcdszf_pewERBy0/edit#gid=769890791)

Story points: 1.0
Add JSON formatter for Puma logs

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Port the GitLab JSON formatter for Puma log messages to get them into fluentd and ELK.

### Proposal

<!-- Consult with `@gitlab-com/business-technology/enterprise-apps/zuora-architects` on the ~"Enterprise Applications" team if Zuora business logic is involved. -->

### Result

### Next steps (if any)

### How will we measure success?

Story points: 2.0
[SaaS] Send introductory email about user overages after renewal

## Problem to solve

SaaS customers are being surprised and taken off-guard by Quarterly Subscription reconciliations. Because self-service SaaS customers have not historically been charged true-ups, the QSR process and the fact that they have to pay for overages is totally new. See tickets from customers [here](https://gitlab.com/gitlab-org/fulfillment-meta/-/issues/337).

## Intended users

This issue is just for SaaS customers.

* Group Owners of a group who have just renewed (manually or auto-renewed) a subscription
* Only groups enrolled in quarterly subscription reconciliation. See exactly which use cases we're solving for in the [Epic description](https://gitlab.com/groups/gitlab-org/-/epics/7229#saas-customers).

## Proposal

After a subscription has been **renewed (manually or auto-renewed)** for a group in GitLab, send an email to **all** group Owners 1 day after a subscription has been renewed to make them aware of how users are counted and how overages are handled.

We should send an email to all group Owners (not just the group Owner who purchased the subscription), because all group Owners should be aware of the overage behavior.

### Design

[Design here](https://gitlab.com/gitlab-org/gitlab/-/issues/347267/designs/Email_to_group_Owners_1_day_after_purchase_with_info_about_QSR.png)

### Email copy

>>>

Subject line: Managing users in your subscription

Dear [customer],

Thank you for renewing your GitLab.com subscription! As you get started, we want to make sure you understand some important billing information.

Your subscription cost is based on the maximum number of seats you use during the billing period. If you reach the number of seats in your subscription, you can continue to add users, but GitLab bills you for the overage. These bills occur every quarter.

Read more about quarterly reconciliation in [our product documentation](https://docs.gitlab.com/ee/subscriptions/quarterly_reconciliation.html).

If you have any questions about this process, [please reach out](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293). We are happy to assist.

Thank you for your business!

GitLab Billing Team

>>>

## Implementation Plan

| Issue | Issue link |
|-------|------------|
| Create mailers | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3999 |
| MVC identify CustomersDot emails to send to | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4000 |
| Send introductory email 1 day after purchase | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4007 |
| Send introductory email 1 day after renewal | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4008 |
| Query GitLab.com for group owners emails and send | https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4009 |

Story points: 0.0
Fix the IronBank schema export

### Problem

As [described in the README](https://gitlab.com/gitlab-org/customers-gitlab-com#ironbank), we should be able to update the IronBank schema files using the command `IronBank::Schema.export` in a rails console.  However, when working on https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/4279, it was noticed that this command stalls.  

When the schema is being exported via IronBank, [the Zuora `describe` API endpoint is used](https://github.com/zendesk/iron_bank/blob/main/lib/iron_bank/describe/object.rb#L20) for [each object name](https://github.com/zendesk/iron_bank/blob/main/lib/iron_bank/describe/tenant.rb#L46) returned by [the top level describe API request](https://github.com/zendesk/iron_bank/blob/main/lib/iron_bank/describe/tenant.rb#L15).  In development (which uses Zuora API sandbox), this process stalls when the describe endpoint is called for certain object names (like `ApplicationGroup`).

CustomerDot has overridden the `object_names` method [here](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/266480162afe740fa1cbc9049dabd600c537110e/lib/zuora/describe/tenant.rb#L17) (from https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/669) to be pulled from a hard-coded set of object names [in `config/zuora/rest_objects.yml`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/6b8978605499bc74aa3d3cdee8b1afaf5d10858f/config/zuora/rest_objects.yml#L1).  From https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/669, it's not exactly clear why it's necessary to use a hard-coded list of object names instead of pulling the available set from Zuora like IronBank does by default.  Using the default logic, allows the schema export to work again.

It's not clear when this export stopped working, but I'm thinking it might be after the latest update to our Zuora tenant for API Sandbox.

### Proposal

Determine why the hard-coded list of object names is necessary.  If it is, trim the list to those needed for CDot that are still relevant.  It might be necessary to make environment specific `rest_objects.yml` files.

### Result

A working IronBank schema export.

Story points: 5.0
Remove UpgradeSubscriptionTransaction component

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

While working on https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3086 it was identified that the component `upgrade_subscription_transaction.vue` seems not to be used anywhere, but rather all we are only using `upgrade_subscription_app.vue` instead. 

### Proposal

It should be safe to delete this component but let's investigate if it's indeed not used anywhere

### Result

Deletion of `upgrade_subscription_app.vue`

### Next steps (if any)

### How will we measure success?

Story points: 2.0
Set correct Terms of Service on Community program subscriptions

### Problem

Users that redeem a subscription through the community program self checkout will need to have different fields set on their subscription. We should update the purchase logic to handle this.

Reminder, this is only applicable to FREE subscriptions handled via https://gitlab.com/groups/gitlab-org/-/epics/5712

Additional context here:
- https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3017#note_586712903
- https://gitlab.com/groups/gitlab-org/-/epics/5068#note_569095826
- https://gitlab.com/gitlab-com/marketing/strategic-marketing/product-marketing/-/issues/5149#note_568946170

**NOTE: this is only applicable after Cloud Licensing goes live**

### Update fields according to this logic

### For SaaS

- `ContractAutoRenew__c` = No
- `TurnOnAutoRenew__c` = No
- `ContractSeatReconciliation__c` = No
- `TurnOnSeatReconciliation__c` = No
- `TurnOnCloudLicensing__c` = _NULL_
- `ContractOperationalMetrics__c` = _NULL_
- `TurnOnOperationalMetrics__c` = _NULL_

### For SM

- `ContractAutoRenew__c` = No
- `TurnOnAutoRenew__c` = No
- `ContractSeatReconciliation__c` = No
- `TurnOnSeatReconciliation__c` = No
- `TurnOnCloudLicensing__c` = No
- `ContractOperationalMetrics__c` = No
- `TurnOnOperationalMetrics__c` = No

Story points: 3.0
Use `self_managed_instance_activation` to find or create a seat link record

### Problem

As reported in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3163, the `SeatLinkCreateService` should create unique `LicenseSeatLink` records for each `self_managed_instance_activation` but this isn't happening at this time.  [The logic that finds or initializes a `LicenseSeatLink` record](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/d2a3c2aed8894dd660eed62cdcc323d2427205ec/app/services/seat_link_create_service.rb#L46) only uses `report_timestamp|date` and `zuora_subscription_name`.

### Proposal

Update [the `SeatLinkCreateService`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/d2a3c2aed8894dd660eed62cdcc323d2427205ec/app/services/seat_link_create_service.rb#L46) to find or create `LicenseSeatLink` records using `self_managed_instance_activation`, in addition to the existing fields (`report_timestamp|date` and `zuora_subscription_name`).

Story points: 2.0
Follow-up from "Fix documentation link for Zuora Objects"

The following discussion from !3300 should be addressed:

- [ ] @vij started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3300#note_584510863):  (+7 comments)

> Yes, that does the trick :tada: Still weird why it works without the leading `/` in GitLab but not here :shrug_tone1:

> We should create a (follow-up) issue to fix other broken links to our docs within our docs but also to track the work (weight of 1). Looks like there aren't too many (just looked for `doc/` in the `doc` folder returned 9 results). Do you want to take care of them too? It'll be a quick win.

Story points: 1.0
Simplify mergeUrlParams function

### Proposal

We would like to simplify the implementation of `mergeUrlParams` function defined in [`url_utility.js`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/d031f6b64d0a27db2c902cc3b657ba66814e4ac2/app/frontend/javascripts/lib/url_utility.js).

As a part of this issue please consider iterating on detect-unsafe-regex warning that is silenced now for the `const re = /^([^?#]*)(\?[^#]*)?(.*)/;`

Please also update the GitLab `mergeUrlParams` utility https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/lib/utils/url_utility.js#L122

Suggested by @agulina in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3281#note_584027216

Story points: 2.0
Use class `has-tooltip` when creating a bootstrap tooltip

## Follow-up from "Add ability to resend the activation code"

The following discussion from !3261 should be addressed:

- [ ] @pslaughter started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3261#note_583778633):  (+4 comments)

    > **question (ux,non-blocking):** Does the `title` automatically generate a tooltip for this icon?
    > 
    > I'm wanting to make sure that it's obvious to admins what this link does. If it's not obvious, it could be misused and cause a bad experience for our customers :thinking:

This is mainly a follow-up of [this comment](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3261#note_583878232).

## Problem

The tooltip implementation does not look like a bon-a-fide Bootstrap tooltip, but rather, it looks like the native browser dependent implementation which sometimes works with `title`

Ref: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/title

There's some significant downsides to this browser implementation, most importantly that it doesn't do anything on focus (only sometimes on mouse hover).  The `rails_admin` gem comes baked in with some bootstrap scripts, including the tooltip one, but it's only applied to table headers.

Ref: https://github.com/sferik/rails_admin/blob/97ccc28940d65fee53b30c409c49032fbb0885db/app/assets/javascripts/rails_admin/ui.js#L97


## Proposal

In the main GitLab project, there is a class `has-tooltip` which is used to programmatically create a Bootstrap tooltip for a thing:

https://gitlab.com/gitlab-org/gitlab/blob/6e953772a9ab09c897d0f4b38f06b4db528b3924/app/assets/javascripts/main.js#L119

Story points: 2.0
Add documentation for how we propagate dependency updates to production

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem
It's not clear how we propagate dependency updates to production

### Proposal

Add such documentation on project's README

Story points: 1.0
Update references to bundler version according to Gemfile.lock

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Some places don't have the same bundler version as the `Gemfile.lock` identifies.

### Proposal

Update references to bundler version according to Gemfile.lock


#### What we have on `Gemfile.lock`:

```
BUNDLED WITH
   2.2.15
```

Story points: 3.0
Remove the CustomersDot plans page and direct users to the marketing pricing page

## Problem to solve

* The Customers Dot plans page https://customers.gitlab.com/plans is hard to maintain and update the pricing information to match the marketing site
* We are in the process of moving all SaaS purchase flows into GitLab (Subscriptions + addon products). Once we have completed that work, the `customersDot/pricing` page will become redundant for the purpose of addon products.
* The CustomersDot plans page is not best entry point for SM customers to start their purchasing journey. The marketing site does a much better of job of displaying the tiers with additional information. Not to mention on `customersDot/pricing` we link back to the marketing site for more information.

## Proposal

Remove the CustomersDot plans page.

- For a user who is not logged in, the home page of `customers.gitlab.com` should be the login page
- For a user who is logged in, the Manage purchases page will be the home page
     - Create an empty state for the Manage purchases page (The manage purchases page currently redirects to the plans page if there are no purchases, so this behavior will need to change)
     - The `Buy new subscription` button will link to the [pricing page on the marketing website](https://about.gitlab.com/pricing/)

#### Design

- See design section
- [See Figma for specs](https://www.figma.com/file/cAKBLLR5KluMCux2byl68x/Remove-the-CustomersDot-plans-page?node-id=3%3A1503)


### Documentation

We will need to update all references to the CustomersDot plans page in the docs. 

Is there a way to search the docs for all instances of links to `customers.gitlab.com/plans`? I found a couple and listed them below, but I am sure there are more.

- [Handling CI minutes](https://about.gitlab.com/handbook/support/license-and-renewals/workflows/saas/ci_minutes.html)
- [Purchase additional CI minutes](https://docs.gitlab.com/ee/subscriptions/gitlab_com/#purchase-additional-ci-minutes)

### Development plan

| Task | MR |
| ------ | ------ |
| Add `redirect_plans_page` FF and use to update root controller to<br>- remove redirect to /plans page<br>- show sign_in page to not logged in users<br>- show subscription list to logged in users | xxx |
| For the plans controller redirect /plans to root / (behind Feature Toggle!) | xxx |
| Create and use a component for empty state "Manage subscriptions" page (behind Feature Toggle!)<br>Update link on the "Buy new subscription" button | xxx |
| Update docs | xxx |

After the FF is switched on and stable -> cleanup for FF and obsolete code https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3637

Story points: 3.0
Send actual plans only to gitlab.com

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Right now we send too many data and plans to gitlab.com, like `deprecated`, `hide_card` attributes. 

### Proposal

I believe they are redundant and should not be send at all. I propose to do all the logic on the CustomersDot side and send only relevant information otherwise we are spreading our logic across two applications which leads to issues like https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61990

In addition, changing available plans will require changes in **both** repositories and that increases the delivery time of a feature.

### Availability & Testing

<!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.

What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?

Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change

See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->

An integration test should be added with a real or mocked GitLab instance.  When the data is transferred from CustomersDot, validate the appropriate data is sent.

Story points: 3.0
Inline all email styles

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Gmail and probably some other email services too strips everything from head and does not accept remote styles.

![image](/uploads/3d4b494fbba11322ddec3593f93840d7/image.png)

### Proposal

We must inline all styles to make them working in Gmail. In order to do that
we need to go over email templates and get the list of all used classes. Then copy&paste them into `mailer.scss` pack.
Add the `premailer-rails` to inline it into a template.

### How will we measure success?

Story points: 3.0
Follow-up from "Add ability to resend the activation code"

The following discussion from !3261 should be addressed:

- [x] @rdavila started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3261#note_577702467):  (+1 comment)

    > I'm not using the `LicenseNotificationService` since it also try to send an email to the reseller and I don't think we want to do that in this scenario.

Story points: 2.0
Trial plans available options in purchase flow on GitLab.com

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

As part of https://gitlab.com/gitlab-org/gitlab/-/issues/322043 Trials were created as their own plan.

These plans are there so we can discern which users are on a trial vs. not on a trial.

Users **should not** be able to choose these options in the purchase flow. That's what this issue is about :)

![image](/uploads/6ee5ca0422d73a284e6cdd4a5dc8dc2c/image.png)

This can be seen when hitting https://gitlab.com/-/subscriptions/new?plan_id=2c92a0ff76f0d5250176f2f8c86f305a&test=capabilities

Story points: 2.0