giseldo/neodataset huggingface dataset:
Exclude unnecessary fields from IronBank Schema files

### Problem

In https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2825, we found out that it can be problematic when custom fields are renamed or removed from Zuora, if those fields are listed in the IronBank Schema ([sandbox files](https://gitlab.com/gitlab-org/customers-gitlab-com/-/tree/staging/config/zuora/sandbox/schema), [prod files](https://gitlab.com/gitlab-org/customers-gitlab-com/-/tree/staging/config/zuora/production/schema)).  Since the schema files are used when IronBank builds queries to pull data from Zuora resources, changes to the fields in Zuora without being reflected in the Schema results in `IronBank::BadRequestError` errors.

### Proposal

To help avoid these types of errors, we probably should exclude any fields in queries which are not needed directly by CustomersDot.  This could be done by adding these fields to [`excluded_files.yml`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/6b8978605499bc74aa3d3cdee8b1afaf5d10858f/config/zuora/sandbox/excluded_fields.yml).  This should prevent these fields from being included in the Zuora queries ([ref from IronBank](https://github.com/zendesk/iron_bank/blob/main/lib/iron_bank/metadata.rb#L31)).  If that approach doesn't work, we could simply remove the fields from the schema altogether.

This might be the best course of action:
- Update all the schema files to get the lastest by exporting: `IronBank::Schema.export`.
- Audit each field for each Resource and add any fields not used by CustomersDot to the `excluded_files.yml`
- This will likely cause specs to fail because the Zuora requests are different. VCR cassettes will need to be re-recorded.

The above could be done for Sandbox first, then later for Production.

Story points: 5.0
PlansFinder should search for plans in Zuora if not found within eager loaded plans

### Problem

If a product rate plan is not in the [list of preloaded plan ids](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/aa13252af0a6e69a58c8ac2bc645afaf36d7b12c/app/models/plans_finder.rb#L11) in `PlansFinder` and this plan id is used in [`PlansFinder.find`](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/aa13252af0a6e69a58c8ac2bc645afaf36d7b12c/app/models/plans_finder.rb#L49), the result is `nil`, even if the product rate plan exists in Zuora.  This can lead to errors when a subscription is created or updated which uses a product rate plan not listed in the PlansFinder.  The results in the Zuora worker failing to process the callout which means no Order record is created for the subscription.

I noticed this when testing an MR where we were adding a new SKU to CustomersDot.  See [this discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2925#note_526657938) for more info.  This happens quite often as can be seen by these Sentry errors:

- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2446974
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2458737
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2465402
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2474979
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2069046
- https://sentry.gitlab.net/gitlab/customersgitlabcom/issues/2374919

### Proposal

One way to fix this issue would be to update [the `PlansFinder.find` method](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/aa13252af0a6e69a58c8ac2bc645afaf36d7b12c/app/models/plans_finder.rb#L49) to fallback to searching Zuora if the product rate plan isn't found within the seat of `eager_loaded_plans`.  Since the product catalog is cached locally anyway, this wouldn't be a very expensive operation as it would have been when this eager loading was originally introduced.

In fact, this eager loading is probably not necessary at all, and we have [an issue to investigate removing it](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2680).  While that would be a better longer term solution, this could be a nice, quick short term solution for this problem. If https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2680 get's prioritized first, we can close this issue.

### Result

No more errors like the ones shown above.  We could resolve those in Sentry when this is finished.

Story points: 2.0
Replace local danger files with gitlab-dangerfiles gem

### Problem

GitLab's Dangerfiles has been extracted to its own gem at https://gitlab.com/gitlab-org/ruby/gems/gitlab-dangerfiles.

### Proposal

Evaluate and implement (if applicable) removing our local dangerfiles in favor of using the official gem file while keeping our needed customization.

### Result

Up-to-date dangerfiles and less code to maintain.

### Next steps (if any)

Schedule

### How will we measure success?

Story points: 2.0
Resolve IronBank::BadRequestError errors from change in sandbox schema

### Problem

We're [seeing a bunch of errors](https://sentry.gitlab.net/gitlab/customersstggitlabcom/?query=is%3Aunresolved+end_customer_details__c) like `IronBank::BadRequestError` related to a malformed query like [this one](https://sentry.gitlab.net/gitlab/customersstggitlabcom/issues/2527191/?environment=stg&referrer=alert_email):

```
Body: {"detail"=>{"MalformedQueryFault"=>{"FaultMessage"=>"invalid field for query: Subscription.end_customer_details__c", "FaultCode"=>"INVALID_FIELD"}}, "faultcode"=>"fns:INVALID_FIELD", "faultstring"=>"invalid field for query: Subscription.end_customer_details__c"}
```

The majority of the `IronBank::BadRequestError` errors we've seen are related to above, but there is [one other error](https://sentry.gitlab.net/gitlab/customersstggitlabcom/issues/2527106/?query=is%3Aunresolved%20invalid%20field%20for%20query) we've seen related to another field `Invoice.currencyexchangerate__c`:

```
IronBank::BadRequestError: Body: {"detail"=>{"MalformedQueryFault"=>{"FaultMessage"=>"invalid field for query: Invoice.currencyexchangerate__c", "FaultCode"=>"INVALID_FIELD"}}, "faultcode"=>"fns:INVALID_FIELD", "faultstring"=>"invalid field for query: Invoice.currencyexchangerate__c"}
```

### Proposal

Fix the schema for Zuora Sandbox.  It looks like the custom field was renamed in Zuora Sandbox but we have to match that in our schema.  If we don't use that field in our codebase, then the better fix might be to remove it from our schema in case it changes again.

### Result

This should fix many errors we're seeing in stg at the moment.

### Next steps (if any)

Long term it would be good to notify the Fulfillment team before  custom field changes are made in Zuora production or sandbox.

Story points: 2.0
EoA: Subscription did not renew with the new product name

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem

Customer renewed their Silver subscription, but the name of the subscription did not change to the new product name. 

### Proposal

All renewals after Jan 26 should renew with the new product names.

### Reported examples

1. https://gitlab.zendesk.com/agent/tickets/198984
1. https://gitlab.zendesk.com/agent/tickets/200391 (note: I had to do a [force associate](https://gitlab.com/gitlab-com/support/internal-requests/-/issues/5182) for this one)

### Once completed...

- [x] @tgolubeva to follow up in https://gitlab.com/gitlab-com/business-ops/enterprise-apps/intake/-/issues/160#note_530913862

Story points: 3.0
Assign appropriate Zuora batch for self-service Community subscriptions

## Summary

In https://gitlab.com/groups/gitlab-org/-/epics/5068 we are introducing the ability for community program participants to self checkout. Under the current manual bookings process, the billing team is manually assigning community subscriptions to `Batch50` in Zuora. Since we are automating the booking process, we need to automate this batch assignment as well.


## Proposal

As per [this comment](https://gitlab.com/gitlab-com/marketing/community-relations/community-operations/community-operations/-/issues/63#note_527588922) from our billing manager, we need to make sure that subscription created during this process as assigned to `Batch50` in Zuora.

Story points: 2.0
Follow-up from "Remove gitlab sync from subscriptions#update"

The following discussion from !2886 should be addressed:

- [ ] @cwiesner started a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2886#note_527348783):  (+2 comments)

    > **suggestion (non-blocking):** A nice follow-up could be using a named subject throughout this file. Although I keep seeing the use of `subject` within controller specs in GitLab as well. Am I maybe missing something and we're not supposed to use in in them? :thinking: :shrug_tone1:

Story points: 1.0
Follow-up from "Implement additional logging when expiring orders"

<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. 

For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks.

https://www.learning-styles-online.com/style/visual-spatial/

-->

### Problem
In !2915 a [discussion](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2915#note_526425953) (+2 comments) about why there is a `#to_param` and `#to_i` being called on a parameter ([here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/4809f88aef0442d6f5e79f6b0b407167ba14d541/app/jobs/expire_order_job.rb#L9)) that is only passed in as an Integer ([here](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/4809f88aef0442d6f5e79f6b0b407167ba14d541/app/jobs/expire_order_cron_job.rb#L18)). It turned out that this was recently changed in https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2303#note_467690097 to meet Sidekiq's best practices. Any remaining jobs during the deploy had to be supported then and therefore this was still needed.

### Proposal
Since there is only one usage where an Integer is being passed in and all jobs the `ExpireOrderJob` class are processed (no retires left as well), we can remove the `#to_param` and `#to_i`. The `#to_i` is also not needed since `.find_by_id` handles this anyway:

```
[2] pry(main)> Order.find_by_id(1)
Order Load (0.4ms)  SELECT  "orders".* FROM "orders" WHERE "orders"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
[3] pry(main)> Order.find_by_id(nil)
Order Load (0.3ms)  SELECT  "orders".* FROM "orders" WHERE "orders"."id" IS NULL LIMIT $1  [["LIMIT", 1]]
[4] pry(main)> Order.find_by_id("")
Order Load (0.4ms)  SELECT  "orders".* FROM "orders" WHERE "orders"."id" = $1 LIMIT $2  [["id", nil], ["LIMIT", 1]]
[5] pry(main)> Order.find_by_id("a-string")
Order Load (0.4ms)  SELECT  "orders".* FROM "orders" WHERE "orders"."id" = $1 LIMIT $2  [["id", 0], ["LIMIT", 1]]
[6] pry(main)> Order.find_by_id("1")
Order Load (0.4ms)  SELECT  "orders".* FROM "orders" WHERE "orders"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
```

### Result
Remove unnecessary method calls and therefore clean up the code and avoid possible confusion.

Story points: 1.0
Cloud License: Skip user limit validation when saving cloud licenses

### Problem

As mentioned in https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2801, between the time a customer purchases a new SM subscription and the customer activates the license in the SM instance, there is potential for the number of active users in the SM instance to increase.

Currently, when (legacy) licenses are applied to new SM instances (e.g. instances without historical user data), if the instance tries to apply a license for `X` number of users, but the instance has more than `X * 1.1` (where the user overage exceeds 10%), then the license can’t be applied. 10% is the allowed user overage for new/non-renewal subscriptions (defined [here](https://gitlab.com/gitlab-org/gitlab/blob/25536b04c0c42e250a9d4319eeff7b1d892d47a5/ee/app/models/license.rb#L616)).

This will be a problem for cloud license activation. Let's say a cloud license is purchased for 10 users, but when the customer gets around to activating the license with the code they received in an email, the number active users on their instance is 15. This would exceed the 10% overage allowance and the license would [fail validation](https://gitlab.com/gitlab-org/gitlab/blob/25536b04c0c42e250a9d4319eeff7b1d892d47a5/ee/app/models/license.rb#L619).

### Proposal

The simplest solution to this problem is to skip the user limit validation for cloud licenses and save them regardless of whether the instance is over the user overage allowance for the license. The customer would pay for the increased users at the quarterly reconciliation.  

This also solves the problem for on-going license updates along the lifespan of the cloud license.  Because we will be applying new licenses records during the license sync workflow (even after activation), this will allow the license to be applied even if the license count is lower than the current active user count.

Story points: 2.0
Drop :sync_plan_to_gitlab_from_controller feature flag

### SUMMARY

We want to remove GitLab plan syncs from the subscriptions `create` and `update` controller actions.

This removal will be behind a feature flag that will be off by default:

```ruby
if Unleash.enabled?(:sync_plan_to_gitlab_from_controller)
  # Code we want to remove that handles syncing the plan to gitlab
  Gitlab::Namespaces::UpdatePlanInfoService.new(order, force_sync: true).execute_async
end
```

This feature flag will be disabled by default and only there as a safety net in case we want to "recover" the code.

It is our expectations that we never have to turn on this flag and that given a reasonable amount of time (TBD) we consider it is safe to entirely remove the code and the flag. This issue is to track this.

Story points: 1.0
User can't reenter new card information if card has insufficient funds

### Summary

When the entered card has insufficient funds, the user can't reenter another card information without refreshing the page.

### Steps to reproduce

1. Go to a purchase page (e.g. buy more CI minutes)
1. Enter a card that will produce insufficient funds error (e.g. `4000000000009995`)
1. Notice the error is shown at the top of the page
1. Notice adding a new card iframe isn't refreshed to allow the user to enter a new card


**Note:** the error is not being sent as other errors like "enter a card with a future date".

### What is the current *bug* behavior?

New a card iframe doesn't refresh and stays on

### What is the expected *correct* behavior?

The new card iframe should be refreshed allowing the user to enter a new card.

### Relevant logs and/or screenshots

![Screen_Shot_2021-03-09_at_17.19.55](/uploads/bc7f784279d895b137334e45b9f3b11d/Screen_Shot_2021-03-09_at_17.19.55.png)

### Workarounds

Refresh the page

Story points: 2.0
Allow to retry submitting in upgrade/renew page when there's a payment error

### Summary

On the upgrade page, if the user entered an invalid/expired card the submit button will remain disabled preventing the user from updating the card details and submitting again.

### Steps to reproduce

1. Go to subscription upgrade/renew page
2. Choose add a new card
3. Enter card details with an expired date (e.g. 01/01/2021)
4. Hit submit
5. Notice the error at the top of the page
6. Notice the submit button is still disabled, the user can't adjust the card details and need to refresh the page to proceed with the upgrade.

### What is the current *bug* behavior?

After submitting an invalid credit card the submit button is disabled, not allowing the user to retry.

### What is the expected *correct* behavior?

When there's an error in the payment methods, the submit button should be enabled to allow retrying.

### Relevant logs and/or screenshots

![Screen_Shot_2021-03-09_at_16.37.58](/uploads/c519994c4ba750d273586a51fa86e156/Screen_Shot_2021-03-09_at_16.37.58.png)

### Workarounds

Refreshing the page.

# TODO

- Remove the [`!this.errorMessage`](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/2591/diffs#995e8151c13e03c9af3ab9e2d2cbe613b2d96ee0_126_126) block that prevents the submit button from being enabled in Upgrade and Renew

Story points: 1.0
Order total and price not correctly updated after quantity entered

### Summary

There's a [race condition](https://en.wikipedia.org/wiki/Race_condition) occurring in purchase pages (e.g. CI minutes and storage), the problem is happening mostly on our e2e tests but it could happen to our users.

### Proposal

In purchase pages, debounce HTTP requests that are triggered by user input (e.g. change seats count).

Introducing the debounce might increase our bundle size a bit (or a few bytes) but it will prevent the race condition from happening, which help with:

1. Our tests will fail less often
1. Our users won't encounter strange UX behavior in the total price changing unexpectedly

**Note:** since the problem is happening on multiple pages, we want to implement a holistic solution for all pages.

### Details

Here's an example of order total and price not correctly updated after quantity entered: 

![image](/uploads/cb9f15d649ee7076a6677728b3ed7a8e/image.png)

### Steps to reproduce

Seems to be intermittently failure in pipeline. More investigation needed.

### Reported examples

1. https://gitlab.com/gitlab-org/customers-gitlab-com/-/jobs/1080777644
1. https://gitlab.com/gitlab-org/customers-gitlab-com/-/jobs/1084970324
1.

<!---
Please add both severity and priority level according to: https://docs.gitlab.com/ee/development/contributing/issue_workflow.html#severity-labels
--->

Story points: 2.0
ProServ Training Per-Seat Add-on SKU

# SKU Request Summary

<!---
Provide a brief description of what this SKU will be used for.
--->



## SKU Requirements

In order to expedite your request, please complete all of the following information.
Any questions about this template or this request can be directed to @amandarueda in the comments of this issue.

* [x] SKU Name: ProServ Training Per-Seat Add-on
* [x] SKU Code/ID: [ProServ Training Per-Seat Add-on](https://docs.google.com/spreadsheets/d/1tOGWs86qtiT7p_21qBI_sJCTWWKJKVmj7pIV9lmr6H4/edit?usp=sharing)
* [x] Zuora Plan Name: [ProServ Training Per-Seat Add-on](https://docs.google.com/spreadsheets/d/1tOGWs86qtiT7p_21qBI_sJCTWWKJKVmj7pIV9lmr6H4/edit?usp=sharing)
* [x] Zuora Plan ID: [ProServ Training Per-Seat Add-on](https://docs.google.com/spreadsheets/d/1tOGWs86qtiT7p_21qBI_sJCTWWKJKVmj7pIV9lmr6H4/edit?usp=sharing)
* [ ] Service type: `add-on`
* [x] Does this need to be purchase-able via CustomersDot? `No`
* [x] Does this service renew? `No`
     * [ ] If yes, what is the renewal cadence? `monthly | annually | other`
* [x] Should an email be sent indicating the subscription is coming to an end?: `No`
* [x] Is there a unit of measure, if so what? ` Yes`  `Education Service`
* [x] Does this service provision anything? `other`
     * [ ] If yes, please describe in detail what should be provisioned including the systems affected:
* [x] How soon do you need this SKU ? Timeline: As soon as possible
* [x] Who in Sales-Ops approved this request: @c.yoshida 
* [x] Link to originating Finance issue: https://gitlab.com/gitlab-com/Finance-Division/finance/-/issues/3676
* [ ] Apply the appropriate `BTG Priority` label to this issue.

Story points: 2.0
Cleanup old trials controller and views

### Problem

I noticed we have some views related to `trials` not being used.

- `app/views/trials/new.html.haml`
- `app/views/trials/_simple_gitlab_com_trial_form.html.haml`

### Proposal

Confirm it's not being used and delete related code (e.g. controller, routes, views, etc.)

### Next steps (if any)

Story points: 3.0
Remove font awesome icons and replace them with gitlab-ui SVGs

## Description

At the moment **we're loading the Font Awesome CSS for only one icon** in the invoices view:

| Screenshot | Description|
| --- | --- |
| ![Screenshot_2021-03-06_at_05.49.40](/uploads/37be0b3b691beb3975aba38810ba85f2/Screenshot_2021-03-06_at_05.49.40.png) | Empty state of invoices view (there are no invoices to display) |

This is a `6.5kB` http request to a 3rd party URL that we can avoid:

![Screenshot_2021-03-04_at_10.03.34](/uploads/b14597ea1b55ffd85537de54b21290f5/Screenshot_2021-03-04_at_10.03.34.png)

## Suggested optimization

We should replace this icon with [the info SVG we already have](https://gitlab.com/gitlab-org/customers-gitlab-com/blob/cbded25ee79e137cd795ce3d88b5394c4092e083/app/views/shared/svg/_information_o.svg). 

## font-awesome usage in the customers-gitlab-com project

| Type | RegEx used | Place(s) used |
| --- | --- | --- |
| Library CSS import | | https://gitlab.com/gitlab-org/customers-gitlab-com/blob/1cf439b58cbd13466d270baa23aa6ee01e790709/app/views/layouts/application.html.haml#L12 |
| HAML usage | `[ "\.]fa[ "\.]` | https://gitlab.com/gitlab-org/customers-gitlab-com/blob/7cb5d6b61a25194be52c5a7416bf2225412b1c12/app/views/invoices/index.html.haml#L27 |
| SCSS usage | `\.fa[-\.]+` | https://gitlab.com/gitlab-org/customers-gitlab-com/blob/1593ae6ea0837ecf2236e6df4c097542b81df84e/app/assets/stylesheets/main.scss#L183 |

Story points: 1.0
Upgrade subscription not showing payment method errors

### Problem

When upgrading a subscription (new upgrade app), if an invalid credit card is entered and the page is submitted, the error received from the backend is not shown properly on the frontend.

![Screen_Shot_2021-03-03_at_16.22.17](/uploads/4b329a6d06232f780dcd85c267031235/Screen_Shot_2021-03-03_at_16.22.17.png)

In the example above I submitted the page without entering the credit card data (the same happened when entering an expired credit card).

I tested the same scenario on the renew subscription page and the error doesn't exist there, it was showing the backend error correctly.

Story points: 2.0