giseldo/neodataset huggingface dataset:
Change instructions to recommend pipx instead of manually-created virtual environments

Before joining Meltano, I'd created !1960+ to recommend installations with pipx instead of pip and manually-generated virtual environments. This came up again today and (now that !2496+ is fixed), I'd like to revisit/resurrect the pipx-based install guidance.

## Why this matters

I think we lose people when we ask them to create their own virtual environments. Also - then anything installed into that virtual environment is unavailable until the environment is manually activated, which requires memorizing the path to the activation script, and the whole process is just more cumbersome than I'd like.

Pipx was directly created to solve this problem:

- Virtual environments are created automatically.
- Shims are installed automatically to PATH, so you never need to manually activate the environments.
- The syntax is the same to install and upgrade, except one character: `pip3 install meltano` -> `pipx install meltano`.

Cc @edgarrmondragon

Story points: 4.0
Legacy python 3.6 compatibility code

With the dropping of python 3.6 support in #2991, I also had a very quick search through the codebase to find comments that referenced 3.6 compatibility hacks:

```
(melty-3.8) ➜  meltano git:(2991-python-3-6-eol) ✗ grep "3\.6" * -r | egrep -v "\.js|.vue|.lock|\.svg|.node_modules|\.log|Binary file|CHANGELOG"
src/meltano/core/plugin_install_service.py:        # TODO: use dataclasses.dataclass for this when 3.6 support is dropped
tests/meltano/cli/test_run.py:            # or is hack to work around python 3.6 failures
tests/meltano/cli/test_run.py:            # or is hack to work around python 3.6 failures
tests/meltano/cli/test_run.py:            # or is hack to work around python 3.6 failures
tests/meltano/cli/test_run.py:            # or is hack to work around python 3.6 failures
tests/meltano/cli/test_run.py:            # or is hack to work around python 3.6 failures

# and heres some other code paths that reference 3.7
src/meltano/core/block/extract_load.py:    with suppress(AttributeError):  # `wait_closed` is Python 3.7+
src/meltano/core/logging/utils.py:            with suppress(AttributeError):  # `wait_closed` is Python 3.7+
src/meltano/core/runner/singer.py:            with suppress(AttributeError):  # `wait_closed` is Python 3.7+
src/meltano/core/utils/__init__.py:    # and inspired by Python 3.7's `asyncio.run`
```

There's probably a few more here and there, but these are the most obvious ones that pop'd up in my quick searches.

Story points: 4.0
Update vuepress to support netlify deployment

Related to #3147 (confidential due to some provision/security related topics). We're migrating our docs site to netlify and will be serving docs from a new subdomain of https://docs.netlify.com. 

Since we've long been using Netlify for doc previews on pull requests, Netlify is actually already configured to build on MR's, and deploy on merges to master - so we're already good to go as far as build/deploy goes.

The only real update required is to our vuepress structure and potentially some redirects on the netlify side.

Story points: 4.0
Follow-up from #2716: clickable urls in --help

The following discussion from !2492 should be addressed:

- [ ] @edgarrmondragon started a [discussion](https://gitlab.com/meltano/meltano/-/merge_requests/2492#note_809858053):  (+3 comments)

    > My only nit with these links is that `click` seems to like wrapping them and so clicking them doesn't work:
    > 
    > ```console
    > $ meltano environment --help
    > Usage: meltano environment [OPTIONS] COMMAND [ARGS]...
    > 
    >   Manage Environments.
    > 
    >   Read more at https://meltano.com/docs/command-line-
    >   interface.html#environment
    > 
    > Options:
    >   --database-uri TEXT  System database URI.
    >   --help               Show this message and exit.
    > 
    > Commands:
    >   add     Add a new environment.
    >   list    List available environments.
    >   remove  Remove an environment.
    > ```

Story points: 1.0
Documented environment alias `MELTANO_LOG_LEVEL` setting for cli.log_level is not used

The [documented alias `MELTANO_LOG_LEVEL`](https://meltano.com/docs/settings.html#cli-log-level) for log level is ignored by Meltano:

```console
$ meltano --version
meltano, version 1.90.1

$ MELTANO_LOG_LEVEL meltano invoke tap-gitlab
<no effect, same output>

$ MELTANO_CLI_LOG_LEVEL meltano invoke tap-gitlab
<debug output 👍‍>
```

The fix might be as simple as adding the `env_aliases` entry in [`setting.yml`](https://gitlab.com/meltano/meltano/-/blob/567b33a561b50863a2071dee913b810e04134f38/src/meltano/core/bundle/settings.yml#L29).

![Screen_Shot_2022-01-13_at_14.40.57](/uploads/61daf91b03bb4a4bfc534f0ab1a32de0/Screen_Shot_2022-01-13_at_14.40.57.png)

Story points: 1.0
Setting config value with $ in the value fails due to Bash Expansion

Steps to reproduce
1.  `meltano config target-snowflake set password "@$a"` 

env file contains

`TARGET_SNOWFLAKE_PASSWORD="@"` 

Should be set to `@$a`

--

Two reasons to do this in my opinion
1. Security - bash history shouldn't have your passwords inside of them
2. To prevent bash from expanding the values


I think we could do something like wait for STDIN data to come through to fix this (just like what happens if you run `cat` )

Story points: 1.0
Enable environment ENVARs to be set from command line

The following discussion from !2478 should be addressed:

- [ ] @pnadolny13 started a [discussion](https://gitlab.com/meltano/meltano/-/merge_requests/2478#note_801189369):  (+2 comments)

    > @aaronsteers @edgarrmondragon @pandemicsyn do any of you know if theres a way for me to do this with CLI commands? The [dbt bundle accesses](https://gitlab.com/meltano/files-dbt/-/blob/master/bundle/transform/profile/profiles.yml#L20) these settings as environment variables and the only way I could find to set these was to introduce the environments features and add the `env` key with local settings but I didnt see a CLI command for this so had to instruct users to manually update the yaml.

Story points: 4.0
Bump pyhumps to v3.0.x

This should help a contributor address a bug they're seeing: https://gitlab.com/meltano/meltano/-/merge_requests/2406#note_729844878:

> this seems to be a bug in the old version of pyhumps that we're using:
> 
> **pyhumps==1.2.2** (what he have)
> 
> ```python
> >>> humps.decamelize("SOME_ENV_VALUE")
> 'som_e__en_v__valu_e'
> ```
>
> **pyhumps==3.0.2** (latest)
>
> ```python
> >>> humps.decamelize("SOME_ENV_VALUE")
> 'SOME_ENV_VALUE'
> ```
>
> So changing the version constraint in `pyproject` to `pyhumps = "^3.0.0"` should and running `poetry lock` should do it.
>
> We could also do it on our end by merging this change from a different MR and rebasing this one.

Story points: 2.0
Support incremental jobs for `meltano run`

Spun out from #3094.

This items are necessary for incremental jobs.

1. Job ID support via `--job-id-prefix` xor `--job-id-name-pattern`
    - May be sourced based on the environment name - or via a CLI flag for invocations which do not have `--environment` set. (https://gitlab.com/meltano/meltano/-/issues/2301#note_750941505)
1. Top-level `--no-state-update` to not save state. (https://gitlab.com/meltano/meltano/-/merge_requests/2432#note_757893658)
1. Top-level `--full-refresh` to ignore saved state (which forces a full refresh). (https://gitlab.com/meltano/meltano/-/merge_requests/2432#note_757893658)

Of these three, I think the first is highest priority, while the second and third could optionally be delayed. In special circumstances where advanced control over state is needed, users can still fall back to `meltano elt` to get those functions.

Proposed syntax:

Job ID:

Set it at the front of a run or automatically get one based on the tasks run.

* `meltano run --job-id=<id> tap-gitlab...`

State:

We _could_ set state requirements right after `run`. For many tools (dbt included) there is a concept of incremental vs full-refresh runs. Those commands could propagate down to the specific plugins. Default is an incremental run.

* `meltano run --no-state-update tap-gitab ...`
* `meltano run --full-refresh tap-gitlab ...`

Story points: 8.0
Inadvertent ELBContextBuilder use in meltano run, results in exception when tap run_discover is called

When running a tap that performs discovery with debug mode enabled, an exception is triggered because discovery mode in the plugin is looking for a debug logger but the base_output_logger isn't accessible:

```
Traceback (most recent call last):
  File "/Users/fhines/projects/meltano/src/meltano/core/block/singer.py", line 261, in start
    self.process_handle = await self.invoker.invoke_async(
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin_invoker.py", line 277, in invoke_async
    async with self._invoke(*args, **kwargs) as (
  File "/Users/fhines/.pyenv/versions/3.8.12/envs/melty-3.8.12/lib/python3.8/site-packages/async_generator/_util.py", line 34, in __aenter__
    return await self._agen.asend(None)
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin_invoker.py", line 262, in _invoke
    async with self.plugin.trigger_hooks("invoke", self, args):
  File "/Users/fhines/.pyenv/versions/3.8.12/envs/melty-3.8.12/lib/python3.8/site-packages/async_generator/_util.py", line 34, in __aenter__
    return await self._agen.asend(None)
  File "/Users/fhines/projects/meltano/src/meltano/core/behavior/hookable.py", line 87, in trigger_hooks
    await self.__class__.trigger(self, f"before_{hook_name}", *args, **kwargs)
  File "/Users/fhines/projects/meltano/src/meltano/core/behavior/hookable.py", line 115, in trigger
    raise err
  File "/Users/fhines/projects/meltano/src/meltano/core/behavior/hookable.py", line 107, in trigger
    await hook_func(target, *args, **kwargs)
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin/singer/tap.py", line 270, in discover_catalog_hook
    await self.discover_catalog(plugin_invoker)
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin/singer/tap.py", line 315, in discover_catalog
    await self.run_discovery(plugin_invoker, catalog_path)
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin/singer/tap.py", line 363, in run_discovery
    _debug_logging_handler(self.name, plugin_invoker, handle.stderr)
  File "/Users/fhines/projects/meltano/src/meltano/core/plugin/singer/tap.py", line 52, in _debug_logging_handler
    if not plugin_invoker.context or not plugin_invoker.context.base_output_logger:
AttributeError: 'ELBContextBuilder' object has no attribute 'base_output_logger'
```

Story points: 2.0
Add a default Environment when `meltano init` is invoked

### Problem to solve

Increase awareness and use of the [`Environments` feature](https://gitlab.com/meltano/meltano/-/issues/2869) by including a default `environments: ...` entry in `meltano.yml` for new projects.

### Target audience

Users starting fresh new Meltano projects.

### Further details
(Include use cases, benefits, and/or goals)

- New projects get the benefits of having at least one declared environment in `meltano.yml`.

### Proposal

An initial set of environments that might be sensible is:

```yaml
environments:
- name: dev
  config:
    plugins: {}
  env: {}
- name: prod
  config:
    plugins: {}
  env: {}
```

### What does success look like, and how can we measure that?

We should see an increased use of `environments` for new projects.

### Regression test
(Ensure the feature doesn't cause any regressions)
- [ ] Write adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

### Links / references

Story points: 2.0
Add a default logging.yaml when `meltano init` is invoked

### Problem to solve

Now that structured logs have been added to Meltano by https://gitlab.com/meltano/meltano/-/issues/2772, it'd be good if new Meltano projects were initialized with a `logging.yaml` containing some good defaults.

### Target audience

Users starting fresh new Meltano projects.

### Further details

- Logging configuration is present from the start
- Users are aware of configurable logging by virtue of the file being present

### Proposal

The initial `logging.yaml` should probably have both the option for colored logs during development, and JSON logs for production scenarios.

The file should probably include a header with some instructions, links, etc.

### What does success look like, and how can we measure that?

Users start using and configuring structured logging in their Meltano projects. This increased use should bring more contributions as bug reports and feature proposals.

### Regression test
(Ensure the feature doesn't cause any regressions)
- [ ] Write adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

### Links / references

_Please note that this was taken from GitLab, to be changed accordingly_

Story points: 4.0
Deprecate `env_aliases` from Meltano

Our latest `discovery.yml` file contains ~~36 refs to `env:` and~~ 73 references to `env_aliases:`. I propose we remove these from `discovery.yml` and instead direct users to use the explicit environment variable logic: `<plugin_name>_<setting_name>`.

Optionally, we can preserve this functionality in case users want to add their own mappings within their `meltano.yml` files, but to my understanding, this is fully redundant with just setting the config value to be equal to the value (e.g. `value: $MY_ENV_VAL` is basically equivalent to adding the `env: MY_ENV_VAL` or `env_aliases: [MY_ENV_VAL]`).

As I understand the evolution of this featureset, the custom-named env variable aliases existing prior to the newer `<plugin_name>_<setting_name>` parsing convention. Since adding that newer convention, I think we have seen this convention go mainstream for most use cases, and we run into problems with those legacy variable names being confusing user experience if the ingestion is not expected, and/or if the order of resolution is surprising to users. See #2984 for more info on this.

## Update (2022-01-24)

Following from the notes in !2507+, we think that `env:` is serving a different function, and so this deprecation should only affect `env_aliases`. Further discussion re: `env:` can happen in !2507 and then we'll feed the findings/outcomes back into this issue.

## Best reason not to build

This is a breaking change for users who are using the older (custom) environment variable names for their projects. 

That said, it should be a simple fix to _either_ add the alias back to the `meltano.yml` file (as the `value:` or mapped in `env:`/`env_aliases:`) or just use the newer `<plugin_name>_<setting_name>` variable names.

Story points: 8.0
Research spike: can targets be tested with a zero-length input stream?

In regards to applying a generic test to targets, there is some discussion regarding whether we would need to physically create a new table in the target in order to perform the test. Douwe had posited that it may be possible to test basic connectivity information by simply sending a zero-length input stream.

I (AJ) hadn't considered that option before, and it seems a simple enough test/spike to perform before implementing a version that would necessarily create side-effects in  data landing zones which are likely considered "production" by our users.

Discussion here on a side-effect-free approach: https://gitlab.com/meltano/meltano/-/issues/1370#note_748076418

And "Option B" here: https://gitlab.com/meltano/meltano/-/issues/1370#note_747150933

Story points: 1.0
hardcoded decoding as ascii when log-level=debug

<!---
Please read this!

Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label and verify the issue you're about to submit isn't a duplicate.

If you are submitting an issue with a tap, please include:

- account details
- target details
- entities selected  with meltano select (if you have selected any entities), as the bug may be related to a specific entity
- the full elt command you are running
- full output of the meltano elt command. Logs can get pretty long, so you can add the full log as a snippet in the Meltano project and add a link in the issue.
--->

### What is the current *bug* behavior?

I'm running 
```
meltano --log-level=debug select tap-google-sheets --list --all
```
on a sheet that contains emoji characters, Meltano CLI crashes trying to decode UTF stream as ASCII.

### What is the expected *correct* behavior?

Meltano shouldn't crash when handling non-ascii stream in debug mode

### Steps to reproduce

retrieve a catalog that contains non-ascii characters, in debug mode


### Relevant logs and/or screenshots

```
Traceback (most recent call last):
  File "/home/matte/.local/pipx/venvs/meltano/lib/python3.9/site-packages/meltano/cli/__init__.py", line 53, in main
    raise CliError(str(err)) from err
meltano.cli.utils.CliError: 'ascii' codec can't decode byte 0xc3 in position 83: ordinal not in range(128)
```

### Possible fixes


https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L45

`file_like_obj.write(data.decode("ascii") if write_str else data)`

I guess the encoding should be read from the environment?

### Further regression test
_Ensure we automatically catch similar issues in the future_

- [ ] Write additional adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

Story points: 2.0
Multi-yaml Config does not respect the $MELTANO_PROJECT_ROOT env var

<!---
Please read this!

Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label and verify the issue you're about to submit isn't a duplicate.

If you are submitting an issue with a tap, please include:

- account details
- target details
- entities selected  with meltano select (if you have selected any entities), as the bug may be related to a specific entity
- the full elt command you are running
- full output of the meltano elt command. Logs can get pretty long, so you can add the full log as a snippet in the Meltano project and add a link in the issue.
--->

### What is the current *bug* behavior?

Multi-yaml Config glob resolves relative to _current_ dir even when the Meltano Project root is overridden using `$MELTANO_PROJECT_ROOT` env var.

```yaml
include_paths:
  - ./*.meltano.yml
```

![Screenshot_2021-12-10_at_18.08.22](/uploads/1bad0dd929c8424f8288ed01762d3efc/Screenshot_2021-12-10_at_18.08.22.png)


### What is the expected *correct* behavior?

Globs should be relative to `$MELTANO_PROJECT_ROOT` if it is provided.



### Steps to reproduce

See example above 👆



### Relevant logs and/or screenshots
_Please use code blocks (\`\`\`) to format console output_



### Possible fixes
_If you can, link to the line of code that might be responsible for the problem or suggest a fix_

Probably need to do some path munging [here](https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/project_files.py#L93) to produce absolute paths relative to the defined project root.

### Further regression test
_Ensure we automatically catch similar issues in the future_

- [ ] Write additional adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

Story points: 4.0
Pin meltano/tap-salesforce to tag v1.4.27

An update to allow schema selection [might soon be merged](https://gitlab.com/meltano/tap-salesforce/-/merge_requests/14)
and it may not be considered a simple patch, so we should pin the current latest ([`v1.14.7`](https://gitlab.com/meltano/tap-salesforce/-/tags/v1.4.27)) tag so users migrate at their own pace to the newer version that supports schema selection.

Story points: 2.0
Pinned git tag adswerve/target-bigquery does not exist

<!---
Please read this!

Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label and verify the issue you're about to submit isn't a duplicate.

If you are submitting an issue with a tap, please include:

- account details
- target details
- entities selected  with meltano select (if you have selected any entities), as the bug may be related to a specific entity
- the full elt command you are running
- full output of the meltano elt command. Logs can get pretty long, so you can add the full log as a snippet in the Meltano project and add a link in the issue.
--->

### What is the current *bug* behavior?

Users are not able to install the pinned version/tag of `adswerve/target-bigquery`

### What is the expected *correct* behavior?

Users should be able to install `target-bigquery`.

### Steps to reproduce

1. Try installing the `adswerve` variant of `target-bigquery` with the default _discovered_ `pip_url`.

### Relevant logs and/or screenshots


```shell
Loader 'target-bigquery' could not be installed: failed to install plugin 'target-bigquery'.
  Running command git clone --filter=blob:none -q https://github.com/adswerve/target-bigquery.git /private/var/folders/w4/qkk0hzgd7h59kkrlkxr9ly380000gq/T/pip-req-build-qi2be3sa
  WARNING: Did not find branch or tag 'v0.11.3', assuming revision or ref.
  Running command git checkout -q v0.11.3
  error: pathspec 'v0.11.3' did not match any file(s) known to git
WARNING: Discarding git+https://github.com/adswerve/target-bigquery.git@v0.11.3. Command errored out with exit status 1: git checkout -q v0.11.3 Check the logs for full command output.
ERROR: Command errored out with exit status 1: git checkout -q v0.11.3 Check the logs for full command output.

Failed to install plugin(s)
```

Slack conversation: https://meltano.slack.com/archives/C01TCRBBJD7/p1639015962380400

### Possible fixes

It's a typo, so pin the `0.11.3` tag instead.

### Further regression test

- [ ] Write additional adequate test cases and submit test results
- [ ] Test results should be reviewed by a person from the team

Story points: 1.0
Update templates for website merge requests

Relates to &232 and #6109 

Goals:

* Automatically add relevant details if possible (labels, pings, etc)
* Improve clarity of issues
* Reduce grooming time
* Ensure that all issues contain the [The Five W's](https://en.wikipedia.org/wiki/Five_Ws) : who, what, when, where, why, and how.
* Request [MoSCoW](https://en.wikipedia.org/wiki/MoSCoW_method) information when possible.

Implementation Details:

- [ ] Update the default MR template.
- [ ] Explore the use of checklists in MR templates.

Story points: 1.0
Make sure that we Capture Advanced Search in our feature and feature comparison pages

This was raised in the PM & Engineering meeting agenda.  
https://docs.google.com/document/d/1D9rW65d4w0UwSRw_0jJMRk5wAxdhAd4ynrHLbMMwIjo/edit#bookmark=id.nif5uubi5a0

> It is under Advanced Global Search https://about.gitlab.com/features/

> Sid: We need it to be added to the feature comparison page, let’s check if there are other items missing.

> Sid: Which plan is it in? Can't find on https://about.gitlab.com/pricing/gitlab-com/feature-comparison/ => Add here.

### Tasks

- [ ] Add elastic search to the feature comparison page
- [ ] Check if there is anything else that is missing from the feature comparison page

Story points: 1.0