Repository: pest-parser/pest (language: rust)

Issue:
Better CI hygiene follow-up
My current understanding of best practice is different from what we're currently doing here, though. *Probably in a follow-up*, it would be better practice to have

- check
  - +pin-nightly generate-lockfile -Zminimal-versions
  - +pin bootstrap --locked
  - +msrv check --locked
- lint
  - +pin-nightly generate-lockfile -Zminimal-versions
  - +pin bootstrap --locked
  - +pin clippy --locked -Dwarnings
  - +pin fmt --check
- test
  - +pin-nightly generate-lockfile -Zminimal-versions
  - +pin bootstrap --locked
  - +pin-stable test --locked
- future (allow-fail)
  - +beta generate-lockfile
  - +beta bootstrap --locked
  - +beta test --locked

The goal is to avoid anything changing between CI runs except the local code.

_Originally posted by @CAD97 in https://github.com/pest-parser/pest/pull/654#discussion_r922438938_


Discussion / hints:
An additional challenge mode: we'd *prefer* if we can generate code which is warning free on *all* versions MSRV+, and it'd also be *nice* to test that the project compiles on all versions in the supported range.

[cargo-hack](https://lib.rs/crates/cargo-hack) may be of interest. For practicality reasons, I'd suggest checking a rolling stable-3 or stable-6 with hack rather than all the way down to MSRV.

But also this is probably way overengineered of a CI pipeline for what we actually need for the project (which is to be aware of MSRV bumps and run tests/lints/fmt on a recent ish toolchain)
@CAD97 any good references for CI pipelines?
tokio is using `cargo hack` for testing features: https://github.com/tokio-rs/tokio/blob/master/.github/workflows/ci.yml#L94 but they don't test the version ranges.
Tracing does check the version range; [I have a separate repo component](https://github.com/CAD97/tracing-filter/blob/v0.1/.github/workflows/ci.yaml) with a workflow derived from tracing's workflow; it's [`--version-range`](https://github.com/taiki-e/cargo-hack#--version-range) to test a range of versions.

If we're updating the CI pipeline, it's probably best to move off of actions-rs ([unmaintained](https://github.com/actions-rs/toolchain/issues/216)) and to either just using `run: rustup` or [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) instead.
One more thing to add is to test no_std support: https://github.com/pest-parser/pest/issues/409#issuecomment-1184548309
* https://github.com/hobofan/cargo-nono
* https://blog.dbrgn.ch/2019/12/24/testing-for-no-std-compatibility/
* or this: `rustup run nightly -- cargo build -j1 -Z build-std=core,alloc --target x86_64-unknown-linux-gnu` should fail to compile if any dependency uses std

Fix patch:
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
new file mode 100644
index 00000000..725af30f
--- /dev/null
+++ b/.github/actions/setup/action.yml
@@ -0,0 +1,60 @@
+name: Setup
+description: Initial setup for workflows
+
+inputs:
+  kind:
+    description: Job kind (for cache key)
+    required: false
+  toolchain:
+    description: Toolchain version
+    required: false
+    default: "1.56.1"
+  components:
+    description: Toolchain components
+    required: false
+  targets:
+    description: Toolchain targets
+    required: false
+  tools:
+    description: Additional tools to install
+    required: false
+
+outputs:
+  cache-key:
+    description: Cache key
+    value: ${{inputs.kind}}-${{runner.os}}-${{steps.toolchain.outputs.cachekey}}
+
+runs:
+  using: composite
+  steps:
+    - name: Install Rust nightly # for minimal-versions
+      uses: dtolnay/rust-toolchain@nightly
+    - name: Install Rust ${{inputs.toolchain}}
+      uses: dtolnay/rust-toolchain@master
+      id: toolchain
+      with:
+        toolchain: ${{inputs.toolchain}}
+        components: ${{inputs.components}}
+        targets: ${{inputs.targets}}
+    - name: Install cargo-hack,cargo-minimal-versions
+      uses: taiki-e/install-action@v1
+      with:
+        tool: cargo-hack,cargo-minimal-versions
+    - name: Install ${{inputs.tools}}
+      uses: taiki-e/install-action@v1
+      if: ${{inputs.tools}}
+      with:
+        tool: ${{inputs.tools}}
+    - name: Generate lockfile
+      run: cargo minimal-versions generate-lockfile
+      shell: sh
+    - name: Set up cache
+      uses: Swatinem/rust-cache@v2
+      with:
+        shared-key: ${{inputs.kind}}-${{runner.os}}-${{steps.toolchain.outputs.cachekey}}
+    - name: Bootstraping Grammars - Building
+      run: cargo build --package pest_bootstrap
+      shell: sh
+    - name: Bootstraping Grammars - Executing
+      run: cargo run --package pest_bootstrap
+      shell: sh
\ No newline at end of file
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index a8f0dd89..0abaad7a 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -13,116 +13,90 @@ on:
       - master
 
 jobs:
+  check:
+    name: cargo check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        with:
+          kind: check
+          toolchain: 1.61.0
+      - name: cargo check
+        run: cargo check --all --all-features --all-targets
+
   testing:
     name: Unit, Style, and Lint Testing
+    needs: check
     runs-on: ubuntu-latest
     steps:
-      - name: Checkout source code
-        uses: actions/checkout@v2
-      - name: Install Rust Stable
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: 1.56.1 # Pinned warnings
-          components: rustfmt, clippy
-          default: true
-      - name: Install gcc
-        run: sudo apt-get update && sudo apt-get install -y gcc
-      - name: Bootstraping Grammars - Building
-        uses: actions-rs/cargo@v1
-        with:
-          command: build
-          args: --package pest_bootstrap
-      - name: Bootstraping Grammars - Executing
-        uses: actions-rs/cargo@v1
-        with:
-          command: run
-          args: --package pest_bootstrap
-      - name: Cargo Build
-        uses: actions-rs/cargo@v1
-        with:
-          command: build
-          args: --all --verbose
-      - name: Cargo Test
-        uses: actions-rs/cargo@v1
-        with:
-          command: test
-          args: --all --verbose
-      - name: Cargo Doc
-        uses: actions-rs/cargo@v1
-        with:
-          command: doc
-          args: --all --verbose
-      - name: Cargo Clippy
-        uses: actions-rs/cargo@v1
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
         with:
-          command: clippy
-          args: --all -- -D warnings
-      - name: Cargo Fmt
-        uses: actions-rs/cargo@v1
+          kind: check
+          components: clippy, rustfmt
+          toolchain: 1.61.0
+      - name: cargo fmt
+        run: cargo fmt --all -- --check
+      - name: cargo clippy
+        run: cargo clippy --all --all-features --all-targets -- -Dwarnings
+      - name: cargo test
+        run: cargo test --all --all-features --release
+      - name: cargo test (ignored)
+        run: cargo test -p pest_grammars --lib --verbose --release -- --ignored tests::toml_handles_deep_nesting_unstable
+
+  doc:
+    name: Documentation check
+    needs: check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
         with:
-          command: fmt
-          args: --all -- --check
+          kind: check
+          toolchain: 1.61.0
+      - name: cargo doc
+        run: cargo doc --all --all-features
 
   dependency:
     name: Minimal Versions Testing
+    needs: check
     runs-on: ubuntu-latest
     steps:
-    - name: Checkout source code
-      uses: actions/checkout@v2
-    - name: Install gcc
-      run: sudo apt-get update && sudo apt-get install -y gcc
-    - name: Install Rust Nightly
-      uses: actions-rs/toolchain@v1
-      with:
-        toolchain: nightly-2022-01-01
-        default: true
-        profile: minimal
-    - name: Bootstraping Grammars - Building
-      uses: actions-rs/cargo@v1
-      with:
-        command: build
-        args: --package pest_bootstrap
-    - name: Bootstraping Grammars - Executing
-      uses: actions-rs/cargo@v1
-      with:
-        command: run
-        args: --package pest_bootstrap
-    - name: Cargo minimal-versions
-      run: cargo -Z minimal-versions generate-lockfile
-    - name: Cargo test
-      uses: actions-rs/cargo@v1
-      with:
-        command: test
-        args: --all --verbose --release
-    - name: Cargo test (run ignored)
-      uses: actions-rs/cargo@v1
-      with:
-        command: test
-        args: -p pest_grammars --lib --verbose --release -- --ignored tests::toml_handles_deep_nesting_unstable
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
+        with:
+          kind: msrv
+          tools: cargo-msrv
+      - name: Check msrv
+        shell: sh
+        run: for crate in "derive" "generator" "grammars" "meta" "pest" "vm"; do cd "$crate" && cargo msrv verify && cd ..; done
 
   coverage:
     name: Test Coverage
+    needs: check
     runs-on: ubuntu-latest
     steps:
-      - name: Checkout source code
+      - name: Checkout repository
         uses: actions/checkout@v3
-      - name: Install Rust Stable
-        uses: actions-rs/toolchain@v1
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
         with:
-          toolchain: stable
-          profile: minimal
+          kind: check
           components: llvm-tools-preview
-          default: true
-      - name: Bootstraping Grammars - Building
-        uses: actions-rs/cargo@v1
-        with:
-          command: build
-          args: --package pest_bootstrap
-      - name: Bootstraping Grammars - Executing
-        uses: actions-rs/cargo@v1
-        with:
-          command: run
-          args: --package pest_bootstrap
+          toolchain: stable
       - name: Install cargo-llvm-cov
         uses: taiki-e/install-action@cargo-llvm-cov
       - name: Generate code coverage
@@ -135,3 +109,54 @@ jobs:
           name: pest-ci-coverage
           fail_ci_if_error: false
           verbose: true
+
+  hack:
+    name: cargo hack check --feature-powerset
+    needs: check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
+        with:
+          kind: check
+          toolchain: 1.61.0
+      - name: Check feature powerset
+        run: cargo hack check --feature-powerset --optional-deps --exclude-all-features --keep-going --lib --tests --ignore-private
+
+  no_std:
+    name: check for no_std compatibility
+    needs: check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
+        with:
+          kind: check
+          components: rust-src
+          toolchain: nightly-2022-07-01 # upgrade this regularly
+      - name: check no_std compatibility
+        run: cd pest && cargo build -j1 -Z build-std=core,alloc --no-default-features --target x86_64-unknown-linux-gnu
+
+  semver:
+    name: check for semver compatibility
+    needs: check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Setup
+        uses: ./.github/actions/setup
+        id: setup
+        with:
+          kind: check
+          toolchain: 1.61.0
+          tools: cargo-semver-checks
+      - name: check semver compatibility
+        shell: bash
+        run: ./semvercheck.sh
\ No newline at end of file
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index e5f6d1ba..dc3bac16 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -23,3 +23,5 @@ jobs:
         run: cargo run --package pest_bootstrap
       - name: Publish pest crates
         run: ./release.sh
+        env:
+          CARGO_REGISTRY_TOKEN: ${{secrets.CARGO_REGISTRY_TOKEN}}
\ No newline at end of file
diff --git a/generator/src/lib.rs b/generator/src/lib.rs
index 27b4d816..ce7fb98e 100644
--- a/generator/src/lib.rs
+++ b/generator/src/lib.rs
@@ -155,7 +155,6 @@ fn get_attribute(attr: &Attribute) -> GrammarSource {
 mod tests {
     use super::parse_derive;
     use super::GrammarSource;
-    use syn;
 
     #[test]
     fn derive_inline_file() {
diff --git a/release.sh b/release.sh
index c9b07b8b..741517e8 100755
--- a/release.sh
+++ b/release.sh
@@ -39,8 +39,8 @@ wait_until_available() {
       fi
     fi
   done
-  echo "Waiting an additional 10 seconds for crate to propagate through CDN..."
-  sleep 10
+  echo "Waiting an additional 20 seconds for crate to propagate through CDN..."
+  sleep 20
 }
 
 for crate in ${CRATES}; do
diff --git a/semvercheck.sh b/semvercheck.sh
new file mode 100755
index 00000000..412d2c61
--- /dev/null
+++ b/semvercheck.sh
@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+export CURRENT_GIT_SHA=`git rev-parse HEAD`
+cargo clean
+cargo install cargo-semver-checks || true
+export RUSTDOC_LATE_FLAGS="--document-private-items -Zunstable-options --output-format json"
+cargo build --package pest_bootstrap
+cargo run --package pest_bootstrap
+
+# current
+for crate in "pest_derive" "pest_generator" "pest_grammars" "pest_meta" "pest" "pest_vm"; do
+    cargo +nightly rustdoc -p $crate -- $RUSTDOC_LATE_FLAGS
+    mv target/doc/$crate.json /tmp/current-$crate.json
+done
+
+# the latest 2.1.x release
+export BASELINE_GIT_SHA="7dee2a7770daa213b45f88e6af730868f764927a"
+# baseline
+git fetch origin
+git checkout "$BASELINE_GIT_SHA"
+cargo clean
+cargo build --package pest_bootstrap
+cargo run --package pest_bootstrap
+for crate in "pest_derive" "pest_generator" "pest_grammars" "pest_meta" "pest" "pest_vm"; do
+    cargo +nightly rustdoc -p $crate -- $RUSTDOC_LATE_FLAGS
+    mv target/doc/$crate.json /tmp/baseline-$crate.json
+    echo "Checking $crate"
+    cargo semver-checks check-release --current /tmp/current-$crate.json --baseline /tmp/baseline-$crate.json
+done
+git checkout "$CURRENT_GIT_SHA"


Test patch: