Testing .NET Core Apps with GitHub Actions
Tags: dotnet, github, devops, csharp
Published: 2020-08-09 08:19:50

In the following post I’m going to cover how you can run and publish results for your .NET Core Tests using GitHub actions.

> Updated 2021 to replace deprecated 'set-env' usage with new command

In particular we will be setting up a CI pipeline action to publish test results in Markdown format as both a comment on pull requests (PR), and a build artifact. Running tests automatically on PRs is a good way to make sure you don’t push code that breaks your project.

To make the most of .NET Core’s cross platform builds, and while maximizing GitHub Actions free tier, we will be using the Linux (Ubuntu) runner. At the time of writing this, the GitHub free tier gives you 2000 minutes per month on Linux runners, if you use Windows this drops to 1000 minutes, and macOS only 200 minutes.

## Getting Started

In order to run tests on GitHub we will need these prerequisites;

- **GitHub account** You can [join here](https://github.com/join) if you don't have an account
- **GitHub Repository** 
  If you don’t have a repo, take a look at the following article to get setup: [Create a repo](https://help.github.com/en/enterprise/2.16/user/github/getting-started-with-github/create-a-repo)
- **Repository cloned locally** 
  If you don’t already have a setup with favorite git client, try the official docs: [Cloning a repository](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository)


### Setting up your .NET Core test project

> *The following commands should be ran in your project folder, so make sure to open up the command prompt in the directory you have checked out your project to.*

**Optional Step.** If you already have a test project you can skip to Step 1. 
Otherwise, we will create a new test project now with the following command:

```powershell
dotnet new xunit
```

You should see a newly created test project

**Step 1. Add the Markdown logger to your test project**

We will be utilizing the **LiquidTestReports.Markdown** VS Test logger to generate a test report in the Markdown format (disclaimer: I am the creator of this library)

You can add the library either by getting it from [nuget](https://www.nuget.org/packages/LiquidTestReports.Markdown), or running the following command:

```powershell
dotnet add package LiquidTestReports.Markdown
```

**Step 2. Run the tests locally**

Make sure the tests will run on your machine with the test logger attached, by running the following command:

```powershell
dotnet test -l liquid.md
```

You output should look similar to the image below, note the “Saved report” line. If you check in your project directory you should have a new TestResults folder with the newly created **TestReport.md** file (preview shown below).

**VS code showing dotnet test console and report preview:**
![VS Test console output showing report generation
](https://dev-to-uploads.s3.amazonaws.com/i/gipq1r7vmcx4j2nuda57.PNG)

> You can delete the report for now, as we want to generate them inside the actions each time.

**Step 3. Commit and Push your code**

Now that your test project is ready from the previous steps, it’s time to get it onto GitHub itself. 

To do this, **commit** your local changes including the test project referencing the markdown logger. And **push** your branch to your remote GitHub repository. 

Once again, leaning on the official GitHub help docs, for more info, take a look at [Committing and reviewing changes to your project](https://help.github.com/en/desktop/contributing-to-projects/committing-and-reviewing-changes-to-your-project)

### Creating your first PR

With your code now pushed to the remote, jump back on your GitHub repo, and you should now see a panel saying your branch has recent pushes, click the green 'Compare & pull request' button.

![branch push notification and green 'Compare & pull request' button](https://dev-to-uploads.s3.amazonaws.com/i/ti5erkmwv283fzu7new7.PNG)

You should now see the create pull request page. 
Add a comment describing the changes e.g. 'Add new test project' and press the green 'Create pull request' button.

![Create pull request page with green button](https://dev-to-uploads.s3.amazonaws.com/i/2peeljdmlhto2y114msy.PNG)

You should now see the pull request page. For simplicity, we will just go straight to pressing the green 'Merge pull request' button
followed by the 'Confirm merge' button.

To keep things clean you can also press the 'Delete branch' button. 

With this all done, we are ready to move on to setting up the action.

------

### Setting up the Github Action

In your browser 

1. Go to your GitHub repo and click the ‘Actions’ tab

2. On the Actions tab, you will see lots of workflow options, at the top click ‘Skip this and set up a workflow yourself’

3. You will now see the yaml editor with a default CI pipeline.  Delete the contents of this, so we can start the pipeline from scratch

#### The GitHub Action

Starting with the setup, we will add something similar to the default dotnet workflow. Here we are setting up the environment, restoring the nuget dependencies, and building the project. 

>  You may notice the 'on' conditions at the top, there are two scenarios that we want to trigger a build
> - A PR has been created to merge into the master branch
> - A push to the master branch, created by a merge commit 


```yaml
name: .NET Core Build with Tests

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    # use ubuntu for more build minutes
    runs-on: ubuntu-latest
    # use release mode for all steps
    env:
      config: 'Release'
      
    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.101    
      
      # restore dependencies  
    - name: Install dependencies
      run: dotnet restore
      
      # build project
    - name: Build
      run: dotnet build --configuration $config --no-restore
```

Next, we will be conditionally setting some variables to set the test report name and title, for both cases, we want to include the run number, so we know which build the test results are for. If it is a pull request, then we additionally want to know the pull request number.

```yaml
      # set pr number, if it's a pr build
    - name: set pr build number
      id: PRNUMBER
      if: ${{ github.event_name == 'pull_request' }}
      uses: kkak10/pr-number-action@v1.3
      
      # set report file and title 
    - name: Set Test Title
      run: | 
            if ${{ github.event_name == 'pull_request' }}
            then
              echo "title=Test Run for PR #${{steps.PRNUMBER.outputs.pr}} (${{github.run_number}})" >> $GITHUB_ENV
              echo "file_name=TestReport.${{steps.PRNUMBER.outputs.pr}}.${{github.run_number}}.md" >> $GITHUB_ENV              
            else
              echo "title=Test Run ${{github.run_number}}" >> $GITHUB_ENV
              echo "file_name=TestReport.${{github.run_number}}.md" >> $GITHUB_ENV
            fi
```

With this set, we can now move on to running the tests, this is using the same `dotnet test` method as earlier in the post, but now passing in some parameters for the title, and file name.

```yaml
  # run tests with built project
- name: Test PR      
  run: dotnet test --no-restore --no-build --configuration $config --logger:"liquid.md;LogFileName=${{github.workspace}}/${{env.file_name}};Title=${{env.title}};"
```

Now the report is generated, we want to store it, this next step will attach the markdown file as a build artifact.

```yaml
  # upload report as build artifact
- name: Upload a Build Artifact
  uses: actions/upload-artifact@v2
  if: ${{always()}}
  with:
    name: 'Test Run'
    path: ${{github.workspace}}/${{env.file_name}}
```

Lastly, we want to add one final step, for pull requests. This step will attach the contents of the markdown report as a comment to your pull request.

```yaml
  # add report as PR comment (if PR)
- name: comment PR
  uses: machine-learning-apps/pr-comment@master
  if: ${{ github.event_name == 'pull_request' }}
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    path: ${{env.file_name}}
```

A complete copy of the workflow is available [here on a gist](https://gist.github.com/kurtmkurtm/286b579242c14f523f904183daeeb38c).

**Time to commit**
Pressing the green "start commit" button 

There's two options here, commit to `master` branch, or create a branch. 

I'm going to: 
1. select the '**new branch**' option
2. set the branch name to `ci/test-workflow`
3. press the green 'Propose new file' button

![Create new branch for commit](https://dev-to-uploads.s3.amazonaws.com/i/zvcbxxftkb0fcjejnp5t.PNG)

You should now see the familiar 'Open a pull request' page. Once again, press the green button to create the pull request.

With the pull request created, switch over to the 'Actions' tab. You should now see your workflow, and after a minute or so of running, a green tick next to the workflow result.

![Workflow result with green tick](https://dev-to-uploads.s3.amazonaws.com/i/80t25xx4pnvph6r3lzkm.PNG)

Now, switch back to your pull request, and you should notice a new comment on it containing your test report.

![PR comment report](https://dev-to-uploads.s3.amazonaws.com/i/x5jqv41cdc25wtlczxg9.PNG)

From here, merge this PR, and you will have test reports included in your builds and PRs from now on.

A complete working sample is available here:
https://github.com/kurtmkurtm/.net-core-workflow/actions