Achieving single command Infrastructure deployment using PowerShell DSC.
Tags: powershell, devops
Published: 2022-04-08 11:33:09

So, previously I wrote about [deployment process to IIS servers](https://dev.to/deacdatacenter/bringing-down-aspnet-deployment-interruptions-from-minutes-to-blip-of-a-time-with-azure-devops-112i). However wouldn't it be nice to keep that IIS configuration in sync, moreover between production and staging environment?

## Configuration deployment world.

One way is to create the initial installation and configuration script. Nice and fast to setup additional server, a great first step. But:
- The configuration has to be maintained and your script has to account for that (_if_ this already exists _then_ do nothing _else_ configure).
- You need to have a mechanism to run the script. Scheduler?
- Your systems may drift if some processes or users do some configuration.
- You do not know the diff between the current system configuration and the script.

There are many tools that may be used to provide uniform server configuration, like [Group Policy](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh831791(v=ws.11)), [System Center Configuration Manager/Microsoft Endpoint Configuration Manager](https://docs.microsoft.com/en-us/mem/configmgr/core/understand/introduction), [PowerShell DSC](https://docs.microsoft.com/en-us/powershell/dsc/getting-started/wingettingstarted?view=dsc-1.1) (Desired State Configuration), Docker and various other software configuration management tools etc. All of those could be used to help achieve completely or partially uniform configuration across multiple hosts. Some of them are not free, some of them depends on Active Directory, some of them have GUI and so on.

## Enter PowerShell DSC

Lets just explore PowerShell DSC, that you don't have to license and if you, like me, enjoy using PowerShell, this may be a tool for you.

Things I immediately found satisfying with PowerShell DSC:
- It's PowerShell (with a little bit of additional syntax to learn) and no additional tools to install (apart from PowerShell modules).
- It works over [WinRM](https://docs.microsoft.com/en-us/windows/win32/winrm/installation-and-configuration-for-windows-remote-management)
- It helps you define [Infrastructure-as-Code] (IaC) (https://en.wikipedia.org/wiki/Infrastructure_as_code)
- You can either make report configuration drifts, [check them manually](https://octopus.com/blog/getting-started-with-powershell-dsc#detecting-drift) or [make your host to automatically re-apply](https://octopus.com/blog/getting-started-with-powershell-dsc#automatically-correcting-drift) whatever was instructed in DSC script.

In an ideal world, once our server is joined to domain, I would like a one-click install for servers, be it staging or production environment. PowerShell DSC achieves quite a lot, but not necessarily a single click deployment. We will see why and how to overcome it.

Why did I go out to try PS DSC?
1. Didn't want to write configuration documentation - let there be text that can't be outdated.
2. IaC - wanted to have valid configuration in source control
3. I heavily use PowerShell and this would be a "natural extension".
4. This has come as a bonus: On deployment, check if IIS webserver configuration is as written.

## Writing DSC script

I will be using PowerShell 5.1 with [DSC 1.1](https://docs.microsoft.com/en-us/powershell/dsc/overview?view=dsc-1.1). Within [v2](https://docs.microsoft.com/en-us/powershell/dsc/overview?view=dsc-2.0), DSC has been split out from powershell package and [v3](https://docs.microsoft.com/en-us/powershell/dsc/overview?view=dsc-3.0) will provide cross-platform capabilities.

What I'm going to concentrate on is overcoming hurdles when writing DSC and not the actual configuration itself. You can drill down the [concepts](https://docs.microsoft.com/en-us/powershell/dsc/configurations/configurations?view=dsc-1.1) and [how-to](https://docs.microsoft.com/en-us/powershell/dsc/configurations/write-compile-apply-configuration?view=dsc-1.1) documentation to learn DSC. My requirements were:

1. I want ONE or maybe two simple commands I can use to configure IIS servers
2. I want to be able target all or subset of servers
3. I want to target different environments (prod, staging)

Let's look at a simple `dsc.ps1` script.
```powershell
configuration SimpleConfiguration {
    # DSC provides only simple tasks. Usually modules will be imported to do additional tasks.
    Import-DscResource -ModuleName 'ComputerManagementDsc' -Name 'PendingReboot', 'TimeZone', 'SystemLocale'
    # One can evaluate expressions to get the node list E.g: $AllNodes.NodeName - AllNodes can be passed when calling function.
    node ("Node1","Node2","Node3")
    {
        # Call Resource Provider E.g: WindowsFeature, File
        WindowsFeature WebServerRole { Ensure = "Present"; Name = "Web-Server" }
        SystemLocale EnglishLocale { SystemLocale = "en-US" ; IsSingleInstance = "Yes" }
        File site1Folder { DestinationPath = "$env:SystemDrive\inetpub\sites\site1"; Type = 'Directory' }
    }
}
```
### Understanding resources

What does `SystemLocale EnglishLocale { SystemLocale = "en-US" ; IsSingleInstance = "Yes" }` mean?

It means I'm calling `SystemLocale` resource, giving it whatever friendly  name `EnglishLocale` and passing hashtable with properties resource accepts: `SystemLocale`, `IsSingleInstance`. You can get available properties like that:

```powershell
Get-DscResource SystemLocale -Syntax
SystemLocale [String] #ResourceName
{
    IsSingleInstance = [string]{ Yes }
    SystemLocale = [string]
    [DependsOn = [string[]]]
    [PsDscRunAsCredential = [PSCredential]]
}
```

### Why can't it be a simple install script?
It can, but in real world - without the word "simple". It's written in [Desired State Configuration Overview for Engineers](https://docs.microsoft.com/en-us/powershell/dsc/overview/dscforengineers?view=dsc-1.1). In short: you focus on WHAT to do not on HOW. DSC provides [idempotent](https://en.wikipedia.org/wiki/Idempotence#Computer_science_examples) (repeatable) deployments. Along with goodies like [Continue configuration after reboot](https://docs.microsoft.com/en-us/powershell/dsc/configurations/reboot-a-node?view=dsc-1.1), [Credential encryption](https://docs.microsoft.com/en-us/powershell/dsc/configurations/configdatacredentials?view=dsc-1.1) and other goodies.

### Why this silly syntax?
Well, because this resource [under the hood](https://docs.microsoft.com/en-us/powershell/dsc/resources/get-test-set?view=dsc-1.1) actually implements 3 functions: `Get`, `Set`, `Test`
- **Get** is a representation of state when, for example, running [`Test-DscConfiguration -Detailed`](https://docs.microsoft.com/en-us/powershell/module/psdesiredstateconfiguration/test-dscconfiguration?view=dsc-1.1) or [`Get-DscConfiguration`](https://docs.microsoft.com/en-us/powershell/module/psdesiredstateconfiguration/get-dscconfiguration?view=dsc-1.1)
- **Set** - Applies configuration
- **Test** - Determines whether configuration has to be applied. If not, **Set** is NOT run. That is, if system is already compliant with this resource, it won't re-run.

When Custom resources gets written, these functions get implemented. For example, using built-in `Script` resource, you also have to implement all of them. Take for example a configuration, that ensures `BUILTIN\Users` have no access to particular folder:
```powershell
Script site1FolderPermissions {
  GetScript = { @{Result="Path = $env:SystemDrive\inetpub\sites\site1; Acl = $(Get-NTFSAccess "$env:SystemDrive\inetpub\sites\site1" | %{ $_.ToString() })" } }
  TestScript = {
     (Get-NTFSAccess "$env:SystemDrive\inetpub\sites\site1" | ? Account -eq 'BUILTIN\Users' | measure | select -ExpandProperty Count) -eq 0
  }
  SetScript = {
    Disable-NTFSAccessInheritance -Path "$env:SystemDrive\inetpub\sites\site1"
    $acl = Get-Acl "$env:SystemDrive\inetpub\sites\site1"
    $acl.Access | ?{ $_.IsInherited -eq $false -and $_.IdentityReference -eq 'BUILTIN\Users' } | % {$acl.RemoveAccessRule($_)}
    Set-Acl -AclObject $acl "$env:SystemDrive\inetpub\sites\site1"
  }
  DependsOn = "[File]site1Folder"
}
```

## Drawbacks

### Modules
Drawbacks or pain-points or maybe stuff that hasn't been provided by DSC out-of-the-box: Distributing modules. And this is the reason I can't have a single command to bootstrap my servers. The modules have to be installed on the host that are building the configuration AND the host that gets configuration deployed. Otherwise, when encountering `Import-DscResource -Module CertificateDsc -ModuleVersion 5.1.0`, it will tell you:
```
Could not find the module '<CertificateDsc, 5.1.0>'.
```
![Could not find the module error](https://i.imgur.com/8WLpIpR.png)

However moving `Import-DscResource` command just will result in error:
```
Import-DscResource cannot be specified inside of Node context
```
![Import-DscResource cannot be specified inside of Node context](https://i.imgur.com/Q0OKrxO.png)

So the takeaway: **we can't install a module and use its resources in a single configuration**. These modules are required on the host that are building configuration and on target nodes. Modules are essential to any configuration, except the most [trivial ones](https://docs.microsoft.com/en-us/powershell/dsc/resources/resources?view=dsc-1.1#windows-built-in-resources).

Moreover, the [`PackageManagement`](https://docs.microsoft.com/en-us/powershell/dsc/reference/resources/packagemanagement/packagemanagementdscresource?view=dsc-1.1) resource you see in a screenshot is a module itself that must be installed from PowerShell gallery before I can actually use it to install other modules 🤦‍♂️ 

Remember me telling how DSC enables writing more simple configuration comparing to install scripts? Turns out with DSC comes complexity anyways if I want automatic module provisioning. 🤷‍♂️ I don't want anyone (and my future self) running the deployment having to think about what modules must be installed on source & target machines.

So I will be using another configuration to install modules, but without any dependencies on modules. I'll run this configuration on hosts that require these modules and only then apply main configuration. First, I apply `InstallPSModules` configuration and then the `SampleConfig` configuration. There is a gotcha: I need modules on localhost too. But running `Start-DscConfiguration .\InstallPSModules\ -ComputerName localhost` gives me error:
```powershell
The WinRM client sent a request to the remote WS-Management service and was notified that the request size exceeded the configured MaxEnvelopeSize quota
```
It may be misleading. If I pass `-Credential` parameter, specifying which user I want to use to connect to localhost (admin), then it works. This is due to the way users/computers/winrm are configured in my environment.

The other thing is that it may override some other DSC configuration applied to _this_ computer. Anyways, I had to choose a tradeoff. I have a build task (a function) that installs modules on local computer and another for deploying modules to target nodes. But then I have to remember adding any dependencies in 2 places. Maybe I'm trying too hard for striving to reach the goal of "_1 simple command to perform deployment_"...

What I'm doing is installing from PowerShell gallery. You could set-up so that you have modules available on some share/local machine and copy them to target machine `C:\Windows\system32\WindowsPowerShell\v1.0\Modules` folder. Or perhaps any of these folders:
```console
PS C:\Repos\psdsctest> C:\Tools\PSTools\PsExec64.exe -s powershell.exe -Command "`$env:PSModulePath -split ';'"

C:\Windows\system32\config\systemprofile\Documents\WindowsPowerShell\Modules
C:\Program Files\WindowsPowerShell\Modules
C:\Windows\system32\WindowsPowerShell\v1.0\Modules
```

That's `PSModulePath` for SYSTEM account. The configuration itself is being applied in the context of `SYSTEM` account on target nodes. **Important** to keep in mind when writing configurations. If you need to apply some stuff within context of another user, read about [Credential Options in Configuration Data](https://docs.microsoft.com/en-us/powershell/dsc/configurations/configdatacredentials?view=dsc-1.1).

Here is an example build task for some configuration I wrote that copies files to particular computers before doing deployment (err, read "[Hooking it all up](#hooking-it-all-up)" to see that I'm using a build tool, thus another "weird" syntax):

```powershell
task copyRequiredFilesToNodes {
	exec {
		Write-Host "Copying $BuildRoot\winlogbeat to target nodes"
		$nodeSessions = New-PSSession -ComputerName $Nodes
		Invoke-Command -Session $nodeSessions { Remove-Item -Recurse -Path "C:\Windows\Temp\winlogbeat" -Verbose -ErrorAction SilentlyContinue}
		$nodeSessions | % { Copy-Item -Recurse -Path "$BuildRoot\winlogbeat" -Destination "C:\Windows\Temp\" -ToSession $_ -Force -Verbose }
	}
}
```

### DSC Engine configuration for enabling reboot
Moreover, DSC Engine itself must be configured to, for example, allow reboots with [PendingReboot resource](https://docs.microsoft.com/en-us/powershell/dsc/configurations/reboot-a-node?view=dsc-1.1&viewFallbackFrom=powershell-7.2). DSC Engine configuration is applied with [Set-DscLocalConfigurationManager](https://docs.microsoft.com/en-us/powershell/module/psdesiredstateconfiguration/set-dsclocalconfigurationmanager?view=dsc-1.1) command and uses different configuration:
```powershell
[DSCLocalConfigurationManager()]
configuration DSCConfig
{
    Node $AllNodes.NodeName
    {
        Settings
        {
            #allow reboot with PendingReboot resource from ComputerManagementDsc module
            RebootNodeIfNeeded = $true
        }
    }
}
```

Applying configuration:
```powershell
Set-DscLocalConfigurationManager .\DSCConfig\ -ComputerName localhost -Verbose
```

So, multiple gotchas here. Lets hook it all up within a single build script we can call.

### Targeting other environments

Configuration scripts are powershell scripts. You may choose to pass parameter or detect which env you are in any other way:
```powershell
$environment = if ($env:USERDOMAIN -ieq "prod.example.com") {"production"} else {"staging"}
```

Then you can use if statements or override variables based on environment.

```powershell
if ($environment -eq "production") {
    #Call some resources
} else {
    #Call other resources
}
```

## Hooking it all up

So I need to take a step back and `Start-DscConfiguration` won't be my entry point to deployment. I need a .ps1 script that installs modules. Or, bear with me, I'll be using a PowerShell build tool: [Invoke-Build](https://github.com/nightroman/Invoke-Build).
{% embed https://github.com/nightroman/Invoke-Build %}

You may use other tools too: [psake](https://github.com/psake/psake), [make](https://www.gnu.org/software/make/), [cake](https://cakebuild.net/), [fake](https://fake.build/) or any other [*ake](https://en.wikipedia.org/wiki/List_of_build_automation_software) you are familiar with. I look at them as a tools that make build tasks behind simple commands and help me answer: [How did I run that code again?](https://hippocanvas.com/posts/tools%20for%20remembering%20code%20projects.md)

So, we still must install Invoke-Build? Well, conveniently, there is a [template](https://github.com/nightroman/Invoke-Build.template/blob/main/samples/Script4/Script4.build.ps1) for invoke-build file that detects if Invoke-Build is installed or not and installs it and we'll use it.

The result ended up using 4 scripts. However it can be used as an example/bootstrap project to speed up your implementation, if you choose to go down the same path:
- `build.ps1` - Invoke-Build script. Entry point
- `DSCConfig.ps1` - DSC Engine configuration. Allows reboots
- `InstallPSModules.ps1` - Module deployment
- `SampleConfig.ps1` - Where I would put my configuration.

In the end, I can checkout the code and run:
```powershell
.\build.ps1 deploy -Nodes localhost
# or
.\build.ps1 deploy -Nodes server1, server2, server3 -Credential (Get-Credential)
# By the way, whatever credential you pass to the build script is only for connecting to target node and starting DSC configuration. Remember that resources itself are applied within SYSTEM context.
```

I'v put up the scripts at psdsctest repository:

{% embed https://github.com/janis-veinbergs/psdsctest %}

## Tying in Azure DevOps

In the end, when deploying code to IIS servers, I included a call to `Test-DscConfiguration -Detailed` to inform me whether host configuration has drifted away.

```powershell
$testdsc = Test-DscConfiguration -Detailed
if (-not $testdsc.InDesiredState) {
  Write-Warning "Host configuration has drifted. Test-DscConfiguration returned False"
  Write-Output $testdsc.ResourcesNotInDesiredState
}
```

![Oh no, 2 servers have configuration drift?](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bas27q16rffua0flegit.png)
  

I'm not using any automatic corrections or not yet using DevOps to automatically apply configuration when DSC configuration changes have been committed. But it's possible to do that. However there comes additional complexity - perhaps I wouldn't like to reboot all servers at once if reboot is required. Or implement rolling deployment - apply configuration to some servers, see how they behave and then others.

## Conclusion

I really wish PowerShell DSC would have provided OOTB way of installing modules and requiring them after installation. Anyway, I'll have the "frame-work" ready for the next DSC project.

Otherwise it's a nice tool for configuration deployment, if you can find the modules. But there are plenty.

Moreover, I do recommend looking at [Building a Continuous Integration and Continuous Deployment pipeline with DSC](https://docs.microsoft.com/en-us/azure/devops/pipelines/release/dsc-cicd?view=azure-devops) article - it even shows how to run tests to validate your configuration.

Looking forward to any comments on what could have been done better.