Better HA for Citrix provisioning: PowerShell conditional reboot script for target devices

Citrix has done a good job in helping us make a XenApp or XenDesktop environment based on provisioning high available.
Once your target devices are up, you’re mostly in the green zone:

• A target device that has acquired license has a grace period.
• Offline database support allows your SQL server to take a break.
• 2 or more provisioning servers ensure high availability for your target devices.
• NIC teaming
• And so on…

However, when your target devices go into a (scheduled) reboot, you can go into the red zone.

When a target devices boots, there can be several single points of failure:

• Option 66 in your DHCP scope (the boot server hostname) is a single entry option.
• Your target devices  shutdown in 5 minutes after booting when the Citrix license server is unavailable.
• No DHCP server will cause your target devices to not receive their reservation and scope options.
• No TFTP server will cause your target devices to not receive their bootstrap file.

All of these will result in the same thing: Your target devices will not boot.
Ofcourse these components can be made high available through a load balancer for example, but sometimes options like that are unavailable.

I was at a customer where high availability for his new environment was priority number one, but the budget was very tight.
To eliminate any single point of failure during the daily scheduled reboot (which occurred during the night), I created a PowerShell script to make sure all single points of failure are alive before initiating a reboot.

The script checks if a service on a server is running before rebooting the target device.
If any of the services on the single points of failure are not running it will send an e-mail, reporting the failed server and service. The target device will not reboot.

In the script I included some of the single points of failure I mentioned above, but you can always add or delete your own to tailor it to your environment.
Make sure you run the script under an account which has enough rights to query the service on the remote server.
If your SMTP server uses a custom port, that is included as well.

# Conditional Reboot Script
# Created by Michel Stevelmans - http://www.michelstevelmans.com

$Global:Reboot = $True
function Check-Service {
        param ($Server = '',$Service = '')
        $Status = Get-Service -Computername $Server -Name $Service
            If ($Status.Status -eq 'Running')
            {
            }
            else
            {
            $Global:Server = $Server
            $Global:Reboot = $False
            $Global:ErrorBody = $Global:ErrorBody + "`r`nHost: $Server" + "`r`nService: $Service (" + $Status.Displayname + ")" + "`r`nStatus: " + $Status.Status + "`r`n"
            }
}

# Change these values ##########################################

Check-Service -Server dhcp-server.domain.local -Service DHCPServer
Check-Service -Server tftp-server.domain.local -Service BNTFTP
Check-Service -Server pvs-server.domain.local -Service StreamService
Check-Service -Server license-server.domain.local -Service "Citrix Licensing"

$EmailFrom = "michel.stevelmans@domain.com"
$EmailTo = "michel.stevelmans@domain.com"
$SMTPServer = "smtp-server.domain.local"
$SMTPPort = "25"

################################################################

if ($Global:Reboot)
    {
    Restart-Computer -Force
    }
    else
    {
    $Subject = "Reboot failed: " + (Get-Content env:computername)
    $SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, $SMTPPort)
    $SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Global:Errorbody)
    }