Jere's Techblog

Install Consul Hashicorp trough Powershell

The Consul Hashicorp DNS service is a frequently used and modern service. Enclosed you will find a Powershell Script to install the Consul-Client as a Windows-Service. The script also allows you to run a version update.

When using the script you have to adjust the variables in line 1-13. For the update only the version number has to be adjusted, which is based on https://releases.hashicorp.com/consul/.

The script creates and updates the following config Files:
\config\consul.hcl
\config\node_exporter.json

If you don’t need this, you can comment it out of the script yourself.

#Install/Update Consul Service
#21.09.2020   by J.Kühnis

#Consul Version  based on https://releases.hashicorp.com/consul
$CONSUL_VERSION = '1.7.5'
$CONSUL_URL = 'https://releases.hashicorp.com/consul'
$CONSUL_DIR = "$env:SystemDrive\Consul"
$consulfqdn = "fqdn.consul.server.company.example"
$domainfqdn = "fqdn.server.company.example"

# Load DatacenterLoc from Webrequest
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$DatacenterLoc = (Invoke-WebRequest "http://fqdn.consul.server.company.example:8500/v1/kv/config/datacenter?raw=true" -UseBasicParsing).Content


#Windows Service params
$params = @{
  Name = "Consul"
  BinaryPathName = "$CONSUL_DIR\Consul.exe agent -config-dir=$CONSUL_DIR\config -data-dir=$CONSUL_DIR\data"
  DisplayName = "Consul"
  StartupType = "Automatic"
  Description = "Consul Hashicorp DNS Service."
}
#stop consul service, if exists
IF(get-service -Name $params.Name -ErrorAction SilentlyContinue){
    Stop-Service -Name $params.Name -Force
    do {
        Start-sleep 1
    }
    until((Get-Service -Name $params.name).status -eq 'Stopped')
    start-sleep 1
}

#Create Consul Folder
IF(!(Test-Path $CONSUL_DIR)){
    New-Item -ItemType Directory $CONSUL_DIR
}
Set-Location "$CONSUL_DIR"
#Create Subdir
$arraySubDir = @(
"data"
"certs"
"config")

$arraySubDir | %{
    IF(!(Test-Path "$CONSUL_DIR\$_")){
        New-Item -ItemType Directory "$CONSUL_DIR\$_"
    }
}

<# Set Download Proxy, if needed
$WebClient = New-Object System.Net.WebClient
$WebProxy = New-Object System.Net.WebProxy("http://your.proxy.url",$false)
$WebProxy.Credentials = $Credentials
$WebClient.Proxy = $WebProxy
#>
    #down
$WebClient.DownloadFile("${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_windows_amd64.zip","$CONSUL_DIR\consul_${CONSUL_VERSION}_windows_amd64.zip")
$WebClient.DownloadFile( "${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS","$CONSUL_DIR\consul_${CONSUL_VERSION}_SHA256SUMS")
$WebClient.DownloadFile( "${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS.sig","$CONSUL_DIR\consul_${CONSUL_VERSION}_SHA256SUMS.sig")
get-content "${CONSUL_DIR}/*SHA256SUMS"| select-string  (get-filehash -algorithm SHA256 "${CONSUL_DIR}/consul_${CONSUL_VERSION}_windows_amd64.zip").hash.toLower()

#Unzip Download
Expand-Archive "${CONSUL_DIR}/consul_${CONSUL_VERSION}_windows_amd64.zip" "$CONSUL_DIR" -Force

#Add env var
$env:path += ";${CONSUL_DIR}"
[Environment]::SetEnvironmentVariable("Path", [Environment]::GetEnvironmentVariable("Path", "Machine") + ";${CONSUL_DIR}", "Machine")


#Create/Update ConfigFiles

### Create HCL Config File; consul.hcl ###
$hcl_config = "${CONSUL_DIR}\config\consul.hcl"
IF(Test-Path $hcl_config){
    Remove-Item $hcl_config -Force
}
New-Item $hcl_config

$hcl_filecontent = @(
'datacenter = ' + '"' + $DatacenterLoc + '"'
'retry_join = ["' + $consulfqdn + '"]'
    'domain = "' + $domainfqdn + '"'
'acl = {enabled = true, default_policy = "deny", enable_token_persistence = true }'
)
Add-Content -Path $hcl_config $hcl_filecontent


### Create node_exporter.json config file ###
$json_config = "${CONSUL_DIR}\config\node_exporter.json"
IF(Test-Path $json_config){
    Remove-Item $json_config -Force
}
New-Item $json_config


$body = [pscustomobject]@{
    services= @([ordered]@{
        name = "node-exporter"
        tags = @("monitor")
        port = 9100
    })
}
   
Set-Content $json_config ($body | ConvertTo-Json -Depth 3)

#register Consul as a WindowsService
IF(!(get-service -Name $params.Name -ErrorAction SilentlyContinue)){
    New-Service @params
}
Start-Service -Name $params.Name
Continue reading...

Create Discord Bot

I way toying arround with Discord and Python because I wanted to record the messages and reactions of users.
Therefore I wrote a bot with the library discord.py My first attempts with Python…or let’s say a try and error session….

But in the end I was able toget the reactions and log messages.

below some good references and instructions and an example of my discord bot

Prerequisits:

Installation of Python 3.5<

Python Discord API – https://github.com/Rapptz/discord.py

py -3 -m pip install -U discord.py

Python await – https://pypi.org/project/await/

pip install await

Python async – https://pypi.org/project/async/

pip install async

References:

Documentation

https://discordapp.com/developers/docs/intro

https://discordpy.readthedocs.io/en/latest/discord.html

https://discordpy.readthedocs.io/

Tutorial

https://realpython.com/how-to-make-a-discord-bot-python/

My own Bot

import discord
from discord.ext import commands

client = commands.Bot(command_prefix='.')

@client.event
async def on_ready():
   #a = discord.utils.get(client.get_all_members(),name="Test-User", discriminator="9635").id  #GetUserID
    print('Bot is ready.')


"""  crawl message
@client.event
async def on_message(message):
    if str(message.author) == 'Test-user#111':
         print(f'{message.content} RECEIVED!.')
    else:
        print(f'{message.content} wrong User!.')
"""

#get reaction an log it on another channel crawl message with some condition as an example
@client.event
async def on_reaction_add(reaction, user):
    channel = client.get_channel(6637461198)
    print(f'{reaction}')
    if str(reaction.message.author.id) == '579155970803':
        if str(user.id) != '57915597803':
            await channel.send('[{0.display_name}] -  {0} has reacted with {1.emoji}!; ID = {1.message.id} '.format(user, reaction))
client.run(' some API-DiscordServer String here ')
Continue reading...

Get size of Citrix UPM Profile

With these few lines you get an evaluation of your profile sizes and the AppData, if you have redirectet this to a share.
The script is designed to check multiple shares. You only have to adjust the array “$ProfileFolders” and if necessary the variable “$folder1, $folder2”

#by J.Kühnis 18.12.2019
$ProfileFolders = @(
"\\TSProfileShare1\profile$",
"\\TSProfileShare2\profile$"
)

$infos = @()

Foreach ($folder in $ProfileFolders){
Write-Host $folder

Get-ChildItem $folder | ? { $_.Name -notmatch ".V2" } | % {
    
    Write-Host ($folder + "\" + $_.Name + "\ts\v3\prd\AppData")
   
    $folder1 = $folder + "\" + $_.Name + "\ts\v3\prd\AppData"
    $folder2 = $folder + "\" + $_.Name + "\ts\v3\prd\upm\UPM_Profile"
    IF (Test-Path $folder2) {
        $a = ((gci -Recurse $folder1 -Force | measure Length -s).sum / 1Gb)
        $b = ((gci -Recurse $folder2 -Force | measure Length -s).sum / 1Gb)
        $c = "{0:N3} GB" -f ($a + $b)
        $output = $_.Name + "," +  $c + "," + $folder
        Write-Host $output
        $infos += $output
        Clear-Variable a,b,c,output
    }
    Clear-Variable folder1, folder2
}
}

$infos | ogv
Continue reading...

Compare ActiveDirectory ACL

Here are some examples and a good description of the ActiveDirectory ACL:

https://blogs.technet.microsoft.com/ashleymcglone/2013/03/25/active-directory-ou-permissions-report-free-powershell-script-download/

Script example to compare ActiveDirectory OU ACL ( Security Groups )

by J.Kühnis 25.11.2019

Import-Module ActiveDirectory


$OU1 = Get-ACl -Path 'AD:\OU=Sales,OU=UserAccounts,DC=FABRIKAM,DC=COM' |  Select-Object -ExpandProperty Access | select IdentityReference

$OU2 = Get-ACl -Path 'AD:\OU=Marketing,OU=UserAccounts,DC=FABRIKAM,DC=COM' |  Select-Object -ExpandProperty Access | select IdentityReference

Compare-Object $OU1 $OU2 -IncludeEqual
Continue reading...

Function Count Ad-GroupMember

#by J.Kühnis 12.11.2019
Function Count-ADGroupMember {
    
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)]
        [string]$GroupName
    )


    #Get Data From AD-Group
    $groupname
    $groupdsn = (Get-ADGroup $groupname).DistinguishedName
    $group = [adsi]"LDAP://$groupdsn" 
    $groupmemebrs = $group.psbase.invoke("Members") | ForEach-Object { $_.GetType().InvokeMember("SamAccountName", 'GetProperty', $null, $_, $null) }
    $ADGroupMemberCount = $groupmemebrs.count
    return $ADGroupMemberCount
}

example:

Count-ADGroupMember -GroupName AnyGroupNameHere
Continue reading...

PowerCLI: Create VM Engine

Here is an example of how to make automated VM’s with PowerCLI. You can customize the vLAN, DiskType, etc.

If you reuse the Script below, I would recommend to look at the parameters and adapt them to your environment, e.g. network/storage config like the vLan,ESXI Hosts or the adapter type you are using its VMWare.

example:

Create-CustomVirtualMachine -Hostname 'MyServer' -NumCPU 6 -DiskSize 80 -RAMinGB 12 -OS Srv2016

Ensure you are using this script with PowerCLI or with the PowerCLI Module / Assemblys.

The script can also be saved and imported as a module (.psm1).

Import-Module "Path:\to\your\module\share\Create-VM_onFreeDatastore.psm1" -Verbose

Script to create VMs on Hypervisor with enough free ressources

#by J.Kühnis 06.11.2019
    $VmHostArray = @()

    Class VMHost {
        [string]$Name
        [string]$State
        [string]$Parent
        [Array]$DatastoreIdList
        [int]$MemoryTotalGB
        [int]$MemoryUsageGB
        [int]$FreetoUseMemory
        [int]$ReservedVMmemory
        [String]$VMCreationState
        [String]$VMCacheDiskState
        [String]$ESXiHost
        [String]$PVSCollectionState
        [String]$CTXStudioState
        [String]$CTXMaintState
    }

Function Get-VMHostVirtualMachineVMs {
 
    [CmdletBinding()]
    Param(
        #[Parameter(Mandatory=$true)][string[]]$ServerName
        [Parameter(Mandatory = $true)][String] $VMhost,
        [bool]$ExtendedProperties
    )
      
    IF ($ExtendedProperties -eq $true) {
        IF ($global:VMhostsVM = (Get-VMHost -Name $VMhost | Get-VM | select MemoryGB, VMHost, Name)) {
            return $global:VMhostsVM
        }
                
    }
    Else {
        IF ($global:VMhostsVM = ((Get-VMHost -Name $VMhost | Get-VM).Name | sort)) {
            return $global:VMhostsVM
        }
    }
    return $false
}
Function Get-vmHostDataStore {
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $true, ValueFromPipeline = $true)]
        [String[]]$DatastoreIDList
    )
    $global:DatastoreID = (get-datastore -Id $DatastoreIDList)           
    return  $global:DatastoreID
    
}
  
Function Create-VMEngine {
    
    
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $true, ValueFromPipeline = $true)]
        [string]$VMName,
        [string]$EsxiHost,
        [string]$Datastore,
        [int]$NumCPUCore,
        [int]$MemoryGB,
        [int]$DiskSize,
        $Vlan,
        $HDFormat,
        [int]$TotalCPU,
        $OS
    )

    write-host "Hypervisor Parameters:" $VMName $EsxiHost $Datastore $MemoryGB $Vlan
    write-host (get-date -f HH:mm:ss)"Start VM Creation $VmName, please wait..."
    If (Get-VM *$VMName*) {
        Write-Host "VM $VMName already exists!!!, Skip this Hostname." -ForegroundColor RED -BackgroundColor Black
        break
    }

    IF ($OS -eq 'Win10') {
        New-VM -Name $VmName -ResourcePool $EsxiHost -Datastore $Datastore -NumCPU $NumCPUCore -MemoryGB $MemoryGB -NetworkName $Vlan -GuestID windows9_64Guest -DiskGB $DiskSize -DiskStorageFormat $HDFormat
    }
    Else {
        New-VM -Name $VmName -ResourcePool $EsxiHost -Datastore $Datastore -NumCPU $NumCPUCore -MemoryGB $MemoryGB -NetworkName $Vlan -GuestID windows9Server64Guest -DiskGB $DiskSize -DiskStorageFormat $HDFormat
    }

    # Set SCSI Controller
    Get-ScsiController -VM $VmName | Set-ScsiController -Type VirtualLsiLogicSAS >$NULL -WarningAction SilentlyContinue
    Write-host "Configure VM $VMName"
       
    # Configure vCPU & Core
    $VmName = Get-VM -name $VmName
    set-vm -vm $VmName -numcpu $TotalCpu -Confirm:$false >$NULL

    # Configure MAC-Address and Adapter Type
    $vm = Get-VM -Name $vmName
    $nic = Get-NetworkAdapter -VM $vm
    $global:spec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $devSpec = New-Object VMware.Vim.VirtualDeviceConfigSpec
    $devSpec.Device = $nic.ExtensionData
    $devSpec.operation = "edit"
    $spec.DeviceChange += $devSpec
    $vm.ExtensionData.ReconfigVM($spec)
    Set-NetworkAdapter -NetworkAdapter $nic -Type Vmxnet3 -Confirm:$false >$NULL

    $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $spec.Firmware = [VMware.Vim.GuestOsDescriptorFirmwareType]::bios
    try {
        $VmName.ExtensionData.ReconfigVM($spec)
    }
    catch {
    }


    #special Parameters
    New-AdvancedSetting -Entity $vm -Name ethernet0.pciSlotNumber -Value 192 -Confirm:$false -Force:$true >$NULL -WarningAction SilentlyContinue
    Write-host "VM Config finished"
    write-host (get-date -f HH:mm:ss)"VM Creation $VmName finished!" -ForegroundColor "Green"

}


Function Create-CustomVirtualMachine {
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)][string]$Hostname,
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)][ValidateRange(1, 16)][int]$NumCPU,
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)][ValidateRange(1, 250)][int]$DiskSize,
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)][ValidateRange(1, 32)][int]$RAMinGB,
        [Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)][ValidateSet('Win10', 'Srv2016')][string]$OS
    )

    Write-host "------------------------"`n"| Start VmWare Section |"`n"------------------------"  -ForegroundColor WHITE
    
    (get-vmhost HYPERVISORFQDN1.f.q.d.n, HYPERVISORFQDN2.f.q.d.n | select Name, State, Parent, DatastoreIdList, MemoryTotalGB, MemoryUsageGB) | % {
        #$FreetoUseMemory = ("{0:N0}" -f $_.MemoryTotalGB) - ("{0:N0}" -f $_.MemoryUsageGB)
        $myhost = New-Object VMhost -Property @{Name = $_.Name; State = $_.State; Parent = $_.Parent; DatastoreIdList = $_.DatastoreIdList; MemoryTotalGB = $_.MemoryTotalGB; MemoryUsageGB = $_.MemoryUsageGB; }
        $VmHostArray += $myhost
    }
    
    Foreach ($VmWareHost in $VmHostArray) {
        $VmWareHost.ReservedVMmemory = ((Get-VMHostVirtualMachineVMs $VmWareHost.Name -ExtendedProperties $true).MemoryGB | Measure-Object -sum).sum
        $VmWareHost.FreetoUseMemory = ("{0:N0}" -f $VmWareHost.MemoryTotalGB) - ("{0:N0}" -f $VmWareHost.ReservedVMmemory);
    }

    $VmHostArray = $VmHostArray | Sort-Object -Property FreetoUseMemory -Descending
    

    #Check RAM Space on ESXI Host with 10GB reserves on Host
    IF (($VmHostArray[0].FreetoUseMemory - 10) -ge ($RAM)) {
        #Check if Disk has 10GB Free Storage
        IF (get-vmHostDataStore -DatastoreIDList $VmHostArray[0].DatastoreIdList) {
            $specVMDatastore = (get-vmHostDataStore -DatastoreIDList $VmHostArray[0].DatastoreIdList | Sort-Object -Property FreeSpaceGB -Descending)
            [int]$a = ("{0:N0}" -f $specVMDatastore[0].FreeSpaceGB).Replace("'", "")
            
            IF ($a -ge ($DiskSize + 10) ) {
                $specVMDatastore = $specVMDatastore[0].Name
    
                #StagingParameters
            
                $VMName = $Hostname
                $EsxiHost = $VmHostArray[0].Name
                $Datastore = $specVMDatastore
                [int]$NumCPUCore = 1
                $Vlan = 'Some_VlanName'
                $HDFormat = "Thin"
                [int]$TotalCPU = $NumCPU

                # Create VM on Host
                Create-VMEngine -VMName $VMName -EsxiHost $EsxiHost -Datastore $Datastore -MemoryGB $RAMinGB -Vlan $Vlan -NumCPUCore $NumCPUCore -HDFormat $HDFormat -TotalCPU $TotalCPU -DiskSize $DiskSize -OS $OS
                Write-host "----------------------"`n"| End VmWare Section |"`n"----------------------"  -ForegroundColor WHITE
            
            }
            Else {
                Write-host $Hostname "| not enough storage on ESXI Host! Action stopped." -ForegroundColor Yellow -BackgroundColor Black
            }
            
        }
        Else {
            Write-host $Hostname "| No Datastore found. Action stopped" -ForegroundColor Yellow -BackgroundColor Black
            
        }
    }
    Else {
        Write-Host $Hostname "| not enough on ESXI Host. Action stopped" -ForegroundColor Yellow -BackgroundColor Black

    }
}
Continue reading...

Jobs on PowerShell

Certain processes can take a long time. As an example, if you want to search a specific event log trough several servers.
To counteract this, Powershell has introduced “Jobs, Workflows and Foreach-Parallel”. It should be well estimated what you can use where best. Personally, I like to rely on jobs when it comes to remote querying / remote invocation.

There is a sensational blogpost by Harry Eagles bout the topic:

https://blogs.technet.microsoft.com/uktechnet/2016/06/20/parallel-processing-with-powershell/

I would like to show a small example of how I use Jobs to read out Eventlogs about several machines. In the example, it is only checked if the corresponding log exists or was written in the last 45 minutes. For 128 servers I needed 30 minutes with this parallel Task. I killed the sequential script after 6 hours …

#by J.Kühnis
$Servers = @(
"serverHostname1"
"serverHostname2"
"serverHostname3"
)

Remove-Job *
$outputArray = @()

Foreach ($Server in $Servers){

Start-Job -Name $Server -ArgumentList $Server{
    param($servername)
    IF(Get-EventLog -LogName System -InstanceId '12306' -After (Get-Date).AddMinutes(-45) -ComputerName $servername -ErrorAction Ignore){
    
    Write-Output"$servername  true"
    }Else{
    Write-Output "$servername  false"
    
    }
} 

}

While (Get-Job -State "Running") {
    Clear-Host
    write-host "Jobs Running"  (Get-Job).count
    Start-Sleep 2
}
Clear-Host
Get-Job
write-host "Jobs completed, getting output"

Get-Job | ForEach-Object {
    $a = Receive-Job $_.Id
    $a
    $outputArray += $a
  
}

#Use the variable $outputArray to get or export the Outputdata
Continue reading...