Automating Audit Collection Services in Workgroups with PowerShell

The Case

We all have the same experience with ACS Forwarders, known also as ACS Agents. While enabling ACS on the agent isn’t that hard to master, as you just have to run task inside OpsMgr console on the Health Service object, the pain starts when you need to start working with non-domain or non-trusted domain environments.

Whole point is that nearly all tasks have to be done manually (well, that’s what official documentation says). What we will try to achieve is to automate whole process which was never possible.

The Process

The process is very clearly documented in https://blogs.technet.microsoft.com/udishmudiar/2017/12/25/scom-cross-domain-acs-collector-and-forwarder-configuration/ blog article by Udish. There are several steps which I won’t be covering at all as the process is often different in each company like certificate creation or agent installation process. I would like to explicitly cover three steps which are often done manually and this makes it very boring and overwhelming, especially when a company has a relatively short certificate lifetime.

Process 1 – Computer Account Creation

When you gather security logs from a non-domain or cross-untrusted domain server, your first point is creating a dummy computer account inside domain, where ACS Collector resides. As this is very straightforward. Your friend is New-ADComputer.

New-ADComputer -Name "USER02-SRV2" -SamAccountName "USER02-SRV2" -Path "OU=ApplicationServers,OU=ComputerAccounts,OU=Managed,DC=USER02,DC=COM"

Process 2 – Importing your computer certificate to computer object

The second part of this process is something, which manually is done by selecting Name Mappings on computer account and from there you are choosing a certificate to be imported. Now what does this exactly do? It just creates a special entry in altsecurityidentities property of computer object. And it in reality doesn’t do anything with this certificate except retrieving its Common Name and Issuer. That’s relatively easy to achieve. You can put your certificate in a specific location and take its properties, modify their format to be able to import it into AD property field and we’re done. Because the field takes a bit strange format (X509:<I>DC=com,DC=contoso,OU= IssuingCAs,CN=IssuingCA123<S>DC=com,DC=contoso,CN=USER02-SRV2) we need to somehow parse the fields to reverse order of path values.

$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate "$file" #Just before declare $file as a path to your CER file
$comp = Get-ADComputer -Filter {Name -Like $server} -Server $Domain #Also declare $server and $domain to query for computer object
$paths = [Regex]::Replace($cert.Issuer, ',\s*(CN=|OU=|O=|DC=|C=)', '!$1') -split "!"
$issuer = ""
# Reverse the path and save as $issuer
for ($i = $paths.count -1; $i -ge 0; $i--) {
$issuer += $paths[$i]
if ($i -ne 0) {
$issuer += ","
}
}


# Do the same things for $cert.subject
$paths = [Regex]::Replace($cert.subject, ',\s*(CN=|OU=|O=|DC=|C=)', '!$1') -split "!"
$subject = ""
# Reverse the path and save as $subject
for ($i = $paths.count -1; $i -ge 0; $i--) {
$subject += $paths[$i]
if ($i -ne 0) {
$subject += ","
}
}

 # Format as needed for altSecurityIdentities
$newcert = "X509:<I>$issuer<S>$subject"

# Insert data
try {
$comp | Set-ADComputer -Add @{'altsecurityidentities'=$newcert}
}
catch{
Write-Output "Cannot write value to altsecurityidentities. Check permissions on domain level for your user. Failed server = $server"
}

Now having it done we can go to the most tricky part which is…

Process 3 – Importing certificate to ACS Forwarder

We all know that the one and only way is to fire adtagent -c, choose one and only certificate and then launch task from OpsMgr Console. Well… up to now 😉

I assumed in my process that OpsMgr Agent (MMA Agent) is already up and running also with addition of certificate, so MomCertImport has already been run and agent is green in console. That gives us a big step forward as we have data already. Let’s start our work.

Firstly we need to retrieve the certificate we have to use:

$cert = (Get-ChildItem Cert:\LocalMachine\My | where { $_.Thumbprint -eq (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Machine Settings' -Name ChannelCertificateHash).ChannelCertificateHsdash })

From now on we have our $cert object filled with certificate, which has the same Thumbprint as the one imported into OpsMgr Agent configuration in registry. From this certificate we have to retrieve the thumbprint as this value is later stored in AdtAgent service registry

$certdata = $($cert.Thumbprint)

The certificate’s thumbprint as a string is now stored in $certdata variable… but the value inside registry key is stored in… Binary value. So again, converting is our goal.

$newdata = for ($i = 0; $i -lt $certdata.Length; $i += 2) { [convert]::ToByte($certdata.Substring($i,2), 16) }
# Setting the value inside ADTAGENT service registry
Set-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\AdtAgent\Parameters' -Name CertHash -Value $newdata -Type Binary

Now that step would be enough if the certificate would be imported into AdtAgent for the second or latter time. If it is for the first time, the account on which AdtAgent runs (normally Network Service) has to have access to decrypt data with this certificate and this is done by access to RSA files hidden inside ProgramData folder.

Firstly we will create proper ACL

# Static path for RSA Keys in Windows
$fileRSAlocation = 'C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\'
# Getting Network Service SID Object
$principal = New-Object System.Security.Principal.SecurityIdentifier([System.Security.Principal.WellKnownSidType]::NetworkServiceSid, $null)
# Rule for ACL to be added to key file
$right = 'Read'
$rule = New-Object System.Security.AccessControl.FileSystemAccessRule($principal, $right, 'Allow')

Now we need to find the proper file for this certificate

$certRSAStore = $($cert.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName)

And finally apply ACS and restart agent (if already configured) or you can skip restart and launch Enable Audit Collection Services task in OpsMgr Console.

$path = $fileRSAlocation + $certRSAStore
$acl = Get-Acl -Path $path
$acl.SetAccessRule($rule)
Set-Acl -Path $path -AclObject $acl

#Restarting ACS Agent (adtagent) [Optional]

Restart-Service adtagent

 

Hope this guide will help you somehow automating your ACS deployments 🙂

You can find full scripts here: https://github.com/rem8/psscripts

 

Advertisements

OMS – Change Tracking

Since early days, when System Center Advisor (*RIP*) was introduced as the first cloud enhancement for SCOM… or maybe GSM was first… never mind… it’s main focus was on tracking changes and reporting when it was changed. That evolved, been revamped, tuned, tweaked, few things thrown away, some added.

Right now Operations Management Suite has two main (and only) fields it works on regarding configuration changes. These are:

  • Service changes (not Status, but Startup Type)
  • Application installations

change1

It’s good to have it – it gives you at a glance what has been installed. I’m a gamer, I love to play Football Manager. That means I needed to install on my private laptop Steam Client to download the game. As my laptop is connected to the OMS via Microsoft Monitoring Agent, it gathered the information and shown as below:

change2

I can click “See all…” to also show my Steam client.

change3.PNG

Because Steam is also a Service, I can see on Windows Service Changes pane on previous picture, that something happened. If I click through it I can see that there’s a new service on my PC.

change4.PNG

And that’s all. I can also group, time slice, drill down etc. But that’s not the point. It’s not much we can do. BUT! Let’s assume Microsoft OMS Team is listening to customers… We’re not in a dream – they are. They started to listen to customers few years ago when Azure was starting to fight it’s way to the top. This may lead us to the point, where filling out surveys sent by OMS team to us will be an inevitable mark of what should be done next. From the survey we can read that future my bring us more investments in change tracking solution in OMS. There are multiple fields we can choose from by prioritizing sources of change. Those are:

  • Registry Values (like HKLM changes)
  • File changes (to choose which files need to be monitored, ie. web.config)
  • Active Directory
  • Group Policy
  • Windows Firewall
  • Azure IaaS VM
  • Infrastructure Apps (Exchange, SQL, Sharepoint)
You can also point out which other sources can be nice to have. I added Azure RM Security Groups. You can have your own ones.
Survey is available at “provide your feedback” link. I haven’t been added to chat panel yet, although at every survey I point out that would be interested. Oh well, maybe someday 🙂
OMS: http://oms.microsoft.com – start your own FREE workspace – 500 MB/day for 7 days of retention!

SquaredUp 2.3 summary

So again hello. As always hadn’t much time to write anything but since there’s some free time, let’s check what’s there out on the market. SquaredUp 2.3 dashboard solution is out there for a while. As for today it’s 2.3.7 with some major and minor fixes. The most important solution (and now a stable one) is Open Access dashboards. The SqUp page sums it as…

Open Access dashboards allow you to make non-interactive dashboards available to all of your users, both SCOM and non-SCOM, with a single-click and no additional set-up or configuration.

The whole point of that is:

  • This solution doesn’t “eat” your license. You can have as many OA dashboards as you like.
  • This solution doesn’t require SCOM access for users
  • This solution is based on rendering static picture out of prepared dashboards in the background. It’s refreshed every xx seconds and rendered back so it’s a pretty fine solution for your widescreen.

There are many scripts you can use later on to get for example 10 links for such dashboards and rotate them every 10 seconds or so on separate page. You can also use add-ons like https://addons.mozilla.org/de/firefox/addon/tab-slideshow/ for FireFox which will cycle through open tabs.

https://download.squaredup.com/downloads/download-info/squared-2-3/

 

 

OM 2016 and Nano Server

Looking at the latest build of System Center Operations Manager 2016 Technical Preview 4 (available for download at MSDN), I’ve noticed that there a small new folder in the installation media: “C:\SC TECHNICAL PREVIEW 4 SCOM\NanoAgent”

nano.PNG

This is the special agent for Nano Server available for latest SCOM release. Because of special architecture of Nano Server, this agent is installed via PowerShell. Whole information is residing in NanoServer folder:

nano2

  • BinaryFileList – list of all files and folders that needs to be copied into specific folder in Nano Server. All files and folders are in the folder above (Pic. 1)
  • InstallNanoServerScomAgentOnline (and Uninstall…) – PS scripts to automatically install agent.
  • msvcp120 and msvcr120 dlls – binary libs
  • StaticRegistryEntry and VariableRegistryEntry – reg files for putting registry items into Nano Server

Let’s stop at the PowerShell first. The script is being launched with four parameters: Management Server FQDN, Management Group Name, Nano Server FQDN and Binary Folder

The script does 6 basic steps:

#1. Adding a rule to firewall
#2. Agent not installed previously
#3. Registry changes done for MG
#4. Folders created in Nano server
#5. Registry changes for Nano Agent in Nano server
#6. Performance counters installation for MomConnector and HealthService

So it’s more or less the same as standard agent installation. This script is being run remotely which is reflected in one the lines:

$NanoServerSystemDriveLetter = (Invoke-Command {$env:SystemDrive} -ComputerName $NanoServerFQDN)[0]

There are lots of scripts later on which are adding necessary information into Nano Server like registry entries:

$SubKeyName = “SYSTEM\CurrentControlSet\Services\HealthService\Parameters\Management Groups\$ManagementGroupName

$ManagementServerId = Invoke-CimMethod -ComputerName $ManagementServerFQDN -ClassName StdRegProv -MethodName GetStringValue -Arguments @{sSubKeyName=$SubKeyName;sValueName=“ID”} -ErrorAction Stop

… or copy items

Copy-Item $BinaryFolder\$BinaryFile $InstallLocation\$DestinationSubDirectory -Force

… or uploading files for later use

Copy-Item $BinaryFolder\NanoServer\StaticRegistryEntry.reg $InstallLocation\StaticRegistryEntry.reg -Force -ErrorAction Stop

… or creating a service

sc.exe create HealthService binPath= $InstallLocationOnNano\HealthService.exe” type= share start= auto depend= rpcss DisplayName= “@$InstallLocationOnNano\HealthService.dll,-10500″

… or adding performance counters

lodctr $env:SystemDrive\Program Files\Microsoft Monitoring Agent\Agent\HealthServiceCounters.ini”

 

So to install the agent on Nano Server, we can get some help from the manifest inside the script:

 

<#

.SYNOPSIS

Uninstalls the SCOM Nano Agent from the Nano Server and unregisters it from the Management Server

.EXAMPLE

InstallNanoServerSCOMAgentOnline.ps1 -ManagementServerFQDN MySCOMManagementServerName -ManagementGroupName MySCOMManagementGroupName -NanoServerFQDN MyNanoServerFQDN -BinaryFolder C:\MyNanoDrop\amd64\

.DESCRIPTION

This script installs the SCOM Nano Agent TP4 version on the given Nano Server machine. This script can run from both the management server and the

nano server. This script has to be run with administrative privileges. The user account which is used to run this script must also have administrative rights on the

Nano Server (if running remotely). Also make sure that the SCOM powershell module is imported before running the script.

#>

 

So this gives you an idea of how Nano Server will be managed in future. Hopefully that will be integrated into Discovered Inventory to allow one pane of glass for agent installation.

 

SquaredUp 2.2 out now!

SquaredUp Portal is used in our company for several months. And we are very pleased with it. Especially since, in the version 2.2 saw the light of day. There are a few elements that are a major factor in the use of this, and not another console. They are: speed, transparency and heterogeneity. We do not want to deprive the ability to see the console on Chrome…  The aim of our company is to give the opportunity to all view of their environment and we have succeeded. But why I write about version 2.2? Two elements in our opinion were problematic and caused problems with the service console.

First one was the inability to see the server name of an object which didn’t have unique name in environment. That was for example disk C:. We wanted a dashboard with state of all disks there. Problem? All disks were just C’s and D’s and You had to click inside to see what was the path underneath.

Secondly we had few Operator roles and those roles were unable to use Maintenance Mode feature because of small bug inside the code. Whenever an Operator tried to launch MM on any object, pop-up appeared that you cannot put MM onto Management Server… On well, Operator had to use standard console for that. But now the problems are GONE!

 

The version 2.2 of SquaredUp Portal brings great enhancements, which for me are really good ones and are the right path SquaredUp team has going. From the http://www.squaredup.com:

  • new – Page-level scope and Clone allow the rapid creation of new dashboards
  • new – Grouping and sorting options for status sections, including label customization
  • new – Groups and distributed applications can be put into maintenance mode (requires page level scope to be set)
  • new – Support for loading extended properties (advanced)
  • new – In-line confirmation for delete and discard actions
  • fixed – Dashboard import did not work as expected in Internet Explorer 9
  • fixed – “Services delivered by this object” was failing due to a locking issue
  • fixed – Favorite metric stars under component view were disappearing when clicked
  • fixed – Operators were unable to put servers into maintenance mode
  • fixed – ‘aggregate’ and ‘top N’ options were not automatically re-selected when reconfiguring a performance section
  • change – The installer no longer prevents installation on Windows Server 2016 TP3 / IIS10

The first one is a great addition which you can see here (credits to SquaredUp):

 

This is really powerfull possibility to fully scope your dashboard when you are creating one view per group or distributed application. Again, this is a great stuff from SqUp team and a massive improvement in flexibility and usability. Moreover, the bugs and tweaks have been corrected and now we can fully go forward with great dashboards from SquaredUp. For me, SquaredUp is a one big step forwards towards Operations Manager management.

 

More info here: https://download.squaredup.com/downloads/download-info/squared-up-2-2/

 

 

 

 

Cluster not appearing in Agentless Managed

Ode to SCOM

It was a pain and a disaster
When you not find yourself a cluster
You search a Virtual Cluster class
And what you found is an empty glass

You know you have agents and a proxy there is
But you hear from behind your co-admin fizz:
“Where is my SQL, where is my node?”
“We thought that the SCOM is an informational lode!”

So I dig on the web and I search for a year
Until on my face all saw a drifting white tear
And we found a solution which might be a bit strange
A cluster co-admin made an unsupported change

There was some resources which were ages offline
But on MMC Snap-In everything was quite fine
So we run powershell and we search for those ones
And looking at output we found them at a glance

So there are some offline which are not in UI
So we run powershell and we told them goodbye
As we closed PS window and we switch once to SCOM
We saw some new objects and knew where they come from

And there you have it – my old admin friend
Every SCOM problem have come to an end
“The solution is gone” you might think at the start
At the end it will look like a mosquito fart.

So basically – when your cluster doesn’t appear in agentless managed nor as a Virtual Server class object, check you cluster and search from powershell if there are any resources missing, which are offline.

Get-ClusterResource | where {$_.state -eq 'offline'}

If so, remove them, for example:

Remove-ClusterResource "Cluster Disk 4"

Maintenance Mode Scheduling revealed!

Which feature was the most expected in Operations Manager? Maintenance Mode Scheduling. Which feature was given for us in SCOM Technical Preview 2? Maintenance Mode Scheduling. I thought WOW, this must something the whole admin population must be thrilling about. And as I saw on twitter – it was. So let’s see what’s all the fuss about.

After installation of Technical Preview 2 from VHD file available from my previous post I was given a nice little pop-up as always.

scomtp1   scomtp2

After launching, nothing really has changed in the main monitoring view. So my first click was an Administration panel in SCOM wunderbar (bottom left pane). So except Operational Insights (which will be described in later posts) my sight was already there – Maintenance Schedules entry.

scomtp3

Clicking it revealed a new pane for Maintenance Mode Scheduling.

scomtp4

Nothing else to do but start playing with it – thought. So let’s try doing the first schedule. Click on the right pane to Create Maintenance Schedule.

scomtp5

Because I have only one server (the MS itself) I had to choose something, that will not affect the total health of it. After thinking for a while I’ve chosen Operating System class and all objects underneath.

scomtp6

Accepting this moved us to second pane – scheduling. This is the best part as you can schedule a maintenance once, daily, weekly, monthly, set start and end date, length and expiry date of the schedule!

scomtp8

Lastly, the details of maintenance. So let’s give it a name and reason then…

scomtp9

After quick run through the wizard, I was given a proper entry in the Maintenance Schedules pane.

scomtp10

…and after few minutes…

scomtp11

… both MM was enabled and the object was properly put in MM too.

scomtp12

CONCLUSION: IT WORKS!

Backyard

It’s all there in the database. If you like to play with it, you can drill down into OperationsManager database and what will be found is:

  • VIEW: dbo.MaintenanceModeView
  • TABLES: dbo.MaintenanceMode, dbo.MaintenanceModeSchedule and dbo.MaintenanceModeHistory

Querying those give you good insight on what’s going on and what happened in the past:

scomtp16

QUERYING VIEW

scomtp14

QUERYING SCHEDULE TABLES

scomtp17

QUERYING HISTORY TABLES

scomtp18

QUERYING MAIN TABLE

scomtp15

As you see, there’s a ScheduleId column (which is not added in the view) which goes exactly to MaintenanceModeSchedule table. You can link both tables to get more info in one result set.

Whole work (adding, deleting, launching, editing etc.) is being done by stored procedures, which are added to handle schedules.

scomtp19

Happy scheduling 🙂

System Center TP 2 (VHD) do pobrania

Krótka notka – pobieramy 🙂

https://www.microsoft.com/en-us/download/details.aspx?id=46912

Z rozwiniętego linku Details można pobierać także pozostałe składniki SC TP 2.

Operations Manager Technical Preview 2 – Co nowego?

Pomimo, że nie ma jeszcze możliwości pobrania, to dość ważna informacja wypłynęła na witrynie https://technet.microsoft.com/en-US/library/dn997273.aspx. Główną nowością jest możliwość planowania trybu serwisowego (Maintenance Mode). Nowy kreator trybu serwisowego będzie miał możliwość wyboru typu klasy obiektu do włączenia w tryb serwisowy oraz planowania jego wykonywania (jednorazowo, codziennie, raz na tydzień, raz na miesiąc). Oprócz tego będzie osobny panel do przeglądania wszystkich harmonogramów trybu serwisowego oraz możliwość włączania dla jednej encji różnych harmonogramów.

Mogę tylko powiedzieć – nareszcie!

Update Rollup 6 dla System Center 2012 R2

Trochę poprawek się pojawiło w najnowszym UR6, a zatem co się zmieniło/poprawiło:

W Operations Manager zmiany kosmetyczne – głównie naprawa timeout dla polecenia Remove-SCOMDisabledClassInstance, naprawa widgetów i wyświetlania danych w konsoli Web. (https://support.microsoft.com/en-us/kb/3051169/)

Więcej ciekawostek jest dla Service Managera – włączono support dla SQL 2014 jako platformy bazodanowej, a więc już robić upgrade (nie czystą instalację) swoich platform SQL do 2014 (zarówno jako in-place upgrade, jak i przeniesienie bazy oficjalną procedurą). Ważne jest też (w końcu) umożliwienie konfiguracji konektora Active Directory i czasu jego synchronizacji z konsoli UI oraz PowerShell. Do tego ponad 15 dodatkowych poprawek do procedur i mechanizmów samego SMa (https://support.microsoft.com/en-us/kb/3039363/)

Data Protection Manager także uzyskał możliwość migracji do SQL Server 2014. Do tego 15 fixów do obsługi backupu online, klastrów oraz Hyper-V, a także wywalania się samego agenta DPMa i konsoli. (https://support.microsoft.com/en-us/kb/3030574/)

SMA to głównie krytyczny fix, który umożliwiał na portalu podejrzenie hasła do konta maszyny, gdyż było przesyłane ono clear-textem! (https://support.microsoft.com/en-us/kb/3050307/)

VMM to duża ilość fixów oraz trochę dodatków:

  • Dodanie opcji  Add Azure Subscription
  • Ulepszony scenariusz ochrony E2A ASR
  • Możliwość użycia Generation 2 VMs w Usługach (Services) oraz VMRoles
  • Total Networking Usage Exposure w Management Packu do VMMa
  • Opcja przeciążenia Cloud i Host Group dla maszyn repliki (obecnie tylko główny site miał te opcje) (https://support2.microsoft.com/kb/3050317)

Na koniec Azure Pack i wsparcie dla wykonywania zdalnych skryptów w postaci schedule lub on demand, Site Slots oraz HttpPlatformHandler np. do obsługi Javy oraz masa poprawek związanych z certyfikatami (takze SSLv3), co wiąże się także z dodaniem dodatkowych portów do obsługi IP-based SSL. (https://support.microsoft.com/en-us/kb/3051142/)

Miłego pobierania i instalacji.

%d bloggers like this: