Measuring Configuration Compliance: CIS Benchmarks and Drift Tracking

Security settings can change during maintenance and troubleshooting. Record which setting changed when, who approved the exception, and how to return to the expected setting.
Technical summary
Technology
CIS Benchmarks, Microsoft Security Baseline, Security Compliance Toolkit, GPO, Intune, OpenSCAP, PowerShell
Topic
Applying a security configuration baseline and measuring drift
Level
Intermediate–advanced
Prerequisites
  • A server and client inventory
  • GPO or Intune management access
  • Knowledge of PowerShell and basic Linux administration
In this article7 sections

A server hardened on installation day isn’t the same server six months later. SMB signing turned off during troubleshooting, NTLMv1 enabled for an application install, a local administrator account added as “temporary”. None of them get reverted. Saying “we have a hardening standard” in an audit is easy; the hard part is showing in numbers how many servers comply with that standard today. This guide explains for system and security teams how to derive a security baseline from CIS Benchmarks or the Microsoft Security Baseline, how to apply it, and how to measure drift regularly.

What a security baseline is

A security baseline is an approved, versioned set of security settings for a specific platform and role: password and lockout policy, audit policy, user rights, service states, protocol settings (SMBv1, NTLM, TLS), remote access settings and the like. It should have three properties:

  • Its scope is clear: It’s defined per platform and role, such as “Windows Server 2022 member server”, “Windows 11 enterprise client” or “RHEL 9 web server”.
  • It’s versioned: Which source and version it was derived from and on what date it was approved are recorded.
  • It’s measurable: The expected value of each setting is written in a form a machine can check.

A baseline isn’t a document applied once and shelved; it’s the reference point of the implementation and measurement cycle in an enterprise cybersecurity program. This article covers the device-level part of the cycle. Prioritizing the security program at the organizational level is covered in the CIS Controls implementation groups article.

CIS Benchmarks and the Microsoft Security Baseline

There are two common sources in Windows environments. The goal is the same; the approach differs:

Feature CIS Benchmark Microsoft Security Baseline
Publisher Center for Internet Security, community consensus Microsoft
Platform coverage Operating systems, network devices, databases, cloud, applications Windows, Windows Server, Microsoft 365 Apps, Edge
Format PDF with rationale, audit and remediation steps GPO backups, reports (Security Compliance Toolkit)
Strictness Level 1 and level 2 profiles A single profile, low application compatibility risk
Measurement tool CIS-CAT (Lite/Pro), third-party tools Policy Analyzer, Intune and Defender reports

Microsoft security baselines are published per version and device role; because some settings can affect application behavior, they should be tried in a pilot. CIS Benchmarks are more comprehensive; they give a rationale and impact description for each recommendation and offer the same structure for platforms beyond Windows. A common route in mixed environments: take the Microsoft baseline as the basis on Windows, assess CIS’s additional recommendations one by one and add them, and use CIS Benchmarks directly for Linux and network devices. Whichever is chosen, the source version the organization’s baseline was derived from should be written down. A baseline of unknown origin can’t be updated.

Choosing level 1 or level 2

In CIS Benchmarks, recommendations are divided into profiles. Level 1 consists of basic settings that can be applied without noticeably breaking functionality. Level 2 includes additional settings for environments requiring high security and may restrict some functions. Windows benchmarks also have additional profiles such as BitLocker and Next Generation Windows Security.

Practical rules for choosing:

  • The target on all servers and clients is level 1; settings where level 1 can’t be met are recorded as exceptions.
  • Level 2 is assessed for high-value assets such as domain controllers, management servers, backup infrastructure, internet-facing servers and systems processing sensitive data.
  • Level 2 settings aren’t put into production without being tested one by one in a pilot group; some settings break application compatibility. An accounts department that can’t print on Monday morning shows this test was skipped.

Implementation methods: GPO, Intune, configuration management

Applying the baseline by hand doesn’t scale and is drift itself. The method is chosen according to the device’s management model:

Method Suitable environment Drift behavior
Group Policy (GPO) Domain-joined Windows servers and clients Security settings are reapplied every 16 hours even if the GPO doesn’t change
Intune security baseline and settings catalog Clients joined to Entra ID or co-managed Device compliance and profile status are reported
Ansible, DSC, etc. Linux servers, mixed environments The expected state is reapplied on regular runs
Golden image New installations Doesn’t prevent post-installation drift; insufficient on its own

The Microsoft Security Compliance Toolkit provides baselines as GPO backups; they can be imported directly. But don’t link the baseline GPO as-is to the domain root. Layering the organization’s differences in a separate GPO and linking them to role-based OUs makes comparison easier when the baseline version is updated, and reduces rollback to removing a single link. For topics that are part of the baseline but managed separately, such as local administrator passwords, see the Windows LAPS migration article.

Drift measurement and tool comparison

Drift measurement means comparing a device’s actual configuration with the baseline and listing non-compliant settings. Options vary with scale and budget:

  • CIS-CAT Lite / Pro: Produces scores and setting-level reports against CIS Benchmarks. The Lite version supports a limited number of benchmarks; the Pro version comes with CIS SecureSuite membership and provides bulk scanning and reporting.
  • Policy Analyzer: Part of the Security Compliance Toolkit; it compares GPOs with each other and with the effective local policy. It’s useful in GPO design and for baseline version differences; it isn’t designed for fleet measurement.
  • Microsoft Defender Vulnerability Management: Its security baselines assessment feature continuously assesses devices with the agent against CIS or STIG profiles (check the license requirement).
  • OpenSCAP: On Linux, it assesses against CIS profiles with SCAP Security Guide content and produces an HTML report.
  • Compliance modules of vulnerability scanners: Most scanners that do credentialed scanning also run configuration audit policies. Using existing scanning infrastructure is cheaper than deploying a new agent.

An example assessment against a CIS profile with OpenSCAP on RHEL 9:

sudo dnf install -y openscap-scanner scap-security-guide
oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml | grep -i cis
sudo oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
  --results /var/tmp/cis-l1-results.xml \
  --report /var/tmp/cis-l1-report.html \
  /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

In small environments without a commercial tool, or for a quick check of critical settings, a targeted audit can be written in PowerShell. The example below checks a few registry-based settings from the organization’s baseline and writes the result to CSV:

$expected = @(
  @{Name='LM compatibility level'; Path='HKLM:\SYSTEM\CurrentControlSet\Control\Lsa'; Value='LmCompatibilityLevel'; Expected=5},
  @{Name='LM hash storage off'; Path='HKLM:\SYSTEM\CurrentControlSet\Control\Lsa'; Value='NoLMHash'; Expected=1},
  @{Name='WDigest clear text off'; Path='HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest'; Value='UseLogonCredential'; Expected=0},
  @{Name='SMB server signing required'; Path='HKLM:\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters'; Value='RequireSecuritySignature'; Expected=1}
)
$result = foreach ($e in $expected) {
  $actual = (Get-ItemProperty -Path $e.Path -Name $e.Value -ErrorAction SilentlyContinue).($e.Value)
  [pscustomobject]@{
    Computer  = $env:COMPUTERNAME
    Setting   = $e.Name
    Expected  = $e.Expected
    Actual    = $actual
    Compliant = ($actual -eq $e.Expected)
  }
}
$result | Export-Csv -Path ".\baseline_$env:COMPUTERNAME.csv" -NoTypeInformation -Encoding UTF8

If the value isn’t defined (empty), this script doesn’t count the default behavior as compliant; for some settings the operating system default is already secure, so note this in the baseline. The script doesn’t replace a tool, but it’s enough for a quick measurement of selected critical settings across the whole fleet; it can be run on remote servers with Invoke-Command. Servers with WinRM turned off don’t appear in the result list at all. Count them separately.

Three metrics that make the measurement meaningful:

  • Per-device compliance: How many of the baseline settings are met.
  • Per-setting compliance: On how much of the fleet is a given setting non-compliant? Widespread non-compliance usually points to a GPO scope error, a wrong WMI filter or the need for an organization-wide exception.
  • Coverage: How many devices in the inventory could be measured; a device that can’t be measured doesn’t count as compliant.

Measurement should be done at least monthly and the results presented in the same reporting cycle as the vulnerability management program. Misconfiguration is a vulnerability too. An attacker uses an open SMBv1 as easily as unpatched software.

Recording and reviewing exceptions

There will be settings that depart from the baseline. What matters is that the departure is deliberate and recorded. Unrecorded non-compliance is drift; recorded and approved non-compliance is an exception. Each exception record contains these fields:

  • The setting and the affected device or group.
  • Justification: because of which application or business need.
  • Compensating control: for example, the server where the setting can’t be applied has been moved to a separate segment and its access restricted.
  • Approver and end date; open-ended exceptions aren’t granted.
  • Removal condition: a new version of the application, decommissioning the system, etc.

The measurement tool should recognize exceptions, or the report should be merged with the exception list. Otherwise, the same known non-compliances fill the report every month, the team stops reading it and new drift goes unnoticed. Exceptions are reviewed quarterly; expired ones are either renewed or closed.

Preparing and applying the baseline can be handled within our system and server hardening service, and independent measurement of existing drift within our control and configuration audit service. Benchmarks can be downloaded from the CIS Benchmarks page, and Microsoft baselines and Policy Analyzer through the Microsoft Security Compliance Toolkit.

Frequently asked questions

Should I use CIS Benchmarks or the Microsoft Security Baseline?

In a Windows-heavy environment, the Microsoft baseline is a starting point with low compatibility risk. If there are different platforms such as Linux, network devices and databases, or if a rationale and impact description is needed for each setting, CIS Benchmarks are more comprehensive. Many organizations take the Microsoft baseline as the basis on Windows and add CIS recommendations after assessing them.

Is it right to apply level 2 settings to all servers?

Usually not. Level 2 can restrict some functions; it’s assessed for high-value assets such as domain controllers, management and backup servers, and tested in a pilot group first. The target for the general fleet is level 1.

If I’ve applied a GPO, do I still need to measure drift?

Yes. Devices outside the GPO scope, machines where the policy couldn’t be applied, settings not managed by GPO, and changes made with local administrator rights only show up with an independent measurement. Measurement is the evidence that the implementation really works across the whole scope.

When can a non-compliant setting be accepted as an exception?

When the setting breaks a business function, the reason is written down, a compensating control is applied and the exception is approved for a limited time. Unrecorded non-compliance isn’t an exception; it’s drift, and it gets fixed.

Support

Contact us for support or a discovery session on this topic.

Contact form
[email protected]

Eretek Cybersecurity

Firewall, VPN, endpoint security and threat prevention. We reduce the attack surface and measure and report the risks.

Knowledge Center

Other articles