Microsoft 365 Security as Code — Part 2: One benchmark, four tools

This is part 2 of a series on managing Microsoft 365 security as code, based on the CIS Microsoft 365 Foundations Benchmark. Each part can be read on its own. Part 1: Before you write any code · Part 2: One benchmark, four tools (this post) · Part 3: Running Microsoft365DSC on Linux

The CIS Microsoft 365 benchmark tells you what to set and how to check it. It does not tell you how to automate it, and that is where the work is. The settings live in different places: some are in Microsoft Graph, some only in Exchange Online PowerShell, some in a portal that has no API at all. No single tool reaches all of them. This post is about how I choose a tool for each control, what I do when no tool fits, and why the result is several repositories, each with its own identity, and not one big one.

TLDR: I ask four questions in a fixed order. Is there a Terraform provider that reaches it? Is there a Microsoft365DSC resource? Can a small script do it safely? If none of these, is a person going to do it, and where do I record that? The repositories are split mainly to limit the damage if one pipeline runner is compromised.

The four tools

1. Terraform, when Microsoft Graph has an endpoint for it. Our standard for infrastructure is Terraform, so this is the first question. The azuread provider covers Entra ID: Conditional Access policies, groups, service principals and so on. The msgraph provider can call any Graph endpoint, including beta ones, which reaches many admin center settings that have no dedicated resource. This is what it looks like for two Level 1 controls in the admin center section:

# 1.3.4 (Level 1): restrict User owned apps and services (Office Store + trials).
resource "msgraph_update_resource" "apps_and_services_settings" {
  url         = "admin/appsAndServices"
  api_version = "beta"
  body = {
    settings = {
      isOfficeStoreEnabled         = false
      isAppAndServicesTrialEnabled = false
    }
  }
}

# 1.3.5 (Level 1): Microsoft Forms internal phishing protection.
resource "msgraph_update_resource" "forms_settings" {
  url         = "admin/forms"
  api_version = "beta"
  body = {
    settings = {
      isInOrgFormsPhishingScanEnabled = true
    }
  }
}

Terraform gives me what I want: a plan before every change, state, review in a merge request, and drift that shows up as a plan that is not empty. If a control can be written like this, it is always the first choice.

2. Microsoft365DSC, for Exchange Online, SharePoint and Teams. There is no mature Terraform provider for most Exchange Online objects: anti-spam and anti-phishing policies, transport rules, DKIM signing, mailbox auditing and so on. The community providers for Microsoft 365 wrap Graph, and these settings live in Exchange Online PowerShell. Microsoft365DSC is the tool that covers this surface. It is built for Windows, which is why part 3 exists.

3. A small script, for the gap that is left. Sometimes the tool I picked does not expose one setting. Then I write a short script, and I keep it as small and as safe as I can. I have a few of these. One sets who is allowed to share externally in SharePoint (control 7.2.8): when I wrote the script, there was no DSC resource for it at all. Another handles Defender for Cloud Apps, where Microsoft publishes no policy schema or policy API, so the policies are created once in the portal, exported as files, committed, and checked from then on.

4. Maester and people, for the rest. Some controls cannot be automated. In my reading of the benchmark, 17 recommendations have no automated audit at all, because they are a process or a human decision (for example reviewing who holds privileged roles). For those the right output is not code. It is a named owner, a review date and a record that the review happened. Maester, an open source test framework for Microsoft 365, covers the assessment side: it reads the tenant and reports what it finds. I come back to this in part 8.

The order matters. Each step down costs more to maintain than the one above it, so a control should land on the first tool that can really do it, not on the tool I like most.

Example: a setting that no tool exposes

Every accepted domain in Exchange Online has a setting that decides whether mail can be sent from that domain. People think about it as "allow sending". In PowerShell the property is SendingFromDomainDisabled, and the meaning is the opposite: $true means sending is switched off. This is easy to get wrong when you read a table that says "Allow sending: No" and write $false.

First, a note to keep two things apart. This setting is not the benchmark's Direct Send control. That one is control 6.5.5, an organisation-wide setting (RejectDirectSend), and it is set in the DSC configuration. The per-domain setting is an extra hardening step that I found useful on top of it, and it is not part of the benchmark.

The Microsoft365DSC resource for accepted domains, EXOAcceptedDomain, has parameters for the domain type, subdomain matching and outbound-only, plus the usual connection parameters. I checked the resource documentation, and it has no property for this setting. So DSC owns the domain type, and a script owns the sending flag.

The way I built the script is like this:

• A table, not a list in the code. The complete desired inventory of accepted domains is one table in a data file: domain, type, whether it is the default, whether sending is allowed. In the documentation of the repository this is a small table that a person can read and approve.
• Read everything before changing anything. The script reads all accepted domains and fails when one is missing, when a property cannot be read, or when the default domain is not the expected one. A script that guesses is worse than one that fails.
• Translate the meaning in one place. The inversion happens in exactly one line:

$actual = @{
    DomainType   = [string]$current.DomainType
    Default      = $current.Default
    AllowSending = -not $current.SendingFromDomainDisabled
}

• Change only what differs, then read it back. The apply step writes only the values that are different and reads them back to confirm.
• Report what it may not change. Exchange Online has no command to remove an accepted domain. So any domain in the tenant that is not in the table is reported as drift, and a person removes it in Entra ID.
• Test without a tenant. Offline tests for this logic run in the lint stage of the pipeline.

What I would add, and what I recommend for every script of this kind, is a note that says when it can be deleted, which is the day the DSC resource gets the property. Without it, a workaround quietly becomes "semi-permanent".

Why the code lives in several repositories

The repositories are split like this: one for Exchange Online (DSC), one for SharePoint, Teams and the admin center (Terraform and DSC), one for Entra ID tenant settings, one for Conditional Access, and one for the assessment with Maester. DNS records such as SPF and DMARC live in separate repositories, because they are DNS and not Exchange objects. The identities and their permissions are created in yet another repository.

The main reason for the split is blast radius. Each repository has its own pipeline runner, and each runner has its own managed identity with only the permissions its job needs. If one runner is compromised, the attacker holds the permissions of that one identity and not the permissions of everything.

There are eight runner identities in total, and each runner tag maps to exactly one identity. This is what a few of them hold, as it stands at the time of writing:

• Exchange Online. The Exchange application permission and nothing in Graph that can write. The Exchange roles it needs are assigned to the application by hand (more on why in part 3). It cannot change Conditional Access or Entra settings.
• Conditional Access. One write permission, for Conditional Access policies, and five read-only ones. No directory roles. It cannot change mail settings.
• Reporting. Read-only permissions in Graph. It can find problems but it cannot fix them.
• Entra ID tenant settings. Much wider: it can write to the directory, applications, groups and several policy areas.
• Role eligibility (PIM). It can write role management settings, which in principle lets this runner grant itself almost any role, so this one has extra protection around in in who can actually push code to it and merge to main. Additional monitoring with Sentinel can be configured when Gitlab is integrated with Sentinel.

That last pair is the honest part. Splitting reduces what one compromise can do, and it does not make every runner harmless. Some identities are still powerful, and a few of them still hold fixed admin roles. The plan is to replace those with roles that the pipeline activates only for the length of a run, and part 4 covers that together with the approval step and the other controls around the runners. The rule I follow for now is: one identity per job, and then write down which permissions are still strong and what protects them.

What the split costs

I will not pretend this is free.

• More to maintain. Eight identities means eight sets of permissions to keep correct, and several pipelines. All permissions are set in code though (duh) so we have traceability.
• Copied code. The parser that makes DSC work on Linux is copied into two repositories, and the other scripts have grown apart. There is no plan to merge them. Part 3 explains why.

The register is how I find these. It joins each control to the repository that implements it, an owner, the evidence from the assessment, and any exceptions. The rule it enforces is simple: one setting, one owner.

If you are starting from nothing

1. For every control, write down which of the four tools owns it before you write any code. A table is enough.
2. Choose the first tool in the order above that can really do it. Do not choose by habit.
3. Keep scripts for gaps small, make them read before they write, and write down when they can be removed.
4. Give each pipeline its own identity from the start. It is much harder to split one shared identity later.
5. Check for settings with two writers. They are the ones that break first.

In part 3 I go into the hardest of the four: running Microsoft365DSC on a platform it was not built for.

Get in touch

I help teams that need Microsoft 365 security to be written down as code, reviewed like code, and provable to an auditor, especially in financial services and other regulated environments. If you are choosing tools for the CIS benchmark, or you have a setting that none of your tools reaches, I would like to hear about it. Connect with me on LinkedIn, or reach out via ascode.nl.

This post is part of the series Microsoft 365 Security as Code, where you can find all the parts.