Stefan Stranger's Blog

Lessons learned while developing a custom Azure Policy


I learned something new on an issue while trying to create a Custom Azure Policy to audit if an Azure Event Hub has a Private Link configured.

Azure Private Link Service enables you to access Azure Services (for example, Azure Event Hubs, Azure Storage, and Azure Cosmos DB) and Azure hosted customer/partner services over a private endpoint in your virtual network.

A private endpoint is a network interface that connects you privately and securely to a service powered by Azure Private Link. The private endpoint uses a private IP address from your virtual network, effectively bringing the service into your virtual network.

Before discussing the encountered issue let’s first go through the steps I followed to develop this custom Azure Policy.

Azure Policy Development steps

I normally follow the following steps:

  1. Deploy Azure Resource with required configuration

  2. Search for Azure Built-in Policies as a starting point

  3. Search for Azure Event Hub aliases

  4. Create custom Policy Definition

  5. Deploy Policy Definition

  6. Assign Policy Definition

  7. Trigger on-demand Azure Policy compliance evaluation

Deploy Azure Event Hub

Just follow the steps documented on Microsoft Docs to deploy your Azure Event Hub. Make sure you select the standard pricing tier otherwise you won’t be able to create a Private endpoint connection later on.

You should now have deployed the Event Hub and as a next step you need to create the Private endpoint connection. Again there are multiple ways to do so just choose the option you are most comfortable with using the Microsoft Docs documentation.

You should now have something similar deployed as is being showed below.

Event Hub configuration screenshot

With a private endpoint configured in the Network configuration.

Private endpoint configuration screenshot

Search for Azure built-in Policies for Event Hub

Why reinvent the wheel if there might be some existing Event Hub Azure Policies? Start searching the built-in Azure Policies first, using the Azure Portal and with some filters.

Event Hub built-in Azure Event Hub Policies screenshot

None of the above Azure Policy Definitions seems to evaluate against a Private link connection for Event Hub.

Let’s change the search criteria a bit.

All Categories overview of built-in Policies for Private Endpoint

Open one of the above Policy Definitions to see if we can use this is a starting point for our custom Azure Policy to audit if a private endpoint is enabled for Azure Event Hub.

Private endpoint should be enabled for PostSQL Servers Policy Definition screenshot

The interesting part in this Policy Definition configuration is the existenceCondition.

The effect of the Policy is AuditIfNotExists. AuditIfNotExists enables auditing of resources related to the resource that matches the if condition, but don’t have the properties specified in the details of the then condition.

So in this example if the if condition Resource of type “Microsoft.DBforPostgreSQL/servers” is met we can audit resource related to that resource being a Private endpoint connection.

AuditIfNotExists runs after a Resource Provider has handled a create or update resource request and has returned a success status code. The audit occurs if there are no related resources or if the resources defined by ExistenceCondition don’t evaluate to true.

When a Resource type “Microsoft.DBforPostgreSQL/servers” is found in the AuditIfNotExists then condition, an existence condition is executed against Resource Type “Microsoft.DBforPostgreSQL/servers/privateEndpointConnections” for a field “Microsoft.DBforPostgreSQL/servers/privateEndpointConnections/privateLinkServiceConnectionState.status” with a value equal to “Approved”

In an flow chart this looks as follows.

Flow chart over of PostSQL Servers Policy Definition

This seems like a good starting point for creating the new custom Azure Policy.

Search for Event Hub Policy aliases

Aliases in resource policies enable you to restrict what values or conditions are permitted for a property on a resource.

Using Azure PowerShell we can search for Private Endpoint connection aliases for Eventhub:

# Install Graphical tools for easy searching
Install-Module Microsoft.PowerShell.GraphicalTools
Install-Module Microsoft.PowerShell.ConsoleGuiTools
# Retrieve Azure Policy aliases
Get-AzPolicyAlias -NamespaceMatch 'eventhub' | Out-ConsoleGridView -OutputMode Multiple | Select-Object -ExcludeProperty Aliases

Animated gif

Based on the output from above alias query we should be able to use the following alias:

“Microsoft.EventHub/namespaces/privateEndpointConnections/privateLinkServiceConnectionState.status” in our custom Private endpoint should be enabled for Event Hub Policy Definition.

Create custom Policy Definition

Follow the steps decribed at Microsoft Docs - Tutorial: Create a custom policy definition.

When you reached the compose the definition step use the following definition:

{
  "properties": {
    "displayName": "Private endpoint should be enabled for Event Hub",
    "policyType": "Custom",
    "mode": "All",
    "description": "Private endpoint connections enforce secure communication by enabling private connectivity to Azure Event Hub. Configure a private endpoint connection to enable access to traffic coming only from known networks and prevent access from all other IP addresses, including within Azure.",
    "parameters": {
      "effect": {
        "type": "String",
        "metadata": {
          "displayName": "Allowed Azure Policy Effect",
          "description": "The allowed effect for this Azure Policy"
        },
        "defaultValue": "AuditIfNotExists"
      }
    },
    "policyRule": {
      "if": {
        "field": "type",
        "equals": "Microsoft.EventHub/Namespaces"
      },
      "then": {
        "effect": "[parameters('effect')]",
        "details": {
          "type": "Microsoft.EventHub/namespaces/privateEndpointConnections",
          "existenceCondition": {
            "field": "Microsoft.EventHub/namespaces/privateEndpointConnections/privateLinkServiceConnectionState.status",
            "equals": "Approved"
          }
        }
      }
    }
  }
}

You can copy and past the following code from above Policy Definition in the Azure Portal where you can do a first test of the Policy Definition deployment.

{
  "mode": "All",
  "parameters": {
    "effect": {
      "type": "String",
      "metadata": {
        "displayName": "Allowed Azure Policy Effect",
        "description": "The allowed effect for this Azure Policy"
      },
      "defaultValue": "AuditIfNotExists"
    }
  },
  "policyRule": {
    "if": {
      "field": "type",
      "equals": "Microsoft.EventHub/Namespaces"
    },
    "then": {
      "effect": "[parameters('effect')]",
      "details": {
        "type": "Microsoft.EventHub/namespaces/privateEndpointConnections",
        "existenceCondition": {
          "field": "Microsoft.EventHub/namespaces/privateEndpointConnections/privateLinkServiceConnectionState.status",
          "equals": "Approved"
        }
      }
    }
  }
}

Screenshot of Policy Definition in Azure Portal

And save the Policy to deploy the Policy via the Azure Portal.

Assign Azure Policy

After deploying the “Private endpoint should be enabled for Event Hub” custom Azure Policy we need to assign the Azure Policy at the correct scope.

You can follow the documentation at Microsoft Docs called Quickstart: Create a policy assignment to identify non-compliant resources to assign the previously deployed custom Azure Policy.

In our example we are assigning the custom Azure Policy at the highest scope in our Enterprise-Scale environment at Management Group ES scope.

Screenshot of Policy Assignment in Azure Portal

Screenshot of Policy Assignment in Azure Portal

Trigger on-demand Azure Policy compliance evaluation

So the custom Azure Policy Definition is assigned, let’s now trigger a Policy compliance evaluation to see if the earlier deployed Event Hub with Private link connection is compliant.

If found that using the Azure CLI is the easiest way to trigger an on-demand Azure Policy compliance evaluation. Especially because we don’t want to wait on the Azure platform to trigger that evaluation, because that can take some time (30 mins) to be triggered.

Run the following Azure CLI commands.

az policy state trigger-scan --resource-group "es-demo-rg"

Keep in mind this can take a couple of mins to finish. So get yourself a cup of tea or coffee ☕ Below trigger run for 8 minutes.

Animated gif showing the Azure CLI trigger Policy scan

You can check in the Azure Portal the status of the Policy.

Screenshot of Azure Policy Compliance state in Azure Portal

Validate non-compliancy

The final test if the custom Azure Policy works as expected is by removing the Private link connection from the earlier deployed Event Hub.

Screenshot of Event Hub Private link connection removal in Azure Portal

Run the on-demand Azure Policy compliance evaluation Azure CLI command again after you have removed the Private link connection and look at the result.

Run again the following Azure CLI commands.

az policy state trigger-scan --resource-group "es-demo-rg"

We now see that the Event Hub is non-compliant because there is no Private link connection configured.

Screenshot of non-compliant Event Hub in Azure Portal

Issue

So what was the issue where it all started with? When I first started developing this custom Azure Policy I started with the following Policy Definition.

{
    "properties": {
      "displayName": "Not working Private endpoint should be enabled for Event Hub",
      "policyType": "Custom",
      "mode": "All",
      "description": "Private endpoint connections enforce secure communication by enabling private connectivity to Azure Event Hub. Configure a private endpoint connection to enable access to traffic coming only from known networks and prevent access from all other IP addresses, including within Azure.",
      "parameters": {
        "effect": {
          "type": "String",
          "metadata": {
            "displayName": "Allowed Azure Policy Effect",
            "description": "The allowed effect for this Azure Policy"
          },
          "defaultValue": "AuditIfNotExists"
        }
      },
      "policyRule": {
        "if": {
          "allOf": [
            {
              "field": "type",
              "equals": "Microsoft.EventHub/Namespaces"
            },
            {
              "field": "Microsoft.EventHub/namespaces/privateEndpointConnections/privateLinkServiceConnectionState.status",
              "notLike": "Approved"
            }
          ]
        },
        "then": {
          "effect": "audit"
        }
      }
    }
 }

Do you see the difference between the working version and this version?

Let’s compare the Policy Rule side-by-side:

Table comparing working and non working custom Policies

So why does not the right audit Azure Policy works?

The reason the audit policy does not work is because it is auditing a resource of type namespaces, but attempting to apply evaluation criteria to a different resource type, namespaces/privateEndpointConnections. Evaluating the subtype requires another GET request after GETting the parent namespaces resource, and this is not performant enough.

You can validate above using the Azure Resource Explorer.

Screenshot over Azure Resource Explorer showing different resource types

This exact problem is why DeployIfNotExists (DINE) and AuditIfNotExists (AINE) are provided: it allows the policy writer to specifically evaluate only the subtypes and fields they need to apply conditions to, and either audit or update the resource asynchronously. Doing this much processing in a deny or audit policy would cause serious performance problems in ARM, since such evaluations must be done in the PUT path and can’t be done asynchronously.

The key to understanding this is to realize that the if {} section of a policy rule evaluates only one resource type at a time.

Compare this with the following custom Azure Policy to audit the deployment of an Azure Container Registry if the Admin user setting is set to Enabled.

"policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.ContainerRegistry/registries"
          },
          {
            "field": "Microsoft.ContainerRegistry/registries/adminUserEnabled",
            "equals": "True"
          }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]"
      }
    }

Screenshot Resource Explorer for Microsoft.ContainerRegistry

In this case there is no different resource type to be evaluated.

I want to thank Chris Stackhouse helping me understand the logic of the existenceCondition in Azure Policies.

Hope you also learned something new.

Reference