AKS Azure Firewall Web Application Firewall

AKS Security Focused Architecture reference (network) with Azure Firewall


Background architecture is similar to one defined by Microsoft in AKS Security Baseline, with bit of simplification as Ingress connections bind directly to POD’s IP via AGIC configuration (without specific ingress service configuration)

Documents purpose is to give brief overview for integration possibilities with Azure Firewall, and various levels of network and web security in AKS, mostly referencing MS Docs, and simplified design pattern drawn here.

Solutions chosen for various areas concerning this post

Cheat Sheet

ServiceWeb Application FirewallAdvanced Egress FilteringInter-Cluster Network SecurityThreat Intel
Azure Application Gateway
Azure Firewall
Kubernetes Network Policies

Firewall and Application Gateway in parallel

Design reference chosen concerning Azure Firewall and Application Gateway

Source: https://docs.microsoft.com/en-us/azure/architecture/example-scenario/gateway/firewall-application-gateway#firewall-and-application-gateway-in-parallel

Design reference for overall Architecture

  • Private cluster model (this actually simplifies the integration with Azure Firewall if you are doing egress filtering in Azure Firewall)
Source: https://docs.microsoft.com/en-us/azure/aks/limit-egress-traffic#azure-global-required-network-rules
  • API server is only available via private IP, but there is public DNS record (this simplifies management quite the bit)
  • Egress traffic is routed via Azure Firewall
  • Ingress traffic outside to the cluster for web workloads traverses via AGIC enabled with Azure Web Application Firewall
  • Inter-cluster workloads are protected by network policy
  • Traffic to Azure PaaS Service traverses via Azure Firewall ( you could also bypass Firewall in these scenarios, especially for VNET service endpoints, but there is benefit of centralization in terms of logging and connecting via Azure Firewall)
Cluster traffic flow
Source: Baseline architecture for an AKS cluster – Azure Architecture Center | Microsoft Docs

AKS Rule 1 – Plan for change

Design your AKS Architecture in a way that you can redeploy the cluster just like you would redeploy an application. The basis of this idea is that AKS is fast moving target. You will eventually be in a situation where in-place upgrade of cluster does not give the security enhancement you are looking for – Good example is Network Policies (1), if you had not enabled those in the first place, you won’t be enabling them later either, (network policy does not support in-place upgrade).

Plan the cluster to be rather a supporting piece for the application, that has seperate flexible lifecycle, as opposed to permanent service generating technical dept.

Source: https://docs.microsoft.com/en-us/azure/aks/use-network-policies#create-an-aks-cluster-and-enable-network-policy

Briefly about network policies

Since you cant use NSG’s and inter-subnet NSG rules in AKS you must then use network policies to limit any traffic you want to have locked down inside the cluster

”Blocking internal subnet traffic using network security groups (NSGs) and firewalls is not supported. To control and block the traffic within the cluster, use Network Policies.” https://docs.microsoft.com/en-us/azure/aks/limit-egress-traffic#background

Use network policies to segregate and secure intra-service communications by controlling which components can communicate with each other. By default, all pods in a Kubernetes cluster can send and receive traffic without limitations. To improve security, you can use Azure Network Policies or Calico Network Policies to define rules that control the traffic flow between different microservices. For more information, see Network Policy.


At first glimpse the user defined route with the next-hop address to Azure Firewall might feel quite obvious. All egress traffic should flow there? Or should it… MS has great article on this, but to summarize:

  • Egress Internet traffic goes to AZFW
  • VNET service endpoint delegations on AKS Subnet bypass firewall
    • More here about what might not follow the rule :
      • If you place the VNET service endpoint policy to Azure Firewall subnet, and remove it from AKS subnet, you will get centralized access point to PaaS services enabled for Service Endpoint via Azure Firewall
    • Response to traffic from AGIC flows back to AGIC and so on.
    • Cluster and inter-pod inter-vnet-subnet traffic stays within


AZ CLI reference for the cluster

Notes from testing

# Base variables, including AAD Admin group, names of groups etc

az group create -n $NAMER \
--tags $TAGS

az network route-table create --name ${RT}_PODS \
--resource-group $NAMER \
--tags $TAGS

az network route-table route create -g $NAMER --route-table-name ${RT}_PODS -n ${RT}_PODS_RULE \
--next-hop-type VirtualAppliance --address-prefix --next-hop-ip-address $FWPrivateIp

az network vnet create --name ${NET} \
--resource-group ${NAMER} \
--address-prefix \
--subnet-name ${NET}_AKS \
--subnet-prefix \
--tags $TAGS

VNET_ID=$(az network vnet show --resource-group ${NAMER} --name ${NET} --query id -o tsv)
VNET_ID2=$(az network vnet show --resource-group ${AZRG} --name ${FWNET} --query id -o tsv)
# Create Peer 

ASSIGNEDID=$(az identity create --name ${NAME}_CONTROL_PLANE -g ${NAMER} --tags $TAGS --query id -o tsv)

az network vnet peering create -g $NAMER -n ${NET}_AKS_PEER  --vnet-name ${NET}  \
--remote-vnet $VNET_ID2 --allow-vnet-access --allow-forwarded-traffic

az network vnet peering create -g $AZRG -n ${FWNET}_AKS_PEER  --vnet-name ${FWNET}  \
--remote-vnet $VNET_ID --allow-vnet-access --allow-forwarded-traffic

az network vnet subnet create -g ${NAMER} --vnet-name ${NET} -n ${NET}_AGIC \
--address-prefixes \
--route-table ${RT}_PODS 

SUBNET_ID1=$(az network vnet subnet show --resource-group ${NAMER} --vnet-name ${NET} --name ${NET}_AKS --query id -o tsv)
SUBNET_ID2=$(az network vnet subnet show --resource-group ${NAMER} --vnet-name ${NET} --name ${NET}_AGIC --query id -o tsv)

az aks create \
-n ${NAME} -g ${NAMER} \
--enable-private-cluster \
--enable-public-fqdn  \
--enable-pod-identity \
--enable-aad \
--aad-admin-group-object-ids $AADG \
--disable-local-accounts \
--enable-azure-rbac \
--network-plugin azure \
--private-dns-zone none \
--network-policy azure \
-a ingress-appgw --appgw-name AKS-AGIC \
--appgw-subnet-id $SUBNET_ID1 \
--tags $TAGS \
--vnet-subnet-id ${SUBNET_ID1} \
--node-resource-group ${NAMER}_AZ_RES \
--attach-acr $ACR \
 --assign-identity $ASSIGNEDID

Azure Key Vault logs for POD identity accessing Azure Key Vault via service endpoint now show the Azure FW subnet internal IP address as the clientIP

AzFW IP shown on key vault logs
Pod-identity accessing the Key Vault
Address from AZFW address space shown on Key Vault logs

End of document

1 comment on “AKS Security Focused Architecture reference (network) with Azure Firewall

  1. hello, thanks for this article. I had a question. in an AKS Cluster environment with Nginx ingress controller is it possible for incoming traffic to use an application gateway in front of an azure firewall. I haven’t been able to find any similar architecture. thank for your help. regards
    HENNEBERT Mickael



Täytä tietosi alle tai klikkaa kuvaketta kirjautuaksesi sisään:


Olet kommentoimassa WordPress.com -tilin nimissä. Log Out /  Muuta )


Olet kommentoimassa Twitter -tilin nimissä. Log Out /  Muuta )


Olet kommentoimassa Facebook -tilin nimissä. Log Out /  Muuta )

Muodostetaan yhteyttä palveluun %s

%d bloggaajaa tykkää tästä: