Alert on indicators for the illegitimate use of by-design Azure AD Bypass for AKS cluster access

I’ve been lately working bit more with AKS than I typically do. One thing that can easily be confusing is how Azure AD integration works at AKS cluster level (at least for me)

First thing to understand, that this is not bypass on Azure level RBAC, or Azure AD in the first place, but rather bypassing the authentication option of Azure AD when accessing the cluster via kubectl. There is major distinction here, and it definitely requires more research on my part to understand fully… if the bypass (which is by-design) can be mitigated with just good overall governance and design of the AKS (Which I believe is the case)

Monitoring in absence of perfect governance action

Meanwhile as I don’t have the perfect strategy/design to mitigate this, I wanted be able to have at least one alert condition on a first use case, when somebody gets the credentials from AZ CLI with admin credentials instead using Azure AD authentication

There are two flags here which I took into account (username correlation with azure activity log action preceding the request of creds) – In normal use context the cluster shows user context as the Azure AD object of the user; not the admin-user of the cluster. Bare in mind, that I am just monitoring one use case

Disclaimer If attacker knew what behavior you are monitoring, they can easily evade the logs. By performing subsequent actions from different IP and or use cached credentials for kubectl. I have another use case to monitor those evasions, but that requires cluster to accumulate bit more logs create trend, and anomaly scores.

Legitimate use

When listClusterAdminCredential action is performed it should be followed by event in kube-audit log containing the same caller (UPN)

Possible illegitimate use

AzureDiagnostics | extend logs = parse_json(log_s)
| extend tostring(logs.user.username), tostring(logs.requestURI), 
 tostring(logs.userAgent)
| where logs_user_username contains "@" or logs_user_username contains "master"
| mv-expand logs.sourceIPs
| summarize make_set(logs_userAgent) by logs_user_username, tostring(logs_sourceIPs)
| join (AzureActivity
| summarize make_set(OperationNameValue) by Caller, CallerIpAddress
) on $left.logs_sourceIPs == $right.CallerIpAddress
| where tolower(Caller)  !=  tolower(logs_user_username)

Some background

When does this apply?

• The operator has sufficient permissions in the first place configured in Azure RBAC
• Operator uses the –admin option to get credentials for AKS cluster

Where this might be undesirable

When you want the most high privileged users to interface the cluster with Kubectl using Azure AD authentication (one use case is targeting the specific CA policy for the use of Kubectl) // Please note you can mitigate this also by just applying the desired CA policy for wider scope for such management operations (Eg. user is prompted for trusted device for all azure related administration events)

Use Azure AD in Azure Kubernetes Service – Azure Kubernetes Service | Microsoft Docs

References

While I discovered this and the simple monitoring strategy independently (this is no secret information on by-design bypass) it was not an much of an surprise that there were similar findings written before me, so credit is due here The revenge of system:masters, return of the AKS (raesene.github.io)