Send Security Alerts From Microsoft Cloud To 3rd Party SIEM With Logic Apps and Event Hub

Introduction

This blog is the next part of the Microsoft Security monitoring blog series. Earlier post you can find from the following link

• Microsoft 365 – Security Monitoring

Even though Azure Sentinel is on top of its hype in the Microsoft ecosystem, there are many other SIEM products in the market. In this post, I will walk through what’s needed to send alerts from the Microsoft cloud security solutions to 3rd party SIEM with one integration. The SIEM role in this specific example case is played by IBM QRadar.

Special greetings to my colleague Tommi Äijälä who helped me with IBM QRadar configuration.

Disclaimer: there are other ways to integrate Microsoft security solutions to SIEM and in this example, I’m covering one of them by leveraging Intelligent Security Graph (ISG) with example configuration in a demo environment.

Setting Up The Scene

In a nutshell, the following resources are needed to achieve the aimed scenario, have alerts in a centralized SIEM solution:

• Intelligent Security Graph (ISG)
• Azure Logic Apps
• Azure Event Hub
• SIEM – QRadar in this scenario

Intelligent Security Graph (ISG) aka Microsoft Graph Security API

I like acronyms so I’m used to ISG but Graph Security API is also seen as a solution name quite often. In this post, it’s referred to as ISG later on. This is how Microsoft describes the Intelligent Security Graph: “The Graph Security API can be defined as an intermediary service (or broker) that provides a single programmatic interface to connect multiple security providers. Requests to the graph are federated to all applicable providers. The results are aggregated and returned to the requesting application in a common schema.”

• 

At the time of writing the following security solutions act as providers to the ISG. As you can see from the list below, it’s quite impressive.

Pretty many of the Microsoft cloud security solutions are covered and more are coming. The whole idea with ISG integration is that in case of 3rd party SIEM only one integration would be enough to get all the necessary alert from the cloud back to down earth.

From technical perspective, you can integrate solution individually (API / SIEM agent) and get alerts from MDATP, MCAS, etc. On the other hand, in my opinion there is clear admin overhead comparing to one individual integration.

Logic App

According to Microsoft: ”Azure Logic Apps is a cloud service that helps you schedule, automate, and orchestrate tasks, business processes, and workflows when you need to integrate apps, data, systems, and services across enterprises or organizations”.

Event Hub

According to Microsoft: ”Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters”.

QRadar

According to IBM: ”Security Information and Event Management (SIEM) helps security teams accurately detect and prioritize threats across the enterprise, and it provides intelligent insights that enable teams to respond quickly to reduce the impact of incidents”.

Configuration

Okay, the scene is set, and then to actual beef in this blog post, how this should be configured to get alerts from the cloud to the QRadar? Let’s get started!

Intelligent Security Graph (ISG)

Security solutions act as a provider to ISG out of the box and configuration to ISG itself is not needed. When the alert is created in a provider, for example in Azure ATP, it will be synchronized automatically to ISG.

• 

Azure Event Hub

Azure Event Hub is a standard integration method for many 3rd party SIEM’s. In a nutshell, you need to send alerts/events from the source you want to Event Hub and make the integration between Event Hub and Radar. Needed configurations for the Event Hub are:

Create Event Hub Namespace

• This is the top level of your Event Hub hierarchy and place where you create the actual Event Hubs (EH). For me, it reminds DFS service and DFSRoot from on-prem and Active Directory world
• Allow access from all networks. If you don’t, Radar is not able to connect to EH.

Create Event Hub and configure it to support the integration

• Select a logical name for your Event Hub. Mine is ”isgalertslogicapps” to identify what’s the reason for this EH later on
• Create shared access policies to EH. What’s needed is
• Create a consumer group. The name doesn’t matter in here as long as you don’t use ”$Default” which is the default EH consumer group name
• Enable the capture feature to test the integration. I highly recommend enabling capture after the initial configuration to verify that EH contains ISG alerts
  • Be aware that capture adds consumption. In this particular case, I would recommend to use it only for testing purposes

• 

• 

• 

How to use Event Hub capture feature?

The Azure Event Hub capture feature was a game-changer for troubleshooting Event Hub connections and integrations. With the capture feature, you can deliver the streaming data in Event Hubs to Azure blob storage or Azure Data Lake Store account.

When you enable the capture feature ”avro” file is created to storage account of your choice. It can be converted to JSON with java developer tools. It took a while until I found proper tools to convert the files and for saving your time here is a short instruction on how to do it.

How to convert *.avro to json?

• Install Java SE Development Kit 14
  • https://www.oracle.com/java/technologies/javase-jdk14-downloads.html
• Verify that JAVA is added to environment variables
• Download tools from Apache
  • www.apache.org/dist/avro/stable/java/avro-tools-1-9-2.jar
  • https://charbelnemnom.com/2020/04/export-azure-security-center-alerts-and-recommendations-to-azure-event-hubs/
  • https://avro-cli.readthedocs.io/en/latest/
• Run conversion
  • Java -jar avro-tools-1.9.2.jar tojson 31.avro > 31.json

• 

• 

• 

• 

Azure Logic Apps

The logic behind this idea is to query alerts from ISG to Logic Apps and send them to Event Hub. QRadar has connector to Event Hub where it will pull the alerts.

There are multiple ways to build the Logic App query and below is just an example. In my environment, I have many alerts in ISG and for that reason in the first run, I’m pulling all of them with a safe top=100 parameter from the ISG.

Did you know that ISG supports updating alert status? One alternative option is to update alert status in every run from ”NewAlert” to ”InProgress”. By doing so, the query will only have alerts that have ”NewAlert” as a status. This would be very beneficial if you would like to send the alerts to the ticketing system.

• 

I updated alert status from ”NewAlert” to ”InProgress” in my own environment. For some reason, comments did not update even the Playbook result was a success.

• 

On the left, before the update. On the right, after the update.

• 

• 

IBM QRadar

QRadar needs following parameters

• Azure Event Hub connection string
• Azure Event Hub consumer group name
• Azure Storage Account connection string

• 

• 

Next phase – validating the incoming data. After configuration was deployed we saw that the ISG alerts were coming to the QRadar, awesome! This means that my work was done but it doesn’t mean that we’re done.

At the time of writing, IBM Radar doesn’t have a supported connector for Intelligent Security Graph which means that the SOC personnel needs to do ”magic” in the QRadar end to get most out of this integration.

More information from this link.

Caveats

I made the first integration with 3rd party SIEM and Azure in 2018 and I have to say that it was a tough road. Now, after a couple of years, I have banged my head enough to the wall and find most caveats, at least I think so 🙂

• An Azure storage account cannot be V2 version, otherwise authentication part fails. It needs to be V1. Yeah, I know what you might think about this, but it is what it is.
• Azure Event Hub consumer group cannot be named as $Default. When Event Hub is created the default consumer group name is @Default. Create whatever name you want instead of $Default.
• Currently, IBM Radar doesn’t support the direct Security Graph API connection. It’s coming but the schedule is still unknown.
  • We are currently working with IBM QRadar to add complete support for Microsoft Graph Security API alerts.
  • Currently, you will be able to receive the Microsoft Security Graph API alerts and view them in your IBM QRadar console. 
• Event Hub Namespace IP restrictions cannot be used. Otherwise, Radar is not able to connect to the Event Hub.

Conclusion

Even though, Azure Sentinel is a hot topic a the moment there are many environments where the organization is still using 3rd party SIEM, such as QRadar or Splunk.

In such environments, the needed scenario might be similar to described in this blog post. The standard method to send alerts from Microsoft security solutions to IBM QRadar is to make the integration between Azure Event Hub and QRadar. I have been involved in multiple integration cases and it’s straightforward operation when you get familiar with it.

I would rather prefer to use one integration instead of many. For that reason, I always encourage to use the Intelligent Security Graph for the integration which contains processed events from the aforementioned security providers.

Managing alerts in Microsoft ecosystem is the next natural topic which I will address in the coming posts. Stay safe!

More reading

I encourage to read Wortell Jeroen Niesen’s blog about Azure Logic Apps & Graph Security API integration with the ticketing system. The blog was an inspiration for me to build integration with ISG & QRadar.