Send data to Microsoft Sentinel

Overview

You can send logs to Microsoft Sentinel via HTTP using our HTTP Data sink.

Microsoft Sentinel configuration

Follow these steps to

1

Create a Microsoft Entra application

• Save the application Client ID

• Save the application Client Secret Value

• Get “OAuth 2.0 token endpoint (v2)” from the application endpoints

• Scope is always: https://monitor.azure.com/.default

2

Create a data collection endpoint

3

Create a data collection rule using an ARM template.

You can download the required template here:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "dataCollectionRuleName": {
            "type": "string",
            "metadata": {
                "description": "Specifies the name of the Data Collection Rule to create."
            }
        },
        "location": {
            "defaultValue": "[resourceGroup().location]",
            "type": "string",
            "metadata": {
                "description": "Specifies the location in which to create the Data Collection Rule."
            }
        },
        "workspaceResourceId": {
            "type": "string",
            "metadata": {
                "description": "Specifies the Azure resource ID of the Log Analytics workspace to use."
            }
        },
        "endpointResourceId": {
            "type": "string",
            "metadata": {
                "description": "Specifies the Azure resource ID of the Data Collection Endpoint to use."
            }
        }
    },
    "resources": [{
        "type": "Microsoft.Insights/dataCollectionRules",
        "apiVersion": "2021-09-01-preview",
        "name": "[parameters('dataCollectionRuleName')]",
        "location": "[parameters('location')]",
        "properties": {
            "dataCollectionEndpointId": "[parameters('endpointResourceId')]",
            "streamDeclarations": {
                "Custom-CommonSecurityLog": {
                   "columns": [
                        {
                            "name": "AccessList",
                            "type": "string"
                        },
                        {
                            "name": "AccessMask",
                            "type": "string"
                        },
                        {
                            "name": "AccessReason",
                            "type": "string"
                        },
                        {
                            "name": "Account",
                            "type": "string"
                        },
                        {
                            "name": "AccountDomain",
                            "type": "string"
                        },
                        {
                            "name": "AccountExpires",
                            "type": "string"
                        },
                        {
                            "name": "AccountName",
                            "type": "string"
                        },
                        {
                            "name": "AccountSessionIdentifier",
                            "type": "string"
                        },
                        {
                            "name": "AccountType",
                            "type": "string"
                        },
                        {
                            "name": "Activity",
                            "type": "string"
                        },
                        {
                            "name": "AdditionalInfo",
                            "type": "string"
                        },
                        {
                            "name": "AdditionalInfo2",
                            "type": "string"
                        },
                        {
                            "name": "AllowedToDelegateTo",
                            "type": "string"
                        },
                        {
                            "name": "Attributes",
                            "type": "string"
                        },
                        {
                            "name": "AuditPolicyChanges",
                            "type": "string"
                        },
                        {
                            "name": "AuditsDiscarded",
                            "type": "int"
                        },
                        {
                            "name": "AuthenticationLevel",
                            "type": "int"
                        },
                        {
                            "name": "AuthenticationPackageName",
                            "type": "string"
                        },
                        {
                            "name": "AuthenticationProvider",
                            "type": "string"
                        },
                        {
                            "name": "AuthenticationServer",
                            "type": "string"
                        },
                        {
                            "name": "AuthenticationService",
                            "type": "int"
                        },
                        {
                            "name": "AuthenticationType",
                            "type": "string"
                        },
                        {
                            "name": "AzureDeploymentID",
                            "type": "string"
                        },
                        {
                            "name": "CACertificateHash",
                            "type": "string"
                        },
                        {
                            "name": "CallerProcessId",
                            "type": "string"
                        },
                        {
                            "name": "CalledStationID",
                            "type": "string"
                        },
                        {
                            "name": "CallerProcessName",
                            "type": "string"
                        },
                        {
                            "name": "CallingStationID",
                            "type": "string"
                        },
                        {
                            "name": "CAPublicKeyHash",
                            "type": "string"
                        },
                        {
                            "name": "CategoryId",
                            "type": "string"
                        },
                        {
                            "name": "CertificateDatabaseHash",
                            "type": "string"
                        },
                        {
                            "name": "Channel",
                            "type": "string"
                        },
                        {
                            "name": "ClassId",
                            "type": "string"
                        },
                        {
                            "name": "ClassName",
                            "type": "string"
                        },
                        {
                            "name": "ClientAddress",
                            "type": "string"
                        },
                        {
                            "name": "ClientIPAddress",
                            "type": "string"
                        },
                        {
                            "name": "ClientName",
                            "type": "string"
                        },
                        {
                            "name": "CommandLine",
                            "type": "string"
                        },
                        {
                            "name": "CompatibleIds",
                            "type": "string"
                        },
                        {
                            "name": "Computer",
                            "type": "string"
                        },
                        {
                            "name": "DCDNSName",
                            "type": "string"
                        },
                        {
                            "name": "DeviceId",
                            "type": "string"
                        },
                        {
                            "name": "DisplayName",
                            "type": "string"
                        },
                        {
                            "name": "Disposition",
                            "type": "string"
                        },
                        {
                            "name": "DomainBehaviorVersion",
                            "type": "string"
                        },
                        {
                            "name": "DomainName",
                            "type": "string"
                        },
                        {
                            "name": "DomainPolicyChanged",
                            "type": "string"
                        },
                        {
                            "name": "DomainSid",
                            "type": "string"
                        },
                        {
                            "name": "EAPType",
                            "type": "string"
                        },
                        {
                            "name": "ErrorCode",
                            "type": "int"
                        },
                        {
                            "name": "ElevatedToken",
                            "type": "string"
                        },
                        {
                            "name": "EventID",
                            "type": "int"
                        },
                        {
                            "name": "EventData",
                            "type": "string"
                        },
                        {
                            "name": "EventSourceName",
                            "type": "string"
                        },
                        {
                            "name": "ExtendedQuarantineState",
                            "type": "string"
                        },
                        {
                            "name": "FailureReason",
                            "type": "string"
                        },
                        {
                            "name": "FileHash",
                            "type": "string"
                        },
                        {
                            "name": "FilePath",
                            "type": "string"
                        },
                        {
                            "name": "FilePathNoUser",
                            "type": "string"
                        },
                        {
                            "name": "Filter",
                            "type": "string"
                        },
                        {
                            "name": "ForceLogoff",
                            "type": "string"
                        },
                        {
                            "name": "Fqbn",
                            "type": "string"
                        },
                        {
                            "name": "FullyQualifiedSubjectMachineName",
                            "type": "string"
                        },
                        {
                            "name": "FullyQualifiedSubjectUserName",
                            "type": "string"
                        },
                        {
                            "name": "GroupMembership",
                            "type": "string"
                        },
                        {
                            "name": "HandleId",
                            "type": "string"
                        },
                        {
                            "name": "HardwareIds",
                            "type": "string"
                        },
                        {
                            "name": "HomeDirectory",
                            "type": "string"
                        },
                        {
                            "name": "HomePath",
                            "type": "string"
                        },
                        {
                            "name": "ImpersonationLevel",
                            "type": "string"
                        },
                        {
                            "name": "IpAddress",
                            "type": "string"
                        },
                        {
                            "name": "IpPort",
                            "type": "string"
                        },
                        {
                            "name": "KeyLength",
                            "type": "int"
                        },
                        {
                            "name": "Level",
                            "type": "string"
                        },
                        {
                            "name": "LmPackageName",
                            "type": "string"
                        },
                        {
                            "name": "LocationInformation",
                            "type": "string"
                        },
                        {
                            "name": "LockoutDuration",
                            "type": "string"
                        },
                        {
                            "name": "LockoutObservationWindow",
                            "type": "string"
                        },
                        {
                            "name": "LockoutThreshold",
                            "type": "string"
                        },
                        {
                            "name": "LoggingResult",
                            "type": "string"
                        },
                        {
                            "name": "LogonHours",
                            "type": "string"
                        },
                        {
                            "name": "LogonID",
                            "type": "string"
                        },
                        {
                            "name": "LogonProcessName",
                            "type": "string"
                        },
                        {
                            "name": "LogonType",
                            "type": "int"
                        },
                        {
                            "name": "LogonTypeName",
                            "type": "string"
                        },
                        {
                            "name": "MachineAccountQuota",
                            "type": "string"
                        },
                        {
                            "name": "MachineInventory",
                            "type": "string"
                        },
                        {
                            "name": "MachineLogon",
                            "type": "string"
                        },
                        {
                            "name": "ManagementGroupName",
                            "type": "string"
                        },
                        {
                            "name": "MandatoryLabel",
                            "type": "string"
                        },
                        {
                            "name": "MaxPasswordAge",
                            "type": "string"
                        },
                        {
                            "name": "MemberName",
                            "type": "string"
                        },
                        {
                            "name": "MemberSid",
                            "type": "string"
                        },
                        {
                            "name": "MinPasswordAge",
                            "type": "string"
                        },
                        {
                            "name": "MinPasswordLength",
                            "type": "string"
                        },
                        {
                            "name": "MixedDomainMode",
                            "type": "string"
                        },
                        {
                            "name": "NASIdentifier",
                            "type": "string"
                        },
                        {
                            "name": "NASIPv4Address",
                            "type": "string"
                        },
                        {
                            "name": "NASIPv6Address",
                            "type": "string"
                        },
                        {
                            "name": "NASPort",
                            "type": "string"
                        },
                        {
                            "name": "NASPortType",
                            "type": "string"
                        },
                        {
                            "name": "NetworkPolicyName",
                            "type": "string"
                        },
                        {
                            "name": "NewDate",
                            "type": "string"
                        },
                        {
                            "name": "NewMaxUsers",
                            "type": "string"
                        },
                        {
                            "name": "NewProcessId",
                            "type": "string"
                        },
                        {
                            "name": "NewProcessName",
                            "type": "string"
                        },
                        {
                            "name": "NewRemark",
                            "type": "string"
                        },
                        {
                            "name": "NewShareFlags",
                            "type": "string"
                        },
                        {
                            "name": "NewTime",
                            "type": "string"
                        },
                        {
                            "name": "NewUacValue",
                            "type": "string"
                        },
                        {
                            "name": "NewValue",
                            "type": "string"
                        },
                        {
                            "name": "NewValueType",
                            "type": "string"
                        },
                        {
                            "name": "ObjectName",
                            "type": "string"
                        },
                        {
                            "name": "ObjectServer",
                            "type": "string"
                        },
                        {
                            "name": "ObjectType",
                            "type": "string"
                        },
                        {
                            "name": "ObjectValueName",
                            "type": "string"
                        },
                        {
                            "name": "OemInformation",
                            "type": "string"
                        },
                        {
                            "name": "OldMaxUsers",
                            "type": "string"
                        },
                        {
                            "name": "OldRemark",
                            "type": "string"
                        },
                        {
                            "name": "OldShareFlags",
                            "type": "string"
                        },
                        {
                            "name": "OldUacValue",
                            "type": "string"
                        },
                        {
                            "name": "OldValue",
                            "type": "string"
                        },
                        {
                            "name": "OldValueType",
                            "type": "string"
                        },
                        {
                            "name": "OperationType",
                            "type": "string"
                        },
                        {
                            "name": "PackageName",
                            "type": "string"
                        },
                        {
                            "name": "ParentProcessName",
                            "type": "string"
                        },
                        {
                            "name": "PartitionKey",
                            "type": "string"
                        },
                        {
                            "name": "PasswordHistoryLength",
                            "type": "string"
                        },
                        {
                            "name": "PasswordLastSet",
                            "type": "string"
                        },
                        {
                            "name": "PasswordProperties",
                            "type": "string"
                        },
                        {
                            "name": "PreviousDate",
                            "type": "string"
                        },
                        {
                            "name": "PreviousTime",
                            "type": "string"
                        },
                        {
                            "name": "PrimaryGroupId",
                            "type": "string"
                        },
                        {
                            "name": "PrivateKeyUsageCount",
                            "type": "string"
                        },
                        {
                            "name": "PrivilegeList",
                            "type": "string"
                        },
                        {
                            "name": "Process",
                            "type": "string"
                        },
                        {
                            "name": "ProcessId",
                            "type": "string"
                        },
                        {
                            "name": "ProcessName",
                            "type": "string"
                        },
                        {
                            "name": "ProfilePath",
                            "type": "string"
                        },
                        {
                            "name": "Properties",
                            "type": "string"
                        },
                        {
                            "name": "ProtocolSequence",
                            "type": "string"
                        },
                        {
                            "name": "ProxyPolicyName",
                            "type": "string"
                        },
                        {
                            "name": "QuarantineHelpURL",
                            "type": "string"
                        },
                        {
                            "name": "QuarantineSessionID",
                            "type": "string"
                        },
                        {
                            "name": "QuarantineSessionIdentifier",
                            "type": "string"
                        },
                        {
                            "name": "QuarantineState",
                            "type": "string"
                        },
                        {
                            "name": "QuarantineSystemHealthResult",
                            "type": "string"
                        },
                        {
                            "name": "RelativeTargetName",
                            "type": "string"
                        },
                        {
                            "name": "RemoteIpAddress",
                            "type": "string"
                        },
                        {
                            "name": "RemotePort",
                            "type": "string"
                        },
                        {
                            "name": "Requester",
                            "type": "string"
                        },
                        {
                            "name": "RequestId",
                            "type": "string"
                        },
                        {
                            "name": "RestrictedAdminMode",
                            "type": "string"
                        },
                        {
                            "name": "RowKey",
                            "type": "string"
                        },
                        {
                            "name": "RowsDeleted",
                            "type": "string"
                        },
                        {
                            "name": "SamAccountName",
                            "type": "string"
                        },
                        {
                            "name": "ScriptPath",
                            "type": "string"
                        },
                        {
                            "name": "SecurityDescriptor",
                            "type": "string"
                        },
                        {
                            "name": "ServiceAccount",
                            "type": "string"
                        },
                        {
                            "name": "ServiceFileName",
                            "type": "string"
                        },
                        {
                            "name": "ServiceName",
                            "type": "string"
                        },
                        {
                            "name": "ServiceStartType",
                            "type": "int"
                        },
                        {
                            "name": "ServiceType",
                            "type": "string"
                        },
                        {
                            "name": "SessionName",
                            "type": "string"
                        },
                        {
                            "name": "ShareLocalPath",
                            "type": "string"
                        },
                        {
                            "name": "ShareName",
                            "type": "string"
                        },
                        {
                            "name": "SidHistory",
                            "type": "string"
                        },
                        {
                            "name": "SourceSystem",
                            "type": "string"
                        },
                        {
                            "name": "Status",
                            "type": "string"
                        },
                        {
                            "name": "StorageAccount",
                            "type": "string"
                        },
                        {
                            "name": "SubcategoryId",
                            "type": "string"
                        },
                        {
                            "name": "Subject",
                            "type": "string"
                        },
                        {
                            "name": "SubjectAccount",
                            "type": "string"
                        },
                        {
                            "name": "SubjectDomainName",
                            "type": "string"
                        },
                        {
                            "name": "SubjectKeyIdentifier",
                            "type": "string"
                        },
                        {
                            "name": "SubjectLogonId",
                            "type": "string"
                        },
                        {
                            "name": "SubjectMachineName",
                            "type": "string"
                        },
                        {
                            "name": "SubjectMachineSID",
                            "type": "string"
                        },
                        {
                            "name": "SubjectUserName",
                            "type": "string"
                        },
                        {
                            "name": "SubjectUserSid",
                            "type": "string"
                        },
                        {
                            "name": "SubStatus",
                            "type": "string"
                        },
                        {
                            "name": "TableId",
                            "type": "string"
                        },
                        {
                            "name": "TargetDomainName",
                            "type": "string"
                        },
                        {
                            "name": "TargetInfo",
                            "type": "string"
                        },
                        {
                            "name": "TargetAccount",
                            "type": "string"
                        },
                        {
                            "name": "TargetLinkedLogonId",
                            "type": "string"
                        },
                        {
                            "name": "TargetLogonId",
                            "type": "string"
                        },
                        {
                            "name": "TargetOutboundDomainName",
                            "type": "string"
                        },
                        {
                            "name": "TargetOutboundUserName",
                            "type": "string"
                        },
                        {
                            "name": "TargetServerName",
                            "type": "string"
                        },
                        {
                            "name": "TargetSid",
                            "type": "string"
                        },
                        {
                            "name": "TargetUser",
                            "type": "string"
                        },
                        {
                            "name": "TargetUserName",
                            "type": "string"
                        },
                        {
                            "name": "TargetUserSid",
                            "type": "string"
                        },
                        {
                            "name": "Task",
                            "type": "int"
                        },
                        {
                            "name": "TemplateContent",
                            "type": "string"
                        },
                        {
                            "name": "TemplateDSObjectFQDN",
                            "type": "string"
                        },
                        {
                            "name": "TemplateInternalName",
                            "type": "string"
                        },
                        {
                            "name": "TemplateOID",
                            "type": "string"
                        },
                        {
                            "name": "TemplateSchemaVersion",
                            "type": "string"
                        },
                        {
                            "name": "TemplateVersion",
                            "type": "string"
                        },
                        {
                            "name": "TimeCollected",
                            "type": "datetime"
                        },
                        {
                            "name": "TimeGenerated",
                            "type": "datetime"
                        },
                        {
                            "name": "TokenElevationType",
                            "type": "string"
                        },
                        {
                            "name": "TransmittedServices",
                            "type": "string"
                        },
                        {
                            "name": "UserAccountControl",
                            "type": "string"
                        },
                        {
                            "name": "UserParameters",
                            "type": "string"
                        },
                        {
                            "name": "UserPrincipalName",
                            "type": "string"
                        },
                        {
                            "name": "UserWorkstations",
                            "type": "string"
                        },
                        {
                            "name": "VendorIds",
                            "type": "string"
                        },
                        {
                            "name": "VirtualAccount",
                            "type": "string"
                        },
                        {
                            "name": "Workstation",
                            "type": "string"
                        },
                        {
                            "name": "WorkstationName",
                            "type": "string"
                        },
                        {
                            "name": "Opcode",
                            "type": "string"
                        },
                        {
                            "name": "Version",
                            "type": "int"
                        },
                        {
                            "name": "DeviceDescription",
                            "type": "string"
                        },
                        {
                            "name": "InterfaceUuid",
                            "type": "string"
                        },
                        {
                            "name": "Keywords",
                            "type": "string"
                        },
                        {
                            "name": "LogonGuid",
                            "type": "string"
                        },
                        {
                            "name": "SubcategoryGuid",
                            "type": "string"
                        },
                        {
                            "name": "TargetLogonGuid",
                            "type": "string"
                        },
                        {
                            "name": "Type",
                            "type": "string"
                        },
                        {
                            "name": "SystemThreadId",
                            "type": "int"
                        },
                        {
                            "name": "SystemProcessId",
                            "type": "int"
                        },
                        {
                            "name": "Correlation",
                            "type": "string"
                        }
                    ]
                },
                "Custom-Syslog": {
                    "columns": [{
                            "name": "Computer",
                            "type": "string"
                        },
                        {
                            "name": "EventTime",
                            "type": "datetime"
                        },
                        {
                            "name": "Facility",
                            "type": "string"
                        },
                        {
                            "name": "HostIP",
                            "type": "string"
                        },
                        {
                            "name": "HostName",
                            "type": "string"
                        },
                        {
                            "name": "ManagementGroupName",
                            "type": "string"
                        },
                        {
                            "name": "ProcessID",
                            "type": "int"
                        },
                        {
                            "name": "ProcessName",
                            "type": "string"
                        },
                        {
                            "name": "SeverityLevel",
                            "type": "string"
                        },
                        {
                            "name": "SourceSystem",
                            "type": "string"
                        },
                        {
                            "name": "SyslogMessage",
                            "type": "string"
                        },
                        {
                            "name": "TimeCollected",
                            "type": "datetime"
                        },
                        {
                            "name": "TimeGenerated",
                            "type": "datetime"
                        },
                        {
                            "name": "OpCode",
                            "type": "string"
                        },
                        {
                            "name": "version",
                            "type": "int"
                        }
                    ]
                },
                "Custom-WindowsEvent": {
                    "columns": [{
                            "name": "Channel",
                            "type": "string"
                        },
                        {
                            "name": "Computer",
                            "type": "string"
                        },
                        {
                            "name": "EventData",
                            "type": "string"
                        },
                        {
                            "name": "EventID",
                            "type": "int"
                        },
                        {
                            "name": "EventLevel",
                            "type": "int"
                        },
                        {
                            "name": "EventLevelName",
                            "type": "string"
                        },
                        {
                            "name": "EventOriginId",
                            "type": "string"
                        },
                        {
                            "name": "ManagementGroupName",
                            "type": "string"
                        },
                        {
                            "name": "Provider",
                            "type": "string"
                        },
                        {
                            "name": "RawEventData",
                            "type": "string"
                        },
                        {
                            "name": "SourceSystem",
                            "type": "string"
                        },
                        {
                            "name": "Task",
                            "type": "int"
                        },
                        {
                            "name": "TimeGenerated",
                            "type": "datetime"
                        }
                    ]
                }
            },
            "destinations": {
                "logAnalytics": [{
                    "workspaceResourceId": "[parameters('workspaceResourceId')]",
                    "name": "logAnalyticsWorkspace"
                }]
            },
            "dataFlows": [{
                    "streams": [
                        "Custom-CommonSecurityLog"
                    ],
                    "destinations": [
                        "logAnalyticsWorkspace"
                    ],
                    "transformKql": "source",
                    "outputStream": "Microsoft-CommonSecurityLog"
                },
                {
                    "streams": [
                        "Custom-SecurityEvent"
                    ],
                    "destinations": [
                        "logAnalyticsWorkspace"
                    ],
                    "transformKql": "source",
                    "outputStream": "Microsoft-SecurityEvent"
                },
                {
                    "streams": [
                        "Custom-Syslog"
                    ],
                    "destinations": [
                        "logAnalyticsWorkspace"
                    ],
                    "transformKql": "source",
                    "outputStream": "Microsoft-Syslog"
                },
                {
                    "streams": [
                        "Custom-WindowsEvent"
                    ],
                    "destinations": [
                        "logAnalyticsWorkspace"
                    ],
                    "transformKql": "source",
                    "outputStream": "Microsoft-WindowsEvent"
                }
            ]
        }
    }],
    "outputs": {
        "dataCollectionRuleId": {
            "type": "string",
            "value": "[resourceId('Microsoft.Insights/dataCollectionRules', parameters('dataCollectionRuleName'))]"
        }
    }
}

Once your DCR is created, you must grant access to it for the application that you created in the first step. From the Monitor menu in the Azure portal, select Data Collection rules and then the DCR that you created. Select Access Control (IAM) for the DCR and then select Add role assignment to add the Monitoring Metrics Publisher role.

Data sink configuration

To start sending data to Microsoft Sentinel, follow these steps:

1

Create a new HTTP Data sink. To do it, go to Data sinks > New Data sink and double-click HTTP.

2

Give your Data sink a Name and, optionally, add a Description and some Tags. Click Finish when you're done.

3

Now, drag your Data sink to the required Pipeline canvas. Link it to the required Listener/Action and double-click it to configure it.

4

Fill the following parameters as follows:

• HTTP method* - Choose POST

• URL*Enter your Microsoft Sentinel endpoint, which should have the following format:

  {Data Collection Endpoint URI}/dataCollectionRules/{DCR Immutable ID}/streams/{Stream Name}?api-version=2023-01-01

  Note that a URL only points to one single table inside the DCR (Stream Name).

• Choose the field that contains the messages you would like to send

• Content-Type, Support special characters, Use gzip, HTTP headers - Set as required.

5

In the Bulk configuration section, fill in the parameters as required.

6

Set the Authentication type* to OAuth2 and fill in the parameters in the OAuth configuration section as follows:

• OAuth URL* - Enter From 1.c

• OAuth method* - POST

• Send body as - Form URL encoded

• OAuth request body - Set as required

• OAuth token path* - access_token

• Enter the following query key/value pairs:

  • grant_type: client_credentials

  • client_id: from 1.a

  • client_secret: from 1.b

  • scope: https://monitor.azure.com/.default

• Enter the following header key/value pair:

  • Content-Type: application/x-www-form-urlencoded

7

Fill in the rest of the parameters and required, and click Save.