As a transitional step, this site will temporarily be made Read-Only from July 8th until the new community launch. During this time, you can still search and read articles and discussions.
While the community is read-only, if you have questions or issues requiring TIBCO review/response, please access the new TIBCO Community and select "Ask A Question."
You will need to register or log in or register to engage in the new community.
Advanced Correlation Rules in TIBCO LogLogic® LMI
TIBCO LogLogic® provides the industry's first enterprise class, end-to-end log management solution. Using LogLogic® log management solutions, IT organizations can analyze and archive log and machine data for the purpose of compliance and legal protection, decision support for security remediation, and increased system performance and improved availability of overall infrastructure.
LogLogic® LMI 6.2.0 allows creating correlation rules to correlate two or more log sources for use in alerting. It also helps to search historical data and analyze the patterns in the data. In order to create advanced alerts, we need to follow some steps. This article covers the lifecycle of an advanced correlation alert by explaining the steps required to create Advanced Correlation Alerts and interact with the advanced alerts
Note: This article assumes that you are familiar with advanced search features in LogLogic LMI. To know more about the terminology used in this article, check the documentation here.
Throughout the article, the terms 1. "correlation blok" and "ECL blok" 2. "blok" and "rule" are used alternatively
1. Create a Correlation Filter Blok
The lifecycle of a correlation alert starts with a Correlation blok. Go to: Home > Management > Advanced Features > Bloks
Select ‘Correlation’ to create new Correlation blok. The Correlation blok uses Event Correlation Language (ECL). The ECL format is described here: https://docs.tibco.com/pub/loglmi/6.2.1/doc/html/GUID-043EB729-8744-4622-A2FE-7A97E39A46CD.html
Let us consider a simple Correlation blok:
USE LogLogic_Appliance WITHIN 30m EVENT GROUP [success] AT LEAST 1 EVENTS WHERE [ll_deviceTypeID] ="17" AND [ll_eventStatus] = "success" WITH THE SAME [ll_sourceUser],[ll_sourceIP] EVENT GROUP [failed] AT LEAST 1 EVENTS WHERE [ll_deviceTypeID] ="17" AND [ll_eventStatus] ="failed" WITH THE SAME [ll_sourceUser],[ll_sourceIP] CORRELATION success->[ll_sourceIP]== failed->[ll_sourceIP] success->[ll_sourceUser]== failed->[ll_sourceUser]
There are some constructs to know in this ECL blok. We'll go over them one-by-one:
USE: The list of log sources (Data Models) used by the rule. Also supports multiple log sources (Comma separated)
WITHIN: The time bucket scope to consider the events from log sources. Defined in seconds, minutes, hours, or days
EVENT GROUP: Each event group describes the criteria that must combine events to be grouped together as part of the blok/ rule. This is equivalent to a single search in EQL. There can be multiple event groups in a blok.
WITH THE SAME: Represents GROUP BY field(s)
HAVING: Represents the filter criteria on the aggregated functions
CORRELATION: A join condition describing which fields should be equal in two event groups
Test the correlation blok and save it with a name. E.g. TestECLBlok.
2. Test the blok on historical data (query)
You can test the ECL blok by executing a query by choosing the ECL blok. (This query is a snapshot query) The correlation blok name should have “correlation” prefix while executing the query on the UI. For example. correlation.TestECLBlok
Provide a time filter for the query example. -1h. You will notice that the matching results are grouped by the time buckets defined in the WITHIN criteria.
3. Create a Trigger
Once you have verified the ECL blok is working as expected on the historical data, it is time to create a Trigger using the ECL blok.
Go to: Home > Management > Advanced Features > Rule Management
Click on add a new 'Trigger'. Enter trigger details like 'Name', 'Description', 'Severity', 'Category', and finally select the Correlation Blok created in the step above. Optionally you can create a new Correlation Blok by clicking on the Blok icon on the same page.
Optionally configure the Notification(s):
Email Notification: Enter To, CC, Subject, Message fields
Syslog Notification: Enter Host, Protocol, Facility, Severity, Message fields
Note: After creating the trigger, you will need to “Sync all triggers”. This is an important step, it reinitializes the correlation instance with all the existing triggers including the newly created trigger.
The deployed triggers internally subscribe to real-time logs in LMI and evaluate the rules for any matching filter (i.e. correlation) criteria.
As the trigger(s) are evaluated, Advanced Alerts would be generated with optionally configured Email or Syslog Notifications.
4. Advanced Alerts Dashboard
Go to Home > Alerts > Advanced Alerts.
The 'Alerts' page shows all the Advanced Alerts triggered by various rules. Each 'Alert' shows 'Severity', 'SLA' (Time based on the severity - high severity alert needs to be addressed sooner compared to low severity alert, etc)
Clicking on an 'Alert' shows the alert details pop-over - more details, the rule, the actual evaluated values which triggered the alert and also a drill down URL which redirects to the Advanced Search page with a pre-filled query to show all the actual log events responsible for the alert.
The operator can 'Acknowledge' one or multiple Alerts. The acknowledgment pop-over allows the operator to provide a comment.
Note: The default 'Alerts' view shows only 'Unacknowledged' alerts. Use the drop-down to view or filter 'All/ Acknowledged/High severity' alerts. By default, the alerts remain in the system for 90 days irrespective of whether they are acknowledged.
If you have issues implementing the above procedure let us know by posting the issues on our community page at https://legacycommunity.tibco.com/answers/product/701891
To learn more about how to use Advanced Data Models and Advanced Search refer to our documentation at https://docs.tibco.com/products/tibco-loglogic-log-management-intelligence
For issues that require a support case open a case with us at https://support.tibco.com