As a transitional step, this site will temporarily be made Read-Only from July 8th until the new community launch. During this time, you can still search and read articles and discussions.

While the community is read-only, if you have questions or issues requiring TIBCO review/response, please access the new TIBCO Community and select "Ask A Question."

You will need to register or log in or register to engage in the new community.

Advanced Correlation Rules in TIBCO LogLogic® LMI

By:
Last updated:
11:09am Oct 03, 2019

Back to HomePage


TIBCO LogLogic® provides the industry's first enterprise class, end-to-end log management solution. Using LogLogic® log management solutions, IT organizations can analyze and archive log and machine data for the purpose of compliance and legal protection, decision support for security remediation, and increased system performance and improved availability of overall infrastructure.


LogLogic® LMI 6.2.0 allows creating correlation rules to correlate two or more log sources for use in alerting. It also helps to search historical data and analyze the patterns in the data. In order to create advanced alerts, we need to follow some steps. This article covers the lifecycle of an advanced correlation alert by explaining the steps required to create Advanced Correlation Alerts and interact with the advanced alerts

Note: This article assumes that you are familiar with advanced search features in LogLogic LMI. To know more about the terminology used in this article, check the documentation here. 

Throughout the article, the terms  1. "correlation blok" and "ECL blok"  2. "blok" and "rule" are used alternatively

1. Create a Correlation Filter Blok

The lifecycle of a correlation alert starts with a Correlation blok. Go to: Home > Management > Advanced Features > Bloks

Select ‘Correlation’ to create new Correlation blok. The Correlation blok uses Event Correlation Language (ECL). The ECL format is described here: https://docs.tibco.com/pub/loglmi/6.2.1/doc/html/GUID-043EB729-8744-4622-A2FE-7A97E39A46CD.html

Let us consider a simple Correlation blok:

USE LogLogic_Appliance

WITHIN 30m
EVENT GROUP [success] AT LEAST 1 EVENTS
WHERE [ll_deviceTypeID] ="17" AND [ll_eventStatus] = "success"
WITH THE SAME [ll_sourceUser],[ll_sourceIP]

EVENT GROUP [failed] AT LEAST 1 EVENTS
WHERE [ll_deviceTypeID] ="17" AND [ll_eventStatus] ="failed"
WITH THE SAME [ll_sourceUser],[ll_sourceIP]

CORRELATION
success->[ll_sourceIP]== failed->[ll_sourceIP]
success->[ll_sourceUser]== failed->[ll_sourceUser]

There are some constructs to know in this ECL blok. We'll go over them one-by-one:

  • USE: The list of log sources (Data Models)  used by the rule. Also supports multiple log sources (Comma separated)

  • WITHIN: The time bucket scope to consider the events from log sources. Defined in seconds, minutes, hours, or days

  • EVENT GROUP: Each event group describes the criteria that must combine events to be grouped together as part of the blok/ rule. This is equivalent to a single search in EQL. There can be multiple event groups in a blok.

  • WITH THE SAME: Represents GROUP BY field(s)

  • HAVING: Represents the filter criteria on the aggregated functions

  • CORRELATION: A join condition describing which fields should be equal in two event groups

Test the correlation blok and save it with a name. E.g. TestECLBlok.

2. Test the blok on historical data (query)

You can test the ECL blok by executing a query by choosing the ECL blok. (This query is a snapshot query) The correlation blok name should have “correlation” prefix while executing the query on the UI. For example. correlation.TestECLBlok

Provide a time filter for the query example. -1h. You will notice that the matching results are grouped by the time buckets defined in the WITHIN criteria.

3. Create a Trigger

Once you have verified the ECL blok is working as expected on the historical data, it is time to create a Trigger using the ECL blok.

Go to: Home > Management > Advanced Features > Rule Management

Click on add a new 'Trigger'. Enter trigger details like 'Name', 'Description', 'Severity', 'Category', and finally select the Correlation Blok created in the step above. Optionally you can create a new Correlation Blok by clicking on the Blok icon on the same page.

Optionally configure the Notification(s):

Email Notification: Enter To, CC, Subject, Message fields

Syslog Notification: Enter Host, Protocol, Facility, Severity, Message fields

Note: After creating the trigger, you will need to “Sync all triggers”. This is an important step, it reinitializes the correlation instance with all the existing triggers including the newly created trigger.

The deployed triggers internally subscribe to real-time logs in LMI and evaluate the rules for any matching filter (i.e. correlation) criteria.

As the trigger(s) are evaluated, Advanced Alerts would be generated with optionally configured Email or Syslog Notifications.

4. Advanced Alerts Dashboard

Go to Home > Alerts > Advanced Alerts.

The 'Alerts' page shows all the Advanced Alerts triggered by various rules. Each 'Alert' shows 'Severity', 'SLA' (Time based on the severity - high severity alert needs to be addressed sooner compared to low severity alert, etc)

Clicking on an 'Alert' shows the alert details pop-over - more details, the rule, the actual evaluated values which triggered the alert and also a drill down URL which redirects to the Advanced Search page with a pre-filled query to show all the actual log events responsible for the alert.

The operator can 'Acknowledge' one or multiple Alerts. The acknowledgment pop-over allows the operator to provide a comment.

Note: The default 'Alerts' view shows only 'Unacknowledged' alerts. Use the drop-down to view or filter 'All/ Acknowledged/High severity' alerts. By default, the alerts remain in the system for 90 days irrespective of whether they are acknowledged.


Additional Resources