Log messages virtually always follow a pattern: most log messages are generated based on a template, with different variables interpolated in between the static text. And the pattern is unfortunately not available after the log leaves your application.
Wouldn't it be great being able to tell how many and which patterns are there? Or how many logs match each pattern, and how often patterns associated - for example - with errors occur? Or even querying the interpolated variables as if they were log attributes?
Now you can do that all that and more, with the LogAI pattern recognition.
- As logs are ingested, our AI identifies patterns, extracts named variables and makes them available as log attributes.
- Logs can be filtered and grouped by pattern or by extracted attributes -- directly in the UI, or by referencing them in your PromQL expressions, in the query builder or even in the Triage!
This feature is currently in open beta, as we work on model improvements to increase the accuracy of the patterns and the quality of the extracted attributes.
Please note that:
- LogAI works on unstructured logs; logs with structure body, like JSON logs, are not modified by LogAI
- At the time of writing, LogAI does not extract the trace/span ID attributes for correlation of logs with tracing
- LogAI requires logs to have sufficient resource attributes for contextualization; the minimum combinations of resource attributes required are listed here