FIRE OPS (Anomaly detection and correlation)

“Fire Ops” is an internal platform of UST Global for the anomaly detection and resolution internal
IT infrastructure. Fire Ops is a 3-stage project in which aims at the complete automation in the
field of anomaly detection. Fire Ops aims at making a self-learning anomaly detection system in
involving the process of “Unsupervised Learning” in the Machine Learning and their algorithms
to learn and understand the pattern of the CPU utilization and memory utilization. The first stage
of this project is to detect anomaly from different types of logs such as system logs, IOT logs,
server log, application log etc. In this project we use ELK stack for anomaly detection. ELK stack
is the acronym for three open source projects: Elasticsearch, Logstash, Kibana. In our project
elasticsearch act as database, Logstash is a platform used for log parsing with the help of grok
debugger, Kibana lets users visualize data with charts and graphs in elasticsearch. When an
anomaly is detected an alert is send to an email id with the help of a watcher. The second stage of
this project is to find the root cause of each anomaly. For that, here we used some of the modules:
Auto discovery, CMDB, Dependency Mapping, Correlation and Root cause analysis. In this
project, we are using adaptive threshold method because it learns automatically. This project is
useful for it infrastructure, healthcare, business etc. The future scope of our project is to predict
and forecast the future patterns, availability and performance of an infrastructure.

MSC DA 2016-2018 INT Dr. Manoj Kumar T K