FIRE OPS (Anomaly detection and correlation)
- MSC DA 2016-2018
“Fire Ops” is an internal platform of UST Global for the anomaly detection and resolution internal IT infrastructure. Fire Ops is a 3-stage project in which aims at the complete automation in the field of anomaly detection. Fire Ops aims at making a self-learning anomaly detection system in involving the process of “Unsupervised Learning” in the Machine Learning and their algorithms to learn and understand the pattern of the CPU utilization and memory utilization. The first stage of this project is to detect anomaly from different types of logs such as system logs, IOT logs, server log, application log etc. In this project we use ELK stack for anomaly detection. ELK stack is the acronym for three open source projects: Elasticsearch, Logstash, Kibana. In our project elasticsearch act as database, Logstash is a platform used for log parsing with the help of grok debugger, Kibana lets users visualize data with charts and graphs in elasticsearch. When an anomaly is detected an alert is send to an email id with the help of a watcher. The second stage of this project is to find the root cause of each anomaly. For that, here we used some of the modules: Auto discovery, CMDB, Dependency Mapping, Correlation and Root cause analysis. In this project, we are using adaptive threshold method because it learns automatically. This project is useful for it infrastructure, healthcare, business etc. The future scope of our project is to predict and forecast the future patterns, availability and performance of an infrastructure.