Anomaly detection

Anomaly detection is used in many parts of modern society and is a high demand field in future industries. The anomaly detection technology can be applied in various fields including industrial machine anomaly detection, medicine and health care, fraud detection in banks and stocks, intrusion detection, and defect detection in certain patterns of images.

There are four main stages of technology for identifying system problems. The first stage is to rely on the domain knowledge of the system and detect a failure based on a predetermined rule. The second stage is an abnormality detection stage using various models. Unlike the first stage, it is used to create a model that detects anomalies by analyzing sufficient amount of given data using artificial intelligence techniques. The third stage is to diagnose the failure based on various models. It is mainly used to create a model that not only detects the abnormality but also finds the cause of the failure. In this step, Other than normal data and abnormal data, data labeling by an expert is required for determining the causes of each abnormal data to be classified as abnormal data. Finally, the fourth stage is to create a model that analyzes data trends and predicts when a failure may occur in the future. With these various stages, you can share your skills to identify system problems.

The anomaly detection technology has also been approached in various ways, and it attempts to detect anomalies with various techniques, such as classification-based anomaly detection, nearest data-based anomaly detection, clustering-based anomaly detection, probability-based anomaly detection, and information theory anomaly detection. Traditionally used methods are isolation forest method and one-class SVM method. Recently, various attempts based on deep learning have been made such as the Deep One-Class Classification method that developed the One class SVM technique, the technique using the Autoencoding and Gaussian Mixture Model (GMM), and the technique using Generative Adversarial Networks (GAN) and Long Short-Term Memory Models (LSTM).