SXAD: Shapely eXplainable AI-Based Anomaly Detection Using Log Data
- Kashif Alam(corresponding author),
- ,
- Gabriel Avelino Sampedro,
- Vincent Karovic(corresponding author),
- Tariq Naeem
- Air University, Islamabad,
- De La Salle-College Of Saint Benilde,
- Comenius University
Research Output:
Contribution to journal
Article
Research
Peer-reviewOpen access
Abstract
<p>Artificial Intelligence (AI) has made tremendous progress in anomaly detection. However, AI models work as a black-box, making it challenging to provide reasoning behind their judgments in a Log Anomaly Detection (LAD). To the rescue, Explainable Artificial Intelligence (XAI) improves system log analysis. It follows a white-box model for transparency, understandability, trustworthiness, and dependability of Machine Learning (ML) and Deep Learning (DL) Models. In addition, Shapely Additive Explanation (SHAP), added to system dynamics, makes informed judgments and adoptable proactive methods to optimize system functionality and reliability. Therefore, this paper proposed the Shapely eXplainable Anomaly Detection (SXAD) framework to identify different events (features) that impact the models' interpretability, trustworthiness, and explainability. The framework utilizes the Kernel SHAP approach, which is based on Shapley values principle, providing an innovative approach to event selection and identifying specific events causing abnormal behavior. This study addresses the LAD by transforming it from a black-box model into a white-box one, leveraging XAI to make it transparent, interpretable, explainable, and dependable. It utilizes benchmark data from the Hadoop Distributed File System (HDFS), organized using a Drain parser, and employs several ML models, such as Decision Tree (DT), Random Forest (RF), and Gradient Boosting (GB). These models achieve impressive accuracy rates of 99.99%, 99.85%, and 99.99%, respectively. Our contribution are novel because no earlier work has been done in the area of Log Anomaly Detection (LAD) with integration of XAI-SHAP.</p>
Research Output Information
Output type
Research Output:
Contribution to journal
Article
Research
Peer-reviewOriginal language
EnglishPages from-to (Number of pages)
Pages 95659-95672 (14 pages)Journal (Volume, Issue Number)
IEEE Access (Volume 12)Publication milestones
- Published - 2024
Publication status
Published - 2024
Publication IDs
- Scopus: 85198304309
Documents and links
Publication metrics
Metrics
SciVal
FWCI
3.37
PlumX, opens in new tab
Citation count
37
Captures
82
