Fraud Detection in E-Commerce Transactions Using Autoencoder Anomaly Scoring and XGBoost Classification

Abstract

The rapid expansion of e-commerce platforms has intensified the risk of digital transaction fraud, which is particularly challenging to detect due to highly imbalanced datasets where fraudulent transactions represent a small minority. This study proposes a two-stage hybrid fraud detection model that integrates an Autoencoder for unsupervised anomaly detection with XGBoost as a supervised classifier. The objective is to evaluate whether incorporating reconstruction error scores from the Autoencoder as an additional feature improves XGBoost classification performance on highly imbalanced e-commerce fraud data. The dataset used is the Credit Card Fraud Detection dataset from Kaggle (ULB), consisting of 284,807 transactions with a fraud ratio of 0.17%. The research pipeline includes stratified train-validation-test splitting, StandardScaler normalization, Autoencoder training exclusively on non-fraud data to produce anomaly scores (AE_Score), and XGBoost training with scale_pos_weight to handle class imbalance. Threshold optimization was performed using the precision-recall curve on the validation set. Results demonstrate that the two-stage model achieved a ROC-AUC of 0.9749, PR-AUC of 0.8442, precision of 0.8971, recall of 0.8243, and F1-score of 0.8592 on the test set. The AE_Score showed strong discriminative power, with a fraud mean of 4.79 compared to 0.02 for non-fraud transactions. These findings confirm that integrating Autoencoder-based anomaly scoring into gradient boosting classification effectively addresses data imbalance and improves fraud detection performance in large-scale e-commerce environments.

GS Cited Analysis