Reinforcement Learning Evaluation for Dependent Task Offloading in Mobile Edge Computing Systems

Abstract

Dependent-task applications represented as directed acyclic graphs require each task to be executed locally or offloaded to a mobile edge computing server, creating a sequential decision problem in which latency, communication cost, and device energy are jointly affected. This study presents a controlled evaluation of deep reinforcement learning for task offloading under a common simulator, dataset, and testing protocol. A recurrent policy trained with proximal policy optimization and a double deep Q-network are reproduced as baselines. Three extensions are evaluated: a Transformer policy, a discrete soft actor-critic agent, and a stabilized proximal policy optimization configuration combining advantage normalization, scheduled optimization parameters, divergence-based early stopping, and orthogonal initialization. All agents and five heuristics are tested on graphs of ten to fifty tasks under latency-oriented and energy-aware quality of experience objectives, using five random seeds and paired significance tests. Across both analyses, stabilized PPO achieved the highest mean QoE (0.35–0.43) and was among the fastest-converging methods, reaching 90% of peak performance within 5–20 updates. It outperformed HEFT by 25–70%, but its gain over PPO-LSTM was below 1% and significant at only some bandwidths. A component-wise ablation shows that advantage normalization drives this benefit. The Transformer policy is competitive on the tested graph sizes, while discrete soft actor-critic reduces device energy more effectively than latency. Training stabilization thus reliably enhances dependency-aware task offloading without increasing model size.

GS Cited Analysis