Rediscovering heuristics: A litmus test for RL in networked systems

Abstract

Reinforcement learning (RL) holds promise for outperforming human-designed heuristics in networked systems, yet training remains brittle and opaque. We propose heuristic rediscovery as a litmus test: a compact policy capable of representing a strong heuristic should reliably learn to match its behavior and performance. We visualize the return landscape, optimization path, and correctness properties in this compact policy space to diagnose training failures. Our experiments on adaptive bitrate streaming and congestion control reveal when and why RL training succeeds or fails.