RL Controllers Can Self-Organize Traffic Into 'Green Waves' Without Formal Coordination, Study Shows
A new study applies a capacity region perspective to multi-junction traffic networks, demonstrating that RL-based traffic controllers can achieve emergent coordination without explicit communication.
The Research
Researchers trained and evaluated three types of RL controllers for urban traffic corridor networks:
- Centralized: Single controller manages all intersections
- Fully decentralized: Each intersection operates independently
- Parameter-sharing decentralized: Shared architecture but independent execution
Key Finding: Emergent Green Waves
When parameter-sharing controllers trained on one network were deployed on a larger network than originally trained on, traffic self-organized into 'green waves' — synchronized signal patterns that allow vehicles to flow through multiple intersections without stopping.
"Even though the junctions are not formally coordinated, traffic may self-organize into green waves."
This is remarkable because:
- No explicit coordination protocol between intersections
- Controllers were trained on a smaller network
- Green waves emerged spontaneously on the larger deployment
Compared to Classical Methods
The RL controllers were benchmarked against MaxPressure, a classical traffic control algorithm:
- RL controllers matched or exceeded MaxPressure in capacity region coverage
- Average travel times (ATT) were competitive
- Parameter-sharing approach offered the best scalability
Why It Matters
- Smart cities: RL traffic control can reduce congestion without expensive centralized infrastructure
- Scalability: Parameter-sharing enables deployment to new areas with zero additional training
- Emergent behavior: Simple rules leading to complex coordination — a hallmark of multi-agent AI systems
- Practical impact: Green waves typically require expensive coordinated timing systems; RL achieves this naturally