The Real-Time Signal Control System Using Reinforcement Learning considering Priority Signaling for Emergency Vehicle

Sangchul Jung; Seungjae Lee; Jooyoung Kim

doi:10.7470/jkst.2021.39.3.329

All Issue

2021 Vol.39, Issue 3 Preview Page Next Page

Article

The Real-Time Signal Control System Using Reinforcement Learning considering Priority Signaling for Emergency Vehicle 긴급차량 우선신호 부여를 고려한 강화학습 기반 지체 시간 최소화 실시간 신호 제어 체계 개발

30 June 2021. pp. 329-344

PDF XML

Abstract

Recently, with the development of autonomous vehicles and V2X (Vehicle-to-Everyting) technology, researches are being conducted to reduce vehicle delay time and unnecessary stop-and-go phenomenon at the urban signalized intersection. In particular, current urban networks, in which the fixed-time traffic signal control system is dominant, can not reflect the real-time traffic situation in the signaling system. Even if the autonomous vehicle is commercialized, it could not exhibit its performance. At this time, if the artificial intelligence traffic signal control system based on accurate real-time vehicle information obtained through V2I technology is developed, more efficient traffic signal control will be possible and it will affect not only traffic flow but also the performance of the V2X technology-equipped vehicles. Especially, when there is unnecessary waiting time for the emergency vehicle in the urban intersection, it may affect the travel time to reach the destination. In case of driving while leaving the lane for the purpose of shortening the travel time, it can threaten the safety of a patient and surrounding vehicles. At this point, if the traffic signal control system that detects the entrance of the emergency vehicle to an intersection through the V2I technology and gives the priority signal to the emergency vehicle is developed, the safer emergency vehicle operation could be guaranteed. In this paper, the change of traffic flow according to the change of traffic signal control system is analyzed based on micro traffic information at single signalized intersection with V2I technology commercialization. A reinforcement learning model that expresses the optimum signal in real-time by learning the traffic information at the single signalized intersection was constructed based on the deep learning through the tensor flow. The performance of the developed traffic signal control system was verified through Vissim. The developed reinforcement learning model expresses a specific signal phase through real-time traffic information and expresses appropriate signal display in the changed network situation. The traffic signal control system developed in this paper is expected to contribute to achieve system optimization in a complex road network and to contribute to the analysis of traffic flow. In addition, it will contribute to building a smart city when autonomous vehicles are operated in the future V2X environment.

Keywords

artificial intelligence

priority signal

reinforcement learning

traffic signal control

v2i

최근, 자율주행차량 및 V2X 기술의 개발에 따라 도심부 신호 교차로에서 차량의 지체시간과 불필요한 stop-and-go 현상을 줄이기 위한 연구가 시도되고 있다. 특히, 정주기식 신호 제어 체계가 주를 이루는 현재의 도시 네트워크에서는 실시간 교통상황을 신호체계에 반영할 수 없어 자율주행차량이 등장하여 상용화된다고 하더라도 그 성능을 모두 발휘할 수 없다. 이 때, V2I 기술을 통해 얻어지는 정확한 실시간 차량 정보를 바탕으로 한 인공지능 신호 제어 체계를 개발한다면 정주기식 신호 제어 체계에 비해 불필요한 신호 대기 시간을 감소시킬 수 있어 더욱 효율적인 신호제어가 가능하며 교통류 흐름뿐만 아니라 V2X 기술 탑재 차량의 성능 향상에도 영향을 미칠 것이다. 특히, 도심부 교차로에서 긴급 차량이 목적지까지 도달하는 데에는 불필요한 신호 대기 시간이 존재할 경우 현장까지의 통행 시간에 영향을 끼칠 수 있으며, 통행 시간 단축을 위해 차선을 이탈해가면서 운행할 경우 오히려 긴급 차량에 탑승한 환자 및 주변 차량의 안전을 위협할 수 있다. 이 때, V2I 기술을 통해 긴급 차량의 교차로 진입을 감지하고, 긴급 차량에 우선 신호를 부여하는 신호 제어 체계를 개발한다면 보다 안전한 긴급 차량의 운행을 보장할 수 있을 것이다. 본 논문에서는 V2I 기술 상용화를 전제로 단일 신호교차로에서 미시적 교통 정보를 바탕으로 신호 제어 체계의 변화에 따른 교통류 변화를 분석하였다. 단일 신호 교차로에서 이동류별 실시간 교통 정보를 학습하여 실시간으로 최적 신호를 표출하는 강화학습 모형을 구축하였으며, 개발한 신호 제어 체계의 성능은 미시적 교통 시뮬레이터인 Vissim을 통해 측정하였다. 개발한 강화학습 모형은 이동류별 실시간 교통 정보를 통해 특정한 신호를 표출하고, 변화된 네트워크 상황에 적절한 신호 현시를 표출한다. 본 논문에서 개발한 신호 제어 체계를 통해 복잡한 도로 네트워크에서 체계 최적을 달성할 수 있는 신호 제어 체계를 개발하는 바탕이 되고자 하며, 교통류 흐름을 분석하는 데 기여할 것으로 기대된다. 또한, 미래 V2X 환경에서 자율주행차량이 운영될 때 스마트시티를 구축하는 데 기여할 것이다.

키워드

강화학습

신호 제어

우선 신호

인공지능

V2X

References

Alvarez I. et al. (2007), Urban Traffic Control Problem via a Game Theory Application, 46th IEEE Conference on decision control, 1(14), 2957-2961. 10.1109/CDC.2007.4434820

Baher A. et al. (2003), Reinforcement Learning for True Adaptive Traffic Signal Control, Journal of Transportation Engineering, 129(3), 278-285. 10.1061/(ASCE)0733-947X(2003)129:3(278)

Chen C. et al. (2009), Adaptive Traffic Signal Control Using Approximate Dynamic Programming, Transportation Research Part C, 17, 456-474. 10.1016/j.trc.2009.04.005

Cheng S.-F., Epelman M. A., Smith R. L. (2006), CoSIGN: A Parallel Algorithm for Coordinated Traffic Control System, IEEE Transactions on Intelligent Transportation Systems, 7(4), 551-564. 10.1109/TITS.2006.884617

Choy M. C., Srinivasan D., Cheu R. L. (2006), Neural Networks for Continuous Online Learning and Control, IEEE Transactions on neural network, 17(6), 1511-1531. 10.1109/TNN.2006.88171017131665

Elise van der P. (2016), Deep Reinforcement Learning for Coordination in Traffic Light Control, Master Thesis, University of Amsterdam.

Enrique D. F., Pradeep K. K. (2000), Multi Agent Collaboration Using Distributed Value Functions, IEEE Intelligent Vehicles Symposium, 404-409.

Han E. et al. (2016), The Development of an Algorithm for the Optimal Signal Control for Isolated Intersections under V2X Communication Environment, The Journal of The Korea Institute of Intelligent Transport Systems, 15(6), 90-101. 10.12815/kits.2016.15.6.090

James C. S., Daniel C. C. (1997), Traffic-responsive Signal Timing for System-wide Traffic Control, Transportation Research-part C, 5(3), 153-163. 10.1016/S0968-090X(97)00012-0

Jan-Dirk et al. (2007), Multi-objective Signal Control of Urban Junctions: Framework and a London Case Study, Transportation Research Part C, 16, 454-470. 10.1016/j.trc.2007.09.004

Jian Q. et al. (2011), Two-stage Fuzzy Logic Controller for Signalized Intersection, IEEE Transactions on Systems, Man, and Cybernetics-part A: Systems and Humans, 41(1), 178-184. 10.1109/TSMCA.2010.2052606

Lee J.-H., Lee K. H. (1999), Distributed and Cooperative Fuzzy Controllers for Traffic Intersections Group, IEEE Transactions on Systems, Man, and Cybernetics-part C: Applications and Reviews, 29(2), 263-271. 10.1109/5326.760570

Lee M. H. (2017), Development of Traffic Signal Control Methodology Using Average Travel Time, PhD Dissertation, University of Seoul.

Mingming Z., Shuguang Z., Jie L., Yuda Q. (2009), Multi-phase Urban Traffic Signal Real-time Control with Multi-objective Discrete Differential Evolution, 2009 International conference on electronic computer technology, 296-300.

Mnih V. et al. (2015), Human-level Control through Deep Reinforcement Learning, Nature, 518, 529-533. 10.1038/nature1423625719670

Park S. Y., Jeong Y. G. (2016), A Real-time Traffic Signal Control Algorithm based on Travel Time and Occupancy Rate, The Journal of the Korea Contents Association, 16(8), 671-680. 10.5392/JKCA.2016.16.08.671

Sutton R. S., Barto A. G. (1998), Reinforcement Learning: An Introduction, Cambridge, The MIT Press, Cambridge. 10.1109/TNN.1998.712192

Van Hasselt H. et al. (2016), Deep Reinforcement Learning with Double Q-learning, Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).

Zhang J. et al. (2019), City Brain: Practice of Large-scale Artificial Intelligence in the Real World, IET Smart Cities, 1(1), 28-37. 10.1049/iet-smc.2019.0034

Information

Publisher :Korean Society of Transportation
Publisher(Ko) :대한교통학회
Journal Title :Journal of Korean Society of Transportation
Journal Title(Ko) :대한교통학회지
Volume : 39
No :3
Pages :329-344
Received Date : 2020-11-25
Revised Date : 2021-01-27
Accepted Date : 2021-03-23
DOI :https://doi.org/10.7470/jkst.2021.39.3.329

Journal of Korean Society of Transportation ISSN:1229-1366(Print) 2234-4217(Online) 대한교통학회지

All Issue