All Issue

2024 Vol.42, Issue 5 Preview Page

Article

Development and Evaluation of Closed Lane Avoidance Strategy Using CAV Control Algorithm Based on Reinforcement Learning

강화학습 기반 CAV 거동 제어 알고리즘을 통한 통제차로 회피 전략 개발 및 평가

Seunghwan KIM1
,
Juneyoung PARK2*
,
Inchul YANG3
김 승환1
,
박 준영2*
,
양 인철3
1Master’s Course, Department of Transportation and Logistics Engineering, Hanyang University, Gyeonggi 15588, Korea
2Associate Professor, Department of Transportation & Logistics Engineering / Smart City Engineering, Hanyang University, Gyeonggi 15588, Korea
3Reaserch Fellow, Korea Institute of Civil Engineering and Building Technology, Gyeonggi 10223, Korea
1한양대학교 교통물류공학과 석사과정
2한양대학교 교통물류공학과, 스마트시티공학과 부교수
3한국건설기술연구원 도로교통연구본부 연구위원
*Corresponding Author
31 October 2024. pp. 565-578
PDF XML Citation
Abstract

Autonomous vehicles (AVs) are expected to positively impact traffic flow by eliminating human factors and ensuring more ideal and consistent driving behaviors. However, due to the limitations of vehicle-centric autonomy, AVs still fail to appropriately handle various unexpected situations, adversely affecting traffic flow and causing crashes. Therefore, for AVs to achieve their anticipated positive impact on traffic flow, it is necessary to evolve into Connected Autonomous Vehicles (CAVs), including the cooperation of infrastructure based on Vehicle to Everything (V2X) communication. The cooperative driving effect of CAVs will be evident in the case of lane closed situations caused by unexpected events such as traffic accidents or construction. If CAVs can acquire information about the lane closed situation through infrastructure communication in advance and control their maneuvers of CAVs appropriately, it is expected that the systematic movement of CAVs will alleviate issues such as bottlenecks caused by closed lanes and ensure a smoother traffic flow. This indicates the necessity for research on strategies to smoothly avoid and pass through closed lane situations in future LV4 autonomous driving situation where CAV maneuver control is possible. Reinforcement learning, which can accommodate large state spaces like traffic situations, has recently emerged as a promising framework for addressing the control challenges of CAVs, which must flexibly respond to real-time dynamic traffic conditions. Therefore, this study developed a reinforcement learning-based strategy for avoiding closed lanes and evaluated its impact on traffic flow, thereby demonstrating the utility of reinforcement learning as a methodology for developing closed lane avoidance strategies for CAVs in future connected environments.

Keywords
CAV
control algorithm
DQN
reinforcement learning
VISSIM

자율주행차량은 운전자의 인적요인을 제거하여 보다 이상적이고 일정한 주행행태를 교통류에 반영하여 이를 통해 긍정적인 영향을 미칠 것으로 기대되는 반면 아직 차량 중심 자율주행의 한계로 인해 다양한 돌발상황에서 적절한 대처를 하지 못하고 교통류에 악영향을 끼치며 사고 또한 끊이지 않고 있다. 이에 따라 자율주행차량이 교통류에 기대하는 만큼의 긍정적인 영향을 가져오기 위해선 V2X(Vehicle to Everything, V2X) 통신 기반 인프라의 협력을 포함한 자율협력주행차량(Connected Automated Vehicle, CAV)으로 발전할 필요가 있다. 교통사고나 공사와 같은 돌발상황으로 인한 차로 통제상황의 경우 CAV의 협력 주행 효과는 두드러질 것이다. 통제상황을 인프라 통신을 통해 CAV가 사전에 정보를 획득하고 이를 통해 CAV들의 거동을 적절히 제어할 수 있다면 체계적인 CAV의 움직임으로 인해 기존의 통제상황으로 인해 발생하는 병목현상 등의 문제점을 해소하고 교통류를 원활하게 유도할 수 있을 것으로 기대된다. 이는 CAV 기동제어가 가능한 향후 LV4 자율주행 상황에서 폐쇄차로 상황을 원활하게 회피하고 통과하기 위한 전략에 대한 연구가 필요함을 의미한다. 교통상황과 같은 큰 차원의 상태 공간을 고려할 수 있는 강화학습은 실시간 동적으로 변하는 교통상황에 따라 유연하게 대응할 수 있어야 하는 CAV의 제어 문제를 해결하기 위한 유망한 프레임워크로써 최근 부상하고 있다. 이에 따라 본 연구에서는 통제 차로를 회피하기 위한 강화학습 기반 전략을 개발하고 교통류에 미치는 영향을 평가함으로써 향후 커넥티드 환경에서 CAV의 통제 차로 회피 전략 개발 방법론으로서 강화학습의 유용성을 입증하였다.

키워드
CAV
제어 알고리즘
DQN
강화학습
VISSIM
References
1

Atkins (2016), Research on the Impacts of Connected and Autonomous Vehicles (CAVs) on Traffic Flow, Modelling and Analysis Technical Report, Department for Transport.

2

Chen Wang, Yuanchang Xie, Helai Huang, Pan Liu (2021), A Review of Surrogate Safety Measures and Their Applications in Connected and Automated Vehicles Safety Modeling, Accident Analysis & Prevention, 157, 106157.

10.1016/j.aap.2021.10615733975090
3

Granà A., Curto S., Petralia A., Giuffrè T. (2024), Connected Automated and Human-Driven Vehicle Mixed Traffic in Urban Freeway Interchanges: Safety Analysis and Design Assumptions, Vehicles, 6(2), 693-710.

10.3390/vehicles6020032
4

Guériau M., Billot R., El Faouzi N. E., Monteil J., Armetta F., Hassas S. (2016), How to Assess the Benefits of Connected Vehicles? A Simulation Framework for the Design of Cooperative Traffic Management Strategies, Transportation Research Part C: Emerging Technologies, 67, 266-279.

10.1016/j.trc.2016.01.020
5

Jang K.J., Yu S.M. (2021), A Study on Autonomous Vehicle Lane Change Method using Cooperative Maneuver, Journal of the Koread Contents Association, 21(1),139-146.

6

Jo young (2023), Mixed Traffic Management Strategy for Proactive Safety in Cooperative Driving Automation Systems (Doctoral dissertation), Hanyang University, Seoul.

7

Jung Sangchul, Lee Seungjae, Kim Jooyoung (2021), The Real-Time Signal Control System using Reinforcement Learning considering Priority Signaling for Emergency Vehicle, J. Korean Soc. Transp., 39(3), Korean Society of Transportation, 329-344.

10.7470/jkst.2021.39.3.329
8

Kang K., Park N., Park J., Abdel‐Aty M. (2024), Deep Q‐network Learning‐based Active Speed Management under Autonomous Driving Environments, Computer‐Aided Civil and Infrastructure Engineering.

10.1111/mice.13283
9

Kim Y., Kang K., Park N., Park J., Oh C. (2024), Reinforcement Learning Approach to Develop Variable Speed Limit Strategy using Vehicle Data and Simulations, Journal of Intelligent Transportation Systems, 1-18.

10.1080/15472450.2024.2312808
10

Mark Mario M., Qingyun T., Long T. T., Hai L. V. (2018), Studying the Safety Impact of Autonomous Vehicles using Simulation-Based Surrogate Safety Measures, Journal of Advanced Transportation, 22, 1-11

10.1155/2018/6135183
11

Ma Q., Zhang S., Zhou Q. (2021), Development of a Conflict-free Unsignalized Intersection Organization Method for Multiple Connected and Autonomous Vehicles, Plos One, 16(3), e0249170.

10.1371/journal.pone.024917033784368PMC8009356
12

Minh V., Kavukcuoglu K., Silver D., Graves A., Antonoglou I., Wierstra D., Riedmiller M. (2013), Playing Atari with Deep Reinforcement Learning, arXiv preprint arXiv:1312.5602.

13

Park J.E., Byun W., Kim Y., Ahn H., Doh K. S. (2021), The Impact of Automated Vehicles on Traffic Flow and Road Capacity on Urban Road Networks, journal of Advanced Transportation, 03, 1-10

10.1155/2021/8404951
14

Park N.R., Kim Y.J., Kang K.W., Park J.Y., Oh C. (2022), Assessing The Real-Time Crash Risk-based Variable Speed Limit Strategy using Reinforcement Learning, The Korea Institute of Intelligent Transport Systems International Conference, 494-496.

15

Ren T., Xie Y., Jiang L. (2020), Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment, Transportation Research Record, 2674(10), 363-374

10.1177/0361198120935873
16

Roy A., Hossain M., Muromachi Y. (2022), A Deep Reinforcement Learning-based Intelligent Intervention Framework for Real-time Proactive Road Safety Management, Accident Analysis & Prevention, 165, 106512.

10.1016/j.aap.2021.10651234890919
17

Suh G. H., Kang S.E., Yun Y.K., Yun K. S. (2022), A Strategy for Controlling Road Network and Autonomous Vehicles Under Incident or Disaster Situation in Mixed Traffic, The Korea Institute of Intelligent Transport Systems International Conference, 140-143.

18

Uhlemann E. (2015), Introducing Connected Vehicles [Connected Vehicles], IEEE Vehicular Technology Magazine, 10(1), 23-31.

10.1109/MVT.2015.2390920
19

Wang H., Hao W., So J., Xiao X., Chen Z., Hu J. (2023), A Faster Cooperative Lane Change Controller Enabled by Formulating in Spatial Domain, IEEE Transactions on Intelligent Vehicles.

10.1109/TIV.2023.3317957
20

Zhou W., Chen D., Yan J., Li Z., Yin H., Ge W. (2022), Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic, Autonomous Intelligent Systems, 2(1), 5.

10.1007/s43684-022-00023-5
Information
  • Publisher :Korean Society of Transportation
  • Publisher(Ko) :대한교통학회
  • Journal Title :Journal of Korean Society of Transportation
  • Journal Title(Ko) :대한교통학회지
  • Volume : 42
  • No :5
  • Pages :565-578
  • Received Date : 2024-05-24
  • Revised Date : 2024-06-13
  • Accepted Date : 2024-08-19
  • DOI :https://doi.org/10.7470/jkst.2024.42.5.565
Journal Informaiton Journal of Korean Society of Transportation Journal of Korean Society of Transportation
Archives
: 2015 Vol.34~
  • NRF
  • KOFST
  • crossref cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close