All Issue

2025 Vol.43, Issue 1

Article

Determining Optimal Personal Mobility Service Sites using Markov Decision Process

마코브 디시젼 프로세스 기반 최적 공유 Personal Mobility 배치지역 선정

Sangjae LEE1
,
Woowon KIM2
,
Juneyoung PARK3*
이 상재1
,
김 우원2
,
박 준영3*
1Postdoctoral Researcher, Department of Transportation and Logistics Engineering, Hanyang University, Gyeonggi 15588, Korea
2Master Student, Department of Transportation and Logistics Engineering, Hanyang University, Gyeonggi 15588, Korea
3Assosiate Professor, Department of Transportation & Logistics Engineering / Smart City Engineering, Hanyang University, Gyeonggi 15588, Korea
1한양대학교 교통물류공학과 박사후과정
2한양대학교 교통물류공학과 석사과정
3한양대학교 교통물류공학과, 스마트시티공학과 부교수
*Corresponding Author
28 February 2025. pp. 1-14
PDF XML Citation
Abstract

The proper deployment of shared personal mobility (PM) can increase mobility for residents who have difficulty traveling short distances to their destinations. This study utilizes Geographic Information System (GIS) and reinforcement learning to find the optimal locations for shared PMs. This study focused on Seoul, a city with open source data and a large population and PM usage. Among the data provided by the Seoul Open Data Plaza, we used public transportation-related data, socioeconomic indicators, road and terrain data, and shared mobility data. The data was inputted after subdividing the area to be analyzed into a hexagonal grid structure, which minimizes decision making for computation. The reinforcement learning structure is composed of the commonly used Markov decision process (MDP). The MDP is based on Open AI GYM, which can build a simple reinforcement learning environment. The reward function of the MDP is designed so that the smaller the distance required for PM placement, the higher the reward. The optimization of reinforcement learning was performed through the Q-learning algorithm, which is simple and applicable in various environments. Every hour, PM users generated from random points were moved according to the collected shared mobility start and end points, and the predicted PM deployment areas were visualized using QGIS. The analysis showed that the predicted PM deployment areas are Hwayang Intersection and Gasan Digital Complex, which are predicted to have a large number of commuters near industrial areas. In addition, Jamsil Station and Magoknaru Station, which are predicted to use PM for leisure purposes, are predicted to need PM deployment. The results of this study can improve the operational efficiency of service providers and the mobility of service users, and can help policy developers prepare legal grounds.

Keywords
big data
geographic information system
markov decision process
reinforced learning
shared personal mobility

공유 Personal mobility(PM)의 적절한 배치를 통해 목적지까지 단거리 이동이 불편한 거주민의 Mobility를 증가시킬 수 있다. 본 연구는 지리정보시스템(Geographic Information System; GIS) 및 강화학습을 활용하여 공유 PM의 최적 배치지역을 선정하였다. 분석대상지역은 오픈소스 데이터를 제공하고 유동인구 및 PM 이용이 많은 서울시를 선정하였다. 서울 열린 데이터 광장에서 제공하는 데이터 중, 분석에 활용한 데이터는 대중교통 관련 데이터, 사회경제지표, 도로 및 지형 데이터와 공유 모빌리티 데이터이다. 분석대상지역을 헥사곤 그리드로 세분화 후 데이터를 입력하여, 연산을 위한 의사결정 최소화를 수행하였다. 강화학습 구조는 주로 활용되는 Markov decision process(MDP)로 구성하였다. MDP는 간단한 강화학습 환경을 구축가능한 Open AI GYM을 기반으로 구축하였다. MDP의 보상함수는 PM 배치를 위해 필요한 이동거리가 작을수록 더 많은 보상을 줄 수 있도록 설계하였다. 강화학습의 최적화는 단순하고 다양한 환경에서 적용가능한 Q-learning algorithm을 통해 수행하였다. 매 시간 마다 무작위 지점에서 생성된 PM 이용객은 수집된 공유 모빌리티 시·종점에 따라 이동하고, 이동 결과에 따라 예측된 PM 배치지역은 QGIS를 활용하여 시각화 하였다. 분석결과 예측된 PM 배치지역은 공업지역 인근의 출·퇴근 이용객이 많을것으로 예측되는 화양사거리, 가산디지털단지역으로 나타났다. 그리고 PM을 여가목적으로 이용할 것으로 예측되는 지역인 잠실역과 마곡나루역이 PM 배치가 필요한 것으로 예측되었다. 본 연구결과를 통해 서비스 업체의 운영효율성 및 서비스 이용자의 이동성을 향상 시킬 수 있으며, 정책개발자의 법적 근거마련에 도움을 줄 수 있다.

키워드
빅데이터
지리정보시스템
마코브 디시젼 프로세스
강화학습
공유 개인형 이동수단
References
1

Aziz H. A., Nagle N. N., Morton A. M., Hilliard M. R., White D. A., Stewart R. N. (2018), Exploring The Impact of Walk-Bike Infrastructure, Safety Perception, And Built-Environment on Active Transportation Mode Choice: A Random Parameter Model Using New York City Commuter Data, Transportation, 45, 1207-1229.

10.1007/s11116-017-9760-8
2

Brodsky I. (2018), H3: Uber's Hexagonal Hierarchical Spatial Index, https://eng.uber.com/h3/, 2019. 06. 22

3

Chen J., Yang S. T., Li H. W., Zhang B., Lv J. R. (2013), Research on Geographical Environment Unit Division Based on the Method of Natural Breaks (Jenks), The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 40, 47-50.

10.5194/isprsarchives-XL-4-W3-47-2013
4

Currie G. (2010), Quantifying Spatial Gaps in Public Transport Supply Based on Social Needs, Journal of Transport Geography, 18(1), 31-41.

10.1016/j.jtrangeo.2008.12.002
5

Desai A., Patel C. R. (2023), Leveraging GIS to Deploy Demand-Driven Public Charging Infrastructure in An Indian Metropolitan City, Spatial Information Research, 31(4), 467-474.

10.1007/s41324-023-00514-4
6

Gao Y., Jiang D., Xu Y. (2018), Optimize Taxi Driving Strategies Based on Reinforcement Learning, International Journal of Geographical Information Science, 32(8), 1677-1696.

10.1080/13658816.2018.1458984
7

Garcia F., Rachelson E. (2013), Markov Decision Processes, Markov Decision Processes in Artificial Intelligence, 1-38.

10.1002/9781118557426.ch1
8

Hadimlioglu I. A., King S. A. (2019), Visualization of Flooding Using Adaptive Spatial Resolution, ISPRS International Journal of Geo-Information, 8(5), 204.

10.3390/ijgi8050204
9

Howard R. A. (1960). Dynamic Programming and Markov Processes. MIT Press(Cambridge, Massachusetts), 39-47.

10

Kakao Mobility (2017), 2017 Kakao Mobility Report, 226-233.

11

Kim A., Lim S. H. (2024), Study on Mobility Service Operational Strategies in Transportation-Underserved Rural Through the Analysis of Public Taxi Operational Status, J. Korean Soc. Transp., 42(1), Korean Society of Transportation, 60-76.

10.7470/jkst.2024.42.1.060
12

Kim S.T., Sung K.Y., Kim J.C., Kim H. (2023), Exploring the Optimization of Demand Responsive Transit Vehicle Routing Using a Deep Q Network, J. Korean Soc. Transp., 41(6), Korean Society of Transportation, 724-738.

10.7470/jkst.2023.41.6.724
13

Koorey G. (2009), Using an Interactive Display to Demonstrate Transportation Planning and Design Issues: Getting from A to B, Transportation Research Record, 2109(1), 31-36.

10.3141/2109-04
14

Luo J., Zhang W., Su J., Xiang F. (2019), Hexagonal Convolutional Neural Networks for Hexagonal Grids, IEEE Access, 7, 142738-142749.

10.1109/ACCESS.2019.2944766
15

Luo M., Zhang W., Song T., Li K., Zhu H., Du B., Wen H. (2021), Rebalancing Expanding EV Sharing Systems with Deep Reinforcement Learning, In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, 1338-1344.

10.24963/ijcai.2020/186
16

Lv Y., Zhi D., Sun H., Qi G. (2021), Mobility Pattern Recognition Based Prediction for The Subway Station Related Bike-Sharing Trips, Transportation Research Part C: Emerging Technologies, 133, 103404.

10.1016/j.trc.2021.103404
17

Miller P., de Barros A. G., Kattan L., Wirasinghe S. C. (2016), Public Transportation and Sustainability: A Review, KSCE Journal of Civil Engineering, 20(3), 1076-1083.

10.1007/s12205-016-0705-0
18

Oh J., Kim J. (2021), Where to Ride? An Explorative Study to Investigate Potential Risk Factors of Personal Mobility Accidents, International Journal of Environmental Research and Public Health, 18(3), 965.

10.3390/ijerph1803096533499255PMC7908115
19

Radzimski A., Dzięcielski M. (2021), Exploring The Relationship Between Bike-Sharing and Public Transport in Poznań, Poland, Transportation Research Part A: Policy and Practice, 145, 189-202.

10.1016/j.tra.2021.01.003
20

Silver D., Huang A., Maddison C. J., Guez A., Sifre L., Van Den Driessche G., Hassabis D. (2016), Mastering the Game of Go with Deep Neural Networks and Tree Search, Nature, 529(7587), 484-489.

10.1038/nature1696126819042
21

Singh D., Singh B. (2020), Investigating The Impact of Data Normalization on Classification Performance, Applied Soft Computing, 97, 105524.

10.1016/j.asoc.2019.105524
22

Son S. O., Park J. (2022), Identifying Micro-Level Hotspots of Urban Network Infrastructures Using GIS Data, J. Korean Soc. Transp., 40(4)

10.7470/jkst.2022.40.4.585
23

Sutton R. S., Barto A. G. (2018), Reinforcement Learning: An Introduction, MIT Press(Cambridge, Massachusetts), 49

24

Tong Y., Shi D., Xu Y., Lv W., Qin Z., Tang X. (2021), Combinatorial Optimization Meets Reinforcement Learning: Effective Taxi Order Dispatching at Large-Scale, IEEE Transactions on Knowledge and Data Engineering, 35(10), 9812-9823.

10.1109/TKDE.2021.3127077
25

Wang H., Tang H., Hao J., Hao X., Fu Y., Ma Y. (2020), Large Scale Deep Reinforcement Learning in War-games, In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 1693-1699, IEEE.

10.1109/BIBM49941.2020.9313387
26

Watkins C. J., Dayan P. (1992), Q-learning, Machine learning, 8, 279-292.

10.1023/A:1022676722315
27

Zagorskas J., Burinskienė M. (2019), Challenges Caused by Increased Use of E-Powered Personal Mobility Vehicles in European Cities, Sustainability, 12(1), 273.

10.3390/su12010273
28

Zhang L., Darzi A., Ghader S., Pack M. L., Xiong C., Yang M., Hu S. (2023), Interactive COVID-19 Mobility Impact and Social Distancing Analysis Platform, Transportation Research Record, 2677(4), 168-180.

10.1177/0361198121104381337153196PMC10152223
29

Zhao P., Li S. (2017), Bicycle-Metro Integration in A Growing City: The Determinants of Cycling As A Transfer Mode in Metro Station Areas in Beijing, Transportation Research Part A: Policy and Practice, 99, 46-60.

10.1016/j.tra.2017.03.003
Information
  • Publisher :Korean Society of Transportation
  • Publisher(Ko) :대한교통학회
  • Journal Title :Journal of Korean Society of Transportation
  • Journal Title(Ko) :대한교통학회지
  • Volume : 43
  • No :1
  • Pages :1-14
  • Received Date : 2024-05-31
  • Revised Date : 2024-07-18
  • Accepted Date : 2025-01-02
  • DOI :https://doi.org/10.7470/jkst.2025.43.1.001
Journal Informaiton Journal of Korean Society of Transportation Journal of Korean Society of Transportation
Archives
: 2015 Vol.34~
  • NRF
  • KOFST
  • crossref cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close