Pix2Pix-Based Data Augmentation Method for Building an Image Dataset of Black Ice

Hojun Lee; Minhee Kang; Jaein Song; Keeyeon Hwang

doi:10.7470/jkst.2022.40.4.539

All Issue

2022 Vol.40, Issue 4 Preview Page Next Page

Article

Pix2Pix-Based Data Augmentation Method for Building an Image Dataset of Black Ice 블랙 아이스의 학습 데이터셋 구축을 위한 Pix2Pix 기반 데이터 증강 방안

31 August 2022. pp. 539-554

PDF XML

Abstract

Recently, the field of artificial intelligence is growing rapidly, and image recognition technology using deep learning technology has evolved to a level that exceeds human recognition rate. To this end, it is essential to build large-capacity datasets, and efforts are being made to secure datasets by disclosing large-capacity open datasets such as CIFAR-10, Fashion-MNIST, MS-COCO, and Cityscapes. However, as data collection takes a lot of time and money, data augmentation technology is being used, and data augmentation research using deep learning such as Autoencoder and Generative Adversarial Network is being actively conducted. In particular, image recognition technology and data enhancement techniques are used to prevent traffic accidents, but black ice, which has a high mortality rate and leads to large accidents, has limitations in data acquisition, making it difficult to build large datasets. Therefore, this study proposes a method for enhancing black ice image data using Pix2Pix. Data collection and preprocessing were conducted to learn the Pix2Pix model, and black ice images created on new roads were secured through model learning. In the analysis of learning results, a quantitative evaluation was performed to derive loss and Frechet Inception Distance score, and a qualitative evaluation was also performed on the generated image. As a result of the learning, loss and Frechet Inception Distance score decreased as the learning progressed, and it was confirmed that the black ice image generated on the road was successfully generated through qualitative evaluation. This study confirmed that it is possible to generate a black ice image generated on the road using Pix2Pix, and through this, it is judged that object recognition research using the dataset should be conducted in the future by constructing a black ice dataset.

Keywords

black ice

data augmentation

deep learning

generative adversarial network

pix2pix

최근 인공지능 분야는 급속도로 성장하고 있으며, 딥러닝 기법을 활용한 이미지 인식 기술이 인간의 인식률을 뛰어넘는 수준으로 진화하였다. 이는 대용량의 데이터셋을 활용한 학습이 선행 되어야 함에 따라 CIFAR-10, Fashion-MNIST, MS-COCO, Cityscapes와 같은 대용량 오픈 데이터셋을 공개하는 등 데이터셋을 구축하기 위한 노력이 이루어지고 있다. 그러나, 데이터 수집 과정에서 많은 시간과 비용이 소모됨에 따라 데이터 증강 기술을 활용하고 있으며, Autoencoder, 생성적 적대 신경망 등 딥러닝을 활용한 데이터 증강 연구가 활발하게 이루어지고 있다. 특히 교통 분야에서는 교통사고를 예방하기 위해 이미지 인식 기술과 데이터 증강 기법을 활용하고 있지만, 치사율이 높고 대형 사고로 이어지는 블랙 아이스의 경우 데이터 구득에 한계가 존재하므로 대용량 데이터셋을 구축하는 것에 어려움을 겪고 있다. 이에 본 연구에서는 Pix2Pix를 활용한 블랙 아이스 이미지 데이터의 증강 방안을 제시한다. Pix2Pix 모델의 학습을 위해 데이터 수집과 전처리를 진행하였으며, 모델 학습을 통해 새로운 도로에 생성된 블랙 아이스 이미지를 확보하였다. 학습 결과 분석에서는 loss와 Frechet Inception Distance score를 도출하는 정량적 평가와 생성된 이미지를 활용하여 정성적 평가도 수행하였으며, 이를 통해 블랙 아이스 이미지를 성공적으로 생성하는 것을 확인하였다. 본 연구는 Pix2Pix를 활용하여 도로 위에 생성된 블랙 아이스 이미지의 생성이 가능한 것을 확인하였으며, 이를 통해 블랙 아이스 데이터셋을 구축하여 향후 이를 활용한 객체 검출 연구가 수행되어야 할 것으로 판단된다.

키워드

블랙 아이스

데이터 증강

딥러닝

생성적 적대 신경망

pix2pix

References

Ali-Gombe A., Elyan E., Jayne C. (2019), Multiple Fake Classes GAN for Data Augmentation in Face Image Dataset. In 2019 International Joint Conference on Neural Networks (IJCNN), 1-8, IEEE. 10.1109/IJCNN.2019.8851953

Alzraiee H., Leal Ruiz A., Sprotte R. (2021), Detecting of Pavement Marking Defects Using Faster R-CNN, Journal of Performance of Constructed Facilities, 35(4), 04021035. 10.1061/(ASCE)CF.1943-5509.0001606

Borji A. (2019), Pros and Cons of GAN Evaluation Measures, Computer Vision and Image Understanding, 179, 41-65. 10.1016/j.cviu.2018.10.009

Chung Q. M., Le T. D., Dang T. V., Vo N. D., Nguyen T. V., Nguyen K. (2020), Data Augmentation Analysis in Vehicle Detection from Aerial Videos. In 2020 RIVF International Conference on Computing and Communication Technologies (RIVF), 1-3. IEEE. 10.1109/RIVF48685.2020.9140740

Fang W., Ding Y., Zhang F., Sheng J. (2019), Gesture Recognition Based on CNN and DCGAN for Calculation and Text Output, IEEE Access, 7, 28230-28237. 10.1109/ACCESS.2019.2901930

Fang W., Zhang F., Sheng V. S., Ding Y. (2018), A Method for Improving CNN-based Image Recognition Using DCGAN, Computers, Materials and Continua, 57(1), 167-178. 10.32604/cmc.2018.02356

He K., Gkioxari G., Dollár P., Girshick R. (2017), Mask r-cnn. in Proceedings of the IEEE International Conference on Computer Vision, 2961-2969. 10.1109/ICCV.2017.322

He K., Zhang X., Ren S., Sun J. (2016), Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770-778. 10.1109/CVPR.2016.9026180094

Heusel M., Ramsauer H., Unterthiner T., Nessler B., Hochreiter S. (2017), GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. arXiv preprint arXiv:1706.08500.

Isola P., Zhu J. Y., Zhou T., Efros A. A. (2017), Image-to-image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1125-1134. 10.1109/CVPR.2017.632

Krizhevsky A., Sutskever I., Hinton G. E. (2012), Imagenet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems, 25, 1097-1105.

Li B., Wang K. C., Zhang A., Yang E., Wang G. (2020), Automatic Classification of Pavement Crack Using Deep Convolutional Neural Network, International Journal of Pavement Engineering, 21(4), 457-463. https://www.tandfonline.com/doi/full/10.1080/10298436.2018.1485917 10.1080/10298436.2018.1485917

Lucic M., Kurach K., Michalski M., Gelly S., Bousquet O. (2017), Are Gans Created Equal? a Large-scale Study. arXiv Preprint arXiv:1711.10337.

Madani A., Moradi M., Karargyris A., Syeda-Mahmood T. (2018), Chest X-ray Generation and Data Augmentation for Cardiovascular Abnormality Classification. In Medical Imaging 2018: Image Processing, 10574, 105741M, International Society for Optics and Photonics. 10.1117/12.2293971

Maeda H., Sekimoto Y., Seto T., Kashiyama T., Omata H. (2018), Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone. arXiv Preprint arXiv:1801.09454.

Muñoz-Bulnes J., Fernandez C., Parra I., Fernández-Llorca D., Sotelo M. A. (2017, October), Deep Fully Convolutional Networks with Random Data Augmentation for Enhanced Generalization in Road Detection, In 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) (pp. 366-371). IEEE. 10.1109/ITSC.2017.8317901

Redmon J., Divvala S., Girshick R., Farhadi A. (2016), You Only Look Once: Unified, Real-time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779-788. 10.1109/CVPR.2016.91

Ren S., He K., Girshick R., Sun J. (2015), Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks, Advances in Neural Information Processing Systems, 28, 91-99.

Simonyan K., Zisserman A. (2014), Very Deep Convolutional Networks for Large-scale Image Recognition. arXiv Preprint arXiv:1409.1556.

Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Rabinovich A. (2015), Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-9. 10.1109/CVPR.2015.7298594

Tanaka F. H. K. D. S., Aranha C. (2019), Data Augmentation Using GANs. arXiv Preprint arXiv:1904.09135.

Tran N. T., Bui T. A., Cheung N. M. (2018), Dist-gan: An Improved Gan Using Distance Constraints. In Proceedings of the European Conference on Computer Vision (ECCV), 370-385. 10.1007/978-3-030-01264-9_23

Wang K., Fang B., Qian J., Yang S., Zhou X., Zhou J. (2019), Perspective Transformation Data Augmentation for Object Detection, IEEE Access, 8, 4935-4943. 10.1109/ACCESS.2019.2962572

Xing Y., Ge Z., Zeng R., Mahapatra D., Seah J., Law M., Drummond T. (2019), Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 757-765, Springer, Cham. 10.1007/978-3-030-32226-7_84

Yang H. J., Jang H., Jeong D. S. (2013), Detection Algorithm for Road Surface Condition Using Wavelet Packet Transform and SVM. In The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 323-326, IEEE. 10.1109/FCV.2013.6485514

Information

Publisher :Korean Society of Transportation
Publisher(Ko) :대한교통학회
Journal Title :Journal of Korean Society of Transportation
Journal Title(Ko) :대한교통학회지
Volume : 40
No :4
Pages :539-554
Received Date : 2022-03-31
Revised Date : 2022-04-25
Accepted Date : 2022-05-28
DOI :https://doi.org/10.7470/jkst.2022.40.4.539

Journal of Korean Society of Transportation 대한교통학회지 ISSN:1229-1366(Print) 2234-4217(Online)

All Issue