Generative Adversarial Networks Based Scene Generation on Indian Driving Dataset
DOI:
https://doi.org/10.5614/itbj.ict.res.appl.2023.17.2.4Keywords:
artificial intelligence, deep learning, driving dataset, generative adversial networks, scene generationAbstract
The rate of advancement in the field of artificial intelligence (AI) has drastically increased over the past twenty years or so. From AI models that can classify every object in an image to realistic chatbots, the signs of progress can be found in all fields. This work focused on tackling a relatively new problem in the current scenario-generative capabilities of AI. While the classification and prediction models have matured and entered the mass market across the globe, generation through AI is still in its initial stages. Generative tasks consist of an AI model learning the features of a given input and using these learned values to generate completely new output values that were not originally part of the input dataset. The most common input type given to generative models are images. The most popular architectures for generative models are autoencoders and generative adversarial networks (GANs). Our study aimed to use GANs to generate realistic images from a purely semantic representation of a scene. While our model can be used on any kind of scene, we used the Indian Driving Dataset to train our model. Through this work, we could arrive at answers to the following questions: (1) the scope of GANs in interpreting and understanding textures and variables in complex scenes; (2) the application of such a model in the field of gaming and virtual reality; (3) the possible impact of generating realistic deep fakes on society.
Downloads
References
Hossam, H., Elgmmal, E. & Elnabawy, R.H., A Review of Generative Adversarial Networks Applications, in 4 Novel Intelligent and Leading Emerging Sciences Conference (NILES), pp. 142-146. 2022.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. & Bengio, Y., Generative Adversarial Nets, in Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
Radford, A., Metz, L. & Chintala, S., Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016.
Hong, W., Wang, Z., Yang, M. & Yuan, J., Conditional Generative Adversarial Network for Structured Domain Adaptation,. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1335-1344. Salt Lake City, UT, USA, 2018.
Isola, P., Zhu, J. -Y., Zhou, T. & Efros, A.A., Image-to-Image Translation with Conditional Adversarial Networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017. Honolulu, HI, USA. DOI: 10.1109/CVPR.2017.632. 2017.
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J. & Catanzaro, B., Video-To-Video Synthesis, in Advances in Neural Information Processing Systems, 31, 2018.
Varma, G., Subramanian, A., Namboodiri, A., Chandraker, M. & Jawahar, C.V., IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments, in IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1743-1751, 2019.
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S. & Schiele, B., The Cityscapes Dataset for Semantic Urban Scene Understanding, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213-3223, 2016.
Wang, T. C., Liu, M. -Y., Zhu, J. -Y., Tao, A., Kautz, J. & Catanzaro, B., High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs., in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8798-8807, 2018.
Lin, G., Milan, A., Shen, C. & Reid, I.D., RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5168-5177, 2017.
Mottaghi, R., The Role of Context for Object Detection and Semantic Segmentation in the Wild, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 891-898, 2014.
Ghosh, A., Bhattacharya, B. & Roy Chowdhury, S.B., SAD-GAN: Synthetic Autonomous Driving using Generative Adversarial Networks, 30th Conference on Neural Information Processing Systems (NIPS), 2016.
Regmi, K. & Borji, A., Cross-view image synthesis using geometry-guided conditional GANs, Computer Vision and Image Understanding, 187, 2019.
Turkoglu, M.O., Thong, W., Spreeuwers, L.J. & Kicanaoglu, B., A Layer-Based Sequential Framework for Scene Generation with GANs, AAAI Conference on Artificial Intelligence, 2019.
S Liu, S., Wang, T., Bau, D., Zhu, J.-Y. & Torralba, A., Diverse Image Generation via Self-Conditioned GANs, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 14274-14283, 2020.
Alotaibi, A., Deep Generative Adversarial Networks for Image-to-Image Translation: A Review, Symmetry, 12(10), 2020.
Liu, M.Y., Huang, X., Yu, J., Wang, T.C. & Mallya, A, Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications, Proceedings of the IEEE, 109(5), 2021.
Shamsolmoali, P., Zareapoor, M., Granger, E., Zhou, H., Wang, R., Celebi, M.E. & Yang, J., Image Synthesis with Adversarial Networks: A Comprehensive Survey and Case Studies, Information Fusion, 72, pp.126-146, 2021.
Alokasi, H. & Ahmad, M.B., Deep Learning-Based Frameworks for Semantic Segmentation of Road Scenes, Electronics, 11(12), 1884, 2022.
Jozdani, S., Chen, D., Pouliot, D. & Johnson, B.A., A Review and Meta-Analysis of Generative Adversarial Networks and Their Applications in Remote Sensing, International Journal of Applied Earth Observation and Geoinformation, 108, 2022.
Pradhyumna, P. & Mohana., A Survey of Modern Deep Learning based Generative Adversarial Networks (GANs), 6th International Conference on Computing Methodologies and Communication (ICCMC), pp. 1146-1152, 2022.
Porkodi, S.P., Sarada, V. & Maik, V., Generic image application using GANs (Generative Adversarial Networks): A Review, Evolving Systems, 2022.
Liu, Y., Gal, R., Bermano, A., Chen, B. & Cohen-Or, D., Self-Conditioned Generative Adversarial Networks for Image Editing, SIGGRAPH '22: ACM SIGGRAPH 2022, 16, pp.1-19, 2022.
Mehralian, M. & Karasfi, B., RDCGAN: Unsupervised Representation Learning with Regularized Deep Convolutional Generative Adversarial Networks, 9th Conference on Artificial Intelligence and Robotics and 2nd Asia-Pacific International Symposium, pp. 31-38, 2018.
Yurtsever, E., Yang, D., Koc, I.M. & Redmill, K.A., Photorealism in Driving Simulations: Blending Generative Adversarial Image Synthesis with Rendering, IEEE Transactions on Intelligent Transportation Systems, 23(12), pp. 23114-23123, 2022.
Yurtsever, E., Yang, D., Koc, I.M. & Redmill, K.A., Photorealism in Driving Simulations: Blending Generative Adversarial Image Synthesis with Rendering, IEEE Transactions on Intelligent Transportation Systems, 23(12), pp.23114-23123, 2022.
Ronneberger, O., Fischer, P. & Brox, T., U-Net: Convolutional Networks for Biomedical Image Segmentation, in N. Navab, J. Hornegger, W. Wells, A. Frangi (Eds.), Medical Image Computing and Computer-Assisted Intervention ? MICCAI, Lecture Notes in Computer Science, 9351, Springer, Cham., 2015.
