Improvement in video inpainting in presence of moving subjects

Document Type: Research Paper

Authors

1 Department of Computer Engineering, Ayandegan Institute of Higher Education, Tonekabon, Iran

2 Department of Industrial Engineering, Ayandegan Institute of higher Education, Tonekabon, Iran.

10.22105/riej.2020.215689.1116

Abstract

Inpainting or completion is used with the purpose of restoring damaged images and video frames. This paper proposes an applicable algorithm to inpaint corrupted subjects in video frames. To begin with, background and foreground (moving subject) are separated from each other in each frame, with the aim of getting to a more visually pleasant result. Static background inpainting is done using a patch-based method. To inpaint the corrupted moving subject, a subject-based method that is an improvement on the rigid object-based method is used in this study to consider the special issue of inpainting the human body. To fill in the holes created by occluding subjects the most appropriate template is found using a similarity measure which is based on both contour and pixel values. The inpainted video is acquired by superimposing the completed foreground on the inpainted background

Keywords

Main Subjects


[1]      Ghanbari A., & Majdi M. (2017). Video inpainting using a contour-based method in presence of more than one moving objects. International journal of advanced engineering and management, 2(2), 37 - 44.

[2]      Javed, S., Mahmood, A., Al-Maadeed, S., Bouwmans, T., & Jung, S. K. (2018). Moving object detection in complex scene using spatiotemporal structured-sparse RPCA. IEEE transactions on image processing28(2), 1007-1022.

[3]      Pawar, A. J., Jadhav, H. L., Ukarande, V. V., & Chavan, S. R. (2018, January). A comparative study of effective way to modify different object in image and video using different inpainting methods. 2018 2nd international conference on inventive systems and control (ICISC) (pp. 1098-1102). IEEE.

[4]      Pinjarkar, A. V., & Tuptewar, D. J. (2019). Robust exemplar-based image and video inpainting for object removal and region filling. In computing, communication and signal processing (pp. 817-825). Singapore: Springer.

[5]      Bertalmio, M., Bertozzi, A. L., & Sapiro, G. (2001, December). Navier-stokes, fluid dynamics, and image and video inpainting. Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, CVPR 2001 (Vol. 1, pp. I-I). IEEE.

[6]      Le, T. T., Almansa, A., Gousseau, Y., & Masnou, S. (2018). Removing objects from videos with a few strokes. SIGGRAPH Asia 2018 technical briefs (pp. 1-4). https://dl.acm.org/

[7]      Talouki, A. G., Majdi, M., & Edalatpanah, S. A. (2017). An introduction to various algorithms for video completion and their features: a survey. Journal of computer sciences and applications, 5(1), 1-10.

[8]      Soryani, M., Ghanbari, A., & Koochari, A. (2014). Dynamic video texture inpainting using improving LDS. British journal of mathematics & computer science, 4(20), 2872-2883.

[9]      Le, T. T., Almansa, A., Gousseau, Y., & Masnou, S. (2017, September). Motion-consistent video inpainting. 2017 IEEE international conference on image processing (ICIP) (pp. 2094-2098). IEEE.

[10]   Bouwmans, T., Maddalena, L., & Petrosino, A. (2017). Scene background initialization: a taxonomy. Pattern recognition letters, 96, 3-11.

[11]   Zakir, Y. K., Prajot, R., Krishna, K., & Anish, K. (2016). A hierarchical super resoloution based video inpainting tool. International journal of advance research and innovative ideas in education, 2(2), 236-240.

[12]   Gaur, S., Degadwala, S., & Mahajan, A. (2019). Multiple objects tracking under occlusion detection in video sequences. In emerging trends in expert applications and security (pp. 189-196). Singapore: Springer.

[13]   Cheung, S. C. S., Zhao, J., & Venkatesh, M. V. (2006, October). Efficient object-based video inpainting. 2006 international conference on image processing (pp. 705-708). IEEE.

[14]   Zhang, Y., Xiao, J., & Shah, M. (2005, January). Motion layer based object removal in videos. 2005 seventh IEEE workshops on applications of computer vision (WACV/MOTION'05 (Vol. 1, pp. 516-521). IEEE.

[15]   Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000, July). Image inpainting. Proceedings of the 27th annual conference on computer graphics and interactive techniques (pp. 417-424). ACM Press/Addison-Wesley Publishing Co.

[16]   Shiratori, T., Matsushita, Y., Tang, X., & Kang, S. B. (2006, June). Video completion by motion field transfer. 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06) (Vol. 1, pp. 411-418). IEEE.

[17]   Ghanbari A., Soryani M., (2012). Contour-based video inpainting in presence of more than one moving object.Paper presented at the meeting of international conference on image processing and pattern recognition (IPPR 2015), Chennai, India.

[18]    proc. 2nd int. conf. image processing and pattern recognition.

[19]   Xu, Z., Min, B., & Cheung, R. C. (2019). A robust background initialization algorithm with superpixel motion detection. Signal processing: image communication, 71, 1-12.

[20]   Xiang, S., Deng, H., Zhu, L., Wu, J., & Yu, L. (2019). Exemplar-based depth inpainting with arbitrary-shape patches and cross-modal matching. Signal processing: image communication, 71, 56-65.

[21]   Aadhirai S., Srinivasan M. N., Rekha R. L., (2016). An approach of video inpainting technique to remove undesired objects using mean shift algorithm. International journal of advanced research in electronics and communication engineering (IJARECE), 5, 1181-184.

[22]   Ghanbari, A., & Soryani, M. (2011, November). Contour-based video inpainting. 7th Iranian conference on machine vision and image processing (pp. 1-5). Tehran, Iran: IEEE.