Image is a basic input for many machine learning-based algorithms, as the majority of the algorithms processed on image data. In nowadays world the data of the image is increasing rapidly. Image deconvolution is used in many applications
like intelligent systems to trace the culprits or to retrieve highly secret information. It is also used in astronomical observations, where the observations mainly focus on the study of extra-terrestrial life and study planets.
These scientists need sharp images to study so. They use image deconvolution algorithms. Image's deconvolution is also used in biomedical used to detect the disease of the patient. Image deblurring occurs due to many reasons,
mainly due to camera shake, some noise, not focused properly and many more. In this case, we cannot retake the picture again as it has already happened, which we can't change, this blurred image can contain important information,
or it is a picture of loved ones. So in order to retrieve such information from the image, we used a deconvolution strategy to update the frequencies in the pixels of an image, such that blurred manage is deblurred not entirely
but to some extent.
Today, when looking around, the imaginative advances recently have been gigantic. As seen in driverless vehicles, sans hands contraptions that can turn on the lights, and robots are working in handling plants, which show that
shrewd machines are possible. Preposterous late four years in the Indian startup climate, the terms that were used (handled to some degree) more than sponsoring, valuation, and exit were human-made thinking (PC based insight)
and computer-based intelligence (ML). Furthermore, saw monetary patrons expeditiously setting in their money in fire up's that indirectly used or purported to use these emerging advancements. From deep tech, online business,
fintech, and conversational chatbots to flexibility, food tech, and clinical benefits, re-enacted insight and ML have changed. The business has immediately moved from mentioning engineers to deal with tons from code to the
machine to getting terabytes of data and crunching it to collect significant reasoning. All through the long haul, development has changed our existence and consistently lives. Development had made amazing devices and resources
for humankind, putting important information which has promptly been accessible. Current improvements in technology have arranged for many multi-utilitarian-based contraptions like a smartwatch and the phone. Computers are
logically faster, more minimal, and more impressive than at some other time. With these miracles, development has moreover simplified our lives faster, better, and more fun. Moreover, in order to build the required models for
real-world based applications, there are an enormous set of libraries in a programming language like python. These libraries are specially built for models or an algorithm to solve real-world problems. There are so many changes
in the functionality of a programming language in the last ten to fifteen years.
The above flowchart is the basic pipeline for this project. At first, the basic task is to import the required package. May include necessary packages or modules to use the existing module, which are already in-built in the
language in this mainly use TensorFlow, which is a system backend usable module which is majorly used for image processing and remaining applications that majorly need a high amount of hardware equipment and power to process
the input and many modules used in this project, namely argparse, UUID, base64, cv2 and many more, are image processing modules in python. Then the warnings have to be ignored for the smooth run of the project. For this, use
the warnings module, which is already existed in the python language. This basically checks for the hardware requirement for the model implementation. As already said, this image processing uses system backend or kernel access
of the system. TensorFlow is developed by Google for the fast and robust image processing technology specially created for python. So, for the hardware, the system must need GPU (graphical processing unit) inbuilt in the system.
NVIDIA or AMD the tycoons of the GPU industry. For this project, if the system has NVIDIA GPU, it must be CUDA enabled or else the power of the system is not sufficient for the processing of an image. After importing the libraries,
the next step is to create the necessary folders to save or retrieve the data from that folders. For this, use the OS module, which was designed to move freely between directories or from directories to sub-directories, even
sub-sub-directories of the particular folder. This majorly works on the root directory. For this project, it needs some folders in order to store the input image and to store the output image. By using the OS module, create
many directories like templates for storing HTML files, static for CSS and JS files for the HTML file, images folder to store the images, uploads folder to store output images which are deblurred by the model. Checkpoint's
folder to save the checkpoints of the model where it can be used directly without training the model each and every time. After completing the creation of folders or directories, the next step is to take the input from the
web application, which is already created by using a flask. Flask is a web application program just as react and angular js in java, flask used to run the program in the web browser as the server-side application just as apache
tomcat server. The main reason for using flask is it is a lightweight application compared to the Django framework, which is similar to angular, which is a complete server-side application framework more complicated than a
flask. In this project, the flask application is built with HTML and CSS, which are web-based programming languages. In this user can give the image as input by selecting the required image from the system documents.
But before giving input, the flask application takes a port number as the source to run the application, and then this port number need to be pasted on the browser URL search bar, then the project successfully runs in the
web browser it doesn't contain any errors. Then it shows an HTML page on the browser, which is stored in the templates folder. As soon as the image is selected, then the image needs to be cropped (i.e., need to select some
part of the image which want to be deblurred). The main reason for this is model is only trained with specific dimensions (i.e., x-height and x width). Since image sizes vary from each other, it is highly important to crop
the image in order to get the best accuracy deblurred image. If the input size doesn't match the training set of the model, then the model doesn't deblur the image properly or doesn't deblur the image. As the user presses the
crop button, which is already created in the HTML document, then the cropped image is directly stored in the cropped folder, which is stored in the uploads directory acts as a root directory. Then the cropped image is passed
through the encoder as the image need to be feed to the nodes of the neural network. But, before encoding, the image needs to be taken as raw input where NumPy concept comes into the picture. As the image is encoded, it is
passed through the convolution layer of the neural network. Encoder and decoder networks are nothing but the structures of symmetric CNN. In which the input image is converted or divided to feature maps and feed to the model
and in a similar decoder manner as encoder but in opposite where the deblurred feature maps are converted back to the image in order to display. Basically, encoding contains convolution layers with strides and activation layers
Tanh activation and whereas, in a decoder, it contains deconvolution layers for resizing the image. As the image is encoded, the image is saved with hexadecimal name with a .png extension and with a unique id is also given
such that no two images have the same name and stored in the text file, the saving is important to track exactly what output got for which input.
Then after completing the encoding process, the image is passed to a different set of nodes as the neural network use for this Project is RNN. So, it has a backtracking ability and a memory storage capacity in order to remove
the vanishing gradient problem. The feature maps of the image will be deblurred parallelly, and the frequencies of the pixels are updated based on the blur on the image pixels. Then, these updated feature maps are combined
in the decoding process, which has deconvolution layers to resize the image to the actual size of the input image. After adjusting the frequencies of the feature maps then the restored image is saved with the same unique id
of the input image in the finished folder in uploads. Then this image which is stored in the finished folder, is retrieved and displayed on the web pages where it can be visible to the user. This is the entire flowchart for
the project natural image deblurring via ringing artefacts.
References
- S. Sun, H. Zhao, B. Li, and M. Yang, ``Kernel estimation for robust motion deblurring of noisy and blurry images,'' J. Electron. Imag., vol. 25, no. 3, pp. 112, 2016.
- J. Pan, D. Sun, H. Peter, and M.-H. Yang, ``Blind image deblurring using dark channel prior,'' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR), Jun. 2016, pp. 16281636.
- X. Xu, J. Pan, Y.-J. Zhang, and M.-H. Yang, ``Motion blur kernel estimation via deep learning,'' IEEE Trans. Image Process., vol. 27, no. 1, pp. 194205, Jan. 2018.
- O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, ``Deblur-GAN: Blind motion deblurring using conditional adversarial networks,'' in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 81838192.
- D. Wu, H. Zhao, and S. Zheng, ``Motion deblurring method based onDenseNets,'' J. Image Graph., vol. 25, no. 5, pp. 890899, 2020.
- J. Pan, D. Sun, H. Peter, and M.-H. Yang, ``Deblurring images via dark channel prior,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 10, pp. 23152328, Oct. 2018.
- https://aditi-mittal.medium.com/understanding-rnn-and-lstm-f7cdf6dfc14e
- https://en.wikipedia.org/wiki/Deblurring
- https://en.wikipedia.org/wiki/Deconvolution
- http://gwyddion.net/documentation/user-guide-en/convolution-deconvolution.html