In image processing, super-resolution refers to the process of improving spatial resolution, i.e. the level of detail in an image. The aim of this article is to present super-resolution based on Deep Learning, with a few useful applications and a simple example to visualize.
What is super-resolution?
Super-resolution is a set of algorithms and techniques used to improve, increase and oversample the resolution of an input image.
In other words, it involves taking an input image and increasing its size. That is, its width and height with minimal degradation of quality. Ideally, quality is not degraded, and that's what super-resolution is all about. Indeed, almost everyone has tried to enlarge an image and found that it becomes less sharp, or even blurred.
In fact, most photo editing software (Photoshop, GIMP, etc.) all use classic interpolation techniques and algorithms:
- nearest neighbor interpolation
- linear interpolation
- bicubic interpolation
- ...
Why use super-resolution?
There are many reasons to use super-resolution. But super-resolution is not a free process in terms of computing power (choice of processing platform, energy, space requirements, etc.). Its use must therefore be justified! For example, if you're using a standard, reasonably-priced visible camera, it's better to change the camera than to carry out super-resolution processing. For the following examples, on the other hand, super-resolution processing is indispensable:
- Expensive cameras: this is the case, for example, with MWIR (Middle Wave Infrared) cameras. These cameras are generally limited to a resolution of 640×480 pixels, and can cost several tens of thousands of euros.
- Existing camera in an industrialized product: it is often simpler to implement super-resolution than to change the camera in an industrialized, qualified product (shock, vibration, EMC, etc.). This may be the case for certain military products, for example.
Super-resolution models (compatible with OpenCV)
A number of super-resolution models can be implemented. Here are just a few of them. Most are based onArtificial Intelligence (Deep Learning).
- EDSR - Enhanced Deep Residual Networks for Single Image Super-Resolution
- ESPRC - Real-Time Single Image and Video Super-Resolution Using an Efficient Subpixel Convolutional Neural Network
- FSRCNN - Accelerating the Super-Resolution Convolutional Neural Network
- LapSRN - Fast and Accurate the Super-Resolution with Deep Laplacian Pyramid Networks
These models can be implemented to run in real time, on embedded platforms such as NIVDIA TX2, Xavier, ... It is then possible to implement them to run on a GPU.
Example of super-resolution
As an example, we present the result of an implementation of our Deep Learning-based super-resolution model. We took our low-resolution Imasolia logo and injected it into our model. The result is significantly less pixelated and more pleasing to the eye. We increased the resolution by a factor of around x4.
Another example, taken from the literature this time, is shown below. Different models have been used and compared on the Lenna image. The results are not discussed here, as this is not the aim of the article. The aim is to show that it is possible to use several models with their own advantages and disadvantages.
Conclusion
In this article, we introduced the notion of super-resolution with models based on Deep Learning. We explained its interest and how it can be implemented on GPUs and embedded platforms for real-time processing. In fact, we are specialists in implementing this type of image processing on NVIDIA embedded platforms!
Imasolia offers super-resolution algorithms with Deep Learning for your specific applications. Please contact us to discuss your requirements.
