PerfectColor - Methods and algorithms for real-time color image enhancement

Project Summary

Color image enhancement is an important problem in many image processing applications. The development of image sensors and the growing market demand for advanced solutions that are based on image processing technology are the main factors for the increasing importance of image enhancement techniques. In particular, image processing algorithms used for embedded systems such as digital cameras have strict requirements in terms of real-time operation and hardware constraints imposed by specific technology used for embedded implementation, such as FPGA technology. Most methods and algorithms published in the literature are not suitable for real-time operation due to their complexity or due to being hardware unfriendly so their embedded implementations are too complex and unable to satisfy the strict requirements of real-time applications.

The overall goal of this project is to improve two important image enhancement tasks of color constancy and tone mapping found in many embedded systems such as digital cameras by proposing new practically implementable methods that are capable of real-time operation.

The main objectives of this project are research and development of:

  1. methods for real-time color constancy, and
  2. methods for real-time tone mapping.

In addition, two new image datasets that are to be created during this research will certainly have an important impact on the whole research field of color constancy.

     

     

Project Duration

The project duration is from March 1, 2017 to February 28, 2021.

Project Team

  • Prof. Sven Lončarić, principal investigator
  • Assoc. Prof. Marko Subašić, researcher
  • postdoctoral fellow - full-time position open
  • Karlo Koščević, doctoral student, researcher
  • Nikola Banić, researcher
  • Ivana Geček Tuđen, researcher

Results

In the scope of the project, four journal articles and eleven conference papers have been published. Three new datasets for the research and development of color constancy methods have been created.

Journal articles

Koščević, Karlo; Subašić, Marko; Lončarić, Sven. "Iterative Convolutional Neural Network-Based Illumination Estimation", IEEE Access,  vol. 9, 2021. pp. 26755-26765, doi: 10.1109/ACCESS.2021.3057072

Koščević, Karlo; Subašić, Marko; Lončarić, Sven. "Deep Learning-Based Illumination Estimation Using Light Source Classification", IEEE Access, vol. 8, 2020. pp. 84239-84247, doi: 10.1109/ACCESS.2020.2992121

Ershov, Egor; Savchik, Alexey; Semenkov, Illya; Banić, Nikola; Belokopytov, Alexander; Senshina, Daria; Koščević, Karlo; Subašić, Marko, Lončarić, Sven. "The Cube Illumination Estimation Dataset", IEEE Access, vol. 8, 2020. pp. 227511-227527, doi: 10.1109/ACCESS.2020.3045066

Banić Nikola; Lončarić, Sven. "Green Stability Assumption: Unsupervised Learning for Statistics-Based Illumination Estimation", Journal of Imaging, vol. 4, no. 11, 2018. pp. 127-137, doi: 10.3390/jimaging4110127

Conference papers

Banić, Nikola; Koščević, Karlo; Subašić, Marko; Lončarić, Sven. "CroP: Color Constancy Benchmark Dataset Generator", Proceedings of the 2020 4th International Conference on Vision, Image and Signal Processing, Bangkok, Thailand, 2020, 9 pages, doi: 10.1145/3448823.3448829

Koščević, Karlo; Subašić Marko; Lončarić, Sven. "Guiding the Illumination Estimation Using the Attention Mechanism", Proceedings of the 2020 2nd Asia Pacific Information Technology Conference, Bali, Indonesia, 2020. pp. 143-149, doi: 10.1145/3379310.3379329

Banić, Nikola; Koščević, Karlo; Subašić, Marko; Lončarić Sven. "On Some Desired Properties of Data Augmentation by Illumination Simulation for Color Constancy", Computer Science & Information Technology, Sydney, Australia, 2020. pp. 27-38, doi: 10.5121/csit.2020.101903

Banić, Nikola; Koščević, Karlo; Subašić, Marko; Lončarić, Sven. "The Past and the Present of the Color Checker Dataset Misuse", Proceedings of the 11th International Symposium on Image and Signal Processing and Analysis, Dubrovnik, Croatia, 2019. pp. 366-371, doi: 10.1109/ISPA.2019.8868882

Koščević, Karlo; Subašić, Marko; Lončarić, Sven. "Attention-based Convolutional Neural Network for Computer Vision Color Constancy", Proceedings of the 11th International Symposium on Image and Signal Processing and Analysis, Dubrovnik, Croatia, 2019. pp. 372-377, doi: 10.1109/ISPA.2019.8868806

Banić, Nikola; Lončarić, Sven. "Blue Shift Assumption: Improving Illumination Estimation Accuracy for Single Image from Unknown Source", Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, Prague, Czech Republic, 2019. pp. 191-197, doi: 10.5220/0007394101910197

Koščević, Karlo; Banić, Nikola; Lončarić, Sven. "Color Beaver: Bounding Illumination Estimations for Higher Accuracy", Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, Prague, Czech Republic, 2019. pp. 183-190, doi: 10.5220/0007394101910197

Banić, Nikola; Lončarić, Sven. "Illumination Estimation Is Sufficient for Indoor-Outdoor Image Classification", 40th German Conference on Pattern Recognition, Stuttgart, Germany, 2018. pp. 473-486 doi: 10.1007/978-3-030-12939-2_33

Banić, Nikola; Lončarić, Sven. "Flash and Storm: Fast and Highly Practical Tone Mapping based on Naka-Rushton Equation", Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, Funchal, Portugal, 2018. pp. 47-53, doi: 10.5220/0006621600470053

Banić, Nikola; Lončarić, Sven. "Unsupervised Learning for Color Constancy", Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, Funchal, Portugal, 2018. pp. 181-188, doi: 10.5220/0006621801810188

Banić, Nikola; Lončarić, Sven. "Towards Hardware-Friendly Retinex Algorithms", Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis, Ljubljana, Slovenia, 2017. pp. 104-108, doi: 10.1109/ISPA.2017.8073578

Color Constancy Resources

A repository of source codes for methods as well as other color constancy-related resources that were created by the Image Processing Group are available on this page.

Acknowledgment

This research project has been fully supported by Croatian Science Foundation under the project IP-2016-06-2092.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of Croatian Science Foundation, Ministry of Science and Education and European Commission.