Cogent Embedded Surround View Color Correction greatly improves the output image on the screen for the user. It removes artefacts on surround view and improves visual experience. The algorithm requires low performance overhead, low level of complexity and low risk of introducing additional artefacts. A reference implementation on Renesas hardware added no additional load on R-Car IMR engine. These advantages allow easily implementing this algorithm as an additional feature to the existing system.

• Surround View

A key challenge facing designers of automotive embedded 360° surround view system is the vast difference in ambient lighting which each of the cameras around the vehicle can be exposed to. In result, the same objects appear substantially different in color and brightness in images produced by cameras looking in different directions with overlapping fields of view. This creates substantial artefacts in stitched panoramas along the seams between parts produced by different cameras as shown below.

Color mismatches are the result of optical effects caused by Auto Exposure and Auto White Balance (AWB) in camera Image Signal Processors (ISP). While it may be possible to fix common Auto Exposure and AWB parameters between cameras, it is generally not a good idea to do so as it causes inefficient usage of camera output dynamic range leading to information loss due to image saturation. As camera data are used for many purposes including critical safety features the information loss is unacceptable.

The aim of color correction is to harmonize images in order to make the same objects appear similar in images produced by different cameras. Compared to blending which has local effect and can lead to blurriness of the panorama, color correction effect is more global and does not affect the sharpness of the image. The effect of color correction is similar to the low-frequency band blending. With good color correction it is possible to completely avoid blending between images.

One approach to color correction is reverting the gamut transformation in the camera based on the processing parameters read from the camera. This requires an accurate model of processing in the camera and interacting with the camera through the vendor defined interface.

Cogent has developed color correction algorithms based on color difference statistics in overlap-zones which solve this problem efficiently and independently of the camera interface.

As an automotive embedded software supplier we are intimately familiar with writing code that runs efficiently. Our color correction algorithms have been implemented and tested on embedded hardware platforms based on Renesas V3M and Renesas V2H processors, on TI TDA2 and TDA3 processors. In each case the average ARM utilization was below 10%. Depending on performance limitations, our algorithm can be tweaked to fit into even smaller processing footprints. Of course, our color correction can also run on more powerful platforms and has been successfully implemented on GPU-based infotainment systems based on NVIDIA, Intel and Qualcomm chips.