• 2D and 3D projections
• History view (transparent car)
• Pre-defined views (e.g. front/rear, tow-hitch)
• Animated UI overlay (car, doors, wheels, projected path/guidelines)
• Factory calibration (intrinsic and extrinsic)
• Advanced Color correction (brightness, color, gamma)
• Dirt/Blockage detection
• Autocalibration
• Built-in debug capability includes video recording and video file playback

Compatible with Cogent Computer Vision Components:

• Lane detection
• Pedestrian detection
• Object detection
• Trailer angle detection
• Blind spot detection
• Parking slot detection
• Cross traffic alert
• Structure from Motion

• Scalable Architecture: Supports anything from TI-TDA3 or Renesas V3M to high-end X86, R-Car or NVIDIA Tegra processors
• Adaptable Software Components
• Used in MY2019 production program.
• Coded in C and C++ to Automotive Standards
• MISRA compliant

• Camera Placement Toolkit

Cogent Surround View provides a set of embedded C++ software components and associated tools that are ready for production implementation.

The projection engine is the heart of surround view. It accepts inputs from the video input framework and provides a perspective-corrected output view, merging the input from four cameras.

The projection engine is compatible with OpenGL to take advantage of existing GPU resources, but has been successfully implemented also on lower end DSP platforms.

Cogent Surround view is implemented with a “zero-copy” approach: Images are never copied within the application, resulting in significant performance improvement.

Cogent Surround View is compatible with FMVSS 111. The system includes an early video mode in which a rear-camera only view is displayed while additional surround view functional blocks are initialized. The transition from early video to standard operation is seamless.

Combining four camera images requires more than just stitching the images together: Visually pleasing surround view requires color and brightness of the four camera images to be coordinated. Cogent surround view includes the necessary algorithms.

When available, computationally expen­sive tasks (e.g. color correction) are hardware accelerated on a GPU. The corresponding functional blocks have been implemented in GLSL-shading language.

Cogent Surround View comes with powerful overlay capability, allowing the placement of 2D and 3D objects in the image. The 3D images can be animated, providing an augmented reality view in which the host vehicle appears realistically with working brake lights, turning wheels that follow the steering direction.

A frozen or delayed camera image displayed to a driver might pose a safety risk. Cogent Surround View has been designed to address this risk. A surround view supervisor daemon constantly monitors the validity of incoming video data, and switches the surround view output into a safe state (e.g. by showing a warning screen) when an input anomaly has been detected.

Every car is unique. And every camera installation is subject to mechanical tolerances. Cogent Surround View includes an offline (factory) calibration component to account for these inevitable built tolerances.

Offline calibration detects visual targets, e.g. black dots on a white background that are placed around the vehicle within the overlapping view of two cameras. Combining the recognition of targets in each image with the knowledge of where the targets are in respect to the vehicle allows offline calibration to calculate a correction factor that is applied to the nominal camera position and orientation.

Offline calibration makes sure that objects transition smoothly and without visual artifacts like double-images from one camera view to the next.

Even a perfectly offline calibrated surround view system is subject to changes over time: Aging mechanical components, vehicle loading and stress due to thermal expansion and contraction can cause slight but noticeable changes to camera alignment. Online calibration continuously monitors for such changes and takes corrective action.

Online calibration is based on scale-invariant feature transform (SIFT) and speeded up robust features (SURF). Using vehicle sensor information the movement of features in the different camera images is compared with the expected movement based on the vehicle’s egomotion. A partial image remap is applied based on pose correction estimations.

Cogent Surround View leverages several open source packages.

• GStreamer is a library for constructing graphs of media-handling components.
• Cairo is a 2D graphics library with support for multiple output devices.
• EGL is an interface between Khronos rendering APIs (such as OpenGL, OpenGL ES) and the underlying native platform windowing system.
• GLib is a general-purpose utility library, which provides many useful data types, macros, type conversions, string utilities, file utilities, a mainloop abstraction, and so on.
• OpenGLES is a royalty-free, cross-platform API for full-function 2D and 3D graphics on embedded systems – including consoles, phones, appliances and vehicles.
• Wayland is intended as a simpler replacement for X, easier to develop and maintain.
• LibPCAP is a system-independent interface for user-level packet capture.
• OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision.
• NLopt is a free/open-source library for NonLinear OPTimization, providing a common interface for a number of different free optimization routines available online as well as original implementations of various other algorithms.
• GLM (OpenGL Mathematics) is a header only C++ mathematics library for graphics software based on the OpenGL Shading Language (GLSL) specifications.
• ZLib is a software library used for data compression.
• LibSPNAV is a library to access 3D-input-devices.

Cogent’s embedded surround view code is highly customizable. In fact, there are hundreds of parameters that can be tweaked to achieve just the right look for any given vehicle. A set of PC tools is available to simplify the task of adapting Surround View to a given vehicle. The tools allow easy configuration of camera locations, provide intrinsic and extrinsic camera calibration, mesh configuration, virtual camera positioning, and user interface configuration.

For the US market the toolchain enables developers to create an “FMVSS 111” scene to verify that the vehicle meets the federal motor vehicle safety standard.

A camera calibration tool is available for demo installations in which camera intrinsic parameters may not be known.

Cogent Surround View comes with built-in video recording and video file playback capability. This eliminates the need for complex and expensive external tools, allowing developers to test the system with a standard hardware prototype.

* Optional but beneficial