HomeProductsProducts Details

STMicro's lowlight image sensor saves one LED in driver monitoring system

Date: 19/09/2022
AI based human physical behaviour monitoring solution provider Smart Eye has used STMicroelectronics' devices in developing 1-LED Driver Monitoring System (DMS) for automotives. Smart Eye has applied its People monitoring AI algorithms and used STMicro's lowlight supporting highly sensitive automotive-grade global-shutter VB56G4A image sensor in developing this new DMS with lower power consumption and lesser cost by using only one single illumination LED lamp, saving two or more LEDs which are used in present DMS systems. This DMS requires a very simple low-power, non-visible near infrared LED emitter to provide adequate illumination for the sensor. Since it is near infrared which is outside visible spectrum, makes DMS work well during both day or night driving and in bright or overcast conditions, to take care safetly problems arising from potential driver-fatigue/slipping to sleep while on wheels.

“The excellent performance and high sensitivity of ST’s automotive-grade global-shutter imaging sensor is an excellent match for Smart Eye’s industry-proven driver-monitoring algorithms,” said Eric Aussedat, Executive Vice President, Imaging Sub-Group General Manager, STMicroelectronics. “In developing a sensor with 2.6µm pixel size and double the quantum efficiency of previous-generation sensors, we’ve simplified DMS development, reduced its power consumption, and lowered costs without compromising performance.”

“Building on our two decades of automotive experience proven by nearly 100 design wins from leading car manufacturers, we’ve worked with ST to combine our technology with ST’s advanced image sensor to eliminate at least half of the expensive LEDs/lenses,” said Martin Krantz, CEO and Founder of Smart Eye. “Even with this smaller and more economical design, we’ve created an exceptional no-compromise DMS demonstrator that shows how automakers can significantly improve road safety with reduced cost of ownership. We are happy to support our partners and customers with our HW and SW reference design and expertise for flawless product integration.”

This solution is demonstrated at the ST stand at recently held AutoSens on September 12-14 at the Autoworld Museum in Brussels and in the same location at InCabin on September 15.

VG56G4A is manufactured using advanced 3D-stacked back-side illuminated (BSI-3D) process making it more sensitive to near-infrared light, and small in size, and more reliable compared to front-side illuminated (FSI) sensors normally used in first-generation DMSs.

VG56G4A offers high Quantum Efficiency (QE), reaching 24% at 940nm near-infrared (NIR) wavelength, with linear dynamic range up to 60dB.

Another interesting feature of this sensor is it uses global-shutter approach instead of rolling-shutter imagers, where it simultaneously exposes all pixels to the scene, allowing simple synchronization with NIR illumination. This feature to save power consumption further.

ST says "Combining a pixel size of just 2.6µm with the high QE and global shutter helps optimize total power consumption and camera size. In addition, integrated automatic exposure control eases use and simplifies the application-software design by minimizing system interaction with the sensor."

Other interesting feature of this sensor is it offers flexible operating modes, programmable sequences of 4-frame contexts, illumination control outputs synchronized with sensor integration periods, and an input for an external frame-start signal. Also features such as automatic dark calibration, dynamic defective-pixel correction, image cropping, and a mirror/flip-image readout, makes it a good choice for DMS systems.

It has eight programmable general-purpose I/O (GPIO) pins and a dual-lane MIPI CSI-2 transmitter interface operating up to 1.5 Gbps per lane. It can capture 88 frames per second (fps) at full resolution. The typical power consumption is 145mW at 60 fps.


Smart Eye driver monitoring systems gaining popularity in automotive safety market, where Smart Eye reported more than 1 million cars are installed with Smart Eye's driver monitoring system software. It has 14 automotive OEM customers, among which six have already started series production of car models having Smart Eye technology to monitor the driver. BMW is one of the top customer of Smart Eye.
Though it did not name the other customers but stated "one of US largest automakers, two of China’s largest automakers, a Japanese-European multinational car alliance and a South Korean multination automobile manufacturer" as its other OEM customers.

Here is another quote from Martin Krantz, CEO and Founder of Smart Eye:
“For the past 20 years we have worked towards this important milestone, but I see this as only the beginning. In most of these first million cars, DMS has been an optional feature. Thanks to regulation and safety requirements, almost all the remaining car models we have won will have mandatory DMS in every single vehicle. To put things in perspective: if all 94 car models we have won so far were to reach peak production at the same time, we could be looking at a yearly volume of as much as five million cars produced with our technology – every year”.