Automotive interior sensing is expected to experience substantial growth in the next five years, with driver monitoring systems (DMS) and occupant detection systems (ODS) soon becoming mandatory. According to Yole Développement’s latest report, Automotive Interior – From lighting and sensing to display technologies 2020, the sensing market for DMS and ODS could grow from $226 million in 2020 to $2.6 billion in 2025, representing a 63% compound annual growth rate. DMS could see the emergence of 3D cameras, and ODS could see a combination of camera and radar-based sensors.
Cédric Malaquin and Pierrick Boulay, Market and Technology analysts at Yole Développement, had the opportunity to talk about in-cabin sensing with Vinay MK, Co-Founder & VP Products at PathPartner Technology. Discover the detail of their discussion below.
Cédric Malaquin (CM): Can you please introduce yourself, your activities, and the activities of PathPartner?
Vinay MK (VM): My name is Vinay MK. I am Co-Founder & VP Products @ PathPartner Technology. PathPartner is a product design house founded in 2006. I lead the team designing driver monitoring (DMS) and in-cabin monitoring solutions (ICMS) for both new car OEMs and aftermarket product makers. My team works synergistically with other teams in PathPartner, which excels in providing advanced software integration, validation, and system testing to all the major global automotive Tier-1s. Through innovative technologies built around deep learning, computer vision, multimedia, imaging, and the internet of things, PathPartner solves complex business challenges for our customers. We have development centers in Bangalore & Cochin, India, Frankfurt, Germany & Fremont CA, USA.
CM: The adoption of Driver Monitoring Systems (DMS) is increasing, and EU regulation will make it mandatory in new vehicles starting in 2022.
Do you see strong demand from OEMs? Are they ready to implement such a feature? Do you think some OEMs will ask for a delay to implement such a feature?
VM: The pandemic year 2020 has definitely created considerable logistical and demand-related business challenges to automotive OEMs worldwide. Euro NCAP is more likely to defer DMS adoption dates by a year or so. However, in the long run, DMS and ICMS are essential for passive safety, autonomous driving & for new user experience use-cases. Innovation and R&D teams of OEMs are definitely becoming involved in understanding & differentiating their offerings through these technologies. Therefore, OEMs are now eligible for an additional point for DMS implementation, with a clear roadmap of doubling it by 2024.
Pierrick Boulay (PB): Current driver monitoring systems are based on the combination of a 2D camera and IR LEDs.
How will this technology evolve (using IR-RGB cameras, stereo cameras, or 3D cameras)? What would be the benefits of using a combination of a 3D camera and VCSELs (as in smartphones)? When do you think it will be implemented by OEMs?
VM: PathPartner is at the forefront of adopting RGB-IR sensor technology. New camera sensors now come with RGB-IR sensor elements, though existing application processors are not equipped to handle this new Bayer pattern. We’ve not only productized ISP adaptations for the TDA4x + OV2312 combination but have also built a custom ICMS pipeline leveraging an RGB path for facial recognition and an IR path for DMS. We are working with STMicroelectronics’ NIR sensor to develop and productize 2.5D facial recognition technology for automotive use cases. Life presence detection and other Airbag related passive safety use cases open further opportunities for 2.5D cameras + 4D radar sensors (radar at 60GHz) for in-cabin applications.
PB: What is the relation between the implementation of driver monitoring systems and automation levels of vehicles? Do you think that OEMs could use DMS not only to face the drowsiness issue, but also in combination with ADAS features to confirm that the driver has seen an alert raised by the vehicle?
VM: Autonomous driving stack providers are the first and most intense partners for our DMS box. We are working with a US-based AD stack provider for commercial trucks and have already installed a few tens of vehicles for road validation. Higher automation would also mean a greater need for in-cabin context understanding. Combining ADAS features with DMS would be needed to make semantically relevant decisions. Turn indicators and speed are two of the basic parameters our DMS takes in before raising alerts on drowsiness & distraction. Our ADAS partners combine contextual information coming from the camera and DMS to make more informed decisions. There will always be a shared authority between the driver and the vehicle over the driving tasks until the highest level of automation comes, i.e., level 5, which only expects the driver to provide the destination input and takes care of the rest. Till, then DMS is a necessity for all vehicles.
PB: Occupant Detection System (ODS) is being extended from front to rear seat. Child Presence Detection (CPD) systems will be mandatory by 2022.
Do you already see the demand from OEMs?
VM: Life presence detection (LPD) is going to be a mandatory safety feature in passenger vehicles across Europe and the US. OEMs are looking at cost-effective options to enable this safety feature. In-cabin radar provides a viable option. With the introduction of the Hot Cars Act, detecting the presence of children and pets left in the car is a real need. PathPartner is probably the only company providing both Camera-based and Radar-based interior sensing SDKs to the OEMs.
While LPD is the need of the hour, this will eventually lend itself to generic occupant detection systems to be used in both passive safety and comfort use cases. Covering rear seat passengers from a single camera at the front is not going to be satisfactory. While safety is a concern, violation of privacy and security of occupants also need to be addressed. 4D radar technology for in-cabin sensing helps secure safety and also ensures the privacy of occupants.
CM: Multiple technologies are under consideration for this feature, such as camera, ultrasonic, and radar.
Which one of those is driving more interest from the OEMs and why?
VM: A combination of Radar and Camera acts as the best solution for Occupant Monitoring. Ultrasonic technology is still in proof of concept, camera alone systems have constraints concerning range and privacy, but 4D radar is emerging as an option that covers reliability, safety, privacy, and cost constraints. Combining Camera + Radar for occupant monitoring will be a game-changer in achieving safety-critical functions with cost-effective options. This sensor fusion model not only detects the occupants but also classifies them into different age groups and triggers seat belt detection. The capability of radar to detect the occupants is pretty high in a variety of conditions and it can be perfectly combined with the camera to reduce false alarms and increase the field of vision, offering therefore, high accuracy.
CM: Radar-based sensors for in-cabin detection are not standardized yet. Various operating frequencies can apply, such as 60 or 79 GHz.
Do you see one or another being demanded more by the OEMs and why?
VM: Right now, we are working closely with a Tier-1 for a start-of-production program using our 60-64 GHz frequency band radar solution. As you are aware, there is no clear standard on radar usage for In-cabin applications. However, given our understanding of EU, ICNIRP & FCC regulations, we are fairly certain that this band will be standardized in Europe and the US regions.
The compliance with these regulations goes beyond the frequency band and extends to the various parameters of the RF, like antenna power, EIRP, duty cycle, dwell time, and other such constraints. Worldwide acceptance may vary, as defined by the respective regulatory bodies. We expect this to be settled soon, given that OEMs are now convinced of radar’s viability as a preferred sensor for LPD and OMS use cases.
Vinay MK is a part engineering manager, a part entrepreneur, a part data analyst and a whole of a dsp engineer. He has been around in the industry for over 2 decades now. He has played various avatars in these many years and in his current form plays the role of VP-Engineering at Pathpartner. He has been the principal engineer designing andimplementing video decoders for customers in formative years of Pathpartner. He gets his daily dose of high from coding and has been managing staff-augmentation and consulting business.
Vinay is a co-founder of PathPartner.
Vinay received his B.E. degree in Electronics and Communications from National Institute of Engineering, University of Mysore, India in 1998 & his M.Tech degree in Signal Processing from Indian Institute of Technology, Kanpur, India.”
As a Technology & Market Analyst specialized in RF devices & technologies within the Power & Wireless division at Yole Développement (Yole), Cédric Malaquin is involved in the development of technology & market reports as well as the production of custom consulting projects.
Prior to his mission at Yole, Cédric first served Soitec as a process integration engineer for 9 years, then as an electrical characterization engineer for 6 years. He contributed significantly to FDSOI and RFSOI product characterizations. He has also authored or co-authored three patents and five international publications in the semiconductor field.
Cédric graduated from Polytech Lille in France with an engineering degree in microelectronics and material sciences.
As part of the Photonics, Sensing & Display division at Yole Développement (Yole), Pierrick Boulay works as Market and Technology Analyst in the fields of LED, OLED and Lighting Systems to carry out technical, economic and marketing analysis. He has experience in both LED lighting (general lighting, automotive lighting…) and OLED lighting. In the past, he has mostly worked in R&D department for LED lighting applications. Pierrick holds a master degree in Electronics (ESEO – France).
Automotive interior enrichment is becoming a key market segment for electronic and semiconductor industries.
Worth more than $20B in 2019, the radar industry is experiencing a major transformation prior to entering the commercial era.
Related Reports and Monitors
Status of the Radar Industry: Players, Applications and Technology Trends 2020
Market & Technology
CMOS Image Sensor Quarterly Market Monitor