Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Display portlet menu

Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Display portlet menu

Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Graphic of a car's outline

At a time when social media interactivity is also making its way into cars, an evolution of the instrument control panel is necessary if drivers are to connect and perform tasks simultaneously, and safely. Improved gesture and speech recognition are two of the more prominent cockpit human-machine interface (HMI) technologies being developed by carmakers to deliver a safe, reliable interaction between driver and vehicle.

Gesture recognition: Hands in command

Gesture recognition technology is widely expected to be the next generation in-car user interface. Gesture recognition determines whether the driver has performed recognizable hand or finger gestures within an allotted space without contacting a touchscreen. For example, an approaching hand can activate the in-car infotainment system, or in a more sophisticated system, the driver can touch the steering wheel then tilt his head left or right to turn the volume of the stereo up or down. A camera placed in the steering wheel or on the dashboard is programmed to watch for certain gestures. When it sees them, it sends a signal to the processor that handles the connected infotainment hardware. The data is analyzed to determine what the driver is doing, ascertain which display controls the driver wants to adjust and then activate the appropriate features.

This technology may well be familiar to gamers. That’s because it is not unlike Microsoft’s Kinect system for the Xbox game console, which detects motion from distances of up to approximately 10 feet. However, rather than tracking a user’s entire body motion as is done with a Kinect system, only the users’ hand gestures are analyzed for automotive infotainment applications.

A market study conducted in 2013 by IHS Automotive examined gesture-recognition technology and proximity sensing. According to IHS, the global market for automotive proximity and gesture recognition systems that allow motorists to control their infotainment systems with a simple wave of their hand will grow to more than 38 million units in 2023, up from about 700,000 in 2013. Automakers including Audi, BMW, Cadillac, Ford, GM, Hyundai, Kia, Lexus, Mercedes-Benz, Nissan, Toyota and Volkswagen are all in the process of implementing some form of gesture technology into their automobiles.

Hyundai’s HCD-14 is a luxury four-door concept sedan featuring gesture controls for audio, HVAC, navigation and smartphone connectivity functions. The driver selects a main function by gazing at a heads-up display (HUD), presses a thumb button on the steering wheel to confirm his/her selection and then performs gestures, which include moving a hand in or out from the dashboard to zoom in or out on the navigation system, a dialing motion to adjust volume and a side-to-side gesture to change radio stations.

Similarly, Visteon’s Horizon cockpit concept, which the company has been demonstrating to global vehicle manufacturers, uses 3-D gesture recognition to transform the way a driver controls such features as interior temperature, audio and navigation. In the Horizon cockpit concept controls can be manipulated by moving the hand or just a finger. Radio volume, for example, can be adjusted by making a turning motion with one’s hand without making contact with the instrument cluster. The gesture recognition technology uses a camera system to map the user’s hand and replicates a virtual hand on the center stack display.

The industry answers with new sensors and tools

Semiconductor suppliers are developing the hardware needed to enable user command input with natural hand and finger movements. For example, Microchip’s MGC3130 is a three-dimensional gesture recognition and tracking controller chip based on the company’s patented GestIC technology, which uses an electric field (E-field) to provide gesture information as well as positional data of the human hand in real time. E-fields are generated by electrical charges and propagate three-dimensionally around the surface carrying the electrical charge. Applying direct voltages (DC) to an electrode results in a constant electric field. In case a person’s hand or finger intrudes the electrical field, the field becomes distorted. GestIC technology uses a minimum number of four receiver (Rx) electrodes to detect the E-field variations at different positions to measure the origin of the electric field distortion from the varying signals received. The information is used to calculate the position, track movements and to classify movement patterns (gestures).

A number of automotive touchscreens now employ proximity sensing, a technology that can enable touch-free interfaces in infotainment systems, keyless entry systems and lighting controls. The Cadillac User Experience was the first system to offer proximity sensing in a mass-market production vehicle. A pair of infrared sensors just below the screen can detect when a user’s hand approaches the screen and activates frequently used menus, such as a list of mixed presets and navigation options.

Cypress Semiconductor’s CapSense touchscreen sensor, which can detect fingers without contact, is also being used for proximity sensing in automotive and other applications, making it feasible for drivers to move fingers up or down to scroll to the next page, among other actions. CapSense sensors leverage the supplier’s PSoC (Programmable System-on-Chip) architecture. CapSense works by measuring the capacitance of each I/O pin on the CapSense controller that has been designated as a sensor. Proximity sensing wake displays from sleep modes when the sensor detects the presence of a hand or finger. And the CY8C20xx7/S CapSense and CapSense Plus controllers consume just 28 µA/sensor in run time with a wake-up and scan once every 125 ms. To get engineers started, Cypress offers a PSoC 4 Pioneer Kit CY8CKIT-042 that includes a CapSense slider, an RGB LED and a user button. 

Fig. 1: Cypress CapSense sensors do not require actual contact. (Source: Cypress Semiconductor)

A detector IC can be used together with up to three separate infrared LEDs to form a sensor that can detect 3-D movements of objects in front of the device. With this information the content of a display can be controlled just by hand waving. Detection ranges up to 20 cm are achievable, and the range can easily be extended by using high power emitters. The SFH 7770 E6 from OSRAM Opto Semiconductors combines the functions of a digital ambient-light sensor with those of a digital proximity sensor. It determines the ambient brightness, detects the presence of an object nearby and can tell whether the object is moving closer or moving further away. This product can be used wherever short-range gesture recognition is needed.

Voice control: Tell me what you want

When traffic situations make driver interaction with infotainment apps potentially hazardous, voice control based on speech recognition can be a useful complement to conventional HMIs. Market research company IHS expects more than half of new automobiles in 2019 to integrate voice technologies such as voice recognition, text-to-speech and speech-to-text to enable drivers to control entertainment and navigation systems simply by using their voices.

In order to do so, speech interfaces in vehicles must overcome a low-adoption rate among drivers, primarily due to perceived accuracy issues. Voice functions must actually work, providing true, reliable voice integration. If it is to make the leap from purely task-oriented command and control to a more sophisticated, user-centric interface designed to negate distraction issues, voice systems must interact in a more conversational way instead of being restricted to a list of fixed, predefined menu phrases.

The good news is that speech recognition systems in the form of hands-free text-to-voice or voice-to-text operations are now delivering greater accuracy and employing more flexible grammar libraries. For example, Ford’s SYNC, based on the Microsoft Windows CE operating system, supports up to 10,000 voice commands with no training required for the system to recognize the commands.

It will take a lot of computer muscle to power next-generation speech-based infotainment applications, enabling them to be context-aware and respond more quickly and with increased accuracy. Some speech functions will be performed on-board by the infotainment system and others, like dictating email or web content, may be performed off-board by servers in the Internet service provider’s data center.

Processors such as the Intel Atom E640 series have the computing headroom to perform critical- noise and echo cancellation functions, thereby eliminating the need for a digital signal processor (DSP) and providing speech applications and far-away listeners on phone calls with a cleaner input signal.

Multimedia information consoles are now the predominant feature of dashboards in new cars. Safety concerns are driving demand for in-car hand gesture and voice recognition technology as a potential solution to driver distraction issues that will be more conspicuous as connectivity on the road becomes a default feature of most new cars. The potential for both technologies in human-machine interface development is immense, and IC manufacturers are quickly integrating the necessary functionality into MCUs and sensors to make it a reality.

Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Display portlet menu

Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Display portlet menu
Related Articles
interior of autonomous car
The State of Automotive Only Starts with Autonomous Driving
March 6, 2020
Learn how the state of automotive spans a variety of applications, from electrication to in-cabin AI.
view from autonomous vehicle as it interprets street signs
Somewhere Between the Flintstones and the Jetsons
March 6, 2020
Learn more about developing adaptable automotive automation with Xilinx
Smart car branded with Avnet and supplier logos
ADAS: paving the way for autonomous vehicles
November 11, 2019
In order to realize fully automated driving without human intervention, three conditions must be met - the vehicle must be fully aware of the surrounding environment, be able to respond accordingly when the environment changes, and the security of th
road view from inside self-driving smart car
Unreliable autonomous driving? Take a look at how difficult vehicle-mounted vision processing is
November 11, 2019
We all have high expectations for autonomous driving, a fact that’s exemplified by the display of autonomous driving technologies at every major tech expo. Development in this sector has however been accompanied by an increase in accidents related
test car colliding with object
Transportation Safety: 5 Protocols & Processes to Know
May 29, 2019
A dynamic range of protocols can help make our transportation technology safer.
cargo ship at port
Transportation Trends in Commercial & Non-Passenger Vehicles
May 29, 2019
Some of the most exciting technology trends shaping transportation are for commercial vehicles.
Graphic of 5G uses for automotive
5G is here – and maybe it’s time to buy a new car!
May 27, 2019
The mission of 5G is to create the ‘Internet of Everything’, drastically expanding the scale of the Internet in the process. As one of our most valuable possessions, it is inevitable that cars will join the network.
Interior of European self-driving car
Meeting Next-Generation Automotive Design Challenges
April 2, 2018
Thanks to engineering ingenuity and the electronic components that have become pervasive inside vehicles of all types, cars are smarter and, in many ways, safer to drive. As the automotive industry advances toward Level 5 fully autonomous vehicles, a
smartphone showing alarm feature
Key Design Considerations for Selecting the Right RF Antenna
March 16, 2018
Know when to choose standard, when to go custom.
Person using the navigation systemm on their car dashboard
Four tech trends that improve the driver experience
September 13, 2017
Learn about ways to integrate some the most innovative features into cars, including haptic-feedback touch screens, knob replacement, smart glass and driver-notification applications.
smart car on the road at sunset with IoT icons overlay
Mastering the road ahead
August 28, 2017
The market demand for Advanced Driver Assistant Systems (ADAS) has increased by leaps and bounds. In fact, the global automotive electronics market is forecast to grow robustly over the next few years. In particular, the market value of entertainment
side view of a sports car from the ground up
Powering the future—the hidden heroes of electric vehicle technologies
August 14, 2017
The automobile industry in Mainland China has developed at an unimaginably rapid pace in recent years and a general trend can be observed in car sales figures. A little more than twenty years ago, automobile production in Mainland China numbered only
man in dirverless car with hands off the wheel
The Growth and Increasing Sophistication of ADAS in the World
July 5, 2017
As electronics in our cars continue to become more sophisticated, ADAS (Advanced Driver Assistance Systems) are taking center stage. These systems make many of the aspects of driving easier, and---most importantly—safer.
self-driving car at intersection
Top 5 Tech Trends in Advanced Driver Assistance Systems
June 15, 2017
Advanced driver assistance systems (ADAS) promise to enhance vehicle safety by helping to simplify the driving process, reducing sources of driver distraction and inattention that often lead to accidents. With ADAS support, drivers and their passenge
rear view camera on car dash with image of child riding a toy
Top 5 Myths in Automotive Vision: Designing Embedded Vision Systems Is Easier Than You Think
June 11, 2017
Vision has always occupied a special place in information science and popular culture. One does not need to be an engineer to appreciate the vast bandwidth available in normal human vision. Most people understand that the common saying “a picture i
man drawing schetch of an electric car attached to a battery
Technologies and Components for Designing Electric Vehicles
April 29, 2017
Hybrid electric vehicles (HEVs) such as the Toyota Prius and the Chevy Volt and electric vehicles (EVs) such as the Nissan Leaf, BMW i3 and Tesla Model S are growing in popularity amid concern for global warming.
Man navigating an IoT car dashboard
The Internet of Things is Driving The Internet of Autos
April 24, 2017
In the early 1900s Henry Ford made the automobile affordable and accessible. But what he really did was offer people connection. Rural residents could connect with more urban areas to sell crops and buy supplies.
smart car dashboard
New TFT LCD Technology Shape Infotainment for Cars of the Future
April 15, 2017
Although the technology has been around since the 1990s, the use of thin-film-transistor liquid-crystal displays (TFT LCDs) is on the rise in automobiles.
person driving a semi
Getting Started in Automotive Smart Vision Design
March 27, 2017
Advances in embedded vision technology have heightened interest in applying smart vision solutions for automotive safety.
Person using the navigation console in their car
Automotive Electronics: Top 5 Tech Trends of Tomorrow’s Smart Cars
March 3, 2017
In the United States alone, motor vehicles travel well over four trillion miles each year according to the U.S. Department of Transportation.*
WiFi router with man in the background using tablet
How wireless communication protocols make or break designs
March 3, 2017
The design process includes many critical decisions for engineers, such as which wireless communications protocol to use.

Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment

Display portlet menu
Related Events

No related Events found