Getting Started in Automotive Smart Vision

Display portlet menu

Getting Started in Automotive Smart Vision Design

truck driver, highway and IoT icons

Advances in embedded vision technology have heightened interest in applying smart vision solutions for automotive safety. Despite past difficulties in overcoming the complexity of embedded vision design, designers today can take advantage of image-processing software, vision processors and specialized development systems to get started in exploring automotive smart vision applications.

Embedded vision technology offers tremendous potential for enhancing automotive safety by simplifying the driving process. Unlike the simple rear-vision cameras commonly available today, automotive embedded vision provides the foundation for automatic driver assistance systems (ADAS) intended to relieve the cognitive load on drivers and enhance overall safety. Providing 360° coverage, automotive vision design seeks to surround the vehicle with vision systems able to recognize and track potential hazards -- helping drivers through lane-departure warning, collision avoidance, pedestrian detection and more.

Until recently, the concept of automated, vision-based hazard identification and tracking remain largely limited to research efforts or highly funded mil/aero projects. The emergence of readily available software and hardware solutions for embedded vision has dramatically lowered the barrier to entry -- and fueled a rapid growth in interest in leveraging embedded vision technology in automotive safety.

Software solutions

The maturation of the open-source OpenCV library has enabled designers to deploy highly sophisticated computer vision algorithms without requiring specialized knowledge in image-processing theory.  Written in optimized C/C++, the library supports C++, C, Python and Java interfaces for Windows, Linux, Mac OS, iOS and Android platforms and features libraries that span a broad range of computer vision functionality.

For computer vision, object identification is an essential requirement and OpenCV provides developers with a broad range of object classification routines. For example, the object detection library implements some high-level object classification models. For custom object-detection approaches, the machine learning library supports more fundamental classifier methods including Bayes, Support Vector Machines, Decision Trees and Neural Network classifiers, among others.

A particularly compelling feature of OpenCV is its support for associated libraries designed to exploit specific hardware features of advanced processors. For example, the OpenCV library accelerates OpenCV functionality on OpenCL-compatible hardware including standalone processors, graphics processors, core processors and FPGAs. The module uses lower-level OpenCL runtime routines built to take advantage of the device's specialized hardware features provided by each hardware manufacturer. The emerging OpenVX library extends this approach by not only serving to accelerate packages such as OpenCV, but also providing direct support for applications including ADAS.

Beyond these open-source offerings, designers can also find fully supported commercial software packages such as the MathWorks Computer Vision System Toolbox ™. The MathWorks suite provides an extensive array of vision-processing algorithms as MATLAB® functions, System objects™, and Simulink® blocks with automatic C-code generation.

Image-processing hardware

For exploring the design of computer vision systems, designers can turn to open-source hardware such as Arduino. Yet, while offering a platform for evaluating basic vision applications, general purpose hardware will typically lack the performance required for implementing advanced computer vision applications and deploying capabilities such as real-time object identification.

Indeed, computer vision algorithms present significant computational challenges in both quantity of data and complexity of computation. In the past, commercial developers could only dream of real-time object detection for lack of cost-effective hardware solutions able to execute sophisticated algorithms on streaming data sets.

Today, silicon manufacturers have targeted computer vision applications with specialized processors built with dedicated on-chip image processing capabilities. The availability of specialized image-processing hardware such as the Analog Devices BF609 Blackfin® processor and Xilinx Zynq® 7000 All Programmable SoC stands as a key enabler of the emergence of widely available automotive embedded vision solutions.

The BF609 Blackfin processor combines a pair of Blackfin DSP cores with Analog's specialized PVP (pipelined vision processor) to accelerate image processing. Capable of processing up to four data streams simultaneously, the PVP can be dynamically reconfigured to form different pipeline structures required for different vision processing algorithms. At the same time, the BF609 integrates memory, system and watchpoint protection features needed in functional safety applications such as automotive vision. Analog Devices provides its Eclipse-based CrossCore® Embedded Studio for software design and debug.

The Xilinx Zynq 7000 All Programmable SoC supports the computational demands of vision processing through a combination of ARM Cortex™-A9 cores, NEON DSP/FPU engines and an FPGA fabric. While the cores support application execution, the FPGA fabric enables designers to implement custom high-speed image processing pipelines. For software design and debug, the Xilinx Vivado™ suite offers a comprehensive development environment tuned to the Zynq device.

Vision development kit

The combination of specialized computer vision libraries and dedicated image processors offers the fundamental elements required to build sophisticated automotive embedded vision applications. Yet, combining these separate pieces into a complete automotive-vision development platform can be a daunting task in itself for developers more interested in creating vision solutions than dealing with development platform integration.

Getting started with automotive embedded vision has become dramatically simpler thanks to the emergence of complete automotive-vision development platforms such as the Avnet Blackfin Embedded Vision Starter Kit. The centerpiece of the kit is the FinBoard development board featuring a full complement of peripherals required to build a complete computer vision system. Based on the Analog Devices BF609 processor, the FinBoard includes a 720p CMOS color sensor, HBLEDs for illumination, HDMI output, 128 MB RAM, 32 Mb Quad SPI flash and microSD card cage.

Avnet Blackfin Embedded Vision Starter Kit

Fig. 1:  The Avnet Blackfin Embedded Vision Starter Kit builds on the FinBoard, a complete computer-vision system based on the Analog Devices BF609 image processor.(Source: FinBoard)

Along with the FinBoard development board and Analog's CrossCore development suite, the Avnet kit includes the Analog Devices ICE-100B JTAG debugger. Combined with the Blackfin emulator, the ICE-100B enables developers to communicate with the Blackfin processor on a host PC. Using the debugger, engineers can operate in single-step or at full speed with predefined breakpoints as well as alter register and memory contents as needed.

Beyond its comprehensive hardware and software components, the Avnet kit finds extensive support from the community-based FinBoard.org site. Along with online documentation, training resources and community forums, FinBoard.org offers registered users access to reference designs able to kick-start new projects. For example, a reference design for implementing video pass through provides a step-by-step introduction on using the CrossCore suite to implement a basic video application on the FinBoard. Similar reference designs introduce new users to the FinBoard with a basic power-on self test application or show more experienced FinBoard users how to implement a Canny Edge detector.

In combination, powerful image-processing software and specialized vision processors offer compelling solutions for creating sophisticated automotive embedded vision applications. With the availability of comprehensive starter kits, getting started in this fast-growing segment has never been easier.

Related Articles
automobile with sensors collecting information
We’re All Part of the Automotive Evolution
April 23, 2019
ADAS (Advanced Drive Assist Systems) and its evolution into full autonomy could see the end of road-based fatalities, injuries and incidents. Technology is now providing the solutions that can help the automotive industry achieve Vision Zero within j
test car colliding with object
Transportation Safety: 5 Protocols & Processes to Know
April 4, 2019
A dynamic range of protocols can help make our transportation technology safer.
cargo ship at port
Transportation Trends in Commercial & Non-Passenger Vehicles
March 29, 2019
Some of the most exciting technology trends shaping transportation are for commercial vehicles.
futuristic city scene
The Car: A Rolling Smart Device
February 5, 2019
Be it for streaming your favorite music, sending emails or getting real-time information on traffic jams: cars have long since played host to mobile internet—and will continue to via diagnostics, hotspots and ADAS.
man using tablet computer in industrial setting
5 of the best artificial intelligence use cases
January 3, 2019
AI’s upgraded algorithms make predictive analytics, parse data, and help businesses make smarter decisions from the boardroom to the factory floor.
Graphic of a green car flying in the air
Autonomous vehicles are the future
April 9, 2018
Autonomous vehicles do not just replace the driver, helmsman or pilot, but have the potential to create completely new business models worth billions.
Interior of European self-driving car
Meeting Next-Generation Automotive Design Challenges
April 2, 2018
As the automotive industry advances toward Level 5 fully autonomous vehicles, automotive engineers will have to continue driving more functionality out of vehicle subsystems that deliver safety, infotainment, and other features.
smartphone showing alarm feature
Key Design Considerations for Selecting the Right RF Antenna
March 16, 2018
Know when to choose standard, when to go custom.
futuristic automobile interior
How all programmable technology revolutionizes embedded vision
December 14, 2017
Autonomous driving is just the start of EV solutions. Learn how all-programmable is revolutionizing embedded vision.
close-up of orange car headlights
Automotive LEDs Deliver Greater Design Flexibility for Safety, Style and Efficiency
March 7, 2017
LED lighting has been making major inroads into the automotive market for two key reasons: safety concerns and energy consumption.
man drawing schetch of an electric car attached to a battery
Technologies and Components for Designing Electric Vehicles
March 6, 2017
Hybrid electric vehicles (HEVs) such as the Toyota Prius and the Chevy Volt and electric vehicles (EVs) such as the Nissan Leaf, BMW i3 and Tesla Model S are growing in popularity amid concern for global warming.
military plane taking off
Defense/Aero Spinoffs Remain an Exceptional Source for Breakthrough Technology
March 5, 2017
Yes, automatic across the board cuts within the Department of Defense (DoD) will put the squeeze on many within the defense/aerospace supply chain.
self-driving car at intersection
Top 5 Tech Trends in Advanced Driver Assistance Systems
March 4, 2017
Advanced driver assistance systems (ADAS) promise to enhance vehicle safety by helping to simplify the driving process, reducing sources of driver distraction and inattention that often lead to accidents. With ADAS support, drivers and their passenge
rear view camera on car dash with image of child riding a toy
Top 5 Myths in Automotive Vision: Designing Embedded Vision Systems Is Easier Than You Think
March 3, 2017
Vision has always occupied a special place in information science and popular culture. One does not need to be an engineer to appreciate the vast bandwidth available in normal human vision. Most people understand that the common saying “a picture i
hand navigating smart car dasboard
Automotive Electronics: Top 5 Tech Trends of Tomorrow’s Smart Cars
March 3, 2017
In the United States alone, motor vehicles travel well over four trillion miles each year according to the U.S. Department of Transportation.* To gain a sense of this distance, consider that while it takes light approximately eight minutes to travel
blue sketch of a car
Gesture Recognition, Proximity Sensors Drive Advances in Automotive Infotainment
March 3, 2017
Safety must remain paramount when designing interactive interfaces for automotive applications, including guarding against distracted driving.
Man navigating an IoT car dashboard
The Internet of Things is Driving The Internet of Autos
February 14, 2017
In the early 1900s Henry Ford made the automobile affordable and accessible. But what he really did was offer people connection. Rural residents could connect with more urban areas to sell crops and buy supplies.
Related Events
Connector Design Considerations for Advanced Automotive Systems
Date: October 29, 2019
Location: Webinar
XENSIV sensors from Infineon
Date: October 1, 2019
Location: Webinar