Volvo Incorporates SeeingMachines System for Driver Inattention Research
IVsource.net
8 November 2001

Volvo researchers in Sweden, working with partners in Australia, have built and tested an integrated attention monitoring system -- providing a new approach to studying inattention and driving behavior.

Volvo researchers in Sweden, working with partners in Australia, have built and tested an integrated attention monitoring system which provides a new approach to studying how inattention affects driving behavior.  The system includes a novel head- and gaze-tracker based on the system available from SeeingMachines, a lane tracker, and integration with the Car Area Network (CAN) vehicle data bus.  Preliminary road tests show that the head- and gaze-tracker is robust against variations in lighting and vibration, and is quite reliable overall.  Volvo hopes to guide future development of human-system interfaces by learning about the visual demands of interiors and about visual behavior in general.

The research is led by Dr. Trent Victor of the Human Systems Integration Department, Volvo Technological Development Corporation, who also has an appointment to the Graduate School of Human-Machine Interaction at Linköping University in Sweden.  The  research is motivated by the fact that driver inattention is the most prevalent primary cause of collisions, accounting for between 25% and 50% of collisions.

To assist drivers effectively, Volvo believes they need to be able to collect real-time data on driver visual behavior, recognize what the driver is doing (using 'contextual' information such as maneuvers, actions, and states), predict what the driver would likely do next, and, finally, design an interface to assist the driver.  Researchers stress the importance of context -- attention support systems should ideally detect the co-occurrence of inattention and safety critical events in the traffic environment: for example, sudden braking of a lead vehicle and simultaneous eyes-off-road.

Working in collaboration with Volvo, the Australian National University's Robotics Systems Lab at the Research School of Information Sciences and Engineering has developed a unique system for tracking both head and eye movements.  (A presentation of this system can be found on www.volvo.com under The Volvo group -> Research and Technology -> Technology News.)

The method employed is image processing using template matching feature tracking, shown in figure 1 below.  Stereo cameras are used to record data in real time from a number of templates (measuring points) on the face under study.

Figure 1. Images from the Volvo/ANU head and gaze-tracking system. The two images on the bottom right illustrate the output from the stereo cameras, with the tracking templates shown as boxes and reference points as white dots. The top row shows images taken in a Volvo V70 car and the bottom row shows images from a Volvo FH12 truck.

The Integrated Attention Monitoring System

The novel head and eye-tracking system is integrated with vehicle performance data from the multiplex buses found on modern vehicles, e.g. the CAN bus.  These systems are further integrated with a lane tracker for measurement of latitudinal control performance. Together, these devices provide the means to do research based on real-time measurement of visual- and driving behavior.

Future research on inattention will make use of the following technical platform:

• a) A Volvo S80 car with RTI navigation system and Nokia 6110 hands-free cellphone, or
  b) A Volvo FH12 heavy truck with cell phone and the Dynafleet fleet management system.

• A VCT2000 logger to collect vehicle performance data from the CAN bus, i.e. speed, pedal position, gear, steering wheel angle, turn indicator use, telematics control use, display control use, etc.

• The Volvo/ANU head- and gaze-tracker to collect glances (number and length), scan paths, time-off-road-scene, over the shoulder head turns, eye-closure, etc.

• The logging edition of the Assistware SafeTrac lane tracker to collect data on lane excursions, percentage time over line, SD lane position, and lane position variance.

• A National Instruments NI-DAQ measurement card with 16 experimenter annotation switches to collect experimenter input on tasks, events and for synchronization.

• The VAT2000 logging tool.  All data from the CAN bus, the head and gaze-tracker, the lane tracker, and the NI-DAQ measurement card will be synthesized and logged in this software.

• The images from the head and gaze-tracker and the lane tracker will be time coded and mixed together with images from a scene camera inside the car and images from the VAT2000 logging system for offline video analysis, e.g. for validation purposes and for collection of additional contextual information.
  

Preliminary road tests show that the Volvo/ANU system robustly tracks head pose, gaze, and eye closure in real-time, in real vehicle environments. The method is resistant to variations in the lighting intensity (e.g., night driving, passing through a wooded area on a sunny day), vibrations and bumps, distortion and occlusion of features, rotation on the X-, Y- and Z-axis, translation along the Z-axis, and employs 'smart tracking' of features. Tracking reliability has been demonstrated at greater than 95% reliability.  Head pose and gaze can be measured even when the driver is wearing normal glasses, although head pose only can be measured if sunglasses are worn.  The system can successfully differentiate between glance targets such as the speedometer and tachometer.  Natural head and eye movements seem to be able to be measured effectively, but a full characterization of the accuracy and performance of the system is still forthcoming.

A video showing a driver wearing glasses in a Volvo FH12 heavy truck -- driving on a bumpy country road, into the sun, with a shadow cast across the face by the sun visor -- can be viewed on  www.volvo.com under The Volvo group -> Research and Technology -> Technology News. Results of experiments using this system integrated with the multiplex bus and the lane tracker are expected to be published in the near future.

Dr. Victor believes that the technical platform outlined above provides a unique opportunity to study how inattention affects driving behavior.  He writes in a recent technical paper: “This platform will provide real-time data on natural driver visual behavior in real vehicles, will enable us to develop real-time context recognition and prediction (what the driver is- and will be doing), and provides opportunities to assist the driver in attending.  We hope to help guide the development of human-system interfaces by learning about the visual demands of interiors and about visual behavior in general.”  For researchers, there is the possibility that the Volvo/ANU system can potentially automate current methods of measuring visual behavior, accelerating the progress of research.

In future research, Volvo and its partners hope to create and evaluate attention support systems that use algorithms which recognize driving context in real-time.  The research has already spun off the company SeeingMachines, which is marketing a stereo-vision-based gaze-tracker to support driver workload monitoring and drowsy driver detection (see www.seeingmachines.com ... and related IVsource articles in the archives).

[Top]

For More Information ...

... contact Dr. Trent Victor at trent@vtd.volvo.se, or see www.tech.volvo.se/human.html.

[Top]