- Michelle is driving along a busy street. She is alert and aware of her surroundings, however there is a pedestrian beginning to cross the road ahead which Michelle has not yet noticed. The system detects the position of the pedestrian and the fact that Michelle has not looked at them, and provides a warning indication or noise to prompt Michelle to take note of the potential hazard, thereby avoiding a collision.In other words:
- The system monitors the driver’s eye gaze (TOB) and what she is looking at to create meaningful notifications for the driver. In addition to the eye gaze, the system also monitors the driver’s expressions (NV) to identify interest or other emotions generated by observing specific objects/POIs outside of the vehicle.
- Unseen Pedestrians detection: The system leverages a forward facing imager (e.g. camera or camera array) to detect hazards on or near the road, such as a pedestrian about to cross the road. In that case a highlight is shown in the HUD at the position placing it on top of the pedestrian or as a shimmering halo around them. (This requires knowledge of the pedestrian position from the forward facing camera as well as the position of the driver’s eyes relative to the HUD). If the pedestrian/obstacle is outside of the field of view of the driver looking through the HUD, instead of creating a halo on top of the pedestrian, the HUD can visualize the symbol of a pedestrian and an arrow in the direction of the pedestrian (this can also be combined with a distance countdown to the object). If the driver still does not look at the pedestrian a noise or an even stronger warning indication may appear until the driver has looked at the pedestrian. When he has done so, the highlight and sound may disappear or fade. A similar effect is created with wildlife identified in the vicinity of the road which may be a potential hazard.
- A simplification of this feature would be to indicate that there are pedestrians on the right/left side of the road by showing an arrow or directional acoustic signal
- Advertisements: The driver’s gaze detection onto outdoor advertisements is recorded together with the emotional response of the driver to determine if the advert is of potential interest. In case of recorded interest, an audio advertisement is played to help the driver learn more about the product without having to keep looking at the advertisement, enabling them to keep their eyes on the road. This makes use of the outside facing camera and correlates with GPS coordinates of billboards. The driver’s interests in outside advertisements (or objects in general) can also be bookmarked for later.