Human-Centered Autonomous Vehicle Systems:Principles of E�ective Shared Autonomy

Lex FridmanMassachuse�s Institute of Technology (MIT)

fridman@mit.edu

Principle 1

Shared Autonomy:Beyond Levels of Automation

Principle 2

Learn from Data:Machine Learning at Every Level

Principle 3

Human Sensing:Multi-Modal Understanding of the Human

Principle 4

Shared Perception-Control:A Second Pair of Eyes and Hands

Principle 5

Deep Personalization:Human Inside the Machine

Principle 6

Imperfect by Design:Flaws Are Features

Principle 7

System-Level Experience:Greater Than the Sum of Its Parts

Figure 1: Principles of shared autonomy used for the design and development of the Human-Centered Autonomous Vehicle.

ABSTRACTBuilding e�ective, enjoyable, and safe autonomous vehicles isa lot harder than has historically been considered. �e reasonis that, simply put, an autonomous vehicle must interact withhuman beings. �is interaction is not a robotics problemnor a machine learning problem nor a psychology problemnor an economics problem nor a policy problem. It is all ofthese problems put into one. It challenges our assumptionsabout the limitations of human beings at their worst andthe capabilities of arti�cial intelligence systems at their best.�is work proposes a set of principles for designing andbuilding autonomous vehicles in a human-centered way thatdoes not run away from the complexity of human naturebut instead embraces it. We describe our development ofthe Human-Centered Autonomous Vehicle (HCAV) as anillustrative case study of implementing these principles inpractice.

KEYWORDSAutonomous vehicles, shared autonomy, computer vision,machine learning, human-centered arti�cial intelligence.

INTRODUCTION�ree ideas underlie the current popularly held view of au-tonomous vehicles:

(1) �e driving task is easy [5, 11, 18].(2) Humans are bad at driving [8, 27].(3) Humans and automation don’t mix well [22, 24].

In contrast to this view, our work considers (1) that driv-ing is in fact very di�cult, (2) that humans are in fact greatdrivers, and (3) that ge�ing humans and arti�cial intelli-gence systems to collaborate e�ectively is an achievable andworthy goal. In this light, we propose a human-centeredparadigm for engineering shared autonomy systems in thecar that erase the boundary between human and machinein the way the driving task is experienced. Speci�cally, wearticulate seven principles of shared autonomy and discusshow we have applied these principles in practice during thedesign, development, and testing of the Human-CenteredAutonomous Vehicle (HCAV). �e statement and compara-tive details of each principle are provided in the followingsections. �e goal of each principle is summarized here:

arX

iv:1

810.

0183

5v1

[cs

.AI]

3 O

ct 2

018

(1) Shared Autonomy:Beyond Levels of AutomationGoal: Motivate drawing a clear distinction sharedautonomy and full autonomy.

(2) Learn from Data:Machine Learning at Every LevelGoal: Motivate the formulation of many so�ware-based tasks as supervised machine learning prob-lems thereby making them amenable to continuousimprovement from data.

(3) Human Sensing:Multi-Modal Understand of the HumanGoal: Motivate the need for understanding the stateof the driver both on a moment-by-moment basis andacross hours, day, months, and years from multiplesensors streams.

(4) Shared Perception-Control:A Second Pair of Eyes and HandsGoal: Motivate an approach for external perception,vehicle control, and navigation planning that seeksto inform and integrate the human driver into thedriving experience even under highly-automated op-eration.

(5) Deep Personalization:Human Inside the MachineGoal: Motivate adjusting the operation of the AIsystem to the individual driver to a degree of person-alization where the resulting system more representsthe behavior of the speci�c human driver than thegeneric background model of the vehicle as origi-nally manufactured.

(6) Imperfect by Design:Flaws Are FeaturesGoal: Motivate rede�ning the goal for an autonomousvehicle as e�ective communication of �aws and lim-itations instead of �awless fully-autonomous opera-tion.

(7) System-Level Experience:Greater Than the Sum of Its PartsGoal: Motivate removing the focus on e�ectivenessof individual so�ware components and instead focus-ing on the integrated shared autonomy experience.

We implement the seven principles described in this workin a prototype vehicle shown in Fig. 1. �e vehicle uses onlycameras and primarily machine learning approaches to per-form the driving scene perception, motion planning, driversensing, speech recognition, speech synthesis and managingthe seamless two-way transfer of control via voice commandsand torque applied to steering wheel. Video demonstrationsof the vehicle and the concepts described in this paper areavailable online at h�ps://hcai.mit.edu/hcav.

1 SHARED AUTONOMYPrinciple: Keep the human driver in the loop. �ehuman-machine team must jointly maintain su�cient sit-uation awareness to maintain control of the vehicle. Solvethe human-robot interaction problem perfectly and theperception-control problem imperfectly.

�e introduction of ever-increasing automation in vehi-cles over the previous decade has forced policymakers andsafety researchers to preemptively taxonomize automation inhopes of providing structure to laws, standards, engineeringdesigns, and the exchange of ideas. �e six levels of automa-tion (L0 to L5) is the result [25]. �ese de�nitions have moreambiguously stated gray areas than clear, illuminating dis-tinctions, and thus serve as no more than reasonable openersfor public discussion rather than a set of guidelines for thedesign and engineering of automotive systems. We proposethat shared autonomy and full autonomy are the only set oflevels that provide instructive guidelines, constraints, andgoals for success. Moreover, they each provide a distinct setof challenge in both kind and degree. �ese are illustratedin Table 1.

Traditional Approach:�e traditional approach to highly automated vehicles isto skip consideration of the human all-together and focuson perfecting the mapping, perception, planning and otherproblems characterized by the “exceptional” performance re-quirement under the full autonomy column of Table 1. Prac-tically, considering current state-of-the-art hardware andalgorithmic capabilities, this approach puts a lot of emphasison accurate high-de�nition mapping, robust sensor suites,and conservative driving policies.

Human-Centered Autonomous Vehicle Approach:As Table 1 shows, the focus for HCAV is on the driver, fromdriver sensing (see §3) to shared perception-control (see §4)to communication and personalization (see §5). Responsi-bility for the control of the vehicle remains with the driver,but depending on the driver state, driver style, and priorjoint-experience of the human and machine, much of thesteering, acceleration, and deceleration of the vehicle maybe taken care of by the AI system. Tesla Autopilot, a currentLevel 2 system, is used on average over for over 30% of milesdriven [10]. Successful implementation of shared autonomymay see over 50% miles driven under machine control. In ourimplementation of HCAV, the vehicle is always able to main-tain take control with varying degrees of con�dence, and thedriver is always made aware of both the level of con�denceand the estimated risk from the perception system.

2

Performance LevelRequired

SharedAutonomy

FullAutonomy

Sensor Robustness [2] Good ExceptionalMapping [23] Good ExceptionalLocalization [17] Good ExceptionalScene Perception [7] Good ExceptionalMotion Control [4] Good ExceptionalBehavioral Planning [21] Good ExceptionalSafe Harbor Good ExceptionalExternal HMI [14] Good ExceptionalTeleoperation* [9] Good ExceptionalVehicle-to-Vehicle* [16] Good ExceptionalVehicle-to-Infrastructure* [19] Good ExceptionalDriver Sensing [13] Exceptional GoodDriver Communication Exceptional GoodDriver Collaboration Exceptional GoodPersonalization Exceptional Good

Table 1: Technology involved in (1) shared autonomyand (2) full autonomy approaches, including the re-quired performance level of each techology for wide-spread deployement. General terms of “good” and “ex-ceptional” are used to highlight the distinction of notsolving the 1% edge cases in the former case and hav-ing to solve them in the latter case. *Note: Teleopera-tion, V2V, and V2I are not required technologies butif utilized would need to achieve the speci�ed perfor-mance level.

2 LEARN FROM DATAPrinciple: Every vehicle technology (see Table 1) shouldbe data-driven. Each should collect edge-case data andcontinually improve from that data. �e overall learningprocess should seek a scale of data that enables progressaway from modular supervised learning formulations to-ward end-to-end semi-supervised and unsupervised learn-ing formulations.

Traditional Approach:Traditional approach to vehicle autonomy at any level rarelyinvolves signi�cant machine learning except in a specializedo�ine context of lane detection in Intel’s Mobileye vision-based systems or infrared-camera based head pose estima-tion in the GM’s Super Cruise system. Tesla Autopilot hastaken a further step in the so�ware built on top of the sec-ond version of its hardware toward converting more andmore of the perception problem into a supervised machinelearning problem. Nevertheless, much of the control of thevehicle and the estimation of driver state (in the rare cases

it is considered) is engineered without utilizing large-scaledata-driven methods and almost never updated in an onlinelearning process. In the case of fully autonomous vehicleundergoing testing today, machine learning is primarily usedfor the scene understanding problem but not for any otheraspect of the stack. Moreover, the amount of data collectedby these vehicles pales in scale and variability to that able tobe collected by Level 2 vehicles.

Human-Centered Autonomous Vehicle Approach:�e data available in Level 2 vehicles for utilization withina machine learning framework is su�ciently expansive inscale and scope and growing to capture varying, representa-tive, and challenging edge cases. Shared autonomy requiresthat both driver facing and driving scene facing sensory datais collected, mined, and used for supervised learning anno-tation. In our implementation of HCAV, the driving sceneperception, motion planning, driver sensing, speech recogni-tion, and speech synthesis are all neural network models thatare regularly �ne-tuned based on recently collected driverexperience data. In doing data collection, we do not focus onindividual sensor streams but instead consider the drivingexperience as a whole and collect all sensor streams together,synchronized via a real-time clock, for multi-modal anno-tation. �at is any annotation of the driving scene can bedirectly linked to any of the annotation of the driver state.Performing annotation on synchronized sensor streams al-lows for easy transition from modular supervised learningto end-to-end learning when the scale of data allows for it.

3 HUMAN SENSINGPrinciple: Detect driver glance region, cognitive load,activity, hand and body position. Approach the driverstate perception problem with equal or greater rigor andscale to the external perception problem.

Driver sensing refers to multi-modal estimation of overallphysical and functional characteristics of the driver includ-ing level of distraction, fatigue, a�entional allocation andcapacity, cognitive load, emotional state, and activity. Typ-ical driver gaze estimation[20] involves extraction of headand eye pose and estimation of gaze or neural network basedapproaches that instead solve the gaze region classi�cationproblem [11]. Driver cognitive load estimation [15] involvesdetection of working memory load based on eye movement.Driver fatigue and drowsiness estimation [28] aims to detectarousal from blink rates, eye movement, and body move-ment. In the driver state detection context, this is the mostextensively studied computer vision area. Driver emotion[1, 29] uses facial landmark con�guration and facial motionanalysis. Physiological and audio sensors are o�en utilized

3

(a) Glance region classi�cation.

(b) In-cab object detection and activity recognition.

(c) Driveable area and lane detection.

(d) Driving scene entity detection.

Figure 2: Perception tasks in our implemetation of HCAV.Visualization of the perception tasks integated to determinerisk is shown in Fig. 4.

in detection of a�ect. Driver activity recognition [6, 26] usesgaze pa�erns and movements of head, arms, hands, and �n-gers. �is includes detailed gesture recognition and broadactivity type (i.e, smartphone texting) recognition.

Traditional Approach:Driver sensing hardware and so�ware capabilities are miss-ing in almost all manual, semi-autonomous, and autonomousvehicles being tested today. Exceptions include the GM Su-per Cruise system that has a camera on the steering wheelfor head tracking and Tesla Model 3 which has an in-cabcamera that, to the best of our knowledge, is not currentlyutilized for driver state estimation. Besides vision-basedmethods, crude low-resolution methods of driver sensingapproaches include tracking steering reversals as a proxy fordriver drowsiness.

Human-Centered Autonomous Vehicle Approach:Sensing the state of the driver is the �rst and most impactfulstep for building e�ective shared autonomy systems. Auto-mated methods for extracting actionable knowledge frommonocular video of a driver have been actively studied forover two decades in computer vision, signal processing, ro-botics, and human factors communities. �e overarchinggoal for these methods is to help keep the driver safe. Morespeci�cally, detection of driver state facilitates the more ef-fective study of how to improve (1) vehicle interfaces and(2) the design of future Advanced Driver Assistance Systems(ADAS). With increasingly intelligent vehicle interfaces andthe growing role of automation technology in the vehicle, thetask of accurate real-time detection of all aspects of driverbehavior becomes critically important for a safe personalizeddriving experience. Of special interest is the transition acrossdi�erent semi-autonomous driving modes ranging from fullymanual control to fully autonomous driving. �e hando� ineither direction of transition requires that the vehicle hasaccurate information about the driver state. In our imple-mentation of HCAV, we estimate driver glance, cognitiveload, and activity at 30 Hz.

4 SHARED PERCEPTION-CONTROLPrinciple: Perform scene perception and understandingwith the goal of informing the driver of the system capa-bilities and limitations, not with the goal of perfect blackbox safe navigation of the vehicle.

Traditional Approach:�e goal of a fully autonomous vehicle is to perfectly solvethe perception-control task, considering the human driveran unreliable and unpredictable perturbation to the controlproblem. Removing the human being from the formulation

4

1. Monocular Camera

2. Secondary System for Perception-Control

3. Disagreement Notification

4. Steering Commands from Primary System and Secondary System

5. Temporal Difference Input to Secondary System

1

4

5

2

3

Figure 3: Implementation and evaluation of the arguingmachines framework implemented in a Tesla Autopilot and ourHCAVprototype vehicle. �e technical details of the framework are detailed in [12].

of the problem makes the problem appear be�er de�ned andthus seemingly more amenable to the type of approach thatproved successful in the DARPA Urban Challenge over 10years ago [5].

Human-Centered Autonomous Vehicle Approach:Instead of decoupling the human driver from the loop ofperception and movement planning, the human-centeredapproach by de�nition seeks to integrate the human being.�e goal of the perception task then becomes to supportthe human driver with information about the external sceneand more importantly about the limitations of the perceptionsystem. Communication of imperfection as discussed in §6 isthe ultimate goal of the perception-control task in the sharedautonomy paradigm.

In our implementation of HCAV, there are several key algo-rithms that are designed around this principle. Examples areshown in Fig. 2. First, we visually communicate the degree ofuncertainty in the neural network prediction, segmentation,or estimation about the state of driving scene. Second, weintegrate all the perception tasks in a decision fusion stepin order to estimate the overall risk in the scene as shownin Fig. 4. �e decision fusion is across both internal and ex-ternal facing sensors. �ird, we are always doing imitationlearning: using the steering of the human driver when he orshe is in control as training data for the end-to-end steering

network. Fourth, we use the end-to-end network as partof an arguing machines framework (detailed in [12]) to pro-vide human supervision over the primary perception-controlsystem as shown in our implementation of it in Fig. 3.

5 DEEP PERSONALIZATIONPrinciple: Every aspect of vehicle operation should bea re�ection of the experiences the speci�c vehicle shareswith the driver during their time together. From the �rstmoment the car is driven, it is no longer like any otherinstance of it in the world.

Traditional Approach:�e most common approach in designing and engineeringautomotive system is to do no personalization at all, exceptminimally within the infotainment system interaction partof the driving experience. One of the ideas underlying suchengineering design is that people want a system to performas expected, and in order to form correct expectations, the be-havior of the system should be consistent within and acrossinstances of the vehicle. In the rare cases that the systemlearns from the driver (i.e., current implementation of TeslaAutopilot), to the best of our knowledge the learning is in-tegrated into the overall knowledge base as part of �eetlearning.

5

(a) Example of elevated risk under manual control during a period of frequent o�-road glances to the smartphone.

(b) Example of elevated risk under machine control in the presence of a pedestrian.

Figure 4: Examples of elevated risk computed by decision fusion of external and in-cab perception systems.

6

Human-Centered Autonomous Vehicle Approach:One of the most important and novel principles underly-ing the HCAV concept is deep personalization, or the instantand continuous departure of system behavior from the back-ground model to one that learns from the experience sharedby one speci�c instance of the system and one speci�c hu-man driver. Part of the learning process is �eet learningwhere the data is used to update the background model (sys-tem behavior deployed to all drivers). However, in terms ofthe overall system experience, the more impactful part isthe individual learning where the �ne-tuned model controlsthe behavior of the system for only the one speci�c driverassociated with that model.

�is approach has profound implications for several as-pects of semi-autonomous driving. First, liability of manycommon system “failures” rests on the human driver, muchlike a trainer of a horse is in part responsible for the behaviorof that horse when the horse is ridden. �is concept is not alegal framework, but it is a way of creating an experience ofshared autonomy even when the vehicle is in control. It cre-ates an operational and emotional closeness that we believeis fundamental to successful human-machine collaborationin a safety-critical context of the driving task.

In our implementation of HCAV, this principle is appliedin two areas: perception-control and communication. Formotion planning of the vehicle, we use imitation learningto adjust the objective function for choosing between theset of generated candidate trajectories. For communication,we adjust the natural language dialogue system the vehicleuses to inform the driver about changes in risk estimates andhigh-level shared autonomy decisions. �e personalizationis both in the operation of the natural language generationand in the degree of personal feel. For example, the vehicleAI calls the driver by their name and adjusts the tone ofvoice based on what was su�cient in the past to grab theira�ention.

6 PRINCIPLE 6: IMPERFECT BY DESIGNPrinciple: Focus on communicating how the system seesthe world, especially its limitations, instead of focusingon removing those limitations.

Traditional Approach:In the automotive context, for many reasons, engineeringdesign is o�en focused on safety. Naturally, this leads togoals formulated around minimizing frequency and mag-nitude of system failures. In other words, for autonomousdriving, perfection is the goal. �e non-obvious side e�ectof such goals is that revealing imperfections and uncertaintyo�en becomes an undesirable design decision. �e thoughtis: “Why would you want to show imperfections when the

system is supposed to be perfect?” �ere are, of course, legaland policy reasons for such design decisions as well that arein large part outside the scope of this discussion.

Human-Centered Autonomous Vehicle Approach:Rich, e�ective communication is the most essential elementof designing and engineering arti�cial intelligence systems inthe shared autonomy paradigm. Within the context of com-munication, system imperfections are the most information-dense content for exchanging and fusing models of the worldbetween human and machine. Hiding system uncertainty,limitations, and errors misses the opportunity to managetrust and form a deep bond of understanding with the driver.In our view, it is one of the greatest design failings of priora�empts at implementing semi-autonomous systems.

In our implementation of HCAV, limitations of the systemare communicated verbally and visually. We visualize theworld and the driver as the system seems them through thevarious algorithms mentioned in previous sections to helpthe driver gain an intuition of system limits. As opposedto providing warnings or ambiguous signals about systemuncertainty, we found that simply showing the world as thecar sees it is the most powerful method of communication.�ere are technical challenges to this type of communica-tion in that the process of visualization can o�en be morecomputationally intensive than the perception-control anddriver sensing algorithms themselves. However, we believethis is a critically important problem to solve and thus de-serves a�ention from the robotics, HRI, and HCI researchcommunities.

7 PRINCIPLE 7: SYSTEM-LEVEL EXPERIENCEPrinciple: Optimize both for safety and enjoyment at thesystem level.

Traditional Approach:As described in §6, one of the primary goal of the engineeringdesign process in the automotive industry is safety. Anothermajor goal is lowering cost. �is second goal tends to leadto modular, component-based design thinking. �e samepa�ern holds, for di�erent reasons, in the design of arti�cialintelligence systems in robotics, computer vision, and ma-chine learning communities. Considering individual compo-nents (i.e., object detection) without considering the overallexperience (i.e., risk-based bi-directional transfer of control)allows to rigorously test the individual algorithms and pushthe state-of-the-art of these algorithms forward. However,this process narrows the focus on individual algorithms andnot the experience of the overall system.

7

Human-Centered Autonomous Vehicle Approach:�e value of systems engineering and systems thinking hasbeen extensively documented in literature over the pastseveral decades [3]. Nevertheless, this kind of thinking israrely applied in the design, testing, and evaluation of au-tonomous vehicles whether in their semi-autonomous orfully-autonomous manifestations. As articulated in the otherprinciples, both humans and AI systems have �aws, and onlywhen the share autonomy paradigm is considered at the sys-tem level do those �aws have a chance to be leveraged tobecome strengths.

8 CONCLUSIONIt is di�cult to predict which path to vehicle autonomy willprove successful both in the near-term and in the long-term.Furthermore, it is not clear what success looks like. Ourhope is that the goals of increased safety, an enjoyable driv-ing experience, and improved mobility can all be achievedwithout having to strike a di�cult balance between them.Moreover, we believe that while the shared autonomy ap-proach is counter to the approach taken by most people inautomotive industry and robotics research community inthe past decade, it nevertheless deserves serious considera-tion. In the end, the choice rests on the question of whethersolving the driving task perfectly is easier or harder thanperfectly managing the trust and a�ention of a human being.We believe this is far from a close case, and this paper andour HCAV prototype vehicle is a serious a�empt to considershared autonomy as a path forward for human-centeredautonomous vehicle system development.

ACKNOWLEDGMENT�e authors would like to thank the team of engineers andresearchers at MIT, Veoneer, and the broader driving andarti�cial intelligence research community for their valuablefeedback and discussions throughout the development of thiswork. Support for this research was provided by Veoneer.�e views and conclusions of authors expressed herein donot necessarily re�ect those of Veoneer.

REFERENCES[1] Irman Abdic, Lex Fridman, Daniel McDu�, Erik Marchi, Bryan Reimer,

and Bjorn Schuller. 2016. Driver frustration detection from audio andvideo in the wild. In KI 2016: Advances in Arti�cial Intelligence: 39thAnnual German Conference on AI, Klagenfurt, Austria, September 26-30,2016, Proceedings, Vol. 9904. Springer, 237.

[2] Jayanta Kumar Baruah, Rabindranath Bera, and Sourav Dhar. 2018.Ranking of Sensors for ADAS: An MCDM-Based Approach. In Ad-vances in Communication, Devices and Networking. Springer, 563–571.

[3] Benjamin S Blanchard, Wolter J Fabrycky, and Walter J Fabrycky. 1990.Systems engineering and analysis. Vol. 4. Prentice Hall EnglewoodCli�s, NJ.

[4] Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, BernhardFirner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Mon-fort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning forself-driving cars. arXiv preprint arXiv:1604.07316 (2016).

[5] Martin Buehler, Karl Iagnemma, and Sanjiv Singh. 2009. �e DARPAurban challenge: autonomous vehicles in city tra�c. Vol. 56. springer.

[6] Shinko Y Cheng, Sangho Park, and Mohan M Trivedi. 2007. Multi-spectral and multi-perspective video arrays for driver body trackingand activity analysis. Computer Vision and Image Understanding 106,2 (2007), 245–257.

[7] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld,Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, andBernt Schiele. 2016. �e cityscapes dataset for semantic urban sceneunderstanding. In Proceedings of the IEEE conference on computer visionand pa�ern recognition. 3213–3223.

[8] Alex Davies. 2015. Oh Look, More Evidence Humans Shouldn’tBe Driving. (May 2015). h�ps://www.wired.com/2015/05/oh-look-evidence-humans-shouldnt-driving/

[9] Terrence Fong, Charles �orpe, and Charles Baur. 2001. Collaborativecontrol: A robot-centric model for vehicle teleoperation. Vol. 1. CarnegieMellon University, �e Robotics Institute Pi�sburgh.

[10] Lex Fridman. 2018. Tesla Vehicle Deliveries and Autopilot MileageStatistics. (2018). h�ps://hcai.mit.edu/tesla-autopilot-miles/

[11] Lex Fridman, Daniel E. Brown, Michael Glazer, William Angell,Spencer Dodd, Benedikt Jenik, Jack Terwilliger, Julia Kindelsberger,Li Ding, Sean Seaman, Hillary Abraham, Alea Mehler, Andrew Sip-perley, Anthony Pe�inato, Linda Angell, Bruce Mehler, and BryanReimer. 2017. MIT Autonomous Vehicle Technology Study: Large-Scale Deep Learning Based Analysis of Driver Behavior and Interac-tion with Automation. CoRR abs/1711.06976 (2017). arXiv:1711.06976h�ps://arxiv.org/abs/1711.06976

[12] Lex Fridman, Li Ding, Benedikt Jenik, and Bryan Reimer. 2019. ArguingMachines: Human Supervision of Black Box AI Systems �at MakeLife-Critical Decisions. CoRR abs/1710.04459 (2019). arXiv:1710.04459h�p://arxiv.org/abs/1710.04459

[13] Lex Fridman, Philipp Langhans, Joonbum Lee, and Bryan Reimer. 2016.Driver gaze region estimation without use of eye movement. IEEEIntelligent Systems 31, 3 (2016), 49–56.

[14] Lex Fridman, Bruce Mehler, Lei Xia, Yangyang Yang, Laura YvonneFacusse, and Bryan Reimer. 2017. To Walk or Not to Walk: Crowd-sourced Assessment of External Vehicle-to-Pedestrian Displays. arXivpreprint arXiv:1707.02698 (2017).

[15] Lex Fridman, Bryan Reimer, Bruce Mehler, and William T Freeman.2018. Cognitive Load Estimation in the Wild. In Proceedings of the2018 CHI Conference on Human Factors in Computing Systems. ACM,652.

[16] John Harding, Gregory Powell, Rebecca Yoon, Joshua Fikentscher,Charlene Doyle, Dana Sade, Mike Lukuc, Jim Simons, and Jing Wang.2014. Vehicle-to-vehicle communications: Readiness of V2V technologyfor application. Technical Report.

[17] Jesse Levinson and Sebastian �run. 2010. Robust vehicle localiza-tion in urban environments using probabilistic maps. In Robotics andAutomation (ICRA), 2010 IEEE International Conference on. IEEE, 4372–4378.

[18] Kieren McCarthy. 2018. So when can you get in the �rst self-drivingcar? GM says 2019. Mobileye says 2021. Waymo says 2018. (May2018). h�ps://www.theregister.co.uk/2018/05/09/�rst autonomousvehicles/

[19] Vicente Milanes, Jorge Villagra, Jorge Godoy, Javier Simo, Joshue PerezRastelli, and Enrique Onieva. 2012. An Intelligent V2I-Based Tra�cManagement System. IEEE Transactions on Intelligent Transportation

8

Systems 13, 1 (2012), 49–58.[20] Erik Murphy-Chutorian and Mohan Manubhai Trivedi. 2009. Head

pose estimation in computer vision: A survey. Pa�ern Analysis andMachine Intelligence, IEEE Transactions on 31, 4 (2009), 607–626.

[21] Brian Paden, Michal Cap, Sze Zheng Yong, Dmitry Yershov, and EmilioFrazzoli. 2016. A survey of motion planning and control techniquesfor self-driving urban vehicles. IEEE Transactions on intelligent vehicles1, 1 (2016), 33–55.

[22] Raja Parasuraman and Victor Riley. 1997. Humans and automation:Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230–253.

[23] Heiko G Seif and Xiaolong Hu. 2016. Autonomous driving in the iCity-HD maps as a key challenge of the automotive industry. Engineering2, 2 (2016), 159–162.

[24] �omas B Sheridan. 2002. Humans and automation: System design andresearch issues. Human Factors and Ergonomics Society.

[25] Bryant W Smith. 2013. SAE levels of driving automation. Centerfor Internet and Society. Stanford Law School. h�p://cyberlaw. stanford.edu/blog/2013/12/sae-levels-drivingautomation (2013).

[26] Pavan Turaga, Rama Chellappa, Venkatramana S Subrahmanian, andOctavian Udrea. 2008. Machine recognition of human activities: Asurvey. Circuits and Systems for Video Technology, IEEE Transactionson 18, 11 (2008), 1473–1488.

[27] Tom Vanderbilt and Brian Brenner. 2009. Tra�c: Why We Drive theWay We Do (and What It Says about Us) , Alfred A. Knopf, New York,2008; 978-0-307-26478-7. (2009).

[28] Qiong Wang, Jingyu Yang, Mingwu Ren, and Yujie Zheng. 2006. Driverfatigue detection: a survey. In Intelligent Control and Automation, 2006.WCICA 2006. �e Sixth World Congress on, Vol. 2. IEEE, 8587–8591.

[29] Zhihong Zeng, Maja Pantic, Glenn Roisman, �omas S Huang, et al.2009. A survey of a�ect recognition methods: Audio, visual, andspontaneous expressions. Pa�ern Analysis and Machine Intelligence,IEEE Transactions on 31, 1 (2009), 39–58.

9