A Cognitive Constraint Model

We describe an approach to modeling strategic variations in how people might select media content from an
Apple iPod portable music player while driving. An experiment was conducted to determine the time
required to select a target from a list using the scroll wheel on the iPod. It was found that a linear model
accurately predicted the time to scroll through a list to a target. This model was then used in conjunction
with a previously reported steering control model to derive a priori predictions for dual-task performance
over the entire range of possible multitasking strategies. From this set of strategies, we then focused on
identifying the fastest and the safest strategies for completing both a simple selection task and also a more
complex selection task. It was found that the model predictions bracketed the observed human data from a
recent study that investigated the effects of using an iPod while driving. Moreover, the analysis suggests
that in order to compensate for the inherent risks of using devices that demand longer interaction episodes
to complete a task, people might adjust their multitasking strategy by giving more time up to steering
control while completing the secondary task.

INTRODUCTION

People continue at an increasingly alarming rate to perform
distracting tasks while driving — for example, a recent study
of over 5,000 American drivers found that 40% of all drivers
talk on cell phones, 20% of drivers aged 18-24 select songs on
an iPod, and 24% of this younger group send text messages,
all while driving (2006, GMAC Insurance National Drivers
Test). From a human factors standpoint it would be beneficial
if the potential effects of using a mobile device while driving
could be better understood and predicted.

To this end, researchers have started to use cognitive
modeling. One common approach has focused on developing
computational models in a cognitive architecture that interface
with a virtual reality simulation of a driving environment (e.g.,
Salvucci, 2006, 2001; Tsimhoni & Liu, 2003). While these
models have accounted for many performance measures of
human driver behavior under single- and dual-task conditions,
these efforts have generally not attempted to explore possible
strategic variability in behavior.

In contrast, Brumby, Howes, and Salvucci (2007a) have
described a cognitive constraint model (CCM, Howes et al.,
2007) that explores possible strategic variations in performing
a secondary dial task while driving. The CCM approach
focuses on understanding the constraints on the interaction
between the driver and the task environment and allows for
objective functions to represent desired trade-offs in relation to
critical performance variables (e.g., trade-offs between task
time and driver performance). In some respects, the approach
taken by Brumby et al. (2007a) to modeling behavior over the
range of possible strategies is similar to Kieras and Meyer’s
(2000) bracketing heuristic. The bracketing heuristic gives
predictions of the speed of the fastest-possible strategy and the
slowest-reasonable strategy for a complex task — these
brackets are useful because observed performance should
always fall somewhere between the performance of these two
strategies.

The aim of the current paper is to extend Brumby et al.’s
(2007a) previous analysis of the dialing while driving example
to a more complex secondary in-car task; namely selecting
media content from an Apple iPod portable music player while
driving. This is a significant step because an accurate task
model for the iPod is lacking in the field. Moreover, the iPod
is a popular portable device that is frequently used by the
driver of a car, and its use has recently been shown to affect
driver performance (Salvucci et al., 2007). In particular,
Salvucci et al’s study found that selecting media on the iPod
had a significant effect on driver performance as measured by
lateral deviation from the lane center. In addition, participants
in the study were required to make both simple and complex
selections of media content while driving. It was found that
the complex selection task had more of a deleterious effect on
driver performance than the simple selection tasks.

In this paper, we describe an approach to modeling possible
strategic variations in how people select media content from
an iPod while driving. In order to model the iPod selection
task, an experiment was conducted to determine typical timing
estimates for scrolling to and selecting an item in a list. This
model was then used in conjunction with a previous steering
control model (Brumby et al., 2007b) to derive a priori
predictions for dual-task performance, which were compared
to human data.

MODELING THE EFFECTS OF PORTABLE MUSICPLAYER
USE ON DRIVER PERFORMANCE

Modeling Steering Control

Brumby et al. (2007b) describe a cognitive constraint model of
steering control that gives predictions of changes in a vehicle ’s
lateral deviation (i.e., distance from the lane center) over time.
The focus of the model is on how constraints imposed by the
environment (e.g., noise affecting the heading of the vehicle
over time) and constraints imposed by cognition (e.g., people’s
sensitivity to the lateral position of the vehicle in relation to
the center of the lane) interact to determine driver
performance. The model simulates a vehicle moving at a
constant velocity down a straight road. The model performs a
series of discrete steering updates that alter the heading (or
lateral velocity) of the vehicle dependent on its lateral position
in the lane at the time that the steering update is performed. In
particular, given the vehicle’s lateral deviation, a quadratic
function is used to model the mean lateral velocity of the
vehicle following the steering update. In some respects this
approach is similar to control theoretic accounts of lane
keeping (e.g., model 1 in Hildreth et al. 2000), which assume
that adjustments to the heading of a vehicle are motivated by
the goal of minimizing perceptual input quantities that
represent the lateral position and heading of the vehicle.

In order to parameterize the model an analysis of human
steering data was conducted by Brumby et al. (2007b). The
aim of this analysis was to formally characterize how drivers
typically adjust the lateral velocity of a vehicle given its lateral
position in the roadway. Brumby et al. assume that
adjustments to lateral velocity are motivated by the driver
attempting to maintain a central lane position. Brumby et al.
segmented steering data collected from a previous study
(Salvucci, 2001) into a series of steering episodes. A steering
episode was defined as a period of time in which the angle of
the steering wheel did not alter. For each steering episode,
Brumby et al. defined a tuple representing the duration of the
episode, the change in the lateral position of the vehicle, and
the average lateral velocity of the vehicle.

Regression analysis was conducted to estimate a best fitting
curve to predict the average lateral velocity of a steering
episode given the lateral deviation LD of the vehicle at the
start of the episode,

It was found that a quadratic function (Eq. 1) provided a
high degree of correspondence with the human steering data
(r^2 = 0.61).

The quadratic model predicts that as the car drifts farther
from the lane center, drivers tended to react by making sharper
corrective steering movements, which increase the lateral
velocity of the vehicle and, as a consequence, return it to a
central lane position more rapidly. Furthermore, the intercept
of the curve gives some suggestion of the driver’s threshold
for judging the vehicles deviation from the lane center. When
the car is near the lane center (i.e., lateral deviation < 0.30 m),
predicted lateral velocity is close to zero . This means that the
position of the car in the roadway remains more or less
constant over time. This implies that the driver was possibly
satisfied with the vehicle’s position in the roadway if the
lateral deviation of the vehicle was less than 0.3 m from the
lane center. Moreover, the model provides a computationally
efficient formalism for predicting how drivers typically adjust
the heading (or lateral velocity) of a vehicle given its lateral
position in the roadway.

Although the quadratic model gave a high degree of
correspondence with the human steering data, there was
considerable variability with respect to the observed lateral
velocities for a given lateral deviation at the start of an episode
(S.D. = 0.10 m/s). This suggests that people’s adjustments to
the heading of the vehicle were stochastic. In order to develop
a stochastic model, random values were sampled from a
Gaussian distribution and added to the value of the updated
lateral velocity. Based on an estimate of the average standard
deviation observed in the human data, the Gaussian
distribution had a mean of 0.00 m/s and standard deviation of
0.10 m/s.

The steering control model assumes that in between steering
updates the vehicle continues along its heading, but that it’s
heading is permuted by noise. This noise represents the fact
that, if left unattended by the driver, the heading of a car will
be influenced by external factors in the environment (e.g.,
bumps in the road, wind, the camber of the road, etc).
Consistent with Hildreth et al.’s (2000) model, a value from a
Gaussian noise was added at a rate of every 50 ms to the
lateral velocity of the vehicle determined at the previous
update. The Gaussian noise distribution had a mean 0.00 m/s
and standard deviation 0.10 m/s.

Finally, it is worth pointing out that we do not make any
strong theoretical commitment to the duration of a typical
steering update; the model is solely dependent on parameters
derived from the analysis of steering performance data. For
the purposes of the current analysis we assume that in normal
driving conditions steering updates occur at an frequency
interval of once every 500 ms. At this baseline interval
between steering updates, lateral deviation predictions given
by the model (M = 0.33 m, S.D. = 0.02 m) are comparable
with reported baseline lateral deviation (M = 0.35 m, S.D. =
0.08 m) in Salvucci’s (2001) experiment

Modeling the iPod Selection Task

In order to model the iPod selection task, an experiment was
conducted to determine the time required to scroll to and
select a given item i in an ordered list using the device
interface. The study used an Apple iPod (5th-generation video
iPod) that was modified to run RockBox, an open source
firmware replacement. The RockBox software allowed
interaction protocols to be recorded (i.e., button-presses and
left and right scrolls). In the study, 10 participants were
required to search for a target number within a sorted list (e.g.,
scroll to and select “15” within the list of numbers 1-200). The
position of the target item within the list was varied across
trials.

Figure 1 shows the average time for participants to scroll to
and select a target located at position n in the list. Participants
were timed from when they began scrolling (i.e. they moved
from line 0 to line 1) until they clicked on the target line.
There is a clear linear trend to the data in Figure 1, such that
the time required to select a target in the list is proportional to
the number items that need to be scrolled through in order to
reach the target. Regression analysis was conducted to
estimate a best fitting curve. It was found that a linear function
(Eq. 2) gave a high degree of correspondence with the human
data (r^2 = 0.996), t (35) = 88.81, p < .001.

Figure 1: Data plot showing the average time for participants to
scroll to and select a target at position n in an ordered list on the
Apple iPod.

The linear scroll model was used to derive timing
predictions for a set of tasks used in a recent study that
investigated the effect of iPod interaction on driver
performance (Salvucci et al., 2007). In Salvucci et al’s study,
participants were required to select various media content
(e.g., songs, podcasts, and videos) while driving a simulated
car down the middle of three-lane highway environment. All
media content in the study were arranged in their default menu
structure on the Apple iPod. The linear model was used to
make detailed a priori predictions from a set of task
procedures used in the study of the time required to complete
each task. These task procedures and the resulting task time
predictions are detailed in Table 1. The benefit of using the
model is that it allows for detailed predictions of the amount
of time taken at each step in the overall task procedure.

Table 1: Procedure and predicted duration for selecting a song,
podcast, and video for Salvucci et al. (2007) study.

Modeling Dual-Task Performance

We next use the models of steering control and iPod selection
to derive performance predictions for dual-task performance.
We focus on accounting for human data reported in a recent
study (Salvucci et al., 2007) that investigated the effect of iPod
use while driving on task time and lateral deviation.

We assume that one of the consequences for steering control
of engaging in a secondary task while driving is that the interval
between consecutive steering updates generally increases.
Recall that we assumed that in normal driving conditions
drivers typically adjust the heading of the vehicle once every
500 ms. We assume that engaging in a secondary task while
driving generally disrupts this pattern of checking and adjusting
the heading of the vehicle. In particular, we assume that steering
updates cannot occur while the driver’s attention is directed
towards a secondary in-car task, such as when they are engaged
in scrolling through a list to locate a target. This assumption is
based on the idea that peripheral resources, such as the eyes,
will limit the degree of parallel processing between tasks.
Moreover there are numerous demonstrations in the literature of
central interference affecting driver performance in dual-task
conditions (e.g., Brumby, Salvucci, & Howes, submitted; Levy,
Pashler, & Boer, 2006).

Furthermore, we assume that switching between tasks
carries a cost overhead (or switch cost), which reflects the time
required to move visual attention between the outside of the
car (i.e., to focus on the road) and the inside of the car (i.e., to
focus on the iPod). Instead of developing a detailed model of
the perceptual/motor processes involved, we use a simple
timing estimate of 185 ms to move visual attention between
the iPod and the road, or vice versa. Thus, the benefit to
driving performance of frequently interleaving tasks must be
traded-off against the cost of switching between tasks.

MODELING EXPERIMENT

Given that cognitive resources are limited, the question
addressed here is how tasks should be interleaved over time in
order to guarantee a degree of safety is maintained in the
primary driving task (i.e., so that lateral deviation does not
become too egregious), but also that the secondary iPod task is
completed in a reasonably timely manner (i.e., so as to reduce
the additional time costs that are incurred for too frequently
interleaving tasks). Following Brumby et al.’s (2007a) analysis
of dialing while driving, we evaluate the relative speed and
safety of each of the possible task interleaving strategies. For
simplicity in modeling the iPod task, we assume that each
scroll movement is completed in chunks, such that attention
can only be returned to driving in between steps in the task
procedure (see Table 1 for details). This means that for the
simple (podcast and video) selection tasks there are at least 22
= 4 possible task interleaving strategies and for the complex
(song) selection task there are at least 2^3 = 8 possible task
interleaving strategies.

Furthermore, we also explore the consequences of dedicating
more or less time to steering control before returning attention
to the secondary task. It should be clear that the steering control
model provides an intuitive argument for the value of giving up
more time to steering in certain circumstances by conducting a
series of multiple steering updates in succession. For instance, if
the car is far from the lane center, then an initial steering update
will likely increase the lateral velocity of the vehicle, placing it
in a sharp corrective heading in order to rapidly bring it back to
the lane center. However, left unchecked the vehicle will
continue along this sharp heading, possibly past the lane
center and beyond. Further steering updates are therefore
required in order to gradually stabilize the heading of the
vehicle as it nears the lane center.

Figure 2: Data plot of task time and average lateral deviation
predicted for each of the modeled strategies and human data for
the simple (podcast and video) selection tasks. All error bars
represent 95% confidence intervals of mean. Note: P = human
data for the podcast selection task & V = human data for the
video selection task.

Thus, for each steering episode in a given strategy we varied
the number of steering updates that were conducted before
attention was returned to the secondary task from between one
(i.e., making the total steering episode 0.5 s) and 30 steering
updates (i.e., making the total steering episode 15 s). This upper
limit was chosen because we found that conducting any more
steering updates in sequence gave asymptotic performance
across all of the iPod selection tasks. Each strategy was run for
50 trials and performance averaged.

RESULTS

Figure 2 shows the task time and average lateral deviation
predicted for each of the modeled strategies and human data
for the simple (podcast and video) selection tasks. Figure 3
shows the same data for the complex (song) selection task.
The speed/accuracy trade-off that exists between completing
the secondary task quickly and driving safely is apparent in
each figure: The upper-left portion of each plot represents
faster but less safe performance, whereas the bottom-right
portion represents slower but safer performance. It is clear
from both of the figures that the human data fall within the
speed/accuracy trade-off space predicted by the model. We
next focus on identifying the fastest and the safest strategies
for completing each of the tasks.

The fastest strategy for completing each of the tasks is
simple defined as the strategy that completes each of the
secondary tasks in a single contiguous block without once
returning attention to the primary task of driving. The
predicted task time for fastest strategy is given by the sum of
the task procedures; that is, 10.4 s for the complex song
selection task, and 4.7 s for both of the more simple (podcast
and video) selection tasks. It is clear from the figures that
while adopting this strategy would complete the secondary
task in the least amount of time, not taking the additional time
to make a quick glance back to the road and adjust the heading
of the vehicle would likely have catastrophic consequences for
the primary driving task. Moreover, comparing performance
between the simple and complex tasks (Fig 3 vs. Fig 4)
suggests that lateral deviation predictions for this fastest
strategy are highly dependent on the total time of the
secondary task; lateral deviation increases with longer, more
demanding secondary tasks.

Figure 3: Data plot of task time and average lateral deviation
predicted for each of the modeled strategies and human data for
the complex (song) selection task. All error bars represent 95%
confidence intervals of mean.

Defining the safest strategy for completing each of the
secondary tasks is more problematic because while giving up
more time to steering control generally reduces the lateral
deviation of the vehicle, this improvement gradually
asymptotes. Brumby et al. (2007a) define the safest strategy as
at the point where lateral deviation reaches asymptotic dualtask
performance. In particular, we identify a subset of the
strategies in the strategy space that do not significantly differ
in terms of lateral deviation from the strategy that gives up the
most amount of time to driving; that is the slowest strategy
considered (see the right most data point in Figure 2), which
we might intuitively presume to be the absolute safest strategy.
A series of t-tests were conducted to reject strategies that gave
lateral deviation predictions that were significantly greater
than the lateral deviation of this slowest strategy. From this
subset of strategies, which are basically statistically equivalent
in terms of lateral deviation, the safest strategy is the strategy
that also completes the secondary task in the least amount of
time (shown in Figures 2 and 3).

Comparing the model predictions to the human data, it is
clear that the human data fall within the bounds of the strategy
space predicted by the model. Furthermore, it is interesting to
note that the human data for all of the selection tasks lay
closure to the model predictions of the safest strategy than the
fastest strategy. Clearly this is not unexpected given the likely
risks of adopting the fastest strategy.

However, the analysis also suggests that in order for a
reasonable degree of driver safety to be achieved, particularly
for more complex and time consuming secondary tasks,
people can adjust their strategy by giving substantially more
time up to driving. In particular, if we compare the difference
in task time between the fastest strategy and the safest strategy
for both the simple and complex selection tasks, then we find
that for longer and more complex tasks safety can be achieved,
but that it requires significantly more time to be given up to
steering control in order to complete the task in a safe manner.
Moreover, the analysis offers a potential explanation for why
people behave the way that they do in terms of the trade-off
between task time and safety.

GENERAL DISCUSSION

In this paper we have described a modeling technique for
predicting the effects of using an Apple iPod portable music
player while driving. In order to model the iPod selection task,
an experiment was conducted to determine the time required
to select a target from an order list using the scroll wheel on
the iPod. It was found that a simple linear model accurately
predicted the time to scroll through the list and select a target.
This model was then used in conjunction with a previously
reported steering control model (Brumby et al., 2007b) to
derive a priori predictions for dual-task performance over the
entire range of possible strategies for interleaving the two
tasks. From this set of possible strategies, we focused on
identifying the fastest and the safest strategies for completing
both simple and a more complex selection task. It was found
that the model predictions bracketed the observed human data
from a recent study that investigated the effects of using an
iPod while driving.

There are at least two possible concerns with the analysis
described here. First, in modeling the iPod selection task, we
assumed that each scroll movement was completed in chunks,
such that attention could only be returned to driving in
between steps in the task procedure. The reason for adopting
this assumption was for simplifying the space of strategies that
were evaluated. There is a clear concern that this assumption
may be unrealistic though. In particular, for longer scrolling
movements participants may have tended to pause halfway
through the movement in order to check on the position and
heading of the vehicle. A more elaborate evaluation of the
strategy space could be conducted; however, a more fruitful
approach might be to focus on gathering empirical data in
order to determine the upper bound on the amount of time that
people are prepared to spend on a secondary task in a single
episode before returning to check on the primary driving task.

Second, the linear scroll model for predicting task time on
the Apple iPod was based on an artificially simple search task,
where participants located a known target in an ordered list. It
is not clear to what extent this data applies to more complex
searches with complex real-world content. We might speculate
that the gradient of the scrolling function might be shallower
for more complex searches. Nonetheless the extent to which
the human data from Salvucci et al’s (2007) study fit within
the a priori performance predictions given by the model is a
non-trivial feat.

Given people continue at an increasingly alarming rate to
perform distracting tasks while driving, a clear implication of
the work presented here is that efforts might be directed
towards understanding how to better design mobile devices to
make their use by the driver of a car less dangerous. The
results of the modeling analysis presented here suggest that the
total time that the driver of a car is distracted is less important
than the extent to which they are encouraged to make quick
glances back to the road while actively working on a
secondary in-car task. That is, designing mobile devices that
facilitate short bursts of interaction as opposed to requiring
long stretches of interaction might help to alleviate the effects
of distracted driving.

ACKNOWLEDGMENTS

This research was supported by National Science Foundation
grant #IIS-0426674. We would like to thank Mark Zuber for
implementing the experimental software used in the Apple
iPod selection study.