Autonomous Cars Will Require a Totally New Kind of Map
Nokia HERE is making HD maps that combine detail about roadways with information about traffic flow. Nokia HERE
As the vehicle navigated the labyrinthine streets of London and headed for the countryside of Surrey with uncommon speed, the passengers must have felt a bit unnerved. Having selected their destination, they’d relinquished control. They had no communication with the driver, but they could check their progress on a map.
The map must have been reassuring, says Peter Skillman, lead designer for Nokia HERE, the maps division the Finnish communications company. Skillman visited WIRED’s San Francisco office recently to talk about HERE’s efforts to build high definition maps for autonomous vehicles. But the passengers he was talking about weren’t zipping through London in a sleek Audi prototype. They were riding in a stagecoach circa 1720. Skillman had taken a slight detour to show off a map he’d bought recently at an antiquarian map shop in London. Then as now, Skillman said, maps can smooth the transition to a new technology.
‘The key to making autonomous driving work is to not forget about the driver.’
Autonomous cars will require maps that differ in several important ways from the maps we use today for turn-by-turn directions. They need to be hi-def. Meter-resolution maps may be good enough for GPS-based navigation, but autonomous cars will need maps that can tell them where the curb is within a few centimeters. They also need to be live, updated second by second with information about accidents, traffic backups, and lane closures. Finally, and this was the point Skillman was trying to make with the 1720 road atlas, they’ll need to take human psychology into account and win the trust of their passengers. “The key to making autonomous driving work is to not forget about the driver,” Skillman said.
Fully autonomous cars will be ready to hit the road as soon as 2017 (according to Sergey Brin), or perhaps sometime in the 2020s (according to more conservative forecasts), or maybe never (according to naysayers). The timing may be uncertain, but cars are already becoming more autonomous, creeping across a spectrum from current models with adaptive cruise control and assisted parallel parking to future vehicles that can navigate from A to B while you take a nap or make a sandwich. Much of the attention has focused on the sensors and other technology inside the cars and on the legal questions they raise (if an autonomous car causes an accident, who’s to blame? what if the car was hacked?), but there’s another crucial element: maps.
Probe data collected by HERE from vehicles traveling highways around Berlin. Nokia HERE
Like typical digital maps HERE is using satellite and aerial imagery as a starting point for its HD maps. The maps also incorporate anonymized “probe data” from GPS devices inside fleet vehicles owned by trucking companies and other partners. This data, which HERE collects at a rate of 100 billion points per month, contains information about the direction and speed of traffic on roads and highways. But the most detailed information being fed into the maps comes from hundreds of cars outfitted with GPS, cameras, and lidar, a laser-based method for measuring distances.
This fleet is coordinated from a nondescript building two blocks from the campus of the University of California, Berkeley. The sensors on the cars were developed by John Ristevski, a 38-year-old Australian native. Ristevski is HERE’s head of reality capture, a job title reminiscent of the famous story by Jorge Luis Borges about a 1:1 scale map that is exactly as big as the area it covers. The map Ristevski and his colleagues are creating has similar aspirations.
When the car is in motion, the lidar system—a cylinder about the size of a soda can—spins around, shooting out 32 laser beams and analyzing the light that bounces back. It collects 700,000 points per second, Ristevski says. An inertial sensor tracks the pitch, roll, and yaw of the car so that the lidar data can be corrected for the position of the car and used to create a 3-D model of the roads it has traveled. The lidar instrument’s range tops out about 10-15 stories above the street. At street level, its resolution is just a few centimeters.
A HERE car outfitted with GPS, LIDAR, and cameras. Nokia HERE
Lane markers and street signs stand out in the lidar imagery because they’re coated with reflective paint. HERE uses a combination of computer vision algorithms and manual labor to extract this information and check it against imagery from the cars’ cameras (much like Google extracts similar information from its Street View imagery.)
HERE has outfitted roughly 200 cars with the sensor system Ristevski designed, and the company has a similar number of cars with an older generation of equipment. In Berkeley, Ristevski and I took a quick spin with driver Luke Pulaski in a bright blue Volkswagen Jetta wagon with the sensor equipment mounted on a Thule roof rack. A battery pack and custom Linux box with a terabyte hard drive occupied the space where the legs of a front seat passenger would go. Pulaski logged on to the system with a few taps on a tablet mounted just to his right, and icons turned green to indicate that the cameras and other sensors were working. Turn-by-turn directions appeared, calculated to provide the most efficient route to cover every street in the area to be mapped. “For the most part, the driving is actually boring,” Ristevski said. “It’s designed to be.”
All told, HERE has driven 2 million kilometers (1.2 million miles) in 30 countries on 6 continents, all in the last 15 months. Google, HERE’s main competitor in the race to build maps for autonomous cars, has focused its efforts close to home, reportedly mapping 2,000 miles around its headquarters in Mountain View. (The US road network, for comparison, covers 4 million miles). A live map in HERE’s Berkeley office shows which cars are active. The afternoon I visited, a green tags indicated cars actively mapping roads on the west coast and a couple tags indicated that drivers in Australia were off to an early start. The tags in Europe and the east coast were grayed out, done driving for the day.
HD maps will tell an autonomous car what to expect along its route, Ristevski says. “If you just have a bunch of sensors on the car that detect things in real time and no a priori information about what exists, the problem becomes a lot harder,” he said. “The maps are essential.”
Of course, road conditions can change quickly, and another challenge for mapmakers is how to detect things like accidents and lane closures and update their maps in as close to real time as possible. Sensors on future autonomous cars could feed information over cellular data networks to HERE’s map in the cloud, but that might not be fast enough to avoid an accident. According to Peter Skillman, it could take several seconds for a car in San Francisco to beam its data to a data center in, say, North Carolina, and get a response. Getting response times down to tens of milliseconds—fast enough for a car to switch lanes to avoid some debris in the road spotted by another car ahead of it—will require applications that live inside the LTE networks and can be accessed locally, Skillman says.
A self-driving car that swerves to avoid debris may be a marvel of technology, but it’s also a scary car to be in if you don’t know what’s going on. And this gets back to Skillman’s point about maps as mediators between human psychology and a potentially frightening new technology. A recent survey found that 88 percent of Americans were worried about riding in a driverless car. The key to getting people to trust autonomous cars, Skillman says, is having the experience match their expectations. If the car signals ahead of time that it’s about to change lanes to avoid some debris, and then does exactly that, it will start to gain the trust of its passengers, he says.
Skillman pulled up a few examples on his laptop, short clips that showed the kind of map you’d see in the console of a car with an onboard navigation system. In one, an animated arrow popped up on a map to indicate an impending lane change. In another, yellow brackets and an exclamation point highlighted a man walking near the side of the road—thereby alerting passengers to the possibility of a sudden move to avoid him.
Skillman played another animation, a short 3-D clip of a route through the streets of Chicago. Played at the start of a trip, this sort of overview could help put passengers at ease by showing them where they are and where they’re going, he says. It’s the same purpose his 18th century road atlas served. Passengers would be able to see that there’s a right turn coming up and understand why. Cartographers will have to keep inventing these kinds of solutions as they design maps for autonomous cars, Skillman says. “We need to develop a whole new visual language so you know what the car’s intentions are.”
Detail from a popular 1720 road atlas. Peter Skillman
