Google focusing autonomous driving development on mastering city street driving; patents piling up
29 April 2014
Over the past year, Google has shifted the focus of its autonomous vehicle project onto mastering city street driving, according to Chris Urmson, Director, Google Self-Driving Car Project. (Urmson was the technical team leader of the CMU team that won the DARPA 2007 Urban Challenge, an autonomous vehicle race.)
Google’s autonomous cars use video cameras; 4 radar sensors (front, back, left, right); a laser range finder (Velodyne HDL-64E LiDAR) to “see” other traffic; and a GPS as well as a wheel encoder and very detailed road maps to determine the precise location of the vehicle. Google says that its autonomous vehicles have logged nearly 700,000 autonomous miles (1.13 million km) over the four years they have been on the road. Since its last pubic update in 2012, Google said that it has logged thousands of miles on the streets of Mountain View, California, Google’s home.
A mile of city driving is much more complex than a mile of freeway driving, with hundreds of different objects moving according to different rules of the road in a small area. We’ve improved our software so it can detect hundreds of distinct objects simultaneously—pedestrians, buses, a stop sign held up by a crossing guard, or a cyclist making gestures that indicate a possible turn. A self-driving vehicle can pay attention to all of these things in a way that a human physically can’t—and it never gets tired or distracted.
As it turns out, what looks chaotic and random on a city street to the human eye is actually fairly predictable to a computer. As we’ve encountered thousands of different situations, we’ve built software models of what to expect, from the likely (a car stopping at a red light) to the unlikely (blowing through it). We still have lots of problems to solve, including teaching the car to drive more streets in Mountain View before we tackle another town, but thousands of situations on city streets that would have stumped us two years ago can now be navigated autonomously.
—Chris Urmson
The Google team has been awarded numerous patents on its autonomous driving technology, including:
| Select Google autonomous driving technology patents | ||||||
|---|---|---|---|---|---|---|
| Patent Nº | Title and description | |||||
| 8,712,104 | Traffic signal mapping and detection A system and method provides maps identifying the 3D location of traffic lights. The position, location, and orientation of a traffic light may be automatically extrapolated from two or more images. The maps may then be used to assist robotic vehicles or human drivers to identify the location and status of a traffic signal. | |||||
| 8,706,342 8,670,891 8,433,470 8,352,110 8,346,426 8,260,482 |
User interface for displaying internal state of autonomous driving system Autonomous vehicles use various computing systems to transport passengers from one location to another. A control computer sends messages to the various systems of the vehicle in order to maneuver the vehicle safely to the destination. The control computer may display information on an electronic display in order to allow the passenger to understand what actions the vehicle may be taking in the immediate future. Various icons and images may be used to provide this information to the passenger. | |||||
| 8,688,306 | Systems and methods for vehicles with limited destination ability Aspects of the present disclosure relate generally to limiting the use of an autonomous or semi-autonomous vehicle by particular occupants based on permission data. More specifically, permission data may include destinations, routes, and/or other information that is predefined or set by a third party. The vehicle may then access the permission data in order to transport the particular occupant to the predefined destination, for example, without deviation from the predefined route. The vehicle may drop the particular occupant off at the destination and may wait until the passenger is ready to move to another predefined destination. The permission data may be used to limit the ability of the particular occupant to change the route of the vehicle completely or by some maximum deviation value. For example, the vehicle may be able to deviate from the route up to a particular distance from or along the route. | |||||
| 8,634,980 | Driving pattern recognition and safety control Systems and methods are provided for controlling a vehicle. A safe envelope driving pattern is determined to control the vehicle in an autonomous mode. User identification data and sensor data are received from one or more sensors associated with the vehicle. A driver-specific driving pattern is determined based on the received sensor data and the user identification data. Operation of the vehicle is controlled in the autonomous mode based on the identification of the user in the driver’s seat, the safe envelope driving pattern, and the user-specific driving pattern. | |||||
| 8,559,673 | Traffic signal mapping and detection A system and method provides maps identifying the 3D location of traffic lights. The position, location, and orientation of a traffic light may be automatically extrapolated from two or more images. The maps may then be used to assist robotic vehicles or human drivers to identify the location and status of a traffic signal. | |||||
| 8,509,982 | Zone driving A roadgraph may include a graph network of information such as roads, lanes, intersections, and the connections between these features. The roadgraph may also include one or more zones associated with particular rules. The zones may include locations where driving is typically challenging such as merges, construction zones, or other obstacles. In one example, the rules may require an autonomous vehicle to alert a driver that the vehicle is approaching a zone. The vehicle may thus require a driver to take control of steering, acceleration, deceleration, etc. In another example, the zones may be designated by a driver and may be broadcast to other nearby vehicles, for example using a radio link or other network such that other vehicles may be able to observer the same rule at the same location or at least notify the other vehicle’s drivers that another driver felt the location was unsafe for autonomous driving. | |||||
| 8,473,144 | Controlling vehicle lateral lane positioning Methods and systems for controlling vehicle lateral lane positioning are described. A computing device may be configured to identify an object in a vicinity of a vehicle on a road. The computing device may be configured to estimate, based on characteristics of the vehicle and respective characteristics of the object, an interval of time during which the vehicle will be laterally adjacent to the object. Based on the characteristics of the vehicle, the computing device may be configured to estimate longitudinal positions of the vehicle on the road during the interval of time. Based on the respective characteristics of the object, the computing device may be configured to determine a lateral distance for the vehicle to maintain between the vehicle and the object during the interval of time at the longitudinal positions of the vehicle, and provide instructions to control the vehicle based on the lateral distance. | |||||
| 8,321,067 8,078,349 |
Transitioning a mixed-mode vehicle to autonomous mode Disclosed are methods and devices for transitioning a mixed-mode autonomous vehicle from a human driven mode to an autonomously driven mode. Transitioning may include stopping a vehicle on a predefined landing strip and detecting a reference indicator. Based on the reference indicator, the vehicle may be able to know its exact position. Additionally, the vehicle may use the reference indictor to obtain an autonomous vehicle instruction via a URL. After the vehicle knows its precise location and has an autonomous vehicle instruction, it can operate in autonomous mode. | |||||
| 8,195,394 | Object detection and classification for autonomous vehicles. Aspects of the disclosure relate generally to safe and effective use of autonomous vehicles. More specifically, objects detected in a vehicle's surroundings may be detected by the vehicle's various sensors and identified based on their relative location in a roadgraph. The roadgraph may include a graph network of information such as roads, lanes, intersections, and the connections between these features. The roadgraph may also include the boundaries of areas, including for example, crosswalks or bicycle lanes. In one example, an object detected in a location corresponding to a crosswalk area of the roadgraph may be identified as a person. In another example, an object detected in a location corresponding to a bicycle area of the roadgraph and identified as a bicycle. By identifying the type of object in this way, an autonomous vehicle may be better prepared to react to or simply avoid the object. | |||||
Google also has a number of patent applications filed, with two published last month (20140081573 and 20140081507) on detecting road weather conditions.
Resources
Urmson, C.; Whittaker, W. (2008) “Self-Driving Cars and the Urban Challenge,” Intelligent Systems, IEEE , vol.23, no.2, pp.66,68, doi: 10.1109/MIS.2008.34
Patents piling up...we certainly don't need that. I though Google's motto was "don't be evil".
Posted by: SJC | 29 April 2014 at 06:40 AM
Patent legal wars are part of the American culture?
Even Google cannot fight it and has to play the game.
Posted by: HarveyD | 29 April 2014 at 07:01 AM
If they wanted to be good and not evil, they would make all their developments public domain for the benefit of human kind. They make plenty of money now as it is, they don't need patent monopoly licenses for billions more dollars while restricting advancement.
Posted by: SJC | 29 April 2014 at 07:24 AM
Good idea SJC but we (the new Democracies) are moving the other way!.
Posted by: HarveyD | 29 April 2014 at 07:50 AM
Part of corporate valuation is an item called Goodwill. Google would increase their Goodwill while helping advancement by releasing the developments to benefit all.
Posted by: SJC | 29 April 2014 at 08:18 AM
I wouldn't worry about google doing this, I would worry about Patent trolls trying to shake down google as soon as they start selling the systems.
At least google plan to build these systems, as do other automotive companies - google will need patents as "swaps" to do deals with these people. The problem will be with trolls who have no intention of building anything, popping up and suing google etc. At least google is big enough to take them on - perhaps all the automakers will get together and pool their IP to make it happen, perhaps not.
I see huge patent wars over this technology in the next 1-20 years.
Posted by: mahonj | 29 April 2014 at 08:31 AM
Ditto mahonj view.
Harvey et al, you may not like Intellectual Property protection through patents, and it definitely has its problems, but I haven't seen a more effective way yet. It's odd to me that people benefiting from the development of technology (and in turn contributing to the economic rewards enjoyed by the innovators) decry patents so much. We post our outrage about the evil of patents from our iPhone/Android device (dripping with patents) as we climb into our Model S (bulletproofed with patents) on the way to get our CT Scan on the newest GE machine (jealously guarded with layers of patents) before we leave on holiday on a B787 (carried aloft with patents). All these items (1) dramatically improve our lives and, (2) yes, enrich their creators by restricting non-licensed use by others for a time. Don't like (2)? Then be prepared to significantly impede (1).
Google makes plenty of money? Not enough for shareholders to give away hundreds of $M (I hold GOOG, and likely you do, too, directly or otherwise, if you live in the US). I don't want them to spend R&D on charity.
BTW, Goodwill is an accounting term used to refer to the cost to acquire a business above the fair market asset value. Use the Google acquisition of Boston Dynamics as an example. The actual terms were not published (BD was a privately-held firm), but let's say fair market asset value was $150M, and Google paid $300M. Then $300M-$150M = $150M is added to Google's Goodwill entry under Assets on the Balance Sheet (presently at $11.5Billion, FWIW). If a company goes out to the world with an open book of technology For the Children, it has no financial value. Doesn't mean companies don't do this (Bell Lab's transistor), but it gets them no financial remuneration.
Posted by: Herman | 29 April 2014 at 09:03 AM
Full disclosure: younger of two daughters will sit for her USPTO registration exam soon to become a Patent Agent.
Posted by: Herman | 29 April 2014 at 09:19 AM
This sounds like the war between Thomas Edison and rivals in the development of the motion picture camera. The rivals ended up moving to California to escape Edison's patent enforcers (goons) and develop full-length motion pictures, which Edison limited to 20 minutes, supposedly because the eyes of the audiences would strain. They called them flicks in those days. Guess what? By 1927 movies ran for almost three hours at the current frames per second, on better film stock than could be afforded by talkies for four years.
No, do not thank Edison for modern filmaking. We have yet to thank Google for monopolizing autonomous driving. In the end, competitors will flee the US, and some international standards committee will get even. Others will pirate the technology. This already happened in TV withPAL, SECAM, and NTSB.
This debate over God Bless Patents and the Trolls is like putting the autonomous cart before the cyborg horse.
Posted by: kalendjay | 29 April 2014 at 03:52 PM
As in many other fields, exaggeration and hungry lawyers have made patent over-protection one of the most effective tool to delay progress and general well being.
Those useless legal wars could be stopped if large countries like China and India (and others) set a maximum contribution of 0.5% to 1% of the factory price to patent holders for a maximum of 5 to 10 years.
'No Fault' Vehicle Insurance put an end to most legal battles and delays.
Posted by: HarveyD | 30 April 2014 at 08:21 AM
Something of a strawman, kalendjay. What "war" is happening now in driverless technology, and how does it come the MPPC saga represent its future? Interesting that you do not note that such a trust couldn't possibly exist today against a huge pile of anti-trust and restraint of trade provisions fully established in the CFR and case law. Cool story, though.
I have a pretty good idea where you fall in the internet meme of Tesla vs. Edison...
I'm most struck by your interesting turn of the phrase "thanking" Google (or Edison). I think this goes to the heart of this odd modern populist view of the evil of IP protection. I never thanked Steve Jobs for numerous Apple technologies. I just bought the products. And that's what we do: the market "thanks" the inventor through acquisition of protected products or services at a price that it accepts. That's all. And as Harvey rightly points out, it is a system finding greater acceptance in emerging economies worldwide in similar structures.
So I will only be "thanking" Google as a happy shareholder, if this investment works out. Otherwise, my purchase of a Google-based technology (or not) will be a purely economically-driven decision. I don't care if Eric and Sergey make a buck or not.
Curious that the main real argument you place against patent protection is that people will behave illegally and steal it. Kind of specious, don't you think? Let's say tomorrow I Xerox(TM) TSLA's ESS physical design and software, and implement it in my Model H. I'll bet a significant fraction of this year's billings that Musk will fight vociferously to stop me. Here's my question to you: should he do What's Best for the World and let me build my Model H, greatly devaluing the investment that his company made 2008-12? Or should he fight for the interests of his shareholders, employees and partners? More importantly, if they thought this theft could happen with no recourse, would these people have risked money and careers in the first place?
Posted by: Herman | 30 April 2014 at 08:22 AM
Curious that Herman attributes remarks to me that I never made, and a spectrographic identification of some meme I never made. I was merely quoting history. There is a dark side to trade and patent development and enforcement. I'm all for patents but it really is a rough world out there. Is antitrust policy as wonderful as you think? Do you think most trials, manned by scientifically uninformed juries and sleeping judges are that much better than a crapshoot? Would you trust the Chinese legal system for that matter?
As for Eric, Sergey and other tech moguls making a buck, they do - mostly on high equity to earning ratios. Bubbles do burst. Patents are ultimately about securing earnings. You might check out some business publications on that subject.
Posted by: kalendjay | 05 May 2014 at 02:07 PM