How to Identify Traffic Signals
Top 25 Patents: How to identify and classify traffic lights and vehicle lights
#In this Issue
In last week’s issue we looked at how companies identify static road signs. In this issue we pose the question:
How are companies identifying and classifying traffic signals and vehicle signals?
Let’s begin.
#Data Wrangling
Today’s data set is extracted from the following patent classification category:
CPC: G06K9/00818: Recognizing traffic signs
This class contains a total of 6,829 documents. Let’s apply some some filters and see what’s in there:
Family Filter: By filtering down to single patent per family reduces or set to 2,379 documents
Main Class Filter: By looking only at those document for which this is the main class we get down to 871 documents
Now we can chart out these results to identify the main players in this area:
Another interesting chart, just like last week, where the patent activity in China exceeds all other countries. Let’s chart out who is filing patents in China to see if it is still mainly our usual suspects:
It looks like a mix of known players and universities in the Chinese filings. We will filter these out for today’s newsletter, but will have to noodle on the amount of patent filings in China, and how to deal with them, in a future issue.
#The Data Set
Let’s get down to our final data set. Let’s apply a couple more filters:
US Only Filter: Let’s just look at patent families in the US. This gets us down to 182 documents
Recency Filter: Let’s only look at patent families published from 2018 - Present (August 2020). This gets us down to 110 documents
Here’s our Top 10 Players from this data set:
Still, we are at 110 documents. That’s too many to cover in one issue. To narrow it down further I’m going to try something new. Rather than selecting a reference company I’m just going to select the patent documents that have the highest “Patent Score” in the top secret patent research tool that I use.
The “Patent Score” is calculated by an algorithm that takes into consideration a combination of factors available in the patent itself and the overall patent database. As it turns out, the highest patent score in today’s document set is:
US20180114077 - Use Of Relationship Between Activities Of Different Traffic Signals In A Network To Improve Traffic Signal State Estimation - Waymo (Patent Score of 94/100)
As an initial set, I selected the top 32 documents by patent score, and ended up eliminating 7 of them as non-relevant, to arrive at a final data set of 25 documents.
Let’s take a look!
#Data Analysis
As it turns out, our data includes traffic signals from both traffic lights and from vehicle lights. That’s just how the patent office categorizes them, and so you’re getting a two-for-one patent analysis in this review.
Here are our category definitions:
Traffic Lights: Patents focusing on how to identify traffic lights, how to classify traffic lights and how to predict upcoming traffic lights
Vehicle Lights: Patents focusing on how to identify and classify brake lights and turn signals
Each of the patents (or publications) in our data set are categorized according to the above categories and summarized as shown below:
Format: Keyword: Problem-Solution Statement – Company (Document Number)
Example: Persistent Likelihoods: How to detect changes in a vehicle environment using persistence likelihoods of map features – TOYOTA (US10710599)
We have taken the 10, 20, 50+ page patent document and turned it into a single, simple, insightful problem-solution statement that you can use to quickly assess for relevance, and you are just one click away from access to the full document.
You’re welcome!
#Category 1: Traffic Lights
Here are the most pressing problems being addressed in the field of traffic lights based on our patent data set:
How do you create a traffic light detection training model?
How to train an automatic traffic light detection model using a training model with annotated images - FORD (US20180211120A1)
In general, how do you detect a traffic light?
How to detect a traffic signal by calculating a confidence score of an image portion based on color and luminance values - WAYMO (US10346696B1)
How to detect an intersection image based on the illuminance - TOYOTA (US20180314902A1)
How to detect a traffic light by using synchronized pixel detection in combination with a positional calculation and a signal lamp determination - NISSAN (US20180137379A1)
How to identify a traffic signal using a localization neural network and a classification neural network - GM GLOBAL (US10699142B2)
But what if the traffic light is obstructed?
How to detect an obstructed traffic signal by combining map data and image data to estimate a continuous obstruction state - NISSAN (US9922259B2)
How do you determine the current state of the traffic light?
How to categorize a traffic indicator using YOLO (You Only Look Once) object detection and BLOB (Binary Large Object) analysis on the color components of an image - HONDA (US10614326B2)
How to classify a traffic signal by converting an RGB frame to a HSV frame for classification by an artificial neural network - FORD (US20180144203A1)
How to confirm a traffic light status by comparing a communicated traffic light status to a imaged traffic light status - MAGNA ELECTRONICS (US10235581B2)
Even better, how do you predict the future state of an upcoming traffic signal? This set includes the “Most Valuable” Waymo patent, defined above:
How to determine the future state of an upcoming traffic signal based on the state of a preceding traffic signal and the timing and distance to the upcoming traffic signal - INTEL (US20190130199A1)
How to estimate a state of a traffic signal based on a relationship to a different traffic signal - WAYMO (US20180114077A1)
And finally, how do you solve the yellow light problem - do we slam on the brakes or hit the gas?
How to determine whether a vehicle should run through a light based on image size of the detected light signal in combination with vehicle distance and velocity - RICOH (US10339399B2)
#Category 2: Vehicle Lights
Now we turn to the most pressing questions in vehicle light detection and classification:
First, how do you identify a vehicle signal or light?
How to identify a light source candidate using overexposure of a moving light emission source - SUBARU (US9904860B2)
How to detect vehicle lights by calculation scene brightness based on gray values in a compressed image - APTIV TECH (US20200042809A1)
How to identify signal lights of a vehicle using position-shift amounts as inclusion and exclusion criteria - SUBARU (US10121083B2)
How to identify a vehicle signal light by combining data from a camera and a LIDAR sensor to identify a subsection of an image relating to a vehicle signal light - GM GLOBAL (US10163017B2)
How to detect another vehicle using a neural network to classify a contour in a thresholded LAB image - FORD (US20180211121A1)
What about rear signal lights, specifically?
How to identify rear signal lights of a vehicle using a convolutional network and a long short-term memory recurrent neural network (LSTM-RNN) to classify a brake state or a turn state - TOYOTA (US20190092318A1)
How do you classify these vehicle signals?
How to detect a tail light signal by generating a ground truth of either an invisible signal, a visible but not illuminated signal or a visible and illuminated signal - TUSIMPLE (US10387736B2)
How to determine the pitch angle of a vehicle using a structured light pattern - JOYSON SAFETY SYSTEMS (US10083361B2)
What about classifying vehicle signals at night time versus during the day?
How to classify brake light status while incorporating a night mode or day mode based on pixel brightness - FORD (US20190156132A1)
What if the vehicle signal is obstructed?
How to detect an obstructed vehicle signal by identifying a light aura - VEONEER (US9990551B2)
How do you estimate risk based on a vehicle signal?
How to estimate a risk value based on brake light detection based on distance, relative speed and deceleration rate of preceding vehicle - CONTI TEMIC (US10140531B2)
How do you differentiate between a left turn and a right turn?
How to differentiate between a left turn signal and a right turn signal based on determination of a middle region of a vehicle - MAGNA ELECTRONICS (US20180068191A1)
And, finally, how do you check that the lights of your own vehicle are functioning properly?
How to determine whether a same vehicle lamp is broken by comparing images from before and after the lamp is turned on - MANDO (US10152640B2)
#Data Summary & Insights
We can summarize our data as follows:
Traffic Lights: Training Model (1), Detection (4), Obstructed (1), Current State (3), Future State (2), Yellow Light (1)
Vehicle Lights: Identification (5), Rear Signal (1), Classification (2), At Night (1), Obstructed (1), Risk (1), Left/Right Signal (1), Light Function (1)
#In the Next Issue
Next we look at an even more complicated problem - identifying and navigating construction zones.
#See You Next Week!
Chris Frank, “The Patent Guy”