Nuro Driver™
Release Notes

AI Model Improvements

1. Detecting Human Traffic Controllers: Building on the 10% improvement in recall achieved last month, our module for detecting human traffic controllers has seen another approximate 10% enhancement leading up to an accumulated total progress of 20%. This is crucial as it ensures increased safety and better navigation in complex urban environments where human traffic controllers are common.
2. Detecting Emergency Vehicles: We have Improved thresholds for emergency vehicle detection, which has resulted in an over 20% improvement in recall, further promoting Nuro Driver's compliance with applicable traffic rules and safety for road users.
3. Boosted Perception Performance: By updating how Nuro Driver prioritizes and synchronizes events, we have improved the mainline path latency of our primary perception model by up to 25% and CPU usage by almost 7%. This improvement allows us to scale the parts of the model responsible for non-mainline but still essential tasks (e.g., online mapping) without affecting the overall latency of the system.
4. Optimizing Compute: Reduced our overall CPU usage for Nuro Driver’s autonomous capabilities by 18% while keeping all functionality and latencies unchanged. These types of wins help ensure scalability and cost efficiency as Nuro Driver expands to new operational domains.

1% Better

5. Accelerated Batch Job Execution: We’ve made significant improvements to Nuro’s in-house Batch Task Execution System by enhancing our queueing backend and prioritization processes. These features lead to ~ a 50% increase in job completion speed, particularly during high loads. They also allow for manual adjustments of job priorities and improve resource utilization. These enhancements play a key role in increasing efficiency, saving time, and maximizing resource management.

* All metrics are based on our internal evaluation test sets made up of challenging and diverse on-road and simulation scenarios.

AI Model Improvements

1. Better Performance via unification of Perception Models: By achieving unification of our geometry models including speed estimation, we’ve seen large improvements across our Perception stack. Some highlights include:
   • Improved detection of foreign objects & debris at night by 7.5%.
   • Improved tracking on uncategorized moving objects with 40% reduction in false positive tracks and 30% improvement in speed estimation errors.
2. Smoother Pullover & Pullouts: Enabled reinforcement learning to help improve our pullover and pullout plan selection, resulting in a smoother Nuro Driver pickup and delivery experience.
3. Enhanced Detection of Rain Intensity: We've deployed an upgraded rain detection model that activates sensor clearing, resulting in significant performance improvements:
   • Light rain recall improved by 83%
   • Heavy rain recall increased by 8%
   • Elimination of false-positive rain detections

1% Better

4. Better Emergency Vehicle Detection: Updated Nuro Driver with a new Vision Transformer (ViT) architecture, boosting our emergency vehicle detection by 5% without any increase in false positives.
5. Improvements in Human Traffic Controller Detection: We’ve improved the recall of human traffic controllers by 10% without any false positive regressions by leveraging our unified perception model architecture.

* All metrics are based on our internal evaluation test sets of challenging and diverse on-road and simulation scenarios.

AI Model Improvements

• Perception Enhancements: We've upgraded the Nuro Driver’s perception model, reducing false negatives by over 20% in aggregate across all scenarios. Improvements were particularly notable in demanding cases, such as a 10% increase in orientation accuracy for large vehicles.

• World State Encoding: We've enhanced our world state AI model by integrating continuous occupancy height, improving our navigation around road irregularities like bumps and potholes.

• Optimal Plan Selection via AI: Our reinforcement learning model now assists in making smarter plan selections, optimizing the decision-making process.

• Imitation Learning: By incorporating a Human Feedback reward model into our behavior model, along with the leverage of Active Learning, we encouraged safer behavior in various conditions including narrow corridors, reactions to other vehicles, and unprotected maneuvers leading to more assertive, yet still safe, decision-making at intersections.

1% Better

• Comfort: We enhanced the passenger comfort of the Nuro Driver by tuning lateral acceleration and jerk limits for enhanced turn-taking comfort. Also, by including pedestrians and cyclists in our risk framework, we significantly reduced undesirable braking events by 50% in cyclist scenarios, 35% in pedestrian crosswalk scenes, and 60% in all other pedestrian encounters.

• Improved Tracking in Cold Weather: We've eliminated false-positive tracking on exhausts in cold weather.

• Enhanced Redundancy: We strengthened safety nets around unprotected maneuvers, resulting in a 20% improvement in yielding behavior, an 18% enhancement where the Nuro Driver was moving too slowly, and a considerable 36% increase in circumstances where more conservative behavior was desired.

* All metrics are based on our internal evaluation test sets made up of challenging and diverse on-road and simulation scenarios.

AI Model Improvements

• Unified perception model: In another big step towards a unified perception AI model, we’ve integrated our radar model. Through this work we improved the recall for motorcyclists in far-range scenarios by 4.7%.

• Human traffic controller model: Ensuring that we can safely handle "unusual scenes" is crucial. To facilitate this, we've added a new capability that identifies human traffic controllers, like police officers directing traffic or crossing guards, into the unified perception model. This has resulted in a 43% increase in recognizing human traffic controllers, and an impressive 380% rise in detecting those holding signs, while still maintaining high precision.

• Data quality: By refreshing our dataset and retraining our AI models, we are seeing considerable improvements in reaction to obstacles and interactions with pedestrians and cyclists. Specifically, there's a notable 47% increase in the system's performance when reacting to obstacles such as curbs, vegetation, and unknown objects.

• Active Learning: Leveraging our toughest scenarios where previous AI models struggled, we've made substantial improvements through Active Learning. Key improvements include:
  • A 7.3% improvement in lead vehicle reaction and follow distance.
  • A 20.8% improvement in Nuro Driver’s behavior in complex situations like dealing with other agents in driveways, parking, and executing multi-point turns.

1% Better

• Unprotected maneuvers: We've optimized “go” plan speeds for unprotected maneuvers, resulting in a 5.8% improvement. Our AI has improved by becoming quicker in responding to unprotected maneuvers.

• Perception of reversing vehicles: We've made significant progress in handling difficult perception cases, which include reversing and multi-point turning vehicles. This, paired with Active Learning improvements, markedly boosts our performance in these tricky scenes.

• Vehicles with open doors: Moreover, we've solved almost every instance in our internal test set of false negative and false positive detections related to vehicles with open doors.

• Continuous improvement: This month, we've accomplished significant progress in optimizing driving speeds on different types of roads, making our systems more comparable to expert human drivers. This goes beyond simply adhering to speed limits. It also incorporates situational cues such as the width and type of the road, as well as the density and proximity of parked cars.

* All metrics are based on our internal evaluation test sets made up of challenging and diverse on-road and simulation scenarios.

AI Model Improvements

• Parking lot driving: We’ve further trained our machine-learned driving model with diverse parking lot data. The new model performed 21% more like a human driver in parking lots. We are now training our other ML models to further maximize performance, allowing us to improve driving in varied types of partner parking facilities efficiently in the near future.

• Multi-GPU model: We've developed an in-house solution that utilizes multiple GPUs to handle sophisticated AI models, which would ordinarily exceed a single GPU's capacity. This debut launch has led to an approximately 28% reduction in latency for our on-board perception model.

• More accurate prediction model: Enabled a new technology that boosts the consistency between our predictions about other road users' behavior and the trajectory selected by the Nuro Driver. The updated model shows a 12% improvement in interaction predictions with other vehicles, thereby simultaneously upholding safety standards and enhancing ride comfort.

Our December release shows better intersection handling, fine-tuned steering and improvements across perception.

New Features & Machine Learned Models Improvements

• Improved vehicle handling: We've fine-tuned our steering controller, achieving up to 20% improvement in steering tracking at high speeds. This enhancement leads to more responsive, human-like vehicle handling.

• Better handling of atypical moving objects: By enabling a new perception model, we improved precision by 18.3% and recall by 5.5% for infrequent moving objects like bouncing balls on the road.

• Improved Intersection Interactions: We've enhanced our intersection logic for better understanding of cross traffic patterns.

1% Better

• Improving our perception of the world: December was a great month of incremental improvements to our perception stack including improved orientation detection in reversing vehicles by 65% and better detection of objects on steep roads.

• Improving efficiency in detecting and resolving issues: We've enhanced in-vehicle tooling with new features that allow for better system visualization and health monitoring, including connectivity. This expedites our testing iteration cycles and accelerates our autonomy progress.

Our November release shows better handling of occluded intersections, smoother driving and much more efficient labeling.

New Features

• Enhanced comfort: By refining our model to decelerate more smoothly, we've attained a 56% improvement in stop and go scenarios.

• Improved handling of occluded intersections: We've upgraded our occlusion reasoning and creeping logic, enhancing our handling of obscured, unprotected intersections.

• Improved delivery experience: We have improved the performance in pull-out maneuvers and low-speed steering, which has improved performance by 69%.

• Prioritizing safety: We've activated our full stack parallel backup system on our next generation vehicle platforms, which operates an independent perception and emergency braking system.

1% Better

• Human-like driving: Through our enhanced ML training methods and human preference feedback, we've improved our decision-making in nearly all situations.

• Perception labeling efficiency: We've automated 75% of our 3D cuboid labeling process, resulting in a 67% reduction in error rates, 24% time-savings, and a 25% cost reduction.

• Faster response: Through enhancements in our Machine Learning models' scheduling and synchronization, we've successfully reduced our perception latency by 41% in November. This significant decrease will contribute to more efficient and responsive autonomous behavior.

Our October release shows smoother acceleration and deceleration, expanded object detection range and even better lane changes.

Exciting New Features & Model Improvements

• Smoother driving: We've implemented a new learned model that improves our driving trajectories, resulting in smoother deceleration at stop signs and faster acceleration on high-speed roads. This increases our human-like driving performance by over 15%.

• More efficient tracking: Through selective tracking of agents that might interact with us, we’ve boosted the efficiency of our Nuro DriverTM. Key results include a notable reduction in computational demand and a 98% decrease in latency spikes.

• Perception improvements: We’ve​ expanded our object detection range by 13%, boosting our ability to handle high-speed roads and busy scenes.

1% Better

• Refined lane changes: Our lane change logic is now more intelligent, improving lane change behavior when approaching intersections.

• Platform tuning: We’ve fine-tuned our braking controls, achieving a 95.4% improvement in longitudinal controls performance.

• Better decision-making: We deployed a new, more comprehensive framework for validating that the various plans under consideration are safe.

• Longer distances: By adjusting our route policies and reducing the computational load, we've doubled the maximum distance for a single journey.

* Statistical information regarding percentage improvements is based on internal data.

Our September release shows us raising the bar by prioritizing machine learning, upgrading our on-board compute and upping our testing efficiency. Some resulting improvements include better lane changes, smarter driving and sharper detection.

New Features & Model Improvements

• Prioritizing Machine Learning: We've shifted to using our machine learning system as the primary decision-maker for driving, while our expert rules-based systems serve as safety backups during uncertain situations, underlining our commitment to safety.

• Upgraded Onboard Compute: We've begun transitioning our vehicles to our next-gen computing system, offering a significant uplift in compute capacity, with improved manufacturability for scaled production. This supports complex perception and behavior tasks in our autonomous system while preserving optimal performance.

• Lane Changes on Higher Speed Roads: Following a general increase of driving speed to 35 mph in August across our test fleet, this month we enabled lane changes on 35 mph roads. This enhancement broadens our route possibilities, making journeys to a chosen destination more versatile.

More efficient off-board testing

• Night Driving: We significantly increased the coverage of our offline night-time testing, particularly around scenarios involving pedestrians, cyclists, and cross-traffic.

• Expanded Test Coverage: We expanded testing for difficult driving scenarios, upscaling tests involving unprotected maneuvers 20x and narrow roads with oncoming traffic 100x. A 10x speed increase in dashboard loading paves the way for quicker reviews by the engineering team.

1% Better

• Better Driving: Thanks to smarter data collection & handling, we've upped our human-like driving performance by almost 16% and decreased nudge events by 8x, enabling even smoother rides.

• Managing Latency: For more efficient feature exchange, we’ve combined parts of our localization and perception models, resulting in a 72% drop in latency and significantly lower memory usage, with no performance compromise.

• Better Detection & Response to Unusual Situations: We've focused on sharpening detection of active emergency vehicles and school buses.

* Statistical information regarding percentage improvements is based on internal data.

Each month we track improvements with our latest software release. In August we had a number of exciting developments resulting in a smoother ride with better navigation.

Exciting New Features & Machine Learned Models

• Faster Driving: Activated driving at speeds up to 35 mph on our whole fleet and improved our driving performance to act 36% more like a human expert.

• Better Predictions: Rolled out a new prediction model that significantly improved predictions for vehicles by 16%, cyclists by 19%, and pedestrians by 13%. Greatly improved predictions for vehicles coming out of driveways.

• Improved Traffic Signal Detection: Enhanced our ability to detect traffic signals with a new end-to end traffic signals classifier and a newly deployed perception model.

• Interface Redesign: Upgraded our testing fleet tablet interface with improved audio commenting.

1% Better Every Day

• Vehicle Controls Improvements: Delivered a smoother drive with 25% less jerky throttle/brake, enhanced acceleration/deceleration, and 90% fewer jerky steering motions.

• Better Agent Interactions: Improved navigation around pedestrian & unprotected stop interactions. More assertiveness for narrow oncoming interaction and better behavior around occlusions and on road pedestrians and cyclists leading to fewer brake jabs.

• Perception Improvements: High coverage labels provided by the data mining and labeling pipelines yielded performance improvements across key perception metrics.

  • Recall Improvements: 11% on Pedestrians, 8% on Vehicles, 5% on Cyclists, 4% on Animals, 4% on Motorcyclists

  • Improvements in False Positive Rates: 71% Motorcyclists, 26% Vehicles, 24% Cyclists, 19% Animals

• Up-To-Date and Faster Technology Updates: Sped up our bot updates so every vehicle is always up-to-date with the latest tech.

* Statistical information regarding percentage improvements is based on internal data.