How Do Self-Driving Cars Work? Sensors, AI and Autonomy Levels Explained

Self-driving cars work by combining sensors (cameras, radar and lidar) that see the world, software that works out what everything around the car is and what it will do next, and a planning system that decides how to steer, brake and accelerate, many times every second. Fully driverless robotaxis now carry paying passengers in more than a dozen US cities, while most cars sold today offer only driver-assistance features that still need a human in charge. Here is how the technology works, how autonomy levels are defined and where it is heading.

What Counts as a Self-Driving Car?

The industry uses six levels defined by SAE International in its J3016 standard, which is also referenced by the US National Highway Traffic Safety Administration:

• Level 0, no automation: warnings only, such as a blind-spot alert.
• Level 1, driver assistance: the car helps with either steering or speed, for example adaptive cruise control.
• Level 2, partial automation: the car can steer and control speed together, but the driver must supervise at all times. Most “autopilot” style systems, including Tesla’s Full Self-Driving (Supervised) and GM’s Super Cruise, are Level 2.
• Level 3, conditional automation: the car drives itself in limited conditions, such as slow motorway traffic, but the driver must take over when asked.
• Level 4, high automation: no human driver is needed within a defined area and set of conditions; this is how robotaxis operate.
• Level 5, full automation: the car can drive anywhere a human could, in any conditions. No Level 5 vehicle exists today.

The key line is between Level 2 and Level 3. At Level 2 and below you are driving, even if your hands are off the wheel; at Level 3 and above, the system is driving while it is engaged.

What Sensors Do Self-Driving Cars Use?

Most autonomous vehicles combine several types of sensor, because each has strengths and weaknesses:

• Cameras: cheap and high-resolution, they read traffic lights, signs, lane markings and the colour of objects, but struggle in glare, darkness and heavy rain.
• Radar: uses radio waves to measure the distance and speed of objects, and works well in fog and rain, though at lower resolution.
• Lidar: fires laser pulses to build a detailed 3D map of the surroundings, accurate to centimetres, day or night.
• Ultrasonic sensors: short-range sensors used for parking and very close obstacles.
• GPS and inertial sensors: track position, speed and orientation, often matched against detailed pre-built maps.

Companies disagree about the right mix. Waymo uses lidar, radar and cameras together, a belt-and-braces approach known as sensor fusion. Tesla relies on cameras alone, betting that powerful neural networks can learn to drive from vision, as humans do.

How Does the Software Make Decisions?

The software stack is usually described in four stages:

• Perception: combining sensor data to detect and classify everything nearby, including cars, cyclists, pedestrians, cones and debris.
• Localisation: working out exactly where the car is, often to within a few centimetres, by matching sensor data against high-definition maps.
• Prediction: estimating what each road user is likely to do over the next few seconds, such as whether a pedestrian will step off the kerb.
• Planning and control: choosing a safe, comfortable path and sending precise commands to the steering, brakes and accelerator.

Much of this relies on machine learning trained on huge amounts of real-world and simulated driving. Developers test rare, dangerous scenarios billions of times in simulation before cars meet them on real roads. Increasingly, companies are moving towards large “end-to-end” AI models that learn the whole task, from sensor input to driving output, rather than hand-coded rules for each stage.

How Big Are Robotaxis in 2026?

Waymo, owned by Google’s parent company Alphabet, is the clear leader. According to fleet data reported by TechCrunch in September 2026, Waymo was operating in 15 US cities and providing around 500,000 paid rides a week, with a fleet of about 4,000 robotaxis, roughly 80% of them in California and Texas. Two years earlier, its service was limited to Phoenix, Los Angeles and San Francisco.

The company is adding a purpose-built vehicle, a Zeekr-built minivan called Ojai that runs its sixth-generation driving system, alongside its Jaguar I-Pace fleet. Rivals include Tesla, which runs a robotaxi service in Austin, and Amazon-owned Zoox, which uses a vehicle with no steering wheel or pedals.

Are Self-Driving Cars Safe?

Waymo publishes safety data comparing its driverless miles with human drivers in the same cities, and reports substantially fewer injury-causing crashes. Independent researchers have broadly found similar results, though robotaxis still make headline-grabbing mistakes, such as blocking emergency vehicles or getting confused at roadworks.

Driver-assistance systems are a different matter. The most common safety concern with Level 2 systems is driver inattention: people treat them as self-driving when they are not. The NHTSA requires companies to report crashes involving automated driving systems and advanced driver assistance systems, and has opened several investigations into how these systems monitor drivers.

What Are the Biggest Challenges?

• Edge cases: unusual situations, such as a police officer directing traffic by hand or a mattress in the road, are hard to train for.
• Weather: snow can hide lane markings and block sensors.
• Cost: lidar-equipped robotaxis are expensive, although sensor prices have fallen sharply.
• Regulation: rules vary by state and country, and liability after a crash is still being worked out.
• Public trust: surveys show many people remain wary of riding in, or sharing the road with, driverless cars.

Can You Buy a Self-Driving Car?

Not a fully self-driving one. Consumers can buy cars with advanced Level 2 systems that steer, brake and change lanes on motorways, and a few manufacturers offer Level 3 systems that work only in specific conditions, such as Mercedes-Benz’s Drive Pilot on certain US motorways. If you are shopping, check exactly what the system can do, where it works and what the manufacturer says about supervision. Read the manual, keep your eyes on the road and treat any Level 2 feature as an assistant, not a chauffeur.

What Happens Next?

Expect robotaxi services to keep expanding city by city, motorway freight trucks to be one of the next commercial uses, and AI models trained on ever larger datasets to improve how cars handle rare situations. The same technologies also power other robots, from warehouse machines to robot vacuums, which use similar mapping and obstacle-detection techniques at a much smaller scale.

For riders, the practical question is simpler: is a robotaxi available where you live, and is it priced competitively with ride-hailing? In the cities where Waymo operates, rides are booked through an app much like Uber or Lyft, and the car arrives with nobody in the driver’s seat.

Self-Driving Cars at a Glance

• How they see: cameras, radar and lidar, combined through sensor fusion
• How they think: perception, localisation, prediction and planning
• Autonomy levels: SAE Levels 0 to 5; robotaxis are Level 4
• Most cars today: Level 2, with the driver always responsible
• Market leader: Waymo, about 500,000 paid rides a week in 15 US cities (September 2026)

Featured image: Waymo Jaguar I-Pace in San Francisco 2023 dllu, by Dllu / Wikimedia Commons, CC BY 4.0. Cropped, colour-graded and combined with MediaPavilion branding.

Sources

• SAE International — SAE levels of driving automation (J3016) (accessed 27 October 2026).
• TechCrunch — Waymo is scaling fast. Here’s what the fleet data shows (24 September 2026).
• Waymo — Safety impact (accessed 27 October 2026).
• National Highway Traffic Safety Administration — Automated vehicles for safety (accessed 27 October 2026).