Recently Tesla announced a transition to Tesla vision, it has new upgraded AI-based self-driving tech which will just depend on the data acquired from the cameras on the car that has been trained on a supercomputer - now it has detailed its approach at its AI day event. Elon Musk had earlier revealed months ago that it had been training its self-driving neural network on something called the "Dojo" computer. But now Tesla has unveiled a supercomputer that is more like a developmental prototype to the dojo.
More about this supercomputer was revealed by Tesla's head of AI Andrej Karpathy who claimed it is this supercomputer that allows it to with radar and LiDAR sensors which have been popularized by Google's Waymo self-driving tech. Tesla instead uses high-resolution optical cameras. Karpathy revealed how Tesla trained its supercomputer to understand the environment the way a human would.
Karpathy revealed that it needed an enormous dataset, a massively powerful supercomputer to train the neural network using that dataset. That's why Tesla has been investing in supercomputers including precursors to the "dojo".
undefinedcan you tell us more about your thinking behind the pure vision approach?
— Whole Mars Catalog (@WholeMarsBlog) April 10, 2021
lots of people arguing no radar is a step backwards. why did you guys decide it was better not to use it?
Tesla revealed the new supercomputer has 10 petabytes of NVME storage and can run at 1.6 terabytes per second. It also has 1.8 EFLOPs of computing capacity which Karpathy estimates makes it the 5th fastest in the world - though he has said he's not run the relevant benchmarks that would make it the TOP500 supercomputer rankings.
"That said, if you take the total number of FLOPS it would indeed place somewhere around the fifth spot. The fifth spot is currently occupied by NVIDIA with their Selene cluster, which has a very comparable architecture and a similar number of GPUs (4480 vs ours 5760, so a bit less)," Karpathy explained to Techcrunch.
Since May, Tesla announced that it would be removing all the sensors that its cars have traditionally come with. Tesla's never were equipped with LiDARs but had radar sensors. Its autopilot is now just based on the cameras and the training algorithm that's run on supercomputers.
"The approach we take is vision-based, primarily using neural networks that can in principle function anywhere on earth," said Karpathy in his workshop. Tesla believes that replacing a human with supercomputer results in lower latencies and better 360-degree situational awareness.
Karpathy revealed that this method can correct many bad driver behaviors including in emergency braking situations where the object detection kicks in automatically to save a pedestrian from being hit or a traffic control warning that can identify a yellow light and send an alert to the driver who hasn't yet slowed down.
Tesla's newest cars don't come with any sensors
Tesla also has a feature called pedal misapplication mitigation as the car identifies pedestrians in the path or even a lack of driving path and responds to the driver accidentally stepping in on the accelerator. The supercomputer collects video from all the eight cameras that surround a Tesla vehicle at 36 frames per second which provides incredibly rich real-time environmental information.
Tesla believes this method is more scalable as it keeps collecting data but privacy advocates may not agree. Tesla feels this works better as it is better than manually building high-resolution maps something that both Apple and Google do. Tesla's data is also coming from real-world conditions as it is able to collect large amounts of data at the velocity a human does. It also is able to match the human ability to access depth and speed.
Karpathy touts the ability for the system to auto label can mark out things like roadway hazards instantly and other objects from the millions of videos that are captured by Tesla vehicles around the world. Tesla has accumulated 1 million videos around 10 seconds each and labeled 6 billion objects with depth, speed, and acceleration. This tales up 1.5 petabytes of storage.
