When should a car brake at a stop sign when it’s snowing outside? When will he turn on his lights? How can you tell the difference between a street light and a full moon?
These are some of the questions that Tesla’s Autopilot team makes every day.
Tesla’s driver software relies on a small army of data analysts who analyze thousands of hours of footage from Tesla owners and the company’s drivers. Explainers are gradually teaching the company’s AI to behave like a driver, one clip of 30 seconds at a time. Tesla employs many writers who make about $20 per hour in full-time jobs at the company.
Business Insider spoke with 17 current and former employees of Tesla’s data annotation team to understand what it’s like to support the company’s Autopilot and fully self-driving software. The team is split across three Tesla factories in Buffalo, New York, Palo Alto, California, and Draper, Utah.
The work on the team can take anywhere from months to days and can vary from project to project that requires users to play small clips to locate images or cover satellite data.
“Sometimes it can be worse,” said one former employee. “You could spend eight hours a day for months on end just listing the lines and blocking over a thousand videos.”
‘Look closely at human life’
The clip can provide a unique window into the daily lives of Tesla drivers. At one point, there was a project that required users to copy data from some owners’ cars through Tesla’s Sentry Mode, five employees said.
Another task, called “Selfie,” requires some writers to write down the data obtained from Tesla’s internal cameras, according to two employees who saw the employees working on the project. Four other employees said they knew about the program. The Selfie event was designed to teach the Tesla system how to recognize when a driver is not paying attention to the road while Autopilot is active, they said.
Tesla has said that the camera in the interior of the car “shares short camera videos with Tesla to help us develop future safety improvements and continue to improve the intelligence of the system based on and home cameras,” according to the owner’s manual. Tesla owners must first log in and share their data for Tesla subscribers to be able to access the video, the company said.
In other cases, users found themselves writing down the names of YouTubers and even Elon Musk himself, BI previously reported.
One current employee said: “There’s something strange about having this close view of a person’s life.” “It feels bad to see people’s cars every day, but it’s also an important part of improving and improving the program.”
The videos are filmed from across the US, as well as some locations in Europe and South America, 15 officials said. Two employees recalled video footage they believe was taken from a passenger car in Ukraine during the Russian invasion.
Business Insider reached out to Tesla, Musk, and his legal team for comment but did not receive a response before publication.
Tesla’s driver assistance software can automatically change lanes and stop at traffic lights. Mark Matousek / Business Insider
Users can access data from any country in the same workflow, which means that they must know how to create different paths for each region. At times, Tesla appears to be taking a more relaxed stance on the rules, seven employees and current employees said. For example, some employees said that they were told to ignore the signs “No Turn Red” or “No U-Turn”, which means that they will not train the system to obey the signs.
“It’s the driver’s first impression,” said one former employee. “I think the idea is that we want to train him to drive like a human, not a law-abiding robot.”
Sometimes the service requires users to record videos from accidents and missing people. Seven users recalled playing videos that included Tesla crashes or those involving nearby vehicles. At one point, an employee shared a video among employees of an incident involving a boy in a Tesla that was hit, four employees said. They say it’s one of the many videos and memes users exchange.
Last year, Reuters was the first to report on the small bike and potential privacy issues on the site. Shortly after the news was published, Tesla began restricting access to videos outside of designated users’ services and watermarked some videos and images so they could identify which users were sharing the images, who nine jobs told BI.
Tesla employee monitoring system
Tesla has a strong employee monitoring system at its Buffalo site. The facility has multiple surveillance cameras on the site overlooking the work space, 11 employees told Business Insider.
Users are also monitored effectively using two different software systems.
One system, called HuMans, monitors how long they should spend on each video, four users said. Writers who take longer than the allotted time may receive poor performance reviews or be placed under a performance improvement plan, or PIP, they said. The software was originally developed to assist pilots in the US Air Force and also has the ability to track users’ eye movements and record audio, according to its website. But it’s unclear whether Tesla uses the software to monitor users’ eye movements.
The company also uses a measurement called “Flide Time” to track the working hours of the annotators in the marking software, 17 employees said. They said it can track keystrokes and how much time users spend on open-source software, but it won’t track the time users spend using other tools on their computers. Depending on their level, users can be expected to log anywhere from five to seven and a half hours of Flide Time, which means they will be actively working on the software for at least that amount of time.
If workers are less than five minutes short of their designated Flide Time, they could face fines, six workers said. If they miss the Flide Period three times in six months, they can be terminated, the officials added.
Some Tesla employees have tried to fall back on the company’s metrics for employees with little success.
In February 2023, some workers at Tesla’s Buffalo facility tried to unionize. Union organizers at the Buffalo office told Bloomberg that the company is monitoring their keys and said some workers are fed up with being “treated like robots.”
That same month Tesla laid off several workers at the Buffalo plant. At the time, the National Labor Relations Board (NLRB) filed a complaint that Tesla illegally terminated some workers “in retaliation for union activity and to discourage union activity.” But, Tesla denied the allegations, saying that the workers were fired for poor performance instead. The NLRB did not respond to a request for the current status of the complaint.
When Tesla started building its driver assistance program in 2016 the company removed data signals from a California company with offices in Kenya, but Tesla brought the program in-house in 2019, Reuters reported.
Most recently, Tesla’s Autopilot team failed in April. Tesla has laid off nearly 300 workers in Buffalo, according to a WARN notice.
Tesla has said that its neural network will one day be able to train itself, but for now, it relies on human operators.
The project is essential to Musk’s vision for the auto industry.
Over the years, the Tesla CEO has repeatedly emphasized the importance of Tesla’s efforts to achieve autonomous driving. In 2022, Musk said that Tesla’s self-driving technology is “the difference between Tesla being expensive or cost-free.”
Tesla plans to unveil its autonomous Robotaxi service this year, which is expected to be built on the same self-driving software — and, of course, exhaustive research, video compilations of its subscribers.
Do you work for Tesla or have a tip? Contact the reporter by email and the device does not work gkay@businessinsider.com or by signaling a secure message at 248-894-6012.