An offshore workforce is training Amazon’s warehouse-monitoring algorithms

This report is published in collaboration avec the Bureau of Investigative Journalism.

Squat blue robots with eight-foot yellow shelving units dance around one another in a huge Amazon warehouse in Beaumont. 

Amari* works 42 hours a week there as a stower, placing products on the shelves robots bring to him. “Cameras are trained on your station at all times,” he said. “It’s kind of demeaning to have someone watching over your shoulder at every second.”

But it’s not just Amari’s managers who are watching. An AI camera system also monitors the stowers’ movements — and if it fails, a video is sent to someone thousands of miles away whose input helps to improve Amazon’s machine learning tools.  

Workers like Viraj in Bengaluru (India) review the videos. “It is very hectic work,” he said. “We shouldn’t blink our eyes while reviewing a video because our accuracy will go less. We have to be on screen at least eight hours — which is kind of painful.”

Viraj and Amari may be working in different countries, doing different jobs. But both perform mind-numbingly repetitive tasks whose output is strictly monitored, all of it serving to fine-tune the very system used by Amazon to closely monitor its own workers — and create the seamless experience enjoyed by its customers.

Viraj, a reviewer, can watch up to 8,000 videos per day and have their output compared with that of their peers. They can be physically exhausting due to the relentless pace at which they work. They can earn as low as $252 per month.

Bureau of Investigative journalism interviewed video reviewers. They reported eye pain, headaches and even worsening of their vision. According to them, they were forced to meet punishing targets. Tracking software logs any inactivity beyond the designated breaks. One expert suggested that video annotation users should take screen breaks at least every 30 minutes, but others said that their targets didn’t allow this.

The Bureau interviewed 33 current and former Amazon employees, including 21 video reviewers, to shed light on a little-known outpost of Amazon’s sprawling global operations.

Amazon spokesperson Steve Kelly disputed a number of the allegations in this story, saying the Bureau and Verge had “selected a handful of anecdotes to paint a misleading picture, and we do not believe they represent the vast majority of our team.”

Amazon has developed an extensive range of applications that use computer vision — a branch of machine learning in which computers process large numbers of images and learn to recognize patterns.

The cameras trained on Amari’s station use computer vision to automatically register the location of products in its inventory and flag errors he makes. This technology can also be used inside Amazon Go stores, to monitor compliance by warehouse workers with social distancing guidelines.

Amazon says the system’s algorithm is 95 percent accurate; the remainder of cases require manual checks. This means that millions of images and videos are sent daily to workers in India and Costa Rica who determine if a product has been successfully stowed and indicate its location on the shelving unit. 

Video reviewers said their primary role was stock management — but they can also record errors made by their colleagues overseas: two former workers said reviewers could raise “stow etiquette” issues if they saw stowers breaking Amazon’s rules on camera. 

But, most importantly, their manual labor contributes to improving the computer vision software. The system learns from them and becomes more accurate. But the people teaching Amazon’s computers to see said their own eyes have been damaged by the work. 

“We will not be able to even blink our eyes as we need to keep a watch on the videos,” said Prisha, a former video reviewer based in Hyderabad, India. “That impacted my health a lot. It makes the eyes really dry because you constantly stare at that screen.”

Reviewers stated that they could watch thousands of videos in one day. The videos last anywhere from two to two minutes. Shifts last between eight and nine hours, but can be extended up to 11 hours during busy times like Black Friday or Christmas. Reviewers get around one and a half hours’ break time, with any periods of inactivity outside of that instantly logged by tracking software.

Interviewees from India claimed that they earn 25,000 rupees ($306) a month, while the Costa Rican median was 514,000 colones ($843). 

The humans behind Amazon’s all-seeing computer vision are themselves tightly monitored while they work. Mateo, an ex-reviewer from Costa Rica, spent his shifts ensuring that warehouse workers in the USA were following covid protocols. He noticed something unusual in his feed one time.

It was an Amazon breakroom with chairs, which was very similar to his own. He felt uncannily watched. “Probably someone else, somewhere else, was watching me at the moment I was watching them,” he said.

Managers keep track of reviewers’ performance with real-time analytics, and they must maintain a high accuracy rate, between 95 and 99.5 percent. Nitara, who worked for Amazon in Bengaluru, was unable to complete her probation period because she missed her accuracy targets. “We were not allowed to make mistakes,” she said. “For me, that was quite difficult to handle. I’m human, I’m not a robot.”

A timer on the screen tracks how long it takes them to decide how to categorize a particular video. If they linger too long, their “TAKT” time — the average time to get through a video — will increase, and they may be subject to retraining, disciplinary processes, or even face losing their job.

“You can’t move or do anything,” said Prisha. “If you even give a little gap, your TAKT will increase, and you may land at the bottom.”

The Bureau was provided with a document by an Indian former reviewer. It shows the TAKT ranking system of 25 employees. The bottom four names are highlighted in red. The fastest performer has a time of 5.7 seconds while the lowest-ranked person has a time of 13 seconds.

Jiyan, another former reviewer based in India, said that while the targets were manageable, the work was still “stressful.” What bothered him most, though, was the monotony. “It’s a very boring job,” he said. “The entire day, for seven and a half hours, you’re doing the same thing over and over again. There is nothing new.”

In Amazon’s peak period of October to December, reviewers said their work increased significantly, with less time between videos, and one said bathroom breaks were more difficult to take. Another India-based worker said he couldn’t take time off for the Hindu festival of Diwali.

Amazon said that workers in India had the option to take Diwali off, and the company’s spokesperson Kelly said workers in India and Costa Rica were “encouraged by the software they use to take short breaks throughout their shifts.”

In the summer of 2020, Amazon’s warehouse in Bolton, England, was rolling out a new stowing system — known internally as “Nike” — that relied on computer vision and manual checks from workers in India and Costa Rica.

The workstations were remodeled with three new cameras that are trained on stowing stations. These cameras would record the location of each product and eliminate the need to use handheld scanners. Amazon stated that this would reduce the time required to store each item. 

Naomi, who was working in the warehouse at that time, had difficulty adapting. “It was just quite nitpicky — the way you had to stand, the way you had to move,” she said. “You couldn’t really have your own freedom in the way you did things.”

To maximize the computer’s chance of success, stowers were told to ensure they were in view of the camera and to use “clean and straight movements” when putting an item away. Video footage of any failures was sent to video reviewers for verification.

The new system was also launched at Jade, a former supervisor in California, would often start her work day by reviewing error reports from previous days. “There were like 30 or 40 different rules of how things had to be specifically stowed,” she said. 

Many of these reports, containing photos of the stowers’ violations, were generated by the new camera system. If enough of these errors accumulate, it could lead to a disciplinary process known as a “write-up.”

Amazon claims that the automated system was not used to discipline employees. “Stowing metrics are shared with employees and managers to identify strengths and opportunities for growth,” said Kelly. “We don’t use the Nike system information to coach associates on ‘stow etiquette violations’. Nike cameras are programmed only for inventory purposes … Their focus is product placement.”

Jade would also check that workers were performing well in other key metrics: “rate”, or the number of units they stowed per hour, and “time off task”, or how long they were inactive outside of break times. 

Isaac, a former tower worker at a Michigan warehouse received a write up after he had spent approximately four minutes off his task. He felt unwell so he went to the toilet and took some medication. He was not able to explain his situation to his manager and still received a written warning.

(Amazon insists Isaac’s experience isn’t representative. “Employees are reminded to take short breaks throughout the day,” Kelly told us, “in addition to regularly scheduled longer breaks during each shift.”)

Jade would also check that workers were performing well in other key metrics: “rate,” or the number of units they stowed per hour, and “time off task,” or how long they were inactive outside of break times.

She felt that the productivity targets were realistic and achievable by anyone who tried. However, she also described the work as “mind-numbing.”

“Your brain just kind of dies slowly while you’re doing it,” she said. “Even though the people in Costa Rica aren’t doing the physical side, they’re doing the mind-numbing brain side. You’re treated like a robot.”

During 2020, when covid protocols were in place across the world, the company said it would retool its machine learning systems to help enforce social distancing in its warehouses — a program it called Proxemics. 

In the US, warehouse workers saw themselves on large screens that had a six-foot circle of green around them. If workers were too close to colleagues, the ring would turn a red color. For additional checks, images of the person were sent overseas in cases where the computer was not sure.

Thiago, who worked on the Proxemics team in Costa Rica, told the Bureau: “It was a difficult job. I think it could be the worst one I ever did in my entire life.”

Prisha felt the same way. It was difficult to look away when you want to keep high metrics. “At the beginning, my eyes would cry,” he said. “Every six seconds or less, you get another image. It was horrible. 

“It was difficult — to the point that you can’t look to the side because if you look to the side, you’ve already had 10 seconds on your screen.” 

Sudip Bhattacharya is an assistant professor at All India Institute of Medical Sciences in Deoghar. He said that video reviewers could be at risk of digital strain. Symptoms include dry eyes and impaired vision as well as headaches. “If the resolution is low,” he said, “there is a risk of permanent eye damage.”

He also suggested screen breaks every 20-30 seconds. However, some employees interviewed by the Bureau found it difficult or impossible for them to take breaks outside their allocated times.

Thiago said he was given around six minutes each day as a designated “eye refresher,” with an additional 10 minutes to use the bathroom outside of scheduled break times.

When he was promoted to team leader, he was given access to Amazon’s employee monitoring software, which tracks workers’ periods of inactivity. “They could tell how much time you didn’t touch something on the screen or you moved your mouse,” he said.

The reviewers also used footage from Amazon Go grocery shops, which use computer vision and detect the customer’s purchase, automatically billing their card without having to scan them through a checkout. 

Amazon Go’s marketing trades heavily on the futuristic novelty of an unstaffed store. However, in reality the role of shop assistant was outsourced to India’s video reviewers.

Ishan, a member of the Amazon Go team, stated that he suffered from headaches and was allowed only four minutes per day to go to the bathroom, outside of his scheduled breaks. “Some who are lucky will survive, and the rest will have to leave,” he said. “An employee is a replaceable asset for them.”

 “[It was] very minimal pay for the continuous high-level stress of a never-ending workload.”

Amazon stated that employees can use the restroom as they need and that the Bureau accounts do not reflect the majority of the people on the team.

The workers being filmed by Amazon’s cameras know little about the people watching them from the other side of the world. One of the nine stow employees interviewed did not know that footage from their station could potentially be sent abroad for manual review. 

Some workers in India, Costa Rica and other countries expressed concern about how Amazon would use their labor. “We had no idea where this particular data was going,” one worker said. “We were never given such knowledge [of] what exactly is happening in the backend.”

A new California law addresses the conditions faced by Amazon workers. Assembly Bill 701This law was put into effect in January and prohibits performance goals from being set at levels that pose a danger to safety or prevent sufficient toilet or meal breaks. It also gives warehouse workers the right to request three months’ worth of their own productivity data.

Lorena Gonzalez, a former state representative, wrote the bill. She now heads the California Labor Federation. She was worried about workplace productivity quotas causing injury and felt that workers having access to their own data would be a first step to resisting management by algorithms.

“We wanted to make sure that they had the right to that information, especially if they felt like it violated their basic labor rights to health and safety,” Gonzalez said. 

Amari, the stower in California, said the law had made a tangible difference in the warehouse and managers were no longer regularly reprimanding people for falling behind on their productivity rates — although workers could still be disciplined for other violations.

The work of a stower in California is very different from that of a video reviewer in India, but both are vital cogs in Amazon’s optimization machine. They are constantly creating new data points as they work to improve the algorithmic tools that control and discipline them.

“The only way to fight back against [algorithmic management] is to refuse to go quicker,” Gonzalez said. “But an individual can’t do that alone.”

“Until workers come together and organize as a collective against the kind of speed-up that happens with computer management, then they’re just going to … require people to go faster and faster.”