Analysis

Update: Tracking of Real Algorithmic Harms in California as CCPA Enforcement Begins

August 8, 2024 | With contributions from EPIC Law Clerk Alissa Johnson

As part of our project Assessing the Assessmentssupported by the Rose Foundation—EPIC has continued to track instances of specific AI harms to consumers relevant to enforcement of the California Consumer Privacy Act. These examples are listed below with the setting and application of the system, a real illustration of the harm that occurred, and what types of harm are done to consumers. EPIC has previously tracked these types of harms under its ongoing Assessing the Assessments project and will continue to track them in the future.

These examples draw on two taxonomies of AI harms to describe the types of harms done: 

  1. Danielle Citron’s and Daniel Solove’s Typology of Privacy Harms, comprising physical, economic, reputational, psychological, autonomy, discrimination, and relationship harms; and 
  2. Joy Buolamwini’s Taxonomy of Algorithmic Harms, comprising loss of opportunity, economic loss, and social stigmatization, including loss of liberty, increased surveillance, stereotype reinforcement, and other dignitary harms. 

These taxonomies do not necessarily cover all potential AI harms. EPIC uses these taxonomies to help readers visualize and contextualize AI harms without limiting the types and variety of AI harms that readers consider.

While these examples are focused on California, examples of the effects of an AI system in one jurisdiction will often apply to other states and countries as well. 

ExampleExplanationCitationTypes of Harm
Autopilot in autonomous vehiclesTesla in self-driving mode crashed into a police cruiser.Tesla driver using self-driving mode slammed into police cruiser in Orange CountyIndividual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesA self-driving taxi in San Francisco dragged a woman about 20 feet. She was struck by another vehicle, thrown into the path of the taxi, and pinned underneath it. The taxi continued driving 7 mph for 20 feet with the woman trapped underneath.A woman was dragged by a self-driving Cruise taxi in San Francisco. The company is paying her millionsIndividual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesTesla owner says his car drove onto an active train track while in self driving mode.A driver says his Tesla drove onto active train tracks after Autopilot mistook them for a roadIndividual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesTesla on autopilot ran a red light and hit another car, killing two people.A Tesla on autopilot killed two people in Gardena. Is the driver guilty of manslaughter?Individual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesLawsuit claims that autopilot malfunction resulted in fatal crash.Lawsuit claims Tesla ‘Autopilot’ killed father of two in California crashIndividual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesSan Francisco fire chief raises concerns about autonomous taxis interfering with first responders.San Francisco’s fire chief is fed up with robotaxis that mess with her firetrucks. And L.A. is nextIndividual: Financial, Privacy & Autonomy; Societal: Equality
Autopilot in autonomous vehiclesTesla autopilot feature was turned on before a crash which killed the vehicle’s driver.Tesla Says Crashed Vehicle Had Been on Autopilot Before Fatal AccidentIndividual: Financial, Privacy & Autonomy; Societal: Equality
Data breachProvider of AI platform designed to assist school district may have been breached, resulting in leakage of children’s personal information.Supt. Carvalho moves ahead with troubled AI effort despite collapse of tech contractorIndividual: Financial, Privacy & Autonomy; Societal: Equality
PolicingSensitive information from automated license plate readers was improperly shared by law enforcement with out of state agengies.California cops illegally shared license plate details, violating privacy laws, grand jury saysIndividual: Privacy & Autonomy; Societal: Equality
Automated mail sortingAutomated mail sorting system slows down mail processing.Mail Carriers Protest Automation SystemIndividual: Financial; Societal: Equality
Facial recognitionLAPD uses AI technology from a company called BriefCam to review hours of surveillance footage to help authorities “quickly locate people.”Beverly Hills is always watching you with thousands of cameras. And the city isn’t doneIndividual: Privacy & Autonomy, Discrimination; Societal: Equality
Facial recognitionSFPD is being sued for alleged use of facial recognition technology despite the technology being banned in San Francisco.SFPD allegedly violates facial recognition technology ban: suitIndividual: Privacy & Autonomy, Discrimination; Societal: Equality
HiringJudge declines to dismiss employment discrimination claims against Workday for bias in Workday’s AI screening tool. The suit claims that Workday disproportionately weeds out people of color, disabled people, and people over 40.AI vendor with biased algorithm may be liable for employment discriminationIndividual: Privacy & Autonomy, Discrimination; Societal: Equality
Age estimationEven developers’ own whitepapers note that age estimation technology is less accurate for women and people of color.Instagram is testing an AI tool that verifies your age by scanning your faceIndividual: Privacy & Autonomy, Discrimination; Societal: Equality, Democracy & Rule of Law
HealthcareThe California Department of Justice launched investigaitons into racial disparities arising from use of algorithms to help make decisions about patient care.California investigating racial bias in healthcare algorithmsIndividual: Privacy & Autonomy, Discrimination; Societal: Equality
NewsDifficulties optimizing journalism content for search or social media prioritization algorithms have contributed to the decline of California’s news industry.California’s news industry is shrinking while misinformation spreads. Here’s what the numbers tell usIndividual: Financial; Societal: Equality, Democracy & Rule of Law
EntertainmentStudios’ vague contractual provisions create fear that technology will be used to replicate actors’ voices and bodies, supplanting their labor.California lawmakers want to protect actors from being replaced by artificial intelligenceIndividual: Financial, Privacy & Autonomy; Societal: Equality
Viral crime statisticsSocial media recommendation algorithms perpetuate fears about crime, even as incidence of criminal activity falls.Crime is down, but fear is up: Why is L.A. still perceived as dangerous?Individual: Financial, Privacy & Autonomy; Societal: Equality, Democracy & Rule of Law
Election misinformationAI chatbots often provide false information in response to questions about voting and the 2024 election.AI chatbots got questions about the 2024 election wrong 27% of the time, study findsIndividual: Financial, Privacy & Autonomy; Societal: Equality, Democracy & Rule of Law

Support Our Work

EPIC's work is funded by the support of individuals like you, who allow us to continue to protect privacy, open government, and democratic values in the information age.

Donate