What’s in a name? A survey of strong regulatory definitions of automated decision-making systems

There is no consensus definition of automated decision-making and artificial intelligence. Hype-filled marketing expanded what many people think of when they hear “AI,” and exacerbated concerns about automation stealing jobs or taking over like in iRobot.

However, with regulation and enforcement starting to take shape over AI and automated decision-making systems, it’s essential that we understand and build common definition(s) that can protect individuals today and in the future. EPIC believes an expansive definition of AI and automated decision-making is critical. Although it might require compliance from entities using simple tools, EPIC believes an expansive definition is essential to make the regulation meaningful. Just because a system is less complex does not mean it has a lesser capacity to be harmful or inaccurate.

Below are different types of definitions of automated decision-making systems proposed by scholars and government entities. These definitions complement EPIC’s Screened and Scored in DC report, which catalogues and analyzes uses of automated decision-making systems in the District of Columbia. In the report, we use an expansive interpretation of automated decision-making systems to fully capture the extent of it in a single jurisdiction.The report is available on November 1^st, 2022 at epic.org/ai/ScreenedinDC.

Definitions proposed in scholarship and outside of government:

• Defining and Demystifying Artificial Intelligence (Rashida Richardson) — “Automated Decision System” is any tool, software, system, process, function, program, method, model, and/or formula designed with or using computation to automate, analyze, aid, augment, and/or replace government decisions, judgments, and/or policy implementation. Automated decision systems impact opportunities, access, liberties, safety, rights, needs, behavior, residence, and/or status by prediction, scoring, analyzing, classifying, demarcating, recommending, allocating, listing, ranking, tracking, mapping, optimizing, imputing, inferring, labeling, identifying, clustering, excluding, simulating, modeling, assessing, merging, processing, aggregating, and/or calculating.
• Our Data Bodies: Digital Defense Playbook (Seeta Peña Gangadharan, Tawana Petty, Tamika Lewis, Mariella Saba)– “Algorithm” — A sequence of steps for solving a problem (e.g., like a recipe); in digital terms, a set of computational or mathematical formulas that use data as their main ingredient, transforming these data (input) into desired outputs.

Some bills define different types of automated decisions:

• D.C. Stop Discrimination by Algorithms Act
  • “Algorithmic eligibility determination” means a determination based in whole or in significant part on an algorithmic process that utilizes machine learning, artificial intelligence, or 16 similar techniques to determine an individual’s eligibility for, or opportunity to access, important life opportunities.”
  •  “Algorithmic information availability determination” means a determination based in whole or in significant part on an algorithmic process that utilizes machine learning, artificial intelligence, or similar techniques to determine an individual’s receipt of advertising, marketing, solicitations, or offers for an important life opportunity.”

• Washington State SB 5116
  • “Automated decision system” means any electronic software, system, or process designed to automate, aid, or replace a decision making process that impacts the welfare or rights of any Washington resident, and that would otherwise be performed by humans. Automated decision systems include, without limitation, tools that analyze data sets to generate scores, predictions, classifications, or recommended actions that are used by agencies to automate, aid, or replace decision-making processes that impact the welfare or rights of any Washington resident. Automated decision systems do not include tools that do not automate, aid, or replace such decision-making processes, including without limitation junk email filters, firewalls, antivirus – software, calculators, spreadsheets, databases, data sets, or other compilations of data.”
  • “Automated final decision system” means an automated decision
    system that makes final decisions, judgments, or conclusions without human intervention.
  • “Automated support decision system” means an automated
    decision system that provides information to inform the final
    decision, judgment, or conclusion of a human decisionmaker.

• EU AI ACT: Artificial Intelligence System’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;
  • ‘Emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric data
  • ‘Remote biometric identification system’ means an AI system for the purpose of identifying natural persons at a distance through the comparison of a person’s biometric data with the biometric data contained in a reference database, and without prior knowledge of the user of the AI system whether the person will be present and can be identified
  • ‘‘Real-time’ remote biometric identification system’ means a remote biometric identification system whereby the capturing of biometric data, the comparison and the identification all occur without a significant delay. This comprises not only instant identification, but also limited short delays in order to avoid circumvention
  • ‘Post’ remote biometric identification system’ means a remote biometric identification system other than a ‘real-time’ remote biometric identification system.

These definitions illustrate the variance in approaches to regulating AI and automated decision-making systems. As the regulatory examples show, there are many approaches that have not yet been tested, but it’s a strong practice to have a broad general definition and specify additional subsets of technologies that the legislation wants to address in a different way, such as bans.