APA Comments

Comments of EPIC to the Department of Education on Seedlings to Scale

DOWNLOAD EPIC-Comment-DOE-Seedlings pdf 434.5KB

By notice published October 12, 2023, the Department of Education Requests information on “Potential New Program, from Seedlings to Scale.”1 The Electronic Privacy Information Center (“EPIC”) writes to encourage the Department to restrict funding for any projects that use affect recognition or one-to-many facial recognition and more generally limit funding only to projects that meet rigorous privacy and security standards. As a key driver of investment in and adoption of new technologies in the education space, the Department has an obligation to ensure that those systems do not cause harm to students. EPIC is concerned that deployment of data-extractive and mistake-prone AI systems in the education space could cause significant harm, and thus demands special safeguards.

EPIC is a public interest research center established in 1994 to secure the fundamental right to privacy in the digital age for all people through advocacy, research, and litigation. EPIC conducts research and advocates for common sense regulation concerning automated decision-making systems. On December 9, 2020, EPIC filed a complaint with the Office of the Attorney General for the District of Columbia alleging that five major providers of online test proctoring services have engaged in unfair and deceptive trade practices in violation of the D.C. Consumer Protection Procedures Act (DCCPPA) and the Federal Trade Commission Act. Specifically, EPIC’s complaint charges that Respondus, ProctorU, Proctorio, Examity, and Honorlock have engaged in excessive collection of students’ biometric and other personal data and have routinely relied on opaque, unproven, and potentially biased AI analysis to detect alleged signs of cheating.

As the Office of Educational Technology’s May 2023 report “Artificial Intelligence and the Future of Teaching and Learning” recommended, the department should “Focus R&D on Addressing Context and Enhancing Trust and Safety”[1] and create “education-specific guardrails.”[2] The Department has an opportunity to put those recommendations, along with recommendations that align with the recent AI Executive Order[3], into practice. With the proper safeguards, the funds from this program can help promote student learning, bridge gaps in education quality, and preserve privacy and civil liberties.

We believe that the Department of Education should establish safeguards that guide the development and deployment of any AI technologies funded by the Seedlings to Scale program Specifically, EPIC recommends that the Department of Education:

  • Only fund AI systems that use limited, curated datasets that are made public for accountability purposes. This will help reduce risks of false or inappropriate outcomes for a school setting, among other privacy violations.
  • Prohibit the funding of any emotion recognition or facial recognition tools.[4]
  • Require any system receiving funding to limit collection, processing, and/or transferring of the data of minors (anyone under the age of 18) to that which is strictly necessary to achieve the minor’s specific purpose for interacting with the business or strictly necessary to achieve certain essential purposes that provide a clear benefit to the minor.[5]
  • Require any developer or user of funded systems to ensure verifiable parental consent for data that is collected from minors under 13 years old, in compliance with COPPA).[6]
  • Prohibit the developer or user of funded technologies from selling or otherwise transferring any data collected from minors to data brokers.
  • Prohibit developer or user of funded technologies from embedding trackers into student web browsers, software, and/or devices.
  • Allow parent or student to delete children’s data from the technology at any time.[7]
  • Require the technology to have data retention and deletion policies in place to minimize the amount of data that is collected from minors is stored.
  • Require that the users of these technologies provide students and parents with an accurate list of what data it collects from students, when it collects that data, and in what manner it collects that data. This should be made in an active disclosure given to students and sent to parents, in addition to having this information embedded in the privacy policies.[8]
  • Require strong cybersecurity practices, including multifactor authentication, to safeguard children’s data.[9]

The use of automated systems in exam proctoring and cheating detection/suspicion applications has questionable accuracy and has caused significant harm to students. For further reading about potential negative impacts of the technologies EPIC is urging the department not to fund, EPIC recommends the following sources:

As detailed above, facial recognition and emotion recognition systems have been shown to have unavoidable discriminatory impacts, as well as limited utility. For resources on the major problems caused by the widespread use of facial recognition and emotion recognition systems:

Increased surveillance throughout the education ecosystem often interferes with learning and causes privacy and civil liberties concerns. For resources on discrimination, surveillance, and other harms of student monitoring software in general, including exam proctoring:

[1] Department of Education Office of Education Technology, Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations (May 2023), available at https://www2.ed.gov/documents/ai-report/ai-report.pdf at 59

[2] Id. At 60

[3] Executive Order 14110, Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 30, 2023), available at https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/.

[4] See, e.g., James Vincent, Discover the Stupidity of AI Emotion Recognition with This Little Browser Game, The Verge (Apr. 6, 2021), https://www.theverge.com/2021/4/6/22369698/ai-emotion-recognition-unscientific-emojify-web-browser-game; see also Kate Crawford, Artificial Intelligence is Misreading Human Emotion, The Atlantic (Apr. 27, 2021), https://www.theatlantic.com/technology/archive/2021/04/artificial-intelligence-misreading-human-emotion/618696/; Charlotte Gifford, The Problem with Emotion-Detection Technology, The New Economy (Jun. 15, 2020), https://www.theneweconomy.com/technology/the-problem-with-emotion-detection-technology; Crawford, supra note Error! Bookmark not defined.; Lauren Rhue, Emotion-Reading Tech Fails the Racial Bias Test, The Conversation (Jan. 3, 2019), https://theconversation.com/emotion-reading-tech-fails-the-racial-bias-test-108404; U.S. Gov’t Accountability Off., Facial Recognition Technology: Privacy and Accuracy Issues Related to Commercial Uses 6 (Jul. 2020), https://www.gao.gov/products/gao-20-522. [hereinafter GAO]; NIST Study Evaluates Effects of Race, Age, Sex on Face Recognition Software, Nat’l Inst. of Standards and Tech. (Dec. 19, 2019), https://www.nist.gov/news-events/news/2019/12/nist-study-evaluates-effects-race-age-sex-face-recognition-software.; Larry Hardesty, Study finds gender and skin-type bias in commercial artificial-intelligence systems, MIT News (Feb. 11, 2018), https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212; Erik Learned-Miller et al., Facial Recognition Technologies in the Wild: A Call for Federal Office, Algorithmic Justice League 7 (May 29, 2020), https://assets.website-files.com/5e027ca188c99e3515b404b7/5ed1145952bc185203f3d009_FRTsFederalOfficeMay2020.pdf; Kashmir Hill, Microsoft Plans to Eliminate Face Analysis Tools in Push for ‘Responsible AI,’ N.Y. Times (Jun. 21, 2022), https://www.nytimes.com/2022/06/21/technology/microsoft-facial-recognition.html; Kashmir Hill & Ryan Mac, Facebook, Citing Societal Concerns, Plans to Shut Down Facial Recognition System, N.Y. Times (Nov. 5, 2021), https://www.nytimes.com/2021/11/02/technology/facebook-facial-recognition.html; Kate Crawford, Time to Regulate AI That Interprets Human Emotions, Nature (Apr. 6, 2021), https://www.nature.com/articles/d41586-021-00868-5

[5] See e.g. CCPA § 1798.121(a) (enumerating the purposes for which California consumers can ask a business to limit the use of their sensitive personal data to); see also Disrupting Data Abuse: Protecting Consumers from Commercial Surveillance in the Online Ecosystem, EPIC (Nov. 2022), 167-181  https://epic.org/wp-content/uploads/2022/12/EPIC-FTC-commercial-surveillance-ANPRM-comments-Nov2022.pdf 

[6] Children’s Online Privacy Protection Rule (COPPA) 16 C.F.R. § 312.3(b); 16 C.F.R. § 312.5

[7] 16 C.F.R. § 312.3(c); 16 C.F.R. § 312.6

[8] 16 C.F.R. § 312.3(c); 16 C.F.R. § 312.6

[9] 16 C.F.R. § 312.3(e); 16 C.F.R. § 312.8

News

EPIC Releases its AI Legislation Scorecard

June 25, 2024

At White House AI Aspirations Conference, EPIC Urges Agencies Considering AI to Center Public Trust

June 20, 2024

PRESS RELEASE: EPIC Launches AI Legislation Scorecard with Virtual Panel Discussion 

June 13, 2024

See All News

EPIC Releases its AI Legislation Scorecard

June 25, 2024

See All News

Support Our Work

EPIC's work is funded by the support of individuals like you, who allow us to continue to protect privacy, open government, and democratic values in the information age.

Donate