Rodriguez v. Massachusetts Parole Board

Questions Presented

Whether it is proper for the Massachusetts Parole Board to deny parole to an applicant based, in part, on a high “risk score” generated by a third-party risk-assessment tool not designed to make accurate predictions about this class of applicants.

Whether it is proper for the Massachusetts Parole Board and MultiHealth Systems, Inc. (the developer of the tool) to shield key information needed to understand the tool from the public and parole applicants.

Background

Federal and state government use of predictive analytical tools is growing: a February 2020 report found that nearly half of the 142 federal agencies studied had “experimented with AI and related machine learning tools.” The developers that make these tools promise that the tools make efficient, unbiased, data-informed decisions. As a result, they’re used in law enforcement, the broader criminal legal cycle, public benefit administration, housing processes, and more.

Risk-assessment tools are a particular type of predictive analytical tool used in the criminal legal system. They are designed to attempt to predict future behavior by defendants and incarcerated persons and quantify that risk. They may use socioeconomic status, family background, neighborhood crime, employment status, and other factors to reach a supposed prediction of an individual’s criminal risk, either on a scale from “low” to “high” or with specific percentages. A 2020 EPIC report on risk assessments showed that nearly every U.S. state uses a risk assessment tool for one or more of pre-trial sentencing, sentencing, and parole.

Unfortunately, many risk-assessment tools are deeply flawed, and they struggle to bring the benefits they promise. Deploying these tools tends to exacerbate discriminatory policing patterns that already disadvantage minorities. For one example, a 2016 investigation by ProPublica tested the COMPAS system adopted by the state of Florida. ProPublica found that the tool’s scores were unreliable in forecasting violent crime: only 20 percent of the people predicted to commit violent crimes actually went on to do so. Meanwhile, the tool was particularly likely to flag black defendants as future criminals, labeling them as such at almost twice the rate as white defendants. In addition, white defendants were labeled as low risk more often than black defendants.

Compounding the accuracy and bias problems is the fact that many agencies feel the need to keep these tool’s accuracy rates, inner workings, and case-by-case determinations secret. Whether because of a desire to avoid accountability or an aversion to fighting third-party vendors’ trade secrecy claims, many agencies keep these tools’ workings secret from the public. Certain states have pending and enacted legislation that would improve transparency and accountability of these tools state-wide, such as an Idaho law requiring transparency in pretrial risk assessment tools for the public and defendants.

The Case

Like many government organizations, the Massachusetts Parole Board (“MPB”) has started using a predictive analytical tool from a third-party contractor. The MPB uses a tool called the Level of Service/Case Management Inventory (“LS/CMI”) to generate risk scores that purport to predict how likely a parole applicant is to end up back in prison.

The plaintiff-appellant in this case, Mr. Jose Rodriguez, has been consistently denied parole based on the high risk score that LS/CMI gives him. Mr. Rodriguez is a “juvenile lifer,” or a person sentenced to life in prison for a crime they committed while they were still a child. The U.S. Supreme Court has held that juvenile lifers cannot be incarcerated without the chance for parole, and the standard for parole is whether there is “a reasonable probability that, if such offender is released, the offender will live and remain at liberty without violating the law [such that] release is not incompatible with the welfare of society.” 120 Code Mass. Regs. § 300.04. See G.L. c. 127 § 130

Mr. Rodriguez is challenging, among other things, the Parole Board’s use of LS/CMI to give him a high risk score, and the Parole Board’s refusal to share any useful information about how his score was calculated. Mr. Rodriguez, who has been incarcerated for more than 35 years, has been a model inmate. He has received only 8 disciplinary tickets, none in the last fifteen years; completed his GED in 1991; and completed extensive programming aimed at addressing substance use disorders and anger management. Nonetheless, his risk score remains high and the reasons why largely kept secret.

EPIC’s Brief

EPIC filed an amicus brief arguing that the use of predictive analytical tools such as LS/CMI is dangerous and invalid unless the tools are used transparently and independently validated on each group of people for which the tool will make decisions.

EPIC argued that statistical principles call into question whether LS/CMI can ever make accurate predictions about juvenile lifers. LS/CMI was, to the public’s knowledge, not built using data that included juvenile lifers. Versions of LS/CMI that EPIC has accessed from other states show that the system considers factors to predict recidivism that have been proven irrelevant for juvenile lifers. These versions also show that LS/CMI ignores age, while advanced age is one of the strongest factors that predict someone will not recidivate. EPIC also showed that LS/CMI scored low on measures of accuracy for parole applicants in general—the group for which the system was created. Therefore, it is likely to perform even worse for predicting juvenile lifers’ chance of recidivism. EPIC additionally showed how these same concerns can lead these systems to produce results biased against racial and ethnic minorities.

EPIC also argued that these concerns about applicability and bias illustrate why agencies should not use predictive analytical tools in secret. The public and people who are subject to machine-informed decisions deserve to know how these decisions are made.

Legal Documents

Massachusetts Supreme Judicial Court (SJC-13197)

• Opinion (Sep. 6, 2022)
• EPIC’s Amicus Brief (Feb. 14, 2022)
• Plaintiff-Appellant’s Reply Brief (Feb. 2, 2022)
• Defendant-Appellee’s Response Brief (Jan. 18, 2022)
• Plaintiff-Appellant’s Brief (Nov. 23, 2021)

Middlesex County Superior Court (2021-P-0516)

• Plaintiff-Appellant’s Application for Direct Appellate Review (Aug. 17, 2021)