< Previous10 PERSPECTIVESVOLUME 46, NUMBER 1 Automated Breathalyzer Kiosk Technologies David Kreitzer General Manager 2855 Country Drive, Suite 100 Little Canada, MN 55117 Phone: (651) 383.1213 Performance-Based Standards Learning Institute Kim Godfrey Lovett Executive Director 350 Granite Street, Suite 1203 Braintree, MA 02184 National Curriculum and Training Institute Gary Bushkin, President 319 East McDowell Road, Suite 200 Phoenix, AZ 85004-1534 Phone: (602) 252.3100 Northpointe Chris Kamin, General Manager Equivant Office: (608) 416-4302 Mobile: (608) 577-1755 Promise Diana Frappier, Chief Legal Officer 436 14th Street, Ste 920 Oakland, CA 94612 Phone: (415) 305.4560 Reconnect, Inc Sam Hotchkiss, Founder & CEO 1 Faraday Drive Cumberland, Maine 04021 RemoteCOM Robert Rosenbusch, President/CEO 2251 Double Creek Dr. Suite 404 Round Rock, TX 78664 Phone: (866)776-0731 SCRAM +LifeSafer Jed Rosenberg, Senior Marketing Manager Scram Systems 1241 West Mineral Avenue Littleton, CO 80120 Phone: (720) 261-6576 Securus Technologies Katie Desir, Marketing Manager Securus Monitoring Solutions 5353 W Sam Houston Parkway N, Suite 190 Houston, TX 77041 Phone: 773.658.0028 Shadowtrack Robert L. Magaletta Shadowtrack Technologies, Inc. ONE LAKEWAY 3900 North Causeway Boulevard Suite 1200, Metairie, LA 70002 Office: 985-867-3771 Smart Start, Inc. Michelle H. Whitaker Conference and Promotions Coordinator 500 East Dallas Road,Grapevine, TX 76051 Phone: (919) 604.2513 Track Group Matthew Swando, VP of Sales and Marketing 1215 North Lakeview Court Romeoville, IL 60446 Phone: (877) 260.2010 TRACKtech Ben Williams, Vice President - Business Development 6295 Greenwood Plaza Blvd, Suite 100 Greenwood Village, CO 80111 Phone: (303) 834-7519 Tyler Technologies Larry Stanton Director of Sales - Courts & Justice 5101 Tennyson Parkway Plano, TX 75024 Phone: (904) 654.3741 Uptrust Leo Scott, Program Manager 1 Sutter Street, Suite 350 San Francisco, CA 94104 765-469-1593 Website: Vant4ge Sean Hosman National Sales Leader – Public Sector Vant4ge P.O. Box 802, Salt Lake City, UT 84110 Phone: (877) 744-1360 appa associate members appa corporate members cont. THE FUTURE IS NOW: KRISTOFER BRET BUCKLEN, PH.D. PENNSYLVANIA DEPARTMENT OF CORRECTIONS GRANT DUWE, PH.D. MINNESOTA DEPARTMENT OF CORRECTIONS FAYE S. TAXMAN, PH.D. GEORGE MASON UNIVERSITY Establishing State of the Art Standards in Risk and Needs Assessment13 AMERICAN PROBATION AND PAROLE ASSOCIATION RISK NEED ASSESSMENT THE FUTURE IS NOW: ESTABLISHING STATE OF THE ART STANDARDS IN RISK NEEDS ASSESSMENT The use of risk and needs assessment (RNA) tools is now ubiquitous in the field of corrections. These tools are used to forecast an individual’s likelihood of re- offending for such purposes as developing treatment recommendations and setting appropriate supervision levels. Unfortunately, the field of corrections lags behind in technological advances and is overdue for establishing new methods and adopting new standards when it comes to the use of RNA tools. Current RNA practices, both in terms of development and implementation, are often outdated and inefficient and demonstrate suboptimal predictive validity. In an effort to help the field of corrections realize the potential that RNA tools have for improving decision- making and reducing recidivism, we recently prepared a commissioned paper for the National Institute of Justice (NIJ) in which we outlined four principles that must be addressed (Bucklen et al., 2021). Moreover, we drew upon our collective experience to offer guidelines and recommendations in each of these four areas that we hope will become the foundation for new industry standards for RNA use in corrections. While these guidelines draw on an evidence-based approach to RNA, they are relatively novel and innovative recommendations, at least in the field of corrections. We recognize that it may be difficult for correctional organizations and other agencies to meet all of these recommended standards. It is not our intention to outline a set of minimum standards for RNA practice, but rather to lay out optimal (“gold”) standards for achieving maximum benefit in the use of RNA. As such, we hope these recommendations become an aspirational playbook for correctional jurisdictions as they update their RNA practices over time. The full paper provides a checklist to guide in the development and implementation of RNA. We summarize these recommendations below. Principle #1: Fairness The fairness principle holds that RNA tools should be used to yield more equitable outcomes. Recently, RNA tools have been characterized by some as biased, usually in reference to racial or gender groups (Pretrial Justice Institute, 2020). An important question that should be raised, however, is “biased as compared to what?” The alternative to RNA tools would be individuals relying solely on their own subjective judgment. With the field of psychology recognizing over 175 different types of cognitive biases affecting human decision- making (Benson, 2016), it becomes easier to see how the structured use of information through RNA tools can actually reduce bias. We believe that RNA tools can be powerful instruments for reform that help correctional systems achieve more equitable outcomes. However, this potential can only be realized if RNA tools are both properly designed and properly used. One difficulty in approaching the subject of fairness within RNA tools is understanding how “fairness” is defined. Statisticians have identified at least six different definitions of fairness as it relates to RNA tools (Berk et al., 2018). “Overall accuracy” is defined as equal model accuracy between each class (e.g., between races) but does not distinguish between false positives and false negatives. “Statistical parity” is defined as equal marginal distributions of the predicted outcome for each class (e.g., the fraction of black parolees forecasted to recidivate is equal to the fraction of white parolees forecasted to recidivate). “Conditional procedure accuracy” is defined as equal false negative and false positive rates between each class (i.e., equal errors conditioned on the actual outcome). “Conditional use accuracy” is defined as equal positive predictive values and negative predictive values between each class (i.e., equal errors conditioned on the predicted outcome). “Treatment equality” is defined as an equal ratio of false negatives to false positives between each class. Finally, “total fairness” is defined as a situation in which all of the above conditions are met simultaneously. As Berk and colleagues point out, trade-offs are inevitable here, since “total fairness” (i.e., the situation in which all five other measures of fairness are met) is statistically impossible in a situation where different classes have different base rates of the outcome predicted (which is almost always the case in criminal justice). Furthermore, there is often a trade-off between fairness (however defined) and accuracy that must be wrestled with by policy-makers in designing and implementing an RNA tool. Researchers have identified three stages of RNA tool development where strategies can be used to improve fairness, once definitions of fairness are settled on: (a) preprocessing, (b) in-processing, and (c) post-14 PERSPECTIVESVOLUME 46, NUMBER 2 RISK NEED ASSESSMENT processing (Romei & Ruggieri, 2013). Preprocessing requires assessing the source data for various types of biases that might exist due to how the data are collected, stored, measured, and generally reported. In-processing refers to building adjustments in the algorithms and/or classification procedures during RNA development in order to account for any biases that might occur. Post- processing involves making adjustments to the algorithms after they have been created to further reduce biases that may still exist. Although a more detailed discussion of the specific innovative strategies within each of these three phases of development for addressing bias is outside of the scope of this paper, it can be found in the NIJ paper on this topic (Bucklen et al., 2021).`` One final note on this fairness principle is that RNA tools must be properly used in addition to being properly designed in order to maximize fairness and reduce bias. Disparities can be reduced through the way in which practitioners use RNAs, such as delivering more programming resources to those who need it the most (the risk principle) and ensuring proper training and auditing on the use of the tool. Principle #2: Efficiency This principle addresses the need for RNA tools to utilize processes that promote reliability, expand assessment capacity, and do not overburden staff. The vast majority of RNAs rely on time-consuming, cumbersome processes that mimic paper and pencil instruments; that is, forms must be completed and then manually scored by staff. The efficiency of RNA tools can be improved by adopting automated and computer- assisted scoring processes to increase reliability, validity, and assessment capacity. Automated assessments reduce staff time required to pull together data. By reducing the time spent reviewing records or filling out forms, automation increases the capacity to do better assessments through quality assurance. Automation can improve predictive performance by eliminating inter- rater problems and scoring interpretation issues (Duwe & Rocque, 2017). Automation may also address Principle #1 in that it has been demonstrated through previous research to reduce bias such as racial disparities (Duwe & Rocque, 2019). It is important for RNA tool developers to maximize the use of automation to the fullest degree possible. If auto-scoring or automation is not feasible, then the use of technology such as computer-assisted survey software should be considered. If an RNA must be scored manually, then it is also necessary to demonstrate through an inter-rater reliability (IRR) assessment that the tool can be scored consistently. Principle #3: Effectiveness It is not enough for RNA tools to be fair and efficient. They also need to be effective, which is the third key principle. The use of algorithms, even very simple ones, has long been shown to be more accurate in making predictions than professional judgment or “gut instincts.” Moreover, contemporary algorithms, such as machine learning techniques, have been shown to outperform older, much simpler algorithms used in the development of many previous and existing RNA tools. Machine learning approaches differ from earlier statistical approaches in that they are not based on a parametric model that is imposed on the data in advance. Instead, the data itself inductively determine the structure of the model. These newer algorithms can “squeeze more juice” out of the data. Examples of machine learning algorithms include classification and regression trees, k-means clustering, Bayesian networks, artificial neural networks, support vector machines, and ensemble methods like random forests and stochastic gradient boosting. A commonly agreed metric should be utilized for determining the comparative effectiveness (i.e., predictive accuracy) in RNA tool development. The most common metric is the Receiver Operating Characteristic Area Under the Curve, or AUC. An AUC value ranges from 0 to 1, with the worst possible score being 0.5 and the best possible score being either a 1 or a 0. One practical way to interpret an AUC score is the percentage of the time that a recidivist scores higher than a non-recidivist on an RNA tool (assuming a higher score means a higher likelihood of recidivism). A score of 0.5 is like flipping a coin; you have a 50% chance of being right. On the other hand, a score of 1 means that 100% of the time you are right in your prediction (i.e., perfect predictive accuracy). Standards for an acceptable AUC value on an RNA tool are changing as technology advances and new best practices in RNA development are implemented. One review of several popular RNA tools found that published AUC scores averaged 0.68 across tools (U.S. Department of Justice, 2019). However, more recent and improved instruments are consistently producing AUC scores well above 0.7. Scores between 0.65 and 0.7 were once considered acceptable but may become unacceptably low as new standards and improvements in RNA are adopted. 15 AMERICAN PROBATION AND PAROLE ASSOCIATION RISK NEED ASSESSMENT On the topic of metrics for comparing effectiveness of RNA tools, it is also important to point out that every tool has two types of errors: false positives and false negatives. A false positive is a situation in which an individual forecasted to recidivate actually turns out not to recidivate. A false negative is a situation in which an individual forecasted not to recidivate actually turns out to recidivate. The AUC metric represents a compilation of false positive and false negative errors. It is often important and advisable to examine each type of error separately, since there is an inevitable trade-off between these two types of errors, and decision-makers may place more or less emphasis on one type of error over the other. Another important recommendation under the effectiveness principle is that RNA tools should ideally be customized to be jurisdiction-specific by being developed using local data. Wholesale adoption of proprietary, “off- the-shelf” RNA tools should be a thing of the past. To use a sports analogy, research has established that there is a clear “home field advantage” to creating a jurisdiction- specific assessment tool (Duwe & Rocque, 2018). Principle 4: Communication The fourth key principle is that RNA tools should employ strategies that improve risk communication to decision-makers and to clients. Existing RNA tools focus on gathering information from individuals, but that information and what it means is often not effectively communicated back to staff or clients. The goal should be for the tool to facilitate information sharing to guide individual clients as to the actions they should take to improve outcomes. That is, the RNA tool administrator should share the information with the client and ensure that the client understands what the information is and why it is important. Explaining risk factors can be a tool for motivating individuals to change. This is similar to a credit report or other consumer device that communicates factors that drive risk. A good RNA tool should communicate and educate about risk factors and about the best path in order to make headway to reduce the risk of recidivism. This connection between assessment and case planning is often referred to as a 4th generation RNA tool. Effective training of correctional staff who will be using the RNA tool is essential for effective communication, particularly in ensuring ability to explain risks and needs and how to translate them into a case plan. A risk communication system, which includes case plan improvements, treatment-matching algorithms, and graduated sanctions and incentives, provides an integrated model for decision-making that helps increase awareness in clients of their own circumstances and need for programming. Transmitting information effectively is also key to risk communication. Increasingly, information can be shared visually instead of using words to describe a problem. The surge in web-based therapies, cell phones, and apps increases enthusiasm for visual messaging and multimedia messages. Using effective graphics to convey what a risk score means, for example, has the potential to greatly enhance communication. Conclusion Collectively, it is time for the field of corrections to modernize methods for developing and implementing RNA tools. Antiquated tools are burdensome to our agencies, are inefficient, and produce less reliable and valid assessments of individuals. An investment in new and improved methods which have proven to be effective is critical to ensure maximal use of RNA tools and to ensure that the information is truly valuable to correctional agencies. The four principles described in this paper–fairness, efficiency, effectiveness, and communication–are important for improving the use of RNA. At the same time, some of the principles and guidelines outlined here represent a relatively innovative approach to the design and use of RNA tools in corrections. We are not aware of any jurisdiction that has applied all of the guidelines and recommendations included within these four principles. What we have set out to do here is to outline the state of the science in RNA development and application with the goal of providing a blueprint and a goal for correctional jurisdictions to aspire to. We believe that adoption of these principles can help RNA tools mitigate system disparities and achieve better recidivism outcomes. 16 PERSPECTIVESVOLUME 46, NUMBER 2 RISK NEED ASSESSMENT References Benson, B. (2016). Cognitive bias cheat sheet. Berk, R., Hidari, H., Jabbari, S., Kearns, M., & Roth, A. (2018). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 1-42. doi: 10.1177/0049124118782533. Bucklen, K., Duwe, G., & Taxman, F. (2021). Guidelines for Post-Sentencing Risk Assessment. Washington, DC: U.S. Department of Justice, National Institute of Justice. Duwe, G., & Rocque, M. (2017). The effects of automating recidivism risk assessment on reliability, predictive validity, and return on investment (ROI). Criminology & Public Policy, 16: 235-269. Duwe, G., & Rocque, M. (2018). The home-field advantage and the perils of professional judgment: Evaluating the performance of the Static-99R and the MnSOST-3 in predicting sexual recidivism. Law and Human Behavior, 42(3): 269-279. Duwe, G., & Rocque, M. (2019). The predictive performance of risk assessment in real life: An external validation of the MnSTARR. Corrections: Policy, Practice and Research. 10.1080/23774657.2019.1682952. Pretrial Justice Institute. (2020). Updated Position on Pretrial Risk Assessment Tools. Gaithersburg, MD: Pretrial Justice Institute. Romei, A., & Ruggieri, S. (2014). A multidisciplinary survey on discrimination analyses. The Knowledge Engineering Review, 29(5): 1-54. doi: U.S. Department of Justice, Office of the Attorney General. (2019). First Step Act of 2018: Risk and Needs Assessment. Washington, DC: U.S. Department of Justice, Office of the Attorney General. Author Bios: Kristofer “Bret” Bucklen, PhD, is the Director of Planning, Research, and Statistics for the Pennsylvania Department of Corrections. Prior to his current position, he was employed in the Pennsylvania Governor’s Office of Administration, where he worked on projects for the PA Board of Probation and Parole, the PA Department of Corrections, the PA State Police, the PA Commission on Crime and Delinquency, and the Justice Network (JNET) Project. Dr. Bucklen received his M.S. in Public Policy and Management from Carnegie Mellon University’s Heinz School of Public Policy, and his PhD in Criminology and Criminal Justice from the University of Maryland. Grant Duwe, PhD, is Research Director for the Minnesota Department of Corrections, where he develops assessment instruments, forecasts the state’s prison population, and conducts research studies and program evaluations. Dr. Duwe is the author of two books and more than 80 peer-reviewed publications on a wide variety of topics in corrections. He has served as a consultant to the federal government, think tanks, and academic institutions. Faye S. Taxman, PhD, is a University Professor in the Schar School of Policy and Government at George Mason University. Dr. Taxman is a health services criminologist who examine how organizational processes affect program and individual level outcomes. Her work focuses on effective supervision practices including implementation and uptake of innovations. She is currently the co-editor of Health & Justice and has published over 220 articles. She was past editor of Perspectives. MODERNIZING RISK AND NEEDS ASSESSMENTS SCORING BAYLEE ALLEN, M.S., ADDISON KOBIE, M.A. & ZACHARY HAMILTON, PH.D. to Improve Agency Outcomes NEBRASKA CENTER FOR JUSTICE RESEARCH (NCJR) UNIVERSITY OF NEBRASKA – OMAHA (UNO)19 AMERICAN PROBATION AND PAROLE ASSOCIATION RISK NEED ASSESSMENT MODERNIZING RISK AND NEEDS ASSESSMENTS SCORING TO IMPROVE AGENCY OUTCOMES Understanding an individual’s characteristics and programming needs is one of the key principles for rehabilitation (Bonta & Andrews, 2016). Case management is built upon the concept that individuals often need different types of interventions. However, given the large size of many correctional populations, it is difficult to individually create consistent and detailed rehabilitation plans within a reasonable time frame and budget. Nearly all contemporary risk and needs assessment tools have been modernized in the last decade to provide software-generated reports and case management recommendations. With the expanded use of risk and needs assessments, agencies now possess the ability to identify those with the greatest need for programming, maximizing limited resources (Bonta & Andrews, 2016; Sullivan & Childs, 2021). Further, by using the results of these tools within the risk-needs-responsivity (RNR) model, all agencies, whether they represent small jurisdictions or supervise a statewide population, now have the ability to provide more standardized programming and service recommendations. Guided by research evidence, these tools allow staff to feel a greater sense of confidence in their case management recommendations, using programming and referrals to strategically reduce an individual’s recidivism risk. While far from perfect, needs assessment tools can help guide and standardize decisions at different stages in the justice process, increasing the opportunity for successful outcomes. In this article, we first discuss the development of modern risk assessment and the different characteristics that make these assessments powerful tools for justice stakeholders. Next, we introduce the infrequently discussed distinction between risk versus needs assessments, how each is developed, and their intended use. We then describe contemporary concepts to be considered when implementing and validating needs assessments as part of case management and planning, including localization, gender responsivity, and considerations for youth. Finally, we provide an example application of the RNR model using the Modified Positive Achievement Change Tool (M-PACT), highlighting the need to update developed assessments to fit agency needs. The Development of Risk and Need Assessments To understand modern risk assessment, it is important to acknowledge the underlying principles on which most risk assessments are based. Andrews, Bonta, & Hoge’s (1990) seminal work outlined three principles of a Risk-Needs-Responsivity (RNR) model to increase the effectiveness of rehabilitation, given previous evidence that “some things work for some people.” First, the Risk Principle uses the findings of an assessment tool to identify those who are the highest priority for supervision and programming. This is often expressed as some variation of high, moderate, and low risk levels. Second, the Needs Principle outlines criminogenic areas to address, where directed interventions may reduce needs and an individual’s likelihood of recidivism. Third, the Responsivity Principle outlines barriers that reduce programming effectiveness if not accounted for or properly addressed (Andrews et al., 1990). Modern case management teams use the RNR model to organize their treatment plans, with risk assessments being the foundation. These newer assessment tools combine overall risk level and subdomain scores to create a wholistic report of each individual’s needs. Contemporary Risk-Needs Assessments Contemporary assessments, or those developed in the last 15 years, have utilized risk and needs in a combined sequence, where those scoring as higher risk are prioritized for specific interventions using needs scores. Within the items scored in an assessment tool, “needs items” are identified to be dynamic (or changeable over time) and predict recidivism, and these concepts can be combined and described as defined as “criminogenic” needs (Bonta & Andrews, 2016). Clusters of needs items form “domains” that produce sub-scores for targeted areas impacting recidivism. Ideally, evidenced- based practices are available to strategically reduce needs scores in these target areas and thus reduce the individual’s overall risk for reoffending. However, modernization and development have had the unintended consequence of merging the concepts of risk and needs in discussions of justice assessments, thereby losing the historical context and conceptual distinction. Next >