Oregon Youth Authority Audit Report on Information Systems 2009
Download original document:
Document text
Document text
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
Report No. 2009-11 May 7, 2009 State of Oregon Oregon Youth Authority: Improvements Needed in Availability and Reliability of Critical Juvenile Justice Information Table of Contents Page Summary ............................................................................................................. 3 Background ......................................................................................................... 5 Audit Results ....................................................................................................... 9 Recommendations............................................................................................... 14 Objectives, Scope and Methodology .................................................................. 15 1 2 Summary Oregon’s juvenile justice system depends on the work of the Oregon Youth Authority and county juvenile departments to promote effectiveness, accountability, and transparency in the juvenile justice system. To support the goals of the juvenile justice system, the Youth Authority implemented an information system to help decision makers plan and evaluate programs to reduce juvenile crime. We reviewed this information system and its use in four Oregon counties. We found that, although the system captures useful information, publicly available reports were limited and did not answer many important questions about the juvenile justice system. Because this information is important for informing policy discussions, this lack of publicly available reports is significant. In addition, although the majority of the data we reviewed in the system was reliable, key data on offender requirements, such as restitution and community service, was not consistently reliable. As a result, the Youth Authority, county juvenile departments, and the public lack information useful to promote effectiveness, accountability, and transparency in the juvenile justice system. As part of our work, we answered a number of key questions from stakeholders in the juvenile justice system. We present this information in four appendices, one for each of the counties we reviewed–Marion, Polk, Tillamook, and Wasco. We recommend that the Youth Authority: • Continue efforts to develop and publish reports that provide useful information for decision makers, managers, and the public to promote effectiveness, accountability, and transparency in the juvenile justice system. Consider obtaining input from various government and public stakeholders about what types of reports would be useful to them. • In consultation with partner agencies, take steps to ensure the juvenile justice data the Youth Authority and county juvenile departments enter are consistent and reliable. The Oregon Youth Authority generally concurs with the findings and generally agrees with the recommendations. The agency’s response can be found beginning on page 14. 3 4 Background Oregon’s juvenile justice system depends on the work of the Youth Authority and county juvenile departments to promote effectiveness, accountability, and transparency in the juvenile justice system. The purposes of the juvenile justice system are to protect the public, reduce juvenile delinquency, and provide fair and impartial procedures for the handling of delinquent conduct. The system is also intended to be open and accountable to the people of Oregon. Government and Public Stakeholders are Interested in Juvenile Justice Information The Youth Authority, county juvenile departments, state agencies, federal agencies, and the public are interested in juvenile justice information for various purposes. For example, the Youth Authority uses juvenile justice information to provide comprehensive support for managing individual juvenile offender cases and tracking juveniles through the juvenile justice process. Counties use information for making decisions on resource allocation and case management. The U.S. Department of Justice archives national juvenile court data to promote access to data for conducting juvenile justice research and policymaking efforts. Local public advocacy groups regard information as important for informing the public and for evaluating juvenile justice programs, policies, and services. Youth Authority Implemented a Statewide Juvenile Justice Information System The Youth Authority implemented a shared statewide electronic juvenile justice information system designed to centrally track and provide information about youth in the juvenile justice system across state, county, and local agencies. The information system helps decision makers plan, develop, and evaluate programs to reduce juvenile crime. It also supports the common needs of juvenile justice partner agencies. The Youth Authority assigned project oversight for the information system to a steering committee, which is a collaborative partnership comprised of representatives of the Youth Authority, county juvenile departments, and external partners. The Youth Authority produces annual statewide juvenile justice information reports through the Data and Evaluation Advisory Committee, a steering committee subcommittee. 5 Background Youth Authority and Counties Capture Juvenile Justice Information The Youth Authority and county juvenile departments are responsible for capturing juvenile offender information. The Youth Authority enters information about youth committed to its custody, and counties enter information throughout the county juvenile justice process. Key county juvenile justice processes reflected in the information system are described below. • Referrals are received by the juvenile department: A referral is any allegation or group of allegations that are documented by a police report or other formal means. For the purposes of our report, we classified allegations into three categories: non-criminal, misdemeanor, and felony. Non-criminal offenses include status offenses, which are age-related offenses. Non-criminal offenses also include local ordinance violations and traffic violations. Misdemeanor and felony offenses are criminal offenses. • The intake process leads to initial referral decisions: During intake, the juvenile department or district attorney determines the initial handling of the referral (e.g. dismissal, diversion, or referral to another agency).1 Alternatively, the county could file a court petition or the youth could enter into a formal accountability agreement. A formal accountability agreement is a voluntary contract between a youth and a juvenile department whereby the youth agrees to fulfill specified conditions in exchange for not having a petition filed. • A petition starts the formal juvenile court process: A petition is an application for a court order alleging a youth is within the jurisdiction of the court. If jurisdiction is established, the court orders a formal disposition. There are various formal dispositions including detention, probation, or commitment to a youth correctional facility. The court may also dismiss the case. • Offenders may be assigned conditions while under supervision: Conditions are requirements, ordered or directed by the court, juvenile department, or Youth Authority. 1 For example, the youth could be referred to the U.S. Citizenship and Immigration Services. 6 Background Youth offenders must complete or observe conditions during time under supervision (e.g. probation, formal accountability agreement, etc.). Types of conditions include community service, restitution, and program intervention. A program intervention is an action, activity, or treatment designed to change the youth’s behavior. 7 8 Audit Results We found that, although the juvenile justice information system captures useful information, publicly available reports were limited and did not answer many important questions about the juvenile justice system. Because this information is important for informing policy discussions, this lack of publicly available reports is significant. Although the majority of the data we reviewed in the system was reliable, key data on offender requirements was not consistently reliable. As a result, the Youth Authority, county juvenile departments, and the public lack useful information to promote effectiveness, accountability, and transparency in the juvenile justice system. Publicly Available Reports Have Increased, But Reported Information Was Still Limited In 2005, we audited the juvenile justice information system and found that information on data variations and trends available in published reports was limited. Since 2005, the Youth Authority has increased the number of publicly available reports, including publishing some useful statewide, county, and trend reports. Currently the Youth Authority produces various reports on recidivism, referrals, dispositions, detention, restitution, and community service. Many of these reports organize the data by gender, age, and race/ethnicity. In addition, the Youth Authority is planning to develop reports on services and programs, which include treatment. Although the Youth Authority increased the number of reports it makes available, reports were still limited and did not answer many important questions about the juvenile justice system. For example, our appendices in this report present information not currently available in a published report. Additional types of information provided in the appendices include, but are not limited to: • prior referral history; • elapsed time for key processes; • referral handling; and • program intervention information. The four tables below are examples of information we present in the appendices that the Youth Authority does not currently report. We chose one table for each of the four counties. Table I presents prior referral history for Marion County youth with referrals that closed in 2006. 9 Audit Results Table I: Number of Youth by Prior Referral Category Based on the Most Recent Referral to Close during Calendar Year 2006 (Marion County Appendix Table 5)2 Prior Referral Category No Prior Referrals Prior Non-Criminal Referrals Only Prior Criminal Referrals Only Both Prior Criminal and Non-Criminal Referrals Total Number of Youth 1427 331 626 779 3163 Percent of Youth 45% 10% 20% 25% 100% Table II shows the elapsed time from receipt of referral to referral closure for Wasco County referrals that closed in 2006. The length of time a referral is open is based on actions taken by the youths, juvenile department, district attorney, and courts. It includes the length of time a youth is on probation. In some cases, referrals may be handled informally initially, but later handled in court. Table II: Distribution of Elapsed Time from Receipt of Referral to Referral Closure (Wasco County Appendix Table 2) 3 Elapsed Time (Days) Less than 30 30 – 59 60 – 89 90 – 179 180 – 364 365 – 730 More than 730 Total Number of Referrals 173 28 28 57 60 42 34 422 Percent of Referrals 41% 7% 7% 14% 14% 10% 8% 100% Table III shows the number of referrals categorized by initial referral handling for Tillamook County referrals that closed in 2006. 2 3 We based our prior referral analysis on the day referrals were received. Table II excludes eight interstate compact referrals. Percentages may not sum to 100 percent due to rounding. 10 Audit Results Table III: Number of Referrals Classified by Initial Referral Handling Category (Tillamook County Appendix Table 14)4 Number of Referrals 121 23 14 38 21 138 355 Initial Referral Handling Category Dismissed Consolidated Referred to Another Agency Other Diversion Formal Accountability Agreement Petition Filed Total Percent of Referrals 34% 6% 4% 11% 6% 39% 100% Table IV provides information about Polk County adjudicated referrals with ordered program intervention conditions categorized by allegation type. It shows the percentage of referrals with program interventions completely met, partially met, not met, and vacated or replaced. Table IV: The Number of Referrals with Program Intervention Conditions Ordered and the Percentage of Referrals by Completion Category Classified by Disposed Allegation Type (Polk County Appendix Table 38c) Disposed Allegation Type Non-Criminal Misdemeanor Total Felony Felony Person Felony Weapons Felony Substance/ Alcohol Felony Property Felony Public Order Felony Other All Referrals w/ Program Interventions 4 Number of Referrals w/ Program Interventions Ordered 18 46 22 4 1 86 Percent of Referrals w/ Program Interventions Completely Met 78% 63% 55% 75% 100% Percent of Referrals w/ Program Interventions Partially Met 0% 28% 18% 0% 0% Percent of Referrals w/ Program Interventions Not Met 22% 9% 27% 25% 0% Percent of Referrals w/ Program Interventions Vacated or Replaced 0% 0% 0% 0% 0% 4 75% 25% 0% 0% 11 1 1 27% 100% 100% 27% 0% 0% 45% 0% 0% 0% 0% 0% 64% 20% 16% 0% This table excludes three interstate compact referrals. The consolidated category denotes referrals that were dealt with through another charge or were handled as parole or probation violations. 11 Audit Results Existing Information Could Answer Most Stakeholder Questions We determined that the juvenile justice information system captures useful information. Specifically, we identified 46 questions based on input from stakeholders in the juvenile justice system and found that information was generally available to answer these key stakeholder questions. The parties we interviewed included the Office of the Governor, the Juvenile Court Improvement Project, the Criminal Justice Commission, Crime Victims United of Oregon, and the Juvenile Rights Project. We found that data was available to answer 44 of these questions. We have presented the data for these 44 questions in the attached appendices. We were not able to answer the following two stakeholder questions because partner agencies did not track the start date for treatment conditions during the audit period: • For referrals with treatment ordered as a condition of accountability agreements, what is the distribution of elapsed time from the signing of an accountability agreement to the treatment start date? • For adjudicated referrals with treatment ordered as a condition, what is the distribution of elapsed time from the disposition date to the treatment start date? Treatment start date was not a required data element during our audit period. However, the steering committee is in the process of implementing a new policy that requires counties to begin tracking program and service data, including the start date of treatment. Tracking program and service data is now required for programs and services funded by state basic services funds and diversion funds. Some Critical Information Was Not Consistently Reliable Reliable condition data is important for providing consistent and comparable information that can be used to evaluate the effectiveness and efficiency of specific practices designed to change youth behavior and facilitate reparations to victims and the community. Using data that is unreliable could lead to inaccurate conclusions or inappropriate comparisons, and could impair effectiveness, accountability, and transparency in the juvenile justice system. Although the majority of the data we reviewed in the system was reliable, we found that key data on conditions (offender requirements) was not consistently reliable for the four counties we reviewed. Specifically, we reviewed the accuracy of 31 data fields and determined that, with the exception of three condition fields, the data was sufficiently reliable.5 At each county, 5 The three condition fields were Total Ordered, Total Complete, and Condition Status. 12 Audit Results we found that three condition fields either did not always agree with supporting documentation or lacked supporting documentation necessary to verify condition data. We excluded condition data that we determined to be unreliable from the appendices that follow. In addition to testing data fields for accuracy, we tested the completeness of condition data. In 2005, we reviewed the information system and found that restitution, community service, and treatment data were not consistently captured (Report No. 2005-21). Although we found improvement in the completeness of condition data, we also found instances during the current audit period in which condition data was not entered in the system. We also found that the Youth Authority does not verify the entry of condition data and has few policies focused on accuracy and completeness of data. In addition, during our review of Youth Authority policy, we found that entering condition data was not mandatory, with the exception of counties that volunteer to participate in annual reporting of community service and restitution data. However, the entry of program and service condition data for specific grant funds became mandatory in 2008. 13 Recommendations We recommend that the Youth Authority: • Continue efforts to develop and publish reports that provide useful information for decision makers, managers, and the public to promote effectiveness, accountability, and transparency in the juvenile justice system. Consider obtaining input from various government and public stakeholders about what types of reports would be useful to them. • In consultation with partner agencies, take steps to ensure the juvenile justice data the Youth Authority and county juvenile departments enter are consistent and reliable. Agency’s Response: The Oregon Youth Authority generally concurs with the findings in the audit and generally agrees with the recommendations. Our comments include input from two of the four counties: Recommendation: Continue efforts to develop and publish reports that provide useful information for decision makers, managers, and the public to promote effectiveness, accountability, and transparency in the juvenile justice system. Consider obtaining input from various government and public stakeholders about what types of reports would be useful to them. The Oregon Youth Authority agrees with the recommendation and believes this is a worthy goal to continue to strive towards. The agency already invests heavily in these efforts through the ongoing work of the JJIS Steering Committee, JJIS Data and Evaluation, JJIS Reports Committee, JJIS Policy and Standards, and the JJIS Implementation Coordinators. All these committees are a partnership between the counties and OYA. OYA will develop a process in which other stakeholders, who are not already part of the JJIS committees, can also contribute valuable feedback. Recommendation: In consultation with partner agencies, take steps to ensure the juvenile justice data the Youth Authority and county juvenile departments enter are consistent and reliable. The Oregon Youth Authority agrees it is important to assure the data entered into JJIS are consistent and reliable. The audit identified three data elements related to tracking conditions (offender requirements) which were considered not consistently reliable. Since 2006, OYA and the JJIS Steering Committee have taken steps to improve the consistency and reliability of condition data through policy clarification, development of data entry procedures, and the development of statewide and county specific information reports. OYA and the partner agencies will continue to improve consistency by developing one or more audit reports to help monitor the data elements found to be unreliable. The monitoring reports and associated documentation on how to use the reports will be provided to each partner agency. 14 Objectives, Scope and Methodology We conducted this audit to meet the requirements of House Bill 3420, passed in 2007. The bill directs the Secretary of State to audit the performance of at least four county juvenile departments during each of the 2007-09 and 2009-11 biennia. This report, one of two that has resulted from our work performed during the 2007-09 biennium, addresses juvenile justice information. A prior report, which we released in December 2008, assessed whether County Juvenile Departments expended basic services and diversion funds in accordance with intergovernmental agreements (Report No. 2008-39). The purpose of this audit was to determine whether reliable data was available and could be used to answer key questions from stakeholders about the juvenile justice system at four selected Oregon counties. We judgmentally selected Marion County, Polk County, Tillamook County, and Wasco County to include in our review. We interviewed local stakeholders in the juvenile justice system to identify key questions they have about Oregon’s juvenile justice operations. The parties we interviewed included the Office of the Governor, the Juvenile Court Improvement Project, the Criminal Justice Commission, Crime Victims United of Oregon, and the Juvenile Rights Project. We used data we obtained from the juvenile justice information system as the basis for answering the stakeholder questions. For each selected county, we obtained queries from the Youth Authority. Our cohort population for each county was limited to youth with referrals that were within the counties’ jurisdiction and that closed in calendar year 2006. We chose closed referrals so that we could present information that tracked a referral through the system. We excluded expunged and dependency referral records. We chose to use referrals as the primary reporting unit for our analysis, based on input stakeholders provided. This is consistent with federal reporting of juvenile justice information. However, readers should exercise care when interpreting the information, as it may not account for all of the allegations within a referral, all of the referrals combined within a single disposition, or other aspects of a youth’s referral or disposition history. In order to gain a better understanding of the juvenile justice system and the information system, we obtained and reviewed Youth Authority policies and procedures, interviewed Youth Authority officials and staff, and interviewed county juvenile department directors and staff. We reviewed Youth Authority reports published online, the information system record layout and data dictionary, policies and procedures, and steering committee and subcommittee meeting minutes. We developed and tested methodologies to extract, manipulate, and analyze the available data to answer the stakeholder questions. We present the answers to these questions in four appendices, one for each county we included in the audit–Marion, Polk, Tillamook, and Wasco. We shared our results with county staff, who generally agreed with the information and saw value in the expanded reporting of juvenile justice information. 15 Objectives, Scope and Methodology We conducted data reliability testing at each of the four counties we reviewed. We tested the accuracy of 31 data fields by comparing electronic data to supporting documentation on file at the counties. We also tested the completeness of condition data by comparing the electronic data to ordered conditions listed on court orders and formal accountability agreements. We determined the data, with the exception of three condition fields, was sufficiently reliable for our audit purposes. The reliability of the three condition fields was not consistent between counties and varied depending on the condition type and the associated disposition type. We reviewed prior audit reports and followed up on findings related to our audit objective. We conducted this audit in accordance with generally accepted government auditing standards. 16 A BOUT THE S ECRETARY OF S TATE A UDITS D IVISION The Oregon Constitution provides that the Secretary of State shall be, by virtue of her office, Auditor of Public Accounts. The Audits Division exists to carry out this duty. The division reports to the elected Secretary of State and is independent of the Executive, Legislative, and Judicial branches of Oregon government. The division audits all state officers, agencies, boards, and commissions and oversees audits and financial reporting for local governments. Directory of Key Officials Interim Director Deputy Director Deputy Director Drummond Kahn, MS, CIA, CGFM, CGAP William K. Garber, MPA, CGFM Mary E. Wenger, CPA Audit Team James E. Scott, MM, CIA, Audit Manager Rex Kappler, MBA, CMA, CFM, Principal Auditor Alexandra D. Fercak, MPA, Senior Auditor Shanda L. Miller, MPA, Senior Auditor Olivia Ngiraikelau, MPA, Associate Auditor Ramona J. Mitchell, MPA, Staff Auditor Sarah F. Salisbury, Staff Auditor Weston R. Brinkley, Staff Auditor This report, a public record, is intended to promote the best possible management of public resources. Copies may be obtained from: internet: http://www.sos.state.or.us/audits/index.html phone: 5 03-986-2255 mail: Oregon Audits Division 255 Capitol Street NE, Suite 500 Salem, OR 97310 The courtesies and cooperation extended by officials and employees of the Oregon Youth Authority and County Juvenile Departments during the course of this audit were commendable and sincerely appreciated. Auditing to Protect the Public Interest and Improve Oregon Government