Office of the
LEGISLATIVE AUDITOR GENERAL
State of Utah
REPORT NUMBER 2009-03
January 2009

A Performance Audit of the
Drug Offender Reform Act (DORA)
DORA is intended to provide selected felony offenders with drug
treatment and increased community supervision in order to reduce the
costs associated with future criminal behavior. DORA began as a
three-year pilot program in 2005 but expanded statewide in 2007,
prior to the conclusion of the pilot study, at an annual cost of about
$8 million. Ideally, program effectiveness should be judged over
many years as data becomes available to know whether DORA
participants are less likely to commit future crimes. However, early
evidence does not demonstrate reduced criminal behavior. Therefore,
it remains unknown whether the expected savings will be realized.
Legislative leadership asked us to conduct a limited review of
DORA focusing on the cost savings generated by the program. In a
September 2008 meeting, legislators were told that “DORA has saved
the state over $23 million in the last year.” For this report, we tried to
verify these cost savings. In addition, we reviewed the history of
DORA, studied the results of the DORA Pilot Program Evaluation,
and reviewed some of the implementing practices of participating state
and local agencies. It was not within the scope of this audit to focus
on other goals of DORA—such as long-term offender or societal
outcomes.

Office of the Utah Legislative Auditor General

DORA is a process for
rehabilitating offenders
with substance abuse
addiction and provides
resources for treatment
and supervision.

1

This report addresses three main points:

It is too early to draw
conclusions from the
DORA pilot program,
but participants do not
demonstrate reduced
criminal behavior.

•

Although established as a pilot program to evaluate its
effectiveness, DORA was expanded statewide before pilot
program results were available.

•

While it is too early to draw firm conclusions, the report on the
DORA pilot program does not demonstrate reduced criminal
behavior from DORA participants, and, as a result, cost savings
to the state have not been realized.
 

•

State agencies implementing DORA should review some
practices. The Division of Substance Abuse and Mental Health
should review apparent inconsistencies in cost and type of
treatment provided to ensure funds are used efficiently. The
Department of Corrections should review DORA cost
accounting practices to ensure funds are spent on DORA
offenders.

 
 

DORA Program Expanded Statewide
Before Pilot Program Had Expired

DORA was expanded
statewide before the
three year pilot
program was
completed.

DORA began as a three-year pilot program in 2005 but was
expanded statewide after just two years in 2007 before pilot program
results were available. DORA increases up-front costs because it
provides services that offenders might not otherwise receive. Future
savings are envisioned if fewer crimes are committed by DORA
participants, thus reducing the costs of incarceration.
DORA Pilot Was Established
To Test Program Effectiveness

DORA intended to be a
collaborative effort
among state and local
agencies.

Legislation establishing the Drug Offender Reform Act (DORA)
Pilot Program passed during the First Special Session of the 2005
Legislature. Senate Bill 1004 created a three-year pilot program in the
Third Judicial District and appropriated $1.4 million to implement the
program on a limited basis and study its effectiveness.
The purpose of the pilot program was to examine the impact of
providing enhanced services to selected felony offenders. DORA is

2

A Performance Audit of the Drug Offender Reform Act (January 2009)

intended to be a collaborative effort among state and local agencies to
identify and treat convicted felons who will benefit from substance
abuse treatment and who do not require incarceration. Eligible
offenders are identified based on both a drug assessment and a risk
assessment. Both assessments are administered prior to sentencing
and are used to advise the court about the offender’s suitability for
DORA.
•

•

The Addiction Severity Index (ASI) measures a person’s drug
dependency. The ASI is administered by substance abuse staff
to evaluate whether the offender will benefit from treatment.
The Level of Service Inventory (LSI) measures an offender’s
level of risk. The LSI is administered by corrections staff to
determine whether an offender can be released into the
community or should be incarcerated.

Through DORA, judges
are provided with
specific information,
prior to sentencing,
about offenders’
substance abuse
patterns and
recommended
treatment options.

Initially, only felony offenders convicted of violating Utah’s
Controlled Substances Act were eligible for DORA. With the passage
of Senate Bill 185 during the 2006 General Session, the DORA pilot
program criteria were expanded to accept all felony offenders who had
an assessed drug problem.
DORA Program Was Expanded Statewide
Before Pilot Results Were Available
Before the three-year pilot program had expired, the 2007
Legislature passed S.B. 50 – Drug Offenders Reform Act and
appropriated $8 million for fiscal year 2008 for a statewide program.
The statewide DORA program requires the courts to order every
offender convicted of a felony on or after July 1, 2007 to participate in
a screening and assessment prior to sentencing. Based on the ASI and
LSI assessments, the court may order the offender into the DORA
program.
Although DORA was expanded statewide beginning in July 2007,
pilot program results were not yet available. As part of the original
DORA pilot, the Commission on Criminal and Juvenile Justice
(CCJJ) contracted with the Utah Criminal Justice Center at the
University of Utah to evaluate the DORA pilot program. A report
called Evaluation of the Drug Offender Reform Act: DORA Pilot

Office of the Utah Legislative Auditor General

Senate Bill 50
appropriated $8 million
in FY08 for DORA and
required every felony
offender to participate
in screening and
assessments for
DORA.

CCJJ contracted with
the University of
Utah’s Criminal Justice
Center to evaluate the
pilot program.

3

(hereafter, DORA Pilot Evaluation Report) was released in November
2008, and some of its results are discussed later.

The statewide DORA
program allows firsttime parolees to
participate in the
program.

The statewide DORA also allows offenders who are being granted
parole for the first time to participate in the program. Depending on
an offenders ASI and LSI scores, the Board of Pardons and Parole
may order the offender to participate in DORA as a condition of
parole. The Department of Corrections (DOC) reports that the
number of parole offenders admitted to DORA in fiscal year 2008 is
265, or 34 percent of the total number of DORA admissions.
Five state agencies have received DORA funds. As shown in
Figure 1, in fiscal years 2008 and 2009, the Division of Substance
Abuse and Mental Health (DSAMH) and the DOC received the
largest portion of the funding for treatment and supervision.
Figure 1. DORA Appropriations for Fiscal Years 2008 and 2009. Most
of the DORA appropriation goes toward treatment and supervision.
Agency
Division of
Substance Abuse
and Mental Health
Department of
Corrections
Administrative
Office of the Courts
Board of Pardons
and Parole
Commission on
Criminal and
Juvenile Justice
Total

Funding
FY 2008
$ 4,850,000

Funding
FY 2009*
$ 4,683,300

3,039,600

3,419,500

50,400

0

36,000

40,500

24,000

19,500

$ 8,000,000

$ 8,162,800

Purpose
ASI assessment,
treatment, case
management
LSI assessment,
supervision, case
management
case management
case analysis and
case management
administration,
research, and
evaluation of DORA

* As reduced by special session in September 2008

DORA was funded $9
million for FY09, but
funding was reduced
to $8.2 million in the
2008 Second Special
Session.

4

For fiscal year 2009, the Legislature originally appropriated $9
million; however, the funding was decreased to $8.2 million during
the 2008 Second Special Session. The special session eliminated
funding to the Administrative Office of the Courts and reduced
funding to the DSAMH and CCJJ. In addition, some unspent funds
from the fiscal year 2008 appropriation were eliminated.

A Performance Audit of the Drug Offender Reform Act (January 2009)

Early Evidence Does Not Demonstrate
DORA Cost Savings
Not enough DORA recipients from the pilot program have
completed treatment for sufficient time to draw firm conclusions
about DORA effectiveness. However, early indicators do not show
that DORA recipients are less likely than non-DORA offenders to
engage in criminal behavior. Thus, proof is not available that DORA
services will prevent future criminal behavior and its associated costs,
such as prison costs. In addition, we are concerned about the accuracy
of some data used to evaluate DORA. To enable a valid analysis in
the future, data integrity concerns should be addressed.

DORA Pilot Program
participants have not
been out of treatment
long enough to
measure effectiveness.

More Time Is Needed for a
Better Pilot Program Evaluation
The DORA Pilot Evaluation Report released in November 2008
stated that it is too soon to measure the program’s impact. Because
the objective of DORA is to reduce future criminal behavior of
recently convicted felons, enough time needs to pass before a
meaningful evaluation can occur. The DORA pilot program provided
(or is still providing) services to offenders who had recently been
convicted of felonies.

Since the objective of
DORA is to reduce
future criminal
behavior, more time is
needed to evaluate
program effectiveness.

We reviewed the DORA Pilot Evaluation Report and met with
staff at the Utah Criminal Justice Center to better understand the
available information. However, we did not have access to the study’s
data and did not complete an independent analysis of the data.
The study’s results are presented in two separate groups depending
on whether offenders began the program before or after March 26,
2006; at that time, the program was expanded to include felons who
were not necessarily convicted of drug offenses. Thus,
•

Time 1 includes offenders who began the program between
July 1, 2005 and March 25, 2006. Time 1 consists of 85
DORA participants, all of whom were convicted of a drug
felony offense in Salt Lake County.

Office of the Utah Legislative Auditor General

5

•

The control groups
used in the DORA Pilot
Program Evaluation
are not ideal for
comparing with the
DORA participants.

The time that has
passed since offenders
completed probation is
a key factor in
evaluating program
effectiveness.

6

Time 2 includes offenders who began the program between
March 26, 2006 and November 20, 2006. Time 2 consists of
134 DORA participants who were convicted of a drug-related
or other felony offense in Salt Lake County.

The study includes non-DORA comparison groups from Salt Lake
and Davis counties. The study did not use an experimental design that
randomly assigned offenders to DORA and non-DORA groups.
However, UCJC staff report they attempted to make the comparison
groups as similar as possible to the DORA participants. In general,
the comparison groups seem fairly similar to the DORA group. The
Salt Lake comparison group has a somewhat more severe criminal
history and the Davis group has a somewhat less severe criminal
history than DORA participants. However, many in the non-DORA
groups did not have ASI assessments, thus adding to concerns about
the comparability of the groups. We are uncertain why offenders in
the Salt Lake comparison group were not provided DORA services.
Of course, the Davis comparison group was geographically ineligible
for the DORA pilot program.
The time since program completion is a key factor in evaluating
program effectiveness. Figure 2 provides data from the pilot study
that shows some offenders in both Time 1 and Time 2 have not yet
exited probation. Of those who have completed probation, the
average time is about 16 months off probation for Time 1 and 9
months for Time 2. The range of time those averages are based on is
extreme.

A Performance Audit of the Drug Offender Reform Act (January 2009)

Figure 2. Selected Pilot Study Information for DORA Participants
and Comparison Groups. Many offenders remain on probation or have
not exited probation long enough to allow a good assessment of DORA
effectiveness.
DORA

Salt Lake

Davis

Sample Size

85

103

134

Number Exited Probation
Average Days Off Probation

63
485

83
497

107
470

Range of Days Off Probation
Time 2

58-796

19-936

2-1009

134
88

108
63

155
98

285
2-761

333
36-637

304
10-820

Time 1
(only drug felony convictions)

(drug or other felony convictions)

Sample Size
Number Exited Probation
Average Days Off Probation
Range of Days Off Probation

Source: DORA Pilot Evaluation Report (Completed November 2008)

For example, Figure 2 shows that 63 Time 1 DORA offenders have
exited probation for an average length of 485 days. However, the
span of days off probation ranges from 58 to 796.
Cost Savings Remain Uncertain
The DORA model has been designed to identify offenders with
drug problems early on, and treat these offenders, therefore reducing
future criminal behavior. By reducing future criminal activity, savings
can be achieved within the criminal justice system through reduced
costs of incarceration. Also, there should be fewer crime victims, and
more reformed criminals will be productively employed.

The DORA model has
been designed to
identify offenders with
drug problems early
on, and treat these
offenders, therefore
reducing future
criminal behavior.

One important goal of DORA is to reduce Utah’s future need for
prison and jail beds, thereby saving taxpayer funds. At the time
DORA was proposed state agencies estimated that substance abuse
treatment per client costs about $3,200 - $4,200 annually, while the
cost of a prison bed was $25,700 annually. Similarly, when a
statewide DORA bill was being considered by the Legislature,
legislators were told that DORA could save the state $838 million
over a 10-year period. This cost savings was based on 70 to 85
percent of prison inmates qualifying for DORA.

Office of the Utah Legislative Auditor General

7

DORA Increases Current Costs. As shown in Figure 1, DORA
costs about $8 million per year. The DORA model is to spend
somewhat more now by providing more treatment and supervision to
avoid having to spend much more later in prison costs. While future
cost savings remain uncertain, current costs increase.

DORA does not reduce
prison costs because
DORA participants are
very unlikely to be
sentenced to prison.

It is important to understand that DORA is not considered as a
prison diversion program, so it does not reduce current prison costs.
An offender who was convicted of a serious offense, or posed a serious
risk to public safety would not be eligible to participate in DORA.
CCJJ staff told us, and the results of pilot program evaluation report
confirm, that DORA recipients would be very unlikely to otherwise be
sentenced to prison.

Early results from the
Pilot Evaluation do not
indicate a reduction in
criminal activity by
DORA participants.

Future DORA Savings Are Uncertain. Expected DORA
savings are based on the premise that DORA recipients are less likely
to commit future crimes, thus saving future prison costs. We reviewed
the DORA Pilot Evaluation Report for evidence of reduced criminal
activity. While it remains early, we did not see evidence of crime
reduction that could lead to future cost savings.
Figure 3 shows data from the pilot study on criminal activity of
Time 1 offenders who have exited probation. The data does not
indicate any reduction in criminal behavior. In fact, a higher
percentage of DORA recipients appear to have engaged in subsequent
criminal activity than the comparison groups. We show the
information for Time 1 offenders because they began the program
sooner and have longer time for follow-up.
Figure 3. Selected Post-Supervision Data for Time 1 Offenders.
Early indicators do not show that DORA is making a positive impact on
offenders when compared to the control groups.

Number Exited Probation
Number with New Arrests
Number with New Convictions
Number with New Prison Commitments

DORA

Salt Lake

Davis

63
22 (35%)
5 (8%)
3 (5%)

83
22 (27%)
4 (5%)
4 (5%)

107
25 (23%)
5 (5%)
1 (1%)

Source: DORA Pilot Evaluation Report

8

A Performance Audit of the Drug Offender Reform Act (January 2009)

While Figure 3 does not show that DORA recipients are less likely
to commit crimes, it is too soon to know. More time is needed to
draw firm conclusions due to the small number of DORA participants
who have exited probation and accrued a reasonable follow-up period.
Better data about DORA’s impact on future criminal behavior will be
available when all offenders have two or three years of time off
probation. Some Time 1 offenders remain on probation, and those
who have exited probation average just 16 months of post-supervision
time. Time 2 offenders have even less post-probation time.

The impact of DORA
on future criminal
behavior will be better
measured after
offenders have been
off probation for 2-3
years.

In the future, the criminal justice system may see reduced costs that
can be attributed to DORA. However, since it is a new program
more time is needed to measure its ability to reduce future criminal
behavior and thus reduce costs of the criminal justice system.
Data Integrity Concerns
Should Be Addressed
In addition to needing more time, the ongoing analysis of DORA
requires that appropriate outcome measures be based on reliable data.
The DORA Oversight Committee has a Research and Evaluation
Subcommittee to evaluate the impact and results of DORA. This
subcommittee is focusing on two types of impacts—impacts on the
criminal justice system and the impacts on the offenders who receive
treatment and supervision. One challenge the subcommittee is
addressing is the reliability and validity of data about DORA
participants.
DSAMH collects admissions and treatment data from the 13 local
substance abuse authorities for their database. DOC also collects data
including DORA admissions and supervision. However, there are no
special data sets, or central collection for DORA data. The DORA
data are taken from databases from these agencies that are used for
other agency activities. A few examples of data concerns are discussed
below.

Reliability and validity
of DORA data is a
concern. There is no
central collection
process for DORA
data.

Tracking of DORA Offenders by Treatment Providers May
Not Be Adequate. DSAMH uses national outcome measures for
participators in substance abuse treatment programs and can apply the
same outcome measures to DORA participants in the statewide
program. The outcome measures used by DSAMH include an increase
in substance abuse abstinence, decrease in homelessness, increase in
Office of the Utah Legislative Auditor General

9

employment, and decrease in arrests. DSAMH does not have a
unique identifier in their database to identify DORA Pilot Program
participants; however, DSAMH can identify participants in the
statewide DORA implementation.

Data discrepancies
have been found
between DSAMH and
Corrections.

Inconsistencies still
exist in the admissions
data for fiscal year
2008.

Data on DORA Participants Is Not Consistent. Discrepancies
exist when DORA data from the DSAMH is compared to the DOC’s
data. Initially for fiscal year 2008, the DSAMH reported 845
admissions, and the DOC reported 775 admissions (488 on
probation, 231 on parole, 56 other). The Research and Evaluation
Subcommittee as well as the DOC and the DSAMH have been
reviewing the 2008 DORA data and making corrections to provide
more accurate data on DORA participants. Updated admissions data
for fiscal year 2008 show that DSAMH reports 815 admissions, and
DOC reports 781 admissions (516 on probation and 265 on parole).
Inconsistencies still exist in the admissions data. Inconsistency in the
admissions data may be due to data-matching problems, such as
offenders listed by different names in one database, incomplete or
incorrect offender information, duplicated data, or other reasons.
Some Data in Pilot Evaluation Report Raises Questions. The
DORA pilot study relied on data from Salt Lake and Davis County
substance abuse authorities and the DOC to complete the evaluation.
ASI assessment data was available for the DORA participants and the
Salt Lake County comparison group, but was not available for the
Davis County comparison group. In addition, Figure 4 shows an
example of questionable DORA data.
Figure 4. Selected Assessment and Treatment Data from DORA Pilot
Evaluation Report. Some data for DORA offenders raises questions
about data validity.
Description
Percent with Assessments

Time 1
93

Time 2
96

Percent with Treatment Admissions

87

90

Percent who Received Substance Abuse
Treatment During Probation

85

92

Source: DORA Pilot Evaluation Report

Figure 4 indicates a small inconsistency in DORA offender treatment
admissions. For Time 1, local substance abuse authority records show

10

A Performance Audit of the Drug Offender Reform Act (January 2009)

slightly more treatment admissions (line 2) than are shown in DOC
records (line 3); for Time 2, the opposite is the case.
Reliable and valid data needs to be available to accurately evaluate
and measure the impact of DORA and to determine if DORA is
operating as intended. The DORA Research and Evaluation
subcommittee should continue working with treatment and
supervising agencies to improve data quality.

Reliable and valid data
needs to be available
to accurately evaluate
and measure the
impact of DORA.

DORA Implementing Agencies
Should Review Some Processes
Our audit work also reviewed some of the implementing practices,
although not in great depth. After interviewing CCJJ staff and
reviewing oversight meeting minutes, we concentrated on the
treatment and supervising agencies that spend most of the DORA
funds. As a new program, DORA’s practices and policies are still
being developed. This section includes some issues we think agencies
should consider in the future.
The DORA model includes increases in both treatment and
supervision of offenders. The biggest change brought by DORA
seems to be that selected felony offenders are much more likely to get
drug treatment. Figure 5 has data from the DORA Pilot Evaluation
Report showing that the DORA offenders were much more likely
than the comparison groups to have treatment admissions. However,
the difference between the DORA and comparison groups in
frequency of contact with Adult Probation and Parole (AP&P) agents
is more modest. It is notable that the pilot study data, shown in the
table below, indicates that much more contact occurred between
agents and treatment providers for the DORA groups. However, this
greater contact is at least partially due to many comparison group
offenders not having treatment providers.

Office of the Utah Legislative Auditor General

The Pilot Evaluation
showed that DORA
offenders were more
likely to have treatment
admissions and to
have contact between
their AP&P agents and
treatment providers.

11

Figure 5. Selected Treatment and Supervision Information for DORA
Participants and Comparison Groups. DORA offenders appear to
receive much more drug treatment and somewhat more intensive
supervision than comparison groups.
DORA

Salt Lake

Davis

Time 1
(Only Drug Felony Convictions)

Numbers with Treatment Admissions
Numbers with Contact Between AP&P
Agent and Treatment Provider
Average Days Between P.O. Contacts

74 (87%)

47 (46%)

43 (32%)

82 (97%)

39 (38%)

32 (24%)

17

24

27

120 (90%)

44 (41%)

25 (16%)

113 (84%)

36 (33%)

32 (21%)

23

25

Time 2
(Drug or Other Felony Convictions)

Numbers with Treatment Admissions
Numbers with Contact Between AP&P
Agent and Treatment Provider
Average Days Between P.O. Contacts

21

Source: DORA Pilot Evaluation Report

As discussed in the prior section, there are questions about the
accuracy of some of this data. For example, it is unclear how, for both
the DORA group in Time 1 and the Davis group in Time 2, the
number of participants with treatment admissions is less than the
number of participants with contact between their supervising agent
and treatment provider.
The remainder of this report addresses the use of DORA funds by
the two main implementing agencies, DOC and DSAMH. It was
beyond the scope of this report to audit DORA implementation in
detail, but this section discusses some issues of concern.
DSAMH Should Review Apparent Inconsistencies
Among Local Authorities
Local authority financial records provided by DSAMH suggest
that DORA operates inconsistently throughout the state. For fiscal
year 2008, the Legislature appropriated $4,850,000 to the DSAMH
for DORA treatment services. About $100,000 was retained by the
division for administrative expenses, and the remainder was allocated
to local authorities based on adult population and probation or parole
admissions.
Only 58 percent of
DORA funds was
disbursed to local
authorities in fiscal
year 2008.

12

For this first year of statewide operations of the DORA program,
fiscal year 2008, only 58 percent of the available DORA funds was
actually disbursed to the local authorities (leaving about $2 million

A Performance Audit of the Drug Offender Reform Act (January 2009)

unspent). An initial lack of staff and an initial lack of referrals were
both cited by local authority officials as factors that prevented DORA
from being fully operational in some regions during the beginning
months of statewide DORA. This start-up period contributed to
some of the cost inconsistencies observed among the local authorities.
Cost per Case Varies Among Local Authorities. Figure 6
shows the DSAMH calculated average cost per DORA case for each
local authority. Statewide, the DSAMH reports a $3,300 average, as
shown in the following table.

The average cost per
case varies
significantly among
the local authorities.

Figure 6. The Average FY 08 Cost per DORA Case Was Inconsistent
Among Local Authorities. Even among urban and rural regions that
would be expected to have comparable populations, the average DORA
cost per case is often very dissimilar.

In fiscal year 2008, the
average cost per
DORA case throughout
the state was $3,300.

According to the DSAMH’s data, local authorities in both urban
and rural areas have very divergent average costs. Although division
staff told us the cost differences represent real differences among local
authorities, the DSAMH has not yet identified the exact reasons for
these differences in average cost. The following section offers some
possible explanations for the inconsistencies.

DSAMH should analyze
the reasons for cost
differences.

Local Authorities May Vary in Their Treatment Approaches.
One reason for cost differences among local authorities could be that
they provide different types of treatment. It appears that the
proportion of admissions to the different service types varies among

Office of the Utah Legislative Auditor General

13

the local authorities. Figure 7 shows the admissions types for DORA
clients at the four largest local authorities.
Figure 7. Fiscal Year 2008 Distribution of DORA Admissions to
Different Treatment Options by the Four Largest Local Authorities.
Local authorities vary in the type of treatment admissions for DORA
offenders.

The ratio of
admissions to each
service type varies
among the local
authorities.

Note: Offenders may have multiple admissions.

Some differences are expected. However, the figure indicates that
Weber is much more likely to have residential admissions than other
local authorities while Utah County is much more likely to have
intensive outpatient admissions.
We do not know whether the offender population’s therapeutic
needs in each jurisdiction are really as diversified as suggested by the
data in the graph above. One director explained there may be some
inconsistency in applying the process for admitting DORA clients.
He explained that the American Society of Addiction Medicine’s
(ASAM) Patient Placement Criteria is used as a tool for determining
the level of care. He stated that each local authority is responsible for
providing its own ASAM training and that the training is not
standardized throughout the state.
Multiple factors appear
to affect a DORA
client’s care level
placement.

14

Other directors indicated other possible causes for differences in
service types. One suggested that clients may not always truthfully
report their drug abuse patterns; such inconsistency could affect the
client’s placement. Another director explained that, initially, not all

A Performance Audit of the Drug Offender Reform Act (January 2009)

judges complied with the ASAM scoring model and ordered clients to
residential care who may, more appropriately, have been ordered to
jail.
Different treatment philosophies may also play a part. For
example, Utah County has identified that it has a large prescriptionopiate-offender population which can be assisted through
rehabilitation by being given an opiate-effects-blocking prescription
drug called Vivitrol. By being administered Vivitrol, an offender’s risk
of relapse is theoretically reduced, and there is less of a need for many
clients to enter residential treatment. Not all local authorities use
DORA funds to pay for prescription drugs like Vivitrol to treat their
dependent DORA clients.
Some DORA Funding May Be Used
To Supervise Non-DORA Offenders
The Department of Corrections (DOC) receives nearly 40 percent
of the DORA appropriation for their part in supervising offenders.
The accounting of those funds raises questions because supervision
costs for DORA and non-DORA offenders are not clearly separated.
In addition, the DORA Pilot Evaluation Report does not indicate a
great increase in the frequency of contact with DORA offenders;
however, agents report their supervision practices for DORA
offenders are much different.

Corrections received
nearly 40 percent of
DORA appropriations,
70 percent of which
went to personnel.

Accounting for DORA Funds Raises Questions. Most DORA
funds spent by the DOC are used to pay for personnel. In fiscal year
2008, about 70 percent of the DOC’s DORA expenditures were used
for personnel and 15 percent for vehicles. As of June 2008, DOC
reports the DORA organizational units included 31 AP&P agents,
two supervisors, and two urinalysis technicians. Additionally, four
personnel were hired to conduct assessments, help train other agencies
on DORA processes, and work with the transitioning of offenders to
regional offices.
Although these personnel are funded exclusively by DORA funds,
they do not work exclusively with DORA offenders. According to
case lists provided to us by the DOC, as of June 2008, DORA agents
supervised 722 DORA offenders and 746 non-DORA offenders. On
the other hand, DOC also reported that 123 DORA offenders were
supervised by non-DORA agents. Based on this data, it seems that
Office of the Utah Legislative Auditor General

DORA agents are
funded exclusively
with DORA funds, but
supervise non DORA
offenders as well.

15

some DORA funds may subsidize the supervision of non-DORA
offenders. However, DOC also reports that non-DORA funds are
being used to cover some DORA costs.
There are practical reasons for the mixed DORA and non-DORA
caseloads described above. For example, many of the DORA agents—
including all agents from the Salt Lake Region where the pilot
program took place—were previously non-DORA agents with nonDORA caseloads who transferred into a DORA organizational unit.
Rather than transferring existing cases, the agents kept them. In
addition, some rural areas may not have enough DORA cases for a
full-time agent, so a mixed caseload makes sense.

After DORA enabled
DOC to hire 31
additional agents,
statewide caseloads
decreased from 75 to
65.

It appears that the DORA funds have enabled DOC to provide
increased supervision to non-DORA as well as DORA offenders.
According to the DOC, average caseloads for non-DORA agents have
decreased from about 75 to 65 since 2006, despite an increase of
nearly 1,300 offenders. Meanwhile, the DORA agents have a caseload
of about 44, although less than half of the cases are DORA offenders.
The DOC should be able to more accurately account for DORA
funding. By allocating both DORA and non-DORA agents’ salaries
based on the percentage of DORA cases an agent has, DORA funds
would only be used for DORA cases. In the future, DOC may be able
to reduce its number of DORA agents if the number of DORA cases
increases in rural regions, and as non-DORA offenders assigned to
DORA agents complete probation. With 845 DORA cases and 31
agents, the average caseload without a mixed caseload would be just
27 cases.

The pilot study shows
only small increases in
the amount of contact
between agents and
DORA participants.

DORA Offenders May Not Receive Substantially More
Supervision. Figure 5 shows the average number of days between
agent and offender contact, as reported in the DORA Pilot Evaluation
Report. The number of days between contacts was somewhat lower
for both Time 1 and Time 2 DORA offenders, but not by a lot. The
study shows that there has been increased contact between agents and
DORA offenders, but the differences are not very large despite the
decreased caseload.
The DOC has standards for frequency of contact between agents
and offenders on probation, as shown in Figure 8.

16

A Performance Audit of the Drug Offender Reform Act (January 2009)

Figure 8. Department of Corrections Levels of Supervision. The DOC
has specific standards for the amount of supervision an offender should
receive based on their level of supervision.
Level of
Supervision
Intensive

LSI Score

Number of Contacts

41-54

2 office contacts and
2 residence contacts per month
1 Office contact and
1 residence contact per month
1 office contact per month and
1 residential contact every 60 days
1 contact every 90 days

High

24-40

Moderate

14-23

Low

0-13

For DORA offenders, however, there is no specific standard for the
frequency of contacts with agents. According to the Department of
Corrections’ Standards of Supervision,
DORA caseloads are established to provide closer supervision and
a coordinated supervision of drug offenders. DORA focuses on
close and effective relationships and collaboration with treatment
providers and a mutually supportive role.

DOC does not have a
specific standard for
the frequency of
contacts with DORA
participants.

In our conversations with seven agents from five of the six regions
in the state, all agents mentioned that the level of supervision is higher
with DORA offenders. Agents report that meeting with family
members and employers are important components of supervision in
addition to face-to-face meetings with offenders. Agents from the two
biggest regions both said that they try to visit offenders twice per
month, once in the office and once in the community (home or work
visits). However, there is no specific standard that specifies the
number of contacts an agent should have with a DORA offender.
In addition to contact with offenders, agents say that contact with
treatment providers is important. In speaking with agents, we were
consistently told that smaller caseloads allow agents to work closely
with the treatment provider. We observed this in practice at a meeting
where AP&P agents and treatment providers discussed offenders’
progress in the treatment program.

DORA agents report
working closely with
treatment providers.

According to an AP&P supervisor as well as several agents, with
DORA the response to a failure in treatment is less likely to be

Office of the Utah Legislative Auditor General

17

punitive. Instead, the agent can work with the treatment provider and
the offender to determine how to best help the offender and prevent
future problems. Data in the DORA Pilot Evaluation Report
indicated that significantly fewer offenders had their probation
revoked after non-compliance events like substance use and other
supervision violations. The agents were more likely to work with the
offender rather than having them removed from the community.

Recommendations
1. We recommend that the DORA Research and Evaluation
Subcommittee continue to monitor the Pilot Program
participants to determine post-supervision outcomes.
2. We recommend that the DORA Research and Evaluation
Subcommittee continue to work with the agencies that provide
DORA data to correct data errors.
 

3. We recommend that the Division of Substance Abuse and
Mental Health review and evaluate the differences for DORA
offenders’ cost differences among local authorities and provide
additional guidance if needed.
4. We recommend that the Department of Corrections more
accurately account for the DORA funding.
5. We recommend that the Department of Corrections establish
clear guidelines for the number of contacts that DORA
offenders should receive from AP&P agents.

18

A Performance Audit of the Drug Offender Reform Act (January 2009)

Agency Response

Office of the Utah Legislative Auditor General

19

This Page Left Blank Intentionally

20

A Performance Audit of the Drug Offender Reform Act (January 2009)

•.
~

~

i

-

+•.,

"-

....

~

,

.+

State of Utah
COMMISSION ON CRIMINAL AND
JUVENILE JUSTICE

Jon M. Huntsman, Jr.
Governor
Robert S. Yeates
Executive Director

Suite E330, East Capitol Complex • Salt Lake City, Utah 84114
801-538-1031 • Fax: 801-538-1024 • www.justice.utah.gov

January 13, 2009

John M. Schaff, CIA
Legislative Auditor General
W315 Utah State Capitol Complex
Salt Lake City, Utah 84114-5315
Re: A Performance Audit of the Drug Offender Reform Act (DORA)
Report No. 2009-03
Dear Mr. Schaff:
Thank you for the opportunity to provide a response to the Performance Audit of the Drug Offender
Reform Act (DORA). The agencies involved in the audit appreciate the work of the Legislative auditors
and the courtesy shown as they worked with our staff. We are grateful for the observations and
recommendations that identify important areas where we can improve DORA and DORA’s outcomes.
DORA is unique in state government for many reasons, not the least of which is the collaboration it has
promoted among diverse agencies across the substance abuse treatment and criminal justice systems. In
keeping with this collaboration, the three agencies which were given the opportunity to respond to this
audit – Corrections, Human Services, and the Commission on Criminal and Juvenile Justice – have
chosen to offer a single, coordinated response.
We concur with the recommendations of the audit and have already begun the work required to
implement them. Before we respond to the specific recommendations, we would like to take this
opportunity to make a few general comments about DORA and the audit.
The audit identifies aspects of DORA that represent the difficulties inherent in implementing any
sweeping policy reform. Nonetheless, DORA’s vision remains on track. From its inception, those
involved in the implementation of the program have understood that it would be an incremental process.
This is common in any new program such as DORA, particularly one that requires collaboration among
a variety of state agencies, branches of government, local government entities, and service providers in
the community.
DORA Goals
The audit report states the intent of DORA as being “to provide selected felony offenders with drug
treatment and increased community supervision to reduce the costs associated with future criminal

John M. Schaff, CIA
Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03
January 13, 2009

Page 2

behavior”. While this is certainly one of the goals of DORA, it is important to note that the intent of
DORA also includes several process and outcome goals in addition to reduced costs. DORA represents
a change in the way Utah handles offenders with drug problems by addressing the root cause of many
crimes – substance abuse – with the ultimate objectives of reducing both substance abuse and crime and
their associated impacts on individuals and society.
DORA’s process goals may be summarized as follows:
-

“Smarter sentencing” is accomplished by providing the judge with specific information about
the offender’s substance abuse problem and treatment needs prior to sentencing. DORA does
not mandate treatment, but provides judges with this information and at the same time provides
funding for treatment services.

-

“Smarter treatment” is achieved by providing funding to create more treatment slots in both
the community and in correctional facilities, and by conducting a comprehensive assessment of
the offender’s substance abuse and determining the appropriate level of treatment (e.g.,
outpatient treatment, intensive outpatient treatment, day treatment, residential treatment) for the
offender.

-

“Smarter supervision” means the treatment provider and the Adult Probation and Parole
(AP&P) agent are working for the same goals, and offenders/clients receive the same message
from both treatment and AP&P. AP&P and substance abuse treatment staff meet regularly to
discuss goals and progress, and more information is available to both parts of the system which
improves effectiveness and offender accountability.

DORA’s outcome goals may be summarized as follows:
-

Reduce substance abuse and crime/recidivism

-

Reduce Utah’s future needs for prison and jail beds

-

Create more law-abiding and taxpaying citizens

-

Create safer neighborhoods

Successful, long-term implementation of DORA will do more than just save money; it will literally save
lives and sustain families and communities.
The audit mentions that Legislators were told DORA “could save the State $838 million over a 10-year
period.” It does not explain that these figures were based on costs avoided through full funding for the
statewide implementation of DORA. Only half of the initial funding request was appropriated, which
would reduce the estimate of costs avoided by at least 50%. In addition, the audit does not explain that
most of these avoided costs were not in state government or even local government operations, but in
reduced victimization costs resulting from thefts and injuries. Less than 30% of the projected avoided
costs were in government services, both state and local.

John M. Schaff, CIA
Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03
January 13, 2009

Page 3

DORA Pilot Study
The audit acknowledges that it is too soon to draw conclusions from the DORA pilot study regarding the
impact of DORA on later criminal activity. Large numbers of participants are still on probation and
follow-up times for those who have exited probation are short. Generally, criminal justice research
requires at least one year of follow-up post-supervision for recidivism comparisons. More than 30% of
the DORA pilot participants are still on probation, and the fact that they are continuing on probation
after more than two years suggests that they have a good chance of being successful. In criminal justice
research, it is important to remember that failures generally happen quickly, but successes require time
and effort to achieve. It is possible that longer follow-up of the pilot participants will show greater rates
of success for the DORA participants.
While the audit mentions the failure of the pilot study to show a lower rate of criminal activity for
DORA participants, the audit does not acknowledge the positive results found in the DORA pilot. The
study found that essential elements of DORA predicted successful completion of probation, including:
completing a treatment admission, agent and offender contact in the community, and shorter time lags
between conviction and probation start.
Specific Recommendations
Recommendation 1:
The audit recommends that the DORA pilot participants continue to be followed by the DORA
Oversight Committee’s Research and Evaluation Subcommittee to determine their long-term
outcomes. We support this recommendation and plan to continue to follow these offenders and
provide annual updates on their outcomes.
Recommendation 2:
The audit notes problems with the accuracy and completeness of the data on DORA participants
provided by the Division of Substance Abuse and Mental Health (DSAMH) and the Department of
Corrections and recommends that efforts be made to correct these errors. We wholeheartedly
support this recommendation and will continue to work on these issues. Data collection is
improving. Sharing public and private data among a variety of entities is always challenging. The
DORA Research Committee shares the auditors’ concerns related to the reliability and validity of
data, and continues to work toward a solution. In time, as the program is fully implemented and
stabilized, we are confident the data concerns will be resolved. Completely accurate data have been
and will continue to be the goal of the DORA evaluation project.
However, we recognize that hundreds of different individuals working in Corrections’ AP&P
offices, in the prisons, at the Division of Substance Abuse and Mental Health, and in the Local
Substance Abuse Authority and other treatment provider agencies are collecting and entering data
and some inaccuracies are unavoidable. In addition, data collected on substance abuse treatment are
subject to strict Federal and professional privacy regulations that, while they are important to client
confidentiality, make data cleanup very difficult. In spite of these challenges, the Research and
Evaluation Subcommittee will continue to make every effort to provide the most complete and
accurate data possible on DORA.

John M. Schaff, CIA
Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03
January 13, 2009

Page 4

Recommendation 3:
The Division of Substance Abuse and Mental Health has already begun the recommended review
and evaluation of the differences in costs for DORA offenders among the local authorities. As the
Division reviewed and evaluated the cost differences prior to the report’s publication, the Division
concluded that there is no one exact reason for the differences in average cost. In fact, there are
multiple reasons, including the ones mentioned in the audit report. One reason not mentioned in the
audit report is the difference in start-up costs for the new program among different areas of the state.
Some local authorities were able to absorb the initial additional client load with their already existing
staff and programs, while others had to hire new staff members that weren’t fully utilized until the
client population had increased.
Recommendation 4:
The Department of Corrections is thankful for the opportunity to reiterate its commitment to the
vision of DORA and is confident that as the program is fully implemented, there will be limited
mixing of DORA and non-DORA funds.
o The building of DORA caseloads and treatment needs is incremental. With the exception of
rural areas in Utah that will continue to require mixed caseloads, as there are simply fewer
DORA clients in those regions, DORA-funded agents will ultimately supervise DORA
caseloads only. At this time, however, the Department would be concerned if agents in our
urban regions were required to create a separate reporting stream in documenting DORA and
non-DORA time. Their responsibilities are already considerably expanded, and a focus is
needed, now more than ever, to place a premium on time spent supervising offenders in the
community.
o In the interim, UDC has provided necessary services for DORA clients from its ongoing,
non-DORA supervision budget. For example, initial screening costs, most supervision of
DORA-funded agents, support staff, and administrative costs are not supported by DORA
funds. Corrections remains confident that most DORA funds are used almost exclusively for
DORA-related operations; however, as noted above, non-DORA funds have also been used
to sustain this vital program.
Recommendation 5:
The audit recommends that Corrections establish clear guidelines for the number of contacts that
DORA offenders should have with their AP&P agents. While this is an important measure, contact
between an agent and the client is but one part of that agent’s involvement with the client. Outcome
analysis of DORA will include many different measures of agent supervision, such as the number of
contacts and days between contacts with DORA clients. There are many collateral contacts both
with the client and the treatment provider that are not being documented. By only collecting the
number of direct contacts between agents and offenders, we certainly understate the true amount of
work and contact between the DORA client and the agent. Realizations like this are common when
developing an outcome evaluation protocol, and we appreciate the auditors in bringing this particular
issue to the surface.
The auditors recognized the need for “clear guidelines” for the amount of contact with DORA
clients. UDC agrees with this recommendation. As the DORA program matures, with the help of

John M. Schaff, CIA
Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03
January 13, 2009

Page 5

the Research and Evaluation Subcommittee, we continue to gain clearer insight into what these
guidelines should entail. Our intent is to use the analysis results to determine “best practices” in
terms of establishing DORA-specific contact guidelines.
The DORA partner agencies would like to thank the Audit team for their professionalism and courtesy.
We recognize that our shared aim is to provide the most effective state services possible to the people of
our state.
Sincerely,

Lisa-Michele Church, Executive Director
Utah Department of Human Services

1~~-­
1~~
Thomas E. Patterson, Executive Director
Utah Department of Corrections

.~
~
/~
,,y

I

'.I

Robert S. Yeates, Executive Director
Utah Commission on Criminal and Juvenile Justice