Implementing Evidence-Based Practice
in Community Corrections
QUALITY
ASSURANCE
MANUAL

December 27, 2005
1

TABLE OF CONTENTS
I. Introduction ……………………………………………………………………………..…….3
II. Acknowledgements …………………………………………………………………………..4
III. Quality Assurance Plan Development ……………………………………………………..5
IV. Peer Review……………………………………………………………………………….….9
Assessment Scoring and Inter-Rater Reliability…………..….… 10
Motivational Interviewing Critique………………………………15
Cognitive Behavioral Treatment Group Facilitation……………21
Case File Review……………………………………………….…..22
V. Quality Assurance Indicators………………………………………………………………32
VI. Customer Satisfaction………………………………………………………..…………….41
VII. Program Evaluation……………………………………………………...……………….50
Formative Evaluation…………………………………..………… 51
Process Evaluation…………………………………………………51
Outcome Evaluation…………………………………………….…52
Logic Models……………………………………………………… 54
VIII. Individual Performance Measurement……………………………………..………….56
IX. Bibliography………………………………………………………………..……….………62
X. Appendix…………………………………………………………………...………………...63

2

INTRODUCTION
This document was developed as part of a multiyear cooperative agreement between the National
Institute of Corrections (NIC) and the Crime and Justice Institute (CJI) for an initiative entitled
Implementing Effective Correctional Management in the Community. The purpose of this
initiative is to assist state systems in applying an integrated approach to the implementation of
evidence-based principles in community corrections. The project model, designed by a National
Project Team of researchers, consultants, and practitioners, maintains an equal and integrated
focus on three domains: evidence-based principles, organizational development, and
collaboration. The project vision is to build learning organizations that reduce recidivism
through systemic integration of evidence-based principles in collaboration with community and
justice partners.
The Integrated Model incorporates eight evidence-based principles that, when implemented with
fidelity, have been shown to reduce offender recidivism. One of the greatest challenges in
implementing evidenced-based practices is ensuring program fidelity. Ensuring that the
assessments and other tools are reliable and valid and that programs are accurately replicated
requires the same level of planning and staff commitment as program implementation. All too
often agencies put systems and programs in place that have proven efficacy but because of
implementation flaws, these same systems and programs fail to deliver projected results. This
can result in the “baby being thrown out with the bathwater.” Agency staff becomes disheartened
because they have changed their systems and programs but see little difference in outcomes.
Quality assurance programs are designed to support implementation efforts and to ensure
accurate replication and implementation. This manual provides a simple and straightforward
approach to implementing a quality assurance plan. An overview of the principles and
components of a quality assurance plan is provided below. Because each jurisdiction has
different goals and capacities, this is not a rigid, step-by-step formula for quality assurance.
Rather, it presents the basic components of a quality assurance plan and provides options for
developing and implementing the plan. Each jurisdiction can tailor its quality assurance plan to
meet its own unique needs.
A comprehensive quality assurance plan is an invaluable tool in implementing evidence-based
practice. The plan provides a clear blueprint of the organization’s goals and how they will be
achieved. Quality assurance should be incorporated into the implementation of evidence-based
practice from the outset, with the goal of creating a “culture of quality” in the organization. Use
this manual as a reference throughout the process to establish and achieve goals for quality.

3

ACKNOWLEDGEMENTS
The Crime and Justice Institute would like to acknowledge the following individuals for
their contributions to this manual:
Authors:
Meghan Howe, The Crime and Justice Institute
Lore Joplin, The Crime and Justice Institute
Contributors:
Brad Bogue, Justice Systems Assessment & Training
Nancy Campbell, Campbell Consulting
Mark Carey, The Carey Group
Elyse Clawson, The Crime and Justice Institute
Dorothy Faust, National Institute of Corrections
Kate Florio, The Crime and Justice Institute
Sally Kreamer, 5th Judicial District Department of Correctional Services, Des Moines, IA
Kim Sperber, Quest Consulting Network, LLC.
Bill Woodward, William Woodward Associates

______________________
This document was developed under a cooperative agreement award from the National Institute
of Corrections, Community Corrections Division, U.S. Department of Justice. Points of view in
this document are those of the authors and do not necessarily represent the official position of the
U.S. Department of Justice.

4

QUALITY ASSURANCE PLAN DEVELOPMENT
OVERVIEW
A commitment to quality assurance can be highly beneficial to an organization, but the creation
and implementation of a quality assurance plan requires effort and attention to detail.
Maintaining quality is a project in and of itself within a larger program, and should be afforded
the same level of planning and staff commitment that would be given to any other significant
project: a project manager, a committee of stakeholders, and a detailed work plan with timeline
for implementation. Subsequent sections of this manual provide greater detail on implementing
components of a quality assurance plan, but an overview of the principles and components
necessary for success is provided below.
PRINCIPLES
› All key stakeholders must be committed to ongoing quality assurance, with the goal of
creating a “culture of quality.”
The process of quality assurance requires a great deal of collaborative effort to succeed.
Staff at all levels of the organization must be committed to the process of measuring and
maintaining quality on an ongoing basis.
› Specific measurable outcomes and their indicators must be precisely defined.
To accurately measure progress, everyone must be on the same page about what quality
means and how it will be measured. To compare data over periods of time, definitions
must remain consistent. Therefore, it is important that precise, constant, and useful
measures are carefully defined at the outset.
› Appropriate information management systems must be in place.
Staff members who are expected to report data need the means to do so quickly and
easily. This includes an efficient system for maintaining records on individual offenders,
as well as an efficient reporting system. In order for the data collected to be useful,
qualified staff and appropriate technology must be in place to compile and analyze data,
and present it in a concise, comprehensible format to stakeholders and decision-makers.
› Data should be incorporated into ongoing practice.
Quality can only be improved if procedures are in place to incorporate quality assurance
data into quality improvement practice. Once data is collected and analyzed, timely
decisions must be made about how this data will impact policies, procedures, and
expectations at all organizational levels.

5

COMPONENTS
• Convene a program steering committee.
Many individuals and programs are impacted by the decision to implement and monitor
evidence-based practices in community corrections. Each affected individual will have different
needs, concerns, and ideas related to implementation and monitoring of processes. To be sure
that diverse viewpoints are represented and diverse ideas are brought to the table, the quality
assurance process should be overseen by a committee rather than an individual. It will not be
possible to involve every stakeholder in the decision making process. However, steering
committee members should be carefully chosen for their ability to represent the viewpoints of
various constituents, and to exercise leadership in implementing various phases of the project.
In addition, a project lead should be chosen to manage the quality assurance efforts. The
quality assurance process is a substantial project in and of itself, and an experiences manager and
leader is required to see it through.
Choosing Indicators

• Develop a program logic model.
A logic model is a schematic diagram that outlines what
resources are being put into a program, what is expected
to happen in a program, and what outcomes are expected
in the program. Creating a logic model compels decision
makers to clearly define the steps of the program, and
also to answer the questions of why the program should
work (i.e. why would a job training program likely lead
to more jobs for offenders?) and how the outcomes
would be measured (i.e. how will we know if more
offenders are employed?). Once completed, a logic
model provides a quick reference of “the big picture” for
all stakeholders, a guideline for program implementation
and evaluation, and a benchmark for program success.
Logic models are discussed in more detail in the Program
Evaluation section of this manual.
•

Determine the program’s outcomes, and the
indicators of outcome achievement.
Stakeholders cannot know if they’ve achieved quality
without a definition of what quality is. It is important to
define the goals of the program (outcomes), and then to
define how those goals will be measured (outcome
indicators). For example, is a successful outcome for
offenders with substance abuse problems staying clean,
not recidivating, or both? If the chosen outcome is

Often, there are many
indicators that could be used
to measure a certain
outcome. When choosing
which to measure, consider
the following factors:
• Timeline
Some data will be available
more quickly than others.
• Ease of Reporting
The easier the data is to
gather and report, the more
likely it will be reported
quickly and accurately.
• Predictive Value
Some data is more accurate
than others in truly informing
progress towards a longerterm outcome, i.e. there is a
true correlation between the
two factors. This can be
obtained through existing
literature, or can be
measured over time.

6

recidivism, then will that be measured only during probation, or for a period of time after
probation? Will recidivism include new arrests, or only new charges? These definition
decisions must be made before any data can be collected.
• Develop an action plan for measuring key indicators.
Implementing a data system for measuring indicators is a complex, multi-step task that will only
be successful with careful planning and accountability for implementation. Once indicators have
been chosen, an action plan should be created to measure each indicator. The action plan should
include steps for creating, implementing, and maintaining the data collection system, including
who is in charge of each step and a timeline for completion.
SAMPLE ACTION PLAN
INDICATOR: Number of properly completed offender assessments in case files.
IMPLEMENTATION
STEP

TIMELINE

PERSON
RESPONSIBLE

Develop Peer Review form
for reviewing assessments.

06/04

Mary Smith

Develop tracking form.

06/04

Mary Smith

Discuss and amend form with
Tx Team and QA Team

07/04

Mary Smith

Train Peer Review Team on
using the review form.

07/04-08/04

Ricardo Vasquez

Develop sampling methodology
for selecting case files

06/04-08/04

Andrew Morgan

Educate officers on the Peer
Review process

08/04-09/04

Ricardo Vasquez,
Sarah Stein

Train officers on conducting
assessments.

01/04-08/04

Ricardo Vasquez,
Sarah Stein

Complete pilot sample and discuss
with Peer Review Team

08/04-09/04

Mary Smith
Andrew Morgan

Amend procedures, forms

09/04-10/04

Mary Smith

•

Educate stakeholders on the program’s desired outcomes, and the processes that will be
used to achieve those outcomes.
Organizational change can be a time of confusion and stress as well as pride and excitement, and
the atmosphere in the organization is heavily dependent on how the process is managed.
Everyone who will be participating in the process must be kept informed on the changing
organizational process and goals, and what their roles are. These stakeholders must be given the
opportunity to learn the new information, ask questions, and express their opinions if they are
ultimately expected to commit to the process.

7

•

Develop a procedure for peer review of staff performance and progress towards
outcomes.
Peer review provides a supportive environment to assess the progress of individual officers and
departments, and to provide feedback and coaching to promote quality improvement. For the
process to work effectively and efficiently, procedures must be in place for the selection and
training of peer reviewers, and for regular review and feedback for officers.
• Develop a procedure for assessing customer satisfaction.
Interventions intended to reduce recidivism are only useful if the offenders involved benefit from
the supervision and treatment that is provided. For example, if an offender did not find a
substance abuse group relevant to his or her addiction issues, then it is unlikely that he or she
will incorporate any information or skills from the group into his or her lifestyle. (This is true
whether the lack of interest is a result of the program content or the offender’s lack of
motivation. Either way, there is something that is preventing the offender from benefiting from
the program.) Therefore, it is important to survey offenders to determine their satisfaction with
probation services and the extent to which they benefited. This survey data offers one measure
of the effectiveness of service provision.
• Conduct ongoing program evaluation.
Program evaluation encompasses the measurement of outcome indicators, discussed above, as
well as process measures, which measure program implementation. The two types of data taken
together determine whether a program is meeting its goals, and which components of the
program are operating effectually or ineffectually in pursuit of that goal. Both types of measures
are important in the implementation of existing evidence-based practice or the creation of new
evidence-based practice.
• Conduct ongoing appraisals of staff performance.
The performance of individual staff has a significant effect on the quality of services that are
being provided. Ongoing staff appraisal keeps both staff and supervisors informed on the level
of an employee’s performance, areas of strength and weakness, and ongoing training needs.
When appraisal is ongoing, supervisors have the ability to provide regular feedback and coaching
to the benefit of the employee and the department.
• Incorporate quality assurance data into practice.
A key role of the steering committee or a designee must be to determine how the data will be
used. For example, if data indicates that less that half of offenders who should be referred to
substance abuse treatment are receiving treatment, does that mean that officers need more
training on determining treatment needs, or policies on referral follow-up need to be revised, or
that more partnerships with treatment providers need to be developed? Decisions must be made
about how to determine what the root cause of the issue is, and what steps will be taken to
remedy it.

8

PEER REVIEW
Overview
Ongoing quality assurance requires periodic review of the supervision practices and services
being provided to offenders. This review compares actual practices and service provision to the
benchmarks of quality established by the organization. A review can be conducted by someone
internal or external to the organization. An internal peer review process can be highly beneficial
when staff are well-trained in the process of peer review and industry standards for effective
assessment. This process employs a cross-section of staff, as opposed to only supervisors or
management, to conduct case file and service audits. Peer reviewers may participate voluntarily
or be assigned to the role, and the assignment may be ongoing or on a short-term, rotating basis.
Internal reviewers are familiar with the officers, the offender population, and departmental
procedures, and have a sense of the context in which the review is being conducted. In addition,
when deficiencies are identified, officers may be more accepting of constructive criticism and
coaching from a peer. Implementing the process can be difficult if officers are suspicious of the
process and how the information is going to be used; therefore, it is important to involve staff in
the process of creating a culture of quality in the organization.
A variety of methodologies can be used for the peer review process, but they each involve a
standardized review process to record data on key indicators. In the implementation of the eight
guiding principles for reducing recidivism, four components of peer review are instrumental:
review of scoring and inter-rater reliability on assessment tools; critique of motivational
interviewing skill balance; evaluation of cognitive-behavioral treatment groups, and review of
case files. These processes can identify achievements and deficiencies and inform quality
improvement efforts regarding individual and organizational capacity for accurate risk
assessment, the building of intrinsic motivation in offenders, and the provision of or referral to
appropriate treatment.
Principles
› An internal review process must be peer-driven.
A cross-section of staff must be involved in the entire process, from determining the
relevant outcomes to designing the assessment tool to analyzing results. Input from
various staff levels will increase the relevance of the process and the results, as well as
increasing staff commitment.
› The process must be support and coaching-oriented.
To reduce resistance and increase the chance of success, staff should view peer review as
an opportunity for professional development, not as a punitive process. Feedback from
the process should be supportive and constructive, and staff should be given the
opportunity to learn, practice, and be coached to improve performance.

9

› The process should create a culture of learning.
The implementation and evaluation of quality, evidenced-based practice is ongoing, so
staff members never reach the point of “perfection.” This idea of a never-ending process
may be frustrating for some, so the peer review process must create an environment that
promotes the value of ongoing learning and continuous improvement.
› The process should include a feedback loop.
Peer review is only useful if the data is applied. Therefore, the process needs to be
designed so that individuals and workgroups receive well-organized, timely data that can
be applied to practice. Practitioners must be able to communicate changing data needs to
the peer review team, as well as request additional feedback and evaluation as needed.

Components
Assessment Scoring and Inter-Rater Reliability
Many validated assessment tools are available for determining an offender’s risk of recidivism,
as well as identifying the criminogenic needs that influence recidivism risk. When used
correctly, assessment tools can identify individual supervision and treatment goals for an
offender, as well as defining risk levels for a group of offenders. Most tools are administered by
interview, and there is a significant potential for error when the interviewer is inexperienced or
poorly trained. For the individual offender, this could mean assignment to inappropriate levels
of supervision and treatment, and potentially an increase in recidivism potential. On a larger
level, inaccurate scoring affects the statistical measures of risk in a population, and skews
decisions about risk level and resource allocation.
There are two important factors to consider in quality scoring: reliability and validity:
•

Reliability is the extent to which everyone interprets questions and scoring in the same way.
For example, if fifty offenders are asked how many previous offenses they have committed,
some may interpret the question as meaning all of the offenses that they have committed,
while others may interpret it as only those for which they were arrested. When the question
can be interpreted in different ways, it lacks reliability. Inconsistent scoring also affects
reliability. If an offender lists all of his offenses and some scorers count them all and others
count only convictions, then the scoring lacks reliability.

•

Validity describes whether or not a tool truly measures what it is supposed to measure. For
example, offenders who are scored as “low-risk” on an assessment should have lower
recidivism rates than those who are “high-risk.” If the assigned risk level is predictive of the
offender’s behavior, then the tool is valid. Because each population has a different
distribution of risk levels, an instrument’s validity must be reassessed with every new
population (i.e. the instrument must be “validated”).

10

The first component of reliability is ensuring that all officers are trained to administer and score
the assessment in the same way. Ongoing reliability can be measured by case file review, direct
observation of interviews and scoring, or tape review of an interview. The resources available
and the type of information desired will influence the review method chosen. For example,
interviewing skills cannot be assessed through a case file review, but the file review may be
faster and less expensive than tape review.
Peer reviewers can check for several components of reliability:
• Was the interview guide used?
• Were all appropriate questions asked, and were complete responses recorded?
• Did the officer demonstrate good interviewing skills (open-ended questions, etc.)?
• Were the answers verified when possible (arrest records, other data sources)?
• Did the scoring reflect the answers given?
Peer reviewers must also be accurate and precise in their review process (for example, different
reviewers might have different definitions of “good” interview skills). Therefore, reviewers
should receive standardized training and be given a detailed checklist of what and how to assess.
The amount of time and resources devoted to scoring reviews will vary. The system may consist
of a random sample of interviews selected on a monthly or
quarterly basis in order to assess a cross-section of
interviews. Alternately, all officers could be assessed when
Maintaining
they first begin administering the assessment (either once or
inter-rater
multiple times), and then on a yearly basis thereafter.
reliability ensures
Regardless of the sampling methodology, the officers should
that assessment
receive the results of the assessment, and have opportunities
for feedback, coaching, additional training, and reinterview and
assessment, if necessary.
scoring is the

same for all
offenders,
regardless of the
interviewer.

Inter-rater reliability is a measure of whether or not different
officers would score an assessment in the same way. This is
important so that all offenders in the same community are
having their risk level and treatment needs assessed in the
same way. The results of an assessment should depend only
on the offender, not on the probation officer to whom he or she is assigned. For example, if
Officer A consistently scores offenders 4 points higher than Officer B, then Officer A will likely
have several more offenders that are considered medium and high risk, and they will be
supervised at a higher level than if they were on the caseload of Officer B. This makes it nearly
impossible to consistently match offenders to appropriate supervision and treatment.
Inter-rater reliability can be measured very easily by presenting multiple officers with the same
interview, having each score the assessment, comparing the scores, and discussing and resolving

11

any discrepancies. This process should be done in all assessment training, and can be used for
ongoing review in two ways:
•

To measure inter-rater reliability in a training environment, officers can view and score
a validated sample interview. The interview could be viewed on a videotape or DVD, or
interactively online. The scores can be submitted to the peer review team for
comparison to the validated scores.

•

To measure scoring in the field, officers could record an interview and submit the tape
and their scoring. A team of peer reviewers could independently score the interview
and compare their ratings to the officer’s.

After the review, the officer should be provided with feedback and coaching from the peer
review team or a designated coach, with opportunities for training and re-assessment. If there is
significant diversity of answers, policies and procedures may need to be revised, and training on
a wider scale may be needed to clarify scoring expectations.

Observation: In Person, Audio or Video?
Officers can be observed in three ways: direct observation during an interaction
with a client; an audiotape of an interaction; or a videotape. Any of the methods
can convey the basic information needed for an evaluation, but each method has
its benefits and drawbacks. It is up to the agency to decide which method is best
for providing the desired information while limiting the intrusiveness of the
observation. In some cases, there are industry standards for review. For example,
the standard for MI reviews is audiotape, while review of cognitive behavioral
treatment groups requires video or in-person observation.
Direct Observation
• May be more expensive if reviewer travel is required.
• No technological concerns (i.e. tapes with poor audio, etc.).
• More reviewer control of which interaction is reviewed.
• Only one opportunity to view and take notes on the interaction.
• Often considered most intrusive by officer and offender.
• Relationship with reviewer may impact officer’s behavior.
Audio
• Often considered less intrusive than video.
• Less expensive technology than video.
• Only allows evaluation of words and verbal cues.
• Allows for replay and feedback review with officer.
• Officer has more control over which interaction segment is reviewed.
Video
• Often considered more intrusive than audio.
• Most expensive technology.
• Allows for evaluation of officer’s words, verbal cues, and body language.
• Allows for replay and feedback review with officer.
• Officer has more control over which interaction segment is reviewed.

12

Sample: Interview Audit Form
Reviewer:

Date:

Interviewer:

1 = Poor 2 = Fair/Needs Improvement 3 = Good 4 = Excellent
1. Explanation of the purpose of the interview.

1

2

3

4

2. Established structure for the interview.

1

2

3

4

3. Adequate use of open-ended questions.

1

2

3

4

4. Avoidance of double-barreled questions.

1

2

3

4

5. Avoidance of biased/leading questions.

1

2

3

4

6. Adequate use of follow-up questions.

1

2

3

4

1

2

3

4

8. Interviewer overcame problems such as silence or excessive talking.

1

2

3

4

9. Interviewer used the interview guide.

1

2

3

4

10. Notes were made indicating why items were or were not scored.

1

2

3

4

11. Adequate documentation in the case of an override.

1

2

3

4

12. Treatment plan clearly relates to information captured in the assessment. 1

2

3

4

7. Avoided barriers to listening (such as moralizing, disagreeing,
Blaming, shaming, reinforcing).

Total score: ______ divided by _______ =
Reviewer Comments:

5th Judicial District, Department of Correctional Services, Des Moines, Iowa
13

Sample: Level of Service Inventory-Revised
Review Protocol
The following are the most common errors that assessors make when completing an LSI-R
assessment. When reviewing, you should look for the following items:
1.

Lack of interview guide (i.e. failure to use an interview guide)

2.

Yes/No questions and answers (i.e. when reviewing the interview guide or notes, look for
answers other than “yes”, “no”, “good”, “fine” et. This will usually indicate that the
interviewer is relying on closed-ended questions and is not tapping for qualitative
answers.)

3.

Ancient History (i.e., review the questions marked as “current” to be sure that the
information recorded is based on the most recent information.)1

4.

Collateral information (do you see discrepancies between the LSI-R and the Pre-sentence
investigation? Has the rap sheet been reviewed for scoring of the criminal history
section? Is there family information in the file that supports or refutes the scoring of the
LSI-R?)

5.

Inconsistencies between the case plan and the scoring of the LSI-R (i.e. a criminality
class is recommended, but the person was given a “2” or “3” in the area of
attitude/orientation. The offender has no contact with family members, but the
family/marital section is rated as pro-social. The person is on supervision for forgery and
the financial situation is rated as pro-social. These are areas to explore.)

6.

Failing to mark all of the boxes with either a “X” or circling the number.

7.

Adding the numbers incorrectly

8.

Blatant Scoring Errors
The following is a list of “default” scoring rule violations that most commonly are
missed:
a.
Person is on for a sex offense and item 10 is scored “no”.
b.
Person is unemployed and question 18, 19, 20 are not scored “0”.
c.
If a person’s parents are deceased, question 24 must be scored as “0”.
d.
If the assessment is done at time of discharge from jail or a residential facility and
the person does not know where he or she will be living, item 27 must be scored
“yes”.
e.
Question 29 must be scored yes if the person is residing in a residential center or
is incarcerated.
f.
If item 32 is scored yes, then items 35 and 36 must be scored yes.
g.
Questions 41-45 must be scored as “no” if there has been no usage within the last
year.
h.
If question 47 is scored “yes”, then question 46 must also be scored “yes”
5th Judicial District, Department of Correctional Services, Des Moines, Iowa

1

“Ancient History” refers to old or out-of-date information.

14

Motivational Interviewing Critique
Motivational Interviewing (MI) is a directive, client-centered counseling style for helping clients
explore and resolve ambivalence about behavior change. When used by a skilled officer, MI can
help increase an offender’s motivation to comply with supervision requirements, participate in
treatment, and address their criminogenic needs. When a more traditional and confrontational
interaction style is used, the officer can increase resistance and decrease motivation to change.
For many officers, learning and implementing MI is challenging, therefore ongoing support and
feedback are needed to ensure successful incorporation of the skills.
The Motivational Interviewing Treatment Integrity (MITI)
Code is a highly structured feedback system that is used in
MI research, the training of trainers, and as part of ongoing
professional development. A MITI critique uses a video or
audio tape of an MI interview at least 20 minutes in length.
The rater tracks the methods used in the interview, including
MI-adherent and non-adherent behavior, and the frequency
of these interviewer behaviors are used to compute a “skill
balance.” Raters also judge the tape based on global ratings,
such as genuineness and empathy. The interviewer receives
detailed feedback on all components of their interview, as
well as a skill balance rating and an explanation of the rating
form. In addition to the written feedback, the rater provides
verbal coaching to the interviewer. This type of critique has
been proven effective in improving the skills of MI
practitioners. To use the MITI correctly, peer reviewers
would need to participate in special training.
Another option is for the peer review team to develop their
own instrument for evaluating MI skills. Though this
instrument would not have the research support of the MITI,
it may better meet the needs of the peer review team and the
officers. At minimum, it should address:
•
•
•
•

What MI adherent techniques are being used, and
how often?
What MI non-adherent techniques are being used,
and how often?
What is the demeanor of the officer? Is he/she
showing empathy?
What is the demeanor of the offender? Is he/she
showing resistance? Motivation to change?

Regardless of the review
method chosen, the
same general peer
review guidelines apply:
•

A standardized review
format must be
developed and all peer
reviewers must be
trained to use it.

•

Reviews should be
done on a regular
basis (monthly,
quarterly, yearly),
either for all officers or
a sample of officers.

•

All reviewed officers
should receive
feedback, coaching,
and opportunities for
additional training and
re-assessment.

•

Widespread issues
should be addressed
with policy changes
and training for all
officers.

Specific feedback is very beneficial to someone learning MI, so any format that is used should
incorporate detailed feedback on specific skills.
15

VERSATILE MOTIVATIONAL INTERVIEWING CRITIQUE

Open Questions (Raw# =9)

26%

26%

Closed Questions (Raw# = 1)

3%

3%

Affirmations (Raw# = 3)

9%

9%

Reflections (Raw# = 10)

29%

29%

Summarizations (Raw# = 1)

3%

3%

Elicitations (Raw# = 6)

18%

18%

Teaching/Advice (Raw# = 4)

12%

12%

Confrontations (Raw# = 0)

0%

0%

SKILL BALANCE RATING

65%

65%
Agency
Avg. (n=1)

Agency Avg.
(n=1)

Date of Tape Review:
Time Length of Session:
Relationship of Participant(s) to
Client:

Officer

Probation Officer
Community Corrections
11/01/04
#VALUE!
#VALUE!

Officer

Q-Strings

CLINICAL SKILL MEASURES

Therapist Name:
Agency:
Date of Session:
Offender JIS ID:
Tape Reviewed By:

3+ Q's In a Row (Per Hour)

2.3

2.3

Largest Consequtive Q-String

4.0

4.0

GLOBAL RATINGS

Low
Acceptance

4.0

4.0

Egalitarianism

4.0

4.0

Empathy

3.0

3.0

Genuineness

5.0

5.0

Warmth

3.0

3.0

Spirit

5.0

5.0

0%

10%

20%

30%

40%

11/11/04
26
#VALUE!

50%

60%

70%

80%

90%

8

9

0

1

2

3

4

5

6

7

0

1

2

3

4

5

6

7 High

MIC - VERSATILE Page 1 of 5

100%

10

Change Talk Rates (Officer)

190.0
140.0
90.0
40.0
-10.0
-60.0

Desire
Per Hour

Ability
Per Hour

Reasons
Per Hour

Need
Per Hour

Commit
Per Hour

Tk. Steps
Per Hour

Total Chg. Talk
Per Hour

P. Officer

76.2

9.2

11.5

46.2

32.3

0.0

175.4

Average

11.1

6.0

11.3

11.4

10.0

9.5

70.8

MIC - VERSATILE Page 2 of 5

NOTES
General Comments
Officer, at 65, your skill balance shows that you have a good understanding of all of the MI skills and are able to use most of them in appropriate proportions. You
did a great job of remembering to reflect and affirm, asking many questions that produced enthusiastic change talk from the client. Nicely done!

Elicitations:
Your specific questions about the client's steps and strategies for change really helped him to express his desire for lasting change. The strength of a client's
comments expressing desire, ability, reasons, need, commitment, and taking steps regarding change are highly predictive of long-term positive behavior
modification, so offering clients the opportunity to express these sentiments is great. There were so many times that you were able to do this in quite a brief
interview. Nicely done!

Reflections/Summarizations:
One third of your interactions with the client were reflections, which is a great start! Ideally, reflections would make up about 56% of an MI interview because
reflections allow clients to know you are listening, allow them to hear what they have said to decide if that is how they really feel, and prompt them to give more
information that you need without making them feel interrogated by too many questions. Also, you were able to do one summary, which showed that you have a
good attention to detail. Using summaries a bit more often may help you to get clients to understand what you are asking for. For example, you were trying to get
him to say what his treatment needs were, but he kept talking about the same things. Summarizing with, "So you want your treatment to consist of staying
connected to your family and church friends, remembering and spending time with your daughter, and being open with me. What other structures will help?" may
help you to focus the client on the additional things you are looking for, in addition to letting him know you've heard him.
Affirmations:
About half of your affirmations really identified something that was important to the client and expressed your appreciation in ways that really built him up. These
were so well done! The other half were conditional: "I respect you if you are sincere" or "You seem to be off to a good start, I would like to see you make it and I
really think you can...if you can hold this attitude." While you are a probation officer and it is your job to make sure that sentiment translates into behavior,
conditional affirmations undermine the goal of the skill: to convey unconditional positive regard so that the client lowers defensiveness so you get better information.
You can express concern that the client keeps up his good start by using other skills, but when you affirm, see if you can stay with comments that are positive and
empathetic. That way the client will understand the message that you care about him and that it is important that he does well, not "I care about you only when you
do well." The latter feeling tends to make clients more reluctant to reach out to you when they are having a hard time.
Global Measures:
I gave you a mix of above and below average rankings for Global Measures because on one hand, you reflected well, remembered to affirm, elicited lots of change
talk, and avoided confrontation. On the other hand, some of your affirmations were conditional, your percentage of teaching was a bit high, and I am not sure if you
realize how flat and unvaried your tone was throughout. While you may deeply care about your clients, when they do not hear your voice go up and down with
emphasis, it may be hard for them to believe that you care and they may be less motivated to give you good information about themselves that you will need to help
them succeed. I encourage you to vary your tone and give a few more unconditional affirmations to establish rapport with clients, "earning the right" to correct them
later as needed.

Summary of Recommendations
Overall, you gave a good demonstration of the majority of the MI skills and showed that you are able to elicit change talk well. In future interviews,
focus on reflecting and affirming more and varying your tone so that client's sense your regard for them. Keep up the great work!
More Of….
Reflections, Affirmations, Summarizations

Less Of…..

Interesting….
No Comment

Teaching

MIC - VERSATILE Page 3 of 5

Skill Balance Detail*

SKILL BALANCE DETAIL: Probation Officer
OPEN (9)

CLSD (1)

REF (10)

SUM (1)

AFRM (3)

ELICIT (6)

TEACH (4)

CONF (0)

30%

3%

33%

3%

10%

20%

12%

0%

(1)

MODIFIED
DISTRIBUTION

(2)

COLLAPSED
DISTRIBUTION

33%

33%

33%

12%

0%

(3)

CATEGORY
PENALTIES

5.3%

23%

0%

0.07

0.00

(4)

OPEN:CLOSED
RATIO PENALTY

(5)

PENTALY
TOTAL

(6)

0%

FINAL SKILL
BALANCE

[O/C] 0.05 + [REF] 0.23 + [S/A/E] 0 + [TEACH] 0.07 + [CONF] 0 + [O:C RATIO] 0 = 0.35

1 - 0.35 = 0.65

Explanation of Skill Balance:

(1) Modified Distribution:
For purposes of the "Skill Balance", The first six MI categories: Open, Closed, Reflections, Summarizations, Affirmations and Elicitations are isolated from the
last two (i.e., the percentages in these categories will add-up to 100%. Regarding the latter two categories, "Teaching" and "Confrontations", the distributions
are calculated as a proportion of all interactions (i.e., across all 8 categories).

(2) Collapsed Distribution:
The following categores are collapsed and their percentages combined/added: (1) Open Questions and Closed Questions and (2) Summarizations,
Affirmations and Elicitations.

(3) Category Penalties
3 OPEN/CLOSED CATEGORY:
Any value in excess of 28% is assigned as a penalty (no penalty for falling short of 28%).
3 REFLECTIONS CATEGORY:
Any value short of 56% is assigned as a penalty (no penalty for exceeding 56%).
3 SUMMARIZATIONS/AFFIRMATIONS/ELICITATIONS
Any value short of 16% is assigned as a penalty (no penalty for exceeding 16%).
3 TEACHING:
Any value in excess of 5% is assigned as a penalty.
3 CONFRONTATIONS:
Any confrontation value is assigned as a penalty.

(4) OPEN:CLOSED Ratio Penalty
3 OPEN:CLOSED RATIO
When analyzing open & closed questions seperately, closed questions (ideally) should not exceed 30%. For any value in excess of 30% a
1 point (1%) penalty for every 5% of the excess is assigned.

(5) PENALTY TOTAL:
The sum of all penalties noted above.
(6) Final Skill Balance:
One (1.0) less the sum of all penalties.

MIC - VERSATILE Page 4 of 5

GRAPH KEY

Open Questions (Raw# = 44)

Agency Avg.
(n=43)

Open Questions (Previous Interview Score)

This green line represents
the "National Average"
score.

Interviewer

This column presents the specific skill percentage of
the interviewer's present critique score for the given
category ("open questions", in this example). It is
depicted in graph form (in gray) to the right.

72%

53%

This column presents the average skill percentage of the
interviewer's agency. The number of individuals comprising the
average is noted in the heading (n=). This value is depicted in
graph form (in red) to the right.

0%

10%

20%

If the interviewer has submitted a previous tape to be critiqued, the red
marker here indicates the interviewer's most recent score. In this
example, the interviewer's previous score was 34%. If the interviewer
has not previously submitted a tape, this yellow-shaded section will not
appear on the report.
30%

40%

50%

60%

70%

80%

90%

100%

,

The gray-shaded area represents the Interviewer's score
for the present critique. It corresponds with the grey
numeric value on the left, which, is 72% in this example.
The thin red line here indicates the average score of all individuals in the same
agency as the interviewer. In this example, the agency average is 53%, which
corresponds to the red numeric value to the left.

Explanation of Global Rating Categories
Acceptance (also called unconditional positive regard)
Interviewers high on this scale communicate acceptance and respect to the client. Acceptance is person-focused (unconditional positive regard) and
should not be confused with acceptance/approval of the person's behavior. Interviewers at the low-end of this scale may be perceived as judgmental,
harsh, disrespectful, labeling or condescending.

Egalitarianism (the opposite pole is authoritarianism)
Interviewers high on this scale emphasize by the words and manner the client's personal autonomy, choice, and responsibility. They may offer their
expertise when asked for it, or after obtaining the client's permission to inform or advise. Interviewers low on egalitarianism take a more authoritarian
approach of directing, ordering, blaming, threatening and confronting. There is a quality of the interviewer asserting a "one-up" position that implies, "I
know best. Listen to me."

Empathy (also called understanding)
Interviewers high on this scale are able to attain and communicate an accurate understanding of their client's perceptions, situation, meaning and feelings
through high-quality reflective listening. Their manner shows an active interest in an effort to understand the client's perspective, and their responses
actively express an attentive understanding of the client's perspective and experience. They probe to understand more fully, and reflect their
understanding back to the client. Interviewers at the low-end of this scale show little interest in or appreciation of the client's perspective, little overt
understanding or reflection of what the client is experiencing. They evidence little effort at seeking a deeper understanding of the client's perspective.
Interviewers low in empathy may ask many questions to gain factual information or pursue their agenda, but do not seek to understand the client's own
perspective.

Genuineness (also called congruence)

Interviewers high on this scale are perceived as open, responsive, and honest. The interviewer appears to be saying what he or she is experiencing in
this session. They show a quality of congruent transparency, saying what they feel and perceive in the moment. Their response to the client is individual
and personal. Do not confuse this with other scales such as acceptance or warmth. High genuineness, for example, can include expression of negative
affect or criticism. Interviewers low on this scale do not appear to be responding honestly and openly to the client, and may appear unresponsive or
phony. If they self-disclose, it may have the quality of talking about personal history rather than relating in the present. Their response may have flat,
closed, or technical-business quality, or may appear to be rote or mechanical.

Warmth
Interviewers high on this scale are perceived as warm, friendly, engaged, compassionate, helpful, caring and concerned. Interviewers at the low end of
this scale present an impression of being cold, distant, detached, or unfriendly, showing little overt evidence of helpful concern and compassion.

Spirit
This is an overall, global rating of the extent to which the interviewer manifests the fundamental spirit of motivational interviewing. It should not be
regarded merely as an average of the other scales, but rather the rater's judgment of the extent to which the interviewer "gets it", evidencing a grasp of the
"music" and not just the words and techniques. Interviewers high on this scale manifest a directive, client-centered style of facilitating, coaching, and
negotiating. The interviewer honors and values the client's perspective. There is a naturalness, comfort, and loving or artistic quality to the interviewer's
style. The interviewer is attuned to the client, and actively "mines for the client's own motivation." Interviewers low on this scale show a lack of the
balanced directive, client-centered style, erring on the side of passivity or of overcontrol (or both). On the passivity side, the interviewer misses or is
inattentive to significant client material, and may seem indifferent, isolated, ignoring, preoccupied, or detached. On the overcontrolling side, the
interviewer may communicate mistrust, disrespect, disregard, or simply the pursuit of the interviewer's own agenda without sufficiently involving the client.

MIC - VERSATILE Page 5 of 5

Cognitive-Behavioral Treatment Group Facilitation
Evidence-based practice tells us that cognitive-behavioral programming is one of the most
effective treatment modalities for encouraging behavior change in offenders. Many evaluated,
successful curricula are available for use with offenders in addressing a range of issues from
anger management to substance abuse to job skills. These curricula can be implemented in
community corrections settings, and can be facilitated by community-based treatment providers
or by community corrections officers.
In order for cognitive-behavioral programming to be effective, the facilitator must be welltrained, and the curricula must be implemented with fidelity to cognitive behavioral principles,
as well as the specific curriculum. For example, a job skills curriculum may have been proven
effective in one jurisdiction when offenders participated in twenty hours of programming and
used extensive role-plays. If the program is replicated in another jurisdiction in ten hours by
replacing role-plays with lectures from the facilitator, the program is unlikely to be successful.
The peer review process can be used to assess the implementation of cognitive-behavioral
groups, with several benefits: the facilitator can receive feedback on his or her skills; the
curriculum can be assessed for fidelity to the model; multiple models can be reviewed to
determine which is the most appropriate for client referral; and officers can better understand the
services that their clients are receiving. Peer review of treatment groups provides a reciprocal
benefit to the treatment provider and the officers who are referring clients to treatment.
Like other forms of peer review, the methodology must be chosen carefully depending on the
comfort level of the facilitator, the resources available, and the information that is being
assessed. There are models in the field for this type of assessment, such as the Correctional
Program Assessment Inventory (CPAI). Please see the appendix for a comprehensive example
of another assessment form. Like other assessments, review of cognitive programming should
include the following steps:
•
•
•

Use of a standardized assessment tool that addresses content, environment, and facilitation.
Well-trained reviewers conducting in-person or video review (audio is usually insufficient)
on a regular basis.
The opportunity to give feedback to the facilitator and the agency on the group.

Because treatment provision is multifaceted and often provided by a variety of providers, a team
approach can be very effective for this type of peer review. The team can be composed of staff
from state and local community corrections agencies and private providers. This way, the review
is conducted by a group with extensive and diverse experience, and all treatment providers,
public or private, will feel that they are being reviewed by a peer. This process can be used to
coach new facilitators, to evaluate new treatment providers and curricula, and to ensure quality in
programming over time. Like any other new type of appraisal, the facilitators must be involved
in developing the program and understanding its benefit, so that the peer review does not seem
punitive, or as a competitive process between different providers. At the same time, an
organization must have an expectation of consistent quality in the services that it offers.

21

Case File Review
The key component of evidence-based practice is ensuring that the level of supervision and
treatment that an offender receives is commensurate with his or her level of risk. The most
efficient way of assessing this is through periodic review of a random sample of case files. Wellmaintained case files or databases should contain the offender’s case history, assessment
interviewing and scoring, case plan, supervision notes, treatment referrals, and treatment
attendance, as well as any other documentation required by local laws and policies.
A peer review team is able to assess whether:
• Officers are maintaining adequate documentation.
• Offenders are receiving appropriate assessment, supervision, referrals, and treatment.
• Overall services are in compliance with the department’s vision, mission, and goals.
• Case plans are based on assessment data.
• Case plans are followed, and goals are met.
• Ongoing training, support, policy revision, etc. is necessary.
Case file review provides much of the data that are used as barometers of quality performance.
The process of case file review can be tailored to the size and needs of the organization,
following these basic steps:
1. A standardized case file review form is adapted or created,
taking into consideration the range of cases that will be
reviewed (i.e. juvenile and adult, offenders assigned to
various supervision types, and offenders participating in
various forms of specialized treatment). A cross-section of
staff should be involved in the creation of the instrument.
The instrument must be designed to measure the quality
assurance indicators that are being tracked.
2. The peer review team is recruited and trained.
3. A sampling plan is developed: How often will reviews
take place? How many files should be reviewed? In
general, the larger the sample, the more information in
garnered. Resource limitations must be considered.
4. Case review is completed, and the data is compiled and
compared to benchmarks (i.e. 80% of offenders referred to
sex offender treatment will complete the treatment).
Trends and deviations from the benchmarks, both positive
and negative, are noted.
5. The peer review team works with management to address
trends and deviations, as discussed below.

The Peer Review
Team

1. The team should be
drawn from the
population of officers
that they are reviewing.
2. Peer review can be the
reviewers’ sole
responsibility, or a part
of their job.
3. Participation in peer
review can be an
ongoing job
responsibility, or a
large group of staff can
serve on a rotating
basis.
4. The team should be
well trained on the
instruments that they
are using.
5. Team members need to
be as objective as
possible in reviewing
their peers, and cannot
review themselves.

The process of peer review involves more than collecting data;
the process needs to be monitored for efficacy, and the data needs to be put to work.

22

Sample: University of Cincinnati Risk Assessment
Note: This tool provides an “estimate” of an offender’s risk/needs assessment score, based on demographics and
history data. It can be used to quickly re-assess an offender’s risk level for comparison to the score determined by
the probation officer. This tool was validated for use with a population of offenders in Ohio. As always, this tool
needs to be validated on the population with which it is being used.

Client ID __________________
Risk Factor
Age
Education
Marital Status
Psychological Problems
Indicated
Alcohol Problem Ever
Drug Problem Ever
Unemployed at Arrest
Prior Arrests

Prior Incarcerations
Prior Conviction for
Violent Offense
Prior Conviction for Sex
Offense
Previous Community
Control Violation
Current Felony Degree
Current Offense Type

Categories and Weights
17-22
23-36
32+
< H.S. Graduate
> H.S. Graduate
Single
Married
Yes
No
Yes
No
Yes
No
Yes
No
2+
1
0
2+
1
0
Yes
No
Yes
No
Yes
No
3rd, 4th, 5th
2nd
1st
Drug, Property, Sex
Person or Other

Total
ˆ High Risk (76-115)
ˆ Moderate Risk (55-75)
Risk Category: _____________

Score

16.9
7.2
0
7.6
0
7.5
0
1.9
0
4.7
0
9.0
0
6.5
0
12.3
2.9
0
22.8
6.6
0
3.5
0
5.8
0
6.9
0
22.8
6.6
0
5
0

ˆ Low/Moderate Risk (38-54)
ˆ Low Risk (0-37)
Interviewer: _______________
23

Sample: Case Management Audit Instructions
NOTE: This case management audit pertains to case management issues only. Other issues related to
supervision but not to case management can be audited according to local policy.
1. Were the LSI, CMC/Jesness, and case plan completed within the prescribed time frames?
a. Focus: 60 days field and 30 days residential from date of assignment.
b. Source: Compare date of assignment and dates on LSI, Jesness, and case plan.
2. Is the problem prioritization consistent with LSI and CMC/Jesness?
a. Focus: Problems listed on front page of case plan should reflect results from LSI
and CMC/Jesness. Were the Big 4 (criminal history, anti-social companions, antisocial personality, and attitudes) considered in prioritization?
b. Source: Case plan, LSI, CMC, Jesness, generic notes, ask officer to explain rationale
for prioritization (SAQI, ranking, Big 4).
3. Is the problem behavior and the need that it serves correctly identified?
a. Focus: Behavior that is illegal or which leads directly to illegal behavior. What are
needs being met by illegal behavior or conditions under which illegal behavior
occurs?
b. Source: LSI interview notes, PSI, case plan, generic notes from meeting negotiating
case plan, ask officer to articulate needs and/or conditions, observation of
negotiation session.
4. Does this section contain the undesirable results of the offender’s behavior as articulated by the
offender (their motivation to change)?
a. Focus: Look at offender’s entire experience with CJ System – what is significant
negative result of this experience. From offender’s perspective!
b. Source: LSI interview guide, PSI, generic notes describing meeting when case plan
negotiated, observation of negotiation session, other evaluations.
5. Is the goal reasonable and measurable?
a. Focus: Medium or long term behavior change that is a result of intervention. Must
fit offender’s capabilities. Stated in a way that attainment of goal can be measured.
Internalized, not superficial. Include timeframes.
b. Source: LSI interview guide, Jesness, PSI, chronos describing meeting when case
plan negotiated, observation of negotiation session, other evaluations.
6. Do offender interventions, tasks, activities appropriately relate to goals? Do the interventions, tasks,
activities contain the methods, techniques, resources, and timeframes the offender will use to achieve
the stated goal?
a. Focus: Are tasks/activities consistent with and supportive of goals and
interventions? Does officer spell out how task will be completed and what resources
will be used? Are there time frames? Are they realistic?
b. Source: Case plan, chronos documenting sessions where tasks are discussed.

24

7. Does the benefits section clearly show meaningful/positive behavioral changes/rewards for the
offender? Are the benefits in contrast to undesirable results and do the benefits bear a relationship to
the prioritized need?
a. Focus: Congruency between undesirable behavior and the prioritized needs. Is the
benefit meaningful to the offender?
b. Source: LSI, CMC/Jesness, case plan, generic notes, observation of negotiation
session, ask officer to articulate.
8. Does the case management plan reflect intervention and supervision/monitoring strategies consistent
with the CMC/Jesness?
a. Focus: congruency between interventions, techniques, strategies, resources, goals,
tasks.
b. Source: CMC/Jesness type, wording of case plan components, specificity of goals,
tasks, expectations. Generic notes describing negotiation session. Observe
negotiation session. Ask officer to articulate how CMC/Jesness type specifically
applies to offender.
9. Do interventions target the criminogenic need in the right intensity and does the intervention use the
methods and strategies needed by the offender based on risk, need, and responsivity? Are contacts
appropriate and meaningful given the risk, need, and responsivity issues of the offender?
a. Focus: Intensity of interventions and supervision, amount of detail in tasks, degree
of planned follow-up/documentation, type and amount of planned contacts with
offender. How many contacts are personal vs. collateral? How long are personal
contacts? What is discussed in personal sessions? How often are contacts? Do all
of these things fit with CMC/Jesness type?
b. Source: LSI, CMC/Jesness, case plan, generic notes, observation of meetings, ask
officer to describe issues mentioned in focus section.
10. Is the case reviewed on an ongoing basis and does officer make appropriate ongoing case
adjustments, i.e. minor violations, rewards, case plan revisions, techniques, strategies?
a. Focus: Regarding case review: does officer go back periodically and review entire
case to get big picture of case performance. Review should emphasize congruency
between LSI, CMC/Jesness, case plan, response to supervision and treatment. Has
case stayed on the course first set out by case plan? Should it have? Regarding
ongoing case adjustment: focus is not on major responses to major events. Rather,
as officer learns more about offender and offender exhibits behavior during course
of supervision, does officer make appropriate adjustments in approach, type of
contacts, content of contacts, motivational techniques, rewarding techniques, etc.
b. Source: Generic notes, case plan, LSI, CMC/Jesness, staff case with officer (ask
what has learned about offender and how have made case adjustments).

25

11. Are critical incidents handled timely and appropriately, given risk, need, responsivity, and other
circumstances?
a. Focus: Critical incidents refer to those events that relate to case management issues.
Are they handled in a timely manner with risk, need, and responsivity factored into
immediate and subsequent responses.
b. Source: LSI, CMC/Jesness, chronos, ask officer.
12. Is the officer communicating with appropriate sources in order to have current, relevant knowledge of
the offender’s performance in programs?
a. Focus: Are lines of communication opened and maintained? Look at frequency and
content of contact and whether responsivity is considered.
b. Source: Generic notes, ask officer, ask program staff, attend staffings.
13. Does the officer have a basic understanding of the programs in which the offender is participating and
is this knowledge reflected by reinforcing programming goals in meetings with the offender?
a. Focus: Does officer understand programs well enough to have meaningful
conversations with program staff about performance and reinforcement? Does
officer understand programs well enough to have meaningful conversations with
offender about progress in program and to reinforce what ought to be reinforced?
b. Source: Chronos, training records, staffings, observe meetings, ask officer to explain
program to supervisor.
14. Is there a relapse prevention plan that is understood by the officer and discussed with the offender
during meetings?
a. Focus: Relapse prevention plan in global sense. Does a plan exist in some form?
Does officer understand dynamics of offender’s criminal behavior and what
behaviors to watch for to determine if offender is staying on right path or starting
down path to illegal behavior? Is officer anticipating behaviors based on his/her
understanding of offense dynamics? Are they initiating interventions as early as
possible and are they reflective of risk and responsivity? Is the plan discussed with
offender at meetings? Is there follow-up and verification?
b. Source: Chronos, staffings, discuss with officer, observation of meetings.
(auditing instrument follows on next page)

5th JUDICIAL DISTRICT DEPT. OF CORRECTIONAL SERVICES (Des Moines, Iowa)

26

Sample: CASE MANAGEMENT AUDIT
Case Manager Name:

Date:

Offender Name/ ICON Number:
Reviewer____________________________________ Date of Review_____________________
1. Were the LSI, CMC/Jesness, and Case Plan completed within the prescribed time frames?
5

CMC/JESNESS/LSI and Case Plan
completed with 30 days (residential), 60
days (field) of case assignment or
reassessment.

3

CMC/JESNESS/ LSI and Case
Plan completed within 45 days
(residential), 75 days (field) of
case assignment or
reassessment.

0

CMC/JESNESS/ LSI + Case
Plan completed later than 45
days (residential), 75 days
(field) of case assignment or
reassessment

Score

(1)

2. Is problem prioritization consistent with LSI and CMC/Jesness?
5

Problem prioritization consistent with
needs identified by LSI, CMC &
JESNESS.

3

Ranking questionable or not
supported by officer or
documentation.

0

Problem ranking inconsistent
with identified needs in
assessments.
(2)

Base ratings on items 3 through 7 on all active action plans.
3. Is the problem behavior and the need it serves correctly identified?
5

Problem behavior and need it serves
clearly identified and articulated on plan.

3

Only one criterion met or not
clearly articulated on plan.

0

Neither criteria met and not
clearly articulated on plan.
(3)

4. Does this section contain the undesirable (worst) results of the offender’s behavior as
articulated by the offender? (their motivation to change)
5

Results identified clearly specify the
undesirable consequences of the
behavior as articulated by the offender.

3

Results identified however the
relationship to offender’s stated
consequences is unclear.

0

Results not identified and/or
bear little if any relationship
to offender’s stated
consequences.
(4)

5. Is the goal reasonable and measurable?
5

Goal focuses on the behavior changes
the offender is capable of achieving and
expected to make as an outcome of the
case management interventions. Clearly
stated so as progress or completion can
be measured.

3

Goal focuses on merely
attendance/ completion not
behavior change or not clearly
stated thus progress hard to
measure.

0

Goal does not meet either
criteria.

(5)

6. Do interventions, tasks, and activities appropriately relate to goals?
Do the interventions, tasks, and activities contain the methods, techniques, resources, and
time frames the offender will use to achieve the stated goal?
5

Interventions, tasks and activities detail
how the offender will achieve and verify
the behavior changes. Resources are
identified and the frequency/duration of
the tasks/activities are identified.

3

Only one of the criterion is met.

0

Interventions, tasks and
Activities section meets
neither criteria.
(6)

27

7. Does the benefits section clearly show meaningful/positive behavioral changes/rewards for
the offender? Are the benefits in contrast to undesirable results and do the benefits bear a
relationship to the prioritized need?
5

Benefits contain meaningful positive
behavioral changes, which are in direct
contrast to the undesirable behavior and
are clearly tied to the prioritized need.

3

Only one of the criterion is met.

0

Benefits do not reflect
positive behavioral outcomes
and do not relate to the
prioritized need.

(7)

8. Does the case management plan reflect intervention and supervision/monitoring strategies
consistent with the CMC/Jesness?
8

Case plan interventions, supervision and
monitoring activities are consistent with
those most effective with the identified
CMC/Jesness type.

4

Only one criterion is met.

0

Neither criterion is met.

(8)

9. Do interventions target the criminogenic need in the right intensity and does the
intervention use the methods and strategies needed by the offender based on risk, need,
and responsivity? Are contacts appropriate and meaningful given the risk, need, and
responsivity issues of the offender?
10

Criminogenic need targeted in right
intensity. Intervention uses appropriate
methods and strategies.

5

Intensity okay or
methods/strategies okay.

0

Neither intensity nor
strategies appropriate.

(9)

10. Is the case reviewed on an ongoing basis and does officer make appropriate ongoing case
adjustments, i.e. minor violations, rewards, case plan revision, techniques, strategies?
10

Case adjustments ongoing and
appropriate.

5

Some problems with timeliness
and /or appropriateness.

0

Significant problems with
timeliness or
appropriateness.

(12)

11. Are critical incidents handled timely and appropriately, given risk, need, responsivity and
other circumstances?
9

Handled timely and appropriately with
all-important factors considered.

5

Some timeliness and/or
appropriateness problems.

0

Significant problems with
timeliness or
appropriateness.

(13)

12. Is the officer communicating with appropriate sources in order to have current, relevant
knowledge of the offender’s performance in programs?
9

Ongoing communication. Knowledge
of performance relevant and current.

5

Sporadic communication.
Problems with quality of
knowledge of performance.

0

Little or no communication.
Knowledge of performance
not current.

(14)

13. Does the officer have a basic understanding of the programs in which the offender is
participating and is this knowledge reflected by reinforcing programming goals in
meetings with the offender?
9

Thorough understanding of programs.
Goals reinforced in meetings.

5

Some problems with degree of
understanding and/or goals
reinforced sometimes.

0

Little understanding of
programs and/or goals not
discussed or reinforced.
(15)

14. Is there a relapse prevention plan that is understood by the officer and discussed with the
offender during meetings?
10

Plan exists, is understood by officer, and
discussed with offender.

5

Plan exists but understanding
insufficient and/or not often
discussed with offender.

0

No plan, little or no
understanding, little or no
discussion with offender.
(16)

TOTAL

28

SAMPLE PEER REVIEW AUDIT SHEET (FY 2004)
Open Case ο

Section 1: Demographics
Client Name:
Client Codap Number:
Date of Review: ___/___/___
Site: ❏ Program 1 ❏ Program 2
❏ Program 3
Primary CSP:
Client Race/Ethnicity:
❏ Caucasian ❏ African American ❏ Hispanic-American
❏ Native American/Eskimo ❏ Asian American
❏ Pacific Islander ❏ Biracial ❏ Other
Section 2: Intake/Assessment
Yes
No
1. Consent to treatment is complete within
2 business days of admission.
❏
❏
2. The client is appropriate for the program and should
have been admitted.
❏
❏
3. Intake section of record is complete, in proper sequence,
and documented in a timely manner.
❏
❏
4. Release/agreements section of the record is complete, in
proper sequence, and documented in a timely manner. ❏
❏
5. Chemical dependency assessment is complete within
14 days of admission to program.
❏
❏
Total
____ ____
Section 3: Treatment Planning
Yes
No
6. Treatment plan is appropriate to assessment.
❏
❏
7. Treatment goals are pertinent to assessed needs.
❏
❏
8. Treatment goals are written in measurable/objective
terms.
❏
❏
9. Individualized treatment plan is written within 7
days of completion of the assessment and contains:
❏
❏
a) signatures and dates ___
b)target dates for objectives/goals ___
c) reviewed and signed ___
10. Treatment plan revisions are appropriate and
completed every 30 days.
❏
❏
Total
____ ____

Closed Case ο

Section 4: Treatment Management
Yes
No
11. There is adequate documentation in clinical record
describing client status and service interventions.
❏
❏
12. Continued treatment and receipt of program services
are appropriate.
❏
❏
13. Adequate quantity and quality of physician and other
types of consultation according to recommendations
within service plans are documented.
❏
❏
14. Clinical section of the record is complete, in proper
sequence, and documented in timely manner.
❏
❏
15. Service Delivery section of record is complete,
in proper sequence, and documented in timely
fashion.
❏
❏
16. Individual progress notes reflect treatment plan
implementation.
❏
❏
17. All documentation is signed and the appropriate
credentials are held by the person providing or
supervising the service.
❏
❏
18. The service provider is only providing services
authorized by the governing authority.
❏
❏
Total
____ ____
Section 5: Discharge Planning
Yes
No
19. Appropriate continuity of care provided from
assessment through treatment, to discharge.
❏
❏
20. Discharge summary is complete and contains:
❏
❏
a) dates ___
b) signatures and credentials ___
c) reviewed and signed ___
d) level of care/services provided ___
e) client’s response to treatment ___
f) recommendations/referrals ___
g) client ID ___
Total

____ ____

Section 6: Miscellaneous
Yes
No
21. Movement/sign logs are accounted for and in proper
sequence.
❏
❏
Total
____ ____
Section 7: Employment/Financial/Vocational
Yes
No
22. Employment section of record is complete, in proper
sequence, and documented in timely manner.
❏
❏
23. Vocation section of record is complete, in proper
sequence, and documented in timely fashion.
❏
❏
24. Financial section of record is complete, in proper
sequence, and documented in timely fashion.
❏
❏
Total
____ ____
Section 8: Records
Yes
No
25. Group record is complete, in proper sequence, and
documented in timely fashion.
❏
❏
26. Urinalysis record is complete, in proper sequence, and
documented in timely manner.
❏
❏
27. Discipline record is complete, in proper sequence, and
documented in timely manner.
❏
❏
28. Medical record is complete, in proper sequence, and
documented in timely manner.
❏
❏
29. Referral record is complete, in proper sequence, and
documented in timely manner.
❏
❏
Total
____ ____
SCORING:
Total number of Yes Answers = _____
(Total number of Yes answers/29) x 100 =

Section 9: Reviewer’s Comments
Reviewer Comments:

Reviewer Signature

If record review is less than 100% accurate, an action plan must be
submitted that informs when the deficiencies within the record will be
corrected. Forms are due back to Peer Review Committee 10 working
days after receipt of form.

Date Due: ___/___/___ Return to:
Action Plan:

Supervisor Signature
Service Provider Signature

%

_____/_____/_____
Date

_____/_____/_____
Date
_____/_____/_____
Date

SAMPLE: PROGRAM/UNIT NAME
PEER REVIEW TRACKING FORM
For the Month of: _________________
Client ID

Reviewer

Primary PO

Date Peer Review

Date Due

Date Returned

31

QUALITY ASSURANCE INDICATORS
Overview
When evidence-based practice is implemented on a large scale, there may be hundreds of
outcomes that indicate success in various facets of the program, and thousands of pieces of data
that indicate progress towards those goals. If too much data is collected, it can be overwhelming
and indecipherable, as well as resource-intensive. If too little data is collected, then program
staff does not have an accurate sense of their progress and areas needing further development.
Developing a set of key indicators and institutionalizing a process for monitoring these measures
will assist organizations in gauging their progress towards implementation of evidence-based
practices. Ensuring that the fidelity and quality of service are in or at least moving toward
alignment with the organization’s plan and goals is essential to maintain momentum toward
change. Developing a small number of key indicators that can act as windows onto the
landscape of fidelity and quality of service helps to build accountability and maintain the
integrity of organizational goals.
Principles
› Identify key measures
Choosing the right measures to monitor is essential to tracking progress towards goals.
The program logic model should be used to identify key program processes and short,
intermediate, and long-term outcomes. Then the data that are necessary and sufficient to
measure those outcomes and process should be selected for inclusion in the assessment.
› Institutionalize measurement monitoring and discussion.
To collect accurate data on indicators in a timely manner, staff members must be
committed to the process and incorporate it into their daily routine. For this to happen,
staff must receive constant exposure to the process, the reasons for it, and their role in it.
This includes reference to quality assurance in staff meetings, supervision, and
performance reviews.
› Share the information with all levels of staff
Staff members will not be able to support and participate in a process that they do not
understand. Provide formal and informational training, communication, and modeling so
that all staff understands the reason for the quality assurance process, as well as how the
process is working at the organizational level.
› Provide positive feedback and celebrate achievement
In the quality assurance process it is easy to become focused on deficiencies, or on goals
that are several years from attainment. To maintain morale and keep staff members
committed to the process, it is important to celebrate quality work (as reflected in
performance data) and achievement of short-term outcomes.

32

Components
• Use a logic model to identify appropriate measures
The program logic model (discussed in detail in the Program Evaluation section of the manual)
provides information on the processes and outcomes that are necessary to achieving a program’s
goal. For example, if a goal is to have all sex offenders complete a community-based treatment
program, intermediate measures may include making
Sample Measures
contact with potential providers and referring offenders
to the treatment. Measuring one of these “checkpoints”
1) Assessment
can provide information on progress towards program
o % of population with
goals. The logic model should be used to identify
completed assessment.
intermediate measures, and choose the information that
o % of population with recan be tracked on a regular basis to indicate that
assessment completed
progress. In our example, referrals to a treatment
within time frame identified
program might be easy to track, and may be a strong
by policy
indicator of the number of offenders who will complete
o Gain Score (changes in
protective measure score)
the program.
•

Develop regular and easily accessible reporting
capabilities
For data to be useful in monitoring quality, these data
must be accessible to the staff members responsible for
reporting, and the means of reporting must be easy to
use. The more difficult the reporting process, the less
likely that the required data will be reported on a
regular basis.
o Choose indicators that are easy to report on a
regular basis.
Sometimes this will mean
developing a plan to make data more accessible.

2) Case Plans
o % of medium to high-risk
offenders that have case
plans
o % of case plans that
address the top 3
criminogenic needs
according to the
assessment.
3) Average length on
supervision by risk level.

o

Develop standardized reporting instruments, pilot
test them, and solicit feedback from those who
will have to use them. Involve staff from across
the agency to increase buy-in and commitment.

4) Revocations
o Number of technical
violations resulting in
revocation to jail.
o Number of technical
violations resulting in
revocation to prison.

o

Increase technological capabilities. Web-based
reporting tools can increase the ease of reporting,
compiling, and analyzing data, but only if staff
has access to computers and the internet.

5) Treatment
o % of high-risk offenders
referred to treatment.
o % of high risk offenders that
attended treatment.

o

Provide
comprehensive
training
documentation on the reporting process.

and

o

% of total population that
attended treatment that are
high-risk.

o Adhere to deadlines. Reward staff members or departments for on-time and accurate
reporting and enforce consequences for those that don’t adhere to deadlines.

33

SAMPLE: EMPLOYMENT DATA
BENCHMARK: 85% OF CASELOAD WILL HAVE EMPLOYMENT DATA ENTERED
INTO CIS:
PERCENT OF MULTNOMAH COUNTY
CASELOAD WITH EMPLOYMENT DATA ENTERED
100%
90%
80%
BENCHMARK =
70%
85%
60%
50%
40%
30%
20%
10%

N
ov
D 02
ec
Ja 02
nFe 03
bM 03
ar
Ap 03
r-0
M 3
ay
Ju 03
n0
Ju 3
l-0
Au 3
gSe 03
pO 03
ct
N 03
ov
D 03
ec
Ja 03
nFe 04
bM 04
ar
Ap 04
r-0
M 4
ay
Ju 04
n0
Ju 4
l-0
Au 4
gSe 04
pO 04
ct
N 04
ov
D 04
ec
Ja 04
nFe 05
bM 05
ar
Ap 05
r-0
M 5
ay
-0
5

0%

Series1

Nov-02

Feb-03

Mar-03

May-03

Jun-03

Jul-03

Aug-03

Sep-03

Dec-03

Mar-04

Jun-04

Feb-05

May-05

64%

75%

72%

73%

80%

90%

84%

85%

85%

84%

94%

87%

94%

Multnomah County Department of Community Justice

34

SUGGESTED NEW EMPLOYMENT BENCHMARK:
XX% OF ACTIVE CASELOAD WILL BE EMPLOYED
MAY 2005
PERCENT OFFENDERS
EMPLOYED

PERCENT WITH DATA
ENTERED

OVERALL

34%

94%

OFFICE 1

28%

98%

OFFICE 2

38%

97%

OFFICE 3

50%

91%

OFFICE 4

98%

98%

OFFICE 5

22%

94%

OFFICE 6

35%

97%

OFFICE 7

32%

95%

OFFICE 8

35%

97%

OFFICE 9

39%

96%

OFFICE 10

10%

93%

OFFICE 11

33%

94%

OFFICE 12

25%

99%

OFFICE 13

23%

91%

OFFICE 14

52%

94%

Multnomah County Department of Community Justice

35

UNIT ACTIVITY: TREATMENT MODULE
BENCHMARK: 50% OF CASELOAD WILL HAVE AT LEAST ONE ENTRY INTO
THE TREATMENT MODULE
PERCENT OF ACTIVE CASELOAD
WITH TREATMENT MODULE ACTIVITY
100%

no activity
module activity

80%

60%

BE NCHM ARK =

40%

20%

0%

OFFENDER REFERRAL TO. ENTRY INTO,
PARTICIPATION IN AND EXIT FROM TREATMENT
2500
2 2 0 2 2 19 5

2000

Mar-03

Sep-03

Dec-03

May-04

Jan-05

Apr-05

Feb-04

2021

19 8 3
18 3 8

17 7 6

17 8 7

1500

1000

500

2 9 2 3 17 2 8 5

202

82

14 6 17 3

9 7 12 1

18 8 14 5
13 7

226
18 9
13 7 14 6
13 0 16 9
12 7

84 97

0

referral

entry

in tx

Multnomah County Department of Community Justice

exit

36

UNIT ACTIVITY: TREATMENT MODULE
April 2005

OVERALL

Referred

Entered

Open

Exit

173

97

1787

146

1

22

2

OFFICE 1
OFFICE 2

12

9

109

12

OFFICE 3

48

8

339

22

2

33

OFFICE 4
OFFICE 5

13

13

112

13

OFFICE 6

21

14

93

13

OFFICE 7

8

10

101

10

OFFICE 8

10

4

23

4

OFFICE 9

7

1

67

9

OFFICE 10

2

1

119

1

OFFICE 11

11

10

109

14

OFFICE 12

1

21

1

OFFICE 13

15

7

69

25

OFFICE 14

16

17

500

13

OFFICE 15

9

1

70

7

Multnomah County Department of Community Justice

37

UNIT ACTIVITY: NEEDS ASSESSMENT AND CASE PLAN
NEEDS ASSESSMENT AND
CASE PLAN UTILIZATION
(RST AND MTDC EXCLUDED)

90%

Feb-05

May-05

78%

80%

71%

71%
70%
61%
60%

54%

54%
51%
50%
42%
40%

30%
22%
20%
11%
10%

0%
percent of offenders
with needs assessment

Definitions:

•
•

percent of offenders
with timely needs
assessment

percent of offenders
with timely case plan
review

percent of offenders
with open case plan

percent of offenders
with active case plan

Timely = every six months
Open case plan = case plan that has at least one objective or requirement with no initial
action date. An objective is a referral or direction to Offender in regard to objective.

Active case plan = plan that has an initial action date on at least one objective that has not been
completed. An objective is a referral or direction to Offender in regard to objective.

Multnomah County Department of Community Justice

38

UNIT ACTIVITY: NEEDS ASSESSMENT AND CASE PLAN
May 2005
PERCENT OF
OFFENDERS
WITH NEEDS
ASSESSMENT

PERCENT OF
OFFENDERS
WITH ACTIVE
CASE PLAN

PERCENT OF
OFFENDERS
WITH TIMELY
CASE PLAN
REVIEW

PERCENT OF
OFFENDERS
WITH TIMELY
NEEDS
ASSESSMENT

PERCENT OF
OFFENDERS
WITH OPEN
CASE PLAN

OVERALL

78%

54%

22%

71%

54%

OFFICE 1

75%

56%

34%

60%

55%

OFFICE 2

88%

80%

38%

81%

63%

OFFICE 3

68%

32%

19%

62%

53%

OFFICE 4

84%

63%

5%

76%

66%

OFFICE 5

72%

19%

1%

66%

51%

OFFICE 6

81%

64%

16%

76%

45%

OFFICE 7

73%

51%

11%

66%

60%

OFFICE 8

89%

68%

38%

84%

79%

OFFICE 9

79%

74%

31%

73%

64%

OFFICE 10

80%

71%

10%

76%

59%

OFFICE 11

70%

50%

13%

57%

45%

OFFICE 12

87%

37%

248%

85%

33%

OFFICE 13

76%

54%

9%

65%

42%

OFFICE 14

87%

57%

40%

83%

57%

Multnomah County Department of Community Justice
39

RISK ASSESSMENT
BENCHMARK: 80% OF RISK ASSESSMENTS WILL BE COMPLETED WITHIN ONE
MONTH OF DUE DATE.

PERCENT OF ACTIVE CASELOAD WITH NEEDS ASSESSMENT COMPLETED
(MTCB, MTDC AND MTDV DSP CASELOADS EXCLUDED)
90%

77%
80%

73%

84%

83%

68%

70%
60%
50%
40%
30%
20%
10%
0%

Dec-03

Mar-04

1
Aug-04

Feb-05

May-05

Multnomah County Department of Community Justice
40

CUSTOMER SATISFACTION
Overview
Research has shown that offenders are more likely to change their behavior and less likely to
recidivate if they receive positive reinforcement. Assessing customer (i.e. offender) satisfaction
with the experience while on probation gives a sense of whether or not the offender feels that he
or she was treated with fairness and respect, and whether positive reinforcement was provided
when the offender maintained compliance with conditions. Therefore, customer satisfaction
surveys are one way of measuring alignment with evidence-based principles.
The basic procedure for assessing customer satisfaction is very simple: develop a questionnaire,
choose a sample of offenders, administer the questionnaire, and analyze the results. However,
each step has its challenges. A survey questionnaire must be carefully designed to elicit the
desired information, and careful sampling and administration are required to ensure that a
representative sample of offenders responds to the questionnaire. Finally, it is important that
feedback from customers (offenders) is considered and incorporated into practice where
appropriate.
Principles
› The sample of offenders surveyed must be representative.
All offenders must have an equal opportunity to be selected for the sample, and to have
an opportunity to complete a survey. This includes offenders who are incarcerated or
who drop out of treatment before completion.
› Questions should be consistent with quality assurance indicators.
Questions should be carefully chosen to provide data necessary to measure quality,
without the survey being too long and collecting irrelevant information.
› Offenders must be able to respond honestly without fear of retribution.
The data is only useful if it is honest, so responses should be anonymous. Offenders must
also trust that their responses are truly anonymous, and that there will not be
consequences for negative feedback.
› Results should be used to improve program quality and efficacy.
Results should be made available to all of the stakeholders charged with program
improvement and quality assurance. This data should be considered in program
evaluation and decisions on program and policy changes.

41

Components
•

Who will be surveyed?
A sample size is determined based on available resources: will all offenders be surveyed, or
only a sample? Will each offender be surveyed only once, or multiple times during his or her
involvement with community corrections? When choosing a sample, it is important that all
offenders involved with community corrections have the opportunity to be included in the
sample (i.e. select participants from the entire census, not just a “convenience sample” from a
certain jurisdiction or treatment group). Once a sample is selected, it is important that all
sampled offenders are given the opportunity to complete the survey, regardless of the reason
they leave probation (i.e. re-arrest or incarceration). Otherwise, only offenders who
successfully complete their probation will be sampled, and that group may not have the same
viewpoint as all offenders. If a certain group is unavailable for survey, then their exclusion
must be reported.

•

When will the surveys be completed?
This relates to the issue of sampling. If multiple surveys are going to be used, when should
they be administered? This will likely vary based on the average length of probation for
potential participants. If only one survey is being used, it could be administered universally
on a certain date (a “snapshot” of satisfaction across the agency on that date), or at a certain
point in the offender’s probationary period. Surveys administered at the end of probation are
often called “exit interviews.” Exit interviews are common because they allow an offender
to evaluate his or her entire experience with less fear of retribution. Also, administering a
single survey such as an exit interview requires fewer resources than multiple surveys.

•

What questions will be included?
As with all other instruments, the questions should reflect the indicators of quality for the
department. For example, did offenders feel they were referred to treatment in a timely
manner? Did they feel heard and part of the decision
making process? Have they re-offended while on
Survey questions
probation? Questions must be chosen carefully so that
the survey is not too long (which would decrease
must be chosen
completion rates), but still offers enough information to
carefully and
be meaningful.

pilot tested with
the target
audience.

Developing a survey instrument that meets the needs of
the agency and its customers can be complicated.
Careful consideration must be given to the instructions,
questions, and format of the survey. For example, respondents with low literacy levels or
speakers of other languages may have difficulty with complex language. The survey will
also need to be pilot tested for reliability and validity. A staff member or consultant with
experience in survey development should participate in developing the tool.

42

•

How will the survey be administered?
The way in which the survey is administered will influence whether the offender receives the
survey, completes it, and returns it in a timely manner. Also, the method must ensure
confidentiality. There are many methods for administering surveys, but the one that lends
itself to confidential exit interviews is self-administered mail surveys. This type of survey
can either be given to the offender in person or mailed to his or her home. The offender is
given the survey and a self-addressed, stamped envelop in which to return it. The survey will
not identify the offender by name, but it may use a number to track which surveys are
returned and which are not.

Mail Surveys: Benefits and Drawbacks
Benefits:

•
•
•
•

Confidentiality is ensured when no identifying information is on the survey.
Offenders can complete the survey at their own pace.
Mail surveys are less resource-intensive than in-person interviews or phone
surveys, so a larger sample size can be used.
Treatment providers can distribute the surveys as well as probation staff.

Drawbacks:
•
•
•

•
•

•

Mail surveys have low response rates compared to other survey methods,
especially when no reminders are given.
Literacy or language issues may prevent an offender from responding.
Surveys sent to offenders’ homes might not reach them.
Offenders may not feel motivated to respond honestly, completely, or at all.
(This is a risk with any survey; a compelling cover letter or other incentive
may be needed.)
Without tracking the survey, there is no way to determine whether certain
groups of offenders do not respond. (For example, results would be skewed
if offenders who are subsequently incarcerated never respond to a survey
about treatment effectiveness.)

How will the results be used?
The survey questions should be designed to inform the quality assurance process, determine
if the agency is providing professional, respectful services, and to determine if supervision
and treatment are addressing an offender’s criminogenic needs and preventing re-offending.
The results should be analyzed and reported in a useful way, ideally as part of the overall
data management system. The stakeholders in the quality assurance process, including the
peer review team, management, and any advisory groups, should use this data to inform their
review of key indicators and development of new best practice.

43

Interview #____

Interviewers Name: _____________
Interview Questions

1)

What is your gender?
Male: _____

Female: _____

2)

Please indicate your age: ______

3)

What is your relationship status?
Not in a relationship: _____

4)

In a relationship: ____ How long: _____

Which do you most strongly identify with?
1. ___ White (Non-Hispanic)

8. ___ Hispanic – Puerto Rican

2. ___ Black (Non-Hispanic)

9. ___ Other Hispanic

3. ___ Native American

10. ___ Asian

4. ___ Alaskan Native

11. ___ Southeast Asian

5. ___ Asian or Pacific Islander

12. ___ Mixed

6. ___ Hispanic – Mexican

13. ___ Other: ___________

7. ____ Hispanic - Cuban
5)

Is English your primary language?

5a)

If no, which language is? __________

6)

Was language a barrier during your supervision process?
Yes: _____

7)

Yes: _____

No: _____

No: _____

What is the highest grade you have completed? ____________________
1. ___ Less than High School

4. ___ Some College

2. ___ High School Degree

5. ___ College Degree

3. ___ GED Completion

6. ___ Trade School Certificate
44

7a)

Were you home schooled?
How many years?

8)

No: _____

_____

Have you ever had an immediate family member in the corrections system?
Yes: _____

9)

Yes: _____

No: ______

Before your most recent arrest, did you have any close friends involved in the corrections
system?
Yes: _____

10)

No: _____

At what age did you first associate with friends or family in the Criminal Justice system?
_____

11)

Are you on… Probation: ______ or Post-Prison Supervision (Parole): ________?

12)

What crime were you most recently convicted of? ______________________

13)

Is this your first time on supervision?
Yes: _____

13a)

No: _____

How long have you been on supervision? (since the last conviction)
_____ months

14)

Have you been convicted of or found in violation of your supervision conditions
( this cycle) ? Yes: _____

14a)

No: _____

More than once: _____

What was the sanction (what happened)?
1) Jail _____ (# of days) _____

5) Counseling _____

2) Community Service

6) House Arrest _____

_____

3) Forest Project _____

7) Increased Reporting _____

4) Work Release _____

8) Revoke _____
45

15)

Were you employed at the time of your arrest?
Yes: _____

15a)

15b)

16)

No: _____

What is your current employment status?
1. ____Full-time (35 hours or more)

3. ____Irregular (less than 17 hours)

2. ____Part-time (17 – 34 hours)

4. ____Not working or looking for work

If unemployed, what is your reason for unemployment?
1. ___ No desire to work

7. ___ In treatment

2. ___ Student

8. ___ Incarcerated

3. ___ Homemaker

9. ___ Seasonal Worker

4. ___ Retired

10. ___ Temporarily Laid-Off

5. ___ Physical reasons

11. ___ Supported by other

6. ___ Mental illness

12. ___ Looking for work

What programs have you been involved with while under supervision?
1. _____Counseling

If yes, what type? ____________

2. _____Community Service

8. _____ Forest Project

3. _____Victims Panel

9. _____ Drug/Alcohol Treatment

4. _____Cognitive Restructuring

10. _____The Learning Center

5. _____One-Stop Employment

11. _____Other

6. _____Day Reporting Center

12. _____ None

7. _____Work Release

13. _____ Pending Referral

46

17)

17a)

How long have you been with your current P.O.?

__________(months)

On a scale of 1-4 please rate your current Probation/Parole Officer’s performance.
1 = Poor, 2 = Fair, 3 = Good, 4 = Very Good
Timeliness
1 - 2 - 3 - 4
Availability
1 - 2 - 3 - 4
Understanding of your situation
1 - 2 - 3 - 4
Ability to relate to you in a respectful manner
1 - 2 - 3 - 4
Informative about programs
1- 2 - 3 - 4

18)

Have you had more than one Probation/Parole Officer during this supervision?
Yes: _________________________ No: ___________________

19)

If yes, has it been problematic?

20)

Regarding your experience with the supervision process in general, would you describe it
as…

Positive ____ or

Yes: _____

No:_____

Negative _____

Why? __________________________________________________________________

20a)

Has your perception of the supervision process changed since the beginning of your
current supervision?

Yes: _____

No: _____

47

21)

Currently, what is your level of concern in the following areas?
Please answer low, high or none.

Low:_____

Finding a Job
High:_____
None:_____

Managing your finances
Low:_____

High:_____

None:_____

Returning to a neighborhood where criminal activity is common
Low:_____

High:_____

None:_____

Community rejection
Low:_____

High:_____

None:_____

Family rejection
Low:_____

High:_____

None: _____

Facing situations where criminal behavior might be tempting
Low:_____

High:_____

None:_____

Facing situations where drugs or alcohol are easily available
Low:_____

High:_____

None:_____

48

22)

On a scale of 1-4 please rate how your supervision was helpful in the following

areas……………………..
Reducing attitudes that might lead to crime.
1–2–3–4
Increasing educational opportunities.
1–2–3–4
Reducing interest in hanging out with individuals that might be involved with crime.
1–2–3–4
Offering job assistance or employment training.
1–2–3–4
Learning new problem solving skills.
1–2–3–4
Developing an understanding of your impact on the community.
1- 2 – 3 – 4
Developing an understanding that even if family and friends commit crime their actions
should not be followed.
1–2–3–4
Please rate your over all experience with the people in probation and parole
1–2–3–4
23)

Was there anything that you feel you need to add, or that you feel we might have left

out?_________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
Multnomah County Department of Community Justice/Portland State University
49

PROGRAM EVALUATION
Overview
Program evaluation is beneficial in several ways:
• It allows a program to be “tracked” from development through implementation, to ensure that
it has been implemented faithfully.
• It requires that stakeholders carefully define “success” and decide how to measure it.
• It measures a program’s progress towards its goals and objectives, letting stakeholders know
when success has been achieved, or when change is necessary.
• It allows new “best practices” to be created, as organizations can cite evidence to prove their
success.
Evaluation methodologies too numerous to count are available to an organization that wishes to
define and measure success. An evaluation strategy can be tailored to the type of program, its
size, goals, and available resources. Evaluations are most often used in three ways: to pilot test
new or adapted programs or materials, called formative evaluation; to test the implementation of
a program, called process evaluation; and to measure progress towards outcomes, called outcome
evaluation. The design of a comprehensive evaluation plan is beyond the scope of this manual,
but a general overview of the process is provided.
Principles
› Evaluation planning should begin when program planning begins.
The evaluation process is much easier, more comprehensive, and more accurate if
evaluation activities are incorporated into the program from the outset. Also, the
systematic process of designing and implementing an evaluation often results in ongoing
program improvement.
› All key stakeholders should be involved in the development of the evaluation plan.
Input from many people will help ensure that the optimal evaluation design is chosen,
and that staff and resources will be committed to following through with the evaluation.
› A knowledgeable evaluator should guide the process.
Evaluations can be very complex, and many decisions must be made to balance the needs
of stakeholders, available resources, and sound research. An experienced evaluator is
required to do this successfully.
› The more rigorous the evaluation methodology, the more reliable the results.
The more carefully the evaluation is conducted, the more meaningful the results are. This
can have implications not only for determining if the program was successful, but also for
evaluating the fidelity to the program model, whether or not a new “best practice” has
been created, and whether the program is a good candidate for funding or replication.

50

Components
• Formative Evaluation
Formative evaluation, or “pilot testing” is the process of testing program components on a small
group and soliciting feedback before the program is implemented on a larger scale. Formative
evaluation is an important step when implementing a new
program, or when adapting an existing program for a new
Qualitative Data
population. Though it requires more resources initially, formative
are the answers to
evaluation saves resources in the long term by finding and
open-ended
addressing problems that would reduce a program’s efficacy. For
questions.
example, if a small group of officers is trained on an assessment
Ex.: What aspects of
tool and then can give feedback on the training, then the training
the treatment did you
can be improved before all officers are trained.
find helpful?
Any response is
Formative evaluation is often done by collecting qualitative (openpossible. Results
ended) data, so that respondents are able to give whatever
are analyzed by
feedback they feel it appropriate. Sometimes the feedback
grouping similar
provided is completely unexpected, so it is important that
responses.
respondents are not limited in their response options. The data is
usually collected through focus groups, individual interviews, or
written questionnaires after participating in the program being tested. As with survey data, it is
important that the questions are carefully written.
Evaluators could test anything from a worksheet to an entire curriculum. Participants in a
formative evaluation are usually a convenience sample, such as probationers from one region,
rather than a random sample. A convenience sample is less resource intensive and allows
evaluators to target a specific group (for example, testing readability of materials with offenders
for whom English is a second language).
• Process Evaluation
Process evaluation measures the implementation of a program, including whether it was
implemented as intended, whether the intended audience participated, and whether participants
were satisfied with the program.
The first step of a process evaluation is choosing what questions should be answered. In many
cases, the process evaluation questions will align with quality assurance indicators. Are
assessments and case plans being completed for all offenders? Are offenders attending
treatment? Are offenders satisfied with treatment? The number of process questions asked
depends on what is being measured and the resources available to track the answers.

51

The second step is determining how the data is going to be collected. Most process data is
collected using paper or electronic tracking forms that are completed and submitted on a regular
basis, either by the staff member completing the task or an
evaluator reviewing the task. For example, an officer may
Quantitative Data
track how many intake assessments he or she does each
are the answers to
month, then a member of the peer review team may track the
closed-ended
accuracy and completeness of the assessment. Some process
questions.
data may be tracked by survey, such as participant
Ex.: On a scale of 1 to 5,
satisfaction with an activity. Process data may be qualitative
how would you rate the
or quantitative (close-ended). In most cases, quantitative
instructor’s level of
data is collected because it is much easier to gather, analyze,
knowledge?
and report. Some process measures, like client satisfaction,
are amenable to qualitative data collection if resources
Responses are limited.
permit.
Results are analyzed
by assigning
numerical values to
The final step is determining how the data is going to be
responses and using
used. The data that is gathered must be able to be compiled
statistical analysis.
and reported in a timely manner, so that it can be used in a
timely way. Electronic information systems are very helpful
with this. Then, as with all other data, stakeholders must decide how it will be applied. The
stakeholders may also decide to change the evaluation process if the data that they are receiving
is not meeting their needs. However, once evaluation methods have changed, new data cannot
be compared to old data.
• Outcome Evaluation
Outcome evaluation measures whether a program achieved its stated goals. The first step in
conducting this type of evaluation is determining which of the program’s outcomes should be
measured by the evaluation. This could be a short-term outcome, such as an increase in offender
knowledge after one session of group therapy, or a longer-term outcome, such as recidivism. In
general, it is more difficult to measure longer-term goals, and to prove that they are a result of
the program. For example, an evaluator may have to choose between measuring the goal “the
offender will not recidivate during the period of probation” or “the offender will not recidivate
during the two years following probation.” The program manager may want to know whether or
not the program has an impact beyond probation. However, it is much more difficult to keep
track of offenders after they leave probation, to accurately measure their rates of recidivism, and
to be sure that their lack of recidivism is due to the program and not due to something that
happened after probation. These longer-term evaluations provide valuable data, but are much
more resource intensive. So, depending on the resources allocated to the evaluation (and the
timeframe in which results are expected), realistic outcomes should be chosen.
The second step is determining how the data is going to be collected. There are many designs to
choose from and many possible methods of collecting data. Some methods are very inexpensive
but provide questionable proof that a program works, such as a convenience sample where

52

probation officers occasionally ask offenders if the have re-offended. Others are very
methodologically rigorous, but more resource intensive. For example, evaluators could follow a
group of offenders who participated in a treatment program and a similar group of offenders who
did not participate in treatment, and on a monthly basis interview
the offenders and check police reports and court documents for
Good programs
evidence of re-offense. This would be more reliable, but also
become
requires the dedication of more resources. In addition, there are
“Evidence-Based
ethical considerations in design, as the methods cannot violate
offender rights. However, some very simple, effective measures
Practice”
are available, such as conducting a pre- and posttest to measure
through good
the amount of knowledge participants gained in a treatment
outcome
group, and comparing offenders who completed treatment to
evaluation.
those who did not.
Again, the final question is how the data will be used. If the results indicate that the program
was successful and implemented according to plan, the evaluation results could be used to
publicize new best practices. If the outcomes were not achieved, the results can be used to
examine why. In most cases, one set of evaluation results will present more questions, and lead
to another, more detailed evaluation. For example, if an evaluation finds that male offenders
have more success with a program than female offenders, additional evaluation could uncover
why.
• Choosing an Evaluator
Clearly, there are many decisions to be made in designing and implementing an evaluation,
whether large or small. A team of stakeholders should be involved in decision-making, but the
team must be lead by a knowledgeable evaluator capable of guiding the process. The team must
decide whether this task should fall to an internal staff member, or if an external consultant
should be brought in. Either choice could be appropriate, but the following should be
considered:
•
•
•
•
•

The individual should have the necessary knowledge to design and implement the plan.
The individual should have the time to manage the evaluation. It may not be possible for
a staff member to balance an evaluation with other responsibilities.
The individual should be involved from the program’s inception, and should have a good
working knowledge of the program.
The individual should have a good relationship with the team and be able to exercise
leadership.
The individual should be aware of personal interests in the results, and conflicts of
interest should be avoided.

A good evaluator, who contributes to effective program development and implementation as
well, is a key member of the project team. The evaluator’s role should be clarified at the onset of
the program.

53

• Logic Models as Evaluation Tools
To measure whether a program was effective and why, there must be an understanding of how
the program is supposed to work. This includes both the mechanics of the program and the
theories behind its effectiveness. For example, in developing a cognitive-behavioral job skills
program, each lesson would include a chance to practice a job skill such as arranging a job
interview. The reason for this is that the program designers know that if they create the activity,
provide the materials for it, and run it (the mechanics), that offenders are more likely to learn and
use the skill because they have the chance to practice it (the theory). This chain of events is
often implicit, but each link in the chain must be in place in order for the program to be
successful.
Activity Æ Materials Æ Activity
Designed
Obtained
Implemented

Æ Offenders Æ Offenders Æ Offenders
Participate
Learn Skill
Practice Skill

Any of these links in the chain could mean the difference between whether a program works or
does not work. When a program is evaluated, it is important to understand why it did or did not
work. In the example above, even if the activity is perfectly designed, it may not be effective
without enough time to practice. Or, maybe the skill is not one that offenders really need, so
they don’t have an opportunity to use it. When a program does not work, designers need to
know what link in the chain broke down so that it can be fixed (or what link was missing in the
first place). When a program does work, designers want to know which links need to be
replicated and which can be eliminated or redesigned. To achieve that level of understanding, all
of the links in the chain must be explicitly mapped out.
A logic model, also called a process map, is a flexible tool for mapping the mechanics and theory
of a program. Creating a logic model forces designers and stakeholders to identify the resources
that are available for a program, describe what will occur in the program, and describe the
desired outcomes. The model can then be used to design the program itself, to design a program
evaluation, and to analyze evaluation results and determine what made a program succeed or fail.
The model also gives everyone involved in the project an idea of the “big picture.” A great deal
of literature is available on the creation of logic models, and a sample is available in this manual.
Components of a Logic Model
Inputs: The resources being put into the project (staff, materials, etc.)
Activities: What is being done in the project (treatment groups, etc.)
Outputs: Direct product of activities, usually numerical targets (# of offenders
trained on a skill, etc.)
Initial Outcomes: Short-term changes, such as an increase in knowledge.
Intermediate Outcomes: The next step in the change process, often applying new
knowledge or skills.
Long-Term Outcomes: The ultimate goal of the project, often a behavior change.

54

SAMPLE: LOGIC MODEL
Program: County Community Corrections Substance Abuse Intervention
Outcomes
Inputs
•

•

•

•
•

•

•

Department will
provide probation
officers to make
referrals and meeting
space.
Tx provider will
provide certified
facilitator and
educational materials.

Activities
•

•

•

Department will
provide probation
officers to make
referrals.
AA/NA will provide
meeting structure.
Several local churches
will provide space for
meetings.

•

Department will
provide probation
officers, facilities, and
materials for drug
testing.
Local lab will contract
to provide urinalysis
screening and report
results.

•

•

•

•

Probation officers will
complete referrals to
TX group within two
weeks of completing
case plan.
Tx provider will
complete intake
within 2 weeks of
receiving referral.
Offender will attend
groups twice weekly
for 20 weeks.
Probation officers will
complete referrals to
AA/NA within two
weeks of completing
case plan.
Offenders will attend
weekly meetings for a
minimum of 24
weeks.
Probation officers will
begin urine testing of
offenders at first
check-in.
Probation officers will
conduct testing
according to offender
risk level at check-ins
thereafter.
Officers will respond
to test results
according to policy.

Outputs
•

•

100% of offenders
with substance abuse
Tx on their case plan
will be enrolled in the
Tx group.
80% of offenders who
enroll in the Tx group
complete it.

Initial
•

•

•

•

•

•

•

100% of offenders
with substance abuse
Tx on their case plan
will be referred to
AA/NA groups
80% of offenders who
are referred will
attend weekly AA/NA
meetings for 24 weeks

•

100% of high-risk
offenders with
substance abuse Tx on
their case plan will be
urine tested at every
check-in.
100% of medium-risk
offenders with
substance abuse Tx on
their case plan will be
urine tested at least
twice monthly.

•

•

•

Offenders will gain
knowledge of the
impact of their
substance use.
Offenders will gain
skills for addressing
triggers of substance
use.
Offenders will
increase motivation
and self-efficacy to
discontinue use.
Offenders will
develop a prosocial
support network.
Offenders will gain
knowledge of the
impact of their
substance use.

Offenders will gain
knowledge of the
consequences of
continuing substance
use.

Intermediate

•

•

•

Long-term

Offenders will address
triggers for substance
use in prosocial ways.

•

Offenders will utilize
prosocial supports to
avoid substance use.

Offenders will
discontinue substance
use.

•

Offenders will remain
substance-free for the
duration of their
probation.

Offenders
experiencing relapse
will utilize Tx options
to regain sobriety.

INDIVIDUAL PERFORMANCE MEASUREMENT
Overview
To fully incorporate evidence-based practices into the culture of an agency, the language and
methodology of EBP must be incorporated system-wide, and staff must be supported and held
accountable for implementing, maintaining, and measuring those practices. This includes
incorporating evidence-based practices into the performance measurement system. An effective
performance appraisal measures what the organization values. Thus management and line
officers must be evaluated on how aligned their management or line practices are with EBP
practices and principles. As job expectations change, the measures against which performance is
evaluated also need to change, so that everyone is on the same page about what constitutes good
performance, and so that staff members have an opportunity to be acknowledged for what they
do well and to learn how to prepare for advancement opportunities.
It is important that individual performance reviews are designed to support managers and
officers to meet organizational expectations and goals. If performance is being judged by out-ofdate criteria that are not in alignment with the organization’s desire to implement EBP, there is
little or no extrinsic motivation to incorporate EBP into daily practice. Management and line
staff must be supported and held accountable to meet these new expectations. Just as line staff
must focus on what offenders do well, so must managers support staff to continue practices that
align with EBP and to change behaviors that are not consistent with EBP. Following-up on areas
for change with performance improvement plans, providing skill development opportunities, and
when necessary, taking disciplinary action are all necessary steps in the appraisal process.
Principles
› Performance reviews should be ongoing.
To allow staff to incorporate feedback and improve performance, supervisors should
provide ongoing assessment and reinforcement. It does not benefit the officer nor the
agency to ignore behavior, positive or negative, until an annual review.
› Performance criteria should be explicit and measurable.
Staff should have clear expectations of the job responsibilities and expected outcomes.
The process for measuring achievement should be clearly defined and consistent.
› Performance criteria should align with desired outcomes.
To encourage commitment to evidence-based outcomes, staff at all levels should be
measured by their ability to meet desired outcomes rather than unrelated, outdated, and
possibly contradictory criteria.

56

› Reviews should focus on positive behavior and provide opportunities for improvement of
skill deficiencies.
Everyone is more responsive to positive feedback. Procedures should be in place to
recognize and reward positive performance. Organizations must be willing to follow-up
on negative evaluations through skill development opportunities, performance
improvement plans, or discipline when necessary.
› Distinguish between performance review and coaching.
When implementing new practices, staff needs an opportunity to practice without fear of
repercussions. Over time, however, everyone must expect that they will be evaluated on
their use of EBP, and that they may face consequences for failing to use those practices.
Components
•

Establish a regular schedule for formal supervision, encompassing observation and
performance reviews.
Performance appraisals should not be a once a year occurrence. At the annual appraisal meeting,
the supervisor and employee should create a learning contract for the year. This can include
continuing and enhancing current practices to changing and adopting new practices. This
contract should be discussed on a regular basis and no less than once a quarter. By observing
employee performance, reviewing performance indicators, and meeting with employees
regularly, both positive and negative performance feedback is more immediate and useful.
Problems regarding behavior, skill, and attitude can be addressed without delay through
performance improvement plans and opportunities for additional skill development if
appropriate. Leaving performance issues unattended to until an annual performance evaluation
is scheduled leaves the organization suffering from possible poor productivity and employees
blindsided at review time. There should be no “surprises” in a performance evaluation.
• Create an environment open to ongoing communication, learning, and feedback.
Organizations implementing evidence-based practices must constantly collect data about their
progress, analyze that data, and make changes and course adjustments based on that feedback.
The same is true at all levels of the organization, including that of the individual employee.
Without open and ongoing communication, performance feedback, and learning between
supervisors and employees, progress toward providing improved services and reducing
recidivism is stalled. Information sharing and communication at all organizational levels are
critical to achieving performance improvement.
• Staff should evaluate their own performance.
A self-evaluation should be the starting point for the appraisal. The supervisor should always
review the employee’s self-evaluation with the employee in person. This creates more active
participation in the review process, and may increase the likelihood of reaching agreement on
what changes may need to occur. The self-evaluation provides valuable insight and information
regarding the employees’ perceptions of their job responsibilities, whether they are completing

57

them adequately, and whether they have the training and resources necessary to fulfill their
professional goals. Many performance failures are rooted in a lack of understanding and
agreement by supervisors and employees regarding what constitutes effective performance.
• Staff should provide feedback on their supervisor’s performance.
The relationship between an employee and their supervisor, including the quality of the
communication and the ability of the supervisor to train, model, and give feedback on skills can
have a significant influence on ability of an employee to fulfill his or her responsibilities.
Employees should be given an opportunity to evaluate supervisors so that they can give feedback
on the quality of that relationship. (In a way, this is a customer satisfaction survey of the
supervisor/employee relationship. The “services” provided by the supervisor are only effective
if the employee finds them beneficial.) This can be done informally in concert with an
employee’s review, or through a formal written (and preferably confidential) survey
administered by the peer review team or management.

Performance Appraisals:
The Questions

Rewarding
Exceptional Performance

While ongoing performance review
can be an informal process, an
annual performance appraisal
should be done using standardized
guidelines. Both the employee and
the supervisor should be clear on
what is being assessed.

A positive performance appraisal is
most often equated with a salary
increase or a promotion. However,
there are many ways to formally
and informally recognize quality
work.
•

Publicly acknowledge the
contributions of individual
employees or teams (for staff who
enjoy public recognition).

•

Offer incentive gifts, special
activities, or time off.

•

Provide advanced professional
development opportunities.

•

Shift workloads: offer smaller,
specialized caseloads, or more
leadership opportunities with less
administrative work (do not reward
good work with even more work)!

•

Provide opportunities for
leadership and participation in
organizational development
activities.

Potential questions are:
•

•

•

What was the employee expected to
accomplish (i.e. his/her job
description)?
What did the employee
accomplish?
How was the employee able to
achieve these accomplishments
(i.e. skills and strengths)?

•

What goals were not met, and why?

•

What potential for improvement
exists, and what is the plan for
professional development?

58

Sample: Employee Observation Evaluation
Employee Name:

Activity Observed:

Observation Time (amount):
Behavior

Start Time:

Below
Needs
Expectations Improvement

Stop Time:

Place of Observation:

Meets
Exceeds
Comments on Direct
Expectations Expectations Observations

Uses cognitive-behavioral
language during encounters with
clients.
Models appropriate language
and behavior to clients.
Includes: Speaking positively
about program, law, courts, etc.
Does not use derogatory
language/jokes op sarcasm.
Avoids power struggles with
clients (e.g., does not argue with
clients, raise voice at clients,
antagonize clients)
Consistently applies appropriate
consequences for behaviors
(both positive and negative)
Identifies thinking barriers in
clients in value-neutral way
Overall Score

59

Sample: Supervisor Evaluation
Supervisor ______________________________________ Date: ___________________
This evaluation is confidential. It is important to have your honest opinion so that we can address strengths and
needs of all staff as well as provide needed training. Thank you for taking time to assist in bettering our team.
1.

My supervisor is firm and fair and has effective use of authority.
_____strongly agree _____agree

2.

My supervisor models and reinforces pro social behaviors through positive and negative reinforcers.
(Knows how to reward appropriately and how to stop negative behavior without damaging morale or
the dignity of staff.)
_____strongly agree _____agree

3.

_____ disagree _____strongly disagree

I consider my supervisor a leader.
_____strongly agree _____agree

9.

_____ disagree _____strongly disagree

My supervisor is organized and makes good use of his or her time.
_____strongly agree _____agree

8.

_____ disagree _____strongly disagree

My supervisor has a firm understanding of Best Practices and is able to articulate that to staff.
_____strongly agree _____agree

7.

_____ disagree _____strongly disagree

My supervisor is open to suggestions, communicates well, and shows respect.
_____strongly agree _____agree

6.

_____ disagree _____strongly disagree

My supervisor is knowledgeable in community resources.
_____strongly agree _____agree

5.

_____ disagree _____strongly disagree

My supervisor is effective at teaching skills needed to do my job.
_____strongly agree _____agree

4.

_____ disagree _____strongly disagree

_____ disagree _____strongly disagree

My supervisor understands what it takes to do my job and is always looking for ways to increase the
efficiency of the unit.
_____strongly agree _____agree

_____ disagree _____strongly disagree

10. My supervisor cares about me personally and has done things to help me grow professionally.
_____strongly agree _____agree

_____ disagree _____strongly disagree

11. My supervisor is good at not only solving problems, but also anticipating them and gathering the
appropriate information.
_____strongly agree _____agree

_____ disagree _____strongly disagree

60

12. My supervisor isn’t afraid to roll up his/her sleeves and pitch in when help is needed.
_____strongly agree _____agree

_____ disagree _____strongly disagree

13. I have deep respect for my supervisor and appreciate all the hard work that he/she does.
_____strongly agree _____agree

_____ disagree _____strongly disagree

14. My supervisor is a hard worker and follows through with what he/she says he will do.
_____strongly agree _____agree

_____ disagree _____strongly disagree

15. My supervisor is accessible. He/she answers voice mail, email and other requests in a timely manner.
_____strongly agree _____agree

_____ disagree _____strongly disagree

This next section covers technical skills including case planning and case management. The following scale ratings
are listed:
5 = Supervisor is an expert in this area
4 = Strong understanding in this area
3 = Average understanding in this area
2 = Could use training in this area
1 = Not sure training would help
16. ICON business rules and information that is tracked.
5
4
3
2

1

17. Case Planning to include Relapse Prevention Plans.
5
4
3
2

1

18. Understand responsivity issues, to include motivational interviewing, and the Jesness.
5
4
3
2
1
19. Sensitive to diversity issues to include ethnicity and gender.
5
4
3
2

1

20. Understanding of the issues of domestic violence.
5
4
3
2

1

21. Knowledge of substance abuse and treatment.
5
4
3

2

1

22. Knowledge of mental health issues and treatment.
5
4
3
2

1

23. Knowledge of sexual abuse issues and treatment.
5
4
3
2

1

24. Knowledge of interstate compact.
5
4
3

1

2

25. Not only knows policy but also is able to guide employee to think the problem out on his/her own in
the future.
5
4
3
2
1
5th Judicial District Department of Correctional Services, Des Moines, Iowa

61

BIBLIOGRAPHY
Bell, JB. (1994) Handbook of Practical Program Evaluation. San Francisco, CA: Jossey-Bass.
Fowler, FJ. (2002) Survey Research Methods, 3rd Ed. Thousand Oaks, CA: Sage Publications.
Grote, D. (1996) The Complete Guide to Performance Appraisal. New York, NY: American
Management Association.
Grote, D. (2003) Performance Appraisals with More Gain, Less Pain. Cambridge, MA:
Harvard Communication Newsletter.
Lowenkamp, CT and Latessa, EJ (2005) Does Correctional Program Quality Really Matter? The
Impact Of Adhering To The Principles Of Effective Interventions. Available online at
http://www.uc.edu/criminaljustice/Articles/Correctional_Program_Quality.pdf.
Miller, WR and Rollnick, S. (2002) Motivational Interviewing: Preparing People for Change. 2nd
Ed. New York, NY: Guilford Press.
United Way of America. (1996) Measuring Program Outcomes: A Practical Approach.
Washington, DC: United Way of America.

62

APPENDIX: COMPREHENSIVE ASSESSMENT OF TREATMENT PROVIDER
Developed by ICCA, modified by Kim Sperber, Talbot House
Section 1: STAFF EXPERIENCE, EDUCATION, TRAINING

Staff Experience/Education
In the following table, please indicate the percentage of direct treatment staff and non-treatment staff (line staff such
as security officers) that meet the following criteria:
Criteria

Percentage
treatment staff

of

Percentage
of
treatment staff

non-

Have worked a minimum of 5 years with the program
Have a minimum of 5 years previous experience working
with offenders in a treatment setting before being hired by
your program
Have worked a minimum of 10 years with the program
Have a minimum of 10 years previous experience working
with offenders in a treatment setting before being hired by
your program
In the following table, please indicate the percentage of direct treatment staff and non-treatment staff (line staff such
as security officers) that meet the following criteria:
Criteria

Percentage of
Treatment staff

Percentage of
Non-Treatment staff

Have a Bachelors degree in social profession
Have a Masters degree or higher in a social
profession
Have worked a minimum of 2 years with the
program
Have a minimum of 2 years previous experience
working with offenders in a treatment setting before
being hired by your program
Rank the five most important characteristics considered when hiring new staff.
__ Education
__ Experience
__ Philosophy of the candidate
__ Demographics of the candidate
__ If the new staff is in recovery
__ If the new staff is an ex-offender
__ Life experiences of the new staff
__ Ability to be firm but fair
__ Problem solving techniques
__ Empathy
__ Knowledge of population
__ Spontaneity
__ Writing skills
__ Communications skills
__ Other: ________________________________________________________________

63

Staff Training
In the following table, please indicate the average number of hours of initial training and on-going training in each
of the following areas.
Area of training

Initial training hours

On-going training hours

Criminogenic Factors
Anger management treatment
Changing criminal thinking
Changing the peer associations
Cognitive skills
Domestic violence
Family counseling
Family treatment
Substance abuse treatment
Training in parenting
Violence or aggression
Community Functioning Factors
Employment/vocational training
Financial classes
Life skills
Mental health
Substance abuse education
Program Specific Factors
Policy and procedures of the program
Supervision of offenders in the
community
Supervision of offenders while in the
facility
Treating juveniles
Other:
Other:
Do new staff attend training?
__ Externally
__ Internally
__ Supervised
__ Unsupervised
__ Other (please specify: ___________________________________________ )
Which of the following generally applies to the training of new staff (check all that apply)?
__ New staff are trained in the philosophy of the program
__ New staff are trained in rewards used by the program
__ New staff are trained in the punishments used by the program
__ New staff are trained curriculums used by the program
__ New staff are trained in the use of the assessment instruments used by the program
__ New staff are trained in the principles of effective interventions
__ New staff are trained in custody of the offenders
__ New staff are provided safety training

64

How does the training of new staff typically occur (check all that apply)?
__ They read the policy and procedure manual
__ They read the curriculums
__ They attend training sessions on the use of punishments
__ They attend training sessions on the use of rewards
__ They attend training sessions on the use assessments instruments
__ They attend training sessions on the principles of effective interventions
__ They attend training sessions on criminal thinking/criminal behavior
__ They attend training sessions on substance abuse
__ They attend training sessions on violence /aggression
__ They attend other training sessions: ______________________________________________
How are the training opportunities for existing staff determined (check all that apply)?
__ Existing staff are not provided training
__ Surveys
__ Face-to-Face Request
__ Solicited requests
__ Unsolicited requests
__ Other (please specify: ______________________________________________ )
What types of training are available for staff (please check all that apply)?
__ Externally
__ Unsupervised
__ Conferences

__ Internally
__ Supervised
__ Other (please specify: _____________________ )

Are existing staff required to attend on-going training?
__ Yes

__ No

If yes, how many hours per year are staff required to attend:
a.
b.
c.
d.

External training ____________________
Internal training _____________________
Supervised training ___________________
Other ______________________________

Do staff receive on-going training in providing treatment in the following offenders need areas?
Criminogenic Factors
Anger Management
Cognitive-Behavioral Treatment
Criminality
Domestic Violence
Family Treatment
Substance Abuse

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No

__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No

__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply

Community Functioning Factors
Educational/Vocational
Employment
Entitlements
Housing Needs
Life Skills
Mental Health

65

Do staff receive on-going training in assessing the following offenders needs areas?
Criminogenic Factors
Anger Management
__ Yes
__ No
Criminality
Domestic Violence
Family Treatment
Substance Abuse

__ Does not apply

__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No

__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No

__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply

3

4

5
Very adequate

3

4

5
Very adequate

3

4

5
Very adequate

3

4

5
Very adequate

3

4

5
Very adequate

3

4

5
Very adequate

Community Functioning Factors
Educational/Vocational
Employment
Financial
Medical Care
Need for Entitlements
Housing Needs
Life Skills
Mental Health
Mentoring

Are training sessions assessed on the:
a. Quality of training
b. Applicability of the training
c. Practicality of the training
d. Value of training materials
e. Quality of trainer
Are staff cross-trained on other positions?

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No

__ Yes

__ No

Rate the adequacy of training in the following areas:
a. Use of rewards
0
1
2
no training Very inadequate
in this area
b. Use of punishers
0
1
2
no training Very inadequate
in this area
c. Principles of effective interventions
0
1
2
no training Very inadequate
in this area
d. Program model
0
1
2
no training Very inadequate
in this area
e. Curriculums used by the program
0
1
2
no training Very inadequate
in this area
f. Assessment instrument used by the program
0
1
2
no training Very inadequate
in this area

66

Rate the adequacy of initial training for new staff.
0

1

no initial
training

Very inadequate

2

3

4

5
Very adequate

Rate the adequacy of on-going training for existing staff.
0

1

2

3

4

no on-going Very inadequate
training

5
Very adequate

Staff Evaluation
How often are staff evaluations conducted?
__ Every Month
__ Every Three Months
__ Every Six Months
__ Annually
__ Bi-Annually
__ Other (please specify: _____________________________________________ )
Who conducts these staff evaluations/assessments?
__ Treatment staff supervisor
__ Administrator
__ Program director
__ Supervising officer
__ Quality assurance division
__ Other (please specify: _____________________________________________ )
Does this assessment include:
__ Observation of treatment groups
__ File maintenance/file review
__ Continuing Education
__ Complaints
__ Offender assessments
__ Offender feedback
__ Monitoring of offender progress
__ Other (please specify: _____________________________________________ )
Which of the following items are examined on the staff evaluations (please check all that apply)? Please indicate if
the item is assessed for direct contact staff (TX staff) or non-direct contact staff (Non-TX staff).
Item
Attendance
Appearance
Relationship with other staff
Communication with clients
Writing skills
Ability to work with a team
Presentation of tx material
Ability to control classroom
Ability to engage clients in discussions
Ability to assess clients
Ability to reassess clients
Ability in developing tx plans
Ability in modeling behavior
Evaluations not completed
Other:
Other:
Other:
Other:

Tx Staff

Non-Tx Staff

67

Rate the adequacy of staff performance evaluations.
0

1

no staff
evaluations

Very inadequate

2

3

4

5
Very
adequate

What percentage of service delivery staff receive clinical supervision by a licensed professional? ____________%
How often are staff provided with clinical supervision?
__ Weekly
__ Every Month
__ Every Six Months

__ Bi-weekly
__ Every Three Months
__ Annually

Who provides the clinical supervision?
__ Director/Administrator
__ Program Director
__ Treatment Staff Supervisor
__ Supervising Officer
__ Clinical Supervisor
__ Quality Assurance Division
__ Other (please specify: ______________________________________________ )
__ Clinical Supervision is not provided
Is the person responsible for providing clinical supervision licensed by some accrediting body (e.g., CAC certified
for substance abuse programs)?
__ Yes
__ No
__ Not applicable
How is the clinical supervision conducted (check all that apply)?
__ Meetings with the treatment staff and the clinical supervisor
__ Individual meetings with the clinical supervisor
__ Supervisor sits in on groups
__ Not applicable
Rate the adequacy of clinical supervision.
0

1

No clinical
supervision

Very inadequate

2

Do staff support each of the following items:
Item
Does not
apply
Philosophy of program
Reducing the risk of the
offender
Targeting the needs of the
offender
The Leadership
Item
Does
not
apply
TX staff
Non-TX staff
Monitoring the offender
Rewarding the offender
Punishing the offender
Getting the offender hooked
up with other agencies

3

4

5
Very
adequate

TX staff
Yes

TX staff
No

Non-TX staff
Yes

Non-TX staff
No

TX staff
Yes

TX staff
No

Non-TX staff
Yes

Non-TX staff
No

68

Section 2: CLASSIFICATION AND ASSESSMENT
Upon admittance in the program, what are some of the offenders problem areas that are consistently present (check
all that apply)?
__ Substance abuse
__ Criminal thinking
__ Anger management
__ Criminal friends
__ Mental health
__ Attitudes related to sexual offending
__ Lack of education
__ Lack of motivation
__ Lack of job skills
__ Family/parenting issues
__ Other (please specify): __________________________________________________

What are the selection criteria?
__ Nonviolent offenders
__ First time offenders
__ Risk specific offenders (e.g., low risk)
__ Offense specific offenders (e.g., DUI offender, sex offenders)
__ Need specific offenders (e.g., only substances abuse)
__ There are no selections criteria in place
__ Other (please specify):_______________________________________
How well are the selection criteria adhered to?
__ Completely
__ Mostly
__ Somewhat
__ Not at all
__ Do not know
__ Not applicable – no selection criteria in place
What are the exclusionary criteria (check all that apply)?
__ Violent offense(s)
__ History of violence within a certain time period
__ Arsonists
__ Mentally unstable
__ Clients with too little time left on sentence
__ There are no exclusionary criteria in place
__ Other
How well are the exclusionary criteria adhered to?
__ Completely
__ Mostly
__ Somewhat
__ Not at all
__ Do not know
__ Not applicable – no exclusionary criteria in place
Are any of the offenders in your program inappropriate for the treatment that is being offered?
__ Yes

__ No

69

If yes, what percentage of offenders are inappropriate for the treatment?
__ Less than 5%
__ 30% - 40%
__ 5% - 10%
__ 40% - 50%
__ 10% - 20%
__ 50% or more

__ 20% - 30%
If yes, what are some of the reasons offenders are inappropriate for the treatment provided?
__ They are too violent
__ They are not motivated
__ They are low risk offenders __ They are cognitively impaired
__ They are mentally unstable
__ They are too young/ immature
__ They need a higher intensity treatment than we offer
__ Other:____________________________________________________
Rate how appropriate are the selection criteria for services offered by your program.
0
No selection
appropriate
Criteria

1
2
Informal criteria

3

4

5
Very

Rate how appropriate the exclusionary criteria are that would keep an offender from entering your program.
0
1
2
No exclusionary Informal criteria
appropriate
Criteria

3

4

5
Very

What assessment instruments are used when an offender first enters the program? (Please check all that apply. If
assessment instrument does not apply, then please check does not apply).
Instrument

All Offenders
Apply
Standardized Criminality Risk/ Need Instruments ________

Some offenders
____________

No Offenders

Does Not

__________

_______

Level of Services Inventory (LSI)

________

____________

__________

_______

Wisconsin Risk Assessment

________

____________

__________

_______

Wisconsin Need Assessment

________

____________

__________

_______

Client Management System (CMS)

________

____________

__________

_______

Correctional Offender Management Profile for
Alternative Sanctions (COMPAS)
________

____________

__________

_______

Salient Factor Score (SFS)

________

____________

__________

_______

Custody Rating Scale (CRS)

________

____________

__________

_______

Hare Psychopathy Checklist

________

____________

__________

_______

Corrections Risk Analysis System (C-RAS)

_________

____________

__________

_______

Youthful Level of Service Inventory (Y-LSI)

________

____________

__________

_______

70

Instrument

All Offenders

Some offenders

No Offenders

Does Not
Apply

Juvenile Probation Risk Assessment

________

____________

__________

_______

Starting Point

________

_____________

__________

_______

Community Risk/ Need Management

________

_____________

__________

_______

Other (please specify) _______________

________

_____________

__________

_______

HRAM/ HRAF

________

_____________

__________

_______

Bio/ social

________

_____________

__________

_______

Bio/Psycho/ social

________

_____________

__________

_______

Psycho/ social

________

_____________

__________

_______

Other (please specify) _______________

________

_____________

__________

_______

Problem Oriented Screening Instruments for
Teenagers (POSIT)
Alcohol Dependence Scale

________
________

_____________
_____________

__________
_________

________
________

Addiction Severity Index (ASI)

________

_____________

__________

________

______________

_________

________

______________

__________

________

_____________

_________

________

Unstandardized Risk/ Needs

Substance Abuse Assessments

Substance Abuse Subtle Screening Inventory
(SASSI)
________
Multidimensional Addictions and Personality
Profile (MAPP)
________
Juvenile Automated Substance Abuse
Evaluation(JASAE)
_________
Michigan Alcohol Screening Test (MAST)

________

______________

__________

________

Drug Alcohol Screening Test (DAST)

________

______________

__________

________

Adult Substance Abuse Survey (ASUS)

________

______________

__________

________

Offender Profile Index (OPI)

________

______________

__________

________

CAGE

________

______________

__________

________

Other (please specify) __________________

________

______________

__________

________

71

Instrument

All Offenders

Some offenders

No Offenders

Does Not
Apply

Personality
Jesness Inventory

________

Minnesota Multiphasic Inventory (MMPI)

________

Adult Management System
Other (Please specify) _________________

______________

__________

________

______________

__________

________

________
________

______________
______________

__________
__________

________
________

SARA

________

______________

__________

________

Other (please specify) ________________

________

______________

__________

________

Criminal Sentiments Scale

________

______________

__________

________

How I Think Questionnaire

________

______________

__________

________

Beliefs Inventory

________

______________

__________

________

Pride in Delinquency

________

______________

__________

________

______________

__________

________

______________

__________

________

Domestic Violence

Antisocial Attitudes/Cognitive Distortions

Antisocial Attitudes/ Cognitive Distortions, Con’t
Client Self-Rating
Other (please specify) __________________
Sex Offender

________
________

STATIC-99

________

______________

__________

________

SONAR

________

______________

__________

________

JSOAP

________

______________

__________

________

MnSOST-R

________

______________

__________

________

MSI

________

______________

__________

________

Other (please specify) __________________

________

______________

__________

________

72

Instrument

All Offenders

Some offenders

No Offenders

Does Not
Apply

Violence
Hare Psychopathy Checklist

________

______________

__________

________

HCR-20

________

______________

__________

________

WASE

________

______________

__________

________

State Trait Anger Expression Inventory
(STAXI)

________

______________

__________

________

Other (please specify) ________________

________

______________

__________

________

Other
MAYSI

________

______________

__________

________

IQ tests (please list) ___________________

________

______________

__________

________

Educational tests (please list) ____________ ________

______________

__________

________

Other (please specify) _________________

______________

__________

________

________

Has the program validated the assessment instrument(s) checked above on it’s own population?
__ Yes, all of the assessments
__ Yes, most of the assessments
__ Yes, about half of the assessments
__ Yes, less than half of the assessments
__ No
How is the offender’s risk level determined (check only one)?
__ Risk level is determined by standardized assessment instruments
__ Risk level is determined by psycho-social assessment
__ Risk level is determined by the severity of the offense
__ Risk level is determined by judgment of the staff (not by instruments)
__ Other:___________________________________________________
__ Risk level is not determined
Does your program reassess the offender using the following?:
Standardized risk/ need instruments
Substance abuse instruments
Personality instruments
Family instruments
Cognitive/ Attitudinal instruments
Violence instruments

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No

__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply
__ Does not apply

73

If the program reassesses the offenders, when does the reassessment take place?
__ Every three months
__ Every six months
__ Once a year
__ Upon termination of the program
If the program reassesses the offenders, how is the information used (check all that apply)?
__ Used to reassess treatment plans
__ Used to present to the court/parole board/probation department
__ Used as criteria for termination
__ Other: (please specify) _________________________________________________
__ Other: (please specify) _________________________________________________
__ Other: (please specify) _________________________________________________
Rate the adequacy of offender reassessment.
0
1
2
No reassessment Very inadequate

3

4

5
Very adequate

Does your program have “tracks” for special needs offenders?
__ Yes
__ No
What special needs offenders are given this attention? __ N/A
__ Sex Offenders
__ Women
__ Drug Offenders
__ Offenders convicted of Driving Under the Influence
__ Mentally Ill Offenders
__ High Risk Offenders
__ Other: _________________________________________________________________________

74

Section 3: CHARACTERISTICS OF THE PROGRAM
If your program has criteria to determine placement into different groups, describe the criteria.

How is placement into treatment groups typically decided?
__ Placement is made based on risk level
__ Placement is made based on need
__ Placement is made based on characteristics of the offender
__ Placement is made based on openings in each group
__ All offenders participate in every group
Do higher-risk offenders receive (check all that apply)?
__ Treatment groups with only high risk offenders
__ More treatment groups
__ Stay in groups for longer periods of time
__ Stay in the program longer periods of time
__ Have higher intensity treatment groups
__ Identical services to other offenders
How do higher-risk offenders have contact with lower-risk offenders (check all that apply)?
__ Live in the same dorms (institutional setting only)
__ Share rooms (institutional setting only)
__ Participate in same treatment groups
__ Eat together (institutional setting only)
__ Participate in activities together (i.e. support groups, recreational activities, pizza parties)
__ Other (please specify: ____________________________________________________ )
Are males and females placed in the same group?
__ Yes

__ No

__ Not applicable

Does the program have specific interventions and groups that are designed for female offenders?
__ Yes-the program has groups especially for women
__ No-the program does not have special programs for female offenders
If the programs have groups/interventions especially for women, what specific groups have female only members?
__ Cognitive groups
__ Substance abuse groups
__ Anger management
__ Parenting
__ Domestic violence
__ Other: __________________________________________________________________
__ Other: __________________________________________________________________
__ Other: __________________________________________________________________

75

What groups are gender specific (check all that apply)?
__ Not applicable
__ Cognitive groups
__ Parenting
__ Substance abuse
__ Vocational
__ Anger management
__ Life skills
__ Domestic violence
__ Sex offender
__ Employment
__ AA/NA
__ Educational
__ Financial
__ Other: (please specify) ________________________________________________________
__ Other: (please specify) ________________________________________________________
__ Other: (please specify) ________________________________________________________
Rate the adequacy of the process for assigning offenders to groups.
0

1

2

3

4

Very inadequate

5
Very adequate

How are decisions regarding placement of staff made (check all that apply)?
__ Staff facilitate whatever groups are in need of facilitators
__ Staff only facilitate groups in which they are trained
__ Staff facilitate groups based on their qualifications
__ Decisions regarding placement are based on seniority (senior staff have 1st choice)
__ Decisions regarding placement are left to staff
__ Staff facilitate all groups
How are caseloads typically determined (check all that apply)?
__ Randomly
__ Based on caseload size
__ Based on offender offense
__ Based on offender need
__ Based on personality characteristics of the offender and the staff
__ Based on the risk level of the offender
Rate the adequacy of the process for assigning staff to groups.
0

1

2

3

4

Very inadequate

5
Very adequate

Rate the adequacy of the process for assigning offenders to staff.
0
Very inadequate

1

2

3

4

5
Very adequate

76

How often are treatment plans updated?
__ monthly
__ quarterly
__ yearly
__ treatment plans not used

__ bi-monthly
__ every 6 months
__ treatment plans not updated

Rate the adequacy of treatment plans.
0

1

2

3

4

Treatment plans Very inadequate
not used

5
Very adequate

Do any of the following groups that your program offers use a documented curriculum (e.g., manual)? If so, in the
space provided, write the name of the curriculum or manual. If the manual was something that was developed by the
program, please write “Program Developed”. Finally, please indicate the typical length (in weeks) of each group.
Yes

No

Name of Curriculum

Length
(in weeks)

Criminogenic Programs
Anger Management
Cognitive Skills
Criminal Friends
Domestic Violence
Family Treatment
Parenting Skills
Relationship Counseling
Sex Offender
Substance Abuse Treatment

__
__
__
__
__
__
__
__
__

__
__
__
__
__
__
__
__
__

_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________

________
________
________
________
________
________
________
________
________

Community Functioning Programs
Educational
Employment
Entitlements
Financial
Housing
Life Skills
Medical Care
Vocational
AA/NA
Art Therapy
Individual Counseling
Mental Health
Mentoring
Recreational Therapy
Self-esteem
Spirituality
Substance Abuse Education
Yoga
Other: _________________
Other: _________________
Other: _________________

__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__

__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__

_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________
_________________

________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________
________

77

What is the average, minimum, and maximum length of time (in months) an offender spend in the program (not
including aftercare)?
Average: _____________________ (months)
Minimum: ____________________ (months)
Maximum: ____________________ (months)
How many group sessions, on average, does an offender attend per week? If multiple groups are provided, please
indicate the number of sessions per group.
Group: _______________________ Sessions: _______________________
Group: _______________________ Sessions: _______________________
Group: _______________________ Sessions: _______________________
How does your program teach offenders about antisocial associates/friends?
__ The program does not teach offenders about antisocial associates
__ Staff point out characteristics of antisocial friends/associates
__ Staff help them determine which friends are antisocial, but do not point characteristics out
__ Discuss what happened in the past when the offender was with antisocial associates/friends
__ Offenders complete workbook/journal that target antisocial associates/friends
__ Other offenders will tell the individual about their antisocial associates/friends
__ Other: ___________________________________________________________________
__ Other: ___________________________________________________________________
How does the program attempt to change the antisocial peer associations of the offenders?
__ Assign a mentor
__ Make abstaining from these antisocial associates/friends a condition of release/program
completion.
__ Introduce offenders to other groups of associates/friends
__ Other (please specify: ______________________________________________________ )
How does this program attempt to change the offender’s criminal/antisocial thinking patterns, beliefs and values
(check all that apply)?
__ The program does not target antisocial thinking
__ Staff point out antisocial attitudes
__ Staff help them to determine antisocial attitudes but do not point them out
__ Discuss the antisocial attitudes that they had in the past that led to the offender being in the
program.
__ Offenders complete workbook/journal that targets antisocial attitudes
__ Other: _____________________________________________________________________
__ Other: _____________________________________________________________________
How does the program attempt to change the offender’s violent behavior?
__ The program does not target violent behavior
__ Staff discusses problems caused by violent behavior
__ Staff help offenders determine violent behavior issues through groups
__ Staff help offenders determine violent behavior issues through individual counseling
__ Discuss violent behavior issues that led to the offender being here
__ Staff share their own individual experiences with offenders
__ Offenders complete workbooks/journals that targets violent behavior
__ Other offenders will tell the individual about violent behavior
__ Other: _____________________________________________________________________

78

How does the program attempt to change the offender’s substance abuse?
__ The program does not target substance abuse
__ Staff discusses problems caused by substance abuse
__ Staff help offenders determine substance abuse issues through groups
__ Staff help offenders determine substance abuse issues through individual counseling
__ Discuss substance abuse issues that led to the offender being here
__ Staff share their own individual experiences with offenders
__ Offenders complete workbooks/journals that targets substance abuse
__ Offenders participate in self-help groups (i.e., AA/NA)
__ Other offenders will tell the individual about substance abuse
__ Other: _____________________________________________________________________
__ Other: _____________________________________________________________________
What are some of the techniques used by the program to teach offenders the identification of “triggers” that may be
problematic for the offender?
__ The program does not focus on triggers
__ Staff point out their triggers
__ Staff help them determine their triggers but do not point them out
__ Staff point out their red flags
__ Discuss what they did in the past that led to the offender being in the program
__ Offenders complete workbook/journal that provide this information
__ Other offenders will tell the individual
__ Other: _____________________________________________________________________
__ Other: _____________________________________________________________________
Rate how well your program teaches offenders about relapse.
0

1

Does not teach
about triggers

Very inadequate

2

3

4

5
Very adequate

What mechanisms are in place to allow offenders to practice new skills that they have learned?
__ The program does not have opportunities for offenders to practice
__ Offenders have to read material relating to the topic after each class
__ Offenders have to complete homework-such as writing assignments
__ Offenders have to complete journals
__ Offenders have to complete workbooks
__ Staff make the role-plays more demanding or harder for each concept
__ Offenders are required to use the new skill outside the treatment group before the next class
time.
__ Offenders are required to report (at the next class meeting) how they used the skill and what
happened.
__ Other (please specify): _______________________________________________________________
Rate the opportunities fro role-playing in the treatment groups.
0

1

No role-playing

Very inadequate

2

3

4

5
Very adequate

Rate the consistency of staff in using role-plays in their treatment groups.
0
1
2
3
4

5

No role plays

Very consistent

Not at all consistent

79

Please specify the category that best reflects how often the groups role-play and what percentage of offenders
consistently participate in role-play.
Group

Every
Class

Every new
skill

Occasionally

Whenever
needed

1 or 2 times

% of
offenders
Participating

Criminogenic
Programs
Anger
Management
Cognitive Skills
Criminal Friends
Criminal
Thinking
Domestic Violence
Family Treatment
Parenting Skills
Relationship
Counseling
Sex Offender
Substance Abuse
Tx
Other Programs
Employment
Substance Abuse
Education
Other:
__________
Other:
__________
Which of the following incentives are used (please check all that apply)?
__ No incentives used
__ Certificate for completion of the program
__ Certificate of completion fro the specific treatment group
__ Offender of the month award
__ Graduation Ceremony
__ Individual verbal praise
__ Reduction in time of sentence
__ Tokens or points that can redeemed for material items
__ Tokens or points that are used for phase advancement
__ Free time
__ Stickers or notes
__ Food items
__ Parties
__ Other: ______________________________________________
Rate the adequacy of incentives used by the program to encourage participation and completion of the program.
0

1

Incentives
not used

Very inadequate

2

3

4

5
Very adequate

80

Which of the following punishers, consequences and sanctions utilized (please check all that apply)?
__ Punishers, consequences, sanctions not utilized
__ Extra work duty
__ Extra homework
__ Singing songs
__ Wearing signs
__ Time Outs/ Hot seats
__ More time added to sentence
__ Technical violations
__ Terminated from the program
__ Loss of privileges
__ Loss of points
__ Loss of levels
Why does the program use punishments (check all that apply)?
__ To coerce the offenders into abiding by the rules of the program/release
__ To control the offenders while they are in the facility
__ To stop antisocial behavior
__ To change the behavior of the offender
__ To show the offender that his/her actions have consequences
__ To increase accountability
__ Other (please specify: _______________________________________ )
Are staff members trained in the administration of punishments?
__ Yes

__ No

What type of training do staff receive regarding the use of punishment?

When staff have to punish offenders, do they provide alternative behaviors for the offenders after the punishment is
administered?
__ Yes

__ No

When a staff member has to issue a punishment and the offender becomes upset, how does the staff deal with the
offender and/or the situation?
__ Discuss the situation with the offender
__ Walk away from the situation
__ Suggest the offender discusses the situation with another staff member
__ Suggest the offender discusses the situation with another offender
__ Suggest the offender “journal” about the situation
__ Request the offender “take a break to cool off” and then discuss the situation
__ Other (please specify: _________________________________________________________
_____________________________________________________________________________ )

81

Rate the likelihood of offenders receiving sanctions/punishers/consequences every time they deserve to receive one.
0

1

Punishers not
used

Very unlikely

2

3

4

5
Very likely

Rate the immediacy of punishment.
0

1

Punishers not
used

Not immediate

2

3

4

5
Very immediate

Rate how well matched punishers are to the severity of the behavior.
0

1

Punishers not
used

Not matched

2

3

4

5
Very well matched

Rate the adequacy of punishments, consequences, and sanctions utilized by the program.
0

1

Punishers not
used

Very inadequate

2

3

4

5
Very adequate

Indicate where on the continuum the ratio of punishers: incentive falls?
PunishersÅ----------------------------------------- | --------------------------------------------------------->Rewards
50:50
What type of involvement do the family members of the offenders have?
__ The program provides voluntary groups for family members
__ The program has mandatory groups for family members
__ The program does not involve family members
How many weeks do the interventions provided to the family members last?
___________________ weeks
__ Not applicable
How many sessions are the family interventions?
__________________ Sessions

__ Not applicable

How many minutes do the family member groups last each session?
__________________ Minutes
__ Not applicable
Rate the adequacy of family members involvement in the treatment process.
0

1

No family
Involvement

Very inadequate

2

3

4

5
Very adequate

Which of the following topics are generally discussed in the family groups?

82

•
•
•
•
•
•
•

•
•
•

•
•

Policies and procedures of the treatment
__ Yes
__ No
__ Does not apply
Groups that the offender is participating in
__ Yes
__ No
__ Does not apply
Substance abuse education
__ Yes
__ No
__ Does not apply
How to reduce substance abuse
__ Yes
__ No
__ Does not apply
What is criminal thinking
__ Yes
__ No
__ Does not apply
How thinking affects behavior
__ Yes
__ No
__ Does not apply
How to assess the offender at home
__ Yes
__ No
__ Does not apply
How to assist the offender in maintaining
prosocial behavior
__ Yes
__ No
__ Does not apply
Establishing boundaries
__ Yes
__ No
__ Does not apply
Other: _________________________________________________________________________________
Other: _________________________________________________________________________________

What is program completion based on?
__ Length of time to the program
__ Length of the time in the program, regardless of sentence
__ Completion of classes/groups
__ Completion of a certain number of classes
__ Completion of treatment plan
__ Completion is based on phase advancement
__ Completion is based on acquisition of new skills and behavior
Rate the adequacy of completion criteria.
0

1

No completion
criteria

Very inadequate

2

3

4

5
Very adequate

How do staff monitor the whereabouts and activities of the offenders when they are in the community?
•
•
•
•
•
•
•
•

Random home visits
Random work visits
Random drug/alcohol tests
Check passes with offender before leaving
Check passes with offender after they return
Rely on relationship with police officers to monitor
Rely on probation/parole officers to monitor
The program is not responsible for the offenders while
in the community

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No
__ No

__ Yes

__ No

Rate the adequacy if staff monitoring of the whereabouts and activities of the offenders when they are in the
community. _NA: Institutional program, participants are always in the facility.
0

1

No monitoring

Very inadequate

2

3

4

5
Very adequate

Do institutional offenders participate in treatment? (check all that apply):
(N/A program is in the community ___ )
Remain completely separated from the general population?
__ Yes
Live with general population?
__ Yes
Eat with the general population?
__ Yes
Attend other services with the general population?
__ Yes
Section 4: POST-PROGRAMMING OPTIONS

__ No
__ No
__ No
__ No

83

Do offenders receive aftercare from you facility/program or another agency?
__ Offenders receive aftercare from your program
__ Offenders receive aftercare from our agency, but not our program
__ Offenders receive aftercare from another agency
__ Offenders do not receive aftercare services
Do all offenders participate in aftercare (either in-house or with another agency)?
__ Yes
__ No
__N/A
What percentage of offenders participate in aftercare? ______________________ %
How many weeks does aftercare last? ________________ weeks
How many sessions per week do offenders meet for aftercare groups? ____________ sessions

If aftercare is provided, what types of specific aftercare services does your program, either directly or through
referrals, provide to the offenders? Further, in the space provided state what curriculum is used (if any).
Curriculum
Criminogenic Programs
Anger Mgt
Cognitive Skills
Criminal friends
Criminal Thinking
Domestic Violence
Family Tx
Parenting Skills
Relationship Counseling
Sex Offender
Substance Abuse
Community Functioning Programs
Education
Employment
Entitlements
Financial
Housing
Life Skills
Medical Care
Vocational
Other Programs
AA/NA
Mental Health
Mentoring

Directly

Referral

__________
__________
__________
__________
__________
__________
__________
__________
__________
__________

__________
__________
__________
__________
__________
__________
__________
__________
__________
__________

__________
__________
__________
__________
__________
__________
__________
__________

__________
__________
__________
__________
__________
__________
__________
__________

__________
__________
__________

__________
__________
__________

Rate the adequacy of aftercare programming.

84

0

1

Aftercare not
offered

Very inadequate

2

3

4

5
Very adequate

Does the offender meet with the aftercare providers before they are released from your program?
__ Yes
__ No
__ N/A
Which of the following describes the typical practice for your program in setting up aftercare? __ N/A
__ staff give the offender phone number of agencies for him/her to call
__ Staff will allow the offender to call agencies in their office
__ Staff will call the agency for the offender
__ Staff will set-up initial appointment with the agency for the offender
__ Staff will physically transport the offender to the agency
__ Staff will check to make sure that offender went to the initial appointment
Rate the adequacy of formal arrangements between your agency and other programs for providing a continuum of
care to offenders once they leave your program.
0

1

No formal
arrangements

Very inadequate

2

3

4

5
Very adequate

Does your program work with any of the following types of programs to assist offenders with any additionally
needed services?
Mental health agencies
__ Yes
__ No
__ Does not apply
Parenting agencies
__ Yes
__ No
__ Does not apply
Mentoring agencies
__ Yes
__ No
__ Does not apply
Boys/Girls club agencies
__ Yes
__ No
__ Does not apply
YMCA
__ Yes
__ No
__ Does not apply
YWCA
__ Yes
__ No
__ Does not apply
Government agencies offering support
__ Yes
__ No
__ Does not apply
Local community agencies
__ Yes
__ No
__ Does not apply
Housing agencies
__ Yes
__ No
__ Does not apply
Law enforcement agencies
__ Yes
__ No
__ Does not apply
Courts
__ Yes
__ No
__ Does not apply
Medical agencies
__ Yes
__ No
__ Does not apply
Financial support agencies
__ Yes
__ No
__ Does not apply
Local religious institutions
__ Yes
__ No
__ Does not apply
Local community agencies specifically
For minorities
__ Yes
__ No
__ Does not apply
Other: _________________________________________________________________________

85

How does the program initiate assistance for the offenders by these agencies once the offenders have completed the
program (check all that apply)?
__ Have open house for the agencies
__ Have offenders do community service for local agencies
__ Have fundraisers for local agencies
__ Have fundraisers for charity with local agencies
__ A person from our agency is responsible for the partnerships with other agencies
__ Members from other agencies come to talk with our offenders
__ Members from other agencies come to perform a free service for our offenders
__ Other: __________________________________________________________
__ Other: __________________________________________________________
__ Other: __________________________________________________________
Have the activities described above helped the programs relationships with these agencies?
__ Yes
__ No
If the answer above is yes, how has the relationship been helped (checked all that apply)?
__ Opened the lines of communication between the agencies
__ Increased the lines of communication between the agencies
__ Allowed the offenders more opportunities upon release
__ Allowed the offender to practice prosocial skills within the community
__ Allowed the offender to obtain prosocial friends within the community
__ Other (please specify: _________________________________________________
__ Other (please specify: _________________________________________________
__ Our program does not have any ties with any other programs
Rate the collaboration and cooperation between your agency and other agencies.
0

1

Not applicable

Very unsatisfactory

2

3

4

5
Very satisfactory

86

Section 5: ORGANIZATIONAL RESPONSITIVITY
Rate the adequacy of support that your program receives from your parental agency.
0

1

2

3

4

Not applicable

5
Very adequate

Rate the adequacy of the support that your program receives from other treatment agencies.
0

1

Not applicable

Very inadequate

2

3

4

5
Very adequate

Rate the adequacy of the support that your program receives from the probation or parole department.
0

1

Not applicable

Very inadequate

2

3

4

5
Very adequate

Rate the adequacy of the support that your program receives from the local courts.
0

1

Not applicable

Very inadequate

2

3

4

5
Very adequate

How knowledgeable are the local courts in the empirical literature of “best practices” for offenders?
0

1

Not applicable
knowledgeable

Not knowledgeable

2

3

4

5
Very

How supportive are the local courts in allowing your program to adhere to the empirical research on “best practices”
for offenders?
0

1

Not applicable

Not supportive

2

3

4

5
Very supportive

Rate the level of political constraints that are imposed on your program.
0

1

Not applicable

Many political
constraints

2

3

4

5
No political
constraints

Rate the impact these constraints have had on your program.
0

1

Not applicable

Not supportive

2

3

4

5
Very supportive

Rate the level of involvement of the advisory board.
0

1

No advisory

Not involved

2

3

4

5
Very involved

board

87

Does the advisory board include members of the:
Criminal justice community?
__ Yes __ No
The local community?
__ Yes __ No
The larger community?
__ Yes __ No
Your program?
__ Yes __ No
Your parent agency?
__ Yes __ No
Other (please specify): ______________________________ )
Not applicable
__ Yes __ No
Is your program adequately funded to sustain the programs?
__ Yes
__ No
Rate the adequacy of current funding.
0

1

2

3

4

Inadequate funding

5
Very adequate funding

Have there been any changes in the program itself during the past two years that have jeopardized the smooth
functioning of the program?
0

1

2

3

4

No changes

5
Many changes

What changes have occurred to the program within the last 2 years, which have had a negative impact on the
program (check all that apply)?
__ There have been no changes to the program
__ Staff turnover
__ Staff morale
__ Obtaining inappropriate offenders
__ Change in the day-to-day operations of the program
__ Reduction in funding
__ Reduction in the number of groups being offered
__ Other: _________________________________________________________________________
Have there been any changes in the area of program funding during the past two years that have jeopardized the
smooth functioning of the program?
0
No changes

1

2

3

4

5
Many changes

What changes have occurred in terms of the program funding, that has jeopardized the smooth functioning of the
program?
List Changes:
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
__ Changes have not negatively affected the program

88

What kind of relationship does the program perceive to have with the community-at-large?
__ The community is very supportive of the program
__ The community is somewhat supportive of the program
__ The community is not supportive of the program
__ The community does not know that the program exists
Does the program perceive the criminal justice community (i.e., judges, police department, sheriff’s department,
DOC) as supportive of the program?
__ The criminal justice community is very supportive of the program
__ The criminal justice community is somewhat supportive of the program
__ The criminal justice community is not supportive of the program
Have there been any changes in the area of the community support during the past two years that have jeopardized
the smooth functioning of the program?
0

1

2

3

4

No changes

5
Many changes

What changes have occurred within the last 2 years in the following agencies level of support for the program?
Community-at-large
Courts
Law Enforcement
DOC
Other treatment agencies
Advisory board

__ Increased
__ Increased
__ Increased
__ Increased
__ Increased
__ Increased

__ Decreased
__ Decreased
__ Decreased
__ Decreased
__ Decreased
__ Decreased

__ Stayed the same
__ Stayed the same
__ Stayed the same
__ Stayed the same
__ Stayed the same
__ Stayed the same

What changes have occurred in terms of the community support that has jeopardized the smooth functioning of the
program?
List Changes:
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
_____________________________________________________________________________________________
__ Changes have not negatively affected the program
What factors are of concern for the funding source (please check all that apply)?
__ The program is not cost-effective
__ The funding source has cut the funds
__ The program is not receiving enough referrals
__ N/A – the program has adequate funding
__ Other (please specify: ______________________________________________________ )

89

For each the following items please indicate if the agencies (i.e., parent organization, other treatment agencies,
probation, and courts) contribute the following to your program. (Check all that apply)
Parent
agency/organization

Other TX agencies

Probation/Parole or
state

Courts

Financial support
Clinical support
Support for
assessments
Interest in the results
of assessments
Support for
reassessments
Interest in the result
of the reassessments
Support for
curriculum-based
treatment groups
Support for tracking
recidivism of
offenders
Interest in the results
of the tracking of
recidivism
Evaluation process
conducted on the
program
Results form the
evaluation conducted
on the program
Support for training
Support for
implementing
“principles of
effective
interventions”

If financial support was available, would the program:
Implement standardized risk/need instruments
Implement standardized substance abuse instruments
Implement standardized personality instruments
Implement standardized attitudinal instruments
Implement standardized specialized instruments
Implement empirical based curriculums for criminal thinking
Implement empirical based curriculums for substance abuse
Implement empirical based curriculums for specialized
Improve the training for new staff
Improve the training for current staff
Improve the evaluation of staff
Improve aftercare services
Conduct evaluations of the program
Reassess the offenders using instruments
Collect recidivism data

__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes
__ Yes

__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No
__ No

__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A
__ N/A

90

Section 6: PROGRAM EVALUATION
Does your program have a document that outlines the measures of specific program goals?
__ Yes
__ No
If yes, does your program collect data for these measures manually or through a computerized database?
__ Manual
__ Database
Does your program collect data or track the following:
Assessment data
__ Yes
__ No
__ Does not apply
Attitudinal measures (pre-test)
__ Yes
__ No
__ Does not apply
Attitudinal Measures (post-test)
__ Yes
__ No
__ Does not apply
Placement in treatment programs
__ Yes
__ No
__ Does not apply
Placement in outside agencies
__ Yes
__ No
__ Does not apply
Offender progress
__ Yes
__ No
__ Does not apply
Technical violations
__ Yes
__ No
__ Does not apply
Sanctions
__ Yes
__ No
__ Does not apply
Program outcomes
__ Yes
__ No
__ Does not apply
Other (please specify: ____________________________________________________________ )
Does your program have an automated system that tracks:
a. Participate progress?
__ Yes
__ No
b. Risk assessment?
__ Yes
__ No
c. Needs?
__ Yes
__ No
d. Placement in treatment programs?
__ Yes
__ No
e. Placement in outside programs?
__ Yes
__ No
f. Recidivism?
__ Yes
__ No
g. Violations?
__ Yes
__ No
h. Sanctions?
__ Yes
__ No
i. Program outcomes?
__ Yes
__ No
j. Other (please specify: _____________________________________________ )
Rate the adequacy of the data collection processes.
0

1

No data
collection

Very inadequate

2

3

4

5
Very adequate

Does your program track offender recidivism for offenders who:
__ Have successfully completed the program
__ Have left the program prior to completion (voluntarily or by program request)
__ Were eligible for the program but did not participate for various reasons
__ Offenders are not tracked
How does your program track offender recidivism once an offender has left the program?
__ Yes-the program track re-arrests
__ Yes-the program tracks reconvictions
__ Yes-the program tracks re-incarcerations
__ No-the program does not track recidivism
How long does your program track offenders after they have left the program:
__ 30days
__ 2 months
__ 6 months
__ 12 months
__ 18 months
__ 24 months
__ 36 months
__ Not tracked
__ Other: _____________________________________________________________

91

Rate the adequacy of the process to track recidivism once offenders leave the program.
0

1

Recidivism
not tracked

Very inadequate

2

3

4

5
Very adequate

Besides a financial audit, has your program ever been evaluated by:
A Contracted Outside Reviewer?
__ Yes
__ No
A Volunteer Outside Reviewer?
__ Yes
__ No
An Individual Internal Reviewer?
__ Yes
__ No
An Internal Research Division?
__ Yes
__ No
Other (please specify: __________________________________________________ )
What type of evaluation has your program completed? (check all that apply)
__ Process Evaluation
__ Outcome Evaluation
__ Program Assessment (i.e. CPAI)
__ Other : _______________________________
Did the evaluations rate your program?:
__ Favorably
__ As having no effect

__ Unfavorably

__ N/A

Did the evaluator use a comparison group in this evaluation?
__ Yes
__ No
__ N/A
Was this evaluation published in a(n):
__ Edited journal
__ Trade publication
__ Newsletter
__ Unpublished report
__ N/A
__ Other _______________________________________________________
Has there been an outcome evaluation done in the last five years?
__ Yes-an outcome evaluation was conducted with a comparison group
__ Yes-an outcome evaluation was conducted, but there was not a comparison group
__ No-an outcome evaluation was not conducted.
Rate the adequacy of the program’s evaluation protocol.
0

1

No evaluation
protocol

Very inadequate

2

3

4

5
Very adequate

Which of the following mechanisms are in place for offenders to have input into the structure of the program (check
all that apply)?
__ Offender suggestion box
__ Treatment plan
__ Institutional/programmatic chain of command
__ KITE system
__ Unit representative
__ House meeting
__ There is no mechanism in place
__ Other (please specify) ______________________________________________________

92

If there is a mechanism in place for offender input, have there been any changes in the program based on the input of
the offender (check all that apply)?
__ More activities
__ Different activities
__ Different treatment groups added
__ The schedule ahs changed based on the input of the offender
__ The program has implemented different types of rewards
__ The program has added or changed certain foods
__ There is not mechanism for offender input
How is offender satisfaction determined?
__ Offender satisfaction surveys
__ Offender interviews
__ Grievance procedures
__ There are no formal mechanisms for determining offender satisfaction
How frequently is offender satisfaction determined? __N/A
__ Monthly
__ Quarterly
__ Annually
__ At the end of each treatment group
__ When offenders are released from the program
How was the assessment conducted: __ N/A – offender satisfaction is not formerly assessed.
By an internal reviewer
__ Yes
__ No
By an outside reviewer
__ Yes
__ No
By a survey to all offenders
__ Yes
__ No
By a survey to a sample of offenders
__ Yes
__ No
By interviews with all offenders
__ Yes
__ No
By interviews with a sample of offenders __ Yes
__ No
As part of a formal evaluation
__ Yes
__ No
What were some of the issues that were identified by offenders as impacting their level of satisfaction during the
past year?
__ N/A – offender satisfaction is not formerly assessed.
Quality of the program
__ Yes
__ No
Safety
__ Yes
__ No
Ability of the program
__ Yes
__ No
Staff treatment of offenders
__ Yes
__ No
Instability of staff
__ Yes
__ No
Usefulness of the curriculum
__ Yes
__ No
Appropriateness of the treatment modality
__ Yes
__ No
Frustration with the administration
__ Yes
__ No
Lack of ownership of the program
__ Yes
__ No
Use of rewards
__ Yes
__ No
Use of punishments
__ Yes
__ No
Other: (please specify: __________________________________________________ )
Rate the adequacy of the mechanisms for offenders to provide input into the structure of the program.
0

1

No offender
input

Very inadequate

2

3

4

5
Very adequate

93

What mechanisms are in place for staff to give their input into the program?
(TX=Treatment)
__ Staff satisfaction surveys
__ TX staff
__ Non-TX staff
__ Staff meetings to discuss issues __ TX staff
__ Non-TX staff
__ Staff suggestion box
__ TX staff
__ Non-TX staff
__ No staff input
__ TX staff
__ Non-TX staff
__ Other: _____________________________________________________________
Which of the following staff satisfaction issues have been addressed during the past year:
Pay
__ Yes
__ No
Safety
__ Yes
__ No
Quality of life factors
__ Yes
__ No
Change in responsibilities
__ Yes
__ No
Job instability
__ Yes
__ No
Insufficient training
__ Yes
__ No
Lack of adequate staffing
__ Yes
__ No
Job security
__ Yes
__ No
Frustration with the administration
__ Yes
__ No
Lack of ownership of the program
__ Yes
__ No
Staff satisfaction not assessed
__ Yes
__ No
Other: (please specify: ________________________________________ )
Rate the ability of treatment staff to provide input into the structure of the program.
0
1
2
3
4
No input

__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified
__ Not Identified

5
A great deal
of input

Has the level of community satisfaction with your program been assessed?
__ Yes
__ No
Was this assessment conducted on the:
N/A – community satisfaction is not formally expressed __
Criminal justice community?
__ Yes
__ No
The local community?
__ Yes
__ No
The larger community?
__ Yes
__ No
Other (please specify: _____________________________________________ )
Whish of the following issues has been addressed during the past year: __ N/A
Quality of the program
__ Yes __ No __ Not Identified
Safety
__ Yes __ No __ Not Identified
Ability of the program
__ Yes __ No __ Not Identified
Staff treatment of offenders
__ Yes __ No __ Not Identified
Instability of staff
__ Yes __ No __ Not Identified
Usefulness of the curriculum
__ Yes __ No __ Not Identified
Appropriateness of the treatment modality
__ Yes __ No __ Not Identified
Frustration with the administration
__ Yes __ No __ Not Identified
Lack of ownership of the program
__ Yes __ No __ Not Identified
Use of rewards
__ Yes __ No __ Not Identified
Use of punishments
__ Yes __ No __ Not Identified
Other: (please specify: __________________________________________________ )
Rate the adequacy of quality assurance.
0
1
2
No internal
Very inadequate
quality assurance

3

4

5
Very adequate

94

SECTION 7: OTHER
Does the program Manager currently (check all the apply)?
__ Review potential staff resumes
__ Interview potential staff members
__ Make recommendations for hiring potential staff
__ Meet with a committee to make the final decision regarding the hiring of new staff
__ Make the final decision regarding the hiring of new staff
__ Review the policy and procedures of the program with new staff
__ Conduct training sessions with new staff
__ Conduct periodic training sessions in-house for he treatment staff
__ Will allow the new staff to shadow the director
__ Have another person responsible for providing training for new staff
__ Meet at least once a month with the treatment staff
__ Periodically meet with the treatment staff individually
__ Have another person who is responsible for supervising the treatment staff
__ Assess offenders
__ Facilitate groups
__ Co-Facilitate groups
__ Have a regular caseload
__ Have a specialized caseload
Where are the offenders’ records kept?
__ In a central locked filing cabinet
__ In a central locked filing cabinet in a central locked room
__ In an unlocked room
__ In the staff locked office
__ In the staff locked cabinet in a locked office
__ The records are kept in a password-protected computer
__ N/A – offender records are not kept
__ Other: _________________________________________________________________________
Does the program have ethical guidelines that staff are required to review?
__ Yes
__ No
What process is used when making changes to the program? (check all that apply)
__ A review of the literature is conducted
__ A formal testing period is utilized
__ Changes are discussed with staff prior to implementation
__ Once implemented, changes are tweaked as needed on an on-going basis
__ Once implemented, changes are no longer discussed
__ Changes have not been made to the program
Rate the process of testing changes to the program (regarding assessment, treatment groups, etc.)
0
1
2
no testing done Changes made as
needed; no formal
test period

3

4

5
Specific period set
aside for testing
with changes made
in response to the
testing period.

__ Does not apply

95