Special Report
T he B oa r d o f Pa rol e hearings:
p s y c h o l o g i c a l e val uat ions and
mandato ry Tr a i n i ng Requirem ent s

Office of the
Inspector General
Dav i d R . S h aw
Inspector General

S tat e o f C a l i f o r n i a
J u ly 2 0 1 0

Office of the Inspector General

David R. Shaw, Inspector General

July 1, 2010

Matthew L. Cate, Secretary
California Department of Corrections and Rehabilitation
1515 S Street, Room 502 South
Sacramento, California 95814

Dear Mr. Cate:
,
Enclosed is the Office of the Inspector General's special report concerning the Board of Parole
Hearings' (parole board) process for preparing psychological evaluations used for parole
suitability hearings and of the parole board's commissioner training program. The purpose of the
special report was to review concerns expressed by the Senate Rules Committee for two particular
issues: (1) factual errors that may exist in psychological evaluations and (2) certain psychologists
who give elevated risk assessment conclusions when compared to conclusions made in prior
evaluations. Additionally, the parole board's executive officer formally requested us to examine
its new commissioner training program.

The Office of the Inspector General found that the parole board lacks reliable data to determine
the number of factual errors contained in psychological evaluations and lacks reliable data to
determine the number oflow, medium, and high risk assessment conclusions--data that would
allow it to perform certain analytical procedures to measure performance. In addition, we found
weaknesses in the parole board's oversight of the methods it uses to review psychological
evaluations. Specifically, it does not require senior psychologists to use source documentation
when conducting their reviews, limiting the effectiveness ofthe reviewers' ability to deteCt
. certain mistakes. The parole board also does not actively monitor senior psychologists' activities
by requiring them to account for their time by case or by activity. Finally, the parole board failed
to provide commissioners, deputy commissioners, and senior psychologists with the sufficient
number of mandatory training hours.
We would like to tha~ you and your staff for the cooperation extended to my staff in completing
this special report. If you have any questions concerning this report, please contact Jerry Twomey,
Chief Assistant Inspector General, Bureau of Audits and Investigations, at (916) 830-3600.

P.O. Box 348780,

Arnold Schwarzenegger. Governor
95834-8780 PHONE (916) 830-3600

SACRAMENTO, CALIFORNIA

FAX

(916) 928-4684

Mr. Matthew L. Cate, Secretary
Page 2
July 1,2010

Sincerely,

David R. Shaw
Inspector General

cc:

Martin Hoshino, Executive Officer, Board of Parole Hearings
Kim Holt, External Audits Manager, California Department of Corrections and
Rehabilitation

Contents
Executive Summary .................................................................................................................. 1
Introduction............................................................................................................................... 5
Background ............................................................................................................................... 5
Objectives, Scope, and Methodology .......................................................................................10
Finding 1 ...................................................................................................................................12
The Board of Parole Hearings Does Not Have Reliable Psychological Evaluation
Data to Effectively Analyze the Number of Reported Errors or the Types of Risk
Assessment Conclusions

Finding 2 ...................................................................................................................................20
The Board of Parole Hearings Can Strengthen Its Process of Reviewing
Psychological Evaluations

Finding 3 ...................................................................................................................................25
The Board of Parole Hearings Failed to Provide All Required Training to Its
Commissioners, Deputy Commissioners, and Senior Psychologists

California Department of Corrections and Rehabilitation’s Response.....................................28

Executive Summary
The Senate Rules Committee, a standing
committee in the Legislature that reviews
proposed changes to rules and other matters
relating to the business of the Legislature,
requested that the Office of the Inspector
General (OIG) review the Board of Parole
Hearings’ (parole board) process of preparing
psychological evaluations. The committee
was concerned about factual errors in
psychological evaluations and concerned
about certain psychologists whose risk
assessment conclusions are elevated compared
to conclusions made in previous evaluations.
In addition, the parole board’s executive
officer requested that the OIG review its new
commissioner training program.

Findings in Brief
The Office of the Inspector General found that
the Board of Parole Hearings (parole board) lacks
reliable data to determine the number of factual
errors contained in psychological evaluations
and lacks reliable data to determine the number
of low-, medium-, and high-risk assessment
conclusions in evaluations. Reliable data would
allow the parole board to perform certain analytical
procedures to measure performance.
In addition, we found weaknesses in the parole
board’s oversight of the methods it uses to review
psychological evaluations. Specifically, the parole
board does not require senior psychologists to
use source documentation when conducting
their reviews, thereby limiting the reviewers’
effectiveness in detecting certain mistakes. In
addition, the parole board does not actively monitor
senior psychologists’ activities by requiring them to
account for their time by case or by activity.

The parole board is responsible for, among
other things, determining whether inmates
serving life sentences with the possibility
Finally, the parole board failed to provide most of
its commissioners, deputy commissioners, and
of parole can safely re-enter society. In so
senior psychologists with the sufficient number of
doing, the parole board’s 12 commissioners
mandatory training hours.
and approximately 70 deputy commissioners
preside over parole suitability hearings
for eligible adult inmates. When making
determinations of suitability, a hearing panel comprised of at least one commissioner and one
deputy commissioner weighs a variety of factors, one of which is the inmate’s past and present
mental state. Hearing panels evaluate inmates’ mental state by reviewing Comprehensive Risk
Assessments prepared by the parole board’s in-house psychologists. These risk assessments, which
are commonly referred to as psychological evaluations, include a clinical opinion of the inmate’s
potential risk for committing violence if released from prison.
In January 2009, the parole board made changes to its process: it began preparing psychological
evaluations by adopting standardized risk assessment instruments and reporting formats. The
parole board also hired 30 new psychologists in late 2008; however, 14 of those psychologists were
hired for a limited term and will no longer be employed by the parole board after June 30, 2010.
Nevertheless, the parole board’s executive management believes that it will remain current with its
workload of psychological evaluations at least until October 2010.
During this review, we found that the parole board’s data concerning the number of factual
errors discovered in psychological evaluations during hearings held in 2009 and in the risk
assessment conclusions for evaluations in 2009 is not reliable and therefore is less meaningful to
State of California • July 2010	

Page 1

its management. According to the parole board’s Lifer Scheduling and Tracking System (tracking
system), commissioners reported between one and four factual errors in each of the psychological
evaluations of 35 inmates who had parole suitability hearings in 2009. However, we question the
reliability of this number because we found examples in which commissioners both overstated
and understated the number of factual errors in the tracking system. Consequently, the number of
evaluations with factual errors, as well as the total number of factual errors, is unknown.
In addition, the parole board does not have reliable data with which to effectively analyze risk
assessment conclusions. Using a spreadsheet, the parole board began collecting detailed information
about psychological evaluations completed in 2009; however, the central repository of this data was
not complete or always accurate. As a result, the parole board cannot analyze the data for problematic
trends or patterns, such as identifying psychologists who provide risk assessment conclusions that
are potentially inconsistent with the types of inmates they evaluate or psychologists who provide a
disproportionate number of risk assessment conclusions that are too high or too low. If the parole
board had accurate data, it could review summary reports and more effectively measure its quality
control processes, identify new training needs, and make necessary procedural changes.
Moreover, we found weaknesses in the parole board’s oversight of its methods of reviewing
psychological evaluations. For example, the parole board does not require that senior psychologists
use source documentation gathered by their subordinate psychologists when they perform
supervisory reviews of psychological evaluations. Consequently, the senior psychologists may not
be well-equipped to detect misstatements or inaccuracies during their review. In addition, the parole
board does not actively monitor its senior psychologists’ activities by requiring them to account for
their time either by case or by activity. As a result, the parole board lacks a necessary management
tool to measure and monitor its employees’ performance. Although psychological evaluations are
only one of several types of relevant information that commissioners consider in making parole
suitability decisions, the sheer magnitude of these decisions warrants that commissioners have the
most accurate information possible.
Furthermore, we found that the parole board did not provide nearly all of its current commissioners
and deputy commissioners with at least 40 hours of annual training, as the law requires.
Specifically, the parole board did not provide the three current commissioners appointed before
2008 with 40 hours of annual training in 2008, nor did it provide the seven current commissioners
appointed before 2009 with 40 hours of annual training in 2009. Furthermore, the parole board
provided only three of 16 current deputy commissioners (19 percent) hired in 2009 with 40
hours of training within 60 days of their appointment as required, and provided only two of 70
current deputy commissioners (3 percent) hired before 2009 with 40 hours of annual training
as required. Commissioners and deputy commissioners who do not receive training may not be
fully prepared to perform their duties and responsibilities, which could affect hearing decisions. If
commissioners and deputy commissioners make mistakes because they were not trained in certain
areas, those mistakes could result in legal challenges and a weakening of stakeholders’ confidence
in the suitability hearing process. Finally, the parole board did not provide all four of its senior
psychologists with 80 hours of mandatory supervisory training within one year of appointment to
a supervisory position. Consequently, those supervising employees may not be aware of certain
techniques for supervising and instructing employees and protecting employees’ rights.
Bureau of Audits and Investigations, Office of the Inspector General	

Page 2

As a follow-up to this report, the OIG will evaluate the risk assessments for a sample of
psychological evaluations to review the soundness of the clinical opinions and determine whether
the evaluations contain factual errors. We are in the process of retaining an independent licensed
psychologist to perform this work, and we will report on the results of that review.

Recommendations
To strengthen its processes for collecting and using data and reviewing psychological evaluations,
the parole board should take the following actions:
•	Ensure that all of its commissioners who preside over parole suitability hearings receive formal
training on how and when to report factual errors in the Lifer Scheduling and Tracking System
(tracking system). This action should include ensuring that commissioners clearly articulate on
the record the disposition of claimed errors in psychological evaluations and record the result in
the tracking system accurately.
•	Periodically review a report on the number and types of factual errors in psychological
evaluations recorded in the tracking system. This review would identify training opportunities for
certain psychologists who repeatedly make the same types of errors and would also determine
whether changes in the tracking system process are necessary.
•	Ensure that psychologists report to the chief psychologist all of their risk assessment conclusion
information so that the chief psychologist can analyze the data for patterns of conclusions and can
identify trends over time.
•	Ensure that senior psychologists review copies of source documents from the inmate’s central
file when reviewing psychological evaluations for parole suitability hearings. This source
documentation review will increase the integrity of the review process by helping to identify
misstatements or inaccuracies contained within the evaluations.
•	Develop a review checklist for senior psychologists to follow when conducting reviews
of psychological evaluations. This checklist would aid in fact-checking the statements and
conclusions in the psychological evaluations and ensure consistency and thoroughness among
reviewers.
•	Develop a time benchmark for reviewing psychological evaluations to ensure that reviewers
spend at least the minimum amount of time conducting their reviews.
•	Require senior psychologists to report to the chief psychologist their time spent reviewing draft
evaluations. This information would help the chief psychologist track productivity and manage
workload levels.
To ensure that its employees receive the required number of training hours, the parole board should
regularly monitor training activities and the number of training hours that it provides to all staff.
The parole board should then make the necessary scheduling adjustments for those individuals who
have not received enough hours.

State of California • July 2010	

Page 3

Department’s Response
The California Department of Corrections and Rehabilitation (CDCR) indicated that it would
implement many of our recommendations in the coming fiscal year. The CDCR also stated that
it is exploring or has already implemented different methods to ensure that it meets the number of
training hours required for commissioners and deputy commissioners annually. Finally, the CDCR
stated that it will monitor and document the Board of Parole Hearings’ progress in addressing the
report’s recommendations.

Bureau of Audits and Investigations, Office of the Inspector General	

Page 4

Introduction
Background
Among other responsibilities, the Board of Parole Hearings (parole board) conducts parole
suitability hearings for all inmates sentenced to life in prison with the possibility of parole (referred
to as an “indeterminate sentence”) to determine whether the inmate can re-enter society or requires
a longer period of incarceration to ensure public safety.1 Only inmates sentenced to life in prison
with the possibility of parole are subject to suitability hearings, for which they automatically
become eligible one year before completing their minimum sentence. Merely being scheduled for a
suitability hearing does not indicate an inmate’s suitability for release from prison. Rather, a hearing
panel—consisting of one commissioner and one deputy commissioner—determines an inmate’s
suitability for parole.
The parole board is one of several divisions or boards within the California Department of
Corrections and Rehabilitation. The parole board has 12 commissioners who conduct hearings
related to adult offenders. These commissioners are appointed by the governor and confirmed by the
Senate. The parole board also uses about 70 deputy commissioners for parole suitability hearings.
Hearing panels weigh various factors when determining suitability for parole
The California Code of Regulations requires that the hearing panel consider all relevant and reliable
information available to the panel when determining whether an inmate is suitable for release on
parole. When making its determination, a hearing panel weighs a variety of factors, including
an inmate’s social history, past and present mental state, criminal history, commitment offenses,
past and present attitude toward the crime, any conditions of treatment or control, and any other
information bearing on the inmate’s suitability for release.
According to state law, regulations, and case law governing parole suitability decisions, a hearing
panel shall find an inmate unsuitable for parole if, in the judgment of the panel, the inmate will pose
a current, unreasonable risk of danger to society if released from prison. Table 1 on the following
page shows examples of factors that tend to indicate suitability and unsuitability for parole.
Parole suitability hearing results for January 1, 2007, through December 31, 2009
According to the parole board’s Lifer Scheduling and Tracking System, the number of suitability
hearings has increased each calendar year from 2007 through 2009. Figure 1 (on page 7) shows
the percentage of parole suitability hearing results by type for the period January 1, 2007, through
December 31, 2009.
The parole board’s data also shows—as illustrated in Figure 2 (on page 7)—that for the three-year
period, the number of parole grants has increased each year, while denials and postponements have
decreased overall since 2007. Where a decision of grant or denial was reached, the percentage of
1 An indeterminate sentence is a prison term that is not fixed in advance by the court but instead ranges over an
indefinite period, such as “25 years to life.”
State of California • July 2010	

Page 5

Table 1: Factors Commissioners Consider For Determining the Suitability or Unsuitability of Inmates at
Hearings.

Suitability

Unsuitability

•	 No juvenile record (of assaulting or
potentially harming others)

•	 Commitment offense (especially heinous,
atrocious, or cruel manner)

•	 Stable social history

•	 Previous record of serious violence

•	 Signs of remorse

•	 Unstable social history

•	 Motivation for crime was significant stress
in life

•	 History of sadistic sexual offenses

•	 Crime was the result of Battered Woman
Syndrome
•	 Lack of significant history of violent crime

•	 Psychological factors (lengthy history of
severe mental problems)
•	 Institutional behavior

•	 Present age
•	 Realistic plans for future
•	 Institutional behavior
Source: The California Code of Regulations, Title 15, Section 2402 (c) and (d).

suitability hearings in which commissioners granted parole quadrupled, from 4 percent in 2007 to
16 percent in 2009 (refer to Figure 3 on page 8).
The parole board recently made changes to improve its process of preparing
psychological evaluations
Before January 1, 2009, psychological evaluations prepared for parole suitability hearings followed
differing report formats and included a variety of means to determine an inmate’s potential for
violence if released from prison. Some of the evaluations did not include the use of recognized risk
assessment tools and did not have an expiration date. Further, the parole board noted that it did not
employ a sufficient number of psychologists to complete its scheduled psychological evaluations,
which led to postponements and an increase in hearings conducted outside of the required time period.
In late 2008, the parole board hired an additional 30 psychologists, including two senior
psychologists and a chief psychologist. And in January 2009, the parole board adopted a new
approach for preparing psychological evaluations, one that uses standardized risk assessment
instruments and consistent report formats. Generally speaking, this new process includes a mental
status examination and an evaluation of the inmate’s remorse, emotional functioning, insight, selfunderstanding, psychiatric symptoms, and risk factors from the inmate’s history, such as the role
drugs and alcohol played in the commitment offense. The process also includes an explanation of
the commitment offense and the inmate’s attitude regarding it, as well as a history of the inmate’s
prison programming. The evaluation process culminates in the psychologist’s opinion, based on
available data, of the inmate’s potential for future violence. This opinion is expressed in terms
of the inmate’s having a “low,” “moderate,” or “high” risk for future violence. The parole board
explains that it uses the psychological evaluation as relevant information to help commissioners
make determinations about the inmate’s mental health and attitude towards his or her crime.
Bureau of Audits and Investigations, Office of the Inspector General	

Page 6

Figure 1: Composition of the Results of Parole Suitability Hearings for the Period January 1, 2007
through December 31, 2009.
Grants
5%

Other*
12%

Denials
44%

Waivers
7%

Postponements
32%

Source: The Board of Parole Hearings’
Lifer Scheduling and Tracking System.
Unaudited data.
3,500

3,000

Figure 2: Three-Year Comparison of Suitability Hearing Results for January 1, 2007, through
December
2,500 31, 2009.
3,500
2,000

2007
3,000

2008
2009

1,500
2,500
1,000
2,000

2007
2008

500

2009

1,500
-

Grants
1,000

Denials

Postponements

Waivers

Grants

Denials

Postponements

Other*

500
-

Waivers

Other*

Source: The Board of Parole Hearings’ Lifer Scheduling and Tracking System. Unaudited data.

The “Other” category in Figures 1 and 2 includes stipulations, in which an inmate agrees in advance of
a hearing that he or she is unsuitable for parole; split decisions, in which the hearing panel disagrees on
the decision, resulting in a “tie” vote; cancellations, in which the hearing is cancelled for circumstances
such as a change in the inmate’s minimum eligible release date or the inmate’s death; continuations, in
which a hearing is stopped and later reheard by the same panel; and a category titled, “No Result.”
The “Grant” category in Figures 1, 2, and 3 includes decisions made by the parole board but does not
necessarily equate to the number of inmates actually released.
State of California • July 2010	

Page 7

Figure 3: Percentage Comparison of Parole Grants and Denials at Suitability Hearings in 2007 and 2009.
Grants
4%

Percentage
of Grants and
Denials in
2007

96%
Denials

Grants

Percentage of
Grants and
Denials in
2009

16%

84%
Denials

Source: The Board of Parole Hearings’ Lifer Scheduling and Tracking System. Unaudited data.

The process of preparing psychological evaluations begins when a staff psychologist travels to the
prison for a face-to-face interview with the inmate. At this visit, the psychologist reviews the inmate’s
central file and medical information and later uses this information to prepare a draft evaluation using
a number of standardized risk assessment instruments. These instruments include the following:
•	Hare Psychopathy Check List-Revised, which is a diagnostic tool used to identify psychopathic
tendencies. This tool uses a semi-structured interview, file documentation, and collateral
information to measure inferred personality traits and behaviors related to a traditional conception
of psychopathy.
•	Historic-Clinical-Risk-Management-20 (HCR-20), which is an assessment tool that provides
an analysis of an individual’s risk potential. This tool consists of a set of structured professional
guidelines for the assessment of risk for institutional and community violence in people with
mental abnormalities, personality disorders, or substance use disorders.  According to the parole
board, the HCR-20 has been cross-validated for use in correctional, forensic, and psychiatric
settings. 
•	Level of Service/Case Management Inventory, which is a tool designed to assist professionals
in managing and treating offenders in correctional and forensic settings. The tool includes both
an actuarial estimate of criminal recidivism and a full service case management component.
However, the parole board notes that its psychologists use the tool for assessing general
criminal recidivism potential (but not violence per se) and not for treatment planning and case
management.
•	Static-99-R, which is an instrument used for assessing risk for sexual and nonsexual violence
recidivism. This tool is intended for male individuals who have been arrested for, or convicted of,
a sex offense as an adult (and in some cases as a juvenile), or when there is reason to believe that
a sex offense against an identifiable victim had occurred.
The parole board’s chief psychologist estimates that it takes psychologists an average of 21
hours (spanning approximately 45 days) to prepare an evaluation from start to finish. The staff
Bureau of Audits and Investigations, Office of the Inspector General	

Page 8

psychologist forwards the draft evaluation to a senior psychologist for review and approval.
After the senior psychologist approves the evaluation, the parole board sends it to the prison
where copies are provided to the inmate and placed in the inmate’s central file. State law permits
an inmate the opportunity to review his or her file (which presumably includes a psychological
evaluation) at least 10 days prior to any hearing conducted by the parole board.2 According to the
parole board’s orientation manual, evaluations completed on or after January 1, 2009, are valid
for five years; however, evaluations completed before 2009 are valid for three years from the date
of the evaluation or will expire by attrition after use in a hearing occurring after January 1, 2009,
whichever occurs first.
According to the parole board, it will lose 14 staff psychologists on June 30, 2010, when these
employees’ limited term positions expire. After this staffing reduction, the parole board will have
35 psychologists, including four senior psychologists and a chief psychologist. When we spoke
to the parole board’s executive officer and chief deputy of program operations about this issue,
the executive officer said that due to a number of variables affecting the future pace of suitability
hearings, he could not predict if or when the parole board would begin to experience a negative
impact from the staffing reduction. The executive officer noted that one of the variables is the
commissioners’ availability to conduct suitability hearings. The chief deputy of program operations
explained that, due to Marsy’s law, the parole board’s future suitability hearing workload could be
less compared to the workload of previous years because the minimum denial period changed from
one year to three years.3 He also said that because the parole board schedules the preparation of
evaluations 180 days in advance, he believes that even with the loss of 14 psychologists, the parole
board will be able to keep up with its psychological evaluations workload at least until October 2010. 

2 In re Rutherford, the court ordered, in part, that the parole board make psychological evaluations available to
the inmate and the inmate’s attorney at least 60 days prior to the date of the inmate’s scheduled parole suitability
hearing. According to the parole board’s Lifer Scheduling and Tracking System for July 1, 2009, through
December 31, 2009, the average number of days before the inmate’s suitability hearing date that the psychological
evaluation was approved was 159 days.
3 In late 2008, voters passed Proposition 9, the Victim’s Bill of Rights Act of 2008 (known as “Marsy’s Law”),
which, among other things, is intended to limit the number of parole hearings that a convicted murderer can
receive. Under Marsy’s Law, a convicted murderer can receive a parole hearing no more frequently than once
every three years and can be denied a follow-up parole hearing for as long as 15 years.
State of California • July 2010	

Page 9

Objectives, Scope, and Methodology
At the request of the Senate Rules Committee, which is a standing committee in the Legislature that
reviews proposed amendments to rules and other matters relating to the Legislature, and pursuant
to Penal Code section 6126, the Office of the Inspector General (OIG) conducted a review of the
process that the parole board uses to prepare the psychological evaluations for its parole suitability
hearings and conducted a review of the parole board’s commissioner training program. The Senate
Rules Committee expressed concern about two particular issues: (1) factual errors that may exist in
psychological evaluations and (2) certain psychologists who give inmates elevated risk assessment
conclusions compared to conclusions made in those inmates’ earlier evaluations. Furthermore, the
parole board’s executive officer formally requested that we examine its new commissioner training
program. As part of our review, we performed the following actions:
•	Reviewed relevant laws, rules, and regulations.
•	Interviewed key members of the parole board’s management team, commissioners, and deputy
commissioners.
•	Interviewed the parole board’s chief psychologist and two senior psychologists to understand the
process of preparing and reviewing psychological evaluations.
•	Interviewed attorneys who represented inmates at parole suitability hearings to discuss their
perceptions of the prevalence of factual errors in psychological evaluations.
•	Observed parole suitability hearings to understand the process.
•	Reviewed a sample of 38 central files for inmates with recent parole suitability hearings to gain
an understanding of the contents of psychological evaluations and to determine whether there
were certain types of factual errors in the evaluations. For efficiency, we selected our sample from
specific prisons in Northern California instead of from the parole board’s universe of inmates
with psychological evaluations. Although our testing revealed one substantial error wherein an
evaluation incorrectly listed the inmate’s life crime, we did not use our sample to make projections
about the total number of factual errors that may actually exist in psychological evaluations.
•	Reviewed the parole board’s electronic data concerning the number of errors reported by
attendees at parole suitability hearings in 2009.
•	Reviewed the parole board’s electronic data concerning the risk assessment conclusions of
psychological evaluations for parole suitability hearings held in 2009.
•	Reviewed and analyzed the parole board’s training information concerning the number of training
hours the current commissioners and deputy commissioners received in 2008 and 2009.
The OIG’s Bureau of Independent Review (BIR) is currently collaborating with the parole board’s
executive staff to develop a transparent training curriculum for all commissioners and deputy
commissioners. The BIR intends to identify judicial training topics relevant to the parole board’s
work and complementary to the parole board’s ongoing internal training. The training curriculum is
scheduled to be completed by December 2010.
Bureau of Audits and Investigations, Office of the Inspector General	

Page 10

As a follow-up to this report, the OIG will evaluate the risk assessments for a sample of
psychological evaluations to review the soundness of the clinical opinions and to determine
whether the evaluations contain factual errors. We are in the process of retaining an independent
licensed psychologist to perform this work and will report on the results of that review.

State of California • July 2010	

Page 11

Finding 1
The Board of Parole Hearings Does Not Have Reliable
Psychological Evaluation Data to Effectively Analyze the Number
of Reported Errors or the Types of Risk Assessment Conclusions
In January 2009, commissioners began tracking the number of factual errors—labeled
“administrative” or “substantial”—discussed during parole suitability hearings concerning
psychological evaluations. The Board of Parole Hearings (parole board) tracks factual errors
in psychological evaluations in its Lifer Scheduling and Tracking System to identify cases that
the commissioners believe warrant a secondary review by the chief psychologist. Although the
data may serve that purpose, it cannot be used for other useful management purposes, such as
determining the total number of psychological evaluations containing factual errors nor determining
the total number of factual errors, because the data in the system is not complete or accurate. For
example, we found cases in which commissioners overstated the number of factual errors when
they reported errors into the tracking system, and we found other cases in which commissioners
understated the number of factual errors by failing to report some factual errors into the tracking
system. Also in January 2009, the parole board’s chief psychologist began collecting detailed risk
assessment results for psychological evaluations using a spreadsheet. Yet this data, too, is unreliable
as a management tool because it is incomplete and inaccurate. Consequently, the chief psychologist
cannot use the data to effectively analyze patterns or trends in the types of risk assessment
conclusions given by psychologists.
When the parole board adopted its new process of preparing psychological evaluations in January
2009, it also began requiring commissioners to track the number of factual errors identified
in the evaluations during parole suitability hearings, which included classifying each error as
either “administrative” or “substantial.” According to the parole board, administrative errors are
misstatements of fact, such as occasional references in the evaluation to a wrong inmate name,
wrong ethnicity, inaccurate family history, or any similar error. Substantial errors are more serious
misstatements of fact that could affect the validity of the evaluation, such as an incorrect crime for
which the inmate was sentenced to life in prison or an incorrect number of victims.
At the conclusion of every hearing, the commissioners enter into the tracking system a variety of
data elements, such as the hearing result, details of the decision, and the numbers of administrative
or substantial errors identified in the inmate’s psychological evaluation. If a commissioner
reports that an evaluation has three or more administrative errors or just one substantial error, the
parole board’s policy requires a secondary review of the evaluation by the parole board’s chief
psychologist, a process separate from the suitability hearing. As part of the secondary review, the
chief psychologist (or his designee) determines whether the reported errors adversely affect the
validity of the evaluation and then either prepares an addendum to the evaluation or orders a new
evaluation if he or she deems the errors to be significant.

Bureau of Audits and Investigations, Office of the Inspector General	

Page 12

According to our interviews with six of the 12 current commissioners who preside over
suitability hearings, the commissioners collectively believe that the number of factual errors in
psychological evaluations is relatively low. As Table 2 shows (on the following page), four of the
six commissioners perceived the rate of factual errors in psychological evaluations as less than
5 percent; however, one commissioner perceived the rate of administrative errors to be between
10 and 15 percent. We also asked the commissioners to rate, using a scale between 1 (low) and
10 (high), how reliable they believe the psychological evaluations to be, and all but one reported
a score of at least eight. Commissioner Six rated the reliability of the evaluations a five and
explained that psychologists need to be more thorough with their inquiries. This commissioner also
commented that psychologists seldom look at the central files to ascertain if the inmate is being
truthful. Nevertheless, most of these commissioners reported they had a strong positive impression
of the evaluations’ reliability.
In contrast to the commissioners’ generally favorable remarks, we received mostly negative
comments from our discussions with six attorneys who represented inmates at their parole suitability
hearings. The attorneys’ perceptions of the frequency of factual errors in psychological evaluations
ranged from 20 percent to 100 percent. Further, when asked about their perceptions of the reliability
of psychological evaluations, the attorneys gave us responses ranging from “two” to “seven.” The
difference between the commissioners’ and the attorneys’ perceptions of the frequency of factual
errors in psychological evaluations illustrates the importance of the parole board having reliable data.
The parole board does not have reliable data to determine the total number of factual
errors in psychological evaluations
In January 2009, the parole board began tracking administrative and substantial errors contained
in the psychological evaluations discussed at parole suitability hearings. During parole suitability
hearings, commissioners use the tracking system to record up to four administrative errors and two
substantial errors for each psychological evaluation. The system also includes a hearing comment
field where commissioners may type other relevant notes from the hearing that are not otherwise
captured in a specific field. These fields serve as a mechanism for commissioners to request a
secondary review of the psychological evaluation by the chief psychologist. The parole board also
notifies the chief psychologist when commissioners record at least three administrative errors or
one substantial error in the tracking system. The chief psychologist reviews the errors in those cases
to determine what impact the errors may have on the validity of the psychological evaluations. For
the purposes of our audit, we requested a report of all the errors that commissioners recorded during
2009. The parole board produced an ad-hoc report showing that commissioners recorded between
one and four factual errors in each of the psychological evaluations of 35 inmates in 2009.
However, we have concerns about the reliability of this data because we found instances in which
commissioners either overstated or understated the number of errors in the tracking system. For
example, in one case, a commissioner reported in the tracking system that an inmate’s psychological
evaluation contained an error: the staff psychologist had not completed a Static-99 risk assessment
(used for inmates who were arrested for or convicted of a sex offense as an adult). Then in order to
trigger a secondary review of this inmate’s case, the commissioner recorded in the tracking system
that the evaluation contained two additional errors. He checked two more boxes and in those boxes’
State of California • July 2010	

Page 13

Table 2: Perception of Reliability and of the Frequency of Factual Errors in Psychological Evaluations
Observed at Suitability Hearings by a Sample of Commissioners and Attorneys Who Represented
Inmates at Parole Suitability Hearings.

Commissioner
or Attorney

Commissioner
One

Commissioner
Two

Commissioner
Three
Commissioner
Four
Commissioner
Five

Commissioner
Six

Attorney One

Attorney Two
Attorney
Three***

Attorney Four

Attorney Five

Attorney Six

Perception of
the Reliability of
Psychological
Evaluations*

Perception of the
Frequency of Factual
Errors in Psychological
Evaluations

“Seven or eight.”

Substantial errors:
“1 or 2 percent.”
Administrative errors:
“10 to 15 percent”

[The evaluations] are very useful. The newer [evaluations]
are much better, since the [parole board] implemented the
new format.

“Nine point five.”

“Less than 2 percent.”

The amount of errors has decreased recently. When I first
started there were a significant amount of errors. An example
of a common error is when the inmate’s name is wrong on
the evaluation.

“Nine or 10.”

Substantial errors:
“Less than 2 percent.”
Administrative errors:
“Less than 5 percent.”

[The evaluations] have a high degree of reliability. There is
little contradiction within the evaluations.

“1 percent.”

Because the [evaluations] incorporate findings from multiple
validated instruments, and are administered by skilled
clinicians, I regard them as likely to be highly reliable.

“0 percent.”**

The psychologists do a pretty good job. I will usually ask the
inmate at the hearing for their feedback on the evaluation
and most of them usually agree with it.

“5 to 10 percent.”

Evaluations are based on what the inmate said at the time to
the psychologist, not what is in his [central] file. The inmate
can lie and the psychologist will base their opinion on this lie.
The psychologist doesn’t typically read the [central] file.

“100 percent.”

I have never seen an evaluation that did not have factual
errors… Not all of the errors are significant, but the psychologist
often makes conclusions from incomplete information.

“50 percent.”

Fairly often, a fair percentage (maybe as high as 50%) of the
reports have some factual errors. Many will have relatively
minor errors; for example, the date of crime is slightly off, which
is not significant to determining overall risk. Many of the factual
errors are substantial errors. These include factual findings or
findings on risk assessment tools that are stated in conclusory
fashion without any explanation.

“50 percent.”

The errors I encountered varied from simple scrivener’s
errors to material errors. The implication is that no one
reviewed the reports for errors.

“20 percent.”

The doctors are not following standard procedures. One
doctor only reviewed 15 percent of the [central] file in their
report, and we (myself and inmate) identified the missing
information from the report in the [central] file.

“Up to 20 percent.”

The psychologist’s report is subjective, and sometimes they get
it right, and sometimes they do not. The psychologists purport to
make the evaluation a science, and I do not believe it is science.

“95 percent.”

Many of the errors are inconsequential to the evaluation as a
whole, but are still errors. Some of the errors I see are wrong
age and date. Even if the errors are inconsequential, they
can still lead to a misunderstanding.

“Seven or eight.”

“Eight.”

“Five.”

“Seven.”

“Two or three.”

“Five.”

“Three.”

“Five.”

“Five.”

Notable Comments from Commissioners or Attorneys

* Ratings are based on a scale of 1 to 10, with one being the lowest and 10 being the highest.
** Commissioner Five reported that he is a new commissioner and has not yet observed any factual errors in
psychological evaluations during a suitability hearing.
*** Attorney Three said that she has not represented inmates at parole suitability hearings for over a year.
Source: Telephone interviews with commissioners and attorneys

Bureau of Audits and Investigations, Office of the Inspector General	

Page 14

description fields, he wrote, “OK, OK.” When we spoke to the chief psychologist about this case, he
explained that the reported errors were not really administrative errors at all but were instead used by
the commissioner to have the chief psychologist consider another risk assessment instrument for this
inmate. The parole board’s policy specifies that if a psychological evaluation contains three or more
administrative errors or one substantial error, the chief psychologist (or his designee) will conduct a
secondary review. In this case, the commissioner overstated the number of errors in the psychological
evaluation when he reported his concern as errors in the tracking system. The commissioner could
have used the hearing comment field to report his concern. By instead reporting his concern as
additional administrative errors, he rendered the error-tracking data inaccurate.
We also found six examples of errors that commissioners incorrectly recorded in the hearing
comment field instead of in the correct administrative or substantial error fields, causing the number
of reported errors in the correct fields to be understated. All six of these inmates had their suitability
hearings postponed as a result of errors in their psychological evaluations, but the six cases did not
appear on the official error report that the parole board prepared for us because the errors weren’t
recorded in the proper fields. For example, in one of the cases, the presiding commissioner recorded
in the hearing comment field a concern that the psychological evaluation referred to a prior charge
of homicide for which the inmate was acquitted, but it was unclear to the hearing panel how, if
at all, this information was incorporated into the evaluator’s risk assessment. At the hearing, the
commissioner said that
based on the new protocol by the forensic assessment division, any time it appears that there
may be a substantial error in the [psychological evaluation] we as commissioners are afforded
the opportunity to request a [secondary] review with us citing the basis for our request, which
I have done, and essentially the question for the Panel and for the secondary review is a
determination to what degree and extent, if any, the use of a significant crime, essentially
another homicide of which [the inmate] was acquitted, had in [the evaluator’s] risk assessment.
Despite the commissioner’s statements that he believed there may be a substantial error in the
evaluation, he did not correctly record his concern as an error in the tracking system, but rather
documented it in the “hearing comments” field where it would not be counted as an error.
In addition, commissioners do not always enter errors discussed at hearings in the system at
all. Consequently, we do not know how many other examples of unreported errors actually
exist. We expected to see comments on the hearing transcript (record) from the commissioners
to ensure that they acknowledged the concerns raised by the inmate or by his or her attorney.
Without those comments, it is not clear from reviewing the transcript whether the commissioners
disagreed with the claimants and therefore did not record the errors or agreed with the claimants
and should therefore have recorded the errors in the tracking system. For example, in one case,
an inmate’s attorney said at the hearing, “[The inmate] has indicated to me that there are several
gross inaccuracies in the psychological report, to the extent to which [the inmate] believes that
the clinician actually confused [the inmate] with someone else.” The attorney further noted that
the inmate prepared a multiple-page refutation concerning the accuracy of the report and asked
for a new psychological evaluation. Although the attorney did not provide specific examples of
the errors nor the total number of errors on the record, the commissioners did not ask for any
clarifying information nor did they refute the attorney’s claims on the record. In another example,
State of California • July 2010	

Page 15

an attorney representing an inmate at his suitability hearing expressed concerns about the inmate’s
psychological evaluation and said, “One of the things that stands out as rather significant is
there’s reference to another inmate on this report.” As in the previous example, in this case the
commissioner did not refute the claims on the record. However, in neither of these cases did the
commissioners record the errors in the tracking system.
The parole board includes as part of its new commissioner training program a document that
briefly describes the process of recording administrative and substantial errors when requesting
a secondary review of an evaluation. Yet only two of the six commissioners we interviewed said
Table 3: A Sample of Commissioners’ Perceptions Regarding the Sufficiency of Training Received in
Performing Duties as a Commissioner and Tracking Factual Errors.

Commissioner

Perception of the Sufficiency
of Training Received in
Performing Duties as a
Commissioner

Commissioner
One

“Eight or nine.” I felt it was
excellent.

Commissioner
Two

“Seven.” Training has changed
since I started with the board.
There have been several
administrative changes that have
led to improvement in training.

Responses from
Commissioners on whether
they have received training on
tracking factual errors

Notable Comments from
Commissioners Regarding
Their Process for Recording
Factual Errors in the Lifer
Scheduling and Tracking
System (LSTS)

No.

[Errors] are always entered into
the comment section [of LSTS].

No ... [I] was absent the day the
[parole board] held a class on the
matter.

In LSTS … using the ‘Hearing
Comment Field.’ [I] record the
errors using the check-boxes [for
administrative and substantial
errors] only if the panel requests a
secondary review.

Yes … I received training in the
beginning of 2009 when updates
were done.

Based on my judgment … I only
enter errors in LSTS if the hearing
is postponed and the panel
requests a secondary review of the
evaluation … If an evaluation has
one or two administrative errors, I
will not enter the errors in LSTS.

Commissioner
Four

“Nine.” I was one of two people
who went through the class for the
first time. It was very informative.
The class was very effective,
dynamic, and challenging.

Don’t recall having any.

I do not know what is meant by
administrative [versus] substantial
… as a commissioner, I do
not track errors. However, if a
hearing were to be continued
or postponed for any reason,
including an issue with the
[psychological evaluation], I would
document this in LSTS, and have
been trained to do so.

Commissioner
Five

“Eight plus.” I got one-on-one
training. Training was very
expansive … The whole process
made me feel comfortable and
prepared to perform my duties at
suitability hearings.

Yes. When I received my initial
training.

[Errors are] put on record and put
in the comments section of LSTS.

Commissioner
Six

“Three or four.” The initial
training [I] received was poor
...Training has improved, but due
to the experience level of each
commissioner, it is difficult to
establish [training] that will suit
everyone.

No … we don’t ‘track’ errors.

[Errors are] put on record and
are articulated in the comments
section of LSTS.

Commissioner
Three

“Seven to eight.” When I first
came on I needed more training
… I feel adequately prepared to
conduct hearings.

Source: Interviews with commissioners.

Bureau of Audits and Investigations, Office of the Inspector General	

Page 16

that they received training specifically related to recording errors identified in psychological
evaluations. We learned from interviews with the commissioners that the process of recording
errors is inconsistent at best. For example, as shown in Table 3 (on page 16), Commissioner
One told us that errors are always entered, whereas Commissioner Three indicated that he
uses his judgment when deciding to record errors. In the latter example, the commissioner
explained that he only enters errors when the panel decides to postpone the hearing and requests
a secondary review of the evaluation. Additionally, the commissioner told us that he does not
always record errors if there are fewer than three and the panel does not request a secondary
review of the evaluation. Commissioner Four, however, did not understand what we meant by
“administrative” and “substantial” errors, commenting that the term “administrative errors”
could mean many things. The commissioner did not recall having received specific training
about the process of recording errors. Had he received such training, he would have understood
the parole board’s terminology.
The parole board should ensure that commissioners report in the tracking system all of the
factual errors in psychological evaluations unless the commissioners are able to refute the errors
on the record. Once the parole board has complete and accurate data regarding factual errors,
it should periodically review the same type of summary report it prepared for us. This periodic
review would help it gauge the effectiveness of its quality control processes, identify training
needs for staff, and make procedural changes as necessary.
The parole board does not have reliable data to determine the number and types of risk
assessment conclusions for psychological evaluations in 2009
As we discuss in the Scope and Methodology section of this report, the Senate Rules Committee
was concerned that certain psychologists give only elevated risk assessment conclusions when
compared to previous conclusions. We reviewed a sample of 38 inmates who recently had
a parole suitability hearing to gain an understanding of the contents of their psychological
evaluations and to look for the existence of certain types of factual errors. However, as we
describe in the introduction of this report, the parole board standardized the format and use
of risk assessment tools for psychological evaluations beginning in January 1, 2009. Thus,
psychological evaluations prepared before that date do not always follow the same approaches
as later evaluations and therefore do not lend themselves to simple comparisons with the
newer evaluations. Accordingly, we asked the parole board’s chief psychologist whether the
parole board collects risk assessment data that could be used for analytical purposes, such as
identifying patterns or trends in the types of conclusions provided by psychologists.
In response to our query, the chief psychologist said he started capturing detailed risk
assessment conclusions for psychological evaluations using a spreadsheet beginning in January
2009. After a staff psychologist completes a draft evaluation, the psychologist enters the risk
assessment conclusions into his or her individual spreadsheet and periodically reports the
information to his or her senior psychologist, who in turn, sends the information to the chief
psychologist. The chief psychologist compiles the information into a single spreadsheet.
We obtained the chief psychologist’s data for evaluations prepared between January 1, 2009
and November 30, 2009, consisting of 3,077 evaluations prepared by 44 psychologists. The
State of California • July 2010	

Page 17

psychologists presented their conclusions using a range of labels, as
we show in the text box at right.

Psychologists present
risk assessment
conclusions using the
following terminology:

In testing this data for completeness, we compared the data from the
physical copies of a sample of 40 evaluations to the corresponding
•	 Low
data field in the spreadsheet. We could not locate three of the 40
•	 Low to Moderate
evaluations (the missing evaluations were conducted by three different
•	 Moderate
psychologists) in the spreadsheet. In addition, we found one instance
•	 Moderate to High
in which the overall assessment in the physical copy of the evaluation
was labeled as “moderate to high,” but in the spreadsheet, the overall
•	 High
assessment was incorrectly recorded as “moderate.” Consequently,
we found that the data in the spreadsheet is not complete or always
accurate. Without reliable data, the parole board cannot use the
information to determine the total number of low-, medium-, or high-risk conclusions or fully
analyze the types of risk assessment conclusions provided by its psychologists, and it cannot fully
analyze patterns of conclusions associated with particular psychologists or certain trends over time.
Despite the data’s limitations, our review revealed that psychologists provided a wide range of
conclusions and that no particular psychologist gave only “moderate” or “high” risk conclusions.
For example, the psychologists gave an average of 32 “low” or “low to moderate” risk conclusions
as their overall assessment. In contrast, the psychologists gave an average of 14 “high” or
“moderate to high” risk conclusions as their overall assessment. The average number of “moderate”
risk conclusions per psychologist was 24.
After we discussed this issue with the chief psychologist, he prepared a written analysis of the
psychologists who gave the greatest number of “high” conclusions and discussed the reasons, in
his opinion, that their conclusions were appropriate to their workload. Specifically, he pointed out
that although these psychologists gave the greatest number of “high” risk conclusions, they also
gave “low” risk conclusions on other evaluations, suggesting that these psychologists are capable
of identifying low risk when it exists. Furthermore, one particular psychologist tended to evaluate
a large number of sex offenders. The chief psychologist noted that “sex offenders do not receive
treatment for salient dynamic risk factors in prison and so these problems do not get addressed,
which elevates their risk.” He also noted that another of these psychologists evaluated a large number
of inmates with “severe and persistent mental disorders which impacts their level of dangerousness.”

Recommendations
To ensure that the Board of Parole Hearings has and uses reliable data, it should take the following
actions:
•	Ensure that all of its commissioners who preside over parole suitability hearings receive formal
training on how and when to report factual errors in the Lifer Scheduling and Tracking System
(tracking system). This action should include ensuring that commissioners clearly articulate on
the record the disposition of claimed errors in psychological evaluations and record the result in
the tracking system accurately.
Bureau of Audits and Investigations, Office of the Inspector General	

Page 18

•	Periodically review a report on the number and types of factual errors in psychological
evaluations recorded in the tracking system. This review would identify training opportunities for
certain psychologists who repeatedly make the same types of errors and would also determine
whether changes in the tracking system process are necessary.
•	Ensure that psychologists report to the chief psychologist all of their risk assessment conclusion
information so that the chief psychologist can analyze the data for patterns of conclusions and can
identify trends over time.

State of California • July 2010	

Page 19

Finding 2
The Board of Parole Hearings Can Strengthen Its Process of
Reviewing Psychological Evaluations
The Board of Parole Hearings (parole board) employs four senior psychologists who review
psychological evaluations. These senior psychologists, who report to the parole board’s chief
psychologist, each supervise between nine and 12 staff psychologists who prepare psychological
evaluations. We found that the parole board can improve its oversight of the senior psychologists’
methods for reviewing psychological evaluations. Specifically, we found that the parole board
could require senior psychologists to use source documentation when conducting their supervisory
reviews. Not using source documentation limits the senior psychologists’ ability to validate
statements and conclusions presented by staff psychologists in their evaluations and to detect
factual errors in their subordinates’ work. In addition, the parole board could actively monitor its
senior psychologists’ activities by requiring them to account for their time by case or activity. This
would aid the parole board in holding its senior psychologists accountable if reviews are not done
properly. However, without this information, the parole board lacks a necessary management tool to
measure and monitor its employees’ performance.
The preparation and review of psychological evaluations is unquestionably challenging, given
the complex and subjective nature of assessing an inmate’s potential risk for future violence.
We recognize that even with strong controls in place, the process of preparing and reviewing
psychological evaluations will be subject to some margin of error. Further, we understand that
psychological evaluations are one type of relevant evidence that commissioners consider when
making parole suitability decisions. However, because the decision to grant or deny parole is
critical, it is reasonable to expect that decision be made using the most accurate information
available. Therefore, the changes we recommend are intended to strengthen the parole board’s
review process and ultimately lead to fewer errors in evaluations.
The parole board does not require its senior psychologists to review source
documentation when conducting reviews of psychological evaluations
The parole board provides staff psychologists with templates to use when drafting their evaluations.
These templates provide the psychologists with a standardized report format and include
instructions, examples, and guidance concerning each topic area. When conducting reviews of draft
evaluations, the senior psychologists may refer to the templates. In addition, as part of orientation
training, the parole board provides senior psychologists with a list of elements that they must look
for when reviewing psychological evaluations. As shown in the text box on the following page,
this information provides senior psychologists with a review framework and identifies the parole
board’s expectations for a well-prepared psychological evaluation.
Based on the review guidance provided by the parole board, we anticipated finding that the
senior psychologists’ reviews included key source documents to ensure that staff psychologists’
conclusions are supported by evidence and are factually accurate. By agreement between the
Bureau of Audits and Investigations, Office of the Inspector General	

Page 20

prisons and the parole board, employees at
the prison make copies of key records from
the inmate’s central file for the psychologists’
use. These records include Probation Officer’s
Reports providing details of the inmate’s
commitment offense (life crime), social
factors, and criminal history, as well as the
probation officer’s recommendation to the
court. Other examples of records provided
to the psychologists include chronological
lists of rules violation reports and counseling
notes prepared for the previous hearing; the
last board report that includes the inmate’s
case factors; and the Legal Status Summary,
which provides information regarding the
commitment offense and length of prison term.
Surprisingly, although psychologists use these
records in preparing their draft evaluations, the
senior psychologists do not use them in their
reviews of the psychologists’ evaluations.

The parole board provided senior psychologists
with the following review guidelines:
1. The report was comprehensive and provided
adequate summaries of the clinical interview, file
records, and inmate’s self-report.
2. Data Critique: The report analyzed the
available data, pointed out inconsistencies or
discrepancies in the records, changes in the
inmate’s accounts and contradictions in the
records, and critiqued the conclusions and
opinions contained in other psychological
evaluations, particularly if the previous
conclusions were not supported by the data.
3. The opinions and conclusions pertaining to
violence risk and diagnosis were logical, flowed
directly from the data, were empirically valid,
and well supported.
4. Based on the information provided in the report,
the risk assessment instruments appeared to be
scored accurately and the interpretations were
appropriate.

5. Clinical judgments pertaining to either the overall
The parole board’s review guidelines (see
increased or decreased level of risk were
text box at right) direct senior psychologists
well-supported, especially if the actuarial data
to ensure that the evaluations adequately
suggested a different risk category (i.e. clinical
summarize file records and to point out
over-ride).
data inconsistencies or discrepancies in the
6. The case conceptualization was objective,
records, changes in the inmate’s testimony,
non-biased and neutral, and was written with
and contradictions in the records. From
the appropriate audience in mind (i.e. minimal
our discussions with two of the four senior
clinical jargon).
psychologists, however, we learned that when
7. Cultural issues were addressed and integrated
the senior psychologists conduct reviews of
throughout the report when appropriate.
their subordinates’ evaluations, they do not
review the source documents that the staff
psychologists use. One senior psychologist
indicated that he corroborates limited information about the inmate in the parole board’s Lifer
Scheduling and Tracking System. Nevertheless, we question how a senior psychologist can
effectively identify factual mistakes and omissions of information in an evaluation if the senior
psychologist does not refer to the source documents from the inmate’s central file. For example,
if the inmate’s records contain a discrepancy, such as conflicting information about the inmate’s
life crime, and that discrepancy is not discussed in the psychologist’s evaluation, the senior
psychologist would have no independent means of noticing that discrepancy. If copies of the
underlying documents were available, the senior psychologists could, for example, review the
Probation Officer’s Report to ensure that the evaluation accurately described the inmate’s life crime
and the number of victims, or consult rules violation reports and custodial counselors’ notes to
ensure that the psychologist properly accounted for the inmate’s behavior while in prison.
State of California • July 2010	

Page 21

Furthermore, the parole board could also strengthen its process by developing a checklist of
specific procedures for senior psychologists’ use in verifying statements and conclusions in the
evaluations. An example of a relatively simple procedure would be ensuring that the inmate’s name
is spelled correctly and is used consistently throughout the evaluation and confirming that no other
inmate’s name is inappropriately included in the evaluation. The senior psychologists could check
off the task when complete; the checklist itself could serve as evidence of the review if particular
components of the evaluation were later challenged.
Certain types of factual errors could be prevented if the senior psychologist reviewed source
documentation as part of the review process. For example, we reviewed a parole board tracking
report listing factual errors reported by commissioners during suitability hearings in 2009. In one
case listed on the report, the commissioner recorded that an inmate had four administrative errors
in his psychological evaluation during his suitability hearing. Two of the four errors may have
been preventable if the senior psychologist compared the statements in the evaluation to source
documentation. One of the preventable errors involved the use of an incorrect inmate name in the
psychological evaluation, and the other was a reference in the evaluation to a crime that was not
otherwise on the inmate’s record.
Furthermore, our review of 38 evaluations spread across four prisons discovered a substantial error
in an inmate’s evaluation in which the psychologist incorrectly labeled on the first page of the
evaluation the inmate’s life crime as second-degree murder when it should have been first-degree
murder. We found this error by comparing the life crime in the evaluation to the inmate’s Abstract
of Judgment (located in the inmate’s central file). It is logical to assume that a senior psychologist
reviewing this evaluation would be able to replicate our discovery of this error if he or she were
independently comparing the evaluation to the underlying source documents used by the staff
psychologist who prepared the evaluation.
The parole board does not have a time benchmark for reviewing psychological
evaluations nor does it require senior psychologists to account for their review time
The chief psychologist does not specify a minimum length of time that a senior psychologist
should expect to spend reviewing an evaluation nor does the chief psychologist require senior
psychologists to account for their time by case or activity. Consequently, the chief psychologist
lacks the information needed for determining whether his staff spends an adequate amount of
time on their reviews. The chief psychologist primarily oversees the senior psychologists’ work
by reviewing their monthly reports of research data and their weekly evaluation scheduling and
tracking reports; holding monthly in-person status meetings and weekly conference calls; and
conducting additional discussions as needed.
We learned from our conversations with the chief psychologist and two of the four senior
psychologists that, using the parole board’s framework and guidance, the senior psychologists
each have the flexibility to determine the approach they will use in conducting their review and
the time they spend on it. The two senior psychologists estimated that they review between 65
and 80 evaluations per month, and although one of them estimated that it takes an average of one
hour to review a full draft evaluation, the other estimated that it takes between one and three hours.
Bureau of Audits and Investigations, Office of the Inspector General	

Page 22

However, absent a benchmark or standard against which to compare the actual time spent reviewing
each evaluation, the chief psychologist cannot judge whether the senior psychologists are spending
too little or too much time in this process. Comparing the actual time spent reviewing evaluations
with a benchmark can provide the chief psychologist with more assurance that the evaluations are
thoroughly reviewed. Furthermore, given that senior psychologists are not centrally located in an
office and instead work most of their time at home, it is important that the parole board have more
accountability built into its process for reviewing psychological evaluations. Having to verify a list
of specific information and having specified review-time expectations could make the process more
consistent and more thorough for all four reviewers.
In addition to reviewing evaluations, senior psychologists also perform a number of other activities,
such as visiting prisons to observe psychologists’ face-to-face inmate interviews and to meet
with prison staff, conducting performance appraisals and other supervisory tasks, participating in
conference calls with management and staff, participating in training activities, and performing
secondary reviews of evaluations when directed by the chief psychologist. This breadth and variety
of supervisory responsibilities further heightens the need for a time-based monitoring system.
When we discussed these issues with the parole board’s chief psychologist, he said that
the [parole board’s] existing practice of supervisory review of psychological evaluations
conducted by licensed psychologists far exceeds the standard of practice in the field.  
In fact, most licensed psychologists, including those employed by the State, receive
little or no clinical supervision of evaluation reports—let alone have each report
subjected to supervisory review.
In addition, the chief psychologist said that he “believes that [the parole board’s] psychologists
receive more individualized supervisory input and oversight of their written work than most
psychologists in the state.” The chief psychologist also noted that
the individualized nature of psychological evaluations requires that senior psychologists
adopt a flexible approach when reviewing evaluations—even when doing so within a fairly
structured format.  Senior psychologists do not arbitrarily determine the length of their
review time. Instead, their review time is determined by the complexity of the case and the
supervisory needs of the psychologist. 
He added the following:
It is my position that it is the primary responsibility of the psychologist, and not the senior
psychologist, to assure that evaluation findings are based upon accurate reporting of facts.
This responsibility is codified in State licensing requirements, professional ethics codes, and
specialty guidelines of forensic psychologists.
We agree with the chief psychologist’s position that psychologists are responsible for the quality of
their work. However, we also believe that as supervisors, senior psychologists are responsible for
holding their staff accountable for their work and ensuring that factual errors such as those discussed
previously are identified and corrected as soon as possible before the inmate’s suitability hearing.

State of California • July 2010	

Page 23

Recommendations
To strengthen the process for reviewing psychological evaluations, the parole board should take the
following actions:
•	Ensure that senior psychologists review copies of source documents from the inmate’s central
file when reviewing psychological evaluations for parole suitability hearings. This source
documentation review will increase the integrity of the review process by helping to identify
misstatements or inaccuracies contained within the evaluations.
•	Develop a review checklist for senior psychologists to follow when conducting reviews
of psychological evaluations. This checklist would aid in fact-checking the statements and
conclusions in the psychological evaluations and ensure consistency and thoroughness among
reviewers.
•	Develop a time benchmark for reviewing psychological evaluations to ensure that reviewers
spend at least the minimum amount of time conducting their reviews.
•	Require senior psychologists to report to the chief psychologist their time spent reviewing draft
evaluations. This information would help the chief psychologist track productivity and manage
workload levels.

Bureau of Audits and Investigations, Office of the Inspector General	

Page 24

Finding 3
The Board of Parole Hearings Failed to Provide All Required
Training to Its Commissioners, Deputy Commissioners, and
Senior Psychologists
Penal Code section 5075.6 requires commissioners and deputy commissioners who conduct parole
suitability hearings to receive 40 hours of annual training. Government Code section 19995.4
requires senior psychologists (because they are supervisors) to receive 80 hours of supervisory
training within 12 months of their appointment. However, the Board of Parole Hearings (parole
board) failed to provide most of these employees with the required training. Commissioners
and deputy commissioners who are not well trained may not be fully prepared to perform their
duties and responsibilities, which could affect hearing decisions. If commissioners and deputy
commissioners make mistakes because they were not trained in certain areas, those mistakes could
put the state at risk of legal challenges and erode stakeholders’ confidence in the parole suitability
hearing process. Moreover, the parole board did not provide all four of its senior psychologists with
the required supervisory training. As a result, the senior psychologists may not be aware of certain
techniques for supervising and teaching employees and enforcing labor policies.
Commissioners and deputy commissioners did not always receive the required number of
annual training hours
Penal Code section 5075.6 requires commissioners and deputy commissioners who conduct
hearings for the purpose of considering the parole suitability of inmates to receive—within 60 days
of their appointment and annually thereafter—a minimum of 40 hours of training. The law specifies
this training to include such topics as the treatment and training programs provided to inmates,
parole services, commissioner duties and responsibilities, and the laws and regulations applicable
to conducting parole hearings. The parole board noted that it established a new commissioner
training program in July 2008. The duration of the program is five weeks, with the first three weeks
consisting of classroom instructional training. The parole board maintains a variety of instructional
materials, covering such topics as basic introduction to the parole board, ethics, applicable case
law, prison resources available to commissioners such as inmate central file and psychological
evaluations, familiarization with the Lifer Scheduling and Tracking System, mock hearings, and
decision writing. The parole board also provides commissioners with administrative training on
such topics as completing time sheets and defensive driver training. The final two weeks of the
training program allows commissioners to observe parole suitability hearings to better understand
the process.
We found that the parole board provided all seven of its current commissioners appointed during
2008 and 2009 with 40 hours of training within 60 days of their appointment. However, the parole
board failed to provide any of its current commissioners who were appointed before 2009 with 40
hours of annual training thereafter as required by law. Specifically, according to the parole board’s
In-Service Training reports (IST reports), the parole board failed to provide all three commissioners
appointed before 2008 with at least 40 annual training hours in 2008 and all seven commissioners
State of California • July 2010	

Page 25

appointed before 2009 with at least 40 annual training hours in 2009. In 2008 and 2009, the parole
board provided these commissioners with an average of 19 and 10 hours of training each year,
respectively. In addition, we reviewed training records for deputy commissioners who were panel
members for suitability hearings in 2009. According to the parole board’s IST reports, it provided
only three of 16 deputy commissioners (19 percent) whom it hired in 2009 with 40 hours of
training within 60 days of their appointment. The average number of training hours received by the
remaining 13 deputy commissioners was only 28. Furthermore, the IST reports show that the parole
board provided only two of 70 deputy commissioners (3 percent) whom it hired prior to 2009 with
40 hours of annual training. The average number of training hours received by the remaining 68
deputy commissioners was only 15.
If not trained sufficiently, commissioners and deputy commissioners—even those who have served
in the position for a long period of time—may not be as well-equipped or prepared as they could be
to make the best parole suitability decisions at hearings. Commissioners and deputy commissioners
who do not receive training may not fully understand certain aspects of their duties and
responsibilities, which could affect hearing decisions. If commissioners and deputy commissioners
make mistakes or issue poorly articulated decisions because they were not trained in certain areas,
those actions could put the state at risk of legal challenges and erode stakeholders’ confidence in the
parole suitability hearing process.
When we brought this matter to the attention of the parole board’s chief deputy of program
operations, he said that the parole board recognized that training deficiencies existed in 2009 and
had drafted a new training plan. He explained that the new plan addresses training requirements and
describes procedures to bring the parole board staff, including commissioners, into compliance with
statutes and regulations governing training. He also noted that the parole board has a new training
manager who will report quarterly to executive management if there are deficiencies in training.
As stated earlier, the Office of the Inspector General’s Bureau of Independent Review (BIR) is
currently collaborating with the parole board’s executive staff to develop a transparent training
curriculum for all commissioners and deputy commissioners. The BIR intends to identify judicial
training topics relevant to the parole board’s work and complementary to the parole board’s ongoing
internal training. The training curriculum is scheduled to be completed by December 2010.
Senior psychologists did not receive the required number of supervisory training hours
within one year of their appointment
Government Code 19995.4 requires supervisory employees to receive a minimum of 80 hours of
training—40 hours performed by a qualified instructor—within 12 months of their appointment.
The law requires the supervisory training to address such topics as the role of the supervisor,
techniques of supervision, planning, organizing, staffing and controlling, performance standards,
performance appraisal, affirmative action, discipline, labor relations, employment law relating to
persons with disabilities, and grievances. The additional 40 hours may be provided as on-the-job
training by a qualified higher level supervisor or manager.
The Parole Board’s four senior psychologists supervise staff psychologists. However, the parole
Bureau of Audits and Investigations, Office of the Inspector General	

Page 26

board did not provide the 80 hours of supervisor training to any of those senior psychologists within
12 months of their appointment, as required. One senior psychologist, whom the parole board hired
in October 2006, reported that he received supervisory training in February 2009, about 15 months
after the 12-month period expired. However, these training hours were not listed in the senior
psychologist’s IST roster. The chief deputy of program operations acknowledged that the parole
board did not provide the other three supervisors with the required training. He explained that this
is partly because the parole board focused senior psychologists on completing their high-volume
workload. Nevertheless, without proper supervisory training, there is greater risk that staff could
be unaware of employment-related matters such as performance standards and processes related to
discipline and grievances.

Recommendation
To ensure that its employees receive the required number of training hours, the parole board should
regularly monitor training activities and the number of training hours it provides to all staff. The
parole board should then make the necessary scheduling adjustments for those individuals who
have not received enough hours.

State of California • July 2010	

Page 27

California Department of Corrections and
Rehabilitation’s response to the special report

Bureau of Audits and Investigations, Office of the Inspector General	

(page 1 of 1)

Page 28

SPECIAL
REPORT
SPECIAL REPORT
INMATE CELL PHONE USE ENDANGERS PRISON
SECURITY AND PUBLIC SAFETY

The Board of Parole hearings: psychological evaluations and
mandatory Training Requirements
OFFICE OF THE
INSPECTOR GENERAL

OFFICE OF THE INSPECTOR GENERAL
DAVID R. SHAW

INSPECTOR GENERAL

David R. Shaw

STATE OF CALIFORNIA
MAY 2009

INSPECTOR GENERAL



Jerry Twomey
Chief assistant inspector general

William Shepherd

Deputy Inspector General, In-Charge

Bryan Beyer
Deputy Inspector General, Senior

Christina Animo
Deputy Inspector General

Ryan Baer
Deputy Inspector General

STATE OF CALIFORNIA
July 2010
WWW.OIG.CA.GOV