CALIFORNIA STATE AUDITOR
ELAINE M. HOWLE

DOUG CORDINER

STATE AUDITOR

CHIEF DEPUTY STATE AUDITOR

March 27, 2007	

2007-503

The Governor of California
President pro Tempore of the Senate
Speaker of the Assembly
State Capital
Sacramento, California 95814
Dear Governor and Legislative Leaders:

T

his letter report presents the results of a follow-up review the
Bureau of State Audits (bureau) conducted concerning the
California Department of Corrections and Rehabilitation’s
(department) efforts to implement selected recommendations from
a report the bureau issued in September 2005 titled Department of
Corrections: It Needs to Better Ensure Against Conflicts of Interest
and to Improve its Inmate Population Projections (2005-105). During
the follow‑up review we focused on three key recommendations
regarding the department’s inmate population projections, and found
that the department has made minimal progress in implementing
those recommendations. Specifically, we determined that the
department has done little to seek advice from statistical experts to
assist it in establishing a statistically valid forecasting methodology.
Furthermore, we determined that although the department asserts
it has revised its population projection model, it could not provide
us with documentation to support the population projections it
published in the fall of 2006. Finally, we noted that the department
has not begun to update its variable projections using a revised inmate
classification database. Because the department has done little to
address our recommendations and those of our statistical consultant,
we believe the usefulness of the department’s current inmate
population projections remains questionable.

BUREAU OF STATE AUDITS

555 Capitol Mall, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 www.bsa.ca.gov

BACKGROUND
In March 2005 the Joint Legislative Audit Committee requested
that the bureau evaluate the process the department used to
negotiate and enter into two no-bid contracts for private prison
facilities and to analyze the reasonableness and consistency
of the department’s method of tracking and projecting
inmate population. In September 2005 we issued our report
and concluded that although the department’s policies and
procedures for processing the two no-bid contracts and
identifying potential conflicts of interest were consistent with
state requirements, the department did not ensure that these
contracts were free of conflicts. In addition, we concluded that
although the department’s inmate population projections are
reasonably accurate for the first two years, they are significantly
less accurate after the second year. As a result, the projections
are useful for assessing the next two years’ budget needs but
have limited usefulness for longer-range planning, such as
determining when additional facilities should be built.
During our 2005 audit, to determine whether the department’s
method of tracking and projecting inmate populations was
reasonable and consistent, we reviewed forecasts for the
previous three fiscal years to assess whether the assumptions and
variables included were reasonable and consistent, and further
analyzed those that have a material effect on the population
projection. We also consulted with a statistical expert to obtain
an analysis of the department’s use of its microsimulation
model. In addition, we compared the actual inmate populations
to the levels forecasted by the department over the past 10
years to determine their accuracy, and analyzed the sufficiency
of the department’s method of projecting the number of needed
inmate beds for facility planning purposes.
We concluded that although the inmate population projection
of the department has an average error rate of less than 5 percent
for the first two years of its six-year projection, by the last year
the average error rate climbs to almost 30 percent. In addition,
we found that the department did not update the inputs to its
projection model with historical data using a statistical process
but rather adjusted the variables by relying on staff experience.
Also, because it did not update information related to prisoners’
security classifications using the most recent data, we concluded
that the department was projecting security needs based on an
obsolete classification system that may differ significantly from
the current inmate population. Furthermore, our statistical

	

California State Auditor Letter Report 2007-503

expert advised us the department’s process for making inmate
population projections was not based on a statistically valid
method for creating a forecast.
Pursuant to the authority granted to the bureau, including
the audit standards the bureau operates under, it has been a
long-standing administrative practice to require each agency
or department we have audited to report to the bureau on
its progress on implementing our recommendations at three
intervals—60 days, six months, and one year (California
Government Code, Title 2, Section 8543, and Government
Auditing Standards, paragraph 1.27). Under that same authority,
it has also been a long-standing administrative practice of the
bureau to conduct follow-up reviews of audits when resources
are available and the bureau determines it is prudent to do so.
After receiving and reviewing the department’s status reports,
the bureau determined that it was appropriate to conduct a
follow-up review of certain key recommendations. Specifically,
we reviewed the department’s progress in implementing the
following three recommendations:
If the department intends to continue using the projections for
long-term decision making, such as facility planning, it should
ensure that it employs statistically valid forecasting methods. It
should consider seeking the advice of experts in selecting and
establishing forecasting methods that will suit its needs.
To increase the accuracy and reliability of its inmate projections,
at a minimum the department should do the following:
•	 Fully document its projection methodology and model.
•	 Update its variable projections with actual information, such
as new security level data, whenever feasible to do so.

THE DEPARTMENT’S SIX-MONTH RESPONSE 	
INDICATES MINIMAL PROGRESS IN IMPLEMENTING
KEY RECOMMENDATIONS
To address our recommendation regarding the use of statistical
experts to establish a valid method of forecasting inmate
populations, the department stated in its six-month response
that it was working with its Office of Research to establish an
interagency agreement with statistical experts at either the
California State University (CSU) or University of California
(UC) systems to review the existing simulation model and

California State Auditor Letter Report 2007-503	



projections process. The department provided us with e-mail
correspondence that occurred in September 2005 between
department staff indicating that the department should seek
those who have demonstrable practical application of statistical
methods as they relate to projections, preferably as they relate
to prison systems. Department staff stated their belief that a
search for a national expert on prison population projections
was appropriate. One of the department’s staff believed that
CSU Sacramento did not offer classes in applied mathematics
and concluded the department should not approach
CSU Sacramento. However, the department did not provide
us with evidence to demonstrate that it made any attempts to
contact CSU Sacramento’s Department of Mathematics and
Statistics directly to find out whether or not it could actually
assist the department with its projection model. Furthermore,
the department did not have evidence to demonstrate that it
contacted any other campuses within the CSU system.
Additionally, in September 2005, the assistant secretary of
the department’s Office of Research contacted a professor
at UC Davis via e-mail asking for a recommendation of
someone either at UC Davis or within the UC system that
might be able to assist the department in refining its prison
population forecasting model. The professor responded to the
assistant secretary indicating that he did not think anyone at
UC Davis would be willing to work with the department based
upon earlier experiences certain UC Davis staff had with the
department. Specifically, the professor stated that UC Davis had
previously contracted with the department to forecast prison
populations, and the department did not like the projections
furnished by UC Davis because they were too low. The professor
further stated that the department did not want unbiased expert
advice. Beyond this e-mail, the department did not provide
us with evidence showing that it made any other attempts to
contact UC Davis or other UC campuses to find out whether
or not anyone would be willing to assist the department in
evaluating its projection model.
We question the department’s efforts to seek advice from
statistical experts in the CSU or UC systems. As stated earlier, we
consulted with a statistical expert from CSU Sacramento and in
our September 2005 audit report to address the validity of the
department’s projection model, she stated the following:
The department needs to bring a group of
statisticians together to get advice on establishing
a statistically valid forecasting methodology with
	

California State Auditor Letter Report 2007-503

the current data system. A time series specialist, a
non-parametric statistician, and a finite population
sampling theorist along with a statistical computing
expert will be able to design a suitable forecasting
tool for the department’s needs. There are national
experts available in all these areas of statistics within
California’s higher education system. Both the CSU
and UC campuses have researchers specializing
in these areas right here in Northern California.
One of the benefits for the department is that
this group of statisticians will be able to provide
the state-of‑the‑art statistical tools needed for the
department’s short-term and long-term forecasts.
Regarding our recommendation to document its projection
methodology and model, the department asserted that its
documentation of the projection methodology and model
was 50 percent complete and estimated that it would be fully
documented by October 1, 2006. In addition, to address our
recommendation that the department update its variable
projections with actual information, the department reported
that its action plan was to establish an Inmate Classification
Scoring System database, create programs to be used in its
simulation model, and revise the simulation model to include
classification data; however, it had not begun to implement its
action plan and estimated that it would not be complete until
October 1, 2007.

THE DEPARTMENT’S ONE-YEAR RESPONSE SHOWS
A CHANGE IN APPROACH TO implement OUR
RECOMMENDATIONS
In its one-year response to the audit recommendations, the
department changed its proposed action to seek advice to
ensure that it employs statistically valid forecasting methods.
Rather than establishing an interagency agreement with
either the CSU or UC systems as it had asserted it would do in
its six‑month response to our report, the department stated
in its one-year response dated October 23, 2006, that it was
working with its contracts staff to establish a public entity
agreement with Ohio State University (Ohio State) in its search
for a national expert to assist in selecting and establishing
forecasting methods that will suit the department’s needs.
However, as of March 9, 2007, the contract had not been signed
because, according to a department senior staff counsel, the
department and Ohio State have not reached an agreement on

California State Auditor Letter Report 2007-503	



two of the contract’s clauses pertaining to indemnification and
adjudication. The section chief of the Estimates and Statistical
Analysis Section of the Offender Information Services Branch
(section chief) stated that, if the contract is not executed with
Ohio State, the department intends to pursue other methods
to find an expert, such as using a sole-source contract or
competitive bidding process.
Regarding our recommendations to document its projection
methodology and model and to update its variable projections
with actual information, the department again stated that it
was only 50 percent complete in its efforts to document the
methodology and the model and it had not started its process
to establish the Inmate Classification Scoring System database.
Furthermore, as part of the action plan, the department stated
that it intended to hire a retired annuitant to begin working on
the database in the spring of 2007 with an expected completion
date of October 1, 2007.

LACK OF DOCUMENTATION OF RECENT 	
PROJECTIONS CASTS DOUBT ON THE VALIDITY 	
OF THE PROJECTION PROCESS
During our follow-up review, the section chief stated that
beginning in fall 2006, the department reported the Adult
Population Projections for 2007 through 2012 using a projection
model that had been revised since the one we evaluated
for our September 2005 audit report. On March 5, 2007, we
asked to review the supporting documentation for the revised
projections model and were told by the section chief that the
documentation did not completely match the population
projections included in its fall 2006 report. According to the
section chief, this is because three of the four components
comprising the current documentation for the model had
already been updated for the spring 2007 projections, which the
department intends to release by April 6, 2007. Therefore, the
section chief stated that the existing documentation no longer
exactly matched that of the model that produced the fall 2006
projections. Without having the supporting documentation to
review, we cannot determine how the projection model used
for the fall 2006 projections differs from the previous version
we reviewed. Consequently, we are still concerned that the
department publishes its projections without having adequate
supporting documentation. Moreover, similar to the concerns
we expressed in the September 2005 audit, as of March 21, 2007,
the lack of documentation to support the projections casts doubt
	

California State Auditor Letter Report 2007-503

on the validity of the revised projection model that was used for
the fall 2006 projections. This is important because, according to
the assistant secretary of the department’s Office of Research, the
fall 2006 revised statistical model was used to generate the latest
population counts and projections that are among the factors
considered in making capital outlay decisions. The assistant
secretary also indicated that these capital outlay decisions
include other factors, such as new compliance requirements, life
and safety concerns, and maintenance of capacity. Finally, as
stated earlier, we are still concerned about the statistical validity
of the population projections since the department has yet to
seek advice from statistical experts.
The section chief indicated that he would provide us the existing
supporting documentation for the model used for the spring
2007 projections on March 23, 2007. We received the supporting
documentation, and we plan to have a statistical expert
review the sufficiency of the department’s projection model.
However, the section chief also stated that the documentation
we receive on March 23, 2007, would not represent the complete
model the department is developing because, as stated earlier,
it plans to establish a security classification database with
the assistance of a retired annuitant who will begin work
by April 30, 2007. According to the section chief, the retired
annuitant will assist in revising a portion of the model, a
classification database that is used for projecting the number
of inmates by security level. The section chief also stated that
the revised classification database would contain a new method
for scoring an inmate’s security level. According to its one‑year
response, the department estimates that the classification
database will be completed by October 1, 2007.
We conducted this review under the authority vested in the California State Auditor
by Section 8543 et seq. of the California Government Code and according to generally
accepted government auditing standards. We limited our review to those areas specified in
the letter report.
Respectfully submitted,

ELAINE M. HOWLE
State Auditor

California State Auditor Letter Report 2007-503	



cc:	 Members of the Legislature
	
Office of the Lieutenant Governor
	
Milton Marks Commission on California State
		 Government Organization and Economy
	
Department of Finance
	
Attorney General
	
State Controller
	
State Treasurer
	
Legislative Analyst
	
Senate Office of Research
	
California Research Bureau
	
Capitol Press

	

California State Auditor Letter Report 2007-503