For other versions of this document, see http://wikileaks.org/wiki/CRS-RL32680
------------------------------------------------------------------------------

                                                   Order Code RL32680




                  CRS Report for Congress
                                      Received through the CRS Web




                       Peer Review: OMB's Proposed,
                          Revised, and Final Bulletins




                                        Updated February 3, 2005



                                             Curtis W. Copeland
                     Specialist in American National Government
                                Government and Finance Division

                                                   Eric A. Fischer
                     Senior Specialist in Science and Technology
                       Resources, Science, and Industry Division




Congressional Research Service ~ The Library of Congress
 Peer Review: OMB's Proposed and Revised Bulletins

Summary
     In September 2003, the Office of Management and Budget (OMB) published
a proposed bulletin on "Peer Review and Information Quality" in the Federal
Register that sought to establish a process by which all "significant regulatory
information" would be peer reviewed. The scope of the proposed bulletin was very
broad, covering virtually all agencies and defining regulatory information as "any
scientific or technical study that ... might be used by local, state, regional, federal
and/or international regulatory bodies." Such information would be subject to peer
review if the agency could determine that it could have a "clear and substantial
impact on important public policies or important private sector decisions" when
disseminated. The proposed bulletin placed additional peer review requirements on
"especially significant regulatory information," and said agencies were required to
notify OMB in advance of any studies that might require peer review and how any
such reviews would be conducted.

     The proposed bulletin aroused controversy, with some observers expressing
concern that it could create a centralized peer review system within OMB that would
be vulnerable to political manipulation or control by regulated entities. OMB
received nearly 200 comments on the proposal, and published a "substantially
revised" peer review bulletin in April 2004 that was broader in scope than the
proposed bulletin in that it applied to "influential scientific information" (which
includes, but is not limited to, regulatory information) and "highly influential
scientific assessments." However, agencies were given substantial discretion to
decide whether information is "influential" and therefore requires a peer review. The
revised bulletin also allowed agencies to use the National Academy of Sciences for
peer reviews or to use other procedures that had been approved by OMB. It also
provided exemptions for certain classes of information, such as information related
to national security, products by government-funded scientists that are not
represented as views of a federal agency, and routine statistical information.
However, OMB retained significant authority to decide when information is "highly
influential" (and, therefore, requires more specific peer review procedures) and to
approve alternative peer review procedures.

      OMB received more than 50 comments on the revised peer review bulletin,
many of which were supportive of the changes made to the proposal. However, some
commenters believed the changes did not go far enough, while others believed that
OMB had significantly weakened the bulletin. In January 2005, OMB published a
final version of the bulletin with what it described as "minor revisions" to the version
published in April 2004 (e.g., requiring agencies to disclose the identities of peer
reviewers and to prepare an annual report on their peer review activities). A number
of issues regarding the implementation of the bulletin remain unclear (e.g., how much
discretion agencies will be given to decide when and what kind of peer review is
required). This report will be updated when any further revisions to the bulletin are
published or other significant events occur.
Contents
Peer Review and Regulatory Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
     Support for Peer Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
          Legislative Proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
          Presidential Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
     Cautionary Notes About Peer Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
          Procedural Flexibility or Uniformity . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
     Peer Review and the Information Quality Act . . . . . . . . . . . . . . . . . . . . . . . . 7

OMB's Proposed Peer Review Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
   Peer Review of Significant Regulatory Information . . . . . . . . . . . . . . . . . . . 9
   Review of Especially Significant Regulatory Information . . . . . . . . . . . . . . 9
   OMB Notification of Upcoming Peer Reviews . . . . . . . . . . . . . . . . . . . . . . 10
   Other Provisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Comments on the Proposed Peer Review Bulletin and OMB's Responses . . . . . 11
   Need for the Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
   Clarifications of Scope and Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . 12
   Agency Discretion and Peer Review Procedures . . . . . . . . . . . . . . . . . . . . . 13
   Selection of Reviewers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
   Public Participation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
   Cost of Implementing the Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
   Other Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

OMB's Revised Peer Review Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
   Changes in Scope and Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
   Influential Scientific Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
        When Peer Review Is Required . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
   Highly Influential Scientific Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . 19
        When Enhanced Peer Reviews Are Required . . . . . . . . . . . . . . . . . . . 20
        Selection of Reviewers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
        Public Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
        Other Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
        Alternative Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
        OMB Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
   Upcoming Peer Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Comments on the Revised Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
   National Academy of Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
   OMB Watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
   Chamber of Commerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
   Members of Congress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
   AAAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
   Other Scientific Societies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Final Peer Review Bulletin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Concluding Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
    Agency Discretion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
    Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
    Effects on Agencies and Rulemaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

List of Tables
Table 1. Comparison of Proposed, Revised, and Final Peer Review Bulletins . . 31
         Peer Review: OMB's Proposed and
                 Revised Bulletins

      On September 15, 2003, the Office of Management and Budget (OMB) in
coordination with the Office of Science and Technology Policy (OSTP), published
a proposed bulletin in the Federal Register on "Peer Review and Information
Quality." The bulletin, if made final, would have established a process by which all
"significant regulatory information" and "especially significant regulatory
information" would be peer reviewed.1 OMB described the term "peer review" in
this context as "a scientifically rigorous review and critique of a study's methods,
results, and findings by others in the field with requisite training and expertise." The
proposed bulletin placed additional peer review requirements on "especially
significant regulatory information," and said agencies were required to notify OMB
in advance of any studies that might require peer review. The scope of the proposed
bulletin was very broad, covering virtually all agencies and defining "regulatory
information" as "any scientific or technical study that ... might be used by local,
state, regional, federal and/or international regulatory bodies." OMB indicated it was
issuing the bulletin because agencies' peer review practices were inconsistent, and
government-wide standards for peer review would make regulatory science more
competent and credible.

     The proposed bulletin aroused substantial controversy, with some observers
expressing concern that it could create a centralized peer review system within OMB
that would be vulnerable to political manipulation or control by regulated entities.
OMB received nearly 200 comments on the proposal, including comments from
Members of Congress, trade associations, public interest groups, and recognized
experts in the field of peer review and scientific research. As a result of those
comments, OMB (again in consultation with OSTP) published what it described as
a "substantially revised" peer review bulletin in the Federal Register on April 28,
2004.2 In some ways, the revised bulletin was broader than its predecessor. For
example, instead of focusing on "significant" and "especially significant regulatory

1
 Office of Management and Budget, Executive Office of the President, "Proposed Bulletin
on Peer Review and Information Quality," 68 Federal Register 54023 (Sept. 15, 2003). This
proposed bulletin had been released to the public via OMB's website on Aug. 29, 2003. To
view a copy, see [http://www.whitehouse.gov/omb/inforeg/peer_review_and_info_quality.
pdf].
2
   Office of Management and Budget, Executive Office of the President, "Revised
Information Quality Bulletin on Peer Review," 69 Federal Register 23230 (Apr. 28, 2004).
This revised bulletin had been released to the public via OMB's website on April 15, 2004;
see [http://www.whitehouse.gov/omb/inforeg/peer_review041404.pdf]. In this report, the
first draft of the bulletin is referred to as the "proposed bulletin" and the second draft as the
"revised bulletin." Unless otherwise specified, each respective reference includes the
introductory supplemental information as well as the body of the bulletin per se.
                                          CRS-2

information," the revised bulletin centered on "influential scientific information"
(which includes, but is not limited to, regulatory information) and "highly influential
scientific assessments." In other ways, though, the revised bulletin was less inclusive
and directive. For example, it gave agencies more discretion to determine when
information required a peer review, and when the more detailed review requirements
for "highly influential" information were applicable. Also, unlike the proposed
bulletin, the revised bulletin did not exclude individuals from being peer reviewers
if they had received research grants from the agency disseminating the information
being peer reviewed. OMB again requested comments from the public on the
bulletin.

     On December 15, 2004, OMB published a final version of the peer review
bulletin on its website.3 The final bulletin was published in the Federal Register on
January 14, 2005.4 OMB said this version reflects "minor revisions" made in
response to more than 50 comments from the public on the revised bulletin. For
example, the final bulletin requires agencies to disclose the names of peer reviewers
to the public and adds an annual reporting requirement to allow OMB to track how
agencies are using the bulletin. However, agencies are still afforded substantial
discretion to determine when and what type of peer review is required. OMB also
retains substantial discretion in certain areas.

     This report briefly describes OMB's September 2003 peer review proposal, the
concerns that were raised about that proposal in the public comments and elsewhere,
and OMB's April 2004 revised bulletin. It then describes comments on the revision
and OMB's final bulletin. First, however, the report provides some background on
peer review, particularly in the context of regulatory policy and recent efforts to
improve the quality of regulatory information.


               Peer Review and Regulatory Policy
      There is no official, government-wide definition of "peer review," even in the
OMB bulletins.5 Agency-specific definitions and descriptions of the term differ, but
all of them contain the basic concept of a review of scientific or technical merit by
individuals with both sufficient technical competence and no unresolved conflicts of


3
 Office of Management and Budget, Final Information Quality Bulletin for Peer Review,
Dec. 15, 2004, at [http://www.whitehouse.gov/omb/inforeg/peer2004/peer_bulletin.pdf].
4
 Office of Management and Budget, Final Information Quality Bulletin for Peer Review,
70 Federal Register 2664 (Jan. 14, 2005).
5
  While the term "peer review" is not explicitly defined in OMB's bulletins, the
supplementary information in the final bulletin describes peer review generally as "one of
the important procedures used in science to ensure that the quality of published information
meets the standards of the scientific community. It is a form of deliberation involving an
exchange of judgments about the appropriateness of methods and the strength of the author's
inferences. Peer review occurs when a draft product is reviewed for quality by specialists
who were not involved in producing the draft." Arguably, the methods laid out in the
bulletin for conducting peer review could also be considered an implicit definition of the
concept.
                                           CRS-3

interest. In its final bulletin, OMB states that peer review "typically evaluates the
clarity of hypotheses, the validity of research design, the quality of the data collection
procedures, the robustness of the methods employed, the appropriateness of the
methods for the hypotheses being tested, the extent to which the conclusions follow
from the analysis, and the strengths and limitations of the overall product."6

Support for Peer Review
      Some type of peer review has been used for centuries within the scientific
community to judge the quality of science. Peer review can take many different
forms, and is used for a variety of purposes. For example, it is used commonly by
federal agencies to evaluate research proposals, and plays a major role in funding
decisions. In these cases it is often part of a broader category of evaluation known
as merit review. Peer review is also the usual method by which the editors of
scientific publications evaluate proposed research reports. It even plays an important
role in many research institutions, including many government agencies, in decisions
about retention and promotion for individual scientists and in reviewing research
programs. A February 1999 report from the Committee on Science, Engineering, and
Public Policy concluded that expert review (the most common form of which is peer
review) is the most effective means of evaluating federally funded research
programs.7

     Peer review is also used for scientific and technical products relating to policies,
including regulations, to determine whether the underlying scientific findings are
well supported. For example, such peer review was established as Environmental
Protection Agency (EPA) policy in 1993.8 In 1996, a panel of leading economists
concluded that peer review should be used for economic analyses supporting
regulations with a potentially large impact on the economy (e.g., those whose annual
economic costs exceed $1 billion).9 The panel also indicated that reviewers should
be selected based on their demonstrated expertise and reputations.

     In its peer review bulletins, OMB recognized the variety of ways that peer
review is used. OMB said that independent peer review is especially important in the
regulatory arena because federal agencies often develop or fund the science that
underlies their regulations, and then oversee the peer review of those scientific
studies -- thereby creating the appearance of a conflict of interest.




6
    Final Information Quality Bulletin for Peer Review, p. 2.
7
 The Committee on Science, Engineering, and Public Policy, Evaluating Federal Research
Programs: Research and the Government Performance and Results Act (Washington:
National Academies Press, Feb. 1999). The Committee is a joint committee of the National
Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
8
 U.S. General Accounting Office, Federal Research: Peer Review Practices at Federal
Science Agencies Vary, GAO/RCED-99-99, Mar. 1999.
9
 Kenneth J. Arrow, et al, Benefit-Cost Analysis in Environmental, Health, and Safety
Regulation: A Statement of Principles (Annapolis, MD: AEI Press, 1996).
                                       CRS-4

     Legislative Proposals. A number of statutes and legislative proposals have
advocated the use of peer review in a regulatory context for particular issues. For
example, the Safe Drinking Water Act (42 U.S.C. 300g-1(b)(3)(A)) requires EPA,
when taking action under the act based on science, to use the "best available, peer
reviewed science and supporting studies." In the 108th Congress, the Water
Resources Development Act of 2003 (H.R. 2557) would have generally required
project studies to be subject to peer review by an independent panel of experts if the
project has an estimated total cost of more than $50 million. Similarly, the Sound
Science for Endangered Species Act Planning Act of 2003 (H.R. 1662, 108th
Congress) would have directed the Secretary of the Interior, in making decisions
about species protection, to give greater weight to certain kinds of data that had been
peer reviewed by qualified individuals as defined in the bill.

      There have also been legislative efforts to require peer review more broadly.
For example, in the 106th Congress, the Senate considered but did not enact bipartisan
legislation (S. 746) that would have required virtually all agencies to provide for an
independent peer review of any required risk assessments and cost-benefit analyses
of major rules that the agencies or the OMB Director reasonably anticipated were
likely to have a $500 million effect on the economy. The bill would have required
that peer reviews be conducted through panels that were "broadly representative" and
involved participants with relevant expertise who were "independent of the agency."
However, if an agency certified that adequate peer review had already been
conducted, and the OMB Director agreed, no further peer review would have been
required. In its comments on this legislation, the General Accounting Office (GAO)
(now the Government Accountability Office) generally supported the use of peer
review in this context, noting that "the rigorous, independent review of economic
analyses should help enhance the quality, credibility, and acceptability of agencies'
decisionmaking."10 However, GAO cautioned that (given the number of reviews
contemplated) agencies would need to plan carefully for the reviews, and that the
panels would need to reflect all points of view.

     Presidential Support. Recent presidential administrations have also
supported peer review as a preferred means of assessing scientific research, both
prospectively and retrospectively. For example, beginning with the FY1996 budget
cycle, OMB and OSTP have jointly provided annual direction to agencies,
encouraging them to emphasize the funding of peer-reviewed research over nonpeer-
reviewed research for most scientific activities. In 1997, the then-Administrator of
OMB's Office of Information and Regulatory Affairs (OIRA) testified that the
Clinton Administration supported peer review, but also said the administration
recognized that it is not cost-free in terms of agencies' resources or time.11

    On September 20, 2001, the new OIRA Administrator for the George W. Bush
Administration issued a memorandum for the President's Management Council
recommending (among other things) that agencies subject regulatory impact analyses


10
  U.S. General Accounting Office, Regulatory Reform: Comments on S. 746 -- The
Regulatory Improvement Act of 1999, GAO/T-GGD/RCED-99-163, Apr. 1999, p. 6.
11
  Statement of Sally Katzen, OIRA Administrator, before the Senate Committee on
Governmental Affairs, Sept. 12, 1997.
                                         CRS-5

and supporting technical documents for "economically significant" and "major" rules
(e.g., those with a $100 million annual effect on the economy) to independent,
external peer review.12 The OIRA Administrator also recommended certain criteria
for peer review (e.g., disclosure by peer reviewers of prior technical or policy
positions on the issues at hand and their sources of personal and institutional
funding), and said OIRA would give agency analyses that had undergone such a
review "a measure of deference" during its reviews of their regulatory proposals
under Executive Order 12866.13

Cautionary Notes About Peer Review
     Although the concept of peer review has generally received wide support, some
observers have also raised cautionary notes or mentioned certain limitations to the
approach in particular situations. For example, as OMB indicated in its 1997
testimony, peer review costs can be significant for agencies in terms of both time and
agency resources.14 To pay for peer review procedures, agencies may have to divert
resources from other areas (e.g., regulatory enforcement or standards development).
Scientists from academic institutions who perform peer reviews on a voluntary basis
may also incur opportunity costs with respect to other activities such as teaching and
research. Other concerns have focused on how peer reviews have been implemented.
For example, GAO noted that peer review methods varied within and among federal
agencies,15 and that agencies' economic analyses of major rules were often not peer
reviewed.16 A number of observers have expressed concerns about the impact of peer
review requirements on the pace of regulatory activity, with additional requirements
exacerbating what is already often regarded as an "ossified" rulemaking process.17


12
  For a copy of this memorandum, see
[http://www.whitehouse.gov/omb/inforeg/oira_review-process.html].
13
 Executive Order 12866, "Regulatory Planning and Review," 58 Federal Register 51735,
Oct. 4, 1993. For a description of OMB's reviews under this executive order, see CRS
Report RL32397, Federal Rulemaking: The Role of the Office of Information and
Regulatory Affairs, by Curtis W. Copeland.
14
  Statement of Sally Katzen, OIRA Administrator, before the Senate Committee on
Governmental Affairs, Sept. 12, 1997.
15
  GAO/RCED-99-99. For example, GAO reported that some agencies conducted peer
reviews of research proposals primarily by mail, while others generally relied on panels or
committees. All agencies used a combination of external and internal reviewers for these
reviews, but one (Federal Aviation Administration) relied primarily on agency employees
who were not employed in the project but had the required expertise.
16
   U.S. General Accounting Office, Regulatory Reform: Agencies Could Improve
Development, Documentation, and Clarity of Regulatory Economic Analyses, GAO/RCED-
98-142, May 1998. Only 1 of the 20 economic analyses that GAO reviewed had been
independently peer reviewed.
17
   See, for example, Thomas O. McGarity, "Some Thoughts on `Deossifying' the
Rulemaking Process, Duke Law Journal, vol. 41 (June 1992), pp. 1385-1462; Richard J.
Pierce, Jr., "Seven Ways to Deossify Agency Rulemaking," Administrative Law Review, vol.
47 (winter 1995), pp. 59-93; Paul R. Verkuil, "Rulemaking Ossification -- A Modest
                                                                          (continued...)
                                          CRS-6

In fact, some critics have suggested that regulatory relief and delay is the primary
purpose of peer review proposals in this context.

      Still other concerns about peer review have centered on issues of bias and
balance. Experts agree that effective peer review panels must be (and must be
perceived to be) free from any significant conflict of interest and properly balanced,
allowing for a wide range of views and appropriate expertise. However, in June
2001, GAO reported that the policies and procedures developed by EPA's Science
Advisory Board to ensure balance and independence of the Board's peer reviewers
had limitations that reduced their effectiveness.18 For example, GAO said that the
staff office did not routinely ensure that panel members' financial disclosures were
complete or that they contained enough information to determine whether a conflict
of interest existed. Also, panel members reportedly did not have to disclose
information regarding previous positions on the matter being reviewed until the
panel's first meeting, thereby making it difficult to determine the independence and
balance of the panel members until after they had been selected. GAO also reported
that the panel's policies and procedures did not adequately inform the public about
the points of view represented on the panels. GAO made several recommendations
in this report that were designed to better ensure that the Science Advisory Board's
peer review panels are independent and balanced.

      Procedural Flexibility or Uniformity. As noted previously, GAO reported
that federal agencies' peer review practices are not consistent.19 However, several
studies of peer review in the federal government have suggested that rigidly uniform
peer review procedures may not be desirable. For example, a 1996 National Science
and Technology Council report indicated that peer review implementation should be
flexible and "appropriate to the nature of scientific processes."20 Similarly, in its
1999 report on peer review, GAO reported that OSTP believed agencies' peer review
practices should be "flexible and tailored to agency missions and type of research,
and that specific uniform practices should not be dictated for every agency or all
federally funded research."21 GAO also said that agencies viewed a variety of peer
review methods as "both appropriate and essential, reflecting the varying nature of
the research and its purposes, the differences in research timelines, the broad
spectrum of [research and development] performers, and the varying funding


17
  (...continued)
Proposal," Administrative Law Review, vol. 47 ( summer 1995), pp. 453-459.
18
  U.S. General Accounting Office, EPA's Science Advisory Board Panels: Improved
Policies and Procedures Needed to Ensure Independence and Balance, GAO-01-536, June
2001.
19
     GAO/RCED-99-99.
20
   Subcommittee on Research, Committee on Fundamental Science, National Science and
Technology Council, Assessing Fundamental Science (Washington: National Science
Foundation, July 1996). The National Science and Technology Council was established by
executive order in November 1993 to coordinate federal research and development, and to
establish clear national goals for science and technology investments. To view a copy of this
report, see [http://www.nsf.gov/sbe/srs/ostp/assess/start.htm].
21
     GAO/RCED-99-99, p. 5.
                                        CRS-7

mechanisms, such as grants, contracts, and cooperative agreements." Notably,
although GAO pointed out that peer review practices varied across and within
agencies, it did not recommend greater uniformity in those practices.

Peer Review and the Information Quality Act
     OMB said that the final peer review bulletin was being issued "under the
Information Quality Act [IQA] and OMB's general authorities to oversee the quality
of agency information, analyses, and regulatory actions."22 The IQA (also known as
the "Data Quality Act") was enacted in December 2000 as Section 515 of the
Treasury and General Government Appropriations Act for Fiscal Year 2001 (P.L.
106-554). The act supplemented the Paperwork Reduction Act (PRA), which already
required OMB to "develop and oversee the implementation of policies, principles,
standards, and guidelines to apply to federal agency dissemination of public
information."23 The PRA also required agencies to manage their information
resources to "improve the integrity, quality, and utility of information to all users
within and outside the agency."24

     Although little noticed at the time it was enacted, the IQA has subsequently
been the subject of significant debate and controversy.25 The act required OMB to
issue guidance to federal agencies designed to ensure the "quality, objectivity, utility,
and integrity" of information disseminated to the public. It also required agencies to
issue their own information quality guidelines, and to establish administrative
mechanisms that allow affected persons to seek correction of information maintained
and disseminated by the agencies that does not comply with the OMB guidance.
However, it did not require either OMB or the agencies to issue peer review bulletins.

     OMB published its final government-wide IQA guidelines in the Federal
Register in February 2002.26 In them, OMB encouraged (but did not require) the use
of peer reviews in the development of agency-disseminated information.
Specifically, OMB indicated that agencies can presume that data are sufficiently
"objective" if they have been subject to an independent peer review process (e.g., as
used by scientific journals), but said a member of the public could rebut this
presumption "based on a persuasive showing by the petitioner in a particular
instance." OMB also indicated that journal peer review may not be sufficient for


22
  Office of Management and Budget, Final Information Quality Bulletin for Peer Review,
op. cit., p. 2666.
23
     44 U.S.C. 3504 (d)(1).
24
     44 U.S.C. 3506(b)(1)(C).
25
  For a more complete discussion of the IQA and OMB's information quality guidelines,
see CRS Report RL32532, The Information Quality Act: OMB's Guidance and Initial
Implementation, by Curtis W. Copeland and Michael Simpson.
26
  Office of Management and Budget, "Guidelines for Ensuring and Maximizing the Quality,
Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies," 67
Federal Register 8452, Feb. 22, 2002. OMB published proposed guidelines in June 2001
(66 Federal Register 34489), and published an earlier version of the final guidelines in
September 2001 (66 Federal Register 49718).
                                          CRS-8

information likely to have an important public policy or private sector impact.
Finally, the IQA guidelines set minimum standards for the transparency of agency-
sponsored peer review (referencing the standards in the above-mentioned September
20, 2001, memorandum issued by the OIRA Administrator -- e.g., disclosure of
reviewers' prior technical or policy positions and sources of funding).

     The final peer review bulletin also cited Executive Order 12866 and OMB's
authority to "manage the agencies under the purview of the President's Constitutional
authority to supervise the unitary Executive Branch" as other authorities supporting
the bulletin's issuance.27 E.O. 12866 says that OIRA within OMB is the "repository
of expertise concerning regulatory issues," and says that agencies should base their
decisions on "the best reasonably obtainable scientific, technical, economic, or other
information concerning the need for, and consequences of the intended regulation."28
However, the executive order does not specifically mention peer review.


              OMB's Proposed Peer Review Bulletin
     When it published its proposed peer review bulletin in the Federal Register in
September 2003, OMB noted in the preamble to the bulletin the importance of peer
review and the sometimes inconsistent nature of its application. For these reasons,
the office proposed requiring peer review for "the most important scientific and
technical information relevant to regulatory policies that [agencies] disseminate to
the public."29 OMB said the new requirements would supplement, but not supplant,
both its own information quality guidelines and any agency-specific peer review
requirements for regulatory information. Although OMB noted that its bulletins "are
not laws or binding regulations," as a practical matter, OMB's instructions to the
agencies carry substantial weight because of its authority to review agencies' budget
proposals and most significant regulations before they are published.

      In essence, OMB's bulletin proposed that each covered federal agency take three
actions: (1) have all "significant regulatory information" that the agencies intend to
disseminate peer reviewed; (2) have "especially significant regulatory information"
peer reviewed according to even higher standards; and (3) provide OMB with
information at least once each year about upcoming scientific studies that might
support significant regulatory disseminations, and the agencies' plans for conducting
peer reviews of those studies. The proposed bulletin also contained several other,
related requirements. The bulletin defined "agency" broadly, including cabinet
departments and agencies, independent agencies (e.g., EPA), and independent
regulatory agencies (e.g., the Federal Communications Commission and the Nuclear
Regulatory Commission).




27
  Office of Management and Budget, Final Information Quality Bulletin for Peer Review,
op. cit., p. 7.
28
     Executive Order 12866, section 1(b)(7).
29
     68 Federal Register 54026, Sept. 15, 2003.
                                         CRS-9

Peer Review of Significant Regulatory Information
     The proposed bulletin defined "significant regulatory information" as scientific
or technical information that qualifies as "influential" under OMB's information
quality guidelines and is "relevant to regulatory policies." OMB's information
quality guidelines define "influential information" as information that the agency can
"reasonably determine" will have a "clear and substantial impact on important public
policies or important private sector decisions" when it is disseminated to the public.
The proposed peer review bulletin said information is "relevant to regulatory
policies" if it "might be used by local, state, regional, federal and/or international
regulatory bodies." The bulletin indicated that the peer review for this type of
information should be "appropriate" and "scientifically rigorous," but also said the
appropriate peer review mechanism would depend on "the novelty and complexity
of the science to be reviewed, the benefit and cost implications, and any controversy
regarding the science."

     For information at the lower end of those dimensions, the proposed bulletin
indicated that a review by qualified specialists within a separate part of the agency
disseminating the information could satisfy the peer review requirement. Also (as
in the information quality guidelines), agencies were allowed to presume that any
studies that had already been independently peer reviewed did not have to be
reexamined, but the bulletin said this presumption "is rebuttable based on a
persuasive showing in a particular instance." Finally, the bulletin said peer review
was not needed for significant regulatory information related to national defense or
foreign affairs, or information that is disseminated in the course of an individual
agency adjudication or proceeding on a permit application.

Review of Especially Significant Regulatory Information
     The proposed bulletin indicated that regulatory information should be
considered "especially significant" (and therefore subject to more specific peer
review requirements) if (a) the agency intends to disseminate it in support of an
economically significant regulatory action (e.g., with a $100 million annual impact
on the economy),30 (b) its dissemination would otherwise have a clear and substantial
effect on important public policies or important private sector decisions (with an
impact of $100 million in any year), or (c) OMB determines it is of "significant
interagency interest" or "is relevant to an Administration policy priority." The
bulletin laid out a series of proposed requirements for these types of peer reviews.
For example, it said agencies must

     !   select peer reviewers primarily based on their scientific and technical
         expertise and "strive" to appoint reviewers who are independent of
         the agency and do not have any "real or perceived conflicts of
         interest" (e.g., have no financial interest in the issue, do not seek or
         receive substantial funding from the agency, and have not conducted
         a peer review for the agency or advocated a position in recent years


30
  The definition of an "economically significant" regulatory action can be found in section
3(f)(1) of Executive Order 12866.
                                       CRS-10

         on the specific topic being examined). If obtaining the requisite
         expertise requires the selection of a panel member with a perceived
         bias, the bulletin said agencies must "ensure that another reviewer
         with a contrary bias is appointed to balance the panel";

     !   provide peer reviewers with an explicit statement describing the
         purpose and scope of the review, and sufficient information to
         understand the data, methods, and conclusions of the material being
         reviewed;

     !   permit other interested agencies and persons to submit comments
         and provide those comments to peer reviewers for their
         consideration; and

     !   direct peer reviewers to issue a final report describing the nature of
         their review and their findings and conclusions, including any
         dissenting opinions. The report must also describe the names,
         affiliations, and qualifications of all peer reviewers and any prior
         affiliations with the agency.

Also, agencies were instructed to consult with OMB and OSTP regarding the
"sufficiency" of their planned peer review policies and, on request, to discuss with
OMB the sufficiency of their plans to review specific documents.

OMB Notification of Upcoming Peer Reviews
     The proposed bulletin indicated that, at least once each year, each agency must
provide OMB with a summary description of any "existing, ongoing, or
contemplated" studies that might support significant regulatory information that the
agency intends to disseminate within the next year. Each agency was also to provide
OMB with the agency's plan for conducting peer reviews of such studies under the
bulletin's requirements, including an agency contact person. OMB indicated that, to
reduce paperwork, this information was to be included in other reports being
submitted to OMB under Executive Order 12866 or the Information Quality Act.

Other Provisions
     The proposed bulletin also contained several other peer review requirements or
provisions, including the following:

     !   Agencies that are likely to disseminate "significant" or "especially
         significant" regulatory information were required to supplement or
         amend their information quality guidelines to incorporate the
         requirements of the proposed peer review bulletin for "significant"
         and "especially significant" information.         These agencies'
         supplements or amendments were also to address (a) what factors
         could preclude an individual's participation as a peer reviewer, (b)
         how confidential business information and other confidential
         information would be protected, and (c) what information about peer
                                        CRS-11

          reviewers should be disclosed (e.g., sources of funding that could
          suggest a conflict of interest).

      !   OMB could waive the requirements for peer review if an agency
          made "a compelling case" that a waiver was necessary (e.g., an
          imminent health hazard or homeland security threat).

      !   Agencies were permitted to retain an outside firm to oversee the peer
          review process to select and supervise the peer review panels, which
          the bulletin indicated would allow the agencies to avoid the
          requirements of the Federal Advisory Committee Act (FACA).31

OMB initially said these peer review requirements would apply to information
disseminated on or after January 1, 2004. OMB initially requested public comments
on the proposed bulletin by October 28, 2003, but later extended the comment period
to December 15, 2003, and then to January 16, 2004.


     Comments on the Proposed Peer Review Bulletin
               and OMB's Responses
     OMB said that it received 187 comments on the proposed bulletin on peer
review, and posted those comments on its website.32 In addition to this formal
commenting process, OMB and OSTP sponsored an open workshop on the peer
review requirements in November 2003.33 The workshop was hosted by the National
Academy of Sciences (NAS), which subsequently endorsed the bulletin's objectives
but expressed several concerns. OMB said the comments that it received varied
substantially, with some highly favorable and others suggesting that the bulletin be
withdrawn and reconsidered. In general, supporters of the proposed bulletin
reportedly indicated that peer review standards across the government were
inconsistent, and that more consistent use of peer review could increase the technical
quality and credibility of regulatory science. They also reportedly asserted that peer
review could protect science-based regulations from political criticism and litigation.
For example, the U.S. Chamber of Commerce said that OMB had "provided a
Bulletin of high quality and sound judgment," and strongly supported "OMB's efforts
to improve the quality, objectivity, utility, and integrity of information disseminated




31
  The bulletin cited Byrd v. EPA, 174 F.3d 239 (D.C. Cir. 1999), "holding that peer review
panels selected and supervised by outside consultants are not governed by [FACA]." For
more information on federal advisory committees and FACA, see CRS Report RL30260,
Federal Advisory Committees: A Primer, by Stephanie Smith; or CRS Report RL30795,
General Management Laws: A Compendium, coordinated by Clinton T. Brass, pp. 35-37.
32
 To view the comments provided on the proposed peer review bulletin, see [http://www.
whitehouse.gov/omb/inforeg/2003iq/iq_list.html].
33
  To view a transcript of this conference, see [http://www7.nationalacademies.org/stl/
Peer_Review_Transcript.pdf].
                                        CRS-12

by the federal government to the public."34 Similarly, the National Petrochemical &
Refiners Association strongly supported OMB's proposed bulletin, saying that it
would "correct the current variability in federal agency implementation of peer
reviews by providing the needed guidance on process requirements."35

      OMB said that many critics of the proposed bulletin viewed it as an effort to
inject political considerations into the world of science, and to use the uncertainty
that inevitably surrounds science as an excuse to delay new rules that could be costly
to regulated entities. They also reportedly said that the bulletin appeared more
concerned about peer reviewers' possible conflicts of interest with agencies than their
potential conflicts with regulated industries.

     In its summary of the public comments, OMB identified a number of broad
themes and noted how it responded to those comments.36 Specifically, OMB said the
comments centered on such issues as the need for the bulletin, its scope and
applicability, the prescriptive nature of the review process for "especially significant"
information, and the costs of implementing the bulletin.

Need for the Bulletin
      OMB said many commenters indicated that it was unclear why the bulletin was
needed. For example, the American Public Health Association said it was "unaware
of any evidence that the current system is not working or any examples of
inappropriate or flawed federal regulations being promulgated as a result of failure
to peer review."37 In response, OMB said it would provide additional information in
the revised bulletin on the importance of peer review and the variability of agencies'
peer review policies.

Clarifications of Scope and Applicability
     A number of commenters reportedly said the scope and applicability of the
bulletin was unclear.

     !   Some commenters questioned the focus on "regulatory" information,
         and said it would be difficult to know in advance whether a
         particular study might be used in support of future regulatory action.
         In its response to the comments, OMB said it would revise the


34
  For a copy of the Chamber's comments on the proposed bulletin, see [http://www.
whitehouse.gov/omb/inforeg/2003iq/170.pdf].
35
 For a copy of the association's comments on the proposed bulletin, see [http://www.
whitehouse.gov/omb/inforeg/2003iq/111.pdf].
36
   Office of Management and Budget, "Summary of Public and Agency Comments on
Proposed Bulletin on Information Quality and Peer Review, Including Responses by OMB,"
Apr. 15, 2004. A copy of this summary is available at [http://www.whitehouse.gov/omb/
inforeg/peer_review_comment.pdf].
37
 Letter to Dr. Margo Schwab, Dec. 11, 2003. To view a copy of this letter, see [http://
www.whitehouse.gov/omb/inforeg/2003iq/86.pdf].
                                          CRS-13

           bulletin to focus on "influential" scientific information instead of the
           potential regulatory impact of the information disseminated. As
           noted previously, the IQA guidelines state that information should
           be considered "influential" if "the agency can reasonably determine
           that dissemination of the information will have or does have a clear
           and substantial impact on important public policies or important
           private decisions."38

       !   Some commenters expressed concern that the criteria for requiring
           the more stringent peer review requirements for highly significant
           information were too broad. In response, OMB said it would narrow
           the applicability of those requirements to cover only "scientific
           assessments" that either have a $500 million annual effect on
           important public policies or private sector decisions or involve
           novel, complex, or precedent-setting approaches or significant
           interagency interest.

       !   OMB said it agreed with comments questioning the applicability of
           the bulletin's requirements to agencies' regulatory impact analyses,
           and said it would more clearly state that those analyses would not be
           covered by the revised bulletin.39 However, OMB went on to say
           that the bulletin's requirements would cover the models and data
           underlying those analyses, and encouraged agencies to have their
           regulatory impact analyses peer reviewed by "government analysts."

       !   OMB also said it would clarify that the bulletin did not apply to
           products released by government-funded scientists that were not
           represented as the views of the agency supporting the research.

       !   Finally, in response to several concerns, OMB said it would exempt
           from the bulletin's coverage any time-sensitive medical, public
           health, and safety disseminations, and allow agency heads to waive
           or defer the bulletin's requirements "where warranted by a
           compelling rationale."

Agency Discretion and Peer Review Procedures
     Commenters also said that the proposed bulletin was too prescriptive in the
procedures applicable to "especially significant" regulatory information, and said
extensive peer reviews would be costly and could lead to delays in regulatory
decisions. In response, OMB said that it would give agencies additional discretion
in the revised bulletin to determine the level of peer review required for any
particular document, and to use alternative procedures if they could demonstrate that
those procedures would meet the requirements in the government-wide and agency-

38
     Office of Management and Budget, "Summary of Comments."
39
  OMB said that guidance for what constitutes a good regulatory impact analysis is in OMB
Circular A-4. To view a copy of this circular, see [http://www.whitehouse.gov/omb/
circulars/a004/a-4.pdf].
                                       CRS-14

specific IQA guidelines. The response included significant discussion of the current
and potential role of NAS in conducting peer reviews. In its comments, NAS had
pointed to the role its committees have played in performing peer reviews of various
agency programs, activities, policies, and proposals. In determining the appropriate
level of review, OMB said agencies should consider the costs and benefits of peer
review. OMB also emphasized that its role is not to veto studies, but rather to ensure
that they have the appropriate level of review before implementation.

Selection of Reviewers
     OMB also said it was incorporating suggestions regarding the selection of peer
reviewers, including that agencies solicit reviewers from the public and from
scientific and professional societies. One major concern of the commenters was that
the criteria outlined in the proposed bulletin (e.g., discouraging participation of
reviewers who were seeking or receiving funding via research grants from an agency)
could result in the most qualified reviewers being excluded from consideration.
OMB also said that the revised bulletin would make clear that the receipt of research
grants through peer-reviewed competitions does not disqualify a potential reviewer.

Public Participation
      According to OMB, many of the commenters raised concerns about public
participation in the peer review process, questioning (among other things) whether
that participation would be useful and raising logistical considerations. As a result,
OMB said it had decided to allow the agencies to decide whether public participation
would be allowed and in what forms. However, OMB said it would provide for
public comment on the agencies' peer review plans for upcoming documents.

Cost of Implementing the Bulletin
     Finally, OMB said some of the commenters requested that OMB provide an
estimate of the costs associated with implementation of the peer review bulletin. For
example, in a March 2004 letter to OMB, Senators Joseph Lieberman and Richard
Durbin said it was "curious that OIRA -- an office known for insisting on cost-
benefit analysis -- provides no estimate of costs, nor documentation of the benefits
gained or specific harms that will be reduced by the proposal. We suggest that the
proposed rules do not pass OMB's own test of good rulemaking."40

     OMB estimated that there would be about 125 "influential" scientific documents
covered each year that are related to rulemaking, and at least 1,250 other documents
each year that are unrelated to rulemaking. OMB said some of these documents may
be exempt or otherwise not require peer reviews (e.g., they had already been
adequately peer reviewed), but did not quantify that assertion. OMB said that a "high-
quality individual letter review" would cost an agency about $5,000 per document.
Therefore, assuming that 1,000 of the 1,250 documents required peer review and

40
  To view a copy of this letter, see [http://govt-aff.senate.gov/index.cfm?FuseAction=
PressReleases.Detail&PressRelease_id=694&Affiliation=R]. This letter was not part of the
187 comments that OMB listed on the proposed bulletin.
                                          CRS-15

could be reviewed by letter, OMB said the cumulative cost of these reviews would
be $5 million.

      OMB said "one or two dozen" of these documents could be considered "highly
influential scientific assessments" requiring more intense peer review. OMB
estimated that an average peer review of these documents would cost $50,000.
Therefore, if 20 of these "highly influential" assessments were peer reviewed, the
cost would be $1 million. OMB said these costs are "likely large overestimates of
the incremental costs of the Bulletin because many agencies already engage in peer
review practices." The response also included an analysis that provided a
methodology for agencies to use in deciding when a more expensive peer-review
study by NAS would be cost-effective.41

Other Comments
    Several other types of comments were offered on the proposed guidelines that
were not included in OMB's summary. For example:

        !   OMB Watch questioned OMB's authority to issue a bulletin on peer
            review, noting that the PRA and the IQA do not explicitly provide
            the office with that responsibility, and that Congress had previously
            considered and rejected several proposals to mandate peer review.
            OMB Watch said that if OMB believes government-wide peer
            review requirements are needed, "it should instead submit its ideas
            to Congress for consideration and possible authorization."42

        !   The American Bar Association (ABA), noting that the proposed
            bulletin required peer review whenever the impact of the regulatory
            action would have exceeded $100 million, recommended that OMB
            limit the bulletin's categorical requirement for peer review to "those
            situations in which the scientific information is at least arguably
            complex, novel, or controversial."43 The ABA said peer review "is
            simply not the correct mechanism to address significant use of
            routine, established, or accepted scientific information."

        !   The ABA also said it agreed that one factor in determining whether
            an individual should be a peer reviewer is whether that individual is
            receiving or seeking substantial funding from the agency involved.
            However, the ABA also said that an issue of equal concern is
            whether a potential reviewer is receiving or seeking funding from a
            private company or industry or citizen group with a stake in the
            information involved.


41
     OMB estimated that such a study might cost $1 million and take two years to complete.
42
 Letter to Dr. Margo Schwab, Dec. 15, 2003. To view a copy of this letter, see [http://
www.whitehouse.gov/omb/inforeg/2003iq/153.pdf].
43
 Letter to Dr. Margo Schwab, Dec. 23, 2004. To view a copy of this letter, see [http://
www.whitehouse.gov/omb/inforeg/2003iq/186.pdf].
                                        CRS-16

     !   The American Association for the Advancement of Science (AAAS)
         said it was concerned that one of the bulletin's factors in selecting
         reviewers -- whether the prospective reviewer had advocated a
         position on the matter at issue -- could exclude a qualified scientist
         who had simply made statements based on a "preponderance of peer
         reviewed scientific evidence" (e.g., that particulate matter of a
         certain size had been shown to cause asthma in children).44 AAAS
         said such statements should not be considered "advocacy." The
         organization also said that OMB should make clear that individual
         comments in agencies' peer review reports would not be associated
         with individual reviewers. In March 2004, the AAAS Council
         adopted a resolution strongly opposing the proposed bulletin and
         requesting that it be withdrawn.45

     !   The AEI-Brookings Joint Center for Regulatory Studies
         recommended (among other things) that OMB or some other entity
         conduct an evaluation of the peer review program after a certain
         amount of time "to determine whether peer review actually led to a
         material improvement in the quality of the regulatory analysis." 46
         AEI-Brookings said if OMB could not develop a reasonable
         approach to evaluation, "it probably should not proceed with this
         effort."


             OMB's Revised Peer Review Bulletin
     The comments that OMB received on the proposed bulletin led the office to
release what it described as a "substantially revised" bulletin in April 2004. The
changes were characterized in the media as a "partial retreat" in OMB's initiative.47
OMB said the revised bulletin provided a more extensive discussion of why
government-wide peer review guidance was needed, gave agencies more discretion
in determining the appropriate type of peer review for specific products, and more
clearly indicated that the guidance does not create any new rights for litigation. Also,
the revised bulletin permitted an agency (under certain circumstances) to use a
scientist as a peer reviewer who had received research grants from the agency -- a
practice that was discouraged in the proposed bulletin. The table appended to this



44
 Letter to Dr. Margo Schwab, Dec. 12, 2003. To view a copy of this letter, see [http://
www.whitehouse.gov/omb/inforeg/2003iq/81.pdf].
45
  American Association for the Advancement of Science, Council, "On the OMB Proposed
Peer Review Bulletin," Mar. 9, 2004, at [http://archives.aaas.org/docs/documents.php?doc_
id=434].
46
  Robert W. Hahn and Robert E. Litan, Comment on Peer Review and Information Quality,
Regulatory Analysis 03-11, Dec. 2003. For a copy of this comment, see [http://www.aei.
brookings.org/admin/authorpdfs/page.php?id=309].
47
 Rick Weiss, "OMB Modifies Peer-Review Proposal: Guidelines Partly Retreat from Strict
Control of Agencies' Information Process," Washington Post, Apr. 16, 2004, p. A-19.
                                         CRS-17

report provides a side-by-side comparison of the proposed and revised bulletins. The
sections below discuss similarities and differences in the documents.

Changes in Scope and Application
     Among the more notable changes made to the bulletin were revisions to the
scope and applicability of the peer review requirements. In general, the scope of the
revised bulletin was arguably much broader than the proposed bulletin. For example,
whereas the proposed bulletin applied to "significant regulatory information," the
revised bulletin applied to "influential scientific information" -- a scope that
included, but was not limited to, regulatory information.48 However, the revised
bulletin also made it clear that certain types of information within this broad category
did not have to be peer reviewed, including:

     !   information distributed within the government or to contractors or
         grantees;

     !   responses to requests under the Freedom of Information Act (FOIA),
         the Privacy Act, or FACA;49

     !   correspondence with individuals;

     !   press releases;

     !   and distributions made to peer reviewers under the bulletin, provided
         they contain a disclaimer.50

      The revised bulletin also explicitly exempted from its requirements information
that (1) was related to national security, foreign affairs, or international trade or
treaties where compliance with the bulletin "would interfere with the need for secrecy




48
  The explanatory section on requirements in the revised document states that the bulletin
"addresses peer review of scientific information disseminations that contain findings or
conclusions that represent the official position of one or more Departments or agencies of
the federal government." However, the term "official position" is not mentioned in the
bulletin itself. The revised bulletin does not include the term "regulatory information." The
original proposed bulletin defines "study" as "any research report, data, finding, or other
analysis." While this definition is somewhat broader than the meaning usually ascribed to
the term by scientists, it is narrower than the definition of "scientific information" in the
revised bulletin, which includes "any communication or representation of knowledge ... in
any medium or form."
49
  FOIA is described in General Management Laws: A Compendium, coordinated by Clinton
T. Brass, pp. 26-29.
50
  The disclaimer is required to say "This information is distributed solely for the purpose
of pre-dissemination peer review under applicable information quality guidelines. It has not
been formally disseminated by [the agency] and should not be construed to represent any
agency determination or policy."
                                        CRS-18

or promptness";51 (2) was produced by government-funded scientists that was not
represented as the views of the agency; (3) was an agency regulatory impact analysis
or regulatory flexibility analysis under Executive Order 12866;52 or (4) was medical,
health, or safety information that was time sensitive or based on a recent clinical trial
that had already been adequately peer reviewed. The bulletin also allowed agency
heads to waive or defer the bulletin requirements "where warranted by a compelling
rationale."53

     In structure, the revised peer review bulletin was similar to the proposed bulletin
in that it still essentially required agencies to take three actions (to the extent
permitted by law):

     !   have a peer review conducted on all "influential scientific
         information" that the agency intends to disseminate,

     !   have all "highly influential scientific assessments" peer reviewed
         according to more specific and demanding standards, and

     !   indicate what "influential" and "highly influential" information the
         agency plans to peer review in the future.

Each of these required actions is described more fully in the sections below.

Influential Scientific Information
     The revised peer review bulletin defined the term "influential scientific
information" as information the agency "reasonably can determine that the
dissemination of which will have or does have a clear and substantial impact on
important public policies or private sector decisions." Therefore, in contrast to the
proposed bulletin, the peer review requirements in the revised bulletin were not
limited to regulatory policies.

      In addition to the limitations and exemptions discussed above, the revised
bulletin said that agencies were also not required to conduct a peer review of
influential information that had already had an "adequate" peer review. The bulletin
gave agencies substantial discretion in determining whether a prior review was
"adequate," specifically stating that the earlier review "need not comply with all of
the requirements of this bulletin." It also said that the sufficiency of the prior review
depended on the "novelty and complexity of the science to be reviewed and the
benefit and cost implications." Language in the proposed bulletin about rebutting an

51
  The proposed bulletin said agencies "need not have peer review conducted on significant
regulatory information that relates to national defense or foreign affairs."
52
  As noted previously, OMB has said that the data used to generate these analyses are
covered by the peer review requirements. Also, the bulletin did not indicate whether
regulatory flexibility analyses under the Regulatory Flexibility Act were exempt.
53
  The proposed bulletin allowed OMB (not the agencies) to waive many of the bulletin's
requirements if an agency "makes a compelling case" on grounds of an emergency,
imminent health hazard, homeland security threat, or "some other compelling rationale."
                                         CRS-19

agency's presumption regarding whether earlier peer reviews were sufficient was not
included in the revised bulletin.

      The revised bulletin (like the proposed bulletin) also gave agencies discretion
in determining the appropriate peer review mechanism for any information not
previously reviewed, with the level of rigor required again dependent on the "novelty
and complexity of the science to be reviewed and the benefit and cost implications."
The bulletin stated that "appropriate" reviews could range from examinations by
other federal specialists to formal panels composed of independent experts. The
preamble to the bulletin indicated that some peer reviews could be done by letter
(e.g., when a draft document covers only one discipline), but said panels were
preferable when time and resources permitted their use. Peer reviewers were to be
selected based on their relevant expertise, and were not to have participated in
developing the product being reviewed. The revised bulletin also contained
requirements to handle participation by scientists with conflicts of interest,54 and said
that the comments of peer reviewers or a summary of those comments should be
made available to the public.

     When Peer Review Is Required. In response to concerns that the trigger
for peer reviews was too vague, OMB indicated in the revised bulletin that the
reviews were needed whenever an agency "reasonably determines" that scientific
information "will have or does have a clear and substantial impact on important
public policies or private sector decisions."

     This change did not appear to address one of the underlying concerns voiced in
the comments on the proposed bulletin. The definition of "influential" information
is essentially the same as the one used to define "significant" information in the
proposed bulletin; the only difference is the broader applicability to non-regulatory
information. Therefore, agencies still may find it difficult to determine whether a
particular study will, at some point in the future, have a "clear and substantial impact
on important public policies," much less "important private sector decisions." Two
agencies in analogous situations may reach different conclusions regarding the need
for peer review. As a result, the overall purpose of the guidelines -- a more
consistent application of peer review among federal agencies -- may not be achieved.

Highly Influential Scientific Assessments
     The revised bulletin contained a number of additional peer review requirements
for "scientific assessments" that are "highly influential." These requirements were
much more specific than those placed on "influential" scientific assessments, differed
in some respects from the requirements in the proposed bulletin, and appeared to give
OMB and the agencies significant authority to determine when the requirements are
applicable.



54
   Specifically, the revised bulletin says that the agency (or the entity selecting peer
reviewers) must (1) ensure that reviewers who are federal employees comply with federal
ethics requirements, (2) apply or adapt those requirements to reviewers who are not federal
employees, and (3) consider the National Academy of Sciences' conflict of interest policy.
                                        CRS-20

     When Enhanced Peer Reviews Are Required. The revised bulletin
defined a "scientific assessment" as "an evaluation of a body of scientific or technical
knowledge which typically synthesizes multiple factual inputs, data, models,
assumptions and/or applies best professional judgement to bridge uncertainties in the
available information." The term "highly influential scientific assessment" was
defined in the revised bulletin as "influential" scientific information (as defined in
the previous section of this report) that either the disseminating agency or OMB
determined

     !   met the above-mentioned definition of a "scientific assessment"; and

     !   could have a "clear and substantial impact on important public
         policies ... or private sector decisions with a potential effect of more
         than $500 million in any year," or involved "precedent setting,
         novel, and complex approaches, or significant interagency interest."

      As discussed previously, in the proposed bulletin enhanced peer review
requirements applied to "especially significant" influential information expected to
be disseminated in support of a regulatory action that could have a $100 million
impact in any year. Therefore, although the dollar threshold for when enhanced peer
review is required was higher in the revised bulletin ($500 million instead of $100
million), the peer review requirements were more broadly applicable in that they
applied to more than just regulations. Also, the language regarding "precedent
setting, novel, and complex approaches" replaced criteria in the proposed bulletin in
which OMB alone could decide that certain information required enhanced peer
review because it was of "significant interagency interest" or "relevant to an
Administration policy priority."

     Selection of Reviewers. The revised bulletin described in some detail how
peer reviewers of highly influential scientific assessments should be selected, but still
gave agencies substantial discretion in making the final decision. For example, the
bulletin said that reviewers must be selected primarily on the basis of "necessary"
expertise, experience, and skills, and must be diverse enough to "fairly" represent
different perspectives. The revised bulletin also said that agencies must consider
requesting the public to nominate reviewers, and must generally "bar participation
by scientists employed by the agency sponsoring the review." (As noted previously,
the proposed bulletin said agencies should "strive" to select peer reviewers who were
independent of the agency.) In addition, as in the proposed bulletin, the revised
bulletin said agencies should try to avoid repeated use of the same reviewer on
multiple assignments.

     However, in a significant departure from the proposed bulletin, the revised
bulletin also said that research grants awarded to a scientist through a competitive,
peer-reviewed process did not disqualify that scientist from serving on a peer review
panel. (The proposed bulletin indicated that peer reviewers who were "receiving or
seeking substantial funding from the agency through a contract or research grant"
should not be selected as peer reviewers.) Also, whereas the proposed bulletin
advocated balancing panel members' biases, the revised bulletin contained no
language to that effect. Finally, the revised bulletin said that agencies could
                                       CRS-21

commission entities independent of the agency to select peer reviewers and manage
the process.

      Public Comments. The revised bulletin also gave agencies discretion in the
area of public comments. As noted previously, the proposed peer review bulletin
instructed agencies to permit interested agencies and persons to submit comments on
the information being reviewed, and to provide those comments to the peer reviewers
for their consideration. In contrast, the revised bulletin said that agencies could
decide on their own whether to make the information at issue available for public
comment, but said agencies should "consider" having a public comment period and
public meetings before the reviewers "when there is sufficient public interest." The
revised bulletin did not indicate how agencies are to make this determination, or how
much public interest is "sufficient" to trigger the need for public comments. The
bulletin also said that if an agency decides to make a draft assessment available to the
public, it must, "whenever practicable," provide peer reviewers with a summary or
compilation of the comments received.

      Other Requirements. The revised bulletin contained several other types of
requirements that were also in the proposed bulletin. For example, the revised
bulletin required agencies to provide reviewers with "sufficient information" to allow
them to understand the data, assumptions, and analytic procedures used to support
key findings or conclusions. Also, agencies were required to instruct peer reviewers
to prepare a report summarizing their findings and conclusions. The report must
contain the names, affiliations, and credentials of the reviewers. Agencies were, in
turn, required to prepare a written response to the report, and both the report and the
response must be posted on the agency's website.

      Alternative Procedures. The revised peer review bulletin said that an
agency can (1) rely on scientific information produced by NAS, (2) commission NAS
to peer review the information, or (3) use an alternative procedure "specifically
approved by the [OIRA] Administrator, in consultation with OSTP." As noted
previously, the bulletin permitted the agency head to waive or defer some or all of the
peer review requirements for "influential scientific assessments" and "highly
influential scientific assessments" "where warranted by a compelling rationale."55
If the requirements are waived, however, the bulletin said peer review should be
conducted "as soon as practicable thereafter." The proposed bulletin did not provide
for alternative procedures.

      OMB Authority. As is apparent from the preceding discussion, the revised
peer review bulletin (like the proposed bulletin) gave OMB significant authority to
decide when agencies should use the more rigorous form of peer review.
Specifically, the revised bulletin indicated that these procedures should be used when
OMB (or the agency) determines that a scientific assessment (1) "could have a clear
and substantial impact on important public policies ... or private sector decisions with
a potential effect of more than $500 million in any year," or (2) involves "precedent
setting, novel, and complex approaches, or significant interagency interest."


55
  The proposed bulletin would have permitted OMB (not the agency heads) to waive the
peer review requirements.
                                        CRS-22

Therefore, under this definition, OMB could unilaterally determine that certain
information that an agency intends to disseminate is sufficiently "novel" or
"complex" to trigger these "highly influential" additional peer review requirements.56

     Also, whereas the revised bulletin defined "influential"scientific information as
information that the agency can reasonable determine "will have or does have" a
clear and substantial impact on important public policies or private sector decisions,
the bulletin said that information should be considered "highly influential" if either
the agency or OMB determines it "could" have such an impact -- conceptually, a
lower threshold than for information that is only "influential."

     OMB retained significant authority over the peer review process in other ways.
For example, OMB could determine that an agency's alternative peer review
procedures are sufficiently rigorous to replace the requirements in the revised
bulletin.

Upcoming Peer Reviews
     The proposed bulletin required agencies to notify OMB at least once each year
of any existing or upcoming studies that might trigger the peer review requirements
within the next year, and of the agencies' plans for conducting those peer reviews.
In contrast, the revised bulletin required each covered agency to post an "agenda" on
its website every six months delineating any information disseminations subject to
peer review. The revised bulletin was also much more specific about these notices
than the proposed bulletin, requiring each entry on the agenda to contain a detailed
description of the "peer review plan," including:

     !   the title, subject and purpose of the plan, with an agency contact,

     !   how the review will be conducted (panel or individual letters),

     !   the anticipated number of reviewers, and

     !   a "succinct description of the primary disciplines or types of
         expertise needed in the review."

Each peer review plan was also required to describe how reviewers will be selected,
whether the public will be asked to nominate peer reviewers, whether there will be
opportunities for the public to comment on the information being reviewed, and
whether the agency will provide reviewers with copies of any public comments
before beginning their work. The revised bulletin required agencies to establish a
mechanism to allow the public to comment on the adequacy of the peer review plans,
and must consider those comments.



56
  In the proposed bulletin, OMB had similar authority. The agency could be required to
conduct a formal, independent, external peer review if OMB determined that the information
was of "significant interagency interest or [was] relevant to an Administration policy
priority."
                                        CRS-23

               Comments on the Revised Bulletin
     OMB said that 57 individuals and organizations commented on the April 2004
revised peer review bulletin.57 About a dozen of the comments essentially asked
OMB to allow more time for public comments and interagency review. Most of the
substantive comments provided were supportive of at least some of the changes that
OMB made to the bulletin. However, some commenters believed the changes had
weakened the bulletin to such an extent that they withdrew their initial support, while
others believed the changes had not gone far enough. Some of the individuals' and
organizations' comments are summarized below.

National Academy of Sciences
      The presidents of the National Academy of Sciences and its two affiliated
institutions, the National Academy of Engineering and the Institute of Medicine,
released a statement on April 15, 2004 that generally praised the revision, arguing
that it would "improve the quality of the government's scientific assessments and ...
decision-making," and "better accommodate the diverse circumstances of ... federal
agencies."58 The statement suggested, but did not state, that further improvements
were nevertheless desirable.59

OMB Watch
      In May 2004, OMB Watch filed a comment on the revised peer review bulletin,
stating that although OMB had made "significant revisions" to the bulletin, it
nevertheless had "fail[ed] to correct some of the most fundamental complaints.60
Specifically, OMB Watch said that:

     !   OMB lacked the authority to issue the guidelines, given (1) the lack
         of specific authority to do so in the statutes and orders cited and (2)
         that Congress had not enacted legislation designed to require peer
         review.

     !   OMB had not established that there was a government-wide peer
         review "problem" in need of a government-wide solution.



57
  For OMB's response to the comments provided on the revised peer review bulletin, see
[http://www.whitehouse.gov/omb/inforeg/peer2004/peer_response.pdf]. For a link to each
of the comments, see [http://www.whitehouse.gov/omb/inforeg/peer2004/list_peer2004.
html].
58
  National Academy of Sciences, "Academies' Presidents Comment on OMB Peer Review
Guidelines," Press Release, Apr. 15, 2004, at [http://www4.nationalacademies.org/news.nsf/
isbn/s04152004?OpenDocument].
59
  "...the extended comment period will permit further refinements in the proposed
guidance..." (ibid.).
60
 For a copy of OMB Watch's comments, see [http://www.ombwatch.org/info/dataquality/
CommentsRevisedPR-OMBW.pdf].
                                       CRS-24

     !   OMB retained too much control over federal peer review practices
         (e.g., in determining when certain information requires a more
         rigorous peer review and in approving alternative peer review plans),
         and recommends that some of those responsibilities be given to the
         National Academy of Sciences.

     !   the "independence" standard in the bulletin would diminish the
         ability of government scientists to participate as peer reviewers,
         while not affecting scientists from industry.

     !   the revised bulletin would likely significantly delay the regulatory
         process at agencies by adding "several new layers of bureaucracy"
         (e.g., public comments on peer reviews and peer review plans).

In its conclusion, OMB Watch recommended that OMB withdraw the bulletin and
instead convene an interagency committee to investigate peer review techniques and
flaws, or make changes in the bulletin to address their concerns.

Chamber of Commerce
     The U.S. Chamber of Commerce submitted comments on the revised bulletin
in May 2004, stating that the revised bulletin has "fundamental problems that must
be resolved," and that it could not support the issuance of a final bulletin until those
problems are addressed.61 The Chamber said it had "deep concerns" about (among
other things):

     !   the blanket exemption given to NAS peer reviews;

     !   the "excessive" discretion given to agencies in the implementation
         of the bulletin;

     !   the absence of "provisions allowing affected parties to contest any
         agency determination of applicability, peer review type, panel
         selection, charge, or other peer review program element as it applies
         to a specific case;" and

     !   insufficient limitations on the use of proprietary information and
         models.

In summary, the Chamber said that the changes OMB made in issuing the revised
bulletin "are so severe and debilitating as to eliminate the public benefit of having a
common, government-wide minimum standard for peer review."




61
  For a copy of the Chamber's comments, see
[http://www.uschamber.com/NR/rdonlyres/eznlbdbxz7wylilw7now26n2cqigrnx57hqpav
qkxf7xgvfy2foiy5w5bd37suvonnhln7jv7myxop/COMMENTSProposedRevisedBulletino
nPeerReviewandInfo.pdf].
                                       CRS-25

Members of Congress
     In May 2004, twelve Members of Congress provided OMB with comments on
the revised bulletin, stating that the revision did not address previously expressed
concerns that the proposal was "unjustified, overly broad, burdensome, and did not
appropriately guard against appointment of reviewers with conflicts of interest," or
that it provided OMB with "excessive authority over the production and
dissemination of government information.62 The Members said that the need for the
bulletin remained unjustified, that the exemptions in the revised bulletin resulted in
an unbalanced approach, and that it created "considerable new burdens on agencies."
They also said that "safeguards to ensure the integrity of the process for selection of
reviewers have not been included in the Bulletin." In particular, they said there was
"no requirement that agencies disclose publicly whether any panel members with a
conflict were selected." In conclusion, the Members characterized the bulletin as "a
wolf in sheep's clothing," and said it was "designed to prevent the dissemination of
government information, stifle public debate, and delay legitimate government
regulatory action." They also said that the agencies, not OMB, are "the repository
of scientific and technical expertise," and said they should be given "full authority
to govern their own peer review procedures."

AAAS
     AAAS filed a comment with OMB in June 2004, saying that the "revised
version is much improved" but raising several issues.63 For example, AAAS:

     !   questioned why OMB said scientists seeking an exemption to the
         peer review requirements for information produced by government-
         funded scientists were only "advised" to include a disclaimer rather
         than required to do so;

     !   suggested that agencies be required to make public the criteria for
         determining when outside entities would be commissioned to select
         peer reviewers or manage the peer review process, and how these
         entities would be selected and overseen by the agencies; and

     !   endorsed the idea of an interagency working group on peer review,
         and recommended that the group be required to report to the public
         annually on its deliberations and findings, and to conduct studies and
         hold meetings regarding the impact of the guidelines on science and
         the quality of the information produced.

The editor of AAAS's flagship publication, Science, praised OMB for its efforts to
address the concerns of the scientific community, saying that most of the concerns


62
  For a copy of these Members' comments, see [http://www.whitehouse.gov/omb/inforeg/
peer2004/25.pdf].
63
  For a copy of the AAAS comments, see [http://www.aaas.org/spp/cstc/docs/5-28-04OMB
peerltr.pdf].
                                           CRS-26

regarding the proposed bulletin were answered in the revision, and that OMB
"deserves credit for being a good listener."64

Other Scientific Societies
      A number of other scientific societies also commented on the revised peer
review bulletin. For example, an official in the Association of American Medical
Colleges was quoted in April 2004 as saying that OMB had been "very responsive,"
and said the revised document "is much, much more flexible."65 In July 2004 (after
the comment period for the revised bulletin closed), a consortium of 13 scientific
societies, mostly in the life sciences, released a general statement on the role of
scientific peer review in policy development.66 The statement cited peer reviews as
"critically important tools for policy makers" and laid out a list of eight
"considerations" the societies believe will help ensure that agency peer reviews are
well designed and performed. While several of the points were also made by other
organizations, some provided a somewhat different emphasis. For example, the
societies:

        !   stressed that for scientific peer review to be effective, it "should be
            insulated from politics as much as possible."

        !   said peer review is not a way to get the "right" answer for policy; in
            many instances, even the best data are subject to alternative
            interpretations, and there is no "best" answer.

        !   said that, while avoiding and managing conflicts of interests is
            important, most scientific peer review rests on the presumed
            integrity of the reviewers, stemming from professional standards of
            conduct to which scientists subscribe.

        !   said it is critical to acknowledge the cultural differences among
            scientists, policymakers, and the public in developing and
            implementing an appropriate model for peer review. For example,
            science is inherently uncertain, and scientists must acknowledge that
            in their work, including peer review. Managers, in contrast, often
            are not in a position to embrace that uncertainty but must make
            decisions in the face of it.




64
     Donald Kennedy, "Praise, for a Change," Science, vol. 304 (May 21, 2004), p. 1077.
65
  "White House Softens Disputed Peer-Review Plan," Science, vol. 304 (Apr. 23, 2004),
p. 496.
66
  American College of Preventive Medicine et al., "Scientific Peer Review in Policy
Making," Position Statement, July 15, 2004, [http://www.aibs.org/position-statements/
040715_scientific_peer_.html].
                                         CRS-27

                      Final Peer Review Bulletin
     As noted previously, on December 15, 2004, OMB released its final bulletin on
peer review. OMB said that it had made "minor revisions" to the April version of the
bulletin that were "responsive to the public's comments."67 Perhaps most notably,
OMB said that the final bulletin:

     !   requires that agencies disclose to the public the names of peer
         reviewers for both "influential scientific information" and "highly
         influential scientific assessments." However, the bulletin does not
         require agencies to disclose which reviewers made which comments.

     !   adds an annual reporting requirement to address concerns about the
         lack of an enforcement mechanism, thereby allowing OMB to "track
         how agencies are using the Bulletin, including provisions for
         waivers and exceptions." OMB also said it expects the public to
         monitor agencies' actions under the bulletin.

     !   requires agencies to designate at the time they plan a peer review
         whether it should be considered "influential scientific information"
         or a "highly influential scientific assessment."

     !   provides additional criteria for what constitutes a "highly influential
         scientific assessment." Specifically, the term now includes
         influential assessments that are "novel, controversial, or precedent
         setting or have significant interagency interest." OMB also clarified
         that the $500 million impact test covers all impacts (public or
         private sector) that occur in any year.

     !   broadened the applicability of the exemption for time-sensitive
         health or safety information, noting that it is not restricted to medical
         data from clinical trials that were subject to adequate peer review
         before the start of the trial.

     !   requires agencies to adopt or adapt the National Academy of
         Sciences policy for committee selection with respect to evaluating
         conflicts of interests of peer reviewers who are not federal
         employees.

OMB also reportedly made a number of what appear to be less significant changes
to the final bulletin. For example, OMB said that the final version of the bulletin:

     !   retains a substantial degree of discretion for the agencies, but also
         clarifies the intent of its language concerning the "adequacy of prior



67
  The characterizations in this section of OMB's changes in response to public comments
are from OMB's response to the comments provided on the revised peer review bulletin,
available at [http://www.whitehouse.gov/omb/inforeg/peer2004/peer_response.pdf].
                                       CRS-28

         peer review" and the use of deferrals and waivers to address
         concerns that the revised draft was too discretionary.

     !   encourages agencies to provide public participation opportunities for
         reviews of "highly influential assessments," but avoids a strict
         mandate for public participation and adds language "to stress that
         agencies should avoid open-ended comment periods."

     !   "reinstate[s]" instructions that peer reviewers are to comment on
         scientific and technical questions, and not to provide opinions on the
         policy implications.

     !   provides a "rare exception" to allow a premier government scientist
         from the department or agency that prepared an assessment to
         participate in a peer review (under certain circumstances).


                      Concluding Observations
     OMB's peer review bulletin is likely to have a significant effect on federal
rulemaking and other forms of information dissemination and public policy. That
effect is likely to be both direct (through agencies' and OMB's enforcement of the
bulletin's requirements) and indirect (through references to the bulletin by others).
For example, Section 402 of the Specialty Crops Competitiveness Act of 2004 (P.L.
108-465, signed by the President on December 21, 2004) indicated that a required
peer review of the procedures and standards governing the consideration of certain
import and export requests "shall be consistent with the guidance by the Office of
Management and Budget pertaining to peer review and information quality."

     Some of the initial issues and concerns raised by commenters on the proposed
peer review bulletin were clarified or otherwise addressed in the revised and final
versions of the bulletin. Perhaps most notably, the bulletin now makes it clear that
scientists are not prohibited from serving as peer reviewers if they receive research
grants from the agency based on investigator-initiated, competitive, peer reviewed
proposals. The bulletin also provides exemptions from the peer review requirements
for time-sensitive medical, public health, and safety information, and other
compelling circumstances. However, a number of other issues remain unclear,
including (1) the amount of discretion that agencies will actually have in carrying out
the bulletin's requirements, (2) the degree to which the bulletin will accomplish the
stated goal of making federal peer review practices more consistent, and (3) the
effects of the bulletin on agencies and the federal rulemaking process.

Agency Discretion
      The final peer review bulletin appears to give federal agencies substantial
discretion in determining whether peer review is required for specific products and,
if so, what type of peer review mechanism is appropriate and who should serve as
peer reviewers. For example, the bulletin says agencies need not have peer review
conducted on influential scientific information that had already been subject to
                                      CRS-29

"adequate" peer review. To determine whether a prior review was "adequate,"
agencies are directed to consider (among other things) whether the science is "novel"
or "complex," and whether it is "important" to decision making. (Notably, though,
the final bulletin no longer indicates that an "adequate" peer review "need not
comply with" all of the requirements of the bulletin.) Also, the final bulletin says
that information should be considered "influential scientific information" if the
agency "reasonably can determine" that it will have or does have a "clear and
substantial" impact on "important" public policies or private sector decisions. The
bulletin says that peer reviewers should be selected to provide the necessary
expertise, experience, and skills, and the group of reviewers should be "sufficiently"
broad and diverse to "fairly" represent the "relevant" scientific perspectives and
knowledge.

     On the other hand, the final bulletin also gives OMB substantial discretion in
certain areas. For example, the bulletin indicates that OMB can require agencies to
use the more exacting procedures for "highly influential scientific assessments" if
OMB determines the information is a "scientific assessment" that "could" have a
substantial impact on public policies or private sector decisions with a "potential
impact" of more than $500 million in any year, or is "novel, controversial, or
precedent setting," or has "significant" interagency interest. Also, OMB can
unilaterally approve agencies' use of alternative peer review procedures.

     The amount of discretion that agencies will actually have in carrying out their
peer review programs (or, conversely, the amount of control that OMB will retain)
will be apparent only through the bulletin's implementation. The amount of agency
discretion (or OMB control) could vary substantially from one administration to
another.

Consistency
     OMB indicated that stronger peer review policies were needed because of the
importance of the issue and, citing a 1999 GAO report, because of the "variability in
both the definition and implementation of peer review across agencies." OMB went
on to say that, prior to the development of the bulletin, "there were no government-
wide standards concerning when peer review is required and, if required, what type
of peer review processes are appropriate." Therefore, OMB said that the bulletin
"establishes minimum standards for when peer review is required for scientific
information, and the types of peer review that should be considered by agencies in
different circumstances."

     However, the final bulletin may not provide the desired consistency in peer
review definition or implementation. As indicated above, the bulletin leaves many
key terms undefined or subject to interpretation, and gives the agencies substantial
discretion regarding when certain actions should be taken (e.g., when previous peer
reviews are "adequate") and which reviewers should be selected (i.e., those with the
"necessary" expertise, experience, and skills). To the extent that agencies are
allowed to exercise discretion in these areas, consistency may be forfeited. On the
other hand, strict enforcement of uniform procedures established by OMB carries
with it a different set of concerns about aggregation of power within the Executive
Office of the President, and may be resisted by federal agencies. Also, as noted
                                         CRS-30

previously, although GAO reported that agencies' peer review practices were
inconsistent, it did not recommend greater consistency, and some view variation in
those practices as appropriate and desirable.

Effects on Agencies and Rulemaking
      A number of commenters expressed concerns regarding the effect that adding
new peer review requirements would have on what is already viewed by some
observers as a lengthy, "ossified" federal rulemaking process. Somewhat related
concerns have been voiced regarding the cost of the requirements to federal agencies,
with Senators Lieberman and Durbin suggesting that the requirements "do not pass
OMB's own [cost-benefit] test of good rulemaking." In response, OMB provided
estimates in the preamble to the revised bulletin indicating that it did not believe the
costs would be prohibitive to the agencies. As noted previously, though, in 1997,
OMB indicated that peer review costs could be significant in terms of both time and
agency resources.68 Little empirical data are currently available regarding the cost of
peer reviews, how they affect the federal rulemaking process, or their effect on the
quality of the information being reviewed. It is even less clear how the peer review
requirements suggested by OMB will affect those factors, and there appears to be no
mechanism in place for collecting such data. There are some indications, however,
that the requirements could delay regulatory action.69

     The AEI-Brookings Joint Center for Regulatory Studies recommended that
OMB or some other entity build into the peer review program an evaluation to
determine its effect on the quality of regulatory analyses. Any such evaluation could,
at least conceptually, include an examination of the cost of the new peer review
requirements to federal agencies and its effect on the pace of rulemaking. To
determine the effect of the peer review bulletin on these or other factors, baseline
information regarding the current state of the art would need to be gathered before
the bulletin's implementation.

      Even if the data indicate that peer review adds time to the early stages of the
rulemaking process, that time may be worth the investment if doing so reduces the
likelihood of subsequent judicial challenges to the rules. Peer review may also
provide agencies with a preview of likely objections to a rule during the notice and
comment phase, thereby allowing them to minimize any weaknesses and respond
quickly to adverse comments. All of these factors would have to be considered in
any evaluation of the effect of peer review on the regulatory process.




68
  Statement of Sally Katzen, OIRA Administrator, before the Senate Committee on
Governmental Affairs, Sept. 12, 1997.
69
    Ben Geman, "White House Peer Review Requirements Could Delay Standards,"
Greenwire, volume 10, number 9 (Dec. 17, 2004). The article quoted a program manager
in the Department of Energy as saying that efficiency standards for residential furnaces and
boilers, commercial air conditioners and other equipment were being delayed for two years
because of OMB's peer review bulletin.
                                                                             CRS-31

                            Table 1. Comparison of Proposed, Revised, and Final Peer Review Bulletins

Issue                           Proposed Bulletin                            Revised Bulletin                                Final Bulletin
Target of basic level of peer   "Significant regulatory information,"        "Influential scientific information," defined   Same as revised bulletin.
review                          defined as scientific or technical           as scientific information (including but not
                                regulatory information that the agency       limited to regulatory information) that the
                                can "reasonably determine" will have,        agency "reasonably can determine"will
                                when disseminated, a "clear and              have, when disseminated, "a clear and
                                substantial impact on important public       substantial impact on important public
                                policies or important private sector         policies or private sector decisions."
                                decisions."
Target of more rigorous peer    "Especially significant regulatory           "Highly influential scientific assessments,"    Essentially the same as revised
review                          information," defined as significant         defined as influential scientific information   bulletin. "Highly influential
                                regulatory information (1) disseminated      that either the agency or OMB determines        scientific assessments" defined as
                                in support of an economically                to be "scientific assessments" (e.g., state-    influential scientific information
                                significant regulation (e.g., $100 million   of-science reports, meta-analyses, or risk      that the agency or OMB determine
                                annual impact on the economy); (2) that      assessments) that (1) could have a "clear       (1) could have a "potential impact"
                                could, when disseminated, have a "clear      and substantial impact on important public      (changed from "clear and substantial
                                and substantial impact on important          policies or private sector decisions with a     impact") of more than $500 million
                                public policies or important private         potential effect of more an $500 million in     in any year or (2) are "novel,
                                sector decisions"(those with $100            any year; or (2) involve "precedent setting,    controversial [changed from
                                million impact in any year); or (3) OMB      novel, and complex approaches, or               "complex"]or precedent-setting"
                                determines is of "significant interagency    significant interagency interest."              scientific assessments (changed
                                interest" or "is relevant to an                                                              from "approaches").
                                Administration policy priority."
                                                                           CRS-32

Issue                         Proposed Bulletin                            Revised Bulletin                                 Final Bulletin
Selection of peer reviewers   Peer reviewers "should" not have             Same as proposed bulletin.                       Reviewers "shall" not have
(independence and balance)    participated in the development of the                                                        participated in development of work
                              work product.                                                                                 product.

                              Agency "shall strive to appoint experts      Agency (or entity selecting the reviewers)       Essentially the same as revised, but
                              who ... are independent of the agency,       must "bar participation by scientists            adds exception for the "rare case"
                              [and] do not possess real or perceived       employed by the agency sponsoring the            where the agency determines that a
                              conflicts of interest." One of the factors   review unless the reviewer's service as a        premier government scientist (1) is
                              to consider in determining independence      peer reviewer defines the government             employed in a different agency of a
                              is whether the individual "is currently      employment (i.e., special government             Cabinet-level department, (2) is not
                              receiving or seeking substantial funding     employees)." However, "[r]esearch grants         in a position of management or
                              from the agency through a contract or        that were awarded to scientists based on         policy responsibility, and (3)
                              research grant."                             investigator-initiated, competitive, peer-       possesses "essential expertise that
                                                                           reviewed proposals generally do not raise        cannot be obtained elsewhere."
                                                                           issues as to independence or conflicts."


                              Also consider whether the individual has     No similar provision.                            No similar provision.
                              previously "advocated a position on the
                              specific matter at issue."

                              Also consider whether the individual has     Agencies "should avoid repeated use of the       Agencies "shall" avoid repeated use
                              conducted multiple peer reviews for the      same reviewer on multiple assignments            of the same reviewer unless
                              same agency in recent years.                 unless his or her participation is essential."   participation is essential "and cannot
                                                                                                                            be obtained elsewhere."

                              If obtaining the appropriate expertise       No similar provision.                            No similar provision.
                              requires selecting a reviewer with an
                              apparent bias, agencies should ensure
                              that another reviewer with a contrary
                              bias is selected.
                                                                             CRS-33

Issue                            Proposed Bulletin                           Revised Bulletin                                Final Bulletin
                                 Agencies should avoid repeated use of
                                 the same reviewer on multiple
                                 assignments.

Selection of peer review         Agencies should select an appropriate       Same as proposed bulletin.                      Essentially the same criteria, but
mechanism (significant or        peer review mechanism (e.g., review                                                         also (1) the importance of the
influential information)         within agency or formal review by                                                           information to decisionmakers, (2)
                                 independent body) based on the novelty                                                      the extent of prior peer review, and
                                 and complexity of the science to be                                                         (3) the factors regarding
                                 reviewed, the benefit and cost                                                              transparency (see below).
                                 implications, and any controversy
                                 regarding the science.
Public comments                  Agencies are required to "provide an        Agencies are allowed (but not required) to      "Whenever feasible and
                                 opportunity for other interested agencies   obtain comments on draft scientific             appropriate," agencies are required
                                 and persons to submit comments," and        assessments. If they do so, they must           to make the draft scientific
                                 must "ensure that such comments are         ("whenever practical") provide a summary        assessment available to the public
                                 provided to the peer reviewers with         or compilation of those comments that           for comment when submitted for
                                 ample time for consideration before the     address "significant scientific and technical   peer review and to sponsor a public
                                 peer reviewers conclude their review        issues." When there is "sufficient public       meeting. If they do so, agencies
                                 and prepare their report."                  interest," agencies must "consider"             must ("whenever practical") provide
                                                                             establishing a public comment period on         peer reviewers with access to the
                                                                             the peer reviewers' draft report and            public comments.
                                                                             sponsoring a public meeting.
Alternatives to the bulletin's   No comparable language                      Agencies may (1) rely on scientific             Same as revised bulletin.
procedures                                                                   information produced by NAS, (2)
                                                                             commission NAS to peer review the
                                                                             information, or (3) use an another
                                                                             procedure specifically approved by OMB.
                                                                           CRS-34

Issue                       Proposed Bulletin                              Revised Bulletin                               Final Bulletin
Transparency requirements   The agency shall direct peer reviewers         A detailed summary or copy of the              The agency or entity managing the
                            ("individually or often as a group") to        reviewers' comments, as a group or             peer review shall instruct reviewers
                            issue a final report "detailing the nature     individually, shall be made available to the   to prepare a report describing "the
                            of their review and their findings and         public." Also, the report shall be part of     nature of their review and their
                            conclusions." The report should also           the administrative record for related agency   findings and conclusions." The
                            disclose "the names, organizational            actions "where appropriate."                   report must (1) either include a
                            affiliations, and qualifications of all peer                                                  verbatim copy of each reviewer's
                            reviewers, as well as any current or                                                          comments (with or without
                            previous involvement by a peer reviewer                                                       attribution) or represent the views of
                            with the agency or issue under peer                                                           the group as a whole, and (2)
                            review consideration." The report                                                             disclose the names of the reviewers
                            "should be included in the                                                                    and their organizational affiliations.
                            administrative record for any related                                                         The report must be disseminated on
                            rulemakings."                                                                                 the agency's website, discussed in
                                                                                                                          any related rulemaking, and
                                                                                                                          included in the administrative
                                                                                                                          record.
Upcoming peer reviews       Agencies shall provide OMB at least            Agencies shall post on their websites,         Same as revised, but agencies also
                            once each year (1) a description of any        updated at least every six months, an          required to provide a link from the
                            existing or upcoming studies that might        agenda designating all planned and ongoing     agenda to each document made
                            constitute or support significant              information disseminations subject to peer     public pursuant to the bulletin.
                            regulatory information that the agency         review. For each entry, agencies must          Also, agencies' peer review plans
                            intends to distribute within the next          provide detailed information in a "peer        should indicate whether the
                            year, and (2) the agency's plan for            review plan" (e.g., the subject and purpose    dissemination will be "influential"
                            conducting a peer review of such               of the review, how the review will be          or "highly influential," the timing of
                            studies.                                       conducted, the number of reviewers, how        the review, and whether alternative
                                                                           they will be selected, whether the public      procedures will be employed.
                                                                           will be allowed to comment, and whether
                                                                           reviewers will be provided with those
                                                                           comments). Agencies are also required to
                                                                           allow the public to comment on the plan,
                                                                           and to consider those comments.
                                                                    CRS-35

Issue                   Proposed Bulletin                           Revised Bulletin                               Final Bulletin
Exclusions              Peer reviews are not required for:          Peer review is not required for:               Same as revised bulletin, except:
                        -- studies that have already had an         -- information produced by government-         -- time sensitive "medical"
                             "adequate" peer review (with                funded scientists if not represented as       dissemination not excluded
                             review by a scientific journal              the views of the agency (with                 (only "health or safety");
                             considered adequate, but                    disclaimer);                              -- underlying data and analytical
                             rebuttable);                           -- information related to national                 models used in regulatory
                        -- regulatory information related to             security, foreign affairs, or                 impact analyses or flexibility
                             national defense or foreign affairs;        negotiations involving international          analyses are not excluded; and
                             or                                          trade or treaties;                        -- an exclusion added for
                        -- information disseminated in the          -- information disseminated in the                 analyses of routine statistical
                             course of an adjudication or permit         course of an adjudication or permit           information.
                             application.                                application, unless the agency
                                                                         determines it is novel or precedent
                                                                         setting;
                                                                    -- medical, health, or safety
                                                                         dissemination that is time sensitive or
                                                                         based on clinical trial data that was
                                                                         previously peer reviewed;
                                                                    -- regulatory impact analyses or
                                                                         flexibility analyses subject to
                                                                         interagency review under EO12866;
                                                                         or
                                                                    -- routine statistical information.
Waivers and deferrals   OMB may waive the peer review               The agency head may waive or defer some        Same as revised bulletin.
                        requirements if agency makes a              or all of the peer review requirements for
                        "compelling case" that waiver is needed     "influential" and "highly influential"
                        because of:                                 information/assessments "where warranted
                         -- an emergency,                           by a compelling rationale." If deferred, the
                         -- an imminent health hazard,              review should be conducted "as soon as
                         -- a homeland security threat,             practicable thereafter."
                         -- "some other compelling rationale"
                                     CRS-36

Issue            Proposed Bulletin   Revised Bulletin   Final Bulletin
Annual reports   No provision.       No provision.      By December 15, agencies required
                                                        to provide OMB with a report
                                                        containing (1) the number of peer
                                                        reviews conducted during the fiscal
                                                        year; (2) the number of times
                                                        alternative procedures were
                                                        invoked; (3) the number of times
                                                        waivers or deferrals were invoked;
                                                        (4) any decisions to use exceptions
                                                        in reviewer appointments; (5) the
                                                        number of panels conducted in
                                                        public and allowed public
                                                        comments; (6) the number of public
                                                        comments provided on review
                                                        plans; and (7) the number of
                                                        reviewers recommended by
                                                        professional societies.

------------------------------------------------------------------------------
For other versions of this document, see http://wikileaks.org/wiki/CRS-RL32680