What Baseball Can Teach Policymakers

With the Major League Baseball season having just begun, I’m reminded of the truism that the best teams win their divisions in the regular season, but the hot teams win in the post-season playoffs.  Why the difference?  The regular season is 162 games long, but the post-season consists of just a few brief 5-game and 7-game series.  And because of the huge random element that pervades the sport, in a single game (or a short series), the best teams often lose, and the worst teams often win.

The numbers are striking, and bear repeating.  In a typical year, the best teams lose 40 percent of their games, and the worst teams win 40 percent of theirs.  In the extreme, one of the best Major League Baseball teams ever ­- the 1927 New York Yankees – lost 29 percent of their games; and one of the worst teams in history – the 1962 New York Mets – won 25 percent of theirs.  On any given day, anything can happen.  Uncertainty is a fundamental part of the game, and any analysis that fails to recognize this is not only incomplete, but fundamentally flawed.

The same is true of analyses of environmental policies.  Uncertainty is an absolutely fundamental aspect of environmental problems and the policies that are employed to address those problems.  Any analysis that fails to recognize this runs the risk not only of being incomplete, but misleading as well.  Judson Jaffe, formerly at Analysis Group, and I documented this in a study published in Regulation and Governance.

To estimate proposed regulations’ benefits and costs, analysts frequently rely on inputs that are uncertain —  sometimes substantially so.  Such uncertainties in underlying inputs are propagated through analyses, leading to uncertainty in ultimate benefit and cost estimates, which constitute the core of a Regulatory Impact Analysis (RIA), required by Presidential Executive Order for all “economically significant” proposed Federal regulations.

Despite this uncertainty, the most prominently displayed results in RIAs are typically single, apparently precise point estimates of benefits, costs, and net benefits (benefits minus costs), masking uncertainties inherent in their calculation and possibly obscuring tradeoffs among competing policy options.  Historically, efforts to address uncertainty in RIAs have been very limited, but guidance set forth in the U.S. Office of Management and Budget’s (OMB) Circular A‑4 on Regulatory Analysis has the potential to enhance the information provided in RIAs regarding uncertainty in benefit and cost estimates.  Circular A‑4 requires the development of a formal quantitative assessment of uncertainty regarding a regulation’s economic impact if either annual benefits or costs are expected to reach $1 billion.

Over the years, formal quantitative uncertainty assessments — known as Monte Carlo analyses — have become common in a variety of fields, including engineering, finance, and a number of scientific disciplines, as well as in “sabermetrics” (quantitative, especially statistical analysis of professional baseball), but rarely have such methods been employed in RIAs.

The first step in a Monte Carlo analysis involves the development of probability distributions of uncertain inputs to an analysis.  These probability distributions reflect the implications of uncertainty regarding an input for the range of its possible values and the likelihood that each value is the true value.  Once probability distributions of inputs to a benefit‑cost analysis are established, a Monte Carlo analysis is used to simulate the probability distribution of the regulation’s net benefits by carrying out the calculation of benefits and costs thousands, or even millions, of times.  With each iteration of the calculations, new values are randomly drawn from each input’s probability distribution and used in the benefit and/or cost calculations.  Over the course of these iterations, the frequency with which any given value is drawn for a particular input is governed by that input’s probability distribution.  Importantly, any correlations among individual items in the benefit and cost calculations are taken into account.  The resulting set of net benefit estimates characterizes the complete probability distribution of net benefits.

Uncertainty is inevitable in estimates of environmental regulations’ economic impacts, and assessments of the extent and nature of such uncertainty provides important information for policymakers evaluating proposed regulations.  Such information offers a context for interpreting benefit and cost estimates, and can lead to point estimates of regulations= benefits and costs that differ from what would be produced by purely deterministic analyses (that ignore uncertainty).  In addition, these assessments can help establish priorities for research.

Due to the complexity of interactions among uncertainties in inputs to RIAs, an accurate assessment of uncertainty can be gained only through the use of formal quantitative methods, such as Monte Carlo analysis.  Although these methods can offer significant insights, they require only limited additional effort relative to that already expended on RIAs.  Much of the data required for these analyses are already obtained by EPA in their preparation of RIAs; and widely available software allows the execution of Monte Carlo analysis in common spreadsheet programs on a desktop computer.  In a specific application in the Regulation and Governance study, Jaffe and I demonstrate the use and advantages of employing formal quantitative analysis of uncertainty in a review of EPA’s 2004 RIA for its Nonroad Diesel Rule.

Formal quantitative assessments of uncertainty can mark a truly significant step forward in enhancing regulatory analysis under Presidential Executive Orders.  They have the potential to improve substantially our understanding of the impact of environmental regulations, and thereby to lead to more informed policymaking.


Opportunity for a Defining Moment

The inauguration of Barack Obama as the forty-fourth President of the United States is a defining moment in American history. For most Americans and countless others around the world, this is an inspiring political transition. The question we must face, however, is whether compelling inspiration will lead to effective action. As I wrote in a Boston Globe op-ed (November 12, 2008) one week after election day, environment and energy issues — particularly climate change policy — provide a microcosm of the forces that are shaping and will shape the actions of the new Administration and Congress.

About eight years ago, President-Elect George W. Bush promised to be President for all the people, not just those who had voted him into office. Bush’s ability as Texas Governor to bridge differences across the political aisle provided cause for optimism.

But hope for a centrist and sensible Presidency dissolved under the influence of White House political operative Karl Rove and Vice President Dick Cheney. The Bush Administration moved not to the center, but toward solidifying its base on the political right. Nowhere was this more apparent than in energy and environmental policy, with Vice President Cheney running energy policy, and EPA Administrator Christie Whitman virtually driven from office.

Will the environment and energy team of President Obama respond effectively to the serious challenges that lie ahead? Or will we find that the corporate lobbyists who filled so many key environmental positions in the Bush Administration have simply been replaced by strident advocates from the other end of the political spectrum? In other words, will ideology trump reason?

The first sign of trouble will be if the Administration issues an “endangerment finding” for carbon dioxide, as promised by the Obama campaign, thereby pleasing and solidifying President Obama’s political base, but also playing into the hands of those who oppose climate policy action, tying up progress with litigation, driving up costs, and accomplishing little or nothing.

Ultimately, will the Obama White House work with Congress to develop climate strategies that are scientifically sound, economically sensible, and thereby politically pragmatic? Will the new President –with impressive Democratic majorities in both houses of Congress — take on the difficult task of crafting meaningful climate legislation?

The only politically feasible approach that can make a real dent in the problem is a comprehensive, upstream cap-and-trade system to reduce carbon dioxide emissions 50 to 80 percent below 1990 levels by 2050. The declining cap will increase the cost of polluting, thereby discouraging the use of the most carbon-intensive fossil fuels and providing powerful incentives for energy conservation and technology innovation.

The system could start with a 50-50 split of auctioned and free allowances, gradually moving to 100% auction over 25 years. To establish political support in the short term, free allowances should be targeted to sectors that are most burdened by the policy. And the auction revenue — which will increase over time — can be used to compensate low-income consumers, finance research and development, reduce the federal deficit, or cut taxes.

The best option may be to make the program revenue-neutral by returning all of the auction revenue to citizens through direct cash dividends or annual tax credits. This can go a long way towards making the legislation palatable to Republicans and Democrats alike who are reticent to take any actions that even resemble a tax increase.

By making the overall emissions cap gradually become more stringent over time, costs can be greatly reduced by avoiding premature retirement of existing capital stock, reducing vulnerability to siting bottlenecks, and ensuring that long-lived capital investments incorporate appropriate advanced technology.

Still, the costs of meaningful action will be significant, with impacts on gross domestic product eventually reaching up to 1 percent per year. But the longer the world waits to begin taking serious action, the more ambitious will emission reduction targets inevitably become, as atmospheric greenhouse gases continue to accumulate.

The bottom line is that getting serious about global climate change will not be cheap and it will not be easy. Beware of claims to the contrary. In the midst of a significant economic downturn, with businesses closing and unemployment on the rise, it makes sense for the new Administration to give its greatest attention to economic recovery. There is nothing wrong with sequencing policies. But if current predictions about the consequences of another few decades of inaction are correct, this defining moment provides an important opportunity for serious and sensible action.