January 2014

Volume 23, Number 1

Download EBN digital editions

Smartphone/tablet edition

Kindle edition

Get the PDF

perfect for printing

Article Contents

New Energy Data Is Changing How We Judge Efficiency—And LEED

Printer-friendly versionSend to friend

Seeking lessons from New York City benchmarking data, researchers question everything we thought we knew about energy metrics.

By Nadav Malin


The Helmsley Building at 230 Park Avenue in New York boasts an Energy Star score of 78 and a LEED-EBOM Gold re-certification. But what do these metrics really tell us? Not enough, some scholars contend.

Photo: Tom Bastin. License: CC BY 2.0

In the beginning, there was Energy Star.

Supported by the online Portfolio Manager infrastructure and statistical models from the periodic Commercial Buildings Energy Consumption Survey (CBECS), the U.S. Environmental Protection Agency’s Energy Star has been the envy of the energy policy world; no other country has that kind of benchmarking tool. In fact, it was just licensed to Canada. (Energy Star for Homes and Energy Star labels for appliances and other equipment are mostly unrelated programs.)

Thanks to a flurry of energy benchmarking mandates (see “Energy Reporting: It’s the Law”) in the last three years, however, we’re now starting to get unprecedented flows of data about actual energy use in certain cities (just New York and Washington, D.C., so far, but more are on the way). Suddenly, the CBECS database is looking shabby. It’s out of date, thanks to a temporary funding cut that has since been revoked, and it’s awfully sparse in comparison to the new datasets.

As a benchmark for energy performance of New York City office buildings, the national CBECS database has relatively few reference points, and they tend to represent smaller buildings, according to Constantine Kontokosta, Ph.D., P.E., deputy director of the Center for Urban Science + Progress at New York University (NYU) and associate professor at the Polytechnic Institute of NYU, who did a lot of the data crunching behind the reports that New York City has put out for the last two years based on the new data.

In the forthcoming paper “From Transparency to Transformation: A Market-Specific Methodology for a Commercial Building Energy Performance Rating System,” Kontokosta has proposed an alternative to Energy Star that draws on the newly available benchmarking data, as well as on additional data sources from CoStar and the Department of City Planning, as a framework for benchmarking the energy performance of urban buildings.

Going beyond Energy Star

LEED project performance is just one aspect of Kontokosta’s analysis, which attempts to isolate the impact of a different variables on building energy use. Using a range of building characteristics from these three datasets and applying regression analysis to correct for factors such as building age, height, and asset class, Kontokosta concludes that the source energy use intensity (EUI) of LEED office buildings as a whole is not different in a statistically significant way from that of other office buildings in the dataset. He also finds that Energy Star-labeled buildings have a 10% lower EUI and that buildings that are both LEED-certified and Energy Star-labeled have a 20% lower EUI.

This analysis is based on a relatively small number of buildings, however. Kontokosta cross-referenced a total of 685 office properties across the three databases, of which 44 are LEED certified. Given that small sample size, “we can’t really look at rating system type,” Kontokosta told EBN, “as it won’t produce statistically significant results.”

The majority of those 44 buildings are certified under LEED for Existing Buildings: Operations & Maintenance (EBOM), and most of the rest under LEED for Core & Shell (CS), both of which have limitations in their ability to affect overall building energy use. LEED-EBOM projects don’t typically entail major system upgrades, and LEED-CS projects have a limited ability to affect tenant energy use. As a result, while this assessment is likely a valid report on the relative energy performance of LEED buildings as a subset of large Manhattan office buildings, it doesn’t tell us anything about how buildings that are designed and built entirely to LEED for New Construction standards actually perform.

Stopping short of Energy Star

Oberlin College physics professor John Scofield, Ph.D., takes a similarly dim view of Energy Star in his analysis of the NYC benchmarking data, published in the December 2013 edition of Energy and Buildings. Rather than coming up with an alternate method like Kontokosta, however, Scofield simply dismisses the high Energy Star scores achieved by the 21 LEED buildings he identifies in the dataset.

Despite the fact that Energy Star at least attempts to factor in occupant density and operating hours, Scofield reverts to basic source EUI numbers to conclude that LEED buildings as a whole are not more efficient than the overall collection of buildings. More specifically, he finds that the most basic LEED Certified-level buildings are actually less efficient, while the Gold-level buildings are more efficient.

The buildings included in this analysis suffer from the same rating system limitations as Kontokosta’s, in that they are exclusively LEED-EBOM and LEED-CS certified. Again in contrast with Kontokosta, who reins in his conclusions, Scofield uses his findings—as he has in the past—to argue that LEED fails to deliver energy savings. One of his key assumptions is that each of the LEED buildings has “undergone extensive renovation in the last few years” and therefore should be compared to other recently renovated buildings. LEED-EBOM practitioner Jenny Carney of YR&G disputes that assumption, however. “Most commercial office buildings undergo piecemeal upgrades on a running basis; extensive renovations are more rare and are not often a precursor to a LEED project, in my experience,” she reports.

Rethinking efficiency

Energy-efficiency advocates would like us to believe that, as buildings get more efficient and their energy cost goes down, their value increases. That may be the case, but you won’t find any obvious support for that theory here, where the highest-value properties among New York City’s large buildings also tend to use the most energy.

There are a host of possible reasons for this trend: The most valuable properties tend to be tall buildings with high-tech tenants on corner lots. Kontokosta’s regression model reveals that buildings that are freestanding or on corner lots use 9% more energy than those in the middle of a block; that each floor added on a building increases source EUI by 1%; that newer buildings use more energy than old ones; and that a 10% increase in floor area devoted to data centers increases source EUI by 34%.

These confounding factors beg a larger question, suggests the U.S. Green Building Council’s vice president for research, Chris Pyke, Ph.D.: If efficiency is a measure of how much work output we get for our energy input, “what defines efficiency in buildings anyway?” Our conventional metric of energy used per square foot of floor area makes sense if you’re just looking at heating, cooling, and lighting energy, but in many commercial buildings today, other loads dominate the picture.

Btu per dollar

In search of a more relevant metric, akin to the way the energy efficiency of our economy is measured in Btus per dollar of gross domestic product (GDP), researchers from Buro Happold and Happold Consulting mashed up the New York City energy benchmarking data with the CoStar Tenant Database and classified building tenants according to their likely economic contribution based on their Standard Industry Classification (SIC) code.

In work that they published as part of the Greenbuild Research 2013 compilation, the authors defined the “Building Economic Efficiency Coefficient” for 811 properties as the ratio of source EUI to economic intensity. They found a “loose correlation” between energy consumed and presumed economic contribution.

Applying their economic efficiency coefficient to 28 LEED-NC and LEED-CS-certified buildings in the benchmarking data set, they found a suggestion that LEED buildings have higher economic efficiency than non-LEED buildings and that their efficiency increases with the level of certification (although they note the sample set for this last finding is too small to be statistically relevant).

This work would have been much more compelling if they had actual economic output from the relevant buildings rather than presumed output based on the tenants’ SIC codes. The CoStar database actually contains that level of data in the form of tenant staff costs, but the researchers didn’t have access to those details, according to Pyke.

Many numbers, few conclusions

While more data is streaming in, the unfortunate reality is that we still have little insight into the market-wide impact of green building programs on building energy use. Perhaps more information and analysis will eventually help answer that question, though it’s worth noting that LEED certification addresses much more than energy. There are even aspects of LEED, such as increased ventilation, “that might result in additional energy consumption,” Kontokosta notes. And while recent updates to LEED are likely to drive down energy use in future LEED buildings, it will take a number of years before those impacts are seen in the benchmarking reports.

The LEED conversation aside, the fire hose of new information is starting to help us understand energy-use patterns in urban buildings more generally. Kontokosta is now working on analyzing energy patterns in multifamily buildings, for example, including the affordable housing sector, where residents are especially vulnerable to high energy costs.

More generally, the slight improvement in year-over-year performance in New York (see “NYC Buildings Gain Three Energy Star Points in Year Two”) suggests that the window into energy use could, in itself, have an impact on owner decisions and tenant behavior.


Comments (3)

1 relevance of building ENERGY posted by Ellen Broadwell on 12/24/2013 at 08:29 am

I have three comments regarding ENERGY STAR scores for NYC buildings. The first is to note that, at the end of the day, these scores are irrelevant. The end goal for the NYC benchmarking program is to lower GHG emission and consumption of primary energy. If ENERGY STAR scores are useful in achieving this goal -- great. But there is no solace to be found in achieving high ENERGY STAR scores while energy consumption and GHG emission is not reduced. Global climate change will not be mitigated by high ENERGY STAR scores; mitigation requires lowering GHG concentrations. It is like a football game -- the goal is to win the game (determined by final score) -- not to achieve more rushing yards. The first stat, in the end, is all that matters.

The second comment is to realize there are two ways to raise building ENERGY STAR sores. One is to lower a building's source energy consumption. This is hard to do. The second is to increase your "predicted" energy consumption by inflating certain operating parameters -- number of building hours, number of employees, etc. Consultants who file the NYC benchmarking data, have figured out how to game the system and raise scores for their clients without saving energy. No doubt NYC's high ENERGY STAR scores are, in part, due to such gaming.

And third, what evidence has the EPA ever offered to demonstrate the validity of the ENERGY STAR scoring system? There work has never been subjected to peer review. In 2007 the EPA revised their Office building model, overnight changing a building's score by 20 points or more without explanation. My own research has discovered that a building's ENERGY STAR score is generally uncertain by something like 30-40 points! That means there is statistically no significant ifference between a score of 60 and 80. The EPA is propagating a myth that there score provides meaningful comparison of a building's energy performance to similar buildings in the US building stock. It does not.

John Scofieldhttp://thepragmaticsteward.com/

2 Noisy Metrics posted by Stephen Tilly on 12/30/2013 at 11:06 am

To borrow Nate Silver's (drawn from Claude Shannon's)terms, there seems to be a lot of noise and sparse signal in building energy data, and little accounting for margins of error. There is also uncertainty about which metrics are meaningful. Is the best metric primary energy/GHG per square foot; or should we find a way to measure how "useful" a building is? For example, should not a vibrant, renovated mixed-use building that houses many uses 24/7 be preferred to a new luxury condo or office tower with generous space allotments per user that may have a slightly lower EUI on a square foot basis but a much lower human "Use Intensity"?

A related problem with Life Cycle Analyses is that our GHG use needs to be addressed in the short term to bend the curve, but the time periods normally addressed in LCA projections of the carbon footprints of buildings are 60 or 70 years, well beyond and perhaps not representative of the short term impacts.

Steve Tilly

3 What Matters posted by Marcus Sheffer on 12/30/2013 at 11:46 am

Each energy metric and comparative system has its strengths and weaknesses and none of them are perfect. In the grand scheme of things the only energy efficiency metric that matters is - are we using less over time? In aggregate whether we are using less energy due to occupancy, building type, a lighting retrofit, or as a result of dozens of other potential variables is largely irrelevant beyond the lessons we can learn from the examination of the data and apply to future reductions.

So pick your energy metric, track it over time and observe the trend. This is the only way we will see the desired effect - which is the whole point behind energy data disclosure.

If LEED is to fulfill its mission of market transformation EBOM must be about continuous improvement for all existing buildings. As such it should adopt a significant improvement option for energy efficiency in recognition that the only metric or score that matters is the trend over time.

Post new comment

Welcome !
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Glossary terms will be automatically marked with links to their descriptions. If there are certain phrases or sections of text that should be excluded from glossary marking and linking, use the special markup, [no-glossary] ... [/no-glossary]. Additionally, these HTML elements will not be scanned: a, abbr, acronym, code, pre.

More information about formatting options

December 30, 2013