Virtual CommissioningTM (VCxTM) is an innovative energy efficiency program which provides utilities
with a mass market small and medium business (SMB) solution by capitalizing on AMI investment. Want to learn more?
Regression modeling connects energy consumption to independent variables that drive usage (most notably weather and operating schedules), ultimately creating a counterfactual. A counterfactual determines what your consumption would have been had the ECM or change to save energy not happened. You subtract the expected consumption (adjusted baseline) from the actual consumption to calculate the energy saved.
A regression model can be used to calculate whole facility or equipment level energy savings. Weather is the most common variable used in regression modeling. You can either use a forecasted model, where you use weather data in the reporting period to create your adjusted baseline. It essentially says “if we experienced the current day’s weather during the baseline, here’s what we expect consumption would have been”. This approach is common for performance contracting M&V. You can also run a normalized model, where you estimate savings under typical or “normal” operating conditions. A normalized weather dataset is used to provide the average temperatures across multiple years for a specific region or zip code. This model is typically more stable, as it is not affected by unseasonably warm or cold temperatures, and is commonly used in Pay-For-Performance (P4P), monitoring-based commissioning, and Normalized Metered Energy Consumption (NMEC) style programs.
When you run a linear regression M&V model, there are certain statistical thresholds that must be met. Typically, you want a CV(RMSE) less than 0.2, an R2 bigger than 0.75, and uncertainty less than 0.5. Meeting these requirements means you have a “good” M&V model.
CV(RMSE) tells you how well your regression fits the historical or baseline data. It is a measure of how much uncertainty is within your model. So the lower the CV(RMSE), the lower your uncertainty. R2 tells you how much influence your independent variables have on energy consumption. Uncertainty says what percentage of your savings are you certain about. Models with lower uncertainty can claim lower savings.
When creating your baseline model, a failing result from any of the above criteria means you must adjust your model. A failing CV(RMSE) indicates the model doesn’t fit the data very well, and can be improved by adjusting the baseline period, including/excluding your independent variables, or trying a different modeling approach (i.e., forecast vs. normalized model). A failing R2 is less important in determining if your model is “good”, but can be improved upon by including additional variables. However, one must be careful not to “overfit” a model, where you are adding arbitrary variables to have an R2 above 0.75. The BPA’s Regression for M&V: Reference Guide explains why R2 doesn’t necessarily need to be above 0.75 and that M&V professionals must use their judgment when evaluating. Improving uncertainty can be done by running granular M&V models (i.e., hourly or daily vs. a monthly model), ensuring non-routine adjustments have been included if applicable, and trying different modeling techniques.
The example below shows how different modeling approaches in Verify were used to improve the statistical parameters. The standard weather and simple balance point model were not passing CV(RMSE) thresholds. When a time-of-week and temperature model was used, CV(RMSE) passed, R2 increased, and uncertainty decreased.
