An external scientific review by independent experts is a common and valuable practice, particularly when the models have significant management implications. This ethos is why peer review is foundational to science in general. By providing an objective assessment, external reviewers can help ensure the models are robust and appropriate for the management decisions they’re being applied to. For example, the Chesapeake Bay Program’s suite of modeling tools has undergone several expert reviews over the last 30 years. In 2023, the Southern California Coastal Water Research Project Authority similarly launched an independent expert panel to review their coastal eutrophication numerical modeling.

We are navigating complex decisions

Our region is navigating complex decisions on how best to manage nitrogen to maintain healthy habitats. Excess nitrogen from human activities can potentially increase harmful algal blooms, decrease dissolved oxygen, compound ocean acidification, and cause other changes that may harm marine life. Recent regulations are particularly focused on the impacts of excess nitrogen on low dissolved oxygen in Puget Sound. While low dissolved oxygen occurs naturally, excess nitrogen from human activities may change the timing, location, and size of algae blooms and by extension marine life’s exposure to low dissolved oxygen.

Modeling is driving nutrient management in Puget Sound 

The Salish Sea Model was developed to help inform these complex decisions. Washington State uses monitoring data and the Salish Sea Model to determine compliance with Washington’s dissolved oxygen water quality standard and to establish each 303(d) listing. The state also uses the model to explore the effectiveness of potential nutrient reductions and to establish targets for load reductions at wastewater treatment plants. Regulation using the model may result in billion-dollar wastewater treatment plant upgrades. The nutrient management decisions we make now could shape the future of wastewater treatment, water quality, and our communities for decades to come. Consequently, there is heightened interest in assessing the Salish Sea Model’s performance, particularly the regulatory application.

Regulatory Application: Determining Non-Compliance
Since parts of Puget Sound naturally have low dissolved oxygen, it’s important to determine the decrease in dissolved oxygen due to excess nutrients from human activities, specifically. The Department of Ecology uses monitoring data and the Salish Sea Model to determine compliance with Washington’s dissolved oxygen water quality standard and to establish each 303(d) listing. The Salish Sea Model simulates both existing conditions and natural conditions (i.e., an approximation of conditions before western settlement). A location in Puget Sound is predicted to be non-compliant if any of the 10 depth-layers:
1. Fall below the numeric criteria (e.g., 7 mg/L) or natural conditions at that location, whichever is lower
AND
2. The existing condition is at least 0.2 mg/L or 10% lower than the natural conditions, whichever change is smaller.

If a single layer at any location is non-compliant for at least an hour, the day is considered non-compliant.

*Ecology adopted revisions to the natural conditions provision and shared updated Performance-Based Approach guidance for public comment by May 22, 2025.

Learning from global experts  

When addressing complex environmental challenges, valuable insights can be gained from the extensive experience of scientists in other regions like the Chesapeake Bay and the Baltic, where models have been used to manage nutrients for decades. In 2024, the University of Washington Puget Sound Institute convened global experts to advise on how to improve the application of the Salish Sea Model to inform recovery goals and nutrient management decisions in Puget Sound. We were fortunate to benefit from the expertise of scientists who have led cutting-edge research and advised regional managers on the application of modeling and monitoring in nutrient management programs in other regions. The Model Evaluation Group included:

• Bill Dennison: Vice President for Science Application and Professor at University of Maryland Center for Environmental Science
• Jacob Carstensen: Professor at Aarhus University, Denmark
• Jeremy Testa: Associate Professor at University of Maryland Center for Environmental Science
• Kevin Farley: Professor and the Blasland, Bouck and Lee Faculty Chair at Manhattan College
• Peter Vanrolleghem: Professor at Université Laval, Canada

Expert findings

In analyzing the existing literature and available data on the Salish Sea Model at the time, the experts found:

• On average, Salish Sea-wide model simulations have comparable performance to other models used to set water quality standards and nutrient discharge limits elsewhere in the USA. Evaluating the average model skill for the entire region can miss important local nuances in specific areas where non-compliance is predicted. However, while other states use models to set water quality standards and nutrient discharge limits, to our knowledge, they only use monitoring data to assess compliance with nutrient and dissolved oxygen water quality standards.
• Dissolved oxygen non-compliance in Puget Sound occurs primarily in portions of Hood Canal and 16 shallow embayments. Within these non-compliant areas, the model error is greater. For example, the 2014 modeled results for current conditions ranged from 1.04 – 3.05 mg/L DO RMSE across 22 sites in these embayments. These RMSE results are approximately an order of magnitude greater than the 0.2 mg/L DO human allowance that is used to determine regulatory compliance. This highlights the value of looking at model performance analysis specifically in the places and at the times where model outputs are used in regulatory decision-making.
• The regulatory determination of non-compliance was found to be quite sensitive to the natural conditions threshold defined by the state’s water quality standards. For example, in 2014, 58% of the non-compliant area was at most 0.1 mg/L above the 0.2 mg/L human allowance threshold.

Expert recommendations

In 2024, the Model Evaluation Group recommended the following model analysis to increase confidence in the regulatory application and strengthen a process-based approach to understanding water quality drivers of change:

• Focus additional validation studies on shallow embayments and portions of Hood Canal where human activities may further reduce low dissolved oxygen levels. Preliminary analysis suggests there is larger model error in these areas. Long-term, increased monitoring in these areas could also support refined modeling of these shallow embayments.  
• Perform validation studies using sub-sets of data above/below the pycnocline to better understand the importance of and model skill related to processes influencing dissolved oxygen, such as vertical mixing, stratification, phytoplankton growth, and water-sediment interactions.
• Analyze model performance for non-calibration years and across multiple years to characterize model skill beyond the three existing, single-year runs. Consider performing additional sensitivity scenarios for model years that are at opposite ends of the spectrum of interannual variability.
• Use new monitoring data to analyze model performance for sediment oxygen demand and to validate related processes like carbon fluxes and denitrification.
• Characterize the potential propagation of error associated with uncertainties in model parametrization, loadings, etc., and how this may influence confidence in the model-to-model comparison used when determining regulatory compliance.