The paper, “Performance Measures for Validation of Oil Spill Dispersion Models Based on Satellite and Coastal Data”, is available in the Journal of Oceanic Engineering published by the IEEE (the Institute of Electrical and Electronics Engineers). It was authored by Chris Dearden of STFC, and Tim Culmer and Richard Brooke of Riskaware.

When looking to manage the sensitive and substantial issue of oil spills at sea, it is important to know that the source of information used for decision making is credible. Oil spill models are valuable assets in preparing for and responding to incidents – but only when they are accurate.

As part of our ongoing work to understand and improve the accuracy of our MarineAware oil spill dispersion model, we teamed up with the STFC (Science and Technology Facilities Council) Hartree Centre in Warrington, UK. The goal was to develop a set of metrics that could objectively assess a model’s ability to predict the trajectory, spread and coastal impact of an oil spill.

A literature review conducted by STFC found that very little on consistent validation of oil spill models has been published, particularly for commercial models. To help drive improvement in the field, we wanted to find a way to objectively compare the performance of different models across a range of different spills.

Building on proven metrics from other domains with their own innovative new approaches, STFC were able to assess both deterministic and stochastic modelling results, using ground truth data from satellite imagery and coastal reports.

Together, we are keen to encourage the use of the metrics more widely and so as well as making the paper open access, we have open sourced the Python library that STFC developed and used to generate the outputs for the paper.

We think that the industry could draw many benefits from a common approach to validation. Routine quantitative assessment of the models used during a response could lead to better understanding of their strengths and weaknesses, and drive significant improvements in their accuracy.

IEEE is the world’s largest technical professional organisation advancing technology for humanity and it is a great privilege to be published in their Journal of Oceanic Engineering. We would also like to thank Innovate UK who part funded the research for this paper through their A4I (Analysis 4 Innovation) programme.

You can access the paper, Python library, and interviews with the authors at the links below:

• Published paper (open access): https://ieeexplore.ieee.org/document/9543654 
• Python library for applying the metrics, known as OMEN (Oil Model EvaluatioN): https://github.com/riskaware-ltd/omen 
• Video interview for the A4I Programme with the paper’s authors: https://www.a4i.info/a4i_case_studies/riskaware