Introduction

PHOENIX RapidFire users reported that the 'burnt area' output generated by a single run of PHOENIX is sensitive to minor changes in inputs or operating conditions. Fire Prediction Services had Stock Software investigate the following user reported issues:

• Marginal changes in the point-of-origin can result in significant variation in the burnt area generated;

• Different isochrone settings result in different burnt area results;

• Dynamic time stepping is sensitive to weather and isochrones creating different results;

• Identical projects + data can produce different outputs when run on different computers; and

• There are issues with suppression predictions with 'jumping' behaviour.

This technical note summarises the sensitivity testing and includes recommendations to users.

Background

Various users had reported unexpectedly significant variation in the total 'burnt area' produced by PHOENIX simulation runs based on small or marginal changes in initial conditions. Fire Prediction Services requested Stock Software to formally investigate the issues reported.

Stock Software undertook a significant level of sensitivity testing that specifically addressed the user concerns and tested various theories pertaining to the causes of the variations.

Fire Prediction Services reported the findings of the sensitivity testing to the PHOENIX Technical Reference Group (TRG). The TRG issued recommendation to address the issues.

Technical notes

Testing procedure

Testing was conducted on a virtual machine running Microsoft Windows. Approximately 11, 000 simulations were generated using scripts with results automatically summarised. If the burnt area varied significantly visual assessment of the burn footprint was also undertaken. The testing focused on the kind of variations that might be generated by users and addressed theories of the causes of sensitivity such as dynamic time stepping and spotting. Additionally, the testing investigated possible workarounds and solutions.
Several sets of data were tested that included variations in:

• Point of origin in which the point-of-origin varies in location and type (i.e. point vs polygon);

• Isochrones in which outputs of isochrones is varied using different settings;

• Time stepping in which the use of a new static time step feature is tested;

• Input data in which the impact of gridded weather is compared against that of user provided weather;

• Spotting in which spotting is enabled or not;

• Hardware in which a number of different hardware options are tested.

Outcomes of testing

The testing found that:

• All of the sensitivity issues reported by users were able to be reproduced in testing;

• None of the workarounds or solutions investigated were able to mitigate the variations to a level that would be satisfactory to an uneducated user;

• The variation in total burnt area appears to be closely related to the impact of spotting implementation;

• The variation in total burnt area is reflected in how far the fire spreads more so than the shape of the fire; and

• It was found that using the new static time stepping produced no significant improvement over dynamic time stepping.

Conclusion

In summary, because PHOENIX RapidFire is a dynamic simulation, the extent of the fire can be very sensitive to the specified initial conditions. The main cause of this sensitivity is due to how PHOENIX incorporates the spotting process into fire spread. A small change in the ignition time and/or location may have a significant impact on the spotting process and spread and therefore the fire spread.

Determinations

The sensitivity of PHOENIX to initial input conditions is contrary to the expectations of many users but is within the expectations of PHOENIX TRG members, who accept the variation inherent in dynamic simulators. However, the extent to which the variations produced by PHOENIX reflect the chaotic nature of the real world is unknown. Therefore, the TRG have made the following recommendations:

• Any single run of PHOENIX has an unknown amount of inaccuracy. It is highly recommended that agencies and other users run an ensemble of simulations with slightly altered points-of-origin for each fire;

• Each agency or user should choose a setting for isochrones and avoid varying it unnecessarily; and

• The static time step setting introduced in version 4.1 can be used as part of an ensemble run and a value of '1' is recommended.

The TRG will continue to investigate other situations in which users have reported unexpected variation in burnt area such as hardware settings.

Further Reading

For further information refer to PHOENIX Instability Testing (dated April 2019) produced by Stock Software Pty Ltd.