Verification of Complex Models

Our start-up delivers a unique set of weather forecast insights.

We improve existing weather prediction models using observations and a sophisticated software algorithm.

So what is an algorithm?

In simple terms, an algorithm is a set of rules or calculations designed to solve a problem. Our long-term forecasts deliver risk advisories for growers, which enables them to plan further ahead to mitigate the impacts of local variability in weather.

How are complex models composed?

When starting something big, we need to generalise. Complexity arises naturally from the combination of many small and logically connected steps. We didn’t wake up one day with the entire forecast model – a software developer will spend long hours in a process of iterative improvement. We first created a simple framework to describe the general processes of the system – and then by digging deeper into the components of that system we identify new routes for improvement. Einstein reportedly said that: “Everything should be made as simple as possible, but not simpler.” By ensuring each step is first rigorously tested and valid, we then expand our model outward into new avenues.

To build a housing complex, architects first create a picture of how their construction fits into the landscape. The houses form the blocks, which are fitted together to make the complex. The fixtures and fittings are the last step.

Model development requires nothing more than step-wise thinking, ordering each process involved. Without breaking things down into simple steps computer code gets messy and chaotic. Creating too much detail or complexity in a model before the framework of the model is rigorously tested will at some point cause breakage.

What is the process for complex model verification?

This is kind of a trick question. Complex models are no different from any other, but require more detailed thought about what features are most valuable to a customer.

Farmers for instance require field-scale weather information – so can we get access to direct observations in the field to compare our weather predictions against? A standardised weather station recording would be invaluable here.

Growers of carrots need to know the timing of the first significant frost. To tackle this verification problem we need to obtain ground temperature data, and our knowledge suggests that carrots grow well in sandy soil textures that expose them more to nighttime lows. Weather stations are housed in a Stevenson Screen with thermometers 1.25 metres above ground level. To optimise an air frost model, weather station data is not suitable for these growers. Soil sensor probes are a much better option.

Some farmers begin drilling early, exploiting the milder coastlines for reliable crop emergence. So what happens if we examine local weather for farms near the coastlines? Does this tell how to modify our model for future forecasts?

Verification is an ongoing process. If we conduct the right investigations, centred around critical customer requirements, this naturally steers product development to a viable solution much quicker.