Seasonal climate outlook verification: supporting fresh produce supply

Weather Logistics now has proven its technology through an extended period of experimentation, offering seasonal outlooks that can add value to the operational management of fresh produce. Its goal, to ensure sustainable and affordable fresh produce supply within a volatile future of climate change.

Redefining the benchmark. We are tough when it comes to benchmarking. Our decision was to choose the best representation of current climate, projecting forward to the year 2021, calibrating our outlooks to 50:50 – an even warm versus cool probability balance.

Predictive skill, as we define, arises then from insights about the initial state of the Earth system/ natural climate forcing (e.g., from ocean temperature, sea ice, global circulation, or jet stream dynamics). Our assessment ensures therefore that departures in the seasonal weather conditions from the benchmark can neither be acquired through knowledge about the regional climate, nor through local insights into weather extremes [+]. Anthropogenic radiative forcing from carbon dioxide and other greenhouse gases while essential in our predictions, are therefore not assessed. We then define the triple standard, making best efforts to generate a representative field-scale climate for our benchmark references wherever applicable.

We consider this triple standard as rigorous and time consistent. We also deliver an improvement on categorised seasonal outlooks, through daily and field-scale weather, extending localised weather prediction with an ‘ensemble’ or collection of possible futures.

Caveat. Climate science efforts to determine today’s climate at the field or local level is non-trivial, with only the most technologically advanced growers possessing this knowledge. Our results below therefore indicate a minimum predictive accuracy that is possible right now for growers, producer organisations or decision-support providers. [+] Forecast providers that reference to a 1981 to 2010 climate benchmark can show that most seasons are now warmer-than-average – and we don’t call this call this skilful.

Ground-breaking field-scale climate outlooks.

Weather information is more valuable when delivered on farming scales. Even for Eastern England with no high hills or mountains, its fen lowlands and exposed coastline climates are distinctive. Spain’s production areas are much more geographically complex; featuring temperate coastlines, mountains, and valleys. For this reason, seasonal outlooks on spatial scales of more than a few kilometres can present large uncertainties – simply they are unreliable.

Statistical post-processing of a 100km climate outlook (for example) to represent farming scales is challenging, since regions such as Valencia can often see cooler-than-average conditions while inland Spain experiences warm summers. Similarly, mountainous regions can often experience cold night-time temperatures in the autumn when hot and sunny conditions are more common at lower altitudes. Weather Logistics’ forecast system has pre-processed many billions of weather observations from detailed weather records to ensure that the local knowledge is embedded within its forecasts – recognising that postprocessing often fails in complex climates.

Trained on the past, forecasting for the future.

Retrospective training and verification are a challenge for data-driven models. Overfitting can sometimes produce an analysis that corresponds too closely to the set of observations (in this case the 1995 to 2016 model training years). Our main concern, that while our model works on a regional level over a 21-year test period, it may not predict future observations reliably. To test this assumption, forecast assessments outside the training period (post-2016) were combined with active farm trials that supports our approach with good evidence …

Live field trial – the ‘blind’ experiment.

Weather Logistics undertook a field trial with a large-scale Producer Organisation (PO). We operationally delivered seasonal forecasts for the 2020 growing season, without access to local weather station observations. Issued from April to September, we provided their precision farming team with a rolling seasonal outlook covering the 2 to 15 weeks to end of salad harvest. When the trial was complete, the client analysed the results internally with reference to METOS® Pessl weather stations observations. This field study took place mostly in the Eastern counties of the UK (20 field sites) and results were promising …

Month 1 (forecasting 3 to 6 weeks out) reported that 68% of monthly temperatures were on the correct side of climatology. For forecast month 2 (7 to 11 weeks) this reduced to 54% compared to the same climate benchmark. Historical assessments from 1995 to 2016 indicated an expected skill in specificity of 55% (month 1) and 66.5% (month 2), which on average was broadly consistent. Test #1 passed.

Notes. By combining our seasonal forecast with C3S, we produced a 100-member ensemble prediction. This marginally improved forecast accuracy (+1%), though made our predictions increasingly well-calibrated. This is because C3S is underconfident in its temperature predictions (possible weather realisations too broad). We also found that its 1 to 2-month forecast were not as accurate overall.

Post-training review.

Daily, field-level information

Back in spring 2020, we worked with Agrimetrics to review our winter to early-spring forecasts of cold days [see Arable Farming Magazine, Oct’ 2020]. Providing daily weather forecasts, we assessed whether a farmer should have sown a winter (or spring) crop in 2018 and 2019. Our forecast predicted a milder winter and substantial reduction in days below -2°C for 2019-20, compared to 2018, and a winter crop may have been more favourable that year.

We compared our forecasts against **observations and also against NCEP, ECMWF, UK Met Office and Météo-France public weather providers downscaled to the same locations [see results: https://agrimetrics.co.uk/2020/06/03/seasonal-climate-forecasts-for-agriculture/]

… and the results were consistent with observations. Test #2 passed.

The third test was to generate field-level weather forecasts for the past 3 years, testing the assumption that year-to-year variability in the performance could have led to a ‘lucky’ field trial year for our producer organisation in 2020 …

Retrospective review, for Spain and the UK

For this study we used the same analysis period as the PO trial (April to September) when growers in the UK mostly produce fresh produce. We also generated a 3-year forecast run for farm sites in Spain from 2018-2020. Representative farm sites were then selected. These were located in the Vale of Evesham, West Midlands (see A) and La Hoya, Murcia (see B). Both the UK and Spanish field sites are known for growing fresh produce (salads, exotic vegetables, and horticultural cash crops). We verified our forecasts against observations from **DarkSky® API (by Apple), reported for the same fields and months.

Monthly temperature forecasts again proved positive. For the first forecast month specificity scorings were 69% (Vale of Evesham) and 74% for (La Hoya, Murcia), where n = 21 months in each case. For month 2, forecasts still outperformed climatology: 58% (UK) and 82% (Spain). Results from this experiment are summarised in the table below on the right-hand-side. On the left, a review of the 1995-2016 historical reference assessed on a regional as opposed to a field level basis.

Notable accuracy improvement on Copernicus.

For the experiments undertaken, and in all cases the field-level comparisons run for 2018 to 2020 inclusive, Weather Logistics’ forecast performance outperformed a multi-model average of the ECMWF, UK Met Office, Météo-France and NCEP. Test #3 passed.

Our daily and ‘field-level’ insights also add extra benefit to growers, with no reliable alternative available.

Scalability of our seasonal forecast system to Spain, also proved a success. Test #4 passed.

A. United Kingdom farm: Vale of Evesham, West Midlands

3 to 6-week forecast

*59.5% specificity referenced to C3S, 69% benchmarked to climate

7 to 11-week forecast

*52.5% specificity referenced to C3S, 58% benchmarked to climate

B. Spanish farm: La Hoya, Murcia

3 to 6-weeks (seasonal forecast)

*81% specificity referenced to C3S, 74% benchmarked to climate

7 to 11-week (seasonal forecast)

*80.5% specificity referenced to C3S, 82% benchmarked to climate

*n = 21 months (April to September), 2018 to 2020 inclusive

Future work … Cost-benefit assessment

Weather Logistics published a white paper on the “Economic Benefit of Seasonal Forecasts” in March 2020, and more recently presented a challenge to the European Study Group with Industry (ESGI) 165. Here, a team of academics were briefed with the challenge to assess the value (wholesale price outlook) for fresh produce traded between Spain and the UK. Their paper will be published in Cambridge University Press in June 2021.

Notes.

Weather Logistics’ data contains modified Copernicus Climate Change Service (C3S) information 2021. Neither the European Commission nor ECMWF is responsible for any use that may be made of the Copernicus information or that data that it contains.