Climate-Driven Doubling of Maize Loss Probability in U.S. Crop Insurance

🌽 Introduction: Corn is in trouble.

💰 Rates Explorer

✨ Hyper-Parameters Explorer

It can be helpful to see how the neural network responds to different configurations. However, we can also ask a computer to optimize these metrics through a sweep where it tries different combinations of different parameters to try to find the best one (Joseph 2018). Execute this by clicking / tapping the "Run Sweep" button. For more details, see our scientific paper (preprint). We use that optimized configuration in the rest of this explorable explanation.

More about why the sweep selected this configuration >>

Even with a computer trying hundreds of models, we still have to make some choices:

• Do we favor the model's ability to predict the mean equally with its ability to predict standard deviation?
• After chosing a model configuration, we retrain with all available data so do we want to emphasize a model with the best metrics in the validation set or one that has just slightly worse performance but less overfit so might do better with more examples to learn from?
• Do we want a deeper model which might benefit from more data or a shallower model that seems to perform well in the smaller training dataset used in the sweep?

Sadly there is no one right answer but, in the selection we are using for the rest of this online lab, we prefer the ability to predict mean over std by a ratio in weights of 3 to 1. We also elect for a slightly deeper model in anticipation of retraining with all historic data.

The errors in the visualization reflect a smaller dataset than the one used to train the final model. After retraining with more information, the model sees an MAE of 5% for mean prediction and 2% for std prediction.

More about model performance and uncertainty >>

During the model sweep, we have to divide up our data into three different sets. In this process, we split the data by year. For example, all of the observations from 2011 end up in only one of the sets (Shah 2017):

• The validation set allows us to compare different models together (what is used in the visualization).
• The test set provides a prediction of future performance.
• Only the training set teaches the model while we try to optimize parameters like L2, dropout, etc.

That said, after determining our model preferred model configuration, we can have the model learn from the combined validation and training sets before we make a final estimation of error we expect in the future using the test set. In this final trial, we see an MAE of 5% for mean prediction and 2% for std prediction. We can later use an understanding of this model uncertainty in Monte Carlo.

Finally, despite all of this work, we still need to determine if our model sees similar error across different parts of the US Corn Belt and types of yield changes.

Though not technically a test set, follow up visualizations suggest that:

• Error remains generally stable across geography. The southwest corner has smaller sample size so slightly eleveated error makes sense. These are also areas where results are not typically significant.
• The network appears reluctant to predict highly negative changes to yield. That said, these dramatic drops are overall less common and the error across the entire dataset is symmetric (+3% slightly too optimistic) so this likely does not impact system-wide findings. Furthermore, sampling error during Monte Carlo may address this concern.

All things considered, this model offers sufficient performance for our Monte Carlo.

Anyway, with this neural network built, we run Monte Carlo simulations to see what the future of MPCI and YP might look like while incorporating these measures of neural network uncertainty.

Simulate insurance outcomes >>

Citations for this section

The citations for this section are as follows:

• S. Acharya, “What are RMSE and MAE?” Towards Data Science, May 13, 2021. [Online]. Available: https://towardsdatascience.com/what-are-rmse-and-mae-e405ce230383
• P. Baheti, “The Essential Guide to Neural Network Architectures.” V7Labs, Jul. 08, 2021. [Online]. Available: https://www.v7labs.com/blog/neural-network-architectures-guide
• J. Brownlee, “How Much Training Data is Required for Machine Learning?” Guiding Tech Media, May 23, 2019. [Online]. Available: https://machinelearningmastery.com/much-training-data-required-machine-learning/
• J. Brownlee, “How to Control Neural Network Model Capacity With Nodes and Layers.” Guiding Tech Media, Aug. 25, 2020. [Online]. Available: https://machinelearningmastery.com/how-to-control-neural-network-model-capacity-with-nodes-and-layers/
• B. Haugen, “Geohash Size Variation by Latitude.” Mar. 14, 2020. [Online]. Available: https://bhaugen.com/blog/geohash-sizes/
• R. Joseph, “Grid Search for Model Tuning.” Towards Data Science, Dec. 29, 2018. [Online]. Available: https://towardsdatascience.com/grid-search-for-model-tuning-3319b259367e
• S. Karanam, “Curse of Dimensionality — A ‘Curse’ to Machine Learning.” Towards Data Science, Aug. 10, 2021. [Online]. Available: https://towardsdatascience.com/curse-of-dimensionality-a-curse-to-machine-learning-c122ee33bfeb
• W. Koehrsen, “Overfitting vs. Underfitting: A Complete Example.” Towards Data Science, Jan. 28, 2018. [Online]. Available: https://towardsdatascience.com/overfitting-vs-underfitting-a-complete-example-d05dd7e19765
• D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” 2014. doi: 10.48550/ARXIV.1412.6980.
• I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” in International Conference on Learning Representations, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:53592270
• D. B. Lobell, D. Thau, C. Seifert, E. Engle, and B. Little, “A scalable satellite-based crop yield mapper,” Remote Sensing of Environment, vol. 164, pp. 324–333, Jul. 2015, doi: 10.1016/j.rse.2015.04.021.
• A. Maas, A. Hannun, and A. Ng, “Rectifier Nonlinearities Improve Neural Network Acoustic Models,” in Proceedings of the 30th International Conference on Machine Learning, Atlanta, Georgia: JMLR, 2013.
• NCEI, “Corn Belt.” NOAA, 2024. [Online]. Available: https://www.ncei.noaa.gov/access/monitoring/reference-maps/corn-belt
• G. Niemeyer, “geohash.org is public!” Labix Blog, Feb. 26, 2008. [Online]. Available: https://web.archive.org/web/20080305102941/http://blog.labix.org/2008/02/26/geohashorg-is-public/
• A. Oppermann, “Regularization in Deep Learning — L1, L2, and Dropout.” Towards Data Science, Feb. 19, 2020. [Online]. Available: https://towardsdatascience.com/regularization-in-deep-learning-l1-l2-and-dropout-377e75acc036
• T. Shah, “About Train, Validation and Test Sets in Machine Learning.” Towards Data Science, Dec. 06, 2017. [Online]. Available: https://towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7
• D. Shiffman, The nature of code: simulating natural systems with JavaScript. San Francisco: No Starch Press, 2024.
• N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929–1958, 2014.
• E. Williams, C. Funk, P. Peterson, and C. Tuholske, “High resolution climate change observations and projections for the evaluation of heat-related extremes,” Sci Data, vol. 11, no. 1, p. 261, Mar. 2024, doi: 10.1038/s41597-024-03074-w.

Some definition citations are in glossary.

Data downloads for methods

The data downloads for this methods section ("next, a neural network") are as follows:

• sweep_ag_all.csv: Information about candidates considered in our model sweep.

These are available under the CC-BY-NC License.

🔭 Predicted Distributions Explorer

✋ Claims Visualization

🔬 Neighborhood Explorer

ℹ️ About this website

⚙️ Settings

📖 Glossary

Actual Production History

Actual Production History. For our purposes, this will refer to the average of the last ten years of crop yields (FCIC 2020). This has a number of different roles in insurance but is often used to estimate an expected or typical yield.

APH

Abbreviation for Actual Production History.

Artificial Neural Network

A form of AI modeled after how biological brains work. In this system, there are different neurons connected to each other in a network that "learn" over time. For more informaiton see the Nature of Code (Shiffman 2024).

Blocks

Sometimes certain data can be distracting to the model or simply having a large number of variables can become overwhelming (Karanam 2021). This option lets us take out individual climate variables.

CHC-CMIP6

Climate hazards CMIP6 model which offers both historic climate data / growing conditions as well as future projections (Williams et al 2024).

chirps

Variable name referring to a model for daily precipitation (Williams et al 2024).

County average yield

We use this phrase to refer to the average yield (APH equivalent) for all fields in the county. See Plastina and Edwards (2020).

Counterfactual

A projection of future yield changes which does not assume further warming in contrast to SSP245.

County preimum rate

Based on the crop and county, we use this term to refer to how much it costs to insure one future dollar of expected crop. A value of 4% means 4 cents for each dollar insured. See Plastina and Edwards (2020).

Coverage amount

Up to what percent of APH will be covered. If one sees a yield that is only 75% of APH then the policy will pay out 75% - 70% or 5% of APH. See Plastina and Edwards (2020)

Data Science

This is the type of mathematical and computational science in which we build models like through artifical intelligence to make sense of data and predict the future (IBM).

Depth

The number of layers in the network (Brownlee 2020). Deeper networks can learn more sophisticated lessons from the data but can require more data to train (Brownlee 2019) otherwise it may memorize examples instead of learning broader trends (Brownlee 2020).

Dropout

Randomly turn off neurons while training (Srivastava et al 2014). Conceptually, this encourages finding multiple ways to use inputs in making predictions. This avoids relying on one small aspect of the data (though at the risk of disrupting learning). This "knob" turned up means neurons are disabled with higher frequency.

Geohash

We create a grid through a process called geohashing (Niemeyer 2008) where all of the US Corn Belt gets assigned to a cell. Longer geohashes mean smaller cells and, in the 4 character version, each cell is the size of about 28 by 20 kilometers (Haugen 2020).

Grower

Grower is a more general term we use to refer to farmers and producers regradless of their method of production (Mason 2024).

Insured Unit

A set of fields which are insured together in a policy (Munch 2024). For reference, this is sometimes also called a risk unit.

Insured Unit APH

APH for the set of fields in the insured unit for which a policy is being created. See FCIC (2020).

L2

A penalty on the model for very deep pathways between neurons (Oppermann 2020). Conceptually, this avoids focusing too much on small specifics in the data unless there is strong evidence for that connection. Turning this "knob" up sets a higher bar for the evidence required for those deep connections.

MAE

Abbreviation for Mean Absolute Error.

Maize

In scientific settings, we often refer to the corn crop as a whole regardless of its usage as maize.

Mean

This is simply the average like the average of the yields within the grid cell.

Mean Absolute Error

This is a simple average of the error magnitude. In other words, it indicates, on average, how far off predictions from the network are from actuals regardless of if the model was too high or too low. For example, if the model predicts that a neighborhood's yield will drop by 10% and it actually drops by 15%, the MAE would be 5%. See Acharya (2021).

Monte Carlo

A type of simulation technique that works with variables that have some uncertainty. See Kwiatkowski (2022).

MPCI

Abbreviation for Multi-Peril Crop Insurance.

Multi-Peril Crop Insurance

Covering 494 million acres (Tsiboe and Turner 2023), this is the "oldest and most common form of federal crop insurance" (Chite et al 2005).

Overfit

An overfit model has mistaken noise in the training examples as actual trends so may perform poorly when predicting outcomes for data it has not seen before. See Koehrsen (2018).

Portfolio Effect

In the context of this explorable, portfolio effect emerges where combining more fields together into an insured unit. This offers more opportunities for good outcomes in some fields to offset the negative outcomes in others. Our results find that increasing the size of the portfolio tends to decrease risk. See McClain.

Projected Price

Expected price at which the crop will sell often based on daily futures market data. See Plastina and Edwards (2020)

rhn

Variable name for overall relative humidity (Williams et al 2024).

rhx

Variable name for relative humidity peak (Williams et al 2024).

SCYM

Scalable Crop Yield Mapper (Lobell et al 2015) which is a model that allows us to estimate historic yields.

Significant

Statistically significant means unlikely to have occurred by chance alone. See Hill et al (2018)".

Std

Abbreviation for standard deviation.

Standard Deviation

This is a measure of how wide the range of a distribution is. For example, how variable yields are expected to be in a grid cell. Typically this is where we assume yields approximately follow a normal distribution within a grid cell. See Hargrave (2024).

SSP245

A future projection of climate change expecting moderate warming. See Hausfather and Peters (2020).

Subsidy Rate

What percent of the overall policy cost is covered by subsidy. A value of 55% means that the grower pays for 45% of the overall cost of the policy. See Plastina and Edwards (2020)

svp

Variable name for saturation vapor pressure (Williams et al 2024).

Sweep

Asking the computer to try different combinations of options or hyper-parameters to see how different model configurations perform (Joseph 2018.

tmax

Variable name for maximum temperature (Williams et al 2024).

Test Set

This is a set of data we save until the very end. This lets us estimate how our model will perform in data it hasn't seen before. It helps us set expectations for future performance (Shah 2017).

tmin

Variable name for minimum temperature (Williams et al 2024).

Training Set

This is the set of data / examples from which the model learns. These will change the connections between neurons (Shah 2017).

Underfit

An underfit models fails to capture all of the nuances of the data and can conceptually be thought of as "oversimplified" though the amount of learning may be limited by the data sample available. See Koehrsen (2018).

US Corn Belt

A multi-state area within the United States with substantial corn growing activity (NCEI 2024).

Validation Set

This is a set of data we don't show the model while it learns. It lets us compare competing models if we are experimenting with different options in construting our neural network (Shah 2017).

vpd

Variable name for vapor pressure deficit (Williams et al 2024).

wbgt

Variable name for wet bulb globe temperature (Williams et al 2024).

YP

Yield Protection. This is the coverage option within MPCI that we will simulate. That said, climate effects seen here may impact other options available to growers such as Revenue Proection. Regardless, Yield Protection typically guarantees yield itself up to 75% of an APH (Osiensky 2021).

508h

Process by which new insurance products can be suggested. This could include supplemental policies covering more than 75% of yields (Schnitkey et al 2023) though typically programs cannot cover more than 85% as we will see in a moment.