evaluation of integrated assessment model hindcast experiments: a case study of the gcam 3.0 land use module
Clicks: 147
ID: 189931
2017
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This
article has not been analysed, so there is no overall score —
reader engagement is measured and shown alongside.
Reader Engagement
Emerging Content
30.0
/100
147 views
33 readers
AI Quality Assessment
Not analyzed
Readership in this journal
EmergingRanked #41 of 133 articles by views in international journal of quantum chemistry
Most read
Least read
Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 133 in total.
Mint this article as an NFT
Not yet mintedCreate a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.
5
SUSD
one-off · no wallet required
Abstract
Hindcasting experiments (conducting a model forecast for a time period in
which observational data are available) are being undertaken increasingly
often by the integrated assessment model (IAM) community, across many scales
of models. When they are undertaken, the results are often evaluated using
global aggregates or otherwise highly aggregated skill scores that mask
deficiencies. We select a set of deviation-based measures that can be applied
on different spatial scales (regional versus global) to make evaluating the
large number of variable–region combinations in IAMs more tractable. We also
identify performance benchmarks for these measures, based on the statistics
of the observational dataset, that allow a model to be evaluated in absolute
terms rather than relative to the performance of other models at similar
tasks. An ideal evaluation method for hindcast experiments in IAMs would
feature both absolute measures for evaluation of a single experiment for a
single model and relative measures to compare the results of multiple
experiments for a single model or the same experiment repeated across
multiple models, such as in community intercomparison studies. The
performance benchmarks highlight the use of this scheme for model evaluation
in absolute terms, providing information about the reasons a model may
perform poorly on a given measure and therefore identifying opportunities for
improvement. To demonstrate the use of and types of results possible with the
evaluation method, the measures are applied to the results of a past hindcast
experiment focusing on land allocation in the Global Change Assessment Model
(GCAM) version 3.0. The question of how to more holistically evaluate models
as complex as IAMs is an area for future research. We find quantitative
evidence that global aggregates alone are not sufficient for evaluating IAMs
that require global supply to equal global demand at each time period, such
as GCAM. The results of this work indicate it is unlikely that a single
evaluation measure for all variables in an IAM exists, and therefore sector-by-sector evaluation may be necessary.
| Reference Key |
snyder2017geoscientificevaluation
Use this key to autocite in the manuscript while using
SciMatic Manuscript Manager or Thesis Manager
|
|---|---|
| Authors | ;A. C. Snyder;R. P. Link;K. V. Calvin |
| Journal | international journal of quantum chemistry |
| Year | 2017 |
| DOI |
10.5194/gmd-10-4307-2017
|
| URL | |
| Keywords |
Citations
No citations found. To add a citation, contact the admin at info@scimatic.org
Comments
No comments yet. Be the first to comment on this article.