Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Conference paper
Title: Beyond validation: assessing the legitimacy of artificial neural network models
Author: Humphrey, G.
Maier, H.
Wu, W.
Mount, N.
Dandy, G.
Dawson, C.
Citation: Proceedings of the 24th International Congress on Modelling and Simulation (MODSIM2021), 2021, pp.85-91
Publisher: Modelling and Simulation Society of Australia and New Zealand
Publisher Place: Canberra, ACT, Australia
Issue Date: 2021
ISBN: 9780987214393
ISSN: 2981-8001
Conference Name: International Congress on Modelling and Simulation (MODSIM) (5 Dec 2021 - 10 Dec 2021 : Sydney, NSW, Australia)
Statement of
G.B. Humphrey, H.R. Maier, W. Wu, N.J. Mount, G.C. Dandy and C.W. Dawson
Abstract: Artificial neural network models have been used extensively for prediction and forecasting over the last 25 years. As the data used to develop ANNs contain important information about the physical processes being modelled, it is generally implied that a model that has been calibrated (trained) and performs well on an independent set of validation data represents the underlying physical processes of the system being modelled. However, this is not necessarily the case, most likely due to problems with equifinality, where different combinations of model parameters (e.g. connection weights) result in similar predictive performance. Consequently, there is also a need to check the behaviour of calibrated ANN models as part of the validation process, which is commonly referred to as structural, conceptual or scientific validation (Figure 1). This checks whether the input-output relationship captured by the model is plausible in accordance with a priori system understanding. Figure 1. Importance of checking both predictive accuracy and model behaviour during ANN model validation processes In this paper, the importance of considering structural validation is demonstrated. This is achieved by developing ANN models with different numbers of hidden nodes for two environmental modelling case studies from the literature namely, salinity forecasting in the River Murray in South Australia and the prediction of treated water turbidity at a water treatment plant based on raw water quality and the administered alum dose. The validation errors are then compared with corresponding model behaviours. This was done using the validann R-package, which caters to a range of structural validation approaches. Results show that ANN models producing the best fit to the data do not necessarily result in models that behave in accordance with underlying system understanding.
Keywords: Artificial neural networks; multilayer perceptron; structural validation; process understanding; validann
Rights: © the Authors and Modelling and Simulation Society of Australia and New Zealand Inc. (MSSANZ) These proceedings are licensed under the terms of the Creative Commons Attribution 4.0 International CC BY License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you attribute MSSANZ and the original author(s) and source, provide a link to the Creative Commons licence and indicate if changes were made. Images or other third party material are included in this licence, unless otherwise indicated in a credit line to the material.
DOI: 10.36334/modsim.2021.a3.humphrey
Grant ID:
Published version:
Appears in Collections:Civil and Environmental Engineering publications

Files in This Item:
File Description SizeFormat 
hdl_134910.pdfPublished version2.81 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.