Cambridge University Press

9780521113601 - Verification and Validation in Scientific Computing - By William L. Oberkampf and Christopher J. Roy

Frontmatter/Prelims

Verification and Validation in Scientific Computing

Advances in scientific computing have made modeling and simulation an important part of the decision-making process in engineering, science, and public policy. This book provides a comprehensive and systematic development of the basic concepts, principles, and procedures for verification and validation of models and simulations. The emphasis is placed on models that are described by partial differential and integral equations and the simulations that result from their numerical solution. The methods described can be applied to a wide range of technical fields, such as the physical sciences, engineering, and technology, as well as to a wide range of applications in industry, environmental regulations and safety, product and plant safety, financial investing, and governmental regulations.

This book will be genuinely welcomed by researchers, practitioners, and decision-makers in a broad range of fields who seek to improve the credibility and reliability of simulation results. It will also be appropriate for either university courses or independent study.

William L. Oberkampf has 39 years of experience in research and development in fluid dynamics, heat transfer, flight dynamics, and solid mechanics. He has worked in both computational and experimental areas, and taught 30 short courses in the field of verification and validation. He recently retired as a Distinguished Member of the Technical Staff at Sandia National Laboratories.

Christopher J. Roy is an Associate Professor in the Aerospace and Ocean Engineering Department at Virginia Tech. After receiving his PhD from North Carolina State University in 1998, he spent five years working as a Senior Member of the Technical Staff at Sandia National Laboratories. He has published numerous articles on verification and validation in the area of computational fluid dynamics. In 2006, he received a Presidential Early Career Award for Scientists and Engineers for his work on verification and validation in computational science and engineering.

Verification and Validation in Scientific Computing

William L. Oberkampf and Christopher J. Roy

CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo, Delhi, Dubai, Tokyo, Mexico City

Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK

Published in the United States of America by Cambridge University Press, New York

www.cambridge.org
Information on this title: www.cambridge.org/9780521113601

This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press.

First published 2010

Printed in the United Kingdom at the University Press, Cambridge

A catalog record for this publication is available from the British Library

Library of Congress Cataloging in Publication data

Oberkampf, William L., 1944--

Verification and validation in scientific computing / William L. Oberkampf, Christopher J. Roy.

p. cm.

Includes index.

ISBN 978-0-521-11360-1 (hardback)

1. Science -- Data processing. 2. Numerical calculations -- Verification. 3. Computer programs -- Validation.

4. Decision making -- Mathematical models. I. Roy, Christopher J. II. Title.

Q183.9.O24 2010

502.85 -- dc22 2010021488

ISBN 978-0-521-11360-1 Hardback

Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

To our wives, Sandra and Rachel

Preface	xi
Acknowledgments	xiii
1 Introduction	1
1.1 Historical and modern role of modeling and simulation	1
1.2 Credibility of scientific computing	8
1.3 Outline and use of the book	15
1.4 References	17
Part I Fundamental concepts	19
2 Fundamental concepts and terminology	21
2.1 Development of concepts and terminology	21
2.2 Primary terms and concepts	32
2.3 Types and sources of uncertainties	51
2.4 Error in a quantity	57
2.5 Integration of verification, validation, and prediction	59
2.6 References	75
3 Modeling and computational simulation	83
3.1 Fundamentals of system specifications	84
3.2 Fundamentals of models and simulations	89
3.3 Risk and failure	115
3.4 Phases of computational simulation	116
3.5 Example problem: missile flight dynamics	127
3.6 References	137
Part II Code verification	145
4 Software engineering	146
4.1 Software development	147
4.2 Version control	151
4.3 Software verification and validation	153
4.4 Software quality and reliability	159
4.5 Case study in reliability: the T experiments	161
4.6 Software engineering for large software projects	162
4.7 References	167
5 Code verification	170
5.1 Code verification criteria	171
5.2 Definitions	175
5.3 Order of accuracy	180
5.4 Systematic mesh refinement	185
5.5 Order verification procedures	192
5.6 Responsibility for code verification	204
5.7 References	205
6 Exact solutions	208
6.1 Introduction to differential equations	209
6.2 Traditional exact solutions	210
6.3 Method of manufactured solutions (MMS)	219
6.4 Physically realistic manufactured solutions	234
6.5 Approximate solution methods	239
6.6 References	244
Part III Solution verification	249
7 Solution verification	250
7.1 Elements of solution verification	250
7.2 Round-off error	252
7.3 Statistical sampling error	258
7.4 Iterative error	260
7.5 Numerical error versus numerical uncertainty	283
7.6 References	284
8 Discretization error	286
8.1 Elements of the discretization process	288
8.2 Approaches for estimating discretization error	297
8.3 Richardson extrapolation	309
8.4 Reliability of discretization error estimators	317
8.5 Discretization error and uncertainty	322
8.6 Roache's grid convergence index (GCI)	323
8.7 Mesh refinement issues	329
8.8 Open research issues	334
8.9 References	338
9 Solution adaptation	343
9.1 Factors affecting the discretization error	343
9.2 Adaptation criteria	349
9.3 Adaptation approaches	356
9.4 Comparison of methods for driving mesh adaptation	360
9.5 References	366
Part IV Model validation and prediction	369
10 Model validation fundamentals	371
10.1 Philosophy of validation experiments	372
10.2 Validation experiment hierarchy	388
10.3 Example problem: hypersonic cruise missile	396
10.4 Conceptual, technical, and practical difficulties of validation	401
10.5 References	405
11 Design and execution of validation experiments	409
11.1 Guidelines for validation experiments	409
11.2 Validation experiment example: Joint Computational/Experimental Aerodynamics Program (JCEAP)	422
11.3 Example of estimation of experimental measurement uncertainties in JCEAP	437
11.4 Example of further computational--experimental synergism in JCEAP	455
11.5 References	465
12 Model accuracy assessment	469
12.1 Elements of model accuracy assessment	470
12.2 Approaches to parameter estimation and validation metrics	479
12.3 Recommended features for validation metrics	486
12.4 Introduction to the approach for comparing means	491
12.5 Comparison of means using interpolation of experimental data	500
12.6 Comparison of means requiring linear regression of the experimental data	508
12.7 Comparison of means requiring nonlinear regression of the experimental data	514
12.8 Validation metric for comparing p-boxes	524
12.9 References	548
13 Predictive capability	555
13.1 Step 1: identify all relevant sources of uncertainty	557
13.2 Step 2: characterize each source of uncertainty	565
13.3 Step 3: estimate numerical solution error	584
13.4 Step 4: estimate output uncertainty	599
13.5 Step 5: conduct model updating	622
13.6 Step 6: conduct sensitivity analysis	633
13.7 Example problem: thermal heating of a safety component	638
13.8 Bayesian approach as opposed to PBA	664
13.9 References	665
Part V Planning, management, and implementation issues	671
14 Planning and prioritization in modeling and simulation	673
14.1 Methodology for planning and prioritization	673
14.2 Phenomena identification and ranking table (PIRT)	678
14.3 Gap analysis process	684
14.4 Planning and prioritization with commercial codes	690
14.5 Example problem: aircraft fire spread during crash landing	691
14.6 References	694
15 Maturity assessment of modeling and simulation	696
15.1 Survey of maturity assessment procedures	696
15.2 Predictive capability maturity model	702
15.3 Additional uses of the PCMM	721
15.4 References	725
16 Development and responsibilities for verification, validation and uncertainty quantification	728
16.1 Needed technical developments	728
16.2 Staff responsibilities	729
16.3 Management actions and responsibilities	738
16.4 Development of databases	747
16.5 Development of standards	753
16.6 References	755
Appendix: Programming practices	757
Index	762

Verification and Validation in Scientific Computing

Verification and Validation in Scientific Computing

William L. Oberkampf and Christopher J. Roy

Contents