Clinical Biomechanics
Volume 20, Issue 5 , Pages 451-454, June 2005

Extracting clinically relevant data from finite element simulations

Laboratorio di Tecnologia Medica, Istituti Ortopedici Rizzoli, Bologna, Italy

MEM Research Center, Institute for Surgical Technology and Biomechanics, Bern, Switzerland

Clinical Biomechanics, Editor in Chief

published online 14 March 2005.

Article Outline

 

Back to Article Outline

1. Introduction 

The present editorial is the conclusion of a consensus process that started within the Editorial Board of Clinical Biomechanics, and continued with a public discussion on the biomch-l mailing list. As reviewers we are periodically called to evaluate manuscripts in which results obtained from numerical models are used to draw clinically relevant recommendations. Some reviewers tend to consider physical and numerical models of equal value, arguing that both are a simplified representation of the problem being considered. Others consider all results obtained form a numerical model inherently false, and thus discourage any clinical recommendation based on these methods. Most reviewers look for a middle ground between these two extreme positions. In our opinion this disparity of opinions indicates a lack of consensus within our community. The aim of this editorial is to summarise the results of a wide discussion among experts, and to delineate the position of the Editorial Board of Clinical Biomechanics on this important matter.

Back to Article Outline

2. What is a numerical model? 

A numerical model generally consists of a large number of mathematical equations and relies on computers to find an approximate solution to the underlying physical problem. In this sense a numerical model can be considered a particular instance of a mathematical model, which is in physics, the way we represent a theory. Due to their ability to represent complex systems, numerical models are used to simulate and study a large range of problems in biomechanics ranging from classical structural analysis and mass transport, to fluid mechanics, etc. In fact any number of partial differential equations can be simulated concomitantly. The success of this method in engineering and physics is largely due to clear guidelines, such as The International Association for the Engineering Analysis Community (NAFEMS; http://www.nafems.org/), a large professional community, and also the rather simple materials that are considered in most industry applications. In biomechanics and related research our biggest challenge is the uncertain material properties and complex micro- and macro-geometry of the biological tissues we consider in our analysis. The tools themselves, such as finite element analysis, are valid when used correctly. In many types of analyses in this area the input parameters are known (e.g. bone stiffness), but the meaning of the output is sometimes unclear (e.g. von Mises stress). Every effort should be expended to ensure that the finite element model is verified and validated and that its output is correctly interpreted before its predictions can be considered to have any clinical value.

Back to Article Outline

3. Verification and validation 

In extreme synthesis, verification is about solving the equations right; validation is about solving the right equations (Roarche, 1998). The term verification is commonly used to indicate the process ensuring that the numerical model accurately predicts the results of the theoretical model it is based on. In other words a model is verified by assessing its numerical accuracy. On the contrary, the term validation indicates the process that ensures that the numerical model accurately predicts the physical phenomenon it was designed to replicate. Thus, a model is validated by assessing its accuracy tout court, i.e. the accuracy with which it predicts the reality. However, similarly to a theory, it is in general impossible to prove the validity of a numerical model completely. This is why the formation of a scientific truth, which is a fully validated theory, is a slow and sedimentary process. In science we frequently use the expression ‘it is generally believed’ to indicate this consensus process.

It is evident that while verification is a process fully internal to the work of the single scientist, and thus can be considered mandatory for publication, the validation process involves the whole scientific community and in this sense there is a sort of relativism in it. Nevertheless, some general guidelines designed to avoid common pitfalls is required in this area. There is certainly no unique way to approach any specific problem. Minimal requirements, however, must be met before publication can be considered.

Back to Article Outline

4. Validation of numerical models to be used for clinical purposes 

The medical professional must do something when is faced with a suffering patient, even if he or she is not 100% sure that the planned intervention will actually be an effective treatment.

In contemporary medicine one way to formalise this decisional process is by risk–benefit analysis. We propose that our community should start to consider this instrument for the evaluation of the degree of validation of a numerical model to be used in clinical practice. We accept the fact that no numerical model can be totally validated (Oreskes et al., 1994) when applied to simulate biological tissue. Thus, there is always a risk associated to the use of the predictions made by a model in clinical practice. So the first question which should be answered is: what is the level of risk the patient is exposed to when clinical decisions are based on the results of a numerical model? This is not an easy question. When a model has been validated by means of controlled experiments and/or clinical efficacy studies, it is unlikely that it provides results that are completely wrong. But the other question to be considered is: what is the risk for the patient if the model is wrong (within the limits of its validation)? In the end these are questions of ethics, which must be considered by the clinician. However, the clinician must be made aware of the limitations of any model (numerical or experimental). Furthermore, direct transfer of information from the model to patient is rarely possible. Whenever a model is used for these purposes however, correct interpretation as well as assessing the model’s sensitivity becomes vital.

Back to Article Outline

5. Minimal requirements for a numerical study 

The original motivation of this work was related to the need for author and review guidelines for submission based in part or completely on numerical simulations in journals with a clinical target audience. Based on the consensus reached during discussions, and on the lines of thought reported above, our conclusions are:

•Model selection: a clear reason for why any particular study was conducted must be present in any academic publication. It is also important to communicate why a certain method is the best at investigating the topic of the paper. It must be clear to the reader that the author has weighed up the various tools of investigation. When numerical analysis is considered most appropriate, the numerical model must represent the problem at hand to a sufficient level of accuracy. All relevant aspects, such as the geometry (plane strain/stress, axi-symmetry, 3D), physics (static/dynamic, fluid mechanics, mass-transfer, etc.), boundary conditions, and material properties must be discussed to such a level that the reader can judge to what degree this model represents the physical problem being considered.

•Verification: as mentioned, verification seeks to ensure that the numerical solution approximates the mathematical model on which it is based. For linear models convergence of the mesh refinement or even better on post-hoc indicators related to mesh refinement (Zienkiewicz and Zhu, 1987), must be performed and reported. When a problem is non-linear, it should in most cases be solved iteratively. If a commercial finite element package is used, convergence tolerance for the iteration process should be reported only in those cases when the default criteria have been altered. In any case convergence is a basic requirement, and validation of numerical accuracy is required for in-house solvers. When the model is non-linear the verification should be specific for the type of non-linearity present. If you are including non-linear frictional contact it is appropriate to report the peak penetration or equivalent tolerance, and so on.

•Proper parameter identification: another mandatory requirement for any paper based on numerical analysis is proper identification of model parameters. These should be associated with measurable physical/physiological quantities and be independent by the time or the repetition in the experiment used to identify them. When one attempts to validate a model by showing its ability to fit experimental results over an interval of a given parameter, i.e. time, the number of independent parameters in the model should be significantly lower than the ‘order’ of the experimental event to ensure a unique solution. In such cases where a unique solution can not be obtained, or where the model properties are uncertain it may be reasonable to investigate the sensitivity of the model to a few critical parameters.

When a paper presents these features it may be considered for publication in those journals that are interested in theoretical speculation, and deductive reasoning. In practice with a verified model, a theory is put forth, and the numerical model can be used to explore all its implications. At this stage, when deductions are made with respect to the clinical recommendations, it should be made clear that these are theoretical speculations, in need of further support.

•Sensitivity analysis: some problems do have exact solutions; however this is never the case when biological problems are simulated and the uncertainty in model input should be considered in the analysis (Dar et al., 2002). Most modern commercial finite element analysis packages allow for sensitivity to be included. Sensitivity studies are essential in biomedical research, where frequently the few available experimental measurements are affected by large uncertainties. Rather than ‘using’ these uncertainty to say that the predictions of the model fall with the range of the predicted experiment, it is necessary to do a full sensitivity study that shows how this uncertainty impacts on our deductions. This approach utilizes a unique capability of numerical analysis, and should be performed unless the outcome of such effort is obvious.

•Inter-subject variability: this is another type of sensitivity analysis. The physical properties of biological systems vary enormously between various anatomical sites and subjects. Models are usually based on a specific subject or an ideal average subject. Thus, in principle, the effect of inter-subject variability should be considered. It is very difficult to provide strict guidelines here. In some cases the inter-subject variability can be parameterised, and thus included in the sensitivity analysis. In these cases it is considered mandatory. In other cases a new model would need to be built for each new subject in the study, and this prevents systematic exploration on the effects of inter-subject variability. In any case the authors should address this issue and discuss possible implications. One option may be to investigate a few subjects that are representative of the extremes of variability of the population of interest. This is very similar to a design of experiments approach, and it gives a gross indication of the level of variability one may expect in model output.

At this stage of the validation process, we have a theoretical model that is robustly linked to the experiments that are used to identify it. We propose that this second level of validation should be mandatory to publish in those journals aiming at applied biomechanics research.

•Validation against controlled in vitro experiments: this is the first step in the clinical validation process. It is usually very difficult to perform such validation experiments, and when successful they often show unexpected weaknesses in the numerical model. Of course, the outcomes of this comparison between numerical and experimental results must be properly reported. Many authors use a linear regression between measured and predicted value, and report the regression parameters and coefficients. However, the residuals of such regression should also be presented and discussed. One way is provide a root mean square error as an indication of the average residual, and the peak error, as indication of the maximum residual.

This third level of validation should be considered mandatory for all those journals aiming at the space between biomechanics and clinical research.

•Risk–benefit analysis: before results obtained from numerical models can be used in clinical practice the results of a risk–benefit analysis should be reported. To do so a fully verified model is required on which a complete sensitivity study and an in vitro validation study has been conducted. This would provide a quantitative basis to estimate the risk associated with the use of the model. Of course these studies should be conducted in collaboration with clinical researchers, who should have the necessary knowledge to estimate the expected benefits.

•Retrospective studies: along the same lines, but with greater level of confidence, there are the retrospective studies. If a model can be used to answer clinical questions over a population for which the answer to this question is known, good insight may be obtained on the clinical validity of your model. In many cases the best way to report these results may be in terms of specificity and sensitivity, using the R.O.C. curves (Swets et al., 2000). A key issue here is that the clinical question must yield a true value.

•Prospective studies: as for any other method, whenever possible the conclusive word on the clinical accuracy and usefulness of a numerical model comes from prospective clinical trials.

When this type of validation studies is available publication of numerical studies should be considered also in clinical journals.

Back to Article Outline

6. Where is Clinical Biomechanics on the map? 

Clinical Biomechanics aims to strengthen the link between clinic and laboratory by publishing biomechanics research which helps to explain the causes of musculo-skeletal disorders and which provides knowledge contributing to improved clinical management. The readers of the journal reflect its contents, being a balance of scientists, engineers and clinicians. Thus, in order to be published on Clinical Biomechanics a paper based on numerical methods should report complete model verification, proper parameters identification, sensitivity analysis and/or inter-subject variability analysis, as defined in the previous sections. In addition, if the papers draw clinical recommendations, the numerical model should be validated against controlled in vitro experiments. Every manuscript reporting results obtained with a numerical model will be subject to a preliminary screening by the editorial board to ensure it fulfils these requirements. If the standards are not met, the manuscript will not be reviewed, but returned to the authors with simple reference to this editorial.

Back to Article Outline

7. Code reliability 

Besides model verification, there is also code verification. For many problems in biomechanics, such as coupled transport, no existing commercial tool can satisfy the needs of the research questions at hand. In these cases in-house solvers are sometimes developed specifically for the purpose of the study. All such new tools must go through a thorough code verification process to validate the numerical accuracy. The validation of any new tool must be presented in a technical journal before it can be considered suitable to be used for analysing clinically relevant problems. This is similar to the general quality control we should apply to all laboratory instruments. Any new method of investigation must be validated; software is no exception to this rule.

Back to Article Outline

8. Conclusion 

It is fair to say that numerical simulation can be a very useful tool in biomechanics research. However, as for experimentally based research, articles in this area are of very varying quality. Numerical analysis is easy to do poorly and very hard to do well. First and foremost the researcher needs to ask; what questions do I want this model to answer? Then every effort must be made to create a model which simulates the problem to a suitable degree of accuracy in order to answer the research questions at hand with maximum degree of certainty.

In this article some formal requirements have been put forth which we hope will help the researcher report the findings of a numerical study.

Back to Article Outline

References 

  1. Dar FH, Meakin JR, Aspden RM. Statistical methods in finite element analysis. Journal of Biomechanics. 2002;35:1155–1161
  2. Oreskes N, Shrader-Frechette K, Belitz K. Verification, validation and confirmation of numerical models in Earth science. Science. 1994;263:641–646
  3. Roarche P. Verification and validation in computational science and engineering. Albuquerque: Hermosa Publishers; 1998;
  4. Swets JA, Dawes R, Monahan J. Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest. 2000;1:1–26
  5. Zienkiewicz OC, Zhu JZ. A simple error estimator and adaptive procedure for practical engineering analysis. International Journal for Numerical Methods in Engineering. 1987;24:337–357

PII: S0268-0033(05)00027-6

doi:10.1016/j.clinbiomech.2005.01.010

Refers to erratum:

  • Erratum to “Extracting clinically relevant data from finite element simulations” [Clinical Biomechanics 20 (2005) 451–454] , 08 August 2005

    Marco Viceconti, Sigbjorn Olsen, Lutz-P. Nolte, Kim Burton
    Clinical Biomechanics November 2005 (Vol. 20, Issue 9, Page 1010)

Clinical Biomechanics
Volume 20, Issue 5 , Pages 451-454, June 2005

Article Tools