Book notes — Scientific Reasoning: the Bayesian approach

Note: I am a professional researcher in the natural sciences who reads philosophy of science in his free time, not an expert in philosophy of science. I am also not an expert in Bayesian inference techniques, though not a complete beginner either. Treat what follows as an opinion piece. Experts who may wish to correct my inaccuracies or misunderstandings are welcome to do so by e-mail. I am the sole author of this text and I did not use an LLM to generate or edit its content.

A few months ago, I decided to deepen my understanding of Bayesian methods and Bayesian reasoning in general. On the technical side, I started to read (or rather, peruse) books like Andrew Gelman and coll.’s Bayesian Data Analysis (1) and Christian Robert’s The Bayesian Choice (2). I was also interested, as part of my ongoing effort to learn about philosophy of science, in reading what philosophers had to say about Bayesian reasoning and confirmation theory in relation to scientific inference. Samir Okasha’s Very Short Introduction to the philosophy of science (3) points to Colin Howson and Peter Urbach’s Scientific Reasoning: the Bayesian approach as the reference on the topic, and I was lucky enough to find a used copy online (3rd ed., 2006, the most recent).

It is a very interesting book that took me way too long to read¹. Howson and Urbach discuss probabilities and Bayesian reasoning, and contend that this latter defines and should underpin scientific inference. Their thesis is that probability calculus, in the epistemic interpretation, forms an inductive logic system that allows valid probabilistic inference. Deductive logic goes from premises to conclusions; if the deductive argument is valid, and the premises true, then the conclusions are guaranteed to be true. In inductive probabilistic reasoning, the priors play the role of the premises, and Bayes’s theorem is the “inference engine” to evaluate the degree of confidence one can have in the conclusions. This thesis is elaborated upon in much greater details and rigor throughout the book, especially in Chapter 3 (by far the most technical of the book). It is advised to have a mathematical background in probabilities: this book is probably not yet for you if you never heard of Kolmogorov’s axiomatics.

Howson and Urbach’s thesis places them firmly on the “epistemic probability” interpretation: probabilities are degrees of confidence or belief, rather than objective features of the world. The Bayesian framework, they show, solves the problems that arise if one is too strongly attached to deduction as the defining feature of scientific reasoning (looking at you Karl Popper). Howson and Urbach expose very clearly (in Chapter 4) the philosophical issues that make Popper’s celebrated falsificationist position untenable, first and foremost the Quine-Duhem problem: since hypotheses always rely on background hypotheses (because, say, of the incremental construction of scientific theories), it is always possible to “save” a hypothesis by rejecting a background hypothesis instead. For example, say that from my hypothesis as to the enzymatic mechanism, I predict that mutating that specific residue in myosin should make it hydrolyze ATP faster; then the experiment shows that this is in fact not the case. I should then reject my hypothesis, because it has been falsified. But wait — I could also reject the hypothesis that my ATPase assay works correctly. Isn’t the assay itself independently validated, though? Yes — by reference to other background hypotheses. Which I could decide to reject as well… this all sounds nitpicky, but the fact is that falsificationism as formulated by Popper is not equipped to address these objections.

How do we escape the Quine-Duhem problem? Bayesian reasoning — Bayesian confirmation theory accounts for why our belief that the ATPase assay works is increased by all its past successes. Thus, we naturally would put a high prior probability on the assay being trustworthy, much higher than our prior probability on our mechanistic hypothesis being correct. And so our mechanistic hypothesis is the one that ends up being rejected based on the observation that the mutation does not have the effect it predicted. It is rejected because its posterior probability has decreased enough, perhaps past some unspoken threshold. Another point is that Bayesianism, unlike falsificationism, accounts naturally for situations where a hypothesis is not fully abandoned when its predicted consequences are not observed. Bayesianism also incorporates organically, and without drama, the fact that several scientists can have different prior probabilities about the same hypotheses. Howson and Urbach insist that this is a normal, sound, and desirable feature of science as a practice.

This may sound useless to many scientists; why would we even need to explain these things? Perhaps we don’t, perhaps “Philosophy of science is as useful to scientists as ornithology is to birds”². I do think that reflecting a bit about what, if anything, constitutes scientific reasoning is important. Too many scientists still tout falsifiability as the criterion for the scientificity of a hypothesis ³. As it turns out, philosophy of science has had very interesting things to say since the mid-20th century, and for this reason alone I would recommend Howson and Urbach’s Chapter 4 to other scientists.

The central block of the book is devoted to a philosophical attack against frequentist methods. To my dismay, I must confess I am not able to give a simple summary of the charges, and I will have to read this part of the book again. Howson and Urbach point out many issues with frequentist methods. It seems to me most have in common:

• frequentist methods misapply probability theorems (like the Law of Large Numbers) by pretending that large populations are the same as infinite, regular populations
• frequentist methods claim to be objective (as opposed to the “subjective”, therefore unscientific, Bayesian approaches) but actually hide very very strong assumptions, which are so entrenched that they are not questioned by practitioners even though they determine the outcome of the analysis.

After ruining frequentist methods (which probably was bolder and more controversial in 2006, to say nothing of 1989, than it is now), they show how Bayesian approaches give a principled alternative. Again the details elude me, but a broad lesson I got is this: frequentist methods were intended to be rigorous, which meant objective. In so doing, they actually baked in assumptions which are ultimately arbitrary, yet determine the results. A simple, if perhaps a bit overblown, example is the significance threshold for p-values (which is chosen arbitrarily). Bayesianism drew general (and at times, apparently, bitter) criticism because the very notion of a prior seemed too subjective to be scientific. Howson and Urbach point out that there are “agreement theorems” that reassure us that, under quite general assumptions, rational individuals updating their priors from shared evidence will eventually reach the same posterior. More importantly — in Bayesianism, subjectivity is out in the open, and if you don’t like my prior, you are free to use yours. This is a good thing, claim Howson and Urbach.

This is also where they deviate from another Bayesian line of thought, and where they close the book: the debate between subjective and objective Bayesianism. This is the topic of their Chapter 9 which, like Chapter 4, is nearly self-contained and will be of utmost interest to scientists (not the least to computational biophysicists). Objective Bayesians attempt to ground the choice of the prior in objective principles. These sound intuitively very reasonable, if not obvious: for example, the Principle of Indifference, informationless priors, and of course E.T. Jaynes’s Principle of Maximum Entropy (Maximum Entropy approaches). Glossing over technical details, Howson and Urbach’s objection to these approaches is close to that against frequentist methods: the “objective principles”, no matter how carefully formulated to avoid introducing non-observational information, ultimately fail because they rest on arbitrary choices. These choices may be reasonable, but they are still subjective. And this is fine.

As (non-Bayesian) philosopher of biology Elliott Sober says it in the blurb, this book is “full of cool arguments”. One that I liked particularly is when Howson and Urbach point out that the famous No Miracle Argument (NMA), which motivates the scientific realist position, mistakenly compares likelihoods without accounting for the priors. They do not expand on this (and it’s possible Psillos addressed this criticism in his book, how would I know, it melted my brain), but as a closeted antirealist I found this cool. There are quite a lot of typos (especially in the equations) which is ideal to keep the reader on their toes. Still, this book is exceedingly well written: it is clear, engaging, and often delightfully witty, with banger quotes such as:

“There are no informationless priors, and the quest for them is the quest for the chimera” (p 277, Chapter 9).

I found it very enjoyable and valuable for its philosophical perspective on Bayesian approaches, a nice complement to practice-oriented textbooks.

References

1. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., & Vehtari, A. (2014). Bayesian Data Analysis. CRC Press.
2. Robert, C. P. (2007). The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation. Springer.
3. Okasha, S. (2016). Philosophy of Science: A Very Short Introduction. Oxford University Press.
4. Psillos, S. (1999). Scientific Realism: How Science Tracks Truth. Routledge.
5. Godfrey-Smith, P. (2021). Theory and Reality: An Introduction to the Philosophy of Science. The University of Chicago Press.