The Pain of Finding the Truth
Ways I am trying to not fool myself
Finding the truth is hard. But you should try to find it. I am, in general, worried about getting things wrong.
And I don’t mean ‘oh I tried something that I think it’s likely to work, but didn’t for a reasonable reason’. Rather, I’m worried about when I make a claim, reasoned from some assumptions, but then
The chain of reasoning is incorrect (which can be prevented with care and practice), or
The base assumptions from which I start my argument are incorrect.
I can apply this to a lot of fields I’m interested in, but stuff like ethics is prickly and thorny, and can be erroneous not in the way that I’ve written above. But stuff like science or engineering, yeah, errors 1 and 2 happen all that time.
Science is just complicated, and it is likely that one of my pyramids of reasoning might have a faulty brick. For context, try some of the problems in Landau's famed 'theoretical minimum' - a physics test for his unfortunate students. If you are impressive enough to attempt them, how confident can you be in your correctness once you’ve got the answer?
Here's a problem that, in plain English, gets you to calculate the rate of decay of a particle trapped in attraction with a source.
And if it's that hard to understand an electron, what makes you think we are capable of understanding humans to an acceptable level, in the social sciences? Rather, what makes you think that the claims on human behavior and in social policy are trustworthy? To illustrate the irony of this, there was a paper in 2023 on trying to improve the replicability of results in the social sciences. But the paper itself wasn't replicable, and so it got retracted as well.
Unfortunately, I think that this is probably due to the well-flagged pressure to publish, and so faking data from experiments that are not likely reproducible is easy to do. But still, the message is clear - it is hard to trust other people since they probably have incentives.
So I think this skeptic anxiety is worth it, because since science and truth are complicated and hard, having a continual self-skeptical mindset is likely to protect you from future harm. This is why I now consider self-skepticism as a core value of my personality, and I genuinely think that the world would be a better place if more people thought like this. Sure, people would be annoyed.
It's easy to fool yourself into thinking you're right. Which I am even more worried about. This sense seems to be shared by some of the greatest scientists we have ever had.
One line that has stuck with me from Richard Feynman's memoir Surely You're Joking, Mr. Feynman:
The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.
Terence Tao, one of the most eminent mathematicians of our time, also offers similar advice when it comes to self-skepticism in mathematical arguments, which I think can also be applied to trusting others. He argues that because of the complexity of nature, getting something too easily is a signal of an error in logic somewhere. I share this sentiment, that effort is a necessary but not sufficient indicator of novelty.
If you unexpectedly find a problem solving itself almost effortlessly, and you can’t quite see why, you should try to analyze your solution more sceptically.
This is in the context of solving mathematical problems but is applicable advice to literally anything. One just needs to see all of the cases of unintended consequences in policy to see what I mean.
It's one thing to fool oneself in the sciences, but trying to get things right about policy, economics, and ethics is a lot harder because we can't have experiments. For ethical reasons, it is near impossible to create scientific standard experiments with humans. And so we have to rely on Bayesianism, logic, and rationality to figure out what's what.
Minimal Trust Investigations
One of the texts which have influenced my thinking is Holden Karnofsky's piece on minimal trust investigations, which is when you aim to answer a question without straight-away assuming Karnofsky argues that minimal trust investigations help him to think about when to trust and not to trust something:
But I also think it's a good idea to occasionally do a minimal-trust investigation: to suspend my trust in others and dig as deeply into a question as I can. This is not the same as taking a class, or even reading and thinking about both sides of a debate; it is always enormously more work than that. I think the vast majority of people (even within communities that have rationality and critical inquiry as central parts of their identity) have never done one.
And that doing one on GiveWell's strategy of malaria nets helped him to question whether malaria nets worked.
I also think this is a great exercise. From a physics perspective, this can be done by:
Reading historical accounts to gain insight into the conditions of experiments.
Verifying mathematical claims in a paper.
Reading historical papers and comparing them to modern experiments.
Conducting experiments yourself, which is the hardest but most rewarding approach.
If you're an undergrad doing science, the kicker is actually doing the derivations and exercises yourself. If there are problems in the text that you're reading, try to do them. And if you can't, seek assistance or use an AI model to help. Often, the very act of posing the question and then asking on platforms like StackExchange will help.
Terence Tao offers another valuable insight:
'When you read something, don't just read it, fight it!'
A side effect of doing these things is that you'll probably write up some expository content, which would probably be a bit original since you're likely to spend some time focusing on your own problems.
This culture of minimal trust was also prevalent in Feynman's thinking. He once questioned the result of a psychologist:
"I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person—to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know that the real difference was the thing she thought she had under control."
And even if you don't do a minimal trust investigation, you can still err on the side of caution by:
Baking skepticism into your personality and questioning assumptions.
Writing things down to clarify your thoughts.
I’ve found that a practical way to do minimal trust investigations on topics is to map out a network of the connected papers that you’re reading. You can do this by going on here. And then you can try to make sure you can understand the math of each constituent paper individually, thereby creating a tree.
For example, I was trying to look at mechanistic explanations of human memory. I was wondering if any of the electronic structure stuff I was working on was relatable, but to do that, I had to do a deep-dive into this single paper on memory engrams. And to help me out, I used the connected papers website to build a list tree of papers I needed to read.
Another thing that I like to do is ground-up expository texts. Start with any claim, and then work deeper and deeper down the tree in the things behind that claim. For example, I was looking at this paper on microtubules exhibiting quantum properties called ‘superradiance’. But then I didn’t realise what superradiance actually was from a first principles, mathematical perspective. So then I wrote an expository section about it in my notes, and worked through the math assuming nothing from what I read in papers was true, and rechecking each claim.
Bibliography
Landau, L. D., & Lifshitz, E. M. Theoretical Minimum.
https://people.tamu.edu/~abanov/QE/TM-QM.pdf
Stanford retracted social science paper.
[Unavailable, screenshot referenced]
Terence Tao on skepticism in mathematical arguments.
https://terrytao.wordpress.com/career-advice/be-sceptical-of-your-own-work/
Camus, A. (1991). The Myth of Sisyphus and Other Essays. Vintage International.
https://www.goodreads.com/work/quotes/855563
Holden Karnofsky on minimal trust investigations.