Selecting Good Problems as a Part-Time Physics Researcher
On finding efficient ways of collecting good problems.
Background: I work in biophysics / theoretical spectroscopy at a major university, part-time, alongside a day job.
During the time I’ve been trying to get into physics research, I’ve realised that choosing research problems is way more important than solving them. But for the most part, choosing good projects is hard, probably harder than solving them, especially if you’re trying to do research outside of any formal academic career. And unfortunately, this is where the scientific method doesn’t say much. The scientific method tells you how to answer questions, but says nothing about how to chose good questions.
But trying to choose good problems feels worth it. I actually think it’s the only criterion for success. People who make conjectures but haven’t solved them are still considered useful. And I want to choose the good problems because I want to be ‘successful’, just as much as I want to enjoy things. I’ve been primarily convinced about this from an essay by Richard Hamming1. My biggest fear right now is doing research that is not effectual, or work that is useless - because then I have wasted my time.
So how do we do this if we are time constrained with a day job? I found that the most efficient way for me choosing research problems was simply to find someone better than I am to help me - finding a supervisor. All the evidence I have for this though is anecdotal. I used to try and do problems and physics research on my own, from textbooks and papers - but I really think this is suboptimal. I think I spent a total of a year messing around in the dark trying to find one.
I find it weird that no one really teaches us how to find good teachers. We are taught a bunch of stuff at school, but we aren't taught in any formal way how to optimise for mentor choice. We are taught how to do things, but not find how to find who can show us to do things best.
I found my supervisor after emailing tons of academics. I also spoke to a few labs in-person through personal connections. I did this for a grand total of around forty attempts. I also tried three different areas of research which didn’t work out.2 People will probably give you some time of day, but won’t be looking to onboard someone for a long period of time.
My current supervisor was the first one to give me a shot. I work with my current supervisor on molecular physics problems, and am currently working on spectroscopy, protein conformations and computational biochem. When it comes to my physics research, as of today, I think there are opportunities in the intersection of physics and biology for multiple reasons, but I will get into that in a different post.
Fortunately or unfortunately, I think ‘personal circumstance’ and luck plays a lot in trying to find a supervisor. You might live in a country without a developed academic system. Even if you do, there may not be supervisors in the areas you’re interested in. Some supervisors might not take people remotely. It is a lot easier to work with supervisors who are willing to engage outside of working hours or on weekends. 3
After experimenting with different ways to collate problems, I’ve found that another way to get problems is just to ask on reddit. I’ve found this to be really effective, and karma filters usually mean that people know what they are talking about. From an industry standpoint, working in an engineering role in any private firm is bound to prove a great source of inspiration too.
And finally - just read a bunch of books!
If you do not work on an important problem, it's unlikely you'll do important work. It's perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them. Let me warn you, "important problem" must be phrased carefully. The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. They are not important problems because we do not have an attack. It's not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don't work on important problems, I mean it in that sense. The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn't believe that they will lead to important problems.
I also feel like the right problem to solve should be chosen from the situation you’re in. I now think that location forms a big part of the success equation, much more than effort. And I think being part of a system makes things a lot easier.
What I tried worked on in the past
Theoretical Physics
Materials science
Graphene in medical devices
In terms of solving problems, I think location matters, because a lot of success is based on peers and the environment around you. It’s inefficient to do good work otherwise, since scaling effects matter. I think for example, pursuing research in a London university would probably not be as efficient as doing research in the US, where there is more funding available for research projects.