I had a similar situation a few years ago. I could only ride for about 30 minutes before it became unbearable. the answer was CORE WORK. There could be any number of bike fit issues that are fighting your body, but your body is the main issue here. many of us sit in front of computers all day and don't move around enough. if the muscles that support your lower back are not strong, your spine will take the weight and it will hurt. I started working with a chiropractor who emphasized putting me in his gym three days a week to do planks (among other things) until I wanted to die. it worked.
FWIW, I ride singlespeed exclusively and have for most of my time riding bikes. currently on a Karate Monkey with a 120mm fork and a SS steel "gravel" road bike with a drop bar and 47mm tires. whenever things start to hurt, I get nervous that my days riding simple bikes might be over, but I persevere and get stronger until I feel better. I'm approaching 40 now and if I keep this up, I expect at least another decade before I "need" a squishy old man bike.