It was interesting to read your analysis, Sosa, because I used to have the self-esteem issues to which you refer. I've never done drugs and my only gamble is £5 a week on the national lottery. My wife had passed away about 18 months before my first punt and I didn't have the confidence to date in Civvy Street [I was in my mid-50s at the time] so, needing to have sex [rather than as an alternative to drugs or gambling], I started visiting Parlours.
Initially, I went through the "kid in a sweetshop" syndrome which I'm sure many new punters experience, and spent much too much on my punting; and, yeah, looking back on it now, I regret having spent some of that money.
About 12 - 18 months after I started punting, I went through a bad time personally [every month, it seemed as if another family member or long-standing friend died] and my GP arranged for me to see a counsellor, who was really helpful, although the one thing I didn't open up to her about was, of course, my punting. The counsellor helped me get my head round the low self-esteem issues and I then started seeing "Indies" [almost always milfs], rather than visiting Parlours. Which I've carried on doing and will continue to do so for as long as I can.
My wife & I had no kids and I've no mortgage; so with inflation being higher than the rate of interest I can get on my savings, I see little point in putting much in the bank these days. So, while not going silly with my punting, I enjoy it on the no-strings basis you write about. In some ways, it's better than that for me - you didn't say whether or not you have "regulars"; but I do and I reckon it's the best of all worlds, having sex reasonably regularly with attractive women with whom I get on well but don't have any emotional ties to.