And you think the women who service you don't consider you a loser? Putting aside the sexist overtones that you've put how men view you more importantly than how women view you, I think you should look into the mirror and, as a male, consider your own self worth.
The fact there is need for "protection" suggests a higher risk environment, possibly robbery, but more likely punters kicking off for some reason (e.g. drunk, arguing about services withheld, bait & switch). Add to that "Latina", "Streatham", and I'll throw in an assumption of "late at night" and it screams "watch out for yo ass".
The reason for leaving is because of the higher risk of being caught up in some shit going down, being hurt and/or police bring involved. Lots of alpha males in uniforms that will have a view on your "pass time" (sic) then.
You sound like a sensitive guy and I do empathise with the notion of a non-punter judging you. Personally I think it's worse for a woman (who isn't an SP) judging you. A female neighbour staring at you as you enter a well known place can be a demeaning experience.
You need to be a little think skinned to navigate all the perils involved in punting so you might want to think about avoiding these higher risk places. Perhaps you could save up your punting budget and go for a well-reviewed and well-regarded independent who won't only not judge you but should make you feel like a king.
Oh, and you should share some of your previous experiences as reviews. This one for a start. It would be helpful for other punters to know the set up at that brothel.
