It's not a brothel if only one woman is working the same residence all the time. This aspect is totally legal. All the men are behaving well so there is no public order issue.
She could be just using Tinder FFS.
Wonder how the beaky bastard thought it was a "drugs den"? Just a sickening invasion of her privacy. Bunch of nimby tossers who should be buried in their plots early, prior to expiration.