However if a guy pays for sex then he is breaking the law
Only under certain conditions is it illegal to pay for sex, such as underage, trafficked, coerced or kerb crawling.
Paying for sex (or selling) in itself is not illegal in UK.
From what I understand, a private meeting is fine and legal (you going to a house with one girl there or you both meeting somewhere), but managed brothels are illegal.
Point me to which UK law makes it criminal offence to pay for sex in a brothel. There isn't any.
It's only illegal to manage a brothel, NOT illegal to work their (as a prostitute) or pay for sex as a punter.