I don't doubt that there might be a disproportionate issue regarding the behaviours of punters of black/asian backgrounds. But again I go back to my modifiable/non-modifiable argument - it shouldn't necessarily mean that ethnicities be banned due to behaviours that is perceived to be more common within this population.
Whether it happens in Japan or otherwise is of no concern, and can't justify our own actions one way or another.
And why would a black man see a girl who doesn't want to see blacks? In practice they won't (at least more than once!) but again im talking about social change - many didn't wish to house/etc minorities in the past, nor same sex couples more recently.
Ultimately it depends on how you perceive punting in general. Whilst there is a significant emotional and psychological component, I also see it (somewhat coldly) as a business transaction, a service provision, and should be legitimised as such. Sex workers should pay taxes, but also be able to run their business openly. Maybe if that were the case, we wouldn't be having this discussion at all.