Whilst I think it's probably quite common that someone else is either on the premises or nearby, I don't want to see or hear them! If you are visiting someone who claims to be independent, it should be a private encounter. I always look for tell-tale signs such as how many pairs of shoes there are by the door, or how many toothbrushes there are in the bathroom etc, but in all but 1 case I've never had a problem.
A few months ago however, I was with a WG when the door bell rang and I heard a guy come out of the living room, go answer the door and have a good chat with the some guy delivering his 'tobacco'. After that I lost my nerve, ended the session, and walked.
Similarly, a couple of years ago I was trying to make a booking via phone and could a guy in the background on another call. Didn't book.
So to answer the original question, I'm only bothered if it's obvious.