Think of your expectations and lower them by about 30%. don't know if anyone else agrees but another thing to note is that depending on your price range/requirements you might end up in some rough areas. Even at the £100 plus level i've had to get used to going into areas and buildings that don't look great on the inside. Punting isn't always glamorous and can sometimes take getting used to. It's just a matter of taking a moment, getting used to your surroundings, composing yourself and then being assertive if after that you still don't feel good walk away.
HMD