Do the bumps appear intermittently in any case, or
only ever after you use a condom? (I feel like that male doctor off Embarrassing Bodies now
). You see what I'm getting at though - might be a case of correlation not cause.
The fact they found an infection on your previous check up, albeit not a sexually transmitted one, surely suggests that the problem is bacterial. No idea why it would seem to have come back, but surely if you had an allergy, you would have noticed it before now?
Anyway latex free condoms are easily obtained, no harm in switching to them if you want to see, although some prossies don't like punters bringing their own condom so it's possibly something you'd have to mention in advance.