Not going to debate the state of Birmingham.
To answer the OP's question, NO I do not avoid Birmingham. If there's an interesting girl I fancy there, I'll go. Similarly, if there's a tempting SP not too far (within 60 miles) I'm prepared to work on my agenda and try to organise transport. It's part of punting. It doesn't come all sorted.
1) Birmingham is the same as many cities in terms of parking and potential fines. In the city centre (mostly the CAZ) there are plenty of NCP car parks, and meters on the road (with their shitty frustrating apps). And when you're about to blow £100 on a SP, what's an extra £10 anyway? Hiding behind potential parking fines to avoid Birmingham city centre doesn't seem rational. You could get done in much smaller towns with 'resident permits' or '1hour stay' zones just the same. When I go in the centre, I either park on the outskirts of the CAZ in free spots or even some of the last rare free parking areas within the CAZ (and then the 20 minutes walk to the lady is part of the excitation build-up)
2) Even my 10 YO car is CAZ compliant. But even if yours isn't then there's train, buses ... Where there's a will there's a way. I read you said you didn't know Brum, but I've organised trip to close town I didn't know (Shrewsbury, Leamington Spa, Coventry ...) when there were girls I really wanted to see there. Part of the adventure. But thing like Google Streetview are your best friend there to find free parking places and judge the area you're going to go to.
3) Speed limits is the same things. You could get done for speeding anywhere (saw a cop last month next to his motorbike with a speed gun in a little village in the middle of nowhere. Had I not slowed down to 32/33 from 60, I would have been fined just the same as in the city). The only specific danger I'll concede to you regarding going into Birmingham City Centre are those new money-making-racketeering bus gate cameras. Lethal.
In short, punting is a nice occupation which allows varied and otherwise unavailable experiences. But like a nice holiday, it does require a bit of work and planning. To me it's part of the pleasure and even if Brum city Centre might be a bit more complicated logistically speaking I wouldn't avoid it if that's where one of my hotlist girl is there.