The letter refers to the cup size A being the smallest and going up from there, so in layman's terms the letter is what you get in your hand, the amount you can 'cup'
The number ie 32, 34 etc refers to the measurement around the ribcage, so the elastic bit of the bra that hooks up round the back, the measurement is taken from just below the boob or under the overhang if there is one.
So a 32DD is going to look like a large bust as it's on a small frame, whereas a 40DD will look smaller in comparison as the frame its on is larger.
I'm hoping this makes sense