In my experience it usually becomes obvious whether extras are on offer, and asking is not necessary.
"Keep your boxers on" is usually not a positive sign (though I've been asked to take them off halfway through on one occasion).
At the start of the massage when I haven't been told what to wear then I have asked; I've been told a few times "it's up to you", and so have stripped completely, and got extras.
Whenever I've been told to strip completely I have always had extras.