arm: Fix ICE with incompatible values for -mfp16-format [PR98636].