spi: Fix invalid D channel response when flash interface is disabled