The outrage over Prince Andrew has missed one huge scandal