Dyer: Distaste for refugees grows as numbers set to skyrocket