‘I refused to join the Adelaide Writers Week boycott. The attacks on me since have been frightening’