Sorry for the hurt but not the lack of action, says PM