If you don't want him again, you can just tell him and stop giving him attitude and ignoring his calls