After Speaking with Wizard he has said that he owned the base (The front door) and that you are your friends were basing in there. So he asked you to get out of his house as he owned it. However, Since he wasn't using the door he could of let you own the door. But it was his choice as it was his home.