the general argument goes like this:
a) the aliens are constrained by the rules of physics
b) these rules don't allow them to be here
c) therefore, they're not
this works the same whether you apply it to space travel, time travel, or "inter-dimensional" travel. if you accept it, you can...