If there is any completed feature in the vicinity of a vacant castle the magic gate cannot be used to enter it - the castle is already completed.
But still you must remember, if that feature was completed BEFORE or AFTER castle creation. We can count only features, completed AFTER castle creation.
In our company we play that follower in a castle is safe from Dragon or tower. The follower doesn't belong to a particular tile, so it makes him similar to barn, that doesn't belong to one tile - it stands on 4 tiles simultaneously and is safe from all dangers. You can tell that barn doesn't count as follower - ok, but still we prefer to use same mechanics both on barn and on castle occupier.