Just use a door to leave. A door can be defined as a different looking, movable section of a wall. It either slides or swings open, allowing passage through the wall. I know it sounds really weird and esoteric, even barbarian, for walls to have holes by design, instead of people using magic to teleport themselves in and out, like any civilised being would do, but a door really is a thing, believe it or not.