It's actually quite easy, If you notice each of the places you come to on the wall has different symbols. But note how many symbols. For instance if you come across the Castle Tower emblem and it shows 3 towers in all, it will actually be the third from the left in the final room that you set as the castle tower, etc etc.
Former 360 Owner, PS 3 convert. PSN Handle- JDCFlexx