How are stories kept a mystery in your mind?

I woke up from a nightmare where my daughter was injured and it “shocked” me awake. I was having a pleasant dream about a day walking around with family in Montreal, a city I’ve never visited before. It was lovely and fun for what seemed like ages, until a freak accident ended it.

How was this a surprise to me? The story unfolding all came from my brain. But the accident storyline did as well. What’s the mechanism that “hides” the nightmare ending from my conscious mind in order to surprise me? How are dreams ever unknown to us? How did my brain convince me I was in Montreal when I’ve never been and what did it use to fill in the blank spaces? Are storylines simply health related? After my nightmare I had to pee fairly urgently, so was that my autonomic system throwing a wrench in my happy narrative to get me up to relieve myself and feel better?

Are our brains and bodies a bundle of parts of competing systems that all fire independently at random times or is it a brilliantly orchestrated single cohesive system above it all that chooses what to reveal to me along with what it should hide?

How could you even begin to design an experiment to figure out how stories unfold in our dreams?