Driving, especially for long stretches of time, has a way of lulling even the most alert drivers into a state of reminiscing. Maybe the last time you made your way down that street you were with a former flame, or a song on the radio sparks a distant childhood memory. As pleasant as it is to take a trip down memory lane, a new study has shown that daydreaming while driving can be dangerous — even if you keep your eyes on the road.

Researchers from Vanderbilt University found that recalling a visual memory can temporarily “contaminate” vision, making a road riddled with obstacles that much more difficult to navigate. The study used an illusion called motion repulsion to determine whether information held in working memory changes current perception.

Participants were shown a pattern of dots and asked to remember in which direction they moved. Then they were shown a second pattern of moving dots and asked to recall their direction, too. The results showed that the first pattern significantly swayed what they perceived next. For example, if they first saw dots moving in one direction, then looked at dots moving in a slightly counterclockwise course, they exaggerated the counterclockwise movement.

The research provides evidence that working memory may use a part of the brain that also processes senses, like the primary visual cortex. Previously, scientists did not think this area of the brain had anything to do with the act of remembering. It’s another clue that our senses are closely linked to memories.

So next time you’re driving down the road, try to keep both your eyes — and mind — on the road in front of you. You don’t want your primary visual cortex working double duty while you’re operating heavy machinery. Save the daydreaming until you’re safely parked at home.