Visual and Verbal Memories Use the Same Brain Formats

New York University scientists have discovered how working memory is formatted. And they have advanced toward better understanding of how visual memories are stored.

Working memory is the ability to store and retrieve information for short periods of time. And impairments of working memory are associated with many psychiatric and neurologic conditions, including schizophrenia.

"For decades researchers have wondered about the nature of the neural representations that support our working memory," explains research leader Clayton Curtis in a press release issued by New York University. "In this study, we used both experimental and analytical techniques to reveal the format of working memory representations in the brain."

"Although we can predict the contents of your working memory from the patterns of brain activity, what exactly these patterns are coding for has remained impenetrable," adds Curtis.

A paper is published in Neuron. Scientists measured the brain activity of participants in the study. They used functional magnetic resonance imaging (fMRI), while the participants performed visual working memory tasks.

With the help of models of the brain, scientists created a representation of the neural activity of participants. Participants viewed a monitor. And their viewing patterns were translated into coordinates within the space of the monitor. This enabled the scientists to "see" how working memory is encoded.

Participants were presented with two different visual stimuli. In some trials, the visual stimulus was a tilted grating. And in others it was a cloud of moving dots. After a delay, participants had to indicate the exact angle of the grating's tilt or the exact angle of the dot cloud's motion.

The results of the study indicate that working memory uses the same format to encode the direction of the motion of the dots and the orientation of the titled grating.

"We reasoned that only the task-relevant features of the tested stimuli were extracted and re-coded into a shared memory format, perhaps taking the form of an abstract line-like shape angled to match either the orientation of the grating or the direction of dot motion," explains Curtis. “We could see lines of activity across the topographic maps at angles corresponding to the motion direction and grating."

Generalizing, it seems that the brain discards task-irrelevant features. And it also encodes task-relevant features into efficient memory formats.

This confirms and extends a previously known feature of working memory. Visual working memory about letters and numbers is encoded using the same format as verbal working memory.

That the brain uses an efficient format to represent many different things makes a lot of sense to me. It's an optimized strategy developed over the course of evolution. And it seems likely to me that this research program will not only shed more light on how our own brains work, but also permit advances toward human-like artificial intelligence.