Why do we see colors that aren't there?

Has an optical illusion ever prompted you to see colors that weren't actually there? Or have you wondered why the infamous photo of "the dress" was perceived as white and gold by some but blue and black by others?

In essence, how can colors appear different from what they actually are?

In some cases, the answer has to do with lighting; in others, it depends on our memories or what our photoreceptors are doing, experts told Live Science.

In 2015, a photo of a dress sparked a heated debate with a simple question: What color is it? "The dress was so unusual; we really don't have very many controversies about colors," Bevil Conway, a neuroscientist and visual scientist at the National Institutes of Health in Maryland, told Live Science. "We don't disagree about white and gold or blue and black. The disagreement is about whether or not those colors applied to this image."

Conway and his team analyzed the dilemma by asking 1,400 participants what they thought the color of the dress was if the illumination was changed. They found that people's expectations of what kind of lighting the dress was in affected what color they thought the dress was. People who assumed that the dress was shot under a warm or incandescent light thought that the dress was blue and black (its actual color), whereas people who assumed cool or daylight saw white and gold.

The finding showed that people's expectation of the object's surroundings influenced their color perception. 

Related: Why can't we see colors well in the dark?

Memory can play a role in the way we see color. When we view a familiar object, our brains assign it its expected hue or even enhance its color. 

In a 2024 study, researchers asked study participants to bring colored objects to the experiment. The participants were then asked to identify the color of the objects under different room illuminations that would make the objects appear as follows: 

Despite the different lighting conditions, the participants had no issues identifying the objects' original colors. This effect is called color constancy.

This memory color effect also explains why you tend to "see" color in the dark even though there is no light stimulation: It is likely that your brain is constructing color based on a memory. 

On the other hand, when the object is unfamiliar, your brain can assign color based on what you expect the object to look like. The following image of a train was created by Akiyoshi Kitaoka, a psychologist at Ritsumeikan University in Japan. The train does not have a blue pixel, although it may appear that way to some people. 

In other scenarios, an objects' positioning or context can make some colors appear more intense than they actually are. For example, a red object appears "redder" on a green background than on a white background. In other words, neighboring colors can change how we perceive certain hues.

Tired photoreceptors

Occasionally, cones, or color photoreceptor cells in the retina that turn light into signals the brain can interpret, can trick the brain into "seeing" something that's not there. 

Stare at this flag for 30 to 60 seconds, and then direct your eyes to a white space. What do you see?

For most people, the flag's afterimage, or a vivid image retained after the object is removed, would appear red and blue on a white background. This is because our photoreceptors can experience fatigue.

Related: Can cats really see in the dark?

Most people have three types of color photoreceptors, or cone cells, which are named according to the wavelengths they detect: long, middle and short. The "long" and "middle" cone cells are best at perceiving light at the yellow and green wavelengths of the visible spectrum. Meanwhile, the "short" cone is best at catching 'lavenderish' or violet light, Sara Patterson, a neuroscientist at the University of Washington in Seattle, told Live Science.

Our cone cells operate like muscles and can get tired, Conway explained. 

For example, when we look at a red sheet of paper (which has a long wavelength), the long cone works harder than the rest of the middle and short cones. If, after we've stared at the red paper, we turn to a white sheet of paper, the middle and short cones will compensate for the activity of the long cone and create a perceived green color. This color illusion is called a negative afterimage, or an illusion of the object's complementary color. In contrast, the eye can also see an image that is of the same color as an object that no longer exists. This color illusion is also known as a positive afterimage and is usually at a much briefer timeframe. 

The same effect doesn't happen with a white sheet of paper because white contains all of the wavelengths in the visible light spectrum. When we look at white paper, all three types of cones are stimulated equally. Over time, the long, middle and short cones all grow weary to about the same extent. 

There is still much we don't understand about how our brains perceive color. "Most challenging is definitely 'where' it is happening in the brain," Patterson said. We still don't understand how or which neurons are in charge of comparing the activity of cones in the retina.

To make headway in understanding color perception, "we actually need much more productive dialogue between different branches of intellectual activity," Conway said. This includes art, philosophy and science. It is much more than just the visual, he said.