The Worst Theoretical Prediction in the History of Physics
When it comes to scientifically mysterious concepts that begin with the word "dark," dark matter attracts most of the public attention. Dark energy, however, constitutes 68.3 percent of the mass of the universe compared to dark matter's paltry 26.8 percent (and normal matter's minuscule 4.9 percent). It is truly the more consequential of the two "dark" concepts.
Yet we'll never likely be able to "catch" a particle of dark energy as scientists are striving to do with dark matter. That's because dark energy is – most likely – just the energy inherent to space, itself, perhaps arising from Quantum foam, composed of virtual particles that flit in and out of existence. As Einstein reminds us, the energy delivered by these virtual particles briefly protruding into space has mass.
When astronomers attempt to measure dark energy's density in space, they come up with roughly 10^−9 joules per cubic meter, a microscopic but influential amount. However, this observed value, known as the cosmological constant, isn't remotely close to that which is predicted by the time-tested quantum field theory. As detailed in the textbook General Relativity: An Introduction for Physicists:
"The simplest calculation involves summing the quantum mechanical zero-point energies of all the fields known in Nature. This gives an answer about 120 orders of magnitude higher than the upper limits... set by cosmological observations. This is probably the worst theoretical prediction in the history of physics! Nobody knows how to make sense of this result. Some physics mechanism must exist that makes the cosmological constant very small."
Either that, or quantum physicists' understanding of fields, and by extension, the entire universe, is fundamentally flawed! This seems unlikely, however, as boatloads of observational evidence supports current theories.
An so, quantum physicists are in a bit of a pickle. Making slight tweaks to theoretical estimates narrows the ginormous gap considerably, but our best theories from quantum mechanics still overestimate the influence of dark energy by sixty orders of magnitude.
"Yes, we're clueless," astrophysicist Neil deGrasse Tyson admitted in his recent book Astrophysics for People in a Hurry. "But it's not abject cluelessness. Dark energy is not adrift, with nary a theory to anchor it. Dark energy inhabits one of the safest harbors we can imagine: Einstein's equations of general relativity."
The observed energy density of empty space fits wonderfully within the standard model of Big Bang cosmology, which satisfyingly accounts for the cosmic microwave background, the abundance of basic elements, and the acceleration of the expansion of the universe. At the core of the standard model lies general relativity, one of the most successful theories ever devised. So in the battle of dark energy vs. quantum mechanics, dark energy unquestionably has the upper hand.
Originally published on RealClearScience.
