Sometimes nature just throws you a loop. All your carefully laid plans, all your exquisite calculations, all your deeply held beliefs and expectations get blown away in the simple eloquence of real data from the real world. That is how Dark Energy made its appearance into the world of cosmology. Its not just that folks weren't expecting it. They were, in fact, expecting the very opposite.
Last week I explained how Dark Matter was "discovered" (inferred really), based on observations over decades of the gravitational influence it exerts on matter we can see (the stuff we are made of). Dark Energy was discovered in a similar way, except that it arrived all at once in one big, fat surprise package.
The year was 1998 and two highly competitive groups of astronomers were each rushing toward the same goal: they hoped to hunt down the effects of gravitational braking in the universe. Ever since astronomers had accepted the idea of the Big Bang, they had been out hunting for its subsequent cosmic deceleration.
The idea was simple.
While the Big Bang blows space apart (it literally stretches all points of space-time away from each other), the gravitational pull of matter should, over time, slow down that initial burst of cosmic expansion. The two research groups, (Berkeley vs. Harvard), were racing to find the magnitude of deceleration in the universe. It was a critical project since the rate of cosmic braking is directly related to the total density of mass (and energy) in the universe. It would be a Nobel Prize-worthy result.
Things didn't go quite as planned.
"I was, quite frankly, denying [it] was happening," Harvard's Brian Schmidt reportedly said. As data was gathered and analyzed, both the Harvard and Berkeley groups were stunned to find no evidence for deceleration. Instead, everything pointed in the opposite direction.
According to observations, the expansion of the universe was speeding up — it was accelerating. After exhaustively checking and rechecking their data, both groups bit the bullet and announced their results. Cosmic acceleration became big news.
Where does Dark Energy appear in all this? As Newton showed 400 years earlier, accelerations need forces. And, as the physicists of the eighteenth and nineteenth centuries demonstrated, forces need energy. The discovery of cosmic acceleration meant that space was being forced apart and must, therefore, be pervaded by a new form of energy acting as "anti-gravity." While Newtonian gravity only produces attractions, Einstein's more complete description of gravity — as the shape of space-time — demonstrated that repulsion and gravity could go hand-in-hand.
To make his early, static models of the universe work, Einstein had once proposed the existence of a space-filling repulsive energy. Cosmic expansion (discovered by Hubble in 1928) forced him to reject this "cosmological constant" as his greatest blunder. In 1999, however, the newly discovered cosmic acceleration made it clear that some form of anti-gravitational energy (perhaps similar to Einstein's cosmological constant) had to exist. As nothing was known about this energy, it too was called "dark", even though it and Dark Matter were entirely unrelated.
The discovery of cosmic acceleration and Dark Energy upended cosmology almost overnight. In spite of the community's incredulity, further studies, including studies of cosmic geometry, gave new support for the reality of Dark Energy. Like it or not, this unanticipated form of anti-gravity was now a powerful actor on cosmology's stage.
"Like it or not;" that is the key phrase. The history of science is stacked high with ideas and discoveries nobody was expecting, or even wanted. From the discovery of the muon ("who ordered that?" asked physicist I.I. Rabi) to climate change (the ultimate inconvenient truth), we don't get to dictate to nature how it should behave. The greatest and strangest beauty of science is, in fact, this constant reminder of just how wrong we can be.
You can keep up with more of what Adam Frank is thinking on Facebook and Twitter. His latest book is About Time: Cosmology and Culture at the Twilight of the Big Bang.
Copyright 2021 NPR. To see more, visit https://www.npr.org.