Like sneezing, goose bumps (also known as the pilomotor reflex) represent one of your body's automatic responses, meant to increase your chances of survival in the harsh world.
Cold environments and strong emotions (like fear) are both known to give your skin the texture of plucked poultry. When the muscle fiber connected to a hair follicle tightens, the skin surrounding the follicle puckers into a goose bump, pulling the connected hair straight up.
One effect is to generate warmth: straightened hair traps a layer of air against the skin, insulating the body. Unfortunately, human hair is so thin and short as to render the reflex virtually useless, but in hairier mammals goose bumps don't just look silly. In fact, a cat or mouse's battle-ready stance is related to our own pilomotor reflex. In their case the muscles are responding to perceived threats by making the animals appear larger.