Servers operate under more stressful conditions than consumer PCs and are expected to be more reliable, so server-grade parts are built to tougher standards. They will also often times have special features and ports that drive up production costs.
Building a server doesn’t have to entail using these pricey parts, however. Many people integrate consumer-grade parts into their systems with no problems at all. Often times the price of server-rated hardware is inflated and the parts themselves are not always much better than their desktop counterparts.