When you complete a quest, you will receive some bonus experience, and might also receive some kind of other compensation. This can be in the form of coin or an actual item. Frequently, you will have your choice of rewards, and have to select one before the quest will actually be considered complete. Most frequently, these items are pieces of gear.
Occasionally, you may also receive reputation from finishing a quest, or gain access to new content that wasn't previously available to you.