Gems are used where real value comes into play...
Gems are rare, and should be spent with care, while you normally won't have a hard time getting gold.
You cannot store gems anywhere, nor trade them or put into your Dwelling.
Also they cannot be stolen from you, except from events (i.e. Hanzaki's Thieves).