1. Comfort.  You won’t want to be chafing a few miles into the bike or run.
2. Aerodynamics.  Any clothing choice should be tight-fitting, to minimize drag on the swim and bike.
3. Pockets.  In longer races, you’ll probably want pockets to hold the miscellaneous stuff you’ll need to carry.
4. Color.  This is usually personal preference, but lighter colors tend to be cooler than darker colors (and may be more “revealing,” if you know what I mean).

Once you’ve thought about these factors, you can start to look at your specific options:

1. Tri shorts and a tri top.  This is the most common option for triathletes.  It’s comfortable in all three disciplines, aerodynamic on the bike, and often come with pockets for the run.  The shorts have a minimalist pad for some comfort on the bike without getting in the way on the run.  Most triathlon clubs sell their tri kit in this form, and often it’s a cheap and simple option for new racers.  Remember to leave your tri top in transition before the swim, and put it on in T1.
2. Trisuit.  This garment is similar to the above, except it has a one-piece construction.  These have the advantage of slimming your midsection if you’re carrying a little extra in that region (who isn’t).  Check the neckline if you’re buying one–a high neckline will be more hydrodynamic while a low neckline will “scoop” in the water.  Many of these suits do not have pockets, which can be an advantage or disadvantage, depending on the race distance.  I do all of my races in trisuits, mostly because I have one that makes me look kinda like Superman.
3. Sport-specific attire for each sport.  Especially in longer races, you may find that wearing a swimsuit for the swim, bike clothes on the bike, and run clothes on the run is more comfortable and efficient.  You lose time in transition performing a complete clothes change (and in some races, nudity regulations prohibit this), but you may gain time in the end because you won’t be chafing or adjusting your clothes at all.
4. Miscellaneous combinations.  Some wear tri shorts and a looser-fitting bike jersey with sleeves.  Some can tolerate bike shorts on the run.  Some wear just a speedo for the whole race (you had better be good looking to pull that one off)!!

Your best bet is to leave yourself with several options on race day.  Conditions may dictate your choice of attire–for example, a wetsuit swim means you can leave your tri-top on under your wetsuit, while a non-wetsuit swim means you couldn’t wear your bike jersey on the swim.

Most importantly–when you’re done, wash your clothes as soon as you can.  Your domestic partner or roomate(s) will thank you.  Trust me.