Executive Summary: It depends on how much money and how much diving.
Long Answer: I am a scuba instructor and have bought a lot of gear over the years. My best advice to you is to start buying your own gear if you have the budget. If you are only going to be diving once a year on a vacation then renting would be more cost effective, However, if you have the budget the best overall solution is to own your own gear.
I would probably start with a good quality regulator and then a BCD. The exact order is just personal preference, but that is how I did it. I bought a good quality APEX regulator (ATX200) and am still diving with it after 8 years of lots and lots of diving. I do my fair share of cold water diving so that is a factor that I considered in buying the APEX. If you are going to do cold water diving then you should look for an evironmentally sealed first stage. Good options in this range would be Poseidon, Scuba Pro, Apex etc. Poseidon is probably the top of the range with cold water diving. If you are only going to do Caribbean diving then Apex ATX50/100 or the Scuba Pro MK25/S600 are very good options.
Remember that you HAVE to have your regulator serviced every year - especially if you are doing alot of salt water diving. Also when you come back from your vacation, soak the second stage and alternate airsource (octopus) in fresh water for a few hours to get all the salt out.
BCD: There are a lot of options with BCD and again it depends on what kind of diving you will be doing. If you will be doing cold water diving you will need a BCD with more lift capacity than for warm water diving (because of the extra insulation you wear). You have choice here with back inflate and jacket type. For a new diver I would probably suggest a jacket style. Look for a BCD that is comfortable for YOU. Try them on and if possible get into a pool and see how they feel. One thing I like is pocket space for extra stuff like signalling devices, lights, and extra knife etc. Most of the brands are good. ScubaPro Classic Plus could be a place to start.
Some divers like to have an integrated airsource - meaning that you do not need an separate alternate air source. The AAS is integrated into the inflator hose for the BCD. Some divers like this since it eliminates an extra hose. The downside it that when you have to donate air to a buddy you have to donate your primary airsource and use the AAS.
Fins: Personally I love the ScubaPro split fins. I find that they are super comfortable and durable. However, I would recommend that you try out various options and see what works best for you.
Computer: If you are going to be diving alot then I would highly recommend a computer. They are really affordable and provide alot of benefits. Some of the considerations: Nitrox capable, log book function, download data to your computer. Again budget will determine what you get, but a good starting point could be Uwatech or Suunto. However opinions vary and very diver has his/her favorite. Personally I like the air integrated types that monitors the air pressure so that it keeps track of my air consumption rate and predicts how many minutes of air I have left at the present rate of comsumption etc.
Hope this helps. Please ask any more followups if not sure.