My suggestion would be to sign up for, and take an NRA hunters safety class.
They will teach you the basics, a lot more than just safety. And they offer seminars on hunting various types of game, from jack rabbits and squirrels on up.
Get the certificate, go get a hunting license. That's the first step.
Next comes deciding what you want to hunt.
For the least amount of money, jack rabbits and ground squirrels with a .22 would be the best bet. A .22 at Wally world and some ammo, the hunting license and you are good to go for under $200.
Next up on the four footed game list would be coyotes, but only if you can pick up a center fire rifle inexpensively, as a .22 really isn't suitable for them unless you can call them in real close (under 50 yards, preferably closer to 25) and make consistent head shots. You don't want an animal to suffer because of your being under gunned or not being able to place the shot correctly.
For most folks after the rabbits and squirrels, the next step up cost wise, is usually a shotgun and start bird hunting. This one is easy to practice at, by shooting Skeet, Trap and Sporting Clays. There are several folks on this forum who can teach you to do that, myself included. And there are lots of game birds to be had in this area, Dove, Quail, Chukar, Pheasant, Turkeys, Ducks and Geese. And they are good eating too!
A good pump shotgun which will serve you for all types of bird hunting, and could, with slugs be used for deer, and will also serve for home defense, will set you back $200-$500 depending on what you buy, and if its new or used.
You can also find a good side by side double barrel, or an over and under in that range as well, but whatever it is, it should have screw in chokes so that you can change the choke to suit the game being hunted.
Open patterning chokes (Cylinder bore, Improved Cylinder, Light Modified) for doves and quail, Improved Cylinder, Light modified and Modified for chuckar, Modified, Improved Modified, Light Full, Full and extra full for the Turkeys, Ducks and Geese.
Depending on the brand, the chokes can be pretty cheap. Buy a Remington 870 or a Mossberg 500 and the choke tubes will run you only $12-$14 at Wally world.
From there you move on up to Deer, Elk, Antelope, Bear, Sheep, etc. and some big bucks for gear, the tags, the trip, etc.
Finding someone on this board to take you hunting for the first few times shouldn't be too much trouble I wouldn't think!