You could try "Your Dekalb Farmers Market". They have meat market in there and I think they sell goat. If you decide to get one from there let me know how it goes. If you're looking for a live one then look in yellow pages for small farms that sells live goat... (you may have to make an appointment with good butchers and their turn around time might vary, also don't forget to mention how you like your meat cut/packed..etc).

good luck! keep us posted on good findings.