Welcome to Madtown. There's more here than is immediately visible. But cars do help.
There are are a handfull of magicians in-town who regularly work bars, resturants and do local shows. Many of them are listed here: MadMagicians
In particular, Tom Krzystof
is very active--and very good--and has his shows listed on his website. (Click his name to link.)
In addition to Madison, there are regular magic shows and lectures in Wisconsin Dells, Janesville, Milwaukee, and, of course, Chicago.
There are a good number of casual groups that get together. Most of them are very
casual and meet when the spirit moves them. Most of them also have their genesis out of the local IBM ring.
That being said, Wednesday evenings at State Street Brats was a common gathering place/time for much of the past year for the younger--but not too young--set. (There be beer, there.) The Union and Union Terrace have been host to much Gemutlichkeit (how do I do an umlaut in here?) and magic over the years. At the moment, I don't think there is a regular group meeting there. But there should be....
My best advice is to figure out a way to get to the next IBM meeting. There's usually a range of age and gender at meetings. J.T. Whitney's is close to a bus line. There are usually a couple of cliques going, depending on your interests.
There are no magic shops in town at the moment. But the University Book Store carries a fairly large selection of magic books, DVDs, and stocks Genii
in the magazine section.
If you are serious about starting a group for younger folks and would prefer an alcohol free zone--sometimes difficult to find in Madison--call the University Book Store and ask to speak with Erik.
I know there are other Madisonites who lurk and post here. Hopefully they can add to the limited scope of my reply....