Definitely stop by Tannen's, which is found at 45 W. 34th St., Suite 608, between 5th and 6th Aves. http://www.tannens.com
Also, just around the block is Cafe Rustico, where you can find folks like Sol Stone, Wesley James, etc, on Saturday afternoons between 12 and 6 or so. That's on 35th between 5th and 6th. Brian already mentioned Maui Taco, which is where you'll generally find Doug Edwards and other folks.
Finally, check out Monday Night Magic if you have the opportunity. They just recently moved to a new theater which I hear is their best place yet. http://www.mondaynightmagic.com