The University of Iowa has a collection of Chatauqua memorabilia that includes a bunch of ephemera from magicians
Brown University also has a T. Nelson Downs collection
The Providence RI Public Library holds the John H. Percival collection
of magic books and ephemera.
Warner Perry's collection
is at the Univ of Delaware.
Frederic Jessel wrote a bibliography of works on playing cards and gaming in 1892, and updated it in 1905. It included many books on magic. His collection
went to the Bodleian library at Oxford Univ.
Harvard has a magic poster collection
The Henry Evans collection
at the British Library has a good bit of magic-related stuff.
The Smithsonian has a small collection
of magic material within the Warshaw Collection of Business Americana.
Lester Lake's correspondence
is at Ohio State.
UC Santa Barbara has a performing arts collection
lists "the Harry Houdini and the McManus–Young holdings at the University of Texas at Austin’s Harry Ransom Center, the H. Adrian Smith Collection of Conjuring and Magicana at Brown University’s John Hay Library, the Will Alma Conjuring Collection at the State Library of Victoria in Melbourne, the Robbins Stage Magic Collection at the State Library of New South Wales, the Raymond Mander & Joe Mitchenson Theatre Collection at London’s Trinity College of Music, the George DeMott Papers of the Billy Rose Theatre Collection, the Curtiss Show Print Collection at the Ohio State University’s Jerome Lawrence and Robert E. Lee Theatre Research Institute, and the manuscripts collection at the Robert L. Parkinson Library and Research Center at the Baraboo, Wisconsin, Circus World Museum. The Donald C. Davidson Library at the University of California, Santa Barbara, and the Harvard Theatre Collection also have collections containing information on magic history . . ."