Boyd’s directory of Washington & Georgetown (1867) - hot off the scanner! In it we find a listing of local libraries (and asylums) including:
Smithsonian Institution Library : This Library now contains upwards of 40,000 volumes…besides books the Librrary [sic] contains engravings, maps and other articles connected with the art of printing, but it is especially devoted to scientific works….In this speciality, it is the richest in the country. This Library has been transferred for safe keeping to one of the fire-proof wings of the Library of Congress.
Starting in 1866, after the Smithsonian had not one but two tragic fires, the Library collections were temporarily stored at the Library of Congress in their state-of-the-art fireproof rooms. Eventually we got them back.