While he was in medical school and working on his thesis, Billings realized the need for a great medical library and the need for a comprehensive catalog and index of books, periodicals, journals, etc. He increased the size of the surgeon-general's library from 600 volumes in 1865 to 50,000 in 1873. He began Index Medicus, a monthly guide to current medical literature, with Robert Fletcher in 1880 which ran 16 volumes by the time Billings retired from the library in 1895.

In 1873, he drafted plans for organization and construction of Johns Hopkins University Hospital and recommended the pavilion plan for the construction of the hospital. This type of plan was soon used across the country. Billings wrote many reports dealing with hospital administration and the training of hospital personnel. He served as medical adviser to the Hopkins estate trustees in which he played an important role in determining the organization, philosophy, and faculty of the Johns Hopkins Medical School.

He supervised the compilation of statistics for the U.S. Censuses of 1880 and 1890. He influenced Hollerith in the development of punched card system using mechanical methods of calculation of vital statistics which were successfully used in the Census of 1890. Billing's plan was used when constructing the New York Public Library. While the library was being built, Billings headed the reclassification and recataloging of the books from the Astor, Lenox, and Tilden Foundations which were consolidated in 1895 to form the New York Public Library. He persuaded Carnegie to give $5 million to help fund the establishment of this library and its branches.