Ugh i have this issue! Except i'm also torn between loving the smell of old books. So i want to smell them, but not touch them lmao.
The worst for me are the ones in a plastic sleeve, especially when they've yellowed at the corners, ICK!!
Anyway, just keep telling yourself that the likeliness of catching something from a library book is pretty damn slim, and that if the books were "dirty", they wouldn't be out for public consumption.