My recommendation is to start organizing it, and keeping track of it, now. Manage your bibliography in a manner that not only allows you to know what sources you already have, but in a way that allows you to readily search it for quick details that might let you know that a book in your possession already has some of the information you need for a new project. Make notes about each item that can assist with this.
There are amazing reference managers out there that are free, such as Mendeley, which I personally love. ( awanderingelf.weebly.com/blog-my-journey/mendeley-for-the-sca-researcher ) But even a spreadsheet would serve the purpose here.
It is much, much easier to start early, than to start 10 years into it (which is where I am at, and trust me, it is a painful place to be). I am working on a bibliography (with a focus on Norse textiles and costume) now, that I will eventually share publicly, but I am only 200 items into it and have many, many more to go.
So yes, it is in your own best interest to start early!
(As I work on this eternal-seeming project, I might be posting some book reviews here, so keep an eye out for those as well!)