The product backlog should contain all future work to be done.
“A product backlog is a list of the new features, changes to existing features, bug fixes, infrastructure changes or other activities that a team may deliver in order to achieve a specific outcome.” - Scrum Alliance
This is a good definition because it points to a broad target for the product backlog. The backlog is not only for new features customers, the product manager, or the CEO, are clamoring for. It is really about all future work that will create value for customers.
And, yes, when you eliminate bugs, you do create value by making the software more usable and satisfying. Or, when you improve your infrastructure to get better performance or faster time to market, you do increase value for customers.
It is important to remember the product backlog is not, strictly, a product management tool. This is because being a product owner is not the same as being a product manager. Your goals and audience are not the same. Surely, there is overlap, but it is not the same thing.
This means, as a product manager, you may have many more documents, tools, maps, etc to manage the product. Assume the product manager is not the product owner in the development Scrum team, because she does not have to be the product owner. Obviously, this means the product manager uses many documents (product road-map, wish lists, user research, market research etc…), and these documents are not the product backlog, and are not all replicated in the product backlog!
A good approach is to prepare several levels of documents in the product backlog.
These levels are categorized by how soon the backlog items will be worked on. Here is an example you can use the following scale. Yours could be different, but with the same idea to have a clear idea of how soon a backlog item will get worked on.
Learn more about Scrum with our online interactive Scrum study guide, or get this Scrum book.