"I was intending to go to the library."

It certainly doesn't bother him to find someone inviting himself along. It wasn't what he was planning, but he can easily adapt to something like this.

"I am looking for books for research purposes, but it is not an urgent matter. If I can offer some assistance, I would be glad to. On the other hand, if you wish to accompany me to the library, you are certainly welcome to."