Why do I feel relief now that he doesn't answer?

Your relief is understandable; your instincts told you this was not THE relationship, and now that you've realized it in your "conscious" mind, you know you are free to concentrate on what's important to YOU. I wish you well.