Why is every girl I ever interacted with withdrawing from me?

Im an average looking 18yo introverted, rather passive guy with a variety of interests: Playing classical music instruments, anime, video games and programming. I already met multiple nice girls with at least one matching interest, however, even if im primarily interested in a friendship they are always withdrawing from me after a few small conversations. In irl as well as via text. Once a girl even gave me her number and I visited her, we both said that it was great and we wanted to meet again but after that she responded extremely delayed, short and rather rejective.

Idk how to improve, most of the times it feels like I didn't even had a chance to break the ice before they withdraw from me. Any ideas?