What gives

We have been back and forth for years. He told me if I ever “left” he would miss me. He told me he likes me a lot. Last night, we were joking comment about his “mistress”, and often makes joking comments about my “other boyfriends/lovers”. But told me about a month ago “I can’t give you what you want” when I brought up (jokingly) his fear of commitment. To be clear, I never asked for a “commitment” from him, we have always just gone as is. We hang out, talk about everything, know each others friends, he knows and sees my family, his parents have died, his siblings do not live around the area, so I dont know them, we know each others work colleagues… But,that comment has bothered me the last month, and I don’t want to scare him away by bringing it up. I do not understand, do you care about me as a friend or more, or am I just being strung along and wasting time? There has only ever been one other woman in the many years we have been on/off, I dated two other people. The on/off has been maybe because I too fear commitment and have said this isn’t working… But I’ve always gone back. At this point I just don’t know where we stand and don’t know what or how to say it. I don’t need the world from him. I know I’m the only one. I just need he know 100% he cares and isn’t waiting for the “one”