Guy told me that the "connection on his end is only physical." Please help.

I’ve [F,24] been talking to a guy [M,26] for two months now and we hung out once last week. It went really well. We swam in his pool, got pizza, watched movies, I stayed the night (we did not have sex), he made my breakfast in the morning, and we swam all day the next day. Over the weekend things were normal, he message me like normal. Then around Monday this past week I’ve noticed he was distant. He didn’t talk to me on Monday or Tuesday. In the times we message and FaceTime we don’t talk about sex and mutually agreed with each other that we shouldn’t. So I’m having a hard time wrapping my head around this. He’s told me point blank that he likes me not just for my body but likes my personality as well.

He messaged me this morning and said, “So things have been a little distant between us the last few days, and it’s got me thinking that our chemistry was probably more physical on my end. Wanted to be up front about that. You’re a really great girl (and obviously smoking hot), but I just don’t know if that connection is there for me. Sorry, I know that sucks to hear.”
So I said, “that does suck because I feel like hanging out with you went really well.” And he replied, “I think it did and it’s not like I had a bad time or anything, I just think on my end it’s more a physical thing which is kind of what I was worried about before. You didn’t do anything wrong.”

I don’t know what to do with this or how to even respond to this. I haven’t yet. I’ve never been in this situation before. I feel really led on.