This guy I like has been kinda flirty with me. For example, he touches me a lot (on my hand, shoulder, even on my knee once), has given me compliments, asked about who I am living with, etc. And he also started giving me hugs. Only thing is, the last time he hugged me he was patting my back. I thought that means the person wants to END the hug?! Is this a mixed signal or am I reading too much into it?
Yes, you are reading too much into this. The answer is that it doesn’t mean very much at all.
If you want to ask What does it mean when a man does something like touches the small of my back or caresses my face, the answer is Not much, stop reading into it.