Answer for when a guy asks about past relationships

I recently went on a date and when the guy found out I haven’t been in a relationship for 3 years he was very shocked and kind of immediately asked me if I was upset about this issue .
The honest response is that I don’t just fall into relationships easily . I’m emotionally available but I’m concerned now that this puts men on guard. Top that off with him finding out my longest relationship was under a year and judging by his behaviour I think he sees that as a red flag .
I’m not concerned – I’ve been consciously single and working on myself and I’m the type who leaves a relationship at early signs of red flags or incompatibilities and in fact I am quite proud of myself for it darn it! Lol. There have been some guys who wanted to date me for longer and I broke it off, and of course vice versa .

I kind of just told him a brief answer without any dwelling – kind of just said that I’m been happy being single and I’ve dated in the past 3 years, just no relationships .

I was a bit caught like a deer in headlights . Does anyone have a good response for when a man questions / mildly judges a girl based on length/duration/ timing of previous relationships ?
I’m not taking it personally but kind of wish I had given a more refined answer.

Thanks all