Hi Astrea,
When I read it, I thought, this works fine as a single para, so long as it's just the one person speaking. Which it turns out it is. But unless it's crystal clear from what's gone before, I'd attribute the first speech. Then keeping the rest in a single para will make perfect sense to the reader.
I.e. ' Of course you do,' he says. Something is gathering... And then continue the layout exactly as you had it.
If you break it up, especially if you isolate lines of unattributed speech, it would be very hard to work out who says what. Also, technically it is incorrect to break up lines of speech by the same person. Speech and gestures all from the same person flow in unbroken lines, just as you instinctively had it. Though if you had a brilliant reason for isolating them, all rules dismissed, as ever.