Hi Mark,
I feel you. I think about this kind of stuff all the time too. I often analyze what I'm doing, what's working, what's not working, how people respond, and then I try to make myself better. Perhaps record yourself and ask some people you trust to give their opinions. I posted my music here on SZ and I got some good constructive criticism that I do reflect on. When I see others performing perhaps in venues or events that I would also like to do, I try to figure why they have gig and I don't. It can be a lot of things... Political, maybe they have better a hustle, or know the right people. Maybe they are better entertains, play a more popular style of music or are better musicians. I just try to consider this stuff and then keep working at it and try to make any changes.
As for the old act that you saw that wasn't too hot but the crowd seemed to love, I would have to assume that the have been doing this act forever and they have some loyal fans. If the Stones or Sinatra have (had) a bad night, or even consistently perform a very low standard, people or still going fill venues to see them.
Hang in their Mark, it's hard out there. I know I'd probably make more money if I dedicated all the time that I put into music into some sort of more lucrative field, but I can't. Music has chosen me. I'll keep cracking away at it until I can't any more. It doesn't matter if I'm gigging daily or not at all. You got to love it.