Sure, this is all simply 3-D geometry, I just haven't found a good write up of it, and so I've been working it out on my own.
A few useful things I've convinced myself of: the sun's path through the sky on any given day is a portion of a circle on the sky hemisphere. [This is for a model that ignores the earth's orbital movement during the day.] That circle is a great circle (meaning the center of the hemisphere is the center of the circle) precisely on the equinoxes.
That also means that on the equinoxes, you do want to point your solar panel at the elevation of the sun at solar noon. That's because all the sun vectors you are averaging lie in a single plane, so their average (or integral) does as well. And since the average will be due south, it must be due south at the sun's noon elevation. My earlier argument breaks down on the equinox because it is not the case that lowering the panel elevation slightly helps near sunset or sunrise; a slight deviation from solar noon elevation is a second order effect for the entire day on the equinox.
But on other days when sun's path is not part of a great circle, then the argument still applies. I'm still working out the details.
Cheers, Wayne