Yea, I agree the AoA vane is a great idea.
I know someone who built one by shining an LED through 2 polarized films, one attached to the axle of a vane and the other film was stationary. There was a light detector on the other side of the films from the LED and the light intensity varied as the angles between the polarized films changed from perpendicular to parallel. I don't remember if its a linear relationship between angle and light intensity, but it was a cool system with only friction form the bearing. Since you want the vane to be as small and light as possible, the forces can be small and the friction from the potentiometer becomes important.
Another good "non contact" measuring system is the AS5043 magnetostrictive sensor (they give free samples of chips and magnets too!).
http://www.austriamicrosystems.at/Products/Magnetic-Encoders/Rotary-Encoders/AS5043 Depending on the size and accuracy you want, balancing the vane statically and aerodynamically can also be challenging, but I bet you could something functional with a little tinkering.