I agree, it still is a rare car compared to the number of automatics that they sold.
It's main competition in 1965 would be the Impala SS and if you want to compare the number of SS four speeds built vs Sport Fury 4 speeds, you would say quite rare for the Fury.
If you flip it the other way...