Yep, running it through the mp4creator -optimize fixes it. Now that I know what to look for of course I found a whole bunch of files which have the same problem… Gross.
I wonder whether I can just simply run this program on *all* my m4a files instead of hunting for the particular ones that have this missing fast start layout. I am nervous about making itunes angry…
Thanks for helping out with identifying this culprit.