Well, I’ve playing around with this a bit more, and I think the issue may be that all the samples I have are not fast start. I tried installing the server on my Mac (incredibly easy… great job with this), and I have the same problems as before… audio but no video. They play great from iTunes directly, but not from either of the Firefly media servers (Ubuntu or OSX).
I’ve tried the qt-faststart script, and MP4Box on a variety of video clips and neither of those did the trick. Is there a good source for known working samples, or do I just need to bite the bullet and purchase Quicktime Pro?