Erm, EvilRipper is right. But this isn’t rather a limitation in the DAAP protocol. Again, since firefly was developed for the soundbridges, this isn’t a problem because they act differently than iTunes.
The problem is that iTunes downloads THE WHOLE DATABASE upon connecting.
You can speed this delay up by removing all the playlists and smart playlists, then you effectively limit it to the Library itself. 😉