All files are tagged offcourse. I have the same directory structure as you:
Audio/Artist/Album/## – Title
But firefly couldnt care less about file structure. It goes only on the tags. What I mean is the applet does not have a column “Track nbr” to sort by.
It has to do with how the applet present the files, so it is in all means a matter of the applet and not firefly in general.