That is weird.
First of all, cut the swap file. I’ve got about 8K songs and firefly runs just as fine without swap too.
Secondly: What gives you those numbers? ps? top? meminfo?
Other than misinterpretation of numbers I could only think of source file that breaks the meta data scanner so bad, that goes OOM.