It should go, yeah. You’ll need to use the -P flag to keep the pidfiles from stomping eachother, but that’s really it. Two config files, etc.
Re: speed, it must be your format. My testing library is ~6k songs, and it runs pretty snappy (other than initial connect with drive spun down).