You are here: Home » Topic » Preventing full reload

Preventing full reload

This topic contains 19 replies, has 7 voices, and was last updated by  rpedde 11 years, 10 months ago.

Viewing 15 posts - 1 through 15 (of 20 total)
  • Author
    Posts
  • #972

    NeomaD
    Participant

    Sometimes, whem the connection between my NAS and the computer running firefly is broken. Eg. when i turn them of over night or so…

    When restarting FireFly does a full database reload (about 30min – 1h) (15k songs).

    Why?

    Can I prevent it? So it starts with the old database?

    /Regards Martin Stegmark, Sweden

    #8318

    stretch
    Participant

    The latest nightlies have the option to disable the initial start up scan.

    I’m assuming that firefly is on the PC & music is on the NAS?

    Also, what’s your rescan time set to?

    #8319

    fizze
    Participant

    There’s more.
    You can specify an option that will surpress the DB vacuum that is done during the init. This along with no startup-scan tremendously speeds up the startup time on NAS’ 😉

    I believe it was startup_scan = 0 iirc

    #8320

    rpedde
    Participant

    @fizze wrote:

    There’s more.
    You can specify an option that will surpress the DB vacuum that is done during the init. This along with no startup-scan tremendously speeds up the startup time on NAS’ 😉

    I believe it was startup_scan = 0 iirc

    It’s general/quick_startup=1 to suppress the vacuum.

    That is, in the config file add the line “quick_startup=1” in the [general] section. It can’t be added via the web interface, it’s a super secret double-probation config entry.

    The other option is the “skip first scan” option you can set in the web config.

    #8321

    NeomaD
    Participant

    Ok, I do not run the latest nightly, im using the latses stabele release.

    It seems like after I added quick_startup=1 it does not scan the files at startup…

    ..right now its doing a forced full scan because it erased the old songs db at last startup…

    I could not figure out how to add it via the webinterface…
    …but notepad and mt-daap.cfg worked!

    Ill be back if this dont do the trick.

    //Thx for the help so far. Martin Stegmark

    #8322

    NeomaD
    Participant

    Why do Firefly continue to do full reloads?!?!?

    The songs.db is 19Mb… why do it flush it and start all over again???

    Itr does this when my NAS is offline. How do I prevent it???

    It should just tell me:

    “Your songs db looks really great! But I cant find the media” Thats the kind o messege I want when it dont find the mediafolder Ive specified…
    …now it flushes the whole db and start over again and it takes about an hour before I can listen to my songs…

    And, by the way, it dont sees all the songs in the dir…
    …is this a known issue?

    Plzzz tell me that this is going to get around, I DONT want to get stuck with mediplayer or iTunezzz

    Regards Martin

    #8323

    fizze
    Participant

    What NAS are you using?

    Ever thought of running firefly ON your NAS?

    #8324

    NeomaD
    Participant

    Im running a LaCie ethernet dic mini (not the Gigabit version that hava a built in media server).

    Yes, I have been thinking of that, but even if I come to that conclusion, Firefly should STILL NOT just flush the db because it cant find any media in the place ive specified.

    It should report the error and ASK me if it should flush and scan the area again…

    //Martin

    #8325

    rpedde
    Participant

    @neomad wrote:

    Yes, I have been thinking of that, but even if I come to that conclusion, Firefly should STILL NOT just flush the db because it cant find any media in the place ive specified.

    Of course it should. If you don’t have any media in the place you’ve specified, then by definition it isn’t there. It can’t play if someone connects anyway, so there is no point in having it in the library.

    If you don’t want it to work that way, then don’t start the server when the music isn’t there. Set the service up to run manually, and only run it when you’ve turned on your nas, or use the switches listed here to defer scanning.

    I’d also take a patch that works across all platforms, including headless ones, to make it work in a manner you feel more appropriate.

    — Ron

    #8326

    blamm
    Participant

    Hi.

    I disagree. If a large percentage (configurable?) of the files are missing then clearly something may be wrong and Firefly should ask first before clearing the DB.

    #8327

    rpedde
    Participant

    @blamm wrote:

    I disagree. If a large percentage (configurable?) of the files are missing then clearly something may be wrong and Firefly should ask first before clearing the DB.

    Again, I’ll take that patch. I imagine that if you start working on it you’ll see the difficulties inherent in it, particularly for those machines that are headless, or embedded, or basically aren’t windows machines.

    And for what gain? It doesn’t change the fact that the server doesn’t work when someone tries to play a song that doesn’t exist anymore, and in fact works no better than the solutions (skip first scan & quick startup) enumerated above.

    And even if it were to work spectacularly and simply, it violates the dictum of mechanism not policy. I don’t want software that stops me from shooting myself in the foot. Someday I may just want to do that. If I wanted software that stopped me from doing what I wanted, I’d just run windows and WMC and be done with it.

    #8328

    S80_UK
    Participant

    @rpedde wrote:

    If I wanted software that stopped me from doing what I wanted, I’d just run windows and WMC and be done with it.

    Fantastic! Can I have that for my email signature?

    #8329

    CCRDude
    Participant

    To give NeomaD a point, on Windows you could check for each drive what type it is, and Windows could tell you its a removable drive. But then, that’s only Windows, and to finally take that point away again, what about CDs you had in the drive only once etc. 😉

    A general idea for speeding up such rescans would be to not remove songs from the database, but flag them, in a new boolean field (“filexists” or something), as N/A. Any functions processing the database should then ignore entries where this is set (a simple additional parameter to the queries). This way, on a rescan it would only have to remove the N/A flag instead of parsing the same file again.

    That combined with a “lastseen” database column could allow to only remove entries finally from the DB after time X, e.g. when the songs haven’t been seen for a week or two, making accidental short outages of an external or network drive less time-consuming on the rescan.

    As Ron said: if you feel this is important, feel free to provide a patch for that 😀

    PS: My song database is not even half that big, for more files. Are SQLite2 databases three times as big?

    PPS: One occasion where its normal that such a large number of files “disappear”: if you retag your files and change your directory and/or filename structure, for example.
    For a server application, “asking the user” about decisions is always bad (unless its when the user aka admin actively is configuring the server), since who knows if he’ll access the user interface the next day or three months from now? I can go for weeks without looking at Firefly, and if it wouldn’t find new files I moved to its network drive, I would think something is wrong actually, while it just sits there and waits for me to give it an answer what it should do about those missing files…
    Hmmm. One way to combine the last situation with my example above: when > 50% of all files are not found, 1. set the N/A flag, 2. set a global flag that would, if you access the web interface, show you the “problem” and allow you to remove those entries with one click (contrary to the “after time X” described above).

    #8330

    NeomaD
    Participant

    Ok, I get the messege. Ill try to not start the server when my NAS is offline.

    I was just thinking, Its allright that it starts over again if the media isnt there, BUT if it cant contact the place where the media should be eg D: or my NAS, then it maybe should react in another way.

    And my second question still is a mystery? Why dont it index all my songs??
    Is there some advice for me?

    Thx again for great job with the server and answering my questions!
    //Martin

    #8331

    rpedde
    Participant

    @neomad wrote:

    Ok, I get the messege. Ill try to not start the server when my NAS is offline.

    Sorry. Just hit me wrong somehow. This is software I’ve been working on for something like three years, off and on. Mostly on. I spend somewhere around 12 to 15 hours a week answering forum posts and emails. I *try* to do some coding every day on it (although forum volume lately has slow me down significantly).

    My point is that I have a lot of time and effort invested in this, and I take it very personally. So when I read your post, I somehow internalized something like: “This isn’t even good enough for free software. It sucks. You suck.”

    I realize you probably didn’t mean it that way, but I manged to spin myself up a bit. Sorry. So perhaps I was a bit snippy. 😳

    I was just thinking, Its allright that it starts over again if the media isnt there, BUT if it cant contact the place where the media should be eg D: or my NAS, then it maybe should react in another way.

    Maybe. But it’s a much harder problem that it appears. In the case of a windows machine, maybe it’s fairly easy. But to generalize that to a headless nas box sitting in a closet. Thats harder.

    Plus, my point about mechanism versus policy meant that you never know what the right thing is.

    What if someone were using it to plug and unplug an iPod. Such that when they plugged in the iPod, it indexed it and the songs were available in the music server. But when they undocked it, they disappeared. It would be a misfeature to keep the songs around in that case, or to bug them about whether or not they really wanted to remove the files.

    Or (like me), some people might have an “incoming” folder, where stuff gets put when it gets… uh… ripped from an appropriately purchased CD, and digitized only for personal archival use, or other fair use. (eek!)

    Once it gets tagged, arted, and verified, then I move it into my collection. But I don’t want songs I delete (or move into my main collection) still in the database from their old location in the /incoming folder.

    So to me, mechanism versus policy would suggest that the computer doesn’t know how you are using it, and that you have to make that decision yourself. In this case, a reasonable thing to do might be to suppress scanning at all, and only scan when you know the device is online and has updated music (skip_initial set to 1, scan_interval set to 0, scan via the web interface). Or use a script to determine when the disk goes online, and start the server and kick off a manual scan. Then stop the server when the disk goes offline (again, via a script).

    Or any number of other solutions. But I don’t like solving problems based on a single use-case that isn’t shared by everyone else. That’s all I was really getting at. (plus extra snippiness, of course).

    And my second question still is a mystery? Why dont it index all my songs??
    Is there some advice for me?

    It’s either that the user the server is running as doesn’t have permissions to the files in question (hint: change the user it runs as in control panel -> administrative tools -> services -> firefly media server -> properties -> Logon)

    or…

    It’s a file type (.flac, .ogg, .wma, etc) that isn’t in the “extensions” config variable.

    Thx again for great job with the server and answering my questions!
    //Martin

    np. Sorry to be a dick. Didn’t really mean to, just sometimes stuff just hits me wrong. 🙂

    Peace.

    — Ron

Viewing 15 posts - 1 through 15 (of 20 total)

The forum ‘Setup Issues’ is closed to new topics and replies.