Double songs in iTunes list

[hanc61]

This evening I installed the Firefly media server (1696) onto my WHS which went without any problems. I used iTunes on one of my client PC’s with a setup whereby the iTunes database is located on the home server (shares/music/itunes) to create the database on the WHS. About 2200 songs are loaded in this database. The Firefly media server immediately appeared in my client iTunes PC’s. There is only one strange thing, all songs are displayed twice, the number reported by FireFly is about 4400 songs. I have opened the webinterface of FireFly to see if something points me in the direction of a solution. So far unsuccesfull. The only contents of the shares/music folder is the itunes directory. When connecting with the “upload” client PC I do not have double files. It seems that the scan in FireFly detects the song files twice. Has anyone seen this? What am I doing wrong and how to correct it?

[Anonymous]

I am seeing this too. Very strange. I actually have yet to get this thing working to my liking. I am running on Gentoo Linux and have installed 1586 from the tar ball. It was working for a while but I messed around 1696 and haven’t been able to get it back to normal since.

One thing I noticed is that the server name shows up twice in my Roku when I change its configuration. Does this suggest to anybody what might be going on?

Mike

[Anonymous]

For further consideration, this is what I am seeing my log files after I remove the firefly database file and relaunch:

Jan 25 20:58:07 [mt-daapd] Config option general/process_m3u has been replaced by scanning/process_playlists_
Jan 25 20:58:07 [mt-daapd] Firefly Version svn-1586: Starting with debuglevel 9_
Jan 25 20:58:07 [mt-daapd] Plugin loaded: rsp/svn-1586_
Jan 25 20:58:07 [mt-daapd] Plugin loaded: daap/svn-1586_
Jan 25 20:58:07 [mt-daapd] Plugin loaded: ssc-script/svn-1586_
Jan 25 20:58:07 [mt-daapd] Starting rendezvous daemon_
Jan 25 20:58:07 [mt-daapd] Starting signal handler_
Jan 25 20:58:07 [mt-daapd] Error: enum_begin failed (error 1): Misc SQL Error: no such table: config_
Jan 25 20:58:07 [mt-daapd] Can’t get db version. New database?_
Jan 25 20:58:07 [mt-daapd] Initializing database_
Jan 25 20:58:07 [mt-daapd] Error: enum_begin failed (error 1): ?_
– Last output repeated twice –
Jan 25 20:58:07 [mt-daapd] Full reload…_
Jan 25 20:58:07 [mt-daapd] Starting web server from /usr/local/share/mt-daapd/admin-root on port 3689_
Jan 25 20:58:07 [mt-daapd] Registering rendezvous names_
Jan 25 20:58:07 [mt-daapd] Serving 0 songs. Startup complete in 0 seconds_
Jan 25 20:58:07 [mt-daapd] Rescanning database_
Jan 25 20:58:13 [mt-daapd] Starting playlist scan_
Jan 25 20:58:18 [mt-daapd] Updating playlists_
Jan 25 20:58:19 [mt-daapd] Scanned 14475 songs (was 0) in 12 seconds_

I also noticed I had two iTunes library xml files for some strange reason which was causing problems with duplicate playlists. I cleaned that up thinking I’d solved the problem but I am still seeing 2X all my songs.

Any experts out there with some insight?

Thanks,

Mike

[rsl360]

I’ve seen the problem with duplicate entries, but I have not been able to duplicate the problem. I thought it might have something to do with reading both the songs _and_ the .m3u entries, but again, I cannot duplicate the problem. Have you tried deleting the database and restarting mt-daapd? This will recreate the db from scratch. Keep in mind that mt-daapd will not be fully operational until after the db is finished, this could take a while. Do a search for song3.db (or songs.db) to find the db. Mine is in /usr/local/var/cache/mt-daapd/songs3.db but ymmv.

Robbie

[Anonymous]

I would open the songs3.db with the sqlite client and look at the paths listed in the song table and make sure that the duplicated are not coming from a duplicate copy of the song in a different path.

[EVILRipper]

I have seen these problems before aswell.
The usual causes were when songs and the .m3u playlists were processed, processing the songs double.
The same goes for the iTunes xml library. When processing all songs and the iTunes xml library, all songs get processed twice.
Maybe you can look into that. I believe there are settings in the configuration page to turn off the processing of m3u playlists and/or iTunes xml libraries.

[Anonymous]

Thanks for the ideas. I twiddled with sqlite3. Here is an example of a duplicate song. The paths are the same. I don’t see anything obvious:

INSERT INTO “songs” VALUES(12886,’/mnt/raidvol/thevault/music/iTunes/iTunes Music/The Police/Synchronicity (Remastered)/05 Miss Gradenko 1.m4a’,’05 Miss Gradenko 1.m4a’,’Miss Gradenko’,’The Police’,’Synchronicity (Remastered)’,’Rock’,”,’m4a’,”,”,”,”,”,256,44100,119946,4463487,2003,5,0,1,1,0,0,0,0,0,2,’AAC audio file’,1232394226,1232394226,0,1232990073,0,0,0,’mp4a’,0,0,0,0,’The Police’);
INSERT INTO “songs” VALUES(12887,’/mnt/raidvol/thevault/music/iTunes/iTunes Music/The Police/Synchronicity (Remastered)/05 Miss Gradenko.m4a’,’05 Miss Gradenko.m4a’,’Miss Gradenko’,’The Police’,’Synchronicity (Remastered)’,’Rock’,”,’m4a’,”,”,”,”,”,292,44100,120000,4463487,2003,5,0,1,1,0,0,0,0,0,2,’AAC audio file’,1232394226,1232394226,0,1232946489,0,0,0,’mp4a’,0,0,0,0,’The Police’);

I’ve removed the song database each time too. Even despite doing this I have noticed that some times the server pops right back up claiming to have scanned 14,000+ songs in a couple of seconds. Other times it seems to take a couple of minutes. I haven’t isolated this behavior either.

I also tried disabling the processing of .m3u files. The net result of this was that there were about 400 or so less songs and now no playlists but the duplicates song references in the library remain.

Anybody know what the difference between the following files are?:
iTunes Library
iTunes Music Library.xml

Based on some time stamps, I removed the “iTunes Library” file. When I relaunched iTunes it seemed to have no ill-effect and iTunes apparently then recreated the file. Perhaps the problem is within these two files?

Mike

[Anonymous]

It looks to me like duplicate files. Look at the two paths.
/mnt/raidvol/thevault/music/iTunes/iTunes Music/The Police/Synchronicity (Remastered)/05 Miss Gradenko 1.m4a
/mnt/raidvol/thevault/music/iTunes/iTunes Music/The Police/Synchronicity (Remastered)/05 Miss Gradenko.m4a

The first has a ‘1’ in the filename before the extension.

[Anonymous]

Oh frig! You’re right. This seems to be a curious example as now that I’ve spot checked a couple of album directories I have duplicate entries but the issue seems to be that I have two files for each song. One with the track number in the name and one without. No clue how or when this happened.

I guess this absolves Firefly, the problem is with my iTunes library.

Any idea how to clean this up without having to go and manually delete all the duplicates? The duplicate files don’t appear to be part of the actual iTunes library while within iTunes itself. ie. the find duplicates tool doesn’t detect them and the number of songs in the library is correct.

What I need is a tool to sanitize the iTunes library against all the files stored in the “music folder location”. Any suggestions?

Thanks for the help.

Mike

[Anonymous]

After some googling I am not satisfied with the “state of the art” in duplicate detection and removal tools. There are a few, some that rely heavily on the “find duplicates” feature of iTunes and others that do more creative things that I am not sure I trust.

I think the right approach is to parse and dump the “location” fields of my iTunes Music Library.xml file. Then dump the file listing of the iTunes Music directory hierarchy. Massage the two lists so they are compatible and then diff them. All the diffs should be the extraneous stuff. From there I should be able to massage this diff list into a scrip that will delete them all.

Anybody have any concerns or known issues with this approach?

Mike

[Author]

Posts