Getting Playlist in 1586

[peterr]

I installed svn-1586 on an NSLU2. Overwriting a previous installation of mt-daapd 2.4 which used it’s own database instead of sqlite. After rewriting smart.html and smart.js so that I could batch import smart playlists, I created a small smart playlist covering only 286 albums for 3268 tracks. My previous smart playlist for this library under 2.4 consisted of 436 (by album and by composer) items.

With 1586, it takes at least 5 times as long to refresh iTunes with the network server music list with this smaller playlist. This is not too much a problem as once you log in via iTunes, the playlist stays. but I use a cgi perl script to support three different libraries and switching between libraries is a pain now as one library has 768 item smart playlist and I have to wait nearly forever! Really what is the advantage of sqlite vs the much faster old way?

[rpedde]

@peterr wrote:

I installed svn-1586 on an NSLU2. Overwriting a previous installation of mt-daapd 2.4 which used it’s own database instead of sqlite. After rewriting smart.html and smart.js so that I could batch import smart playlists, I created a small smart playlist covering only 286 albums for 3268 tracks. My previous smart playlist for this library under 2.4 consisted of 436 (by album and by composer) items.

With 1586, it takes at least 5 times as long to refresh iTunes with the network server music list with this smaller playlist. This is not too much a problem as once you log in via iTunes, the playlist stays. but I use a cgi perl script to support three different libraries and switching between libraries is a pain now as one library has 768 item smart playlist and I have to wait nearly forever! Really what is the advantage of sqlite vs the much faster old way?

Access to the underlying database is the big thing. Playlist stuff is going to (soon) get moved back out of the database for performance reasons, and to be able to do interesting things that are hard (impossible?) to do with SQL, so that will help a lot.

Also, after the playlists are done, then I’ll finish up the db abstraction and re-implement a gdbm backend for performance on low-memory devices.

I hope that once the playlist stuff is done, the sqlite backend will run as fast as the old gdbm backend did. Mostly the payoffs of the sqlite backend run more to larger memory devices, and the ability to manage the database with a sql client, via a php page, etc.

Lately, though, I’ve been more targeting architecture changes and not performance. Soon I’ll start tackling performance again.

I know I should have released a new stable before this, so it’s hard not to run nightlies because of the feature improvements, but remember this is dev code, and doesn’t necessarily even *work*, much less work fast. I am aware of the speed issues, though, and I’m going to be tackling them soon. I don’t want to release another stable until I get the gdbm backend back in, particularly for the NSLU2. That’s whats really holding back a new stable.

If you have ideas on that front, I’ll take them, but remember that is is a one-man band, and sometimes I move slower than people might like.

— Ron

[peterr]

I see what you mean, without the playlist in the database I got good response disconnecting and re-connecting but once I batch-added in a few hundred playlist, things got boggy. Also by having the playlist in sqlite, I noticed editing and deleting needs the database id. This really complicates batch editing (in an external editor, of course) and deleting. When you deal in large playlist, the one at a time approach in smart.html sucks. So its easier to trash the database and start fresh by retaining everything in an auxiliary file to straighten multiple changes fast. By the way I just posted iTunes Applescripts to generate mt-daapd playlist (both for 2.4 and svn-1586) to import into mt-dappd. Also included is the necessary changes to smart.html and smart.js needed for batch importing to sqlite. It is at http://www.makehell.com/

This afternoon I came across another problem in svn-1586, while playing tracks from an album smart playlist, it would hang between tracks with “getting url” message, canceling that and then clicking on the same track would allow it to continue!

Sorry to hear that you still slave alone at this project. I know the feeling.

[rpedde]

@peterr wrote:

I see what you mean, without the playlist in the database I got good response disconnecting and re-connecting but once I batch-added in a few hundred playlist, things got boggy. Also by having the playlist in sqlite, I noticed editing and deleting needs the database id. This really complicates batch editing (in an external editor, of course) and deleting. When you deal in large playlist, the one at a time approach in smart.html sucks. So its easier to trash the database and start fresh by retaining everything in an auxiliary file to straighten multiple changes fast. By the way I just posted iTunes Applescripts to generate mt-daapd playlist (both for 2.4 and svn-1586) to import into mt-dappd. Also included is the necessary changes to smart.html and smart.js needed for batch importing to sqlite. It is at http://www.makehell.com/

This afternoon I came across another problem in svn-1586, while playing tracks from an album smart playlist, it would hang between tracks with “getting url” message, canceling that and then clicking on the same track would allow it to continue!

Sorry to hear that you still slave alone at this project. I know the feeling.

The pause is probably long time to do lookups by id. (large database?)

I have most of the webserver cleanup done, and the IO abstraction done, so I’m close to a new nightly. Once bugs from that settle down, I’m going to do the db reorg.

Do you have web-fu? I’m badly in need of getting a javascript interface to the database for playlist, searching etc.

There is a start at /playlists.html, and that’s kind of where i want to go with it, except add static and dynamic playlist editing, and a flash player.

If you had web-fu and time, I’d be happy for the help. 🙂

— Ron

[peterr]

Yes, all my comments are for the classical library and it is a large database of 3267 tracks organized via 287 smart playlist (just albums for now, but if I include my previous criteria of artist and composers it would require 457 playlists).

I only just discovered the playlist.html and corresponding playlist.js. this afternoon from your comment in another topic. Taking a quick look it’s probable I could have skipped the iTunes Applescript and javascript massage on smart.js since I only needed the actual contents of the mt-daapd library to feed into the batch type smart playlist. I been working in the dark here, just grokking what I can from the visible parts of svn-1586.

My only question is if you are changing the database access whether the polishing of playlist html and its javascript will hold up to your planned changes without requiring a simple change in access method. Let me look over the playlist.js and html code before I commit.

I do plenty of simple perl cgi mixed with Ajax, HTML and CSS for web based applications. Examples are available at the forementioned website, http://www.makehell.com, including a cgi I did last year for switching libraries under mt-daapd though a web interface (This was written to segregate lectures and novels from music) – it just swaps out config files and restarts mt-daapd. This is why, reconnection times are important to me.

[rpedde]

@peterr wrote:

My only question is if you are changing the database access whether the polishing of playlist html and its javascript will hold up to your planned changes without requiring a simple change in access method. Let me look over the playlist.js and html code before I commit.

The changes won’t impact the xml methods available from javascript. The same methods will continue to work (creating, modifying, searching, retrieving file details, etc).

I do plenty of simple perl cgi mixed with Ajax, HTML and CSS for web based applications. Examples are available at the forementioned website, http://www.makehell.com, including a cgi I did last year for switching libraries under mt-daapd though a web interface (This was written to segregate lectures and novels from music) – it just swaps out config files and restarts mt-daapd. This is why, reconnection times are important to me.

There isn’t any perl, obviously, as the web part is handled by the server, but everything interesting including querying server status, updating configuration, accessing the database, etc, is all available from restful urls that return xml data sets.

The old way (iTunes-ish way) is here: http://trac.fireflymediaserver.org/wiki/XmlApi

Config updating is here:
http://trac.fireflymediaserver.org/wiki/xmlrpc

And there is more xml stuff available (faster) from the rsp module, but I don’t have that documented yet.

— Ron

[peterr]

Sorry, I haven’t responded sooner. Yesterday, I had an eye exam in the morning and they numbed and then dilated my eyes. I could not use the computer til late afternoon, because with wide open pupils, depth-of-field is nearly non-existent and everything was a bit blurry and gave me a headache.

I started looking over the javascript today and found I need to get a little familiar with prototype.js first. since that’s heavily used. Give me about another week with the prototype api doc and some tracing with firebug and firefox and I should be able to bike around without training wheels.

[rpedde]

@peterr wrote:

Sorry, I haven’t responded sooner. Yesterday, I had an eye exam in the morning and they numbed and then dilated my eyes. I could not use the computer til late afternoon, because with wide open pupils, depth-of-field is nearly non-existent and everything was a bit blurry and gave me a headache.

I started looking over the javascript today and found I need to get a little familiar with prototype.js first. since that’s heavily used. Give me about another week with the prototype api doc and some tracing with firebug and firefox and I should be able to bike around without training wheels.

I’m not beholden to prototype necessarily, some of the other js frameworks seems like they’ve become much more mature… dojo, yahoo ui, whatever seems like it would work the best. Results first, substance over flash, so whatever seems like it would work good I’m for.

— Ron

[petsounds]

JQuery seems to be the new hotness in the js framework scene. A lot of our developers at work have switched from prototype to JQuery.

http://jquery.com/

[Author]

Posts