Hawk Talk

A long-requested feature has been a way to decouple Amarok’s collection scanning from its GUI.  There are various use-cases for this.  For one, it can actually help us with debugging, by allowing us to control the inputs into the scan parser.  For another, many people have all of their music stored on a single machine, and would like to do the scanning locally where it’s fast instead of on their e.g. laptop running Amarok, where it’s over wireless and slow.

Yesterday and today (as of r933010) I put half of the solution into trunk.  I say half, because full collection rescans are now supported in batch modes, but I am still working on the methodology for incremental scans (I have a few ideas, but have to sort out which is the most reasonable/doable/makes the most sense). Below, I’ll explain how to do it.  Keep in mind that this is designed to be (lightly) scripted, not done by hand…so it can be done by hand (which I did during testing) but it has some safeguards in place so that if you script it, and forget about it, Amarok still works normally.  However, there are actually some interesting things you can do now if you script the scanner…

One important thing to note: the scanner requires various bits of Amarok code, mainly centering around the extra taglib plugins we support.  So you’re unlikely to have it work on a machine that doesn’t have A2 installed, and certainly won’t be able to compile it without the rest of the Amarok source, although you may be able to get it to work in a binary-only fashion with just kdelibs and taglib…YMMV.

So, here’s the flow:

1. Run amarokcollectionscanner with the -b or –batch option.  Although this doesn’t directly modify the output, it sets some internal flags (some for safety, some to enable other options).  The output goes to stdout; save this output as amarokcollectionscanner_batchfullscan.xml (yes, it must be that specific filename).  You’ll probably want to use the -r flag too…check –help for more options.
2. Cool feature: if your scan is of files that have a different mount point on the local machine than the machine that will be using the output, add the –rpath option and give it the mountpoint of the directory on the consuming machine.  For instance, if on the local machine your music is at /opt/music, and this directory (music) is mounted as /mnt/music on the remote machine, run amarokcollectionscanner from /opt and use –batch –rpath /mnt/music.
3. Cool feature: You can scan multiple directories and simply append the output to the end of the file from step #1.  Yes, directly append, including the xml version/encoding header.  Amarok’s scan manager will detect and handle this.  You can even scan directories that are not defined as collection directories in Amarok (these entries will work fine, but the next time you do a full rescan from within Amarok without using the saved output, they will be gone).
4. Place the amarokcollectionscanner_batchfullscan.xml file in your Amarok data dir.  Usually, this is ~/.kde4/share/apps/amarok.
5. Trigger a full rescan from inside Amarok (in Settings->Collection).  It should be super-fast.    Note that this step deletes the amarokcollectionscanner_batchfullscan.xml file.  This is on purpose, so if you want to keep re-using it, you’ll have to keep copying it over.

As you can see, it’s designed for scripting, but it’s not terribly complicated even to do by hand.  I’ll post more as I get incremental scanning working, and will at some point put a Wiki page up that has all of these instructions in a less transient form.

Poisonous people. They exist everywhere, sucking the light and good out of things and repurposing them for all sorts of nasty activities. Like the rest of KDE, and the rest of the software world (both free and non-free) in general, the Amarok team has taken its fair share of abuse over the years. Normally I ignore it. However, today I got my KDE 4.0 Release Event talk twisted around in malicious ways and the blame placed right at my own feet. So I’m rising to the bait.

The latest perpetrator of bile and venom is one Antal István Miklós. This blathering idiot Web Developer called me out personally on his Great Blog (yes, that’s really what it’s called, if you want to see his post), and turned my well-received talk at the KDE 4.0 Release Event into complaint-a-thon.

I’m not going to pick apart all of the technical wrongheadedness he portrays with MySQL/e, Akonadi, and the capabilities therein — Nikolaj posted a nice response to the blog, and the guy would be very well served to *fully* read my posting about mysqle (as well as many of the comments). Much of the blog can also easily be decoded as baseless, factless trolling ("amarok1.x is the slowest KDE3 program, if not it’s surely in the top 3 slowest KDE3 programs"), and the-developers-don’t-agree-with-me-so-they’re-wrong syndrome. But I *am* going to defend my statements at the Release Event.

Let me explain, up front, the format of my Release Event talk. I presented a short introduction to Amarok, for those that did not know it. I then stated some drawbacks we found with the KDE3 platform. With these drawbacks, plus a desire to go cross-platform like many other media players (VNC, Banshee, iTunes, etc.), we had considered the possibility of switching to a Qt-only architecture. But then, KDE4 comes along, with elegant solutions to our problems — Phonon, Solid, Plasma, targeting multiple platforms, and more. Boom — the thoughts of Qt-only go out of our heads, and we commit ourselves fully to KDE4.

Far from a long series of complaints, it’s a success story, showing how the benefits the KDE4 platform offer to us solved our problems, and how they could solve yours too. Apparently, however, it simply shows — from Antal’s blog post title, and probably because he hasn’t bothered to watch past five minutes in — how we just don’t get KDE4.

This may not be apparent to everyone, but Amarok was an early poster child for adoption of many of the Pillars of KDE. We are the only application, to date, that has embedded Plasma inside of our application (with our developers doing a large amount of work to make that possible). (Update: we are technically the first, outside of the plasma workspace, but there are others playing with that now.) Device detection completely relies on Solid (which is one reason Mac and Windows ports have no device support right now). And we have completely standardized on Phonon for our media engine. We’ve also had Oxygen team members working on our icons and our interface.

It’s hard to imagine ways for us to more fully integrate with KDE4 than what we are doing. We’ve gone for KDE4 whole-hog, and it’s ludicrous to suggest otherwise. Picking out random Pillars that we don’t fully integrate with (yet) does not mean that we are not KDE4-oriented. After all, right now we don’t have a use for Marble (who knows? that could change) — but does that mean we don’t get KDE4?

it just reminds me of one of the KDE4.0 release event, where a KDE dev complained that how KDE3 sucked, because they couldn’t port Amarok to Windows, and KHTML had bad performance

It’d be nice for Antal to realize that there is a difference between complaining, and listing drawbacks of a platform. (This doesn’t really fit into how trolls work, of course.) Yes, I said that two of KDE3′s drawbacks were that it made it impossible to port Amarok to Windows (and Mac) and that KHTML rendering was found to be slow. No, I did not say that KDE3 sucked.

But there’s nothing new here. It’s not like Amarok was alone in wanting to port to other platforms. The Release Event had showcases of KDE4 applications running on both Windows and Mac. But Antal has this fixation that Windows and Mac are suddenly all we care about, taking an out-of-context "consider the majority" statement someone (he doesn’t say who) made on IRC about some topic (he doesn’t say what, only that it’s vaguely somehow about performance):

Using mysqle mostly benefits non-KDE4 desktops, because as I said earlier KDE4 will probably have a mysql server anyway, but isn’t improving the KDE4 user experiance top release priority anymore? Is amarok on Windows on Mac more important than getting the best out of amarok on KDE4?

[then, later]

What did the people in the IRC channel had to say about this?

…….

My favorite quote from here is: "consider the majority"

It’s like saying: "consider the majority, which are Windows and Mac users, and screw the KDE4 users"

I think Antal fails to realize that KDE is not just a desktop. Windows and Mac users that might be using Amarok are going to be using *lots* of KDE technologies in the process. Regardless of his mistake, there is certainly no evidence that Amarok cares more about Windows and Mac users, or thinks them the majority of our users. Speaking as a developer, I can tell you that the exact opposite is true.

Antal also clearly doesn’t realize that Akonadi is not a requirement of KDE to run (even if it’s installed), and therefore the best Amarok could do would be to integrate with Akonadi, but not to depend or rely upon it. Maybe he kind of gets it when he says, my emphasis, "KDE4 will probably have a mysql server anyway" — we can’t rely on probably, or maybe. We need to use what works, always.

He even contradicts himself:

He begins with complaining, how slow was rendering amarok’s context with KHTML, so it looks like performance matters in amarok, not that anyone forced them to use KHTML for rendering context…

Make up your mind, Antal. You’re right that no one forced us to use KHTML for rendering context, although WebKit wasn’t available back then, and hooking into Mozilla was a non-starter. But you want us to be integrating with other KDE technologies…right?

One last point:

Jeff Mitchell the developer who spoke at the event that I was referring to, referenced KDE as a family, but where is the love now? The lack of communication between Amarok and the rest of KDE4(Akonandi) doens’t seem to back up Amarok as being a family member.

It is not surprising, given how little he understands of Amarok, KDE, and the integration thereof, that he thinks both that there is a lack of communication between Amarok and the rest of KDE4, and that he implies that Akonadi is the entire rest of KDE4.

I’ve covered a small fraction of the untruths and inaccuracies in his post, but it’s enough — I’ve made my points. I love KDE, I have not publicly disparaged it, we Amarok developers are fully committed to the platform, and we are not putting the Windows and Mac ports at a higher priority than the *nix base.

Any comments will be read, but I may decide not to post them.