Now, if you *did* specify that it should be recreated for some reason, that will not mess up statistics so long as a scan is performed on the file in the same location with the new ID. This will create a new (location, ID) mapping and all will be well.

BTW, I’ve read all the comments up to this point, and I definitely agree with you on all your arguments (why not use Strigi/Nepomuk, why make afttagger and standalone utility, etc.)

No, I don’t. Strigi’s scanning is very general and very slow. Amarok’s scanner is very fast and highly specialized. There’s nothing wrong with having both.

“If so, maybe Strigi could be used in some other way than default (either within Amarok or just with music-files) so that it would be quicker and more suitable for Amarok’s purposes?”

But that would defeat the point of Strigi…one of the guarantees it makes is SHA hashes, and that is the real killer as far as speed. I don’t really understand the problem you have with both being used. Amarok has support/is getting support for a Nepomuk-based collection. This would query Nepomuk for files scanned by Strigi and present them as a separate collection from the one that Amarok’s own collection scanner will build. People that have Strigi indexing all their music could take advantage of this collection.

I don’t understand the issue here. What’s wrong with having both methods available and letting the user choose what they want?

But do you agree that using Strigi would be more beneficial as the information would be widely accessible and more usable? If so, maybe Strigi could be used in some other way than default (either within Amarok or just with music-files) so that it would be quicker and more suitable for Amarok’s purposes?

And I obviously just mixed T with P, so I meant AFT instead of AFP

It can be integrated into Amarok eventually. The point of making it an external application is two-fold.

One, it will be easy to integrate into Amarok in some way with a script or the GUI itself.

Two, people can run it when Amarok isn’t open. This is a big plus, because doing the tagging can be a very time-consuming process.

“BTW, for me personally…”

From the article…

“…AFT is enabled…with support for stored playlists coming eventually”

With A2 playlists will be stored in the database, and exported to whatever format you desire when you want. As a result, it will be easy to add AFT support to stored playlists, and it will be coming. With A1, the playlists were stored as M3U files on disk, and as a result there was no way for there to be UIDs stored in them.

So it’s not exactly reinventing the wheel that Amarok is using the same collection scanner it was already using in 1.4. It hasn’t really changed. It still uses TagLib, which is a KDE-style framework if not actually a KDE framework, and which Strigi doesn’t use — so perhaps it’s Strigi that reinvented the wheel when it comes to MPEG/FLAC/Vorbis file parsing.

So, why isn’t Strigi used? Well, one of the main reasons you said yourself: “even though the user ain’t using it for other stuff?” There is no requirement by Amarok for Strigi, and making it a requirement, along with necessary setup, is needlessely complex and intrusive.

Not only that, Amarok’s collection scanner is *much* faster. Strigi needs to read every byte of every file. Our scanner often reads only a tiny part of the file. Considering that many people store their files on NFS or SMB/CIFS shares, this is a very big concern.

It’s also a reason why Amarok’s file tracking capabilities doesn’t take hashes of the whole file, or of all of the music data.

Re: “Nepomuk and AFP-stuff with everything else”…what does Apple Filing Protocol have anything to do with this? That’s Mac-only.

BTW, for me personally the ATF in Amarok 1.4.x is useless because it won’t refresh static playlists after renaming files (which I do quite often).

However, this feature is needed and I hope we won’t discourage you. I’m sure you will find an elegant solution.

Thumbs up!
plangin

Why can’t Strigi be used for scanning Amarok’s collection (even though user ain’t using it for other stuff)? If there’s no right backend/engine or smth, maybe it’s worth creating/porting one?
The collection would then be accessible via Nepomuk and AFP-stuff with everything else could come from there? The information would be accessible from the rest of the KDE and some other app using KDE libs etc. Ain’t that the true goodness of all the great libs?

If you’re not open to criticism then why blog?

Amarok has tables in its database with file paths to associate files with scanned (with amarokcollectionscanner) metadata, statistics, etc. When the file path changes, those tables have to be synced somehow, or you lose the association in the database with the actual files. AFT solves this.

As far as hard links, it’s not an option for Amarok to start placing links everywhere on someone’s filesystem. That’s also assuming that the filesystems people are using support hard links, which in many cases is not true.

P.S. Why would I misinterpret your “tone of voice”?

However I’m don’t entirely understand the difficulties that Amarok and NEPOMUK seem to be having with associating metadata with files. Why is it necessary to use file watching rather than hard links?

p.s. Lest you misinterpret my “tone of voice” I know that I’m ignorant about these things, this is a genuine question not a suggestion that you’ve missed something.

Then don’t.

“But an extra utility that needs to be run sounds like a usability nightmare.”

It’s not. It has nothing to do with usability.

“Hell, I’m already confused as to what the options are.”

Then actually read the entry.

“Will it still use Amarok 1.4-style tracking if the tagger isn’t run?”

It’s very clearly stated that it will.

“Eventually there should probably be an option inside Amarok to enable and run the tagger.”

There can be. When everyone is satisfied that it works and is safe. Or it could easily be done through a script.

* Do not track my files
* Track my files based on their current tags
* Embed an ID in my files to track them

The second option should probably be the default.

Text below the radio button group would read the following if the given radio button is selected:

1. If you move a file while this option is enabled, any statistics will be lost in the database
2. This option will re-discover files if the location OR the tags are changed, but not if both are changed between database updates
3. This option is the most robust, but will embed an identifier in your files’ tags. This process may take a long time for large collections.

…or something similar. Choosing the third option and applying the changes would run an Amarok-managed process (with progress bar?) to tag all the files. This would also provide UI hooks (though not necessarily software hooks) in case someone wants to add Nepomuk-based tracking or MusicBrainz-based tracking.

Now, I’m not a usability expert by any means, so I’m open to criticism as to why this sucks.

Remember that the Nepomuk collection is separate from the SQL collection. I see no reason why Daniel’s work couldn’t be used in the Nepomuk collection. But my understanding is that Strigi is required for the kind of situation you’re describing to actually work in Nepomuk, and if this is the case, it’s not something that can be relied upon. We can’t assume that users will have Strigi scanning all their music (especially if their music is on remote shares); we can only assume that users are scanning the music that is configured within Amarok. So it seems to make sense that for the SQL-based local collection, AFT would be the file-tracking method used, whereas for tracks sourced from the Nepomuk collection, a method like you described would be used.

Example: you have foor.mp3 and bar.ogg, both containing song “foo” by artist “bar”. Play them both and you get a playcount of 2 for the one song.

The same should then apply for playlists: they contain of tracks, not of files. When you move a file, the relation to the actual track is not lost but maintained automatically by Nepomuk.

Of course this is not done yet. So at the moment, AFT is the preferred solution.

Re: MBIDs, those are ways to identify a track, not a file. There are reasons to do one or the other, but file tracking is more reliable.

Re: ID3 header, it’s using a UFID frame. It’s the frame designed specifically for Uniquie File IDentifiers, so I think it’ll be obvious to other players’ coders

Two things:

- All my files also have MBIDs (see http://wiki.musicbrainz.org/MusicBrainzTag), a combination of those should also a nice way to identify a track.

- It might be a good idea to document the exact ID3 header you use on the AFT wiki page. maybe some other player will jump the band wagon and use that id as well…

Actually, that’s one of my biggest problems with kdelibs. It’s really hard to find out what’s already there, in no small part becuase the search at api.kde.org sucks. It never finds classes I already do know exist, much less classes I don’t know exist…

http://api.kde.org/4.x-api/kdelibs-apidocs/kdecore/html/classKSaveFile.html

The way statistics are currently implemented, you would only receive statistics updates for plays of one of the files (I could possibly make this smarter at some point, so it checks to see if a file has a UID and if it matches somewhere else in the table, and preferentially uses that).

What won’t happen, however, is that your statistics will be totally lost.

The first is that it can only track what Strigi scans. Many users may never use Strigi/Nepomuk, and of those that do, many won’t have it index their music files.

The second is that Strigi generates SHA hashes of every file, but this includes (AFAIK) all of the data of each file. As soon as a file has changed (i.e. a metadata update), the SHA is invalidated, and we have to wait until Strigi gets it again, and then somehow sync with it. And if a user moves a file at the same time, we may never find the file again.

So at best, it’s no more useful than non-embedded AFT. At worst, it’s almost useless.

What’s wrong with using MD5 of the first x bytes of audio is that it’s very difficult to do. Our tag library does not have a way to access the audio data only, and you can’t simply find the offset — some formats interleave audio and metadata together. So we’d have to write a new library to find this audio data — it’s not worth it.

Besides, many tag specifications already have this built in. ID3v2 has a UFID frame, for instance, which is what is used.

Using the first x bytes are *not* zero quite often. And it’s not the only value being used in non-embedded hashes.

What’s wrong with using the MD5 of the first x bytes of audio? That way if the tags or file length changes the ID won’t change.

Actually you shouldn’t use the first x bytes because they might be zero quite often. Somewhere from the middle.