Back to the main page

Mailing List Logs for ShadowRN

Message no. 1
From: derek@***************.com (Derek Hyde)
Subject: [OT] iTunes/Command Line Script help?
Date: Tue, 09 Jan 2007 21:01:32 -0600
Ok, since there's a plethora of wizards with the command line/itunes thing
going, anyone out there able to throw me a bone and help me find a script
that'll identify all of the duplicates in my library? (one on win32 and OSX
10.4.9) would be marvelous ;) )
Message no. 2
From: maxnoel_fr@*****.fr (Max Noel)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 08:16:51 +0100
On 10 Jan 2007, at 04:01, Derek Hyde wrote:

> Ok, since there's a plethora of wizards with the command line/
> itunes thing
> going, anyone out there able to throw me a bone and help me find a
> script
> that'll identify all of the duplicates in my library? (one on win32
> and OSX
> 10.4.9) would be marvelous ;) )

Have you tried View -> Show Duplicates from iTunes itself?

-- Wild_Cat





___________________________________________________________________________
Découvrez une nouvelle façon d'obtenir des réponses à toutes
vos questions !
Profitez des connaissances, des opinions et des expériences des internautes sur
Yahoo! Questions/Réponses
http://fr.answers.yahoo.com
Message no. 3
From: derek@***************.com (Derek Hyde)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 01:20:26 -0600
> Have you tried View -> Show Duplicates from iTunes itself?
>
> -- Wild_Cat


I have, the problem is, if they show up as being slightly different
spelling, capitolization, or an extra space in the album name it doesn't try
to filter....or at least didn't for me....
Message no. 4
From: gurth@******.nl (Gurth)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 11:05:26 +0100
According to Derek Hyde, on 10-1-07 04:01 the word on the street was...

> Ok, since there's a plethora of wizards with the command line/itunes thing
> going, anyone out there able to throw me a bone and help me find a script
> that'll identify all of the duplicates in my library? (one on win32 and OSX
> 10.4.9) would be marvelous ;) )

iTunes -> View menu -> Show Duplicates should do the trick ...

--
Gurth@******.nl - Stone Age: http://www.xs4all.nl/~gurth/index.html
Van e-mail bakt men cyberbrood.
-> Former NAGEE Editor & ShadowRN GridSec * Triangle Virtuoso <-
-> The Plastic Warriors Site: http://plastic.dumpshock.com <-

GC3.12: GAT/! d- s:- !a>? C++(---) UB+ P(+) L++ E W++(--) N o? K w-- O
M+ PS+ PE@ Y PGP- t- 5++ X(+) R+++$ tv+(++) b++@ DI- D G+ e h! !r y?
Incubated into the First Church of the Sqooshy Ball, 21-05-1998
Message no. 5
From: gurth@******.nl (Gurth)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 11:09:09 +0100
According to Derek Hyde, on 10-1-07 08:20 the word on the street was...

> I have, the problem is, if they show up as being slightly different
> spelling, capitolization, or an extra space in the album name it doesn't try
> to filter....or at least didn't for me....

When I try that, it looks like it simply shows all song titles that
appear more than once. For example, I see two "21st Century (Digital
Boy)" tracks, same artist and song length but from different albums. Or
is that what you mean by "it doesn't try to filter"?

--
Gurth@******.nl - Stone Age: http://www.xs4all.nl/~gurth/index.html
Van e-mail bakt men cyberbrood.
-> Former NAGEE Editor & ShadowRN GridSec * Triangle Virtuoso <-
-> The Plastic Warriors Site: http://plastic.dumpshock.com <-

GC3.12: GAT/! d- s:- !a>? C++(---) UB+ P(+) L++ E W++(--) N o? K w-- O
M+ PS+ PE@ Y PGP- t- 5++ X(+) R+++$ tv+(++) b++@ DI- D G+ e h! !r y?
Incubated into the First Church of the Sqooshy Ball, 21-05-1998
Message no. 6
From: derek@***************.com (Derek Hyde)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 07:57:56 -0600
> When I try that, it looks like it simply shows all song titles that
> appear more than once. For example, I see two "21st Century (Digital
> Boy)" tracks, same artist and song length but from different albums. Or
> is that what you mean by "it doesn't try to filter"?
>
I may have to uninstall and reinstall it cause when I do it, if they're not
the exact same spelling/punctuation/spacing it's not doing it

For example...

Mad World
mad world
Mad World (Donnie Darko Soundtrack)

Are three that I know there are duplicates but are different bitrates, it
won't find the three of them....and if one that I know should give me 3
gives me 1 or 2 depending on the time I try it....what about the ones I
don't know if there are more than just the ones it shows?
Message no. 7
From: korishinzo@*****.com (Ice Heart)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 06:17:05 -0800 (PST)
> For example...
>
> Mad World
> mad world
> Mad World (Donnie Darko Soundtrack)
>
> Are three that I know there are duplicates but are different
> bitrates, it
> won't find the three of them....and if one that I know should give
> me 3
> gives me 1 or 2 depending on the time I try it....what about the
> ones I
> don't know if there are more than just the ones it shows?

If they have different bitrates, different filenames, and probably
different file sizes... how would a computer know they are
duplicates? There is not, to my knowledge, a computer in existance
that can perform abstract comparison. That is a human capability.
Unless Apple cornered the market on a working AI... :p

======Korishinzo
--Cyberware obsolescence is going to suck for the early birds



____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com
Message no. 8
From: adamj@*********.com (Adam Jury)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 07:23:41 -0700
On 10-Jan-07, at 7:17 AM, Ice Heart wrote:

>> For example...
>>
>> Mad World
>> mad world
>> Mad World (Donnie Darko Soundtrack)
>>
>> Are three that I know there are duplicates but are different
>> bitrates, it
>> won't find the three of them....and if one that I know should give
>> me 3
>> gives me 1 or 2 depending on the time I try it....what about the
>> ones I
>> don't know if there are more than just the ones it shows?
>
> If they have different bitrates, different filenames, and probably
> different file sizes... how would a computer know they are
> duplicates? There is not, to my knowledge, a computer in existance
> that can perform abstract comparison. That is a human capability.
> Unless Apple cornered the market on a working AI... :p

A working solution to this problem is: Fix your ID3 tags, then purge.

If you're on OS X, iEatBrainz will help you automagically fix your
tags, and there's software for other OSes available at http://
musicbrainz.org/

It's not a perfect solution, but if you simply have duplicate
versions of commonly-found songs, it should help you clear up a fair
amount of them quickly. When you have bad data, though, you're almost
always going to have to do some grunt work to clean it up.

Adam
Message no. 9
From: gurth@******.nl (Gurth)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 15:34:11 +0100
According to Ice Heart, on 10-1-07 15:17 the word on the street was...

> Unless Apple cornered the market on a working AI... :p

If they have, I'm sure Steve Jobs would have announced that yesterday
instead of showing a phone and a TV-box-type-thingy ;)

--
Gurth@******.nl - Stone Age: http://www.xs4all.nl/~gurth/index.html
Van e-mail bakt men cyberbrood.
-> Former NAGEE Editor & ShadowRN GridSec * Triangle Virtuoso <-
-> The Plastic Warriors Site: http://plastic.dumpshock.com <-

GC3.12: GAT/! d- s:- !a>? C++(---) UB+ P(+) L++ E W++(--) N o? K w-- O
M+ PS+ PE@ Y PGP- t- 5++ X(+) R+++$ tv+(++) b++@ DI- D G+ e h! !r y?
Incubated into the First Church of the Sqooshy Ball, 21-05-1998
Message no. 10
From: marc.renouf@******.com (Renouf, Marc A.)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 09:39:34 -0500
> -----Original Message-----
> From: shadowrn-bounces@*****.dumpshock.com
> [mailto:shadowrn-bounces@*****.dumpshock.com] On Behalf Of Ice Heart
> Sent: Wednesday, January 10, 2007 9:17 AM
>
> If they have different bitrates, different filenames, and
> probably different file sizes... how would a computer know
> they are duplicates? There is not, to my knowledge, a
> computer in existance that can perform abstract comparison.
> That is a human capability.
> Unless Apple cornered the market on a working AI... :p

Digital signal analysis is the way to solve this problem in a
duplicated music context, but it's prohibitively time consuming and
computationally intensive. You could save yourself some time by
downsampling to the lowest common bitrate and only comparing the first
20 seconds or so of music, but if you have an even reomtely respectable
library it would take quite a long time to identify all the duplicates
(as this method would require 1/2 * N^2 comparisons where N is the
number of song in your library).

Marc
Message no. 11
From: DaTwinkDaddy@*****.com (Da Twink Daddy)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 08:40:15 -0600
On Wednesday 10 January 2007 08:17, Ice Heart <korishinzo@*****.com> wrote
about 'Re: [OT] iTunes/Command Line Script help?':
> > For example...
> >
> > Mad World
> > mad world
> > Mad World (Donnie Darko Soundtrack)
> >
> > Are three that I know there are duplicates but are different
> > bitrates, it
> > won't find the three of them....and if one that I know should give
> > me 3
> > gives me 1 or 2 depending on the time I try it....what about the
> > ones I
> > don't know if there are more than just the ones it shows?
>
> If they have different bitrates, different filenames, and probably
> different file sizes... how would a computer know they are
> duplicates?

Just an idea, but the computer to transcode them all to some ridiculously
low bitrate (like 1kbps or less) and then compare them (sans mpe3 header
and ID3 tags) for equality. The low you go in bit rate, the more likely
everything just "compresses" to the same data -- but first the tracks that
are already similar (like 2 encodings of the same recording... then later
2 recordings of the same song...) will "merge" because mp3 intentionally
throws out the "small stuff" first.

It's something to try, although I'm not sure it'll work if any of one of a
set of "duplicates" has a different length.

--
Da Twink Daddy
DaTwinkDaddy@*****.com
ICQ: 514984 (Da Twink Daddy) YM/AIM: DaTwinkDaddy
Message no. 12
From: justin@***********.net (Justin Bell)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 10:29:06 -0500
On 1/10/2007 8:57 AM, Derek Hyde wrote:
>> When I try that, it looks like it simply shows all song titles that
>> appear more than once. For example, I see two "21st Century (Digital
>> Boy)" tracks, same artist and song length but from different albums. Or
>> is that what you mean by "it doesn't try to filter"?
>>
> I may have to uninstall and reinstall it cause when I do it, if they're not
> the exact same spelling/punctuation/spacing it's not doing it
>
> For example...
>
> Mad World
> mad world
> Mad World (Donnie Darko Soundtrack)
>
> Are three that I know there are duplicates but are different bitrates, it

They're not duplicates then.

Though in an analogue sense you think they are, they're different in a
digital sense.
Message no. 13
From: carltondavis@*****.com (Carlton Davis)
Subject: [OT] iTunes/Command Line Script help?
Date: Wed, 10 Jan 2007 09:42:20 -0600
What I did in that situation was export the library as an XML file,
pipe it through an XSLT to create an HTML table with all the info, and
then copy/paste the data into a spreadsheet. Then I used the sorting
and comparison routines in the spreadsheet to pull out a list of all
the duplicate songs weighted towards keeping the higher bit rate,
rated or often played versions. It also let me know which files
needed to have a rating attached in the few cases where the lower bit
rate version was rated, but the higher bit rate wasn't. Then I just
made a batch file to delete all of the duplicates.

I probably could have used an actual DB for this, but being able to
quickly scan through the list and use my judgment to tweak the
settings and output made it much easier and more accurate.

It was a bit of a hassle, but I spent quite a while looking for a
solution that would handle my particular circumstances, and nothing
available for Windows did what I needed. Even the ones which looked
like they might help required iTunes to be running, and since I don't
have enough ram on my Windows box it barely runs at all with iTunes
loaded, let alone anything else.

Further Reading

If you enjoyed reading about [OT] iTunes/Command Line Script help?, you may also be interested in:

Disclaimer

These messages were posted a long time ago on a mailing list far, far away. The copyright to their contents probably lies with the original authors of the individual messages, but since they were published in an electronic forum that anyone could subscribe to, and the logs were available to subscribers and most likely non-subscribers as well, it's felt that re-publishing them here is a kind of public service.

Thwap!