setting up ABX tests at home

Re: setting up ABX tests at home

Greg Wormald <greg.wormald@gmail,com > wrote:
> Steve wrote:
> > I want to set up an ABX test at home on different sources to see if I
> > can hear a difference. I am doing this for my own education and fun -
> > I don't want to start investing in SPL meters, switches gain
> > equalizers of other boxes.

> Steve,

> As long as you are doing it for fun, the set-ups you have specified
> should work. Education is a different matter.

> While ABX tests have their proponents, they also have detractors, and
> such tests may not educate you about equipment at all. What both
> proponents and detractors agree upon is that for short term tests
> absolute sound level matching is a must. Many tests have shown that the
> louder source is usually preferred.

> The very set-up of much ABX testing (which is most often short segments
> of music quickly switched), cannot test for what most music lovers are
> seeking--long term enjoyment of music listening. A goodly number of
> years ago I ABX blind tested some interconnects, and then chose the ones
> I preferred. Three months later I realised I was not listening as much
> or enjoying it at the level I was previously. I found myself turning the
> music down or off. When I went back to my old interconnects my listening
> increased to it's previous levels of enjoyment.

..at which point , if you did an ABX, you would be able to tell
the two apart , right?

This is what alwyas perplexes me about the 'you need to do
long term listening' brigade. If you believe long term'listening is
required to get the true 'sound' of the gear into your brains,
fine, go for it. And *then* demonstrate you're hearing a real difference,
using a ABX comparison. Shoudl be simple to 'pass' at that point,
shouldn't it?

___
-S
"Hey pip squeak, who's L Ron, some new rapper?" -- Nic

Re: setting up ABX tests at home

On Tue, 12 Feb 2008 20:26:31 -0800, Steven Sullivan wrote
(in article <fotrhn02kb0@news2.newsguy,com >):

> Greg Wormald <greg.wormald@gmail,com > wrote:
>> Steve wrote:
>>> I want to set up an ABX test at home on different sources to see if I
>>> can hear a difference. I am doing this for my own education and fun -
>>> I don't want to start investing in SPL meters, switches gain
>>> equalizers of other boxes.
>
>> Steve,
>
>> As long as you are doing it for fun, the set-ups you have specified
>> should work. Education is a different matter.
>
>> While ABX tests have their proponents, they also have detractors, and
>> such tests may not educate you about equipment at all. What both
>> proponents and detractors agree upon is that for short term tests
>> absolute sound level matching is a must. Many tests have shown that the
>> louder source is usually preferred.
>
>> The very set-up of much ABX testing (which is most often short segments
>> of music quickly switched), cannot test for what most music lovers are
>> seeking--long term enjoyment of music listening. A goodly number of
>> years ago I ABX blind tested some interconnects, and then chose the ones
>> I preferred. Three months later I realised I was not listening as much
>> or enjoying it at the level I was previously. I found myself turning the
>> music down or off. When I went back to my old interconnects my listening
>> increased to it's previous levels of enjoyment.
>
> ..at which point , if you did an ABX, you would be able to tell
> the two apart , right?
>
> This is what alwyas perplexes me about the 'you need to do
> long term listening' brigade. If you believe long term'listening is
> required to get the true 'sound' of the gear into your brains,
> fine, go for it. And *then* demonstrate you're hearing a real difference,
> using a ABX comparison. Shoudl be simple to 'pass' at that point,
> shouldn't it?

Not always. Sometimes differences are subtle and many times, certain
recordings simply aren't good enough to allow those differences to show up.
For instance, suppose we are testing the soundstage/imaging of two different
sets of speakers and we use a 20-track recording that places a microphone in
front of each instrument in the ensemble. An ABX or double-blind test will
tell you nothing of the imaging characteristics of those speakers because,
there isn't any imaging in the recording - merely a bunch of instruments
pan-potted to the right, the left, and the center.

Re: setting up ABX tests at home

"Sonnova" <sonnova@audiosanatorium,com > wrote in message
news:fp023j01mr2@news4.newsguy,com
> On Tue, 12 Feb 2008 20:26:31 -0800, Steven Sullivan wrote
> (in article <fotrhn02kb0@news2.newsguy,com >):

>> This is what alwyas perplexes me about the 'you need to
>> do long term listening' brigade. If you believe long
>> term 'listening is required to get the true 'sound' of
>> the gear into your brains, fine, go for it.

>> And *then* demonstrate you're hearing
>> a real difference, using a ABX comparison. Should be
>> simple to 'pass' at that point, shouldn't it?

Exactly. More to the point, not every part of every recording is adequately
or maximally critical for all kinds of audible differences. For example,
consider a subwoofer that distorts only deep, loud bass. If you don't listen
to a recording with deep, loud bass at an appropriate level, you're never
going to hear the problem. Another example would involve audible resonances.
Resonances are narrow-band effects, and if your music doesn't have enough
energy to trigger the resonance to the point where it is audible, you're
never going to hear it.

> Not always. Sometimes differences are subtle and many
> times, certain recordings simply aren't good enough to
> allow those differences to show up.

I agree with the thinking, but I find the wording to be unecessarily vague.
It's not the goodness of a recording that makes the difference, it is its
appropriateness, which of course can include general desirable
characteristics such as clarity. Several researchers including AES/MPEG/Bell
Labs/Microsoft guru Jim Johnson have reported that sometimes it is bad
recordings that make certain audible differences most audible.

> For instance, suppose
> we are testing the soundstage/imaging of two different
> sets of speakers and we use a 20-track recording that
> places a microphone in front of each instrument in the
> ensemble. An ABX or double-blind test will tell you
> nothing of the imaging characteristics of those speakers
> because, there isn't any imaging in the recording -
> merely a bunch of instruments pan-potted to the right,
> the left, and the center.

Again the fault here is that ABX testing is singled out as being part of the
problem. It isn't. The heart of the matter is that no testing paradigm will
reliably reveal differences that aren't there. Testing methodologies that
can't reject false positives can convince people that they are hearing
things that aren't really there.

Very shortly after developing ABX, our listening group became far more
focussed on listening strategies, because if our listening tests didn't cut
it, that became abundently clear almost instantly. We had a rudimentary idea
of the importance of what we call critical passages to the process of
reliably detecting audible differences. This included well-known studio
tricks such as listening with just one ear, and using piano recordings to
test for flutter and wow.

ABX testing allowed us to develop a well-defined collection of segments of
popular recordings that were ideal for hearing a wide variety of kinds of
audible differences. A suite of these recordings was assembled in the
mid-late 1980s using the emerging technology of the CD-R which facilitated
compiling and distributing the collection.

This collection became known as the LTT, and is used by many of us to this
day. Interestingly enough, the LTT remained unchanged for about a decade,
but was augmented to include musical selections from new genres such as rap,
that had new types of content that stressed audio components in ways that
they had not been stressed before.

Obviously, a collection like the LTT is very problematical because of
copyright issues. Therefore, it is generally only distributed to people who
have collected the commercial recordings that it is based on, which is a
library of about 70 CDs. Most of the people who have it are members of large
research organizations with relatively deep pockets.

Re: setting up ABX tests at home

On Thu, 14 Feb 2008 16:36:45 -0800, Arny Krueger wrote
(in article <fp2mqt02u3k@news3.newsguy,com >):

> "Sonnova" <sonnova@audiosanatorium,com > wrote in message
> news:fp023j01mr2@news4.newsguy,com
>> On Tue, 12 Feb 2008 20:26:31 -0800, Steven Sullivan wrote
>> (in article <fotrhn02kb0@news2.newsguy,com >):
>
>>> This is what alwyas perplexes me about the 'you need to
>>> do long term listening' brigade. If you believe long
>>> term 'listening is required to get the true 'sound' of
>>> the gear into your brains, fine, go for it.
>
>>> And *then* demonstrate you're hearing
>>> a real difference, using a ABX comparison. Should be
>>> simple to 'pass' at that point, shouldn't it?
>
> Exactly. More to the point, not every part of every recording is adequately
> or maximally critical for all kinds of audible differences. For example,
> consider a subwoofer that distorts only deep, loud bass. If you don't listen
> to a recording with deep, loud bass at an appropriate level, you're never
> going to hear the problem. Another example would involve audible resonances.
> Resonances are narrow-band effects, and if your music doesn't have enough
> energy to trigger the resonance to the point where it is audible, you're
> never going to hear it.
>
>> Not always. Sometimes differences are subtle and many
>> times, certain recordings simply aren't good enough to
>> allow those differences to show up.
>
> I agree with the thinking, but I find the wording to be unecessarily vague.
> It's not the goodness of a recording that makes the difference, it is its
> appropriateness, which of course can include general desirable
> characteristics such as clarity. Several researchers including AES/MPEG/Bell
> Labs/Microsoft guru Jim Johnson have reported that sometimes it is bad
> recordings that make certain audible differences most audible.

Well, I was thinking about sound stage/imaging as the specific parameter when
I wrote that. You'll not find any differences in sound stage or imaging when
the recording used for the test has none. But I agree with your point too.

>> For instance, suppose
>> we are testing the soundstage/imaging of two different
>> sets of speakers and we use a 20-track recording that
>> places a microphone in front of each instrument in the
>> ensemble. An ABX or double-blind test will tell you
>> nothing of the imaging characteristics of those speakers
>> because, there isn't any imaging in the recording -
>> merely a bunch of instruments pan-potted to the right,
>> the left, and the center.
>
> Again the fault here is that ABX testing is singled out as being part of the
> problem. It isn't. The heart of the matter is that no testing paradigm will
> reliably reveal differences that aren't there. Testing methodologies that
> can't reject false positives can convince people that they are hearing
> things that aren't really there.

Well, I'm not blaming ABX or blind-testing per se, I'm merely saying that it
doesn't work in all cases for a myriad of reasons.

[quoted text deleted -- deb]

Re: setting up ABX tests at home

In article <fotrhn02kb0@news2.newsguy,com >,
Steven Sullivan <ssully@panix,com > wrote:

> Greg Wormald <greg.wormald@gmail,com > wrote:
*snip*
> > A goodly number of
> > years ago I ABX blind tested some interconnects, and then chose the ones
> > I preferred. Three months later I realised I was not listening as much
> > or enjoying it at the level I was previously. I found myself turning the
> > music down or off. When I went back to my old interconnects my listening
> > increased to it's previous levels of enjoyment.
>
> ..at which point , if you did an ABX, you would be able to tell
> the two apart , right?
>

Hi Steve,

I already implied that I was able to tell them apart--when I did the
first ABX test. Since I had a preference for one, that surely implies I
could tell them apart.

> This is what alwyas perplexes me about the 'you need to do
> long term listening' brigade. If you believe long term'listening is
> required to get the true 'sound' of the gear into your brains,
> fine, go for it. And *then* demonstrate you're hearing a real difference,
> using a ABX comparison. Shoudl be simple to 'pass' at that point,
> shouldn't it?

Already passed. See above.

My point was that while short-term ABX testing told me whether or not,
in the short term, I could discriminate between two pieces of equipment,
and then choose the one I preferred, the preference did not survive
long-term, real music listening for pleasure (as opposed to 'test'
listening).

Greg