Re: setting up ABX tests at homeArny Krueger wrote:
> "bear" <bearlabs@netzero,net > wrote in message
> news:fp588l01ah1@news1.newsguy,com
>
>> BECAUSE in order to do a SCIENTIFIC TEST:
>> - the test conditions must be known.
>
> That disqualifies every sighted evaluation because the test conditions are
> not as well known as they can be if blind testing procedures are used.
Don't care one way or the other.
I'm not here to defend sighted evaluations in terms of scientific or engineering
principles.
>
>> - the test's parameters must be known.
>
> Redundant with point one. Test conditions are the same as test parameters.
Only if they are reduced to a parametric quantity or other condition that can be
replicated!
>
>> - the test must be replicable by independent researchers!
>
> That eliminates any subjective evaluation because the subject is an inherent
> part of a subjective evaluation and every subject, being a human being is
> unique. You can't replicate the test independently, because doing a test
> independently eliminates the possibility using the same subject.
Nonsense.
The *test* meaning the acoustic and electronic conditions Arny.
Silly of you to think that! : )
>
>> - controls must be in place.
>
> Redundant with point one. Controls are same thing as test parameters.
>
>> This is basic, no?
>
> If you can live with the caveats above, so can I. ;-)
Sure, as long as you understand that these sorts of issues remain uncontrolled
thus far in all tests that I am aware of.
>
>>> You see, this is a common mistake. You've raised issues
>>> that relate identically to blind and sighted evaluations.
>
>> So what?
>
> So your elimination of sighted tests from your critiques invalidates the
> critiques themselves.
Huh?
Apples and oranges, Arny.
We're discussing your style of testing only.
>
>> I am saying (again and again) that ABX testing, while a
>> useful tool, is not dispositive of any issue whatsever,
>> except to the extent that the conditions for a test that uses a *true
>> scientific method* is done AND
>> those conditions clearly
>
> You just made the same mistake all over again. You've raised issues
> that relate identically to blind and sighted evaluations, but did not
> include sighted evaluations in your discussion.
Why should they be?
You discard them completely no matter what.
So, let's ignore them.
Concentrate on why ABX testing has flaws, and so those results can not be
reliably generalized beyond the specific tests??
>
>>>> EVEN WITH this information, I have yet to read a
>>>> published ABX test that would permit someone to
>>>> precisely duplicate the test. Not enough information is provided.
>
>>> I know of no published ABX tests that are secret. AFAIK
>>> whatever information you might relate to duplicating a
>>> test, can be obtained from the people who performed the
>>> tests.
>
>>>> Show me *one* that has the requisite information and
>>>> controls.
>
>>> Show me a sighted, non-level-matched, non-time-synched
>>> evaluation that has the requisite information and
>>> controls.
>
>> Who cares?
>
> If you don't care about the outcomes of sighted evaluations, that is fine
> with me.
No I don't actually.
Don't care about the outcomes of blind tests either as they now stand, with one
notable exception - already noted.
>
>> YOU are claiming scientific method and certainty!
>
> Actually, using the scientific method eliminates claiming certainty. The
> outcome of any scientific investigation is not certain but merely
> provisional, until more relevant results are obtained. In just my short 61
> years I've seen all kinds of non-trival scientific investigations in just
> about every major area of endeover invalidated by more relevant results.
Fine by me.
I have repeatedly noted some possible and plausible issues with the current
methods that I am aware of using ABX and similar testing...
>
>> THEY are not!
>
> To use a popular phrase, who cares? ;-)
>
>> The onus is upon those who claim science and certainty!
>
> "Science and certainty" is an invalid phrase for the reasons just given.
Yes, but Arny when you post you clearly imply that because of your "scientific
method" that your views are the only valid ones... or so it appears to this
reader...
>
>> I am not supporting any method or any point of view in
>> all this.
>
> Then why all these mentions of ABX, and not listening tests in general?
Because of your claims.
>
>> I am however pointing out the limitations of ABX to this
>> date, since so many seem to be claiming that ABX has resulted in certainty
>> of
>> result which can be applied and generalized.
>
> I would hope that one and all would use, if nothing else, my humble words
> above to dissuade one and all from believing that the results of ABX tests
> are certain.
Geez, I'd like to retain that as a quote for future use.
>
>> It can not, in my view.
>
> That is a truism. I stipulate that it is true.
>
>>> Show me an evaluation whose details are published in a
>>> high end audio publication that has the requisite
>>> information and controls.
>
> <no response to a simple request.>
Never claimed they were any good.
Why should I have to support those people?
>
>>>> Dr. Earl Geddes' research - and he is a hardcore
>>>> academician/scientist/researcher - has shown that
>>>> absolute values of distortion do not translate at all
>>>> to audibility of distortion.
>
>>> Geddes is an advocate of the same kinds of listening
>>> tests as you are criticizing. Why would you use his
>>> tests or conclusions as being authoritative?
>
>> BECAUSE THEY POINT OUT WHY EXACTLY MANY OF THE ABX TESTS
>> TO DATE HAVE A GAPING > HOLE IN THEIR METHOD!!!
>
> That is also a truism. I stipulate that it is true.
>
>> Again, Arny, I know this is difficult to quite figure out
>> but I do NOT have any prejudice against ABX or *any other
>> testing method*!!!
>
> It appears that we are training you to not pick on just ABX tests for having
> a common flaw that besets all tests, and probably many of those other tests
> even worse than ABX.
Nah, you're perhaps finally learning to listen/read what I write/wrote??
>
>> I do have a problem with claiming that
>> one tool (ABX in this case) automatically has results
>> that fully define these issues (the ones we're talking
>> about).
>
> I don't anybody making that claim. For example, like many others including
> Mr. Sullivant, for years I've been mentioning ABX/hr as being superior to
> ABX for many kinds of subjective testing.
>
>> I'm 100% fine with Dr. Geddes results, since they EXPLAIN
>> a whole lot about many many discrepancies between the
>> "low distortion" camp and the "tubes sound better" camp - and why some
>> "low distortion" amps and
>> other gear sounds like *ca-ca*.
>
> I don't think so. If you talk to Geddes and read his papers, the major focus
> of his work is loudspeakers, not amplifiers at all. I know for a fact that
> Geddes personal audio systems use inexpensive receivers as their amplifiers.
> Pioneer, Yamaha, Sony consumer line, and stuff like that.
yeah, I know. He's said as much.
But, iirc, he did some testing which included amplifiers using his "GedLee Metric".
>
>>> You might be interested to know that three of the ABX
>>> partners including myself and David Clark of AES fame
>>> attended a Chinese New Years party given by Earl Geddes
>>> and his wife Lydia Lee at their home last Saturday
>>> night. Dr. Geddes is a long time friend of ABX and other
>>> forms of blind testing, to say the very least! ;-)
>
>> So what?
>
> The point is that you can't use Geddes test for the purpose that you have
> been using it.
What?
His results clearly show something important that pertains to all sorts of
prejudices and presumptions about the audibility of distortions!!
>
>> You've got the wrong idea about this Arny.
>
> Show me!
Just did.
>
>> Since ur his friend, perhaps you should ask him to
>> explain that part of his research to you - and mention
>> that I suggested that to you? Which will be funny, since
>> I doubt that he thinks of me as being "in your camp" on
>> these sorts of things.
>
> It's not my camp or his camp, it is our camp. ;-)
Mmmm... to bad you've taken a dogmatic viewpoint.
I prefer to take a more even-handed and open-minded approach, while still
maintaining a good degree of healthy skepticism.
>
>> Even so, his research is solid, his designs very sharp,
>> and on this matter he has inadvertently put a major bit of science into
>> the
>> hands of those who have long claimed to subjectively note sonic
>> differences (that
>> have long been denied by "objectivists")!
>
> No such thing.
Funny. Then you fail to comprehend the significance of his GedLee Metric concept
at all! It shows that mere distortion figures do NOT correlate to audibility.
>
>> So, I thank him for reducing to
>> practice what I have long held without adequate means to test or otherwise
>> prove!!
>
> It would appear that there is quite a discrepancy between how you perceve
> Geddes' work and how he does.
Sure. No doubt. Apparently he doesn't see the ramifications the same way I do.
But it certainly applies as I have suggested.
>
>> Whether or not his speakers sound any good, or I would
>> like them, is another matter...! : )
>
> His speakers are basically very nice high efficiency 2-way speakers with a
> 15" woofer and a horn tweeter, that are very similar in many ways to EV
> ZX-5 professional audio speakers. Since his speakers are designed to have
> somewhat deeper bass, the enclosures are larger. Since the speaker is not
> designed for the highest possible SPLs, the crossover to the tweeter is at a
> lower frequency. I believe that the horn for the tweeter is one of his
> proprietary designs and uses an partially absorbant plug to reduce
> reflections in the horn.
Yes, I know.
>
The question is does it sound any good? ; )
- -bear