For the sake of realism in medical test exercises

Master / Vertiefung Künstliche Intelligenz II
1

Disclaimer: Dieser Thread wurde aus dem alten Forum importiert. Daher werden eventuell nicht alle Formatierungen richtig angezeigt. Der ursprüngliche Thread beginnt im zweiten Post dieses Threads.

2

For the sake of realism in medical test exercises
Let D be a disease and T a test to determine if a patient has the disease D, then the test has 2 characteristics that determine its quality:

  1. The sensitivity P(T|D), which is the probability that the test is positive if the patient has the disease (the chance that the test registers the disease if it is present) and
  2. The specificity P(-T|-D), which is the probability that the test is negative if the patient does not have the disease (the chance that the test won’t register the disease if it is not there) . These values are given by the manufacturer of the test.

Then you also have the prevalence P(D), which is the probability of the disease appearing in a person of a population (the prior probability).

Now using the above probabilities and Bayes’ rule, you can calculate what the patient is really concerned about, namely the probability that he has the disease if the test is positive ( P(D|T) ) and the probability that he is healthy if the test is negative ( P(-D|-T) ). These are called positive predictive value (PPV) and negative predictive value (NPV) respectively.

So the Type I and II errors ( P(T|-D) ) and ( P(-T|D) ) are only used in Bayes rule to calculate P(T), the probability of a test being positive, using marginalization and are not really displayed anywhere else.

So maybe you could change the medical test exercise for next semester if there are not any didactic reasons to keep it that way.

Let me know if I am wrong.

Good luck in the exams! :S

3

I don’t even understand what you want me to change.

Sensitivity is just 1-(Type II Error) and Specificity is just 1-(Type I Error). The exercise was to compute the positive predictive value by using Bayes Theorem. All that seems different to what you write is a) unnecessary terminology that computer scientists don’t need to know (unlike the terms “Type I error” and “Type II error”, which are used almost everywhere regardless of domain of applicability; unlike “Sensitivity” and “Specificity”) and b) whether you give a probability directly or its complementary one.

Am I missing something?

4

While we’re at it: I already strongly dislike the terminology “Type I” and “Type II”, because they give you no indication for which is which. Neither do “sensitivity” and “specificity”.

If it were up to me I’d just call them “false/true positives/negatives”. There’s zero ambiguity here what’s meant and I don’t need to look up which is which every single time :smiley:

1 „Gefällt mir“
5

I think the intended change here is to specify the question in terms of sensitivity and specificity (without necessarily using the terminology) as that is the information which manufactuers of medical tests usually provide.

So the “realistic” setting would be something like “you are given a test and on the box it says specificity (P(T|D))=x and sensitivity (P(-T|-D))=y. Given that the test is positive what is…”

6

Yeah,I see good reasons not to do that (Type I/II errors are standard terminology), no reason to actually do that (“medical professionals use a nonstandard convention” is no good reason to use the same nonstandard convention) and the difference between the two is a simple 1-x.

I’d rather use the exercise to introduce the notions of Type I/II errors. Those are actually useful.

I suspect the main reasons sensitivity and specificity are used at all is for marketing purposes. “Works in 99% of cases!” sounds better than “Fails in only 1% of cases” :smiley: