Problem 9.1 - Concerning the Information Gain Formula

[system]

Disclaimer: Dieser Thread wurde aus dem alten Forum importiert. Daher werden eventuell nicht alle Formatierungen richtig angezeigt. Der ursprüngliche Thread beginnt im zweiten Post dieses Threads.

[tyr]

Problem 9.1 - Concerning the Information Gain Formula
Hi, I have two question regarding this topic:

1. The formula given for the information gain from an attribute test does not explicitly say, what to do when p_i or n_i is 0. ( Obviously, the log_2(0) is not defined). I believe from my observations of examples, that the whole P_i * log_2(P_i) part of the I(P) formula is just set to 0?

2. Do we have to implement DTL so that the [m]target[/m] attribute could have more than two values? I ask this because in the lecture we implicitly limit the world to binary targets. I mean the step from binary to to an arbitrary number of possible values is not that hard, but I guess you do not want to rework that part of your code once it is done

[Jazzpirate]

---

The formula given for the information gain from an attribute test does not explicitly say, what to do when p_i or n_i is 0. ( Obviously, the log_2(0) is not defined). I believe from my observations of examples, that the whole P_i * log_2(P_i) part of the I(P) formula is just set to 0?

Yes; that will basically render the information gain maximal. However, depending on your implementation you’ll never get to that point, since the p_i/n_i=0 iff all remaining table entries have the same query-value; which means you need to return that anyway.
Nevermind, that can happen in the body of the sum of course.

2. Do we have to implement DTL so that the target attribute could have more than two values?

Yes; you’ll note that the data structure are such that I can query for an arbitrary attribute in the table

[Stella]

---

Just in case: How are we supposed to treat contradictory data?
E.g if we have (Day=Monday, Hour=8, Awake=Yes) and (Day=Monday, Hour=8, Awake=No) and solving for Awake → should we throw an exception or what can be done?

[tyr]

---

@Stella

Just in case: How are we supposed to treat contradictory data?
E.g if we have (Day=Monday, Hour=8, Awake=Yes) and (Day=Monday, Hour=8, Awake=No) and solving for Awake → should we throw an exception or what can be done?

I believe this comes down to the line

else if attributes is empty then return MODE(examples)

MODE(ex) = most frequent value in ex

from the DTL function from the lecture.

So it comes down to what to do in [m]MODE[/m] when there are multiple most frequent values.
(I have no ideas…)

[Jazzpirate]

---

I believe this comes down to the line
else if attributes is empty then return MODE(examples)

MODE(ex) = most frequent value in ex
from the DTL function from the lecture.

Seconded

So it comes down to what to do in MODE when there are multiple most frequent values.
(I have no ideas…)

In that case, just pick the first one / an arbitrary one. There’s little else you can do, except introduce a new „do whatever“ result

[ThiloK]

---

In Java, rows and attributes in the abstract class are not visible in the extension when both are not in the same package unless they aren’t declared as public/protected. Are we allowed to change that?

[Jazzpirate]

---

Oh sorry, yes, please - you’re allowed to change that