In “Principles of Disagreement,” Eliezer Yudkowsky shared the following anecdote:

Nick Bostrom and I once took a taxi and split the fare. When we counted the money we’d assembled to pay the driver, we found an extra twenty there.

“I’m pretty sure this twenty isn’t mine,” said Nick.

“I’d have been sure that it wasn’t mine either,” I said.

“You just take it,” said Nick.

“No,

youjust take it,” I said.We looked at each other, and we knew what we had to do.

“To the best of your ability to say at this point, what

wouldhave been your initial probability that the bill was yours?” I said.“Fifteen percent,” said Nick.

“I would have said twenty percent,” I said.

I have left off the ending to give everyone a chance to think about this problem for themselves. How would you have split the twenty?

In general, EY and NB disagree about who deserves the twenty. EY believes that EY deserves it with probability p, while NB believes that EY deserves it with probability q. They decide to give EY a fraction of the twenty equal to f(p,q). What should the function f be?

In our example, p=1/5 and q=17/20

Please think about this problem a little before reading on, so that we do not miss out on any original solutions that you might have come up with.

I can think of 4 ways to solve this problem. I am attributing answers to the person who first proposed that dollar amount, but my reasoning might not reflect their reasoning.

- f=p/(1+p-q) or $11.43 (Eliezer Yodkowsky/Nick Bostrom) — EY believes he deserves p of the money, while NB believes he deserves 1-q. They should therefore be given money in a ratio of p:1-q.
- f=(p+q)/2 or $10.50 (Marcello) — It seems reasonable to assume that there is a 50% chance that EY reasoned properly and a 50% chance that NB reasoned properly, so we should take the average of the amounts of money that EY would get under these two assumptions.
- f=sqrt(pq)/(sqrt(pq)+sqrt((1-p)(1-q))) or $10.87 (GreedyAlgorithm) — We want to chose an f so that log(f/(1-f)) is the average of log(p/(1-p)) and log(q/(1-q)).
- f=pq/(pq+(1-p)(1-q)) or $11.72 — We have two observations that EY deserves the money with probability p and probability q respectively. If we assume that these are two independent pieces of evidence as to whether or not EY should get the money, then starting with equal likelihood of each person deserving the money, we should do a Bayesian update for each piece of information.

I am very curious about this question, so if you have any opinions, please comment. I have some opinions on this problem, but to avoid biasing anyone, I will save them for the comments. I am actually more interested in the following question. I believe that the two will have the same answer, but if anyone disagrees, let me know.

I have two hypotheses, A and B. I assign probability p to A and probability q to B. I later find out that A and B are equivalent. I then update to assign the probability g(p,q) to both hypotheses. What should the function g be?

Pingback: Even Odds | By Way of Contradiction

As promised, here is my analysis.

First of all, I think that the g and f functions should be the same. My reason is that to be completely satisfied with who gets the 20, both people should update their probabilities to the same value, so f(p,q) seems like the probability that EY and NB should both walk away with saying is the probability that EY deserved the 20. Since EY and NB trust each other, this should be the same if there is a single person with both beliefs learning they are equivalent. Some of the features I want in f are easier to justify in the g example, which might have caused some of my mistakes.

I think this question is inherently about how much we treat p and q as coming from independent sources of information. If we say that the sources are independent, then #4 is the only reasonable answer. However, the dependency of the evidence is not known.

I think f should have the following properties:

A: f(p,p)=p — If we wanted f to be a general rule for how to take two probabilities and output a probability that is an agreement of the two, then there is a danger in setting g(p,p) to anything other than p. In that we can repeatedly apply our rule and get different answers. We update p and q to g(p,q) and g(p,q), but then these are our new probabilities, so we will update them both to g(g(p,q),g(p,q)). Our new answer is not consistent under reflection. Therefore, I think g(p,p) should be p, and this makes me believe that f(p,p) should also be p. (Maybe I am working by analogy, but I still believe this.)

B: f(p,q)=1-f(1-q,1-p) — This is just saying that the answer is symmetric with respect to swapping EY and NB.

C: f(p,q)=f(q,p) — This is saying that the answer is symmetric with respect to swapping who says what. This property seems really necessary in the g problem.

D: f(1,q)=1 — If EY knows he put the 20 in, he should get it back. In the g problem, if A is a theorem, and we learn B if and only if A, then we can prove B also.

E: p1>=p2, then f(p1,q)>=f(p2,q) — If EY is more sure he should get the money, he shouldn’t get less money.

F: f(f(p,q),f(r,s))=f(f(p,r),f(q,s)) — This relation doesn’t mean much for f, but for g, it is saying that the order in which we learn conjectures are equivalent shouldn’t change the final answer.

G: f is continuous — A small change in probability shouldn’t have a huge effect on the decision.

I think that most people would agree with B, C, and E, but looking at the comments, A and D are more controversial. F doesn’t make any sense unless you believe that f=g. I am not sure how people will feel about G. Notice that if B, C, and D together imply that f(1,0) is not defined, because it would have to be both 1 and 0, I think this is okay. You can hardly expect EY and NB to continue trusting each other after something like this, but it is necessary to say to be mathematically correct.

Now, to critique the proposed solutions.

1 Violates C and D (I didn’t check F)

2 Violates D

4 Violates A

3 Does not violate any of my features.

I did not do a good job of describing where #3 came from, so let me do better. #3 chooses the unique value f, such that if we wanted to update EYs probability to f and update NBs probability to f, this would take the same amount of evidence. It satisfies A because if they are both already the same, then it doesn’t take any evidence. It satisfies D, because no finite amount of evidence can bring you back from certainty.

We do not have enough information to say that #3 is the unique solution. If we were to try to, it would look like roughly like this:

If we think about the problem by looking at p* =log(p/1-p). Then #3 just finds the f* =(p* +q* )/2. E and A together tell us that f* should be somewhere between p* and q* but it is not immediately clear that the arithmetic mean is the best compromise. However, I believe this implies that we can apply some monotone function h, such that h(f* ) is the always the arithmetic mean of h(p* ) and h(q* ). B tells us that this monotone function must be an odd function (h(-x)=-h(x)).

From here, #3 assumes that h(x)=x, but if we were to take h(x)=x^3 for example, we would still meet all of our properties. We have freedom from the fact that we can weigh probabilities with different distances from 1/2 differently.