How should transitivity be handled with rated pairwise preferences?

So, I’ve been discussing this idea of allowing voters to score the candidates in pairs, such that you could give a 100% margin to A>B and B>C simultaneously. In essence, it’s Condorcet but each matchup is done based on Score voting.

The main conundrum for this is to decide how its transitivity rules should operate. Suppose a voter gives a 30% margin to A>B (meaning they might have, for example, scored A 80% and B 50% in the A vs B matchup) and a 40% margin to B>C. Obviously, transitivity means they must give a positive margin to A>C i.e. they must score A higher than C in the A vs C matchup. But, since we’re talking about ratings, we also have to take into account how much higher A should be scored relative to C. The two ways I see are:

  • Take the strongest margin in all of the matchups transitively next to each other, and make this the minimal margin to be used for the next matchup in the chain.
    ** This means giving at least a 40% margin to A>C.
  • Add up all of the margins in the matchups transitively next to each other, and make either this or 100% be the minimal margin, whichever is smaller. (This is Score voting’s version of transitivity).
    ** In this case, this means giving at least a 70% margin to A>C.

Either of these transitivity rules gives the same result in Condorcet, because if a voter gives 1 vote to A>B and does not vote B below C, then they’ll automatically give 100% support (1 vote) to A>C as well.
The Score voting-based transitivity rule is more restrictive (i.e. if you score A>B 50% and B>C 50%, then you have to give A>C 100% based on Score’s transitivity but only at least 50% in the other transitivity rule), but I suspect it might be the more logical rule.

This method looks very unwieldy. I don’t think we should expect a voter to compare every pair of candidates individually! And should we disqualify a contradictory ballot? It would lead to a lot of spoilt ballots!

I also don’t think that article is very clear or concise.

The safest application of the idea would probably be for digital/online interfaces.

Well, technically they don’t have to be required to do so. Suppose there are 3 candidates A, B, and C, and a voter gives A a 5 and B a 4 (A:5 B:4) in the A vs B matchup, and B:4 C:1 in B vs C. Then we can reasonably interpret this as if the voter would score A:5 C:1 (or perhaps C:0?) in the A vs C matchup, due to transitivity.

I think if some consistent rules are codified with regard to this (in terms of “fixing” the inconsistency), then the problem is greatly minimized. See the bottom part (“Notes”) of this comment:

It’s equivalent to saying that the voter uses range [0,5] and therefore the vote would no longer be a Rated pairwise preference ballot (to be it, it is strictly necessary for the voter to evaluate each single pair separately from the others).
In short: to eliminate transitivity problems, you have to use ranges [0,5] or rankings (not rated pairwise).

I didn’t quite understand the count used in the Rated pairwise preference ballot, but it seems to me that in the end the winner is the same as the SV (if ranges [0,5] were used as a form of voting).
Can you give an example in which Rated pairwise preference ballot generates a winner other than the SV?
Make A[5] C[1] in this way A[5] C[0] perhaps generates different results from SV.

To show you how the voter can save themselves time in filling out their ballot, I made the example you quoted, where the voter can have their preferences “figured out” by the vote-counters where transitivity would make it somewhat obvious. But I’m pretty sure that one of the two versions of transitivity laid out in the OP is the correct one.

So, let’s say we have 3 voters with these rated pairwise preferences:
2 voters: A:5 B:1, B:5 C:1, A:5 C:0 (to more easily express this, though this representation doesn’t give the full information in all cases, it is “A 80%>B 80%>C”)
1 voter: B:5 C:1, B:5 A:0, C:5 A:0 (This is B 80%>C>A, with the ranks that lack %s assumed to be 100% strength pairwise preference)
So this yields a electorate-wide pairwise total of A:10 B:7 (A>B), A:10 C:5 (A>C) and B:10 C:3 (B>C), making A a “rated CW”.
But now let’s look at the scores this voter would give in SV:
2: A:5 B:4
1: B:5 C:4
The points are A 10, B 13, C 5. So here, the A-top voters, afraid of letting C win (i.e. maybe the example is 51 to 49, rather than 2 to 1), would exaggerate their preference for B, which makes B the winner. It’s a little unrealistic, but I didn’t want to make an example too much more complex than this, given the difficulty of the concept.
If you’re looking for an example where rated pairwise gives a different result than Condorcet, then see the link to the “Scored Pairwise Matchups” at the bottom of my comment above yours. It’s an actual rated pairwise poll I conducted.

Rated pairwise methods were discussed for a long time on the old Election-Methods mailing list.

Around 2004, James Green-Armytage introduced Cardinal Weighted Pairwise, in which pairwise preferences were inferred from a ratings ballot, then two pairwise tables were tabulated; one with standard pairwise votes, the other with the cardinal strength of that pairwise preference. Then the Condorcet completion method of choice was followed using the standard pairwise array to determine defeats, and the weighted pairwise array to determine the strength of the defeat. One could similarly make an Approval pairwise array with an explicit approval cutoff. The Green-Armytage’s hypothesis was that robust methods like Ranked Pairs, Schulze, and River would all get similar results.

Another pair of methods, also from the 2004-2005 time period, also use either approval cutoff or score along with rank-inferred-from-rating ballots: Approval Sorted Margins, and Score Sorted Margins.

With either approach, my preference for single-winner elections would be to use Approval Cutoff rather than score.

But none of those approaches allows a voter to give, say, an 80% margin to A in the A vs B matchup, and a 60% margin to B in the B vs C matchup, do they?

If transitivity is satisfied, then the vote can be represented as a ranking or with a range.
If transitivity is not satisfied, then voting on each pair of candidates is harmful.
The fact of not satisfying transitivity generates serious tactical votes.
Honest Vote: A:5 B:3, B:5 C:2, A:5 C:0 (convertible to SV: A[5] B[3] C[0])
Tactical vote: A:5 B:0, B:5 C:0, A:5 C:0

In your ex. I represent the distance with “-” (worth 1 point each):
2 voters: A:5 B:1 + B:5 C:1 +
A - - - - B - - - - C
A:5 C:0
A - - - - - C
These two representations are inconsistent, so they cannot be converted to a vote with range (SV).

I have not said it clearly: I wanted an example in which Rated pairwise preference ballot, which satisfies transitivity and can therefore really be converted into a vote with range, returns a result different from the SV.
With your example I understood the count better and it seems that I was right, that is: Rated pairwise preference ballot that satisfies transitivity in the votes, is equivalent to SV.

This is a contradiction. Your “tactical vote” involves the voter giving maximal support to their preferred candidate in each matchup, which is exactly how a ranking would be interpreted. If the voter voted A>B>C, A>B would be interpreted as 1 full vote (100% margin) for A in Condorcet, etc.

With a vote like this (in cardinal “RPPB”):
A:5 B:0, B:5 C:0, A:5 C:0
you are more likely to win one of your favorite candidates, than a vote like this:
A:5 B:3, B:5 C:2, A:5 C:0

If all voters use a vote like this (because it’s better):
A:5 B:0, B:5 C:0, A:5 C:0
then cardinality becomes useless, and it falls back to the ranking.

This rating:
A:X B:-, B:X C:-, A:X C:-
fails transitivity referring to cardinality, therefore it cannot be converted into a vote with range, but doesn’t fail transitivity referring to order (i.e. rankings).
To fail transitivity referring to rankings, you need a cycle like this in a vote:
A:X B:-, B:X C:-, A:- C:X


  • If there is transitivity in the order, RPPB can be represented as a ranking.
  • If there is also transitivity cardinal, RPPB can be represented as a vote with range.

In other cases there are problems.

Okay, thanks. I think we can go a step further in codifying the transitivity, though, and let me give a colloquial example to show that. Suppose a voter thinks A is somewhat better than B, and B is a lot better than C. Logically, it’d make no sense for this voter to say that A is also only somewhat better than C; this is because they prefer A over B, and think B is a lot better than C. Given that if the voter thought A=B, A would have to have the same preference in each matchup against other candidates as B had, it makes sense that A ought to be at least a “lot better” than C. The example I posed in the original post was essentially to ask, should we go a step further and require that, because A is preferred over B by some amount by the voter, should we expect that to increase the amount the voter prefers A over C, relative to the amount the voter prefers B over C? In other words, should A>C be even greater than “a lot”?

If you don’t want to satisfy transitivity then you will end up with a problem that you will have to solve arbitrarily (which is not necessarily the way the voter would solve it).
The positive side of the rankings or votes with range, is that problems like the one you proposed (or the cycle), are solved by the voter in his own way (as it should be).

If using a digital interface, we could disallow a voter from submitting their vote until they had met the proper transitivity constraints, while highlighting the error. So I think the question is still relevant: should the minimum margin the voter has to express for A>C be “a lot” or “a lot + somewhat”? This is keeping in mind that if the margin were to exceed “maximally better”, then we’d cap it at maximally better (i.e. if we had to add margins of 0.8 and 0.4 votes, then we’d cap it at 1 vote).

Ok, and in the end you would get the equivalent of a ranking or ranges, with the difference that the ranking and ranges are easier to understand, faster to use and you don’t have to make error messages appear.

Then there is also the problem of the order in which you make them evaluate the pairs (which can generate different votes depending on the order in which the transitivity problems are solved).

“A much better than B, B a little better than C” means that “A much + a little better than C”.

1 Like

One way to do rated pairwise would be to allow voters to score each candidate from 0 to 5, and then below, they indicate their rated pairwise support in the matchup(s) between their 1st choices vs 2nd choices, 1st vs 3rd, 2nd vs 4th, etc. With only 6 scores, that makes for at most 15 matchups to evaluate. There’s no way with this setup for a vote to fail ranked transitivity, and if the vote fails rated pairwise transitivity (i.e. voter says their 1st choice is 20% better than their 2nd choice and 2nd choice>3rd choice is 40%, but 1st>3rd is only 20%), then it should be relatively easy, even with a paper ballot, for the vote-counters to minimally adjust the vote to be transitive (i.e. they count 1st>3rd as 60%, and then adjust the other matchups on the ballot as necessary if this change makes them now fail transitivity, etc.)

I did not understand correctly; if you have candidates A,B,C to be sure that there are no cycles in the ranking (even using scores) you should only evaluate the following pairs:
A-B, A-C
A-B, B-C
A-C, B-C
If the voter has cycles in the head, the couples that I decide to have him evaluated can generate different votes (inaccettable problem).
If instead he had to classify all the candidates together, he would be forced to solve these cycles all at once.

Here is an example of a ballot for your scenario, and its interpretation:

1st choice vs 2nd choice: 4 and 3 (the voter selects both the 3 out of 5 stars and the 4 out of 5 stars, which can reasonably be interpreted as them giving their 1st choice candidate(s) a 4 and their 2nd choice candidates a 3 in this matchup)
2nd choice versus 3rd choice: 2 and 3
1st choice versus 3rd choice: 5

So, the ranking is interpreted from the scoring of each candidate (here it is B>A>C). Then we know the voter’s rated pairwise preference is B:4 C:3, A:3 C:2, and B:5 C:0. This passes both ranked transitivity and rated pairwise transitivity (1st choice is 1 point better than 2nd choice, 2nd>3rd is 1 point, and 1st>3rd: 5 points).

Could you show me how the ballot, with 3 or 4 candidates, is made (I mean graphically)?

So, for 3 candidates:

                       0 stars | 1 star | 2 stars | 3 stars | 4 stars | 5 stars

Candidate A:
Candidate B:
Candidate C:
1st choices vs 2nd choices:
1st choice vs last choices:
2nd choices vs last choices:

So the voter bubbles in how many stars they want to give in next to each of these.

Some points unrelated to the graphical design of this ballot:
If the voter only scores the candidates, but not any of the matchups, then their scores could be used to figure out their pairwise preferences. Specifically, if a voter scores B:4 C:3, then this could either be interpreted as B:4 C:3 or B:5 C:0 for that matchup. So this saves times for voters who don’t want to give a nuanced rated pairwise preference.
If the voter scores some but not all of the relevant matchups, then perhaps their candidate-scores could be used to auto-complete their pairwise-scores.

If, with 5 stars, I voted like this:
A: 5
B: 3
C: 0
1st vs 2nd: 5 and 0
1st vs 3rd: 0 and 5
2nd vs 3rd: 5 and 0
What form does my vote take in counting?