CES's new anti-STAR Hit Piece. :(

“The lackluster improvement over simpler methods is not especially surprising given the method’s origin. A look at existing research such as the French studies referenced earlier—which existed before STAR was developed—would have also given this insight. These kinds of obstacles are inevitable to those outside of a technical field who do not do the requisite research before going all in.”

Hi Aaron and CES,

I saw the new article and am commenting below as I read though. Before diving in I want to say thank you for covering STAR. It’s been a long time coming and it means a lot to us.

A few things I loved:

  • Nice to see CES referencing and citing VSE.

A few suggestions: (CES quotes in italic)

  • The intro line is confusing. Can we reword this please?: “Score candidates, a top-two runoff is simulated through comparing these candidates’ paired scores to determine the winner”
    Suggestion: “Score candidates from 0-5, the two highest scoring candidates advance to an automatic runoff where the winner is the finalist preferred by more voters.”

  • “Inventing one’s own voting method has become popular as it’s so easy to share ideas online.” Line one of the STAR section comes across as an insult. Do we really need to say this?

  • Could we have a section just on STAR? I think it deserves it. A 7th special mention section could also be included to talk over other hybrid methods like 321 etc that don’t have political traction but deserve special mention, or there could be a paragraph or so mentioning this at the end of the STAR section. Mixing these methods together breaks the format of your article and is just confusing.

  • “The stated premise was to create a voting method where votes were able to equally cancel each other out—though virtually every cardinal method already does this.” No. The test of balance (described above) is the stated core tenant of the Equal Vote Coalition, which was founded to fight for equality in the vote and which ran the first statewide initiative for a Unified Primary with Approval Voting in 2014, (just like CES is doing in St. Lous now.)
    There were many reasons why STAR was invented. The goal was a voting method which delivered on the best features of both scoring and ranking methods while addressing legitimate criticisms with both.

  • “[Scoring] takes more concentration for the voter to assign each candidate a score. This is also information that voters aren’t used to seeing on other election ballots.” This is a subjective opinion. Many people find Scoring easier as it’s nuanced enough to actually just express the opinions they already have. Millions of people have used 5 star ratings and it’s likely the most familiar system for expressing a nuanced opinion.

  • If you are concerned with the complexity and voter understanding why use novel terminology to explain it, such as “simulated runoff component.”

  • “To the degree that normal range voting voters are more tactical, STAR could make up for that with a tiny improvement.” It’s not just about voter behavior, it’s about making it safe and smart (and possible) to vote one’s conscience. STAR Voting’s runoff makes it safe to honestly score candidates, even if you don’t like any of the font-runners. And the outcome is notably better for minority voters because of it. Even if none of your favorites can win, with STAR, your (runoff) vote helps prevent your worst case scenario.

  • “The additional complexity added by STAR still appears to add only trivial if any real improvement both according to Quinn’s simulation shown earlier and separate independent analysis.” No, this is incorrect. Quinn’s study (graphic 2 on strategy) clearly shows that STAR significantly outperforms Score, Approval, and RCV in terms of incentivizing strategic voting. In fact, it’s one of the only cardinal methods that does not incentivize it. (See ratio of when strategy works vs backfires in attachment graphic.)

  • “The lackluster improvement over simpler methods is not especially surprising given the method’s origin. A look at existing research such as the French studies referenced earlier—which existed before STAR was developed—would have also given this insight. These kinds of obstacles are inevitable to those outside of a technical field who do not do the requisite research before going all in.” Burn! Is this an opinion piece? A revenge piece? We’ve bent over backwards to speak well of CES despite numerous issues. You are the CENTER FOR ELECTION SCIENCE. People come to you for facts and data analysis. This reads like personal drama. Friendly competition and disagreements on strategy or priorities are understandable but there is NO reason for this.

  • “Range voting, as a base in itself, performs excellently. STAR should perform no worse than this.” All studies to date show it does better. It’s hard work to be this negative in the accuracy section about a method that topped the charts.

  • “STAR voting’s largest issue is that it quests to move an inch nearer to ideal winner selection at any cost to complexity or practicality—despite easier alternatives being available that perform nearly identically” This ignores the political reality that Score Voting’s strategic voting incentives are widely regarded as a deal-breaker. Agree or disagree all you like. (I like Score a lot myself.) It’s a political reality that is supported by the data.

  • “A separate ballot initiative for STAR voting also failed to gather enough signatures.” False. We’ve proved twice over that enough signatures were submitted. How about some solidarity? Campaigning without huge grants is hard. Seriously. This chunk serves no purpose other than being mean.

  • Framing. This piece juxtaposes STAR with Choose-One and Approval, nether of which are even contenders out west. The piece also ignores the fact that STAR’s meaningful competition is IRV. CES clearly sees NO value in bridge building or compromise, but we do see value in reaching out to other reformers and working to deliver on their goals. Preference voting with majority preferred winners is the dominant trend in voting reform. Is it really a bad thing to offer a cardinal system that can do that too, but better?

To the CES Board of Directors. Is the hit piece quoted above representative of the way you all feel? Did the board approve or even read this article before it went live? Do you want us to write and publish a scathing and spiteful article about Approval like this? We could… but we do not think that furthers the movement, and it would go directly against the mission statement of the Equal Vote Coalition, which values solidarity and collaboration to build a movement that can win for generations.

Broken-hearted,
Sara Wol​k

​Director of Campaigns- ​STAR Voting​ for​​ Eugene and Lane County​
Executive Director - ​Equal Vote Coalition
971-222-9364
​www​.​starvo​ting​.​us​
www.equal.vote

7 Likes

Do you have a link?

Yes, exactly correct. Studies have shown that it is faster to score than to approve since the cognifive burdin is lower when the choice is less stark.

Agreed, is that a personal attack? As far as I know the methods origin is @ClayShentrup. He knows his stuff.

To be fair there is no one metric to compare on so this statement is meaningless. For example there are some arguments that it fails NESD where score does not. It gets different winners so there will be tradeoffs. Typically however Bayesian Regret is the most important metric once the standard criteria like Monotonicity and clone resilience are satisfied.

1 Like

Link: https://www.electionscience.org/voting-methods/an-assessment-of-six-single-winner-voting-methods/?fbclid=IwAR15ppj5UKVWox6WgTRsUgdajeY0ZVsRbbeSs3kvyL8U8kSm5of3qz92Y7o#starvoting

“this statement is meaningless”

Yes, it’s factually meaningless, but in terms of it’s negative impact it’s anything but. Election Science isn’t just about the science anymore (but maybe should go back to it’s roots.) It’s about political advocacy. These statements and worse directly undermine 10s of 1000s of volunteer hours working to replace plurality voting and eliminate vote splitting.

As far as I know the methods origin is @ClayShentrup

Mark Frohnmayer and Clay Shentrup, with a little inspiration from a Rob Richie comment.

And also just an instant version of range + runoff that came out top in Warren Smith’s Bayesian Regret simulations long before.

Right. If Rob Richie is the “deadbeat dad” of STAR Voting then Warren Smith is certainly the Godfather.

I agree that CES shouldn’t be criticizing STAR voting. It makes no sense. Both organizations should be cooperating toward a single goal: get rid of plurality with something that solves 99% of the problems, which both STAR and Approval do.

I agree that this statement is really misleading and probably outright wrong:
“this takes more concentration for the voter to assign each candidate a score.”

I would much prefer STAR voting in terms of how much concentration is needed. You don’t have to think so hard about which one is most likely to be a front runner. While you are technically giving more information, it’s very nice to be able to express who your favorite candidate is without worrying that you should be also helping someone who is more likely to be a front runner.

Example: A person who preferred Nader to Gore, but know Gore was more likely to be a front runner and prefer him to the other front runner. With Approval, you have to think “who are likely to be a front runner?” If you think it will be Gore and Nader, vote for Nader only, if you think Gore and Bush, vote for Nader and Gore. With STAR, you just give Nader 5, Gore 4, and Bush zero. You get to express your preference for Nader over Gore, while knowing that in the end, Gore will get your support over Bush. I find this far less cognitively challenging.

But at the end of the day, I’d be thrilled if we adopted STAR, I’d be thrilled if we adopted Approval. They both solve the problem.

3 Likes

Yeah, if a voter really didn’t want to put thought into their scores, they could just min-max and cast an Approval-style ballot.

1 Like

I’d be interested to know how the scores out of 10 for each criterion in the article are calculated, or whether they were plucked from thin air. On the face of it, it looks completely subjective.

The article certainly doesn’t look like an exercise in science.

And I know this is off topic, but I’d be interested to know how @Jameson-Quinn found such a difference between Schulze and Ranked Pairs in his simulations, considering that I can’t imagine that any election ever would actually have a different result under the two methods. It would certainly be rare anyway, but in his simulations it must actually be quite common when there is one-sided strategy.

2 Likes

I generally agree with @Sara_Wolf here. As a fan of both approval and STAR I was disappointed to see this post be so antagonistic. I really hope this does not become the norm for relations between cardinal voting method advocates.

Do you have a citation for this? I’d like to have something to link to in order to back this up if I ever find it relevant to a point I’m making.

I’m guessing the post was referring to the Multnomah County initiative, not the Eugene initiative as you seem to have interpreted it.

This would also be a nice claim to have a citation for.

1 Like

Warren Smith’s web site has info on this here and here

Rob Richie and FairVote are notorious for hammering this point home and it’s one of the few valid points they make. League of Women Voters sites it in their analysis, and used it as the foundation of their anti-cardinal position in Oregon. I recently spoke with Professor Darlington from Cornell, who has a long history simulating methods and who advocates for Condorcet and he said something to the same effect. He regards it to be a deal-breaker, even though he thinks Score would otherwise be one of the best methods. He said that that’s the consensus in ordinal voting circles which are educated on voting theory.

With Score the best strategy is generally to give all your candidates min or max scores, while also being sure to max-score your lesser-evil if you don’t think your favorite can win. Essentially Approval voting on a Score ballot. This is explained in detail on Warren’s Smith’s site and it looks like this article (which I didn’t actually read) has the details. https://www.rangevoting.org/RVstrat.pdf

The point is also backed up by VSE simulations, which show that Score voting has the strongest strategic incentives second only to Plurality voting (Plurality is ~3x worse!) and that the incentives increase the larger the ballot scale.

Interestingly with Approval Voting, as with Plurality, the overall election results improve when voters are strategic. This is why most cardinal advocates who are familiar with it are not concerned by it. (This is the opposite of systems like STAR and Condorcet which are more accurate overall when voters are more honest.)
[Pgf. edited: 1st draft accidentally said Score instead of Approval here.]

Ideally we want a method which does not incentivize strategic voting, and which is also more accurate overall when voters are honest.

Most voting methods incentivize the type of voting behavior which produces the best outcomes, to varying extents. (STAR, and Condorcet are accurate with honest voting and honest voting. Approval and Plurality incentivize strategic voting and perform better when voters are strategic.)
[Edited, to remove Score from this section and add it below. 1st draft had it in the wrong category.]

A notable exception is IRV, which incentivizes strategic voting ~3:1 (according to the VSE graph on strategy works v. backfires) but which is more accurate when voters are honest. Luckily the strategy in RCV isn’t that obvious. That said I figured it out and others can too. Score Voting is also most accurate when voters are honest, but does not incentivize that behavior.



I want to end by stating that Score Voting is a great voting system. That it’s a lot better than Approval, and that this is the only valid concern with it, while most other methods have multiple issues which are worse than this.

This is also the reason that STAR was invented and why most Score fans love STAR. The runoff eliminates this strategic incentive or renders it more or less in-actionable.

3 Likes

Agree with most all you say, except that, while I love the idea of STAR, I don’t like score at all. It would put me in a sort of conflicted state, where I would both want to vote honestly and strategically. At least with approval, I know from the get-go that I am going to vote strategically. Instead of seeming to ask “how much do you like each candidate?”, approval asks “which candidates do you want to give a point to?” It seems to go ahead and own the fact that it is putting you in the position of having to consider how likely each candidate is to win.

In that sense, it seems almost insulting to my intelligence, in the sense that it is providing (suggesting?) all these options to dilute my vote. STAR doesn’t do that, since voting honestly is much more likely to be a perfectly strategic vote as well.

One of the best things about not having incentives to be strategic is that, especially on “downballot” elections, voting is way easier because you really don’t have to try to study the polling data, if it even exists.

I’m a bit confused on how you say score can be so much better than approval, while also saying “with Score Voting … the overall election results improve when voters are strategic”. I guess I can see it, in a way… in that purely naive voters under approval will sort of randomly set a threshold, and for those voters, it makes sense to just gather the richer information score provides. Ok. :slight_smile:

Regardless, your main point, that score’s “incentives are widely regarded as a deal-breaker,” I heartily agree with. I am very happy to see that so many that were advocating score are now getting behind either STAR or approval.

I sincerely hope that the STAR people and the approval people find a way to support one another.

2 Likes

Sorry. Good catch. That was a typo. It should have said “Interestingly with Approval Voting, as with Plurality, the overall election results improve when voters are strategic.”

I also had Score in the wrong category in the paragraphs below. Fixed! You can fact check me by looking that the blue and red dots on the VSE chart, and also looking at the ratios of strategic voting incentives on the 2nd VSE chart.

1 Like

Cool, yeah I see you added “Score Voting is also most accurate when voters are honest, but does not incentivize that behavior.” That pretty much sums up why it feels icky to me and puts my brain in conflict with itself. STAR is a vast improvement in that respect.

And while approval is similar to plurality in that they both are consistent (both incentivizing and benefitting from strategic voting), at least approval ends up with a pretty good result, while plurality still ends up with crap, regardless of whether you vote strategically or not.

1 Like

3 Likes

I like the colors. :slight_smile:

It’s a bit tricky to make a graphic like that since you’ve got five categories/variables and you’re trying to force it onto a 2-dimensional image. Like if you had a method that was simple and accurate, but neither expressive or equal, it would be hard to find where to put it. Regardless, you should be happy to hear that I agree that STAR is pretty well into the sweet spot.

I’m not 100% sure I know what equality means in this context, or how it differs from accuracy. For me, the gold standard of “fairness” is whether it selects the “first choice of the median voter”. For typical “human candidate” elections, the definition of “median voter” can be hard to nail down, but there are some elections – say you are voting for the temperature to set the office thermostat – it is pretty clear what the median voter is. (i.e. the person whose preferred temperature is the median of everyone’s preferred temperatures)

I would argue that such an election is fair because each voter has equal “pull”, regardless of how extreme their views are. If I want it cold enough to see my breath, my vote pulls it downward no more than the vote of someone who just wants it a tad colder than typical. To me, that is the correct result even if some would say it isn’t optimal according to VSE/Baysian regret type measurements.

Here’s a widget I made some time ago that tries to demonstrate the concept: https://pianop.ly/voting/median.html

Would accuracy be an election that selected the average temperature, but somehow magically managed to get people to not exaggerate their preferences? I don’t know.

I do think that Condorcet methods should be considered “equal”, by any reasonable definition of equality I can come up with, though, and it doesn’t show that way in the graphic. (Condorcet certainly fails on simplicity, though)

Some interesting discussion on median vs. mean at:

Yeah I’m the one who posted that one on Reddit. (were you there?) I made that widget and video after people on a different Reddit EndFPTP thread argued that exaggeration was incentivized even in a median-seeking election. I disagreed and still do.

Ok, so vote splitting is the enemy of equality. Vote splitting does indeed cause wild distortions, although my personal view on it is that the main reason vote splitting is bad is because it has the long term effect of forcing us into an ugly and unstable partisan situation where a large percentage of the people hate whoever wins.

Regarding accuracy – I could be wrong regarding Baysian Regret and VSE, but I suspect they favor a sort of thing that is always going to be mutually exclusive with honesty. Going back to the office temperature example and the person who likes it freezing cold, VSE would factor that person’s opinion more heavily than the person who likes it ever-so-slightly warmer than typical. In my view both should have the same pull, but in opposite directions. Any system that gives greater weight to extreme views is in my opinion 1) unfair, 2) incentives exaggeration, and 3) unstable.

I suspect that condorcet methods and STAR all meet my criteria of “median seeking.” Approval probably does if you assume people have an awareness of polling data. Not so sure about score.