Sensible Ways of Measuring Score Distributions

I have been considering methods of evaluating distributions for quality in terms of providing many people with relatively high utilities and few people with relatively low utilities, i.e. placing a “utilitarian” measure on distributions. It is my belief that “minimax” methods are a mistake in terms of voting systems, since political and social landscapes for example are fairly dynamic—that is, I feel like it is OK for few voters to be very dissatisfied, as long as it isn’t the same voters who are very dissatisfied consistently or all the time, and as long as those voters were at liberty in engaging with the risks of dissatisfaction for the potential of reward. “The needs of the many outweigh the needs of the few,” but the “many” shouldn’t be some group that you can materially identify, if you catch my drift. All voters should have the liberty to engage with risks for potential rewards. If the rewards are real, then so are the risks.

But that’s enough waxing philosophical. My recent thoughts have been concerned with bounded score distributions, which is what we would be facing anyway with distributions of discrete scores over candidates. Anyway, suppose candidates are scored on a discrete scale from 0 to N. (By the way, if possible, we should try to get LaTeX formatting to work on the new forum at some point, not sure how that is done). Take a function


such that T(0)=0, T(N)=1, and A<B implies T(A)<T(B). This will constitute a mapping or “translation” of the discrete scores into the real interval [0,1].

Next, for a given score distribution, consider the cumulative distribution function (CDF) given by F:[0,1]—>[0,1], which maps X to the fraction of voters whose score S satisfies T(S)=< X. Note that this distribution function depends on our choice of T. Like all CDFs it is weakly increasing, and in this case it’s always the case that F(1)=1.

To measure a distribution, simply calculate the area beneath the curve of F from 0 to 1. The intuition is that the smaller this area is, the better the distribution. However, this measure depends on the choice of T. It is fairly easy to find distributions whose scores will swap in order according to different translation maps.

The experimental science comes in determining what an appropriate T should be to match with “normal” or “social” intuitions about which distributions are superior to which others. No matter what, it is going to be arbitrary, but ideally it would correspond nicely with what most people would expect most of the time. As a voting method, a single translation map T should be used to compare distributions—essentially it is the measuring instrument. T should be varied or calibrated to optimize “voter satisfaction.” (Which ironically is another matter of distributive justice). To calibrate T, one can simply have the algorithm running side by side with human distribution selections, keep a record of the human selections, and generate random maps for T until its winning distributions match those in the record.

A second, more complicated way of measuring distributions would be to try to take the “average distance” of points in the graph of F to the point (1,0). If this average is minimized, then F is 0 until a sharp jump up to 1 at the very end, which indicates that every voter has given a maximal score—this is the ideal distribution. The same occurs when the area under the curve is minimized, so this method of “average distance” is similar but is more complicated to calculate, so probably isn’t as neat. For either measure though, topologically speaking, lower measures will naturally tend to correspond with distributions that have simultaneously high means and low variances. More directly, they will approach the ideal distribution.

These give straightforward methods of selecting a distribution from a multitude—just select the distribution with the lowest measure. If there are any ties (which is extremely unlikely), then I don’t know, break the tie at random. Still, these methods are arbitrary. Nonetheless, they define a very broad class of voting systems that are simple to evaluate and more than just superficially utilitarian.

Again, any thoughts, suggestions, or concerns are welcome. Thanks for checking it out!

I follow nearly all of what you are saying but miss the point. Perhaps an example would be illustrative.

1 Like

Ok, here is an example:

This shows two score distributions, for two different candidates, along with their cumulative distribution functions on the left. The one on the top has a superior distribution. It also has a smaller area beneath its CDF.

You can also observe that starting from a fixed distribution and a fixed placing of those vertical lines, the quality of a distribution can only increase as the area under its CDF is made smaller.

So my premise is that this may be a sensible way to compare different distributions for quality. If you choose a sensible spacing of vertical lines (which corresponds to choosing a map T as described) then the procedure of selecting the candidate with the minimum CDF-area may yield a very good voting system, i.e. one that is not majoritarian, and that operates to establish a “distributionally just” distribution of “utility” (score points).

In this example, the scores are mapped to equally spaced out spots along [0,1] (or are supposed to be according to my sketch). But it might be better to have something like this:

The spacing is arbitrary, but here I’ve tried to space them out “harmonically.” Basically, 1 is mapped to 1/(1+1/2+1/3+1/4+1/5), 2 is mapped to (1+1/2)/(1+1/2+1/3+1/4+1/5), and so on. My reasoning for that is heuristic. First I drew the picture based on my own feeling for what ought to be right, and then I compared it with the harmonic spacing as a hypothesis and it was basically a complete match. But maybe a different spacing is better. I don’t know.

Yea ok I get it. We have discussed similar systems in the past this is just a generalization. I would think the the spacing (weighting on score) should be even. Also, I think we should take the integral (cumulative sum) from the top down. You would then need to take a cutoff point otherwise all candidates get the same results. A Hare quota seems logical since that is how you get PR. This then produces Sequential Monroe voting. So what you have done is generalized that system. I think thought that it is the best system in the class. Maybe @parker_friedland can comment since he is the inventor of this system.

So actually I think there is a misunderstanding of the system. No cutoff point is required. I am taking the area beneath the whole CDF curve in the pictures. If you understand me, it’s a double integral of the score distribution, not a single integral. Different distributions will almost surely give different areas. So this makes me think perhaps I haven’t explained it very clearly. I wager this is a new kind of system, and not just a generalization of an old kind of system. I could be wrong.

About the spacing, having an even spacing seems like a natural idea, but it doesn’t necessarily line up with how the typical person feels about different distributions. That’s why I think the spacing should be modified. Basically, I want the algorithm to model how the typical person might choose between different distributions.

In fact, I think the most natural spacing would be to have something very close to T(k)=1-(1-T(1))^k. I will try to elaborate on why. Unfortunately this exact spacing can’t be achieved with a finite number of bins.

There are good reasons that you would want it to be an even spacing. It would make the mapping of utility to score obey Cauchy’s functional equation. This I suspect is how people think about it.

I get your meaning better about the two distributions and double integral. Sequential Monroe would then have the second function being a step function.

Perhaps. I don’t necessarily subscribe to a “score~utility” idea. Utility is too abstract. No matter what the spacing is, people in theory should adapt to it strategically over time. Basically I would want the spacing arranged so that as little adaptation is necessary as possible, so that the system becomes stable as quickly as possible as it is used.

I will have to look more into the Monroe method. It’s true that any particular realization of a system in this class can be recast as evaluating the distributions by taking a particular linear combination of the fraction of scores in each bin, but the relationships between the coefficients are constrained geometrically by the diagrams. And rather than looking for the highest score, we are looking for the lowest score, so the mathematics is a bit different.

In any case, I believe this kind of system when implemented correctly is strictly superior to standard score systems. There are still pitfalls I’m sure such as exaggeration. This is why I was envisioning this situation:

The issue with this is that people don’t want to vote more than one time.

That is exactly my point. If that is true then we should choose the simplest. Otherwise voters will just have to invert it with their utility to score map.

Complexity is also an issue. I think you need to decide what system you think is best an lay it out. You are describing a class of systems so it is really too abstract. Come up with the ideal example and give it to the forum to evaluate.

Well the system with even spacing and integer scores from 0 to 5 essentially gives a score of


to each candidate, where p(k) is the fraction/number of voters who scored that candidate as k, and devoid of ties the candidate with the lowest score is declared the winner.

These systems are essential just as easy to implement as any scoring procedure. Unfortunately they probably have the same problems.

But I would like to elaborate about what I picture as an ideal system within this class of systems. Basically it is captured by the purely theoretical system with infinitely many bins and a mapping T with T(k)=1-(1-T(1))^k, and some choice for T(1). This would basically ensure that the process is self similar, in the sense that, for example, if two distributions (candidates) have the same fraction of zero scores as each other, those zero scores can effectively be ignored when comparing the two distributions—you can pretend that 1 is the new 0, and evaluate from there.

Unfortunately, it is not possible to achieve this property with a finite number of bins and still have T(0)=0 and T(N)=1. But in my opinion, a mapping T that tries to approximate this self-similarity so that we can approach the ideal situation would be preferred. The even-spacing does a comparatively poor job of approaching this ideal. The harmonic spacing does a better job. There are probably various other spacings that could serve well. But a key property that should be upheld is that the spacing between larger scores should be smaller than the spacing between lower scores.

So to respond to your light request for an ideal system, I would propose the harmonic spacing and integer scores from 0 to 5 as a starting point. This would effectively give scores of


To be more mathematically rigorous, one can take an information projection of the distribution of distances in the ideal spacing onto the set of spacings with a fixed number N of bins. This establishes a single parameter family of distributions over each number of bins, with the parameter being the choice of T(1).

If this information projection is done with N score bins, you find the somewhat simple formula


where B is a variable parameter in [0,1]. In fact as B approaches 1, T(k) approached k/N, which gives the even spacing. However, as B approaches 1, the “ideal solution” also diverges. So if we want to be able to approach the ideal as N grows larger, then B should be strictly smaller than 1.