Reweighting test validity ratings by test score -- need expert advice

Hey everyone, I’m excited to be here! Some of you know me, some of you don’t… I’m really happy to see all the progress CES (and AV/SV/Star/etc.) has made in the last few years! I have a voting-related question for you experts out there:

I have created multiple tests, which I am having subject matter experts (SMEs) take and then validate. At the end of the test, each SME is asked to rate the validity of the test from 1-5 (5 being highest). High score is 100.

I want to re-weight the SMEs’ validity ratings proportionally by their scores to produce an overall re-weighted test validity rating. In other words, I want the formula to take each SME’s rating more or less seriously, directly proportional to that candidate’s score. (aka to ‘validate the validators’)

This “overall test validity rating” would be used to compare various tests’ validity ratings to each other.

Here’s a mock-up with some sample data: https://docs.google.com/spreadsheets/d/1WPBNuNRwkLOE80I0r2Q730zcLyjL3OOVldtJ46jVXZk/edit?usp=sharing

Does anyone have a suggestion for the best way to do this? (I’m not a mathematician, but ‘play one on TV’.)

I am not sure I understand the question but you want to rescale one value by another. There are a few ways to do it and most involve multiplying by the difference to the mean. This is not really an election problem but I did do this back when I was a professor when I needed to rescale to have a specific mean.

What you want to do is scale one to the others mean.

1 Like

Thanks @Keith_Edmonds! I’m replacing this question with another one, more related to voting methods (same underlying concept): Score Voting re-weighted by voter credibility?