How is reputation calculated?In short, coaches earn or lose reputation whenever Dota Coach receives evidence showing how the coach performed. You can view a full list of reputation events and their associated values here.
This is the overview of the reputation for a top coach.
What is current reputation? Current reputation is the metric used to rank coaches on the leaderboard and used by our matchmaker. Until now, this is the formula we used to calculate a coach’s current reputation. In change #3, a new updated formula is given.
Current Reputation = Reputation from MMR + reputation from students coached within 70 days
We aspire to be an open platform where anybody with the skill and drive to coach professionally can become a top coach. To achieve this, we employ automated, evidence-based, transparent metrics for measuring our confidence in each coach. The current reputation score aims to make it so our leaderboard shows only the best and most active coaches on the site; and that newer coaches still have a chance of making it to the top.
Change #1 - “Coaching Experience” reputation events now factors in the cost of the lessonCoaches previously received 100 reputation per hour for lessons sold. With this change, they now receive 15 reputation per dollar per hour. The old system gave an advantage to coaches with low prices and in rare cases coaches lowered their price in order to artificially inflate their reputation.
Impact: We have recalculated reputation for all lessons going back to July, 2015 to reflect the change. This went live on March 2nd. Coaches who have charged less than $6/hr might see a slight decline in their reputation score, but all other coaches should see an increase in overall reputation.
Change #2 - “Current Reputation” value cannot decay below 5,000Each day at midnight, we calculate the coaches' Current Reputation based on the last 70 days. If the coach doesn’t remain active, their current reputation will appear to “decay.” Extreme levels of decay can move top ranked coaches down to equal ranks with a new coach with zero experience (eg. Kumbomaman has 275 hours coached but less have than 2,000 reputation). Surely this is unfair and the system should have higher confidence in the abilities of these highly experienced coaches.
This change makes it so a coach’s current reputation doesn’t decay until a coach gains more than 5,000 reputation and then it can only decay to a floor of 5,000. Currently, that ranks you 17th on the leaderboard. If you don't see your name on the leaderboard, you have to coach one hour in the last 70 days to be listed (no change there).
Impact: No impact to the top 15 coaches, but a few of the old top coaches have re-appeared in the middle of the leaderboard.
Change #3 - “Current Reputation” uses clearer and more understandable decay windowThe algorithm we were using to calculate current reputation is a bit mysterious and chaotic. It has resulted in radical swings of reputation with no transparency into the cause of the swings. The resulting confusion undermines the system.
We are moving to a simpler algorithm which is more stable and more transparent.
Current Reputation = Reputation up to 5,000 + additional reputation earned within 70 days
Example: This coach has a life-time reputation of about 50,000. The first 5,000 reputation is not subject to decay. The reputation after 5,000 has decayed and is not counted in current reputation. The reputation delta shaded yellow represents the coach's "additional reputation earned within 70 days." This coach has a current reputation of about 22,000.
Impact: This has caused a reshuffling in the top coaches list. Top coaches may see a reduction in their overall reputation.
Join the DiscussionThe factors which make a player into a good coach are varied and complex. Expressing this multi-dimensional concept into a number is hard. Just as Valve’s MMR system allows them to build technologies like a matchmaker and a leaderboard for Dota, so does the reputation system enable our matchmaker and leaderboard for DotaCoach.org.
Just like players feel that MMR often underrepresents their true skill, our reputation system can leave coaches feeling under-represented. Send us any thoughts or suggestions you have on how this can be improved @DotaCoachDotOrg or at firstname.lastname@example.org.
Future ChangesA good measure of a coach’s quality is how impactful he is on his student’s MMR. We will soon start tracking the MMR for each of a coach’s students to attempt to capture evidence of the coach’s impact.
Current reputation isn’t the only interesting attribute about coaches. We will be adding a new page to capture the all-time great coaches.