Back in January I started looking into some strange voting patterns affecting several users who noticed their comments were routinely downvoted within a minute of posting. Some of these users had already reported the issue to reddit admins to no avail, so I wrote a little script to continuously refresh the latest comments and measure how long it takes for each comment's vote score to change from the default '1 point'. Some users reported being affected when posting in /r/btc, so I included that sub as well. I finally started logging on January 30th. With the recent downvote attack against /r/Bitcoin, I figure now is as good a time as any to share this information.MethodStream reddit comments and record how long it takes for the vote score to change.If the vote score changes within three minutes, record whether it was an upvote or downvote.If the vote score changes within roughly one minute, consider it potentially anomalous.Tally data to isolate which accounts are most frequently affected by anomalous changes to vote score.ResultsWhat I found was rather alarming. It didn't take long to see that virtually all the comments by several dozen regular contributors appeared to be getting downvoted to '0 points' within about about a minute, regardless of what they said or how old the thread was. And since I wasn't only measuring downvotes, I also found that a number of accounts had their comments change to '2 points' within the same time frame.You can view the results in this Google Spreadsheet. Please note that one sheet contains the data, while the other 3 sheets contain charts of the data. At least one chart didn't import from Excel correctly.Since January 30th, /r/Bitcoin has received over 10,000 'instant' votes:For 12,451 comments, the vote scores were changed within 180 seconds10,309 comments had their vote scores changed within 60-80 seconds2,137 of those 10,309 comment vote scores were changed to "2 points"8,123 of those 10,309 comment vote scores were changed to "0 points"It's important to note that this activity is observable at all hours of day and without any noticable interruption, except when affected users are not commenting. This even occurs when commenting in very old threads with simple test comments.ChartsChart 1: FrequencyThis histogram shows the number of comments where a vote score change was detected (y-axis) within n seconds of the comment being made (x-axis). The anomaly is the massive spike in vote score changes under ~80 seconds. As the anomaly dissipates, vote score changes appear to be much more organic. Regretfully I didn't save any data logged from comparison subreddits, but they just look like this graph minus the huge bubble.Chart 2: Targeted UsersHere's a histogram based on frequency of specific users affected. Blue bars indicate the number of comments a user made whose vote scores changed to "0 points" within 80 seconds, whereas Orange bars indicate the number of comments a user made whose vote scores changed to "2 points" within 80 seconds. Bars which are more evenly split between blue and orange can be ignored as inconclusive. Longer bars of unform color are more indicative of something weird.Chart 3: ActivityThis shows the number of comments affected within a given hour per day over the course of logging. It shows that this activity has gone on around the clock as long as people are online and commenting.User targetingThe most alarming thing about this data to me is that specific users are being targeted, apparently based solely on their political views. I have not monitored how this might effect comment sorting, but it's certainly plausible that a comment with '2 points' will have an advantage over a comment with '0 points', potentially distorting reader perception.I want to stress that a user having their comments instantly changed to '2 points' is not conclusive evidence of any wrongdoing on the part of that user. It's admittedly strange, but could be explained by an obsessive fan upvoting all their comments as soon as they post something, or perhaps some unknown reddit mechanism.False positivesFalse positives can occur during fast-paced threads where readers are frequently refreshing for threads for the latest comments and replies. It's not uncommon to open a thread and see a comment posted within the last few minutes, then cast a vote. However, given the amount of data accrued and patterns observed, it's seems pretty clear that false positives don't weigh heavily on the results.Vote fuzzingVote fuzzing is one of reddit's anti-vote cheating mechanisms which causes vote scores to fluctuate randomly within a narrow range in an attempt to obscure the actual vote score. This can be observed by refreshing a comment with around 5 votes or more, and watching the score randomly change plus or minus a few points.However, to the best of my knowledge, comments with a default vote score of '1 point' do not get fuzzed until after it receives a few votes. Sometimes you might see vote fuzzing on controversial comments, as indicated by the little red dagger† (if enabled in prefs). You can verify that default vote scores aren't fuzzed by commenting in your own private sub (or a very quiet old thread in the boonies somewhere) and see that the vote score does not change when you refresh.I have no reason to believe that vote fuzzing applies to the data I've collected because I'm only logging the first change to the vote score. That said, it does not rule out the possibility these anomalies could be explained by some proprietary anti-vote cheating measure which reddit does not wish to disclose.Admin responseReddit admins are generally pretty responsive when it comes to isolated cases, but this issue took a few weeks to address, presumeably due to the bulk of users affected and investigation required. They have confirmed that they've dealt with multiple accounts targeting these users with downvotes, but have also caution against drawing firm conclusions from this method due to various anti-vote cheating measures in use. Reddit admins have neither confirmed nor denied whether automated voting is taking place. It appears to still be happening, but the frequency has abated somewhat.Other subredditsI looked at a few other subreddits of comparible size and found that votes occuring within 1 minute are rare by comparison. In fact, I extended the scope from 3 minutes to 15 minutes, and still did not find any anomalous voting patterns. Fast votes do happen, but I have yet to find any sub where they happen as fast as on /r/Bitcoin, nor have I found a sub where it appears specific individuals are targeted. I also looked at some much larger subs whose scores are not hidden (GetMotivated+mildlyinteresting+DIY+television+food) and found that while votes do roll in a bit faster, they still do not occur within seconds of commenting, and still do not appear to target specific individuals. There's room for more research in that area.Edit: I've asked the mod team if they'd object to disabling the temporary hiding of vote scores for a few days in case anyone wants to run the script for themselves. No objections, so comment vote scores are now visible for the time being. The script requires Python 2.7 and PRAW. Provide your own login credentials.Edit 2: We've seen a couple attempts to claim responsibility. This is the most compelling so far. Here's the data he posted. A very quick glance reveals that it's very similar to mine, but I need to look into it. Most compelling is that his earliest logs were before I started recording. I'm now even more convinced by the multiple bot theory than before. Everyone doing this should knock it off because you're only hurting your cause. via /r/Bitcoin http://bit.ly/1U5IXdS
No comments :
Post a Comment