1 line
21 KiB
JSON
1 line
21 KiB
JSON
{"short_id":"tdfoqh","short_id_url":"https://lobste.rs/s/tdfoqh","created_at":"2022-07-22T12:06:59.000-05:00","title":"Ranking comments by sum of replies' scores","url":"","score":18,"flags":0,"comment_count":10,"description":"\u003cp\u003eThere are often cases where comments with a poor score (say, 1 point), invite high-quality replies that receive many upvotes.\u003c/p\u003e\n\u003cp\u003eThese threads are not shown at the top of the story’s comments page because their “root comment” has a low score.\u003c/p\u003e\n\u003cp\u003eAn example, as of 2022-07-22 17:00 UTC, is \u003ca href=\"https://lobste.rs/s/ekvqcf/random_wallpaper_with_just_bash_systemd#c_vop9bt\" rel=\"ugc\"\u003ehttps://lobste.rs/s/ekvqcf/random_wallpaper_with_just_bash_systemd#c_vop9bt\u003c/a\u003e:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ethe root comment has 1 point,\u003c/li\u003e\n\u003cli\u003ethe first reply has 12 points,\u003c/li\u003e\n\u003cli\u003eand, in total, 18 points have been accumulated by all the replies with score \u0026gt; 1 point.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBecause the root comment has only 1 point, it remains at the bottom of the page, bringing down with itself also the good comment with 12 points and all the other upvoted comments.\u003c/p\u003e\n\u003cp\u003eCould an alternative ranking system be put in place, where the threads and subthreads are not sorted by the score of their root comment, but by the \u003cem\u003esum of all the scores of their children\u003c/em\u003e?\u003c/p\u003e\n\u003cp\u003e(Maybe taking into account only children with \u003ccode\u003e\u0026gt;1\u003c/code\u003e point, to avoid giving importance to superficial back-and-forth discussions.)\u003c/p\u003e\n","description_plain":"There are often cases where comments with a poor score (say, 1 point), invite high-quality replies that receive many upvotes.\r\n\r\nThese threads are not shown at the top of the story's comments page because their \"root comment\" has a low score.\r\n\r\nAn example, as of 2022-07-22 17:00 UTC, is \u003chttps://lobste.rs/s/ekvqcf/random_wallpaper_with_just_bash_systemd#c_vop9bt\u003e:\r\n* the root comment has 1 point,\r\n* the first reply has 12 points,\r\n* and, in total, 18 points have been accumulated by all the replies with score \u003e 1 point.\r\n\r\nBecause the root comment has only 1 point, it remains at the bottom of the page, bringing down with itself also the good comment with 12 points and all the other upvoted comments.\r\n\r\nCould an alternative ranking system be put in place, where the threads and subthreads are not sorted by the score of their root comment, but by the _sum of all the scores of their children_?\r\n\r\n(Maybe taking into account only children with `\u003e1` point, to avoid giving importance to superficial back-and-forth discussions.)","comments_url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores","submitter_user":"gioele","user_is_author":true,"tags":["meta"],"comments":[{"short_id":"ncdsfc","short_id_url":"https://lobste.rs/c/ncdsfc","created_at":"2022-07-22T14:04:36.000-05:00","updated_at":"2022-07-22T14:04:36.000-05:00","is_deleted":false,"is_moderated":false,"score":17,"flags":0,"parent_comment":null,"comment":"\u003cblockquote\u003e\n\u003cp\u003eto avoid giving importance to superficial back-and-forth discussions\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eMaybe take the max, instead of the sum? That way it’s still comparing the score of individual comments. A highly-upvoted child would rank higher than a less-upvoted toplevel comment, but long threads wouldn’t get any bonus.\u003c/p\u003e\n","comment_plain":"\u003e to avoid giving importance to superficial back-and-forth discussions\r\n\r\nMaybe take the max, instead of the sum? That way it's still comparing the score of individual comments. A highly-upvoted child would rank higher than a less-upvoted toplevel comment, but long threads wouldn't get any bonus.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_ncdsfc","depth":0,"commenting_user":"dpercy"},{"short_id":"m3wyu5","short_id_url":"https://lobste.rs/c/m3wyu5","created_at":"2022-07-23T01:04:43.000-05:00","updated_at":"2022-07-23T01:04:43.000-05:00","is_deleted":false,"is_moderated":false,"score":8,"flags":0,"parent_comment":null,"comment":"\u003cp\u003eFor better or worse, the deeper you go into threads the less likely it is that the discussion is still about the original submission. Your proposal, or the max variant presented elsewhere, I fear would unduly reward flamebait and off-topic discussion.\u003c/p\u003e\n","comment_plain":"For better or worse, the deeper you go into threads the less likely it is that the discussion is still about the original submission. Your proposal, or the max variant presented elsewhere, I fear would unduly reward flamebait and off-topic discussion.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_m3wyu5","depth":0,"commenting_user":"friendlysock"},{"short_id":"pcvbcd","short_id_url":"https://lobste.rs/c/pcvbcd","created_at":"2022-07-23T09:52:52.000-05:00","updated_at":"2022-07-23T10:09:15.000-05:00","is_deleted":false,"is_moderated":false,"score":2,"flags":0,"parent_comment":"m3wyu5","comment":"\u003cblockquote\u003e\n\u003cp\u003eYour proposal, or the max variant presented elsewhere, I fear would unduly reward flamebait and off-topic discussion.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eSuch off-topic discussions that stray away from the main topic do exist, are a nuisance, and, surely, they should not be rewarded with more attention. However, such sub-threads exists as a long string of short comments with no or very few upvotes. (I wonder if the data confirms this or it is just my impression.)\u003c/p\u003e\n\u003cp\u003eFor this reason I believe that a) not counting in comments with score==1 or b) \u003ca href=\"https://lobste.rs/u/dpercy\" rel=\"ugc\"\u003e@dpercy\u003c/a\u003e’s max variant would avoid rewarding such discussions.\u003c/p\u003e\n","comment_plain":"\u003e Your proposal, or the max variant presented elsewhere, I fear would unduly reward flamebait and off-topic discussion.\r\n\r\nSuch off-topic discussions that stray away from the main topic do exist, are a nuisance, and, surely, they should not be rewarded with more attention. However, such sub-threads exists as a long string of short comments with no or very few upvotes. (I wonder if the data confirms this or it is just my impression.)\r\n\r\nFor this reason I believe that a) not counting in comments with score==1 or b) @dpercy's max variant would avoid rewarding such discussions.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_pcvbcd","depth":1,"commenting_user":"gioele"},{"short_id":"9vyyu5","short_id_url":"https://lobste.rs/c/9vyyu5","created_at":"2022-07-22T13:39:20.000-05:00","updated_at":"2022-07-22T13:39:20.000-05:00","is_deleted":false,"is_moderated":false,"score":5,"flags":0,"parent_comment":null,"comment":"\u003cp\u003eJust like to point out that the assumption here is that upvotes is a metric of quality. On lobste.rs as in any other social context, upvotes/support is a measure of popularity.\u003c/p\u003e\n\u003cp\u003eThere are many instances where the two are correlated, and some important instances where they are not.\u003c/p\u003e\n","comment_plain":"Just like to point out that the assumption here is that upvotes is a metric of quality. On lobste.rs as in any other social context, upvotes/support is a measure of popularity.\r\n\r\nThere are many instances where the two are correlated, and some important instances where they are not.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_9vyyu5","depth":0,"commenting_user":"kghose"},{"short_id":"owddle","short_id_url":"https://lobste.rs/c/owddle","created_at":"2022-07-22T13:57:19.000-05:00","updated_at":"2022-07-22T13:57:19.000-05:00","is_deleted":false,"is_moderated":false,"score":7,"flags":0,"parent_comment":"9vyyu5","comment":"\u003cblockquote\u003e\n\u003cp\u003eJust like to point out that the assumption here is that upvotes is a metric of quality. On lobste.rs as in any other social context, upvotes/support is a measure of popularity.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI fully agree: upvotes are a metric of popularity and not of quality. But this proposal does not assume that.\u003c/p\u003e\n\u003cp\u003eRegardless of what upvotes represent, currently threads are sorted in descending order of (upvotes, time). All that this proposal suggests, is to change that to (sum(upvotes of root-and-children), time).\u003c/p\u003e\n\u003cp\u003eThe only assumption here is: if ranking a thread by upvotes of its root comment is considered good (and at the moment it is), then ranking by the sum of the upvotes of the whole thread is better.\u003c/p\u003e\n","comment_plain":"\u003e Just like to point out that the assumption here is that upvotes is a metric of quality. On lobste.rs as in any other social context, upvotes/support is a measure of popularity.\r\n\r\nI fully agree: upvotes are a metric of popularity and not of quality. But this proposal does not assume that.\r\n\r\nRegardless of what upvotes represent, currently threads are sorted in descending order of (upvotes, time). All that this proposal suggests, is to change that to (sum(upvotes of root-and-children), time).\r\n\r\nThe only assumption here is: if ranking a thread by upvotes of its root comment is considered good (and at the moment it is), then ranking by the sum of the upvotes of the whole thread is better.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_owddle","depth":1,"commenting_user":"gioele"},{"short_id":"lqqn3a","short_id_url":"https://lobste.rs/c/lqqn3a","created_at":"2022-07-22T14:28:14.000-05:00","updated_at":"2022-07-22T14:28:27.000-05:00","is_deleted":false,"is_moderated":false,"score":2,"flags":0,"parent_comment":"owddle","comment":"\u003cp\u003eTangential to your effort here, on a personal note, if I find a discussion interesting I read all the posts regardless of ranking. On the discussions I find worthwhile I find that post quality is unrelated to upvotes.\u003c/p\u003e\n\u003cp\u003eOn some discussions I find the top voted comment has devolved into a long tail of niche discussion, often acrimonious, which is not as useful as later posts.\u003c/p\u003e\n\u003cp\u003eWorst to pick out of the noise are replies to replies which are great but buried in not so great comments.\u003c/p\u003e\n","comment_plain":"Tangential to your effort here, on a personal note, if I find a discussion interesting I read all the posts regardless of ranking. On the discussions I find worthwhile I find that post quality is unrelated to upvotes.\r\n\r\nOn some discussions I find the top voted comment has devolved into a long tail of niche discussion, often acrimonious, which is not as useful as later posts.\r\n\r\nWorst to pick out of the noise are replies to replies which are great but buried in not so great comments.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_lqqn3a","depth":2,"commenting_user":"kghose"},{"short_id":"7b0jgw","short_id_url":"https://lobste.rs/c/7b0jgw","created_at":"2022-07-23T11:25:10.000-05:00","updated_at":"2022-07-23T11:26:03.000-05:00","is_deleted":false,"is_moderated":false,"score":4,"flags":0,"parent_comment":null,"comment":"\u003cp\u003eRanking by max or sum upvotes optimizes for photogenic smackdowns of bad opinions.\u003c/p\u003e\n\u003cp\u003eMaybe disaggregate “agreement upvotes” and “contribution upvotes”?\u003c/p\u003e\n","comment_plain":"Ranking by max or sum upvotes optimizes for photogenic smackdowns of bad opinions.\r\n\r\nMaybe disaggregate \"agreement upvotes\" and \"contribution upvotes\"?","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_7b0jgw","depth":0,"commenting_user":"FeepingCreature"},{"short_id":"ei3nck","short_id_url":"https://lobste.rs/c/ei3nck","created_at":"2022-07-23T10:43:21.000-05:00","updated_at":"2022-07-23T10:48:49.000-05:00","is_deleted":false,"is_moderated":false,"score":2,"flags":0,"parent_comment":null,"comment":"\u003cp\u003eI feel kind of honored to be taken as an example, so please let me share my personal thoughts (TL;DR below):\u003c/p\u003e\n\u003cp\u003eIt is impossible to find \u003cem\u003ethe\u003c/em\u003e definite ranking system, but let me elaborate by using 3 extremes to see where the current and proposed systems fail:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cem\u003ehighly controversial root comment\u003c/em\u003e with score 0 (100 upvotes, 100 downvotes), logically controversial responses with median 0,\u003c/li\u003e\n\u003cli\u003e\n\u003cem\u003eroot comment preaching to the choir\u003c/em\u003e with logically high score and usually high-score responses,\u003c/li\u003e\n\u003cli\u003e\n\u003cem\u003elow-score root comment with high-score responses\u003c/em\u003e (like the presented case)\u003c/li\u003e\n\u003cli\u003e\n\u003cem\u003eOC high-value root comment\u003c/em\u003e with high score and high-value responses\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eIt all depends on personal preference though, but I personally would like more controversial topics to be on top and less controversial ones to be secondary. The reason for that is that you otherwise might end up with echo-chambers, given users are encouraged to preach to the choir for a high karma score. As a tangent, I would prefer two karmas for each user: A “paragon-karma” and “renegade-karma” whose sum would be an “influence” score (in the end up- or downvotes both reflect a certain influence) and whose difference would be a “non-controversiality-score” (a low difference reflects controversiality, a high score the opposite). All in all, maybe we should move away complete from the miriad of downvote-options on lobsters and simply have an “agree” and “disagree”. If something is spam, you can report it, if something is incorrect, you can write a response rectifying it (which would then be subject to a vote of agreement and disagreement, depending on how well you state your case).\u003c/p\u003e\n\u003cp\u003eThe proposal to add replies into the weight would not change the ranking of controversial topics (1), but even moreso weigh comments preaching to the choir (2) because everyone replying would aim to also get some karma. It would solve the presented case (3) though. Case (4) would also be favoured.\u003c/p\u003e\n\u003cp\u003eThe proposal by \u003ca href=\"https://lobste.rs/u/dpercy\" rel=\"ugc\"\u003e@dpercy\u003c/a\u003e to the take the max would not change (1)’s ranking, probably not affect (2) but help (3) as well. (4) is also positively affected.\u003c/p\u003e\n\u003cp\u003eTL;DR, here’s my proposal so more highly-controversial topics (those that are interesting) are more favoured: Consider root comment and replies as equals and only take \u003cem\u003einfluence\u003c/em\u003e (sum of up- and downvotes) as a score.\u003c/p\u003e\n\u003cp\u003eThis benefits (1), which in the current form end up at the bottom, which makes zero sense. It also benefits (2) and (4), which is a forced compromise, given the score does not really show how “original” a comment is (this is why Reddit probably introduced awards to allow users to weigh very good posts). By taking all replies into account, it also, of course, benefits (3), which makes sense to push up.\u003c/p\u003e\n\u003cp\u003eReplies with no votes increase the influence of the thread by 1, indeed, but isn’t the purpose of the score to show what the hivemind thinks? By scoring them as zero (which was proposed here), you effectively value an elaborated response lower than a simple upvote of the original comment. Instead, why not simply “fold” long subthreads so they don’t take up too much space?\u003c/p\u003e\n\u003cp\u003eAs another tangent: By weighing root and children, this might encourage people to respond to “deep” threads instead of writing a new post.\u003c/p\u003e\n\u003cp\u003eLet’s see what \u003ca href=\"https://lobste.rs/u/pushcx\" rel=\"ugc\"\u003e@pushcx\u003c/a\u003e decides in the end. :)\u003c/p\u003e\n","comment_plain":"I feel kind of honored to be taken as an example, so please let me share my personal thoughts (TL;DR below):\r\n\r\nIt is impossible to find _the_ definite ranking system, but let me elaborate by using 3 extremes to see where the current and proposed systems fail:\r\n\r\n1. *highly controversial root comment* with score 0 (100 upvotes, 100 downvotes), logically controversial responses with median 0,\r\n2. *root comment preaching to the choir* with logically high score and usually high-score responses,\r\n3. *low-score root comment with high-score responses* (like the presented case)\r\n4. *OC high-value root comment* with high score and high-value responses\r\n\r\nIt all depends on personal preference though, but I personally would like more controversial topics to be on top and less controversial ones to be secondary. The reason for that is that you otherwise might end up with echo-chambers, given users are encouraged to preach to the choir for a high karma score. As a tangent, I would prefer two karmas for each user: A \"paragon-karma\" and \"renegade-karma\" whose sum would be an \"influence\" score (in the end up- or downvotes both reflect a certain influence) and whose difference would be a \"non-controversiality-score\" (a low difference reflects controversiality, a high score the opposite). All in all, maybe we should move away complete from the miriad of downvote-options on lobsters and simply have an \"agree\" and \"disagree\". If something is spam, you can report it, if something is incorrect, you can write a response rectifying it (which would then be subject to a vote of agreement and disagreement, depending on how well you state your case).\r\n\r\nThe proposal to add replies into the weight would not change the ranking of controversial topics (1), but even moreso weigh comments preaching to the choir (2) because everyone replying would aim to also get some karma. It would solve the presented case (3) though. Case (4) would also be favoured.\r\n\r\nThe proposal by @dpercy to the take the max would not change (1)'s ranking, probably not affect (2) but help (3) as well. (4) is also positively affected.\r\n\r\nTL;DR, here's my proposal so more highly-controversial topics (those that are interesting) are more favoured: Consider root comment and replies as equals and only take *influence* (sum of up- and downvotes) as a score.\r\n\r\nThis benefits (1), which in the current form end up at the bottom, which makes zero sense. It also benefits (2) and (4), which is a forced compromise, given the score does not really show how \"original\" a comment is (this is why Reddit probably introduced awards to allow users to weigh very good posts). By taking all replies into account, it also, of course, benefits (3), which makes sense to push up.\r\n\r\nReplies with no votes increase the influence of the thread by 1, indeed, but isn't the purpose of the score to show what the hivemind thinks? By scoring them as zero (which was proposed here), you effectively value an elaborated response lower than a simple upvote of the original comment. Instead, why not simply \"fold\" long subthreads so they don't take up too much space?\r\n\r\nAs another tangent: By weighing root and children, this might encourage people to respond to \"deep\" threads instead of writing a new post.\r\n\r\nLet's see what @pushcx decides in the end. :)","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_ei3nck","depth":0,"commenting_user":"FRIGN"},{"short_id":"4l8mzk","short_id_url":"https://lobste.rs/c/4l8mzk","created_at":"2022-07-22T12:30:20.000-05:00","updated_at":"2022-07-22T12:30:20.000-05:00","is_deleted":false,"is_moderated":false,"score":2,"flags":0,"parent_comment":null,"comment":"\u003cp\u003eI really like this idea but have no idea what its implementation would represent in terms of additional load to the server.\u003c/p\u003e\n","comment_plain":"I really like this idea but have no idea what its implementation would represent in terms of additional load to the server.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_4l8mzk","depth":0,"commenting_user":"feoh"},{"short_id":"qy0l9k","short_id_url":"https://lobste.rs/c/qy0l9k","created_at":"2022-07-22T16:12:09.000-05:00","updated_at":"2022-07-22T16:12:09.000-05:00","is_deleted":false,"is_moderated":false,"score":5,"flags":0,"parent_comment":"4l8mzk","comment":"\u003cp\u003eI’ve moved most of the \u003ca href=\"https://github.com/lobsters/lobsters/blob/cf52ab9f3f0b3ac805effc2e718f15922cb67332/app/models/comment.rb#L340\" rel=\"ugc\"\u003evote\u003c/a\u003e into the db, but \u003ca href=\"https://github.com/lobsters/lobsters/blob/cf52ab9f3f0b3ac805effc2e718f15922cb67332/app/models/comment.rb#L219\" rel=\"ugc\"\u003enot all\u003c/a\u003e. If \u003ccode\u003ecalculated_confidence\u003c/code\u003e finished its move into the db almost any algorithm would be faster than the current implementation.\u003c/p\u003e\n","comment_plain":"I've moved most of the [vote](https://github.com/lobsters/lobsters/blob/cf52ab9f3f0b3ac805effc2e718f15922cb67332/app/models/comment.rb#L340) into the db, but [not all](https://github.com/lobsters/lobsters/blob/cf52ab9f3f0b3ac805effc2e718f15922cb67332/app/models/comment.rb#L219). If `calculated_confidence` finished its move into the db almost any algorithm would be faster than the current implementation.","url":"https://lobste.rs/s/tdfoqh/ranking_comments_by_sum_replies_scores#c_qy0l9k","depth":1,"commenting_user":"pushcx"}]} |