Actually I find a lot more interesting about Stack Exchange than simply a view of its successes and failures, but I'm known for making the occasional bold and controversial statement so I'd better try and stick to defending my point of view without getting too far off topic.
One of Stack Exchange's greatest strengths is in its use of the community to provide checks and balances on the system. There are moderators elected from the community to do a lot of work policing the system to keep it tidy, and to keep abusive posts to a minimum. Then there is a system of rewards where points are earned for contributions to the system itself, and as the number of points - aka reputation - a user earns grows, the more privilege the user has in the system, allowing the user to flag posts for moderator attention, vote on the quality of posts, participate in chat rooms, edit another user's post, and even vote to have a post closed or removed. Nobody gets paid to do all of this work, and for the most part, the democratic nature of the system works extremely well to deliver a high quality of content to its users. In this, the format of the site works extremely well, and the site has been a resounding success. I'll often search for the answer to a problem on StackOverFlow or ProgrammersStackExchange and if I don't find the answer to my problem, I'll usually find a clue that will lead me to either use a better search phrase, or to solve the problem on my own.
For me, the failure comes because the community itself is defined by its people, and these people are only human after all, so will make mistakes occasionally. That in itself wouldn't be too great a problem as the community ensures that the system is self correcting to a large degree. The problem is where people become polarized on issues due to differing opinions without relying of facts, and then they use their ability to cast a negative vote as a means to attack a persons credibility instead of engaging in a reasoned discussion. In one particular case where this happened to me, I received a negative vote because one individual made an assumption about what my answer was about instead of actually reading the post and thinking about the content. After I expanded my answer to explain out a concept in detail, he decided to remove his down vote. While this might sound reasonable - and I'll admit my answer reads a little better now I've edited it - it shows how people use the power of their votes to influence the way in which people behave. I would rather receive a negative vote for being wrong than as a tool to attempt to manipulate my answers.
Positive votes are likewise abused. I've seen a couple of cases where votes were cast heavily against one answer, and just as heavily for another answer as a result of the comments being made which resulted in little more than a tit for tat argument between two individuals over a certain issue. It was as if the votes were being used as a popularity measure, and the sad thing is that quality of the answers didn't even factor into it. Meanwhile, a couple of really great answers were ignored and received no votes at all, lost in all the noise that a few selfish individuals had engaged in.
Sometimes when a lot of argumentative commentary occurs, the moderator will either delete many of the comments, or they will close the question because it is seen as being too polarizing and resulting in too many highly subjective answers and comments. I can understand this policy if the question itself is quite vague and requires an interpretation to answer, but not when two people start to battle it out over their own personal views over a good question or answer. It's as if the very act of moderation itself ends up being used as a bludgeon and the question is taken out as collateral damage, instead of removing and censuring the combatants themselves.
Whenever I cast a negative vote, it is because an answer is either factually wrong, or because it doesn't ask or answer a question. If I am uncertain about whether to case a negative vote, I might take a moment to post a comment or two and wait to see what the response is, and if the response shows a complete lack of understanding or reasoning, then I might vote a post down. I only vote an answer up if it is a good answer to the question, and not because I necessarily agree with the answer. My reasoning is that the voting system isn't merely about selfishly improving your own reputation, but is supposed to ensure that the quality of the questions and answers remains high to ensure the reliability of the information available in the site. Yet not everybody sees it that way.
So where did the creators of StackExchange go wrong? As a concept itself, the site is brilliant. It has a lot of really great features that make it easier to participate in not only the Q/A, but also in the running of the site. As the following graphic shows, it represents a clever blending of concepts to create something familiar, yet new (yes, that little asterisk in the middle).
Partly, I think it has to do with perceptions. As new people arrive on the site they see questions and answers and they treat the site as if it is a forum, and as if it is simply a help site. So, with the tools at hand the more experienced members will use their votes as a means to influence the moderation of a new user's questions, and to teach the new people to treat the site as a knowledge repository rather than as a mere Q/A site. Another problem is that the commentary in the questions and answers isn't really a good place for a discussion, yet it is common to find everything from seemingly harmless debates to full-blown flame wars occurring in the comments. Again the creators have thought about this, and provided a chat facility to take this sort of thing offline. The moderators have also been given the ability to delete some of the extended comments when they become too abusive, or off topic.
One failing is that it's difficult to find a link between discussions about a question, and the question itself. For example, if a question gets removed, you have to know that the correct place to ask questions about questions is to locate the accompanying "meta" site. This is not made obvious in the FAQ. When you do get to the meta site, you have to post a question, it gets answered, and then you get a whole lot of the sort of debate-styled commentary that is frowned upon in the main site. You can only post a single answer, and then when you add comments, you're heavily restricted in the number of characters your comment can accept. Incidentally, this is also a limitation of the chat, yet it seems to me that if the chat were to allow slightly longer comments to be posted, and if the chat were used as a discussion page linked to each question, then the format would work even better. This is one of the features of MediaWiki for example that makes Wikipedia such a success, and yet is sorely limited. If the chat were to make the commentary more easily to follow in terms of the conversational threads, it would work even better. A linked discussion page would be much easier to find than the meta, and would encourage users to participate more actively in the running of the site in a way that the Meta is presently failing.
Another failing is that the messages the moderators leave as explanations for question closures aren't always clear. The assumption is always that if they point the original poster to the FAQ, that their reasoning will be clear. In some cases however, you are left with a simple this is not a good fit for our site style of answer and when you look to the FAQ for an explanation you often feel that you haven't really found an answer. The way that the FAQ is written and how the meta site is used has mostly been influenced by the way in which the site has been run since the beginning, and there are many things that new users need to know how to do in order to engage fully with the site. From a usability perspective, this is a problem. Great systems are inherently easy to use - and therefore naturally difficult to implement - and it should be clear at first glance that a user has certain options available to follow up when issues arise. Perhaps a link from the question direct to the moderators would help, or perhaps a kind of shortcut page that explains what to do and where to go. I found out after nearly 2 years on the site - ok, I've been heavily active for only 3 months now - that if a question is voted to be closed, that it can be voted to be opened again!! I'm given to understand that the OP can even take a part in the reopen vote. If you want to get a question reopened, you may need to try and garner a little support to get those extra votes. This means creating a meta question to try and change minds with a good discussion. Of course, you have to hope that there are lots of people reading the meta who can vote, but as I said earlier the meta isn't really designed for a good free-flowing discussion.
I mentioned earlier that people are possibly one of the biggest problems. In this case I'm thinking about the way in which people moderate for questions that they feel may be considered off-topic for the site. While I agree with most of the decisions made, I - and many others - have commented about some decisions on questions that can be considered edge-cases, questions that are of interest, but border on areas of expertise that are covered by some of the other stack exchange sites. Sometimes the questions are closed down very quickly, while many in the community would prefer the questions to remain open and answered by people with the expertise to answer from the perspective of the target audience of that site. It seems as if it comes down to questions that moderators - or people with enough reputation - don't like, don't understand, can't answer perhaps? I'm really not sure what their motivation is, but it seems that they just won't be happy unless the question is closed down. I started a chat to address this very issue, and the answers to my comments were oft times very glib, elitist, and in some respects quite selfish, from both sides of the argument. I happen to believe my point of view was well defined and asked questions that really should have been answered, yet when I found intelligent commentary to provide to the debate, it was ignored, and the chat was really only visited by a mere handful of people as far as I could tell. Yet regardless of who is more correct in the discussion, it's the combination of anger, sarcasm, and self-imagined-superiority on the part of many of the participants that makes the site at times unpleasant to use. Sadly, many of the comments by such people are accepted and left by moderators, because of the respect the moderators have for certain select members of the site. Not only should these people be held to a higher standard, but they should also be treated with the same impartiality as the rest of the community is. It's not that it isn't fair, it that it creates a dividing line where one shouldn't exist, and leaves a handful of people feeling entitled to behave badly, while the rest are left feeling as if they are outsiders, and unable to make themselves heard in what is supposed to be an open, democratic, and embracing community.
Lastly, I feel that the site is suffering from a bit of an identity crisis. Is it a help site, a Q/A site, a forum, a wiki... what is it? It has the makings to be a great knowledge-base, and indeed it can be quite useful when attempting to find answers to some of the more esoteric questions you might find yourself asking. It wants to be the sort of Q/A site where the OP gets to pick the answer they feel answers the question best, yet the way in which the moderators and others respond seems to largely rob the OP of their position in the matter. There are many people working very hard on the site to make it appear to be a knowledge base, yet the site doesn't present itself very convincingly that way. It behaves a bit like a wiki, and yet misses some of the best elements that make wiki's so organic and self managing by granting too much individual power to a handful of users while disallowing the rest to have much of a say. While I agree that it's a good thing that you have to contribute to the site to earn reputation to have such power yourself, clever people can and do manipulate the way in which the site works in order to advance themselves, while those that play entirely by the rules find the process incredibly slow going, and once they get there, they find an entrenched group of people and prejudices that are almost impossible to change.
OK, so forgive me, but it appears I am such a slave to the sensationalist media that I seem to have a need to have attempted to raise a little controversy in the title of this post... and the first paragraph... and perhaps even in the content. You don't have to agree with me. Hey, it's my blog so I'll rant as I see fit. The thing is, that I'm not really bashing the StackExchange sites, because with all of the faults that I do see, the Stack Exchange network is still one of the best tools available on the interwebz. Sure, you've got Wikipedia, and the rest of the web to trawl through, but with StackOverflow and ProgrammersSE in particular - and many others - you get to pick the brains of a very large group of very smart people who have between them all a vast pool of information just waiting to be harnessed and tapped into. In this sense, the creators got it very right, and it's probably for this reason above all others that I find myself returning daily to read questions, post answers, and spend a little time flagging useless posts, and throwing my votes around trying to do my own little part to contribute for the good of the site. In spite of the flaws that I see, there the site does have huge potential, and is being improved all of the time. With luck I'll convince enough people to encourage a little change for the betterment of the site, while at the same time learning more and more about not only how to do lots of really great stuff when developing software, but also about the motivations behind the things we do when creating software, and when interacting with our peers on many different levels.
No comments:
Post a Comment