Fabrice Florin responds, my suggestions follow
Interesting project! thing I don’t like about it: the rating process is too much of a “black box” when it need not be so: why not have a “talk page” ala Wikipedia, for instance? (or the comments function for Digg) and let people talk it out… ask each other questions, point out flaws that others might have missed… and make all this viewable not only by those who want to rate (which would very likely result in more accurate ratings) but also by those who just want to read the stories.
Of course, an overall rating number is very useful if you want be able to quickly pick a couple of stories worth reading — it’s just that in plenty of cases the rating number may be meaningless without an insight into what were the raters thinking…
Also, some sort of guidance as to what would constitute 1, 2, 3, 4 and 5 in the rating categories given would probably help a lot. Maybe they could gives examples (5 stories in each rating category that, according to whoever runs the site, would rank 1, 2, 3, 4 or 5 — it would probably help to ask users for input with this). I think some sort of a common ground as far as rating steps (1, 2, 3, 4, 5) needs to be established or most people are just going to rate “safe” and pretty much all stories would end-up being rated “average,” which would be useless…
Delia
Hi Fabrice! You are welcome! why moderate the discussion? (especially if you don’t have the funds for it) — just make the function available AND make it clear it’s there so people can *discuss* among each other… not just post comments (that’s good too but not as useful as having real discussions).
The tips on rating are good (good categories!)– I just think they are a bit too general (the 1, 2, 3, 4, 5 levels for each category are just left undefined) to be very useful to the rater or to those just reading the stories and trying to make sense of the rating.
re: “know to hit 4 when they think it’s good, or 2 when it’s bad”
well… you want them to do more than just give you their gut reaction… (I *think*… — I mean, *I* would… I think it would be much more useful…)
Oh… and, I think it would be good to also have a category by category breakdown of the general score available (maybe a “breakdown” link next to the general score?). Some people may not want to read a story that rates poor at the factual support level but somehow manages to do OK overall…
Further along this idea… I’d consider letting the reader pick the criteria: maybe have a search box that lets the user get a list of stories according to their desired criteria (e.g. no less than 4 on factual support, don’t care about originality etc.)
re: hitting 5 when they really like a story
I’d get rid of that question altogether (re: “how good is this story?”) or at a minimum I’d have it at the end — not at the beginning — (if you *must*…). I think letting the aggregate score on the individual categories give the overall “recommendation” would really help the rater with not allowing things like personal likes and dislikes get in the way (you want them to be as objective as possible… I *think*…). In fact, a message asking them to “please don’t let your personal like or dislike of the story affect you rating (if you don’t think you can do this, please skip this story)” would probably help.
re: “In time me may calibrate everyone’s ratings to adjust for their personal rating style”
I think that would be much more difficult than establishing a common understanding for the rating steps in each category (1, 2, 3, 4, 5), but, of course, I could be wrong…
Delia
P.S. I’ll try to keep an eye on your project but I’m trying to not get sucked-in any particular project. D.
No comments:
Post a Comment