Some results of recent surveys on the ProZ.com term help system 4

For those not familiar with it, the ProZ.com KudoZ term help system is a network through which translators can receive and provide term help mutually. You can see more about terminology resources here: http://www.proz.com/about/overview/terminology/

A couple of surveys on the term help system were run recently; a general survey on KudoZ, and a survey on the quality assessment of the system by participants. Here is a summary of results:

There were over 500 respondents to the general survey.

Over 50% of respondents stated that answering questions was their most frequent term help system activity. Term searches came in second, and asking questions was a distant third, at 8%.

Nearly 50% rated their experience with KudoZ as positive, only 10% rating it as either “somewhat negative” or negative.

Common aspects that respondents found negative about their experience were poor answers, and conflicts among answerers and peer commenters (this was also reflected in the responses to the survey on the assessment of KudoZ quality).

On the subject of the provision of context when asking questions:

  • 64% found the issue of context “very relevant” to the overall quality of the KudoZ system.
  • Nearly 80% agreed that term context is necessary to provide good answers to a term help question, 20% saying that it depended on the term/question.

Of the proposed solutions for ensuring that more or sufficient context is provided in term questions by askers,

  • “Improving the ‘explanation/context’ field to incite more and better context when a question is submitted” ranked first, at just under 35%
  • “Educating/orienting less experienced term help askers on the relation between good context and good answers” ranked second, at 26%
  • “Providing better ways for answerers to request more context when they find it insufficient” came in third, at 25%
  • “A voting system whereby answerers may hide incoming questions from public view until ‘enough context’ has been entered by the asker” ranked fourth, at 13%

Only 5% stated that they had not been aware that the principal purpose of the KudoZ term help system is to provide term help to askers, and that the resulting glossary entries are an additional benefit of the system.

A survey on the quality assessment of the term help system was also run simultaneously with the general survey. The same survey had been run in 2007, and, interestingly, the results of the two surveys do not vary in any significant way. It should be noted that respondents were asked to apply their own definitions of “quality” when responding to the surveys:

  • Overall participation: About 50% rated the overall “quality” of participation in the network as high; 40% rated it as average.
  • Questions: 32% rated the overall “quality” of KudoZ questions as high, with 52% rating it as average.
  • Answers: 44% rated the “quality” of answers as high, 45% rated it as average.
  • Peer comments: 37% rated peer comment “quality” high, 47% rated it average.
  • Glossary entries: 34% said the “quality” was high, 46% rated it average.

When asked to indicate the degree to which they agreed with the statement “Most KudoZ questions are legitimate questions, worthy of the time and effort needed to reply”, 76% agreed with the statement, 17% disagreed somewhat, and 5% disagreed strongly. These percentages remained the same for the statement “There are more high quality questions and answers in KudoZ than there are low quality questions and answers.”

Respondents were asked to select one “wish” with regards to KudoZ quality:

  • “More high-quality answers” ranked first.
  • “A better environment/tone among KudoZ users” and “Higher quality in the glossary/archive” tied for second place.
  • “Fewer low-quality questions” came in third.

Overall, some excellent feedback and comments were also provided in both surveys. Thanks to everyone who participated in the surveys, and to all the translators using the term help system and providing help to colleagues.

Jared

4 comments

  1. Hello mulinder,

    First, thanks for commenting, and for being the first to comment on Translator T.O.

    Most of the findings are not surprising, I would say, in that site staff were aware of them, and some work has already been done (in the area of inciting better context for questions, for example, recent changes have been made to the forum used to ask questions, to help encourage more and better context).

    Aspects of quality will always be a concern (maybe so will the definition of quality), but within a collaborative platform like KudoZ that is also a collaborative effort.

    Jared

    Like

  2. I think there will always be conflicting suggestions from different answerers on this tool – a good translator will nearly always be able to select the most appropriate answer for their context – sometimes I submit a question, already having an idea of the answer, but the suggestions just give a “shove in the right direction” or another angle, particularly when the answerer really is a specialist in the relevant field.
    I think the Kudoz system is an excellent, user friendly tool already, and apart from educating askers regarding sufficient information and context, I don’t really see that it needs too much tweaking.

    Like

  3. Thanks Mark for the feedback, and for commenting on how you use the system! The use of questions to help confirm what the translator already knows, or to bounce possible terms off of colleagues, is a common practice, and a good one to keep in mind when thinking about any possible changes to the term help system.

    Like

Leave a reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s