Showing posts with label requesters. Show all posts
Showing posts with label requesters. Show all posts

Sunday, December 9, 2012

MTurk qualification tests



What are Amazon MTurk qualification tests?

One thing that pretty much every turker comes across sooner or later are the MTurk qualification tests.  These are set by the requester and are generally a way for them to restrict the turkers who do their HITs, essentially by trying to weed out workers who lack certain qualities or skills.  The qualification tests are in addition to the other restrictions that the requesters can impose, such as minimum HIT acceptance rates and nationality.

What do the tests involve?

There is no set answer to this question as there are numerous types of qualification test.  It all depends on which skills or attributes that the requester is looking for.  The test will in most cases reflect this in the form it takes.  For example, if the requester wants to assess your English language skills, you might be given a series of sentences and asked to judge which ones are grammatically correct.

Some tests are as simple as just pressing a button, or answering a yes or no question.  Other qualification tests can involve elaborate tasks which are difficult and time-consuming.  Of course, if you spend a long time taking a difficult and time-consuming test and ultimately you are told you have failed, it can be very frustrating.

Should I take an Amazon MTurk qualification test?

Generally speaking, I would say yes in the majority of cases.  As I mentioned earlier, many of the tests are very easy, and even if they aren’t, you haven’t got too much to lose from trying, apart from your time. 

Don’t take a test if you clearly don’t fit the criteria, however.  If the HIT involves translating text into German and you don’t speak German, then common sense dictates that the test will be checking your German language skills and it isn’t worth taking.

Some tests will also fail you for things that you can’t control, such as where you live.

The advantage of taking tests is that if you succeed, they can sometimes open the door to you doing some of the (relatively) better paid HITs.

Monday, December 5, 2011

MTurk online surveys: 3 peeves of mine

I’ve been working on Amazon MTurk a lot recently in the run up to Christmas - it’s a good way of saving for gifts, especially when I know that what I plan to buy is sold on Amazon.  But some of the problems with the Mechanical Turk online surveys have been annoying me. 

Don’t get me wrong, I enjoy doing surveys.  They seem like less hassle than some of the other tasks sometimes, the pay varies and is maybe going down gradually over time, but often still isn’t bad in MTurk terms.  The requesters are also generally pretty reliable, especially if the MTurk requester is a university academic or student, so I worry less about not getting paid, or there being some sort of scam going on.

But recently when I search for surveys, there have been more and more scams that are calling themselves surveys appearing (you know the sort, they always offer you an "easy money for 5 minutes work" and say they're going to pay you $20!).  I mean signing up for a credit report can in no way be considered a survey!  Neither can “test my website by entering all your personal details and pressing submit”!  I guess I had considered online surveys to be a haven from the worst scams, but now I’m realizing that I was probably being over-optimistic.

Another really annoying thing that happens to me sometimes with surveys is that I approve the hit, open another window then do and the survey – but when I go back to the original window it tells me that I’ve run out of time.  Basically, the requester has given an absurdly short amount of time to do the survey and I have not noticed.  It’s happened to me before and I think it’s actually the same requester who keeps doing it (I hope it’s not deliberate, but who knows?!)

Which brings me onto my third and final gripe, which never ceases to annoy me.  Why is it that MTurk requesters can block specific turkers, but the turker can’t block a requester that they don’t like?  I am maybe being cynical, but I think it tells you where the turker features in the Amazon scheme of things – down at the bottom of the pile!

Anyway, enough moaning.  I can’t say for sure if I will blog again before Christmas.  If I don’t, have a happy festive season, turkers, and a prosperous new year!