MTurk online surveys: 3 peeves of mine

I’ve been working on Amazon MTurk a lot recently in the run up to Christmas - it’s a good way of saving for gifts, especially when I know that what I plan to buy is sold on Amazon.  But some of the problems with the Mechanical Turk online surveys have been annoying me. 

Don’t get me wrong, I enjoy doing surveys.  They seem like less hassle than some of the other tasks sometimes, the pay varies and is maybe going down gradually over time, but often still isn’t bad in MTurk terms.  The requesters are also generally pretty reliable, especially if the MTurk requester is a university academic or student, so I worry less about not getting paid, or there being some sort of scam going on.

But recently when I search for surveys, there have been more and more scams that are calling themselves surveys appearing (you know the sort, they always offer you an "easy money for 5 minutes work" and say they're going to pay you $20!).  I mean signing up for a credit report can in no way be considered a survey!  Neither can “test my website by entering all your personal details and pressing submit”!  I guess I had considered online surveys to be a haven from the worst scams, but now I’m realizing that I was probably being over-optimistic.

Another really annoying thing that happens to me sometimes with surveys is that I approve the hit, open another window then do and the survey – but when I go back to the original window it tells me that I’ve run out of time.  Basically, the requester has given an absurdly short amount of time to do the survey and I have not noticed.  It’s happened to me before and I think it’s actually the same requester who keeps doing it (I hope it’s not deliberate, but who knows?!)

Which brings me onto my third and final gripe, which never ceases to annoy me.  Why is it that MTurk requesters can block specific turkers, but the turker can’t block a requester that they don’t like?  I am maybe being cynical, but I think it tells you where the turker features in the Amazon scheme of things – down at the bottom of the pile!

Anyway, enough moaning.  I can’t say for sure if I will blog again before Christmas.  If I don’t, have a happy festive season, turkers, and a prosperous new year!

My favorite MTurk requester

My favorite Mechanical Turk requester was Content Galore.  For a period of time they were paying $1.80 for 150 word encyclopedia style articles on MTurk.  They would give you a name or a theme, such as “Burt Reynolds” or “Orlando, Florida”, you went away and did some basic research (looked it up in Wikipedia or another reputable source), wrote a summary of the information in your own words and submitted it.  The articles were checked by other turkers and pretty much always accepted (at least in the experiences of my wife and I).

It was a great way of making money from home.  Although there was the odd sparse period here and there, generally there was plenty of work to do.  Content Galore were quick payers too – you would usually have your work accepted within a couple of hours, often almost instantly.  The HITs would take me just over 10 minutes to complete, meaning that I could earn 20 dollars or more in just a couple of hours or so, which is pretty good by general MTurk standards.  My wife could complete them even faster than me, often doing them in half the time that I could!  Content Galore were great and I came to rely upon them, usually ignoring other MTurk HITs during that period.

But then something went wrong.  My favorite MTurk requester disappeared for a long time, which wasn’t entirely unusual for them, but when they returned, something had changed.  My wife noticed it first, she did three HITs for them straightaway, but to her shock had all three of them rejected!  The pattern was repeated, with Content Galore disappearing again and returning, only to reject the HITs of my wife and I.  It was bizarre, as we followed all the instructions and had done countless HITs for them previously without a single rejection.  It resulted in our HIT approval ratings getting damaged and our time being wasted with no money in return.

The rumor was that unscrupulous turkers had been trying to cheat the system, including submitting work that had been spun (generated by special software, which turns out work that reads well to a computer but looks terrible to a native English speaker).  Content Galore had responded by introducing their own software to check work in an attempt to spot the spinners and cheaters – but the software was overzealous and rejecting work from honest turkers as well as the cheats.

Content Galore disappeared and reappeared again and again.  Every time that they returned, my wife or I would try doing a ‘test HIT’ to see if Content Galore had resolved their problems, but each time we did, the test HIT was rejected.  My wife had one HIT rejected on the grounds that she had completed it faster than was humanly possible.  Well, I was with her at the time and can vouch that she did the HIT according to all the instructions and produced a great piece of work, which was researched, read well and had been spell checked too, so I don’t know what was going on.

So nowadays, if I happen to come across a Content Galore HIT, I will ignore it.  But working for them was good while it lasted.

Writing an MTurk blog: A look at the best ones

Sustaining an MTurk blog isn’t always easy.  Like most turkers, I suspect, my enthusiasm for MTurk ebbs and flows.  There are periods where I do a lot of turking, periods where I do just a little each day, and periods where I do no turking at all.  The reasons for this are varied.  Sometimes there are HITs on MTurk that I enjoy and pay reasonably well, so I spend more time on MTurk to exploit them.  Sometimes I am forced to spend more time on MTurk purely because I need some extra money.  Other times, I have other things happening in my life and I haven’t the time, need, or motivation to be on MTurk.

I have been looking at other blogs recently, principally ones that are written by turkers and aimed at other MTurk workers - there aren’t actually that many active ones around, it seems.  The official Amazon MTurk blog, for instance, tends to focus more on the needs of requesters and programmers, than the requirements of turkers.

Judging by some of the other MTurk blogs out there that I have managed to find, many turkers start out with the notion that they can make enough regular money off MTurk to sustain themselves for a long period of time.  Then they gradually become disillusioned if this doesn’t happen and some get angry with being poorly treated, or at the quality of some of the HITs.  Some of the most entertaining and informative writing comes from disillusioned turkers, although their blogs are often short-lived.

“Mechanical Turk Blog” was clearly motivated to write an MTurk blog in order to highlight what he saw as the scams and injustices of MTurk, of which there are many.  I think it is a continual source of disappointment to all of us that Amazon can appear unwilling, or unable, to police the posted HITs better when they are clearly aimed at gathering workers’ contact details for spamming purposes, or worse still, are aimed at collecting personal info to be used for identity theft.  These sort of HITs appear regularly on MTurk, sometimes masquerading as a “survey” or a “website test”.  Other times, turkers are being asked to perform tasks that are of questionable ethics, and/or legality.

Having made his point, “Mechanical Turk Blog” now seems to have melted back into cyberspace.

Turk Lurker’s blog has some entertaining and informative reads in it!  He has a dry sense of humor, referring to Mechanical Turk as the “Slave Yard” and giving his blog the url: “MTurkey”.  Strange that his posts suddenly stop in 2007 and then suddenly reappear in 2011, but these things happen, I guess – maybe his wife had a baby or something?

Eric Cranston wrote an interesting turker blog, but unfortunately it petered out in 2007 after 2 years of posts.

There are many blogs by people on how to make money from home which refer to MTurk in passing, but it they generally just write a single article and they often don’t give you much more information or insight than a general introduction to MTurk for beginners.

Anybody know of any more interesting MTurk blogs?