Recovering from a bad HIT Approval Rate

As any experienced turker will tell you, recovering from a bad HIT approval rate can be a frustrating experience.  It seems to take forever to come back from the damage that just one or two MTurk HIT rejections can do and all the time your HIT approval rate is damaged, your ability to make money online is compromised.

If you are new to turking and aren’t aware, Amazon Mechanical Turk calculates your HIT approval rate by comparing how many of your MTurk HITS have been rejected by requesters for HITs submitted against how many have been accepted.  The resulting percentages are called your HIT rejection rate and your HIT approval rate.  (All these figures can be seen on the Dashboard part of your Mechanical Turk account.)

The problem is that many of the requesters insist on a HIT approval rate of 95% or better as a qualification requirement and if you don’t meet the qualification standard, they don’t allow you do their HITs.  These MTurk HITS are very often some of the best paid ones and so getting a bad HIT approval rate can really damage your ability to make money online.

As I have mentioned before, when I started out with Mechanical Turk, I didn’t appreciate the problems that having a bad HIT approval rate might bring, so I just tried out lots of different HITs without really caring too much whether some of them got rejected or not.  This came back to haunt me and I learnt my lesson accordingly.

More recently, however, I have seen my HIT approval rate drop again through no fault of my own.  This was apparently because, as I found out from turkers at TurkopticonToolbar, a requester who had been reliable in the past suddenly started using new (faulty) software which was wrongly automatically rejecting HITs that were actually completely fine.  I did complain about it, of course, but it was to no avail, as usually happens!  (See 10 reasons why I hate Mechanical Turk)

Anyway, I’ve spent my time recently working my socks off on getting my HIT approval rating back up to 95%.  It isn’t fun.  As well as my choice of HITs being reduced, each time I do a new HIT I am worried about getting another rejection and getting pushed back down again (once bitten, twice shy!). 

I’m not sure if I mentioned it in my comparison of MTurk and Microworkers: ­MTurk or Microworkers, which is best?,  but Microworkers are more generous with when it comes to recovering from a bad HIT approval rate.  With Microworkers your rate will recover naturally with time, even if you don’t do anything - so long as you don’t keep picking up rejections you will be okay.

In conclusion, my advice would be to not let your MTurk HIT approval rate fall below 95% if you can possibly help it.  If it does happen, I would focus on getting it back up by doing ‘safe’ HITs, that’s to say HITs being offered by MTurk requesters that you trust, as you can’t afford to get any more rejections.  Be prepared for a long haul, though - because the HIT approval rate is a proportional figure, you will need many new HIT approvals to make up for just a few rejections.

MTurk Requester Payment Problems

Earning money online can be frustrating sometimes.  Most turkers who have worked on Amazon Mechanical Turk for any length of time has experienced MTurk requester payment problems.  There are times when you know why you have not been paid sometimes, for instance, you had a computer glitch, or an interuption happened, like you were phoned up, and you ran out of time to finish doing the MTurk hit.  Another time, you might have been unsure what you were supposed to be doing on the hit and so not surprised when your work was rejected by the MTurk requester.

Late Payments

It is frustrating when you complete a job and then you are checking your account every day expecting to see your earnings and nothing happens.  In my experience, most MTurk requesters pay up within a couple of days.  Some pay quicker than that, within a matter of hours sometimes.  The annoying thing is where is takes over a week before you receive your MTurk payment.  You can email the MTurk requester in this situation, but be aware that Amazon allows them up to 30 days to either pay or reject your work.  That is way too, long in my opinion, and Microworkers have got a better idea by giving MTurk employers a one week cut off period to resolve outstanding jobs.

Non-payment by MTurk requester because of hits rejected

Occasionally you can get hits rejected by an MTurk requester and it seems unfair.  As well as losing the money you felt you earned, getting hits rejected can also damage your hit rejection rate and effect which hits you are able to do in future.  So even if the amounts of money involved are small, it can still be annoying and upsetting.

If there were technical problems at the time of doing the hit, then you should report the hit as broken using the link in the bottom right.  If you believe that the requester is breaching the Amazon Mechanical Turk policies, then you should click the “Violates the Amazon Mechanical Turk policies” link which is also at the bottom right.  

Sometimes you have no idea why you have had your hits rejected.  In this case it is best to contact the MTurk requester.  I would advise that the tone you adopt with them is polite but firm.  You should explain that you followed all the instructions and ask why your work was rejected.  In many cases the matter can be resolved this way and you will get your money (or they will explain to you your mistake).  Sometimes the MTurk requester will use stalling tactics and you have to write more than once before they pay up.  Note though, that an MTurk requester cannot themselves reverse a rejection decision without the help of Amazon.

If you still have no joy, then your options for complaint as an MTurk worker are very limited (one gripe of turkers is that MTurk requesters can ban certain MTurk workers, but not the other way around).  Though you can, of course, warn other MTurk workers about the bad requester and discuss your experiences either on Turkopticon, or on one of the MTurk forums.

How much money do you earn online with Amazon MTurk?

How much money do you earn online with Amazon MTurk?  This is good question and a relevant one for someone who is weighing up various ideas to make money online.  In my experience Amazon Mechanical Turk is actually a relatively straightforward option for making money online, as it is free to sign up to and once you have got used to the set up of the site, you can easily find out details of what each Amazon MTurk hit (task) entails and how much money that you will earn for its completion before you click to accept it. 

If you want actual earnings figures for MTurk then I will give you an idea of how I work and how much I earn in return for completing MTurk hits.  I tend to work in stints of two to three hours per day on MTurk.  Where possible, I do hits for MTurk requesters that I have worked for before, as then I know that there is a good chance that I will be paid and I also have a rough idea when.  For those two or three hours work, I would normally expect to come away with somewhere between and $8 and $18.  There may well be MTurk workers out there who do much better than that, if so, please feel free to comment.  I think the most that I have made is $27 in a half day but that is not typical for me.

On my bad days, a lot of my time is wasted on trying to find suitable MTurk hits.  On the good days, I find hits straight away and get straight on with it.  I don’t use any programs to cheat, though I am aware that there is software out there that some people have experimented with, such as software for generating reviews, blog posts, written articles etc.   

There are people who say that they have stumbled on reasonably well paid tasks that they can complete a lot of very quickly and they’ve come away with earnings of  $200+ for a day’s work, but that has never happened to me.  I would also remind people to be wary of the very high paying hits, as many of them are scams, designed to get you to hand over personal information.

I am relatively cautious when it comes to choosing jobs, but my wife is more experimental and that means her MTurk earnings tend to be more variable.  You can sometimes find lucrative MTurk hits by looking around and trying things.  But one risk of doing a lot of completely new MTurk hits all at once is that sometimes the requester will reject all your hits, either because you didn’t understand the instructions properly and did the tasks wrong, or because you have been unlucky and got one of the dishonest Turk requesters.  This can damage your rejection stats and effect your future work, as well as waste your time.