Claim
Taking away rewards immediately decreases engagement, but it goes back to the baseline level or above in the long run, which means, any reward introduction, even temporary, has a major positive return.
Moreover, taking rewards away from time to time might have the extra benefit of fulfilling the desire to "take a break" and help users balance their focus on short-term goals vs long-term goals.
Study
Goswami & Urmisky'study (2017)
Goswami & Urminsky, made a series of dynamic tests to test the above hypothesis.
Participants could pick between Task 1: Moderately challenging math problem to solve, or Task 2: watching an enjoying video clip.
Each round took 30 seconds.
The test was created with 100 participants, which yielded 91 survey results and 77 valid tests (after removing some users for failing basic attention tests etc)
The control group would just be solving tasks, while the tested group was told after Round 1 that during the second round (and only the second round) they will be getting a bonus (cash).
Observations were gathered.
Goswami I, Urminsky O. The dynamic effect of incentives on postreward task engagement. J Exp Psychol Gen. 2017 Jan;146(1):1-19. doi: 10.1037/xge0000206. PMID: 28054810.
Results
The control group picked the math task 67% of the time.
Rewards increased the math task popularity to 88% (61% for control group during this period).
After the incentives have finished, the popularity of math tasks dropped to 53% (73% control group).
Overall using the temporary incentive increased the math task favourability to 76% from 63% which is a significant improvement.
Conclusion
Taking the reward away from the task will significantly reduce user engagement. But only for a short time, only to "take a break", and exercise their own "autonomy". After a short period of time, the baseline level comes back.
Use rewards. Even if you can only do it periodically.
Useful reads
https://www.researchgate.net/publication/312100138_The_dynamic_effect_of_incentives_on_postreward_task_engagement -access to the full research paper in pdf