Celery meets Singleton


Celery is a popular framework for deferred, offline, or scheduled processing. Celery itself is written in Python, and it can well fit into docker, stack, and cluster structure.
In real situation, we might have requirement or usage like this: to ensure only one instance of celery task running. Celery itself could not do this out of the box. So there are some patches implementations or usage strategies to support singleton task. However all of them have their pros and cons. Sadly these patches need more dependencies and not robust enough, thus they could not fit to our situations.
In this talk, I will share some implementation strategies, discuss characteristics, explain their weak points. After that, I will introduce a strategy which has different characteristics, pros, and cons from other strategies. And this strategy totally fit our requirement.


celery: http://www.celeryproject.org/ how to implement single task: https://stackoverflow.com/questions/20894771/celery-beat-limit-to-single-task-instance-at-a-time another possibility: https://stackoverflow.com/questions/20091505/celery-task-with-a-time-start-attribute-in-1970/20096342 https://github.com/celery/celery/blob/c0e587e94575d7be9bfdabe40c1d5dfda268e68e/celery/worker/request.py#L432 https://github.com/celery/celery/pull/3684


郭學聰 (Hsueh-Tsung Kuo)

目前是CPPRG(C# Perl Python Ruby Golang)熟手