Celery meets Singleton

Abstract

Celery is a popular framework for deferred, offline, or scheduled processing. Celery itself is written in Python, and it can well fit into docker, stack, and cluster structure.
In real situation, we might have requirement or usage like this: to ensure only one instance of celery task running. Celery itself could not do this out of the box. So there are some patches implementations or usage strategies to support singleton task. However all of them have their pros and cons. Sadly these patches need more dependencies and not robust enough, thus they could not fit to our situations.
In this talk, I will share some implementation strategies, discuss characteristics, explain their weak points. After that, I will introduce a strategy which has different characteristics, pros, and cons from other strategies. And this strategy totally fit our requirement.

Description

celery: http://www.celeryproject.org/ how to implement single task: https://stackoverflow.com/questions/20894771/celery-beat-limit-to-single-task-instance-at-a-time another possibility: https://stackoverflow.com/questions/20091505/celery-task-with-a-time-start-attribute-in-1970/20096342 https://github.com/celery/celery/blob/c0e587e94575d7be9bfdabe40c1d5dfda268e68e/celery/worker/request.py#L432 https://github.com/celery/celery/pull/3684

Slides

https://hackmd.io/@fieliapm/B10XOgeXD#/

Speaker

郭學聰 (Hsueh-Tsung Kuo)

台灣某game會社的小小RD
目前是CPPRG(C# Perl Python Ruby Golang)熟手
Smalltalk與Scheme基本教義派
不是太奇怪的程式語言都會一些

請多多指教⎝(˶′◡‵˶)⎠