Handy Parallel (Distributed) Computing in Python - Liang Bo Wang

Handy Parallel (Distributed) Computing in Python

Liang Bo Wang /英語

We all start our programming with single process in mind. But parallelization from scratch is a real headache. Things get worse when it comes to writing a spaghetti code in both short time and high performance, which is often the case doing data analysis. In fact, cases like computation under different conditions, programs can be easily parallelized (distributed) with a few modifications using existed library like multiprocessing, IPython Parallel, and Celery. This talk makes you not afraid of parallel or distributed computing and provides ways for different level of parallelization from single machine, cluster, to cluster with task queue. I will only talk about the basic scenario, which should be easier for newbies to try on and understand how powerful these tools can be. A machine learning example will be given in the end to compare performance and possible issues with different implementations.


Profile picture
呆呆電雞生,喜歡寫 R / Python,喜歡統計與生物資訊。目前為 Taiwan R Users Group 工作人員及 Taipei.py 常客。 Bioinfo / Stat / R / Python, master student of NTU BEBI. Co-organizer of Taiwan R Users Group and freq attendee of Taipei.py.


R, Python, pandas, pycontw, mldm





HDE, Inc. mongodb Google

Tagtoo Vpon Github Github Github

Quanta Research Institute

AcoMo Technology

CLBC KKTIX QSearch Python Software Foundation Open Source Software Foundry LIVEhouse.in Young Optics Wolf Tea

QSearch Business Next Vpon Inside 硬塞的 DIGITIMES INNOMAMBO 創新曼波

Built with Django and Mezzanine by PyCon Taiwan

Hosting provided by StreetVoice.

網站問題或建議請 回饋給我們