Write your own micro data processing framework in python

語言

英文演講

分類

資料分析

Python 難易度

中階

摘要

Data processing framework is the core element of Big Data. It provided good abstraction for computing resource and logic. In this talk, I will use google mapreduce (written in python) to introduce some key components such as message queue, pipeline, object collection, fault tolerance and task flow which used in common data processing frameworks. Then use a micro framework written in django to demo how data processing works.

David Chen

GliaCloud founder and coorganizer of GCPUG. Gliacloud is a startup focus on AI and data analysis.

I like coding, ramen. and skiing. Most of time, I involve in python coding in Google Cloud Platform for big data processing and cloud architect designing. It is my pleasure to join pycon with lots of passion community members.