The Software Engineering Part of Data Science

Abstract

Data science and software engineering is getting more and more intertwined with each other. But what is the common grounds between the two? How do we define the boundaries of the two disciplines? This talk will discuss the role of python in terms of development process for software engineers working in a data science environment. We will also tackle the different aspects of software engineering in relation to working with a data science oriented team.

Description

In this session, I will discuss the software engineering side of data science and how python bridges the gap between data science and software engineering. Will highlight some of the development process involved in working with data scientists. Will also demonstrate some sample codes and best practices that shows how to work as a software engineer in a data science company. Lastly, I will discuss how software engineers can shift to a more data science related job like a data engineer. **Outline:** - Introduction - Challenges faced by software engineers in the data science field - Working with a data science team - How Python bridges the two domains - Problem definition - Requirements building - Coding standards - Architectural challenges - Deployment - Solutions - Data driven dev (in Python) - Python tools and libraries for collaboration - Establising protocols - Data standardization - Understanding how to translate data scientist code - Performance considerations - Platform considerations - Multiprocessing (in python) - Deployment options - Python packaging - Platform specific python libraries - Overall system deployment - Transitions from software engineering to data science - Role of python in further improvement between the two fields - Conclusion

Slides

https://docs.google.com/presentation/d/1Xos0xRyDtXKPGQ2IkN5OaEDNHsJQTSPHukrMrmayGZ8

Speaker

Niño R. Eclarin

Hi, I'm Ninz.

I am a software engineer in the Philippines. I've worked with several startups in the past and currently working at Hacarus(focus on backend and machine learning integration).

I am currently taking my Masters degree in Computer Science here in the Philippines. My interests are Astronomy, Physics and Cooking :P