What data transformation library should I use? Pandas vs Dask vs Ray vs Modin vs Rapids (Ep. 112)

<p>In this episode I speak about data transformation frameworks available for the data scientist who writes Python code. The usual suspect is clearly Pandas, as the most widely used library and de-facto standard. However when data volumes increase and distributed algorithms are in place (according to a map-reduce paradigm of computation), Pandas no longer performs as expected. Other frameworks play a role in such context. </p> <p>In this episode I explain the frameworks that are the best equivalent to Pandas in bigdata contexts.</p> <p>Don't forget to join our <a href='https://discord.gg/4UNKGf3'>Discord channel</a> and comment previous episodes or propose new ones.</p> <p> </p> <p>This episode is supported by <a href='https://amethix.com'>Amethix Technologies</a></p> <p><a href='https://amethix.com'>Amethix</a> works to create and maximize the impact of the world’s leading corporations, startups, and nonprofits, so they can create a better future for everyone they serve. Amethix is a consulting firm focused on data science, machine learning, and artificial intelligence.</p> <p> </p> References <ul><li> <p> Pandas a fast, powerful, flexible and easy to use open source data analysis and manipulation tool - <a href='https://pandas.pydata.org/'>https://pandas.pydata.org/</a></p> </li> <li> <p>Modin - Scale your pandas workflows by changing one line of code - <a href='https://github.com/modin-project/modin'>https://github.com/modin-project/modin</a></p> </li> <li class="lead"> <p>Dask advanced parallelism for analytics <a href='https://dask.org/'>https://dask.org/</a></p> </li> <li> <p>Ray is a fast and simple framework for building and running distributed applications <a href='https://github.com/ray-project/ray'>https://github.com/ray-project/ray</a></p> </li> <li> <p>RAPIDS - GPU data science <a href='https://rapids.ai/'>https://rapids.ai/</a></p> </li> </ul>