Enhancing Task Scheduling and Implementing MapReduce for Pydra

by Yin Qiu for Oregon State University Open Source Lab

Pydra is a distributed computing framework developed at OSU OSL, which enables both rapid parallel program development and convenient cluster management. It is now on its early stage, and I'm glad to have a chance to make a contribution towards its upcoming first release. To be more specific, my proposed summer project involves 2 tasks: 1) to enhance task scheduling; and 2) to implement the MapReduce mechanism based on task 1. In addition, I'd like to optionally implement node auto-discovery.