The Project

The goal of this project is a seamless continuous development experience. Modified code should be automatically rebuilt, re-launched and re-tested as quickly as possible.

The Yesod and Happstack projects have planned to collaborate on improving their development mode toward this end [1, 2], but have been lacking in resources. Previous attempts in this area, such as shaker [3] were not well advertised and did not offer significantly faster development.

A foundational component to this project is file-watching code that can be used to immediately trigger a rebuild. The file-watching code will use an abstract interface that is interoperable, efficient, and capable of falling back on less efficient techniques when necessary. There are already working libraries for platform-specific file watching on Linux, Mac, and Windows [4, 5, 6], and this project would seek to make these efforts more comprehensive, and unify them. A file-watching library has its own uses outside of the goal of this project. For example, building an application that changes when its config file is edited.

With a unified file watcher in place, the next task for this project is to abstract the pattern of a “lively development environment” common to frameworks such as Yesod and Happstack to make it accessible to any Haskell project. Initial efforts in this direction have been made in the form of a command-line tool [3], and this project seeks to build on these efforts. Abstracting the notion of a “lively development mode” out of these frameworks, and integrating better file watching, and reducing compile-and-link time has the potential to benefit all Haskell users. If any Haskell developer can move their code into a system that aggressively, automatically, and efficiently rebuilds code, every developer can realize a more efficient development environment as a direct result of this project.

Haskell has 2 established techniques for fast code reloading: the GHC API and hs-plugins. In addition to supporting these, the lively development environment will be able to fallback to reliably re-invoking a cabal build.

Once the foundation for continuous development has been laid, further points of development environment integration and performance enhancement and can be explored. The most immediate example of next steps include offering lively development environment integration for Emacs, Vim, and Eclipse, and analyzing issues with using the gold linker for Haskell.

The Applicant

I am a computer science Masters student at the University of Western Ontario. Through a combination of self-directed contract work and internship opportunities at the University of Waterloo where I studied as an undergraduate I have over two years of professional software development experience [7]. This experience has included relevant work on a compiler for a language designed for distributed systems at OptumSoft, Inc. Through internships with Sybase iAnywhere, STAR Group at the University of Waterloo, Tagged, Inc., and Google, Inc. I have been exposed to the technical aspects of JBOSS application server, and various web technologies for server-side and client-side development such as Django, Memcache, and various Google technologies. In addition, my academic experience includes building a compiler for JOOS, a large subset of Java, for x86 machines; developing an object-and-message-based operating system for 68k Coldfire machines; and formal study a broad range of programming languages such as Prolog, Scheme, and ML in the context of a fourth year undergraduate course.

Most of my work has been with imperative programming languages. However, I have completed several academic exercises in Scheme and ML and I am currently working on a Haskell library for multi-level modelling in statistics. Though my experience with functional programming has been limited, I am a fast learner; I am excited to contribute to bringing functional programming—Haskell in particular—into more and more fields of practice. By exploiting my experience with compilers and debugging tools, I am confident that I can make a valuable contribution to a project of this nature in the Haskell community. My hope is that this project will have have a deep impact for the entire Haskell community striving for rapid development of quality code.

Naturally, my developing knowledge of Haskell will require good communication with my mentors to guide the project. Should my mentors be unable to assist with a particular issue I will turn to the Haskell community for advice, taking advantage of contact information for contributors to related projects, as well as mailing lists and IRC channels.

Plan of Action

Progress on this project has many milestones, some of which may not be feasible to reach in three months. Nevertheless, these milestones offer a solid plan for the summer that is sure to occupy the full three months; if some milestones prove unachievable this summer, subsequent milestones offer direction to further work in this area. The milestones, along with time estimates, are as follows:

1. Develop an abstract cross-platform file watcher interface for Haskell. The task is simply to unify existing libraries and code into a coherent and robust interface. (2 weeks -- a lot of padding in the time for the first milestone because it is the first)
2. Abstract GHC API code currently written for Yesod and trigger it with file watching code from the previous step. The GHC API will be required for an efficient generalized Haskell development mode. Luite Stegeman wrote code for this that is used in the latest version of Yesod [8], and is willing to mentor this portion of the project. This code avoids needing to re-link all existing packages by keeping a GHC instance in memory. (1 week)
3. Integrate the hs-plugins interface successfully used by Happstack as an alternative to the GHC API development mode. Each approach has strengths and limitations and may be better suited for different projects. (1 week)
4. Integrate a failsafe technique of building with cabal for when there are issues using the GHC API or hs-plugins. The previous version of yesod devel did this. (1 week)
5. Generalize the notion of a “development mode” to make this methodology accessible to any Haskell project. (2 weeks)
6. Integrate this tool particularly well with web frameworks such as Happstack, Yesod and Snap. Web frameworks can reload specific request handlers. Integration with automatic browser refresh is also important for efficient development. (2 weeks)
7. Develop integration for the newly abstracted Haskell development mode in Emacs, vim, Eclipse, and Leksah. The main goal is an API usable by these environments, not to write specific code for them. (2 weeks)
8. Research and report on issues with using the gold linker with Haskell. The gold linker is much faster than the GNU linker, but Haskellers are unclear on when it is safe to use and when it fails [9]. Clear reports to GHC may help with future compatibility.

As time estimate indicate, it seems feasible to complete the the first six milestones this summer. We would not be surprised if certain tasks turn out to be harder than expected. The milestones are structured such that skipping steps that prove too difficult or time consuming is acceptable because there will still be a huge value to the community for this project even if some are not achieved. The last two items in particular set the stage for future work by other members of the Haskell community.

References

[1] http://permalink.gmane.org/gmane.comp.lang.haskell.yesod/246
[2] https://groups.google.com/forum/?fromgroups#!topic/happs/tJ9uoGzg2ME
[3] http://hackage.haskell.org/package/shaker
[4] http://hackage.haskell.org/package/hinotify/
[5] https://github.com/luite/hfsevents
[6] http://hdiff.luite.com/cgit/Win32-notify/tree/System/Win32/FileNotify.hsc
[7] http://markdittmer.org/resume.html (contact author for access credentials)
[8] http://hackage.haskell.org/package/yesod
[9]https://bugzilla.redhat.com/show_bug.cgi?id=635935