Established in 2006, The Natural User Interface Group is an open source community that creates and shares interaction techniques & standards that benefit designers & developers throughout the world. We offer a collaborative environment for scientists that are interested in learning and developing modern Human/Computer Interaction methods and concepts. Our research includes topics such as: computer vision, touch computing, voice & gesture recognition, and experience design and information visualization. We have participated in every GSoC event since 2008.

Natural User Interfaces are becoming common in our everyday lives. Companies are rapidly introducing new product lines with NUI methodologies interwoven within; Google, Samsung, Apple, Microsoft and others… all with a major push towards enhanced interaction. The class most influenced by the emergence so far is the “smart mobile” as each unit contains 5-10 sensors paired with rich effectors. With 2011 seeming to be the year of the tablet both touch computing and our understanding of UX must scale once again as Android has shown with Fragments. Moving beyond mobile applications, entertainment devices like the Xbox Kinect Sensor give us new sensing abilities which officially registers on Windows 7 as NUI Audio, NUI Camera and NUI Motor devices, hinting towards the future.

By participating in GSoC, we would like to expand the NUI concepts to a wider audience, raising interest from students around the world that do not typically have the opportunity to work with such technologies. We aim to study emerging sensing solutions and translate to applicable scenarios, starting with computer vision, natural language, and signal processing. Beyond that students can ponder on innovative problems that include related developments in clustering algorithms, image processing and machine learning.

With over 12,000 members, our goals is to accelerate the exploration of sensing solutions and interaction scenarios, thus allowing us to find the most effective methods of constructing modern interface devices. Our members are a worldwide gathering of students, researchers, designers, developers and engineers all who share a similar passion for the creation and distribution of open systems.

An important aspect is to create and utilize open standards that allow development to flourish. An example is our Community Core Vision or CCV project which is built in OpenFrameworks and supports several communications modes (Binary, TUIO, OSC and XML). The project is nearing one million total downloads and has introduced computer vision to a whole new audience, lowering the entry bar for new researchers while remaining powerful for advanced use and content creators. We would like to thank Google Summer of Code Project and the amazing contributors for making this a possible.

Everyone in the community is really excited about GSoC 2011 as last year set the stage for our Core Platform, Which aims to manage voice and visual signals and route to clients build in AS3/Java/Python. This year we would like to continue our development with focus on several key projects:

Community Core Services (C/C++):

• Community Core Audio (CCA) (GSoC 2010 - Alpha) - A cross platform server which manages voice inputs, convert voice to text, and output resulting messages to network. Alpha shipped during GSoC 2010 including a new addon (ofxASR) abstracting signal processors and sources. 2011 Goals - Ship 1.0 release stable on all platforms and broaden natural language processor support (SAPI, Sphinx, OSX-SRS) / CCW Management
• Community Core Vision (CCV) (GSoC 2008/2009/2010 - Beta 1.4) - A cross platform computer vision server for high fidelity tracking of objects such as fiducials, hands or fingers. First release during GSoC 2008 and with great community response. In 2009/2010 we continued to develop new tracking techniques and enhanced stability. 2011 Goals - Ship 1.5 with Depth Sensor (Kinect Devices) / Augmented Reality / CCW Management
• Community Core Web (CCW) (GSoC 2010 - Alpha) - A cross platform light weight web server which supports plugins for web development standards such as: Moongoose/PHP/MySQL/SQLite etc. Manages C++/Javascript communications allowing CCA/CCV to be controlled via minimal web client. Alpha started during GSoC 2010 but needs further development until official community beta. 2011 Goals - Ship 1.0, optimize, stable on all platforms, continue development of HTML/JS based management utility and binding to core services (CCA/CCV)

Community Client Frameworks:

• Actionscript/Flash - A client UX framework and set of example applications written in Actionscript (2007+)
• C/C++ - A client UX framework and set of example applications written in C/C++ and OpenGL ES 2
• Java/MT4J - A client UX framework and set of example applications written in Java (GSoC 2009/2010 projects)
• Python/PyMT - A client UX framework and set of example applications written in Python (GSoC 2008/2010 projects) 2011

Framework Goals - Development and standardization of target frameworks including structure, documentation and release packages.

Community Tools:

• Signal Simulators - Utilities to generate signals when hardware is not available. (QMTSim GSoC 2008) 
• Signal Translators - Client/Server Translation Utilities (flosc, wm_touch2tuio, mtvista)

With ongoing developments for supporting projects:

• OpenFrameworks - Continue contributing to Addons/Framework through C/C++ developments
• Protocols – Socket support on clients and servers to broaden TUIO/OSC support UX Design - A general focus on emerging experience design topics related to NUI 
• Web Applications - Community Search, Publishing and Communication (LAMP Stack)

Other than the projects mentioned above, we have our Ideas Page with topics to help give students inspiration and insight when writing their proposals.

Please refer to our ideas page for a full overview of past projects and video showcase: http://nuigc.com/gsocvideos