Search
Research
Faculty Initiatives
Participatory Culture
Our group works to create hybrid [physical-digital] systems that reflect and extend notions of art and culture in contemporary society, particularly in the realms of experiential and multi-dimensional media. Research will focus on cultural production, be it object or environment, that is not static, but emphasizes kinetic, adaptive and dynamic processes in contemporary art making. We will explore and develop tools with applications that can range from robotics to architecture, live performance to video games. This module will provide a place to practice creativity by process of iteration and critique, as well as a potential for opportunities and interactions with professionals from a diversity of fields in art and culture. Themes of interest include aesthetics and inference, the nature of participation and networks, the shifting role of technology in art, and attention to current practices across disciplines. Current activities include an intensive workshop focusing on collaboration amongst students, faculty and working professionals on a single project.
Reflective Living
Members of the Reflective Living initiative are interested in understanding, designing and developing dynamic media computation systems that promote reflective thinking, increase community awareness, support creative collaboration and facilitate purposeful communication between individuals and groups. We engage in research in behavior-based interaction architectures, media-sharing frameworks, analysis of large-scale social networks, multimedia presentation tools and narrative computation.
Project Website: Reflective Living
Modular Human Movement Sensing and Analysis for Experiential Media
In this initiative, we will develop a modular movement sensing and analysis platform for rapid instantiation of experiential media systems with emphasis on embodied learning and training, in complex, real-life scenarios. Modularized components allows for fast system reconfiguration to different real-life scenarios. The resulting sensing platform will be easy to operate and use, and cheap to build. We envision that this modular platform will have a three-level hierarchical structure. The lowest fundamental level is a rich repository of functional modules for movement sensing and analysis, feedback, data archiving, analysis and performance evaluation. Different modules have their individual functionalities and they are flexible to be tuned for different applications. The middle level is a data flow and fusion mechanism allowing for versatile ways of information fusion and integration. For example, a variety of sensor fusion strategies will be built into this level allowing users to try out different multimodal sensing possibilities for a particular scenario. The top level is a network/community of people as developers and users of the instrumentation who use these modules in their daily research and work and at the same time contribute to the instrumentations. Once developed, such instrumentation for embodied training will be widely disseminated in schools, hospitals, and homes for use in movement rehabilitation and embodied learning.
Project Website: Movement Sensing and Analysis
Gesture Recognition Toolbox: Arts, Media and Engineering Library Assortment (AMELiA)
The Arts, Media and Engineering Library Assortment (AMELiA) is a collection of libraries focusing on real-time recognition and synthesis of stochastic sequential patterns. Based on the Arts, Media and Engineeering Patterns Library (a generic template-based C++ library), and ofxPatterns (an easy-to-use, pre-compiled object-oriented C++ library), AMELiA is able to recognize and synthesize sequential patterns from various sensing modalities, such as speech, body gesture, mouse, and pen movement. AMELiA has been successfully used on several publicly available datasets and achieved state-of-the-art results. In the immediate future, we will develop educational applications using AMELiA for teaching and learning concepts and models related to sequential patterns. Meanwhile, we will enhance the documentation of AMELiA and further its ease of use. Over time, we will also add support for more use cases and evaluate the libraries on additional datasets.
Project Website: Gesture Recognition Toolbox