We are investigating human-computer interactions that span across several devices, also known as multi-device interaction or digital ecosystems. Our project has looked at several prototypes as well as commercial systems, in lab as well as real world settings, and qualitative as well as quantitative perspectives
As an example, we have investigated a number of multi-device prototype systems within the music production domain. Music production involves multiple people interacting with many individual and connected systems in parallel and in sequence.
However, current systems and setups are not designed as a whole, but consist of many individually designed systems and devices. We have addressed this opportunity through the design of systems that are specifically meant to fit well with others around them, such as mixing desks, digital work stations etc. One system extends a physical mixing desk with a spatially tracked iPad (using Kinect), allowing the music producer to access sound effects related to a specific track by moving the iPad to that track. Another system extends an on-screen mixing tool with a spatial representation of the different tracks of a mix, allowing the music producer to change their spatial position in the mix by moving them on the screen (surround sound).
Assoc Prof Jesper Kjeldskov
Associate Professor of Human Computer Interaction