DataDancing: Full Body Interaction for 3D Surfaces and Spaces
My research explores how immersive technologies transform the way people interact with information. Traditional data analysis systems are constrained by flat screens and desktop interfaces, whereas Virtual and Mixed Reality enable information to be distributed throughout the surrounding environment. I investigate how people organize, navigate, and manipulate multiple visualizations in these spatial workspaces, with the goal of developing more natural and effective interfaces for data exploration and sensemaking.
A major contribution of this work is the development of DataDancing, a design space for visualization view management in 3D environments. Through a systematic review of immersive visualization systems, I identified key dimensions that influence how information is presented and interacted with in space, including spatial reference frames, display geometry, proximity, mobility, and interaction modalities. This framework provides researchers and designers with a structured way to understand existing immersive interfaces and create new interaction techniques for information-rich environments.
Beyond theoretical frameworks, I design and evaluate novel embodied interaction techniques that leverage the user’s whole body as an input mechanism. My work investigates how hands, feet, body movement, and proxemic relationships can be used to manage and navigate collections of visualizations distributed across virtual walls, tables, floors, and body-centered displays. In particular, I explored some of the first applications of foot-based interaction for visualization tasks in immersive environments, demonstrating how alternative input modalities can support hands-free and eyes-free interaction while expanding the available interaction space.
Through qualitative user studies, I examine how different spatial layouts and interaction techniques affect usability, workload, and user preferences. My findings show that users benefit from a combination of fixed public displays and body-centered personal workspaces, while also highlighting opportunities for multimodal interaction that integrates hand, foot, and proxemic input. These results provide practical design guidance for future immersive analytics systems and collaborative virtual workspaces.
Collectively, this research advances our understanding of embodied cognition and interaction in immersive environments. By investigating how people use space, movement, and the body itself as part of the analytical process, my work contributes new principles for designing next-generation immersive interfaces that support data analysis, collaboration, and decision-making in spatial computing environments.
Figure 1: (a) A hybrid prototype demonstrating interaction possibilities of the DataDancing design space for visualisation view management. (b) One of four prototypes evaluated in a qualitative user study with a body-fixed large display and a floor display with novel foot interactions. (c) The third-person front view of one participant ‘dancing’ with views on the virtual floor in VR (synchronised with scene b). (d) A novel foot device used in the user study for each participant foot with reflective trackers and a pressure sensor system.
Figure 2: A design space illustration for visualisation view management for 3D surfaces and spaces. In this design space, we propose seven design dimensions in two categories for visualisation view presentation (top) and four design dimensions in two categories for user interactions with views (bottom).
Figure 3: Use cases from literature (top row) or proposed by us (middle row). In these tables, a filled cell indicates the design dimension option used. The bottom five figures illustrate implemented prototype surfaces as design applications adapted from recent research: (a) Wall Display , (b) Floor Display , (c) Tabletop Display , (d) Body-fixed Display , and (e) Cockpit Display.
Publication and Thesis
Liu, J., Ens, B., Prouzeau, A., Smiley, J., Nixon, I.K., Goodwin, S. and Dwyer, T., 2023, April. Datadancing: An exploration of the design space for visualisation view management for 3d surfaces and spaces. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (pp. 1-17). Publication Link