Intuitive Virtual Objects Manipulation in Augmented Reality: Interaction between User’s Hand and Virtual Objects

Authors
Makoto Sakamoto1, *, Takahiro Ishizu1, Masamichi Hori1, Satoshi Ikeda1, Amane Takei1, Takao Ito2
1Department of Computer Science and System Engineering, University of Miyazaki, Miyazaki 889-2192, Japan
2Graduate School of Engineering, Hiroshima University, Higashi-Hiroshima 739-8527, Japan
*Corresponding author. Email: [email protected]
Corresponding Author
Makoto Sakamoto
Received 23 October 2018, Accepted 22 November 2018, Available Online 26 February 2020.
DOI
https://doi.org/10.2991/jrnal.k.200221.003
Keywords
Augmented reality; virtual objects; occlusion problem; hidden surface removal
Abstract
Augmented Reality (AR) is a technological advancement which brings to reality the intuitive interaction that a user is attracted to virtual objects. In this paper, we focus on intuitive manipulation of virtual objects with bare hands. Specifically, we propose a method to deform, move, and join virtual objects by intuitive gestures, and aim at intuitive interactions with virtual objects. In order to manipulate virtual objects with bare hands, the position coordinates of the user’s fingers are acquired using the Leap Motion Controller. Additionally, we use Bullet Physics as an engine to represent virtual objects. On the other hand, in AR technology, since the three-dimensional (3D) model is superimposed on the image of the real space afterward, it is always displayed on the front end rather than the user’s hand (occlusion problem). Thus, the user cannot perform an intuitive manipulation. We solve the occlusion problem by performing appropriate hidden surface removal. From the results of a study using data gathered from a questionnaire survey, we found support for the intuitive interaction between the user and virtual objects.
Copyright
© 2020 The Authors. Published by ALife Robotics Corp. Ltd.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)