In this blog post, we’ll explore the types of motion recognition technologies that could replace keyboards and mice, their applications, and their future potential.
You probably clicked on this post while holding a mouse in one hand. As this example shows, keyboards and mice are still essential tools for inputting and receiving information. However, in the not-too-distant future, even these indispensable tools—the mouse and keyboard—may fade into history. This is precisely due to motion recognition technology. By moving beyond traditional, limited data input methods, motion recognition technology is creating more active and rapid ways to input data using sensors, cameras, and other devices. What types of motion recognition technology exist, and in which fields are they being utilized? Furthermore, how might this technology evolve in the future?
There are various methods for recognizing human movements, including data gloves, stereo cameras, depth-sensing cameras, and infrared sensors. Among these, the data glove is arguably the most classic form. The Sayre Glove, an early version of the data glove, was developed in the 1970s at the Visualization Laboratory of the University of Illinois at Chicago. This system utilized sensors attached to the glove to detect finger flexion, converting hand movements into computer input. Since then, data glove technology has continued to advance; today’s data gloves combine sensors that measure finger and hand movements with devices that track position and orientation, enabling the collection of more precise motion data. These data gloves are used in various fields, including virtual reality, motion capture, simulation, and research.
For example, the CyberGlove II, developed by the U.S. company CyberGlove Systems, precisely measures finger flexion and hand movements and converts them into real-time digital data. It can record finger and hand movements using up to 22 joint angle sensors and can be combined with separate position and orientation tracking sensors as needed. If this technology is combined with robotic arms, it becomes possible to create robotic systems that precisely replicate human hand movements. In particular, technology that converts intricate human hand movements into digital data holds great potential for future applications in fields such as medicine, virtual reality, and industrial design. However, since wearing the glove itself can be cumbersome, methods that recognize movements using only cameras or sensors—without the need for additional equipment—are also being developed.
Motion recognition technology using cameras is relatively familiar to the general public. Gaming devices such as Nintendo’s Wii and Microsoft’s Kinect for Xbox have contributed to making motion recognition technology widely known to the general public. However, the motion recognition methods used by these two devices are not the same. The Wii Remote used an infrared sensor and a sensor bar to detect the remote’s position and movement, while Kinect used an infrared light source and a camera to obtain depth information about the space and, based on that, recognize a person’s body movements. In particular, early Kinect models obtained depth information by projecting a pattern into space using an infrared light source and analyzing it with a camera. Therefore, it is not accurate to describe the system as one that detects the infrared radiation emitted by the human body to recognize joint movements. Rather than being the current leading motion-sensing devices, it is more appropriate to view the Wii and Kinect as examples that drove the popularization of motion-sensing technology.
This motion recognition technology, which utilizes cameras and sensors, enables the conversion of human hand and body movements into digital input. Recently, technology has advanced to estimate the positions of hands and body joints using computer vision and artificial intelligence, not only from depth cameras but also from footage captured by standard cameras.
Hand gesture recognition is expanding its scope of application to various fields, including virtual reality, augmented reality, robotics, sign language recognition, and human-computer interaction. Above all, it is gaining attention for its convenience, as it allows human movements to be used as input without the need to directly operate a separate input device. Motion recognition technology not only has the potential to expand human input methods in new directions but can also be utilized in various fields, such as contactless measurement and educational devices.
Motion recognition technology enables the use of human movements—which were difficult for keyboards and mice to capture—as a new form of input data. Of course, keyboards and mice are still used as important input devices today due to their accuracy and convenience, and it is difficult to say that motion recognition technology has completely replaced them. Hand gesture recognition also faces challenges that need to be addressed, such as lighting conditions, obstructions, user-specific variations, recognition accuracy, and processing speed. However, as motion recognition technology continues to advance, the ways in which humans exchange information with computers will become far more diverse than they are today. Moving beyond input methods that rely on keyboards and mice, natural human movements—such as hand gestures, body language, and eye movements—could themselves become new means of input. It will be interesting to see what role motion recognition technology will play as a new way of connecting humans and machines in the future development of civilization.
