Virtual keyboards are a frequent source of frustration for augmented reality (AR) users. The virtual surfaces are slow and error prone, and raising an arm to type on them can cause muscle strain known as “gorilla arm.”
Researchers at the University of Texas at Dallas have developed a novel AR interface that enhances virtual typing by turning ordinary items into functional typing surfaces.
AR System Turns Everyday Objects Into Keyboards
The new system, called PropType and currently patent-pending, projects an augmented keyboard onto handheld objects and even works on curved shapes.
Student developers released a demo video showing PropType in action on items like water bottles, coffee mugs, books and soda cans.
“By using objects that are already part of a user’s environment, PropType transforms how text is entered in AR, blending the physical and virtual worlds more naturally,” said Dr. Jin Ryong Kim, assistant professor of computer science at the Erik Jonsson School of Engineering and Computer Science. “The system takes advantage of the natural tactile feel of these objects, improving key recognition and reducing the need for users to look closely at the keys.”
The approach offers a new option compared to typical AR typing methods, such as external keyboards, which can break the sense of an immersive, device-free experience.
“PropType taps into people’s familiarity with everyday handheld items to deliver a more intuitive and user-friendly typing method, especially in mobile or hands-free contexts where standard keyboards are not practical,” Kim added.
Researchers Showcase PropType at Top Global HCI Events
In April, researchers from Kim’s Multimodal Interaction Lab introduced PropType at the ACM CHI Conference on Human Factors in Computing Systems—the leading global event in human-computer interaction—where their work earned a Best Paper Honorable Mention. The team later presented the project again in September at the 38th Annual ACM Symposium on User Interface Software and Technology in Busan, South Korea.
Developing PropType came with obstacles, especially when adjusting the interface to fit objects of various shapes and sizes. The team studied 16 participants’ grip and typing behaviors on different props. Based on these insights, they created specialized keyboard layouts tailored to each object. They also built an editing tool that lets users personalize the keyboard design and visual effects.
Kim’s work centers on haptics—the study of touch-based and physical feedback in digital environments, such as vibration or temperature changes.
“Touch conveys a rich range of information; it’s an important yet still underexplored communication channel in virtual and augmented reality,” he said.
Award-Winning Research on Thermal–Tactile Interaction
Kim and his students also won the Best Demo Award at the 2025 IEEE World Haptics Conference and an Honorable Mention at the 2024 IEEE Haptics Symposium for their work on thermal masking and thermal–tactile integration.
Thermal masking is a sensory illusion where the brain feels heat or cold in one spot even though the source is elsewhere. For example, researchers might place a heat actuator on a participant’s arm and a tactile actuator a few inches away. When both devices are activated together, the brain interprets the warmth as coming from both positions.
The team presented their findings at the 2024 ACM CHI Conference and created a demo video of the technology.
Kim said, “We aim to design more immersive interfaces by showing how touch can influence or override thermal cues.” “This research could make VR, medical training, and other applications feel almost real.”
Read the original article on: Tech Xplore
Read more: A Multi-Function Mimicking Neuron Moves Robots Closer to Human-Like Abilities