Haptic-temperature fusion and pattern recognition
Multisensory neurons in the superior colliculus of the midbrain are revealed to directly integrate spikes from different senses to initiate a neuronal response to multimodal environmental events.[49-51] For example, when two sensory stimuli appear at the same time, human generally shows higher sensitivity than the cases when the stimuli appear separately. This cognitive ability raises awareness and helps human make the right choices. Since the abovementioned sensory neuron is capable of detecting and encoding pressure and temperature inputs based on the voltage dividing effect and the intrinsic thermal sensitivity of VO2, respectively, it offers a potential platform for fusing and integrating tactile and temperature information directly in the same component, as shown in Figure 6a. The piezoresistive sensor provides the haptic perception while the temperature sensitivity of VO2 offers a mechanism for temperature perception, and both sensory inputs are converted into the oscillation frequency and amplitude of VO2 neuron, therefore achieving multisensory perception (the entire testing system is shown in Figure S13, Supporting Information). Indeed, experimental results in Figure 6b demonstrate that the VO2 sensory neuron can integrate haptic and temperature information. As examples, we use (100 g, 27 °C) to indicate an empty cup, (200 g, 17 °C) to indicate a cup with cold water, and (200 g, 32 °C) to indicate a cup with warm water. Figure 6b shows that the abovementioned cases exhibit oscillation frequency of 0.5 MHz, 0.6 MHz and 1.1 MHz, respectively, showing the cross-modal perception capability of the VO2 neuron. The actual multisensory data collection processes are displayed in Supplementary Video 4-6. Although prior works have reported mapping of input signals into output frequencies, most of the works only realizes single mode perception. Here we take advantage of the intrinsic thermal sensitivity of metal-insulator transition in VO2 and its combination with a pressure sensor has led to multiple sensory perception.