2.4. Haptic-temperature fusion and pattern recognition
Multisensory neurons in the superior colliculus of the midbrain are
revealed to directly integrate spikes from different senses to initiate
a neuronal response to multimodal environmental
events.[49-51] For example, when two sensory
stimuli appear at the same time, human generally shows higher
sensitivity than the cases when the stimuli appear separately. This
cognitive ability raises awareness and helps human make the right
choices. Since the abovementioned sensory neuron is capable of detecting
and encoding pressure and temperature inputs based on the voltage
dividing effect and the intrinsic thermal sensitivity of
VO2, respectively, it offers a potential platform for
fusing and integrating tactile and temperature information directly in
the same component, as shown in Figure 6 a. The piezoresistive
sensor provides the haptic perception while the temperature sensitivity
of VO2 offers a mechanism for temperature perception,
and both sensory inputs are converted into the oscillation frequency and
amplitude of VO2 neuron, therefore achieving
multisensory perception (the entire testing system is shown in Figure
S13, Supporting Information). Indeed, experimental results in Figure 6b
demonstrate that the VO2 sensory neuron can integrate
haptic and temperature information. As examples, we use (100 g, 27 °C)
to indicate an empty cup, (200 g, 17 °C) to indicate a cup with cold
water, and (200 g, 32 °C) to indicate a cup with warm water. Figure 6b
shows that the abovementioned cases exhibit oscillation frequency of 0.5
MHz, 0.6 MHz and 1.1 MHz, respectively, showing the cross-modal
perception capability of the VO2 neuron. The actual
multisensory data collection processes are displayed in Supplementary
Video 4-6. Although prior works have reported mapping of input signals
into output frequencies, most of the works only realizes single mode
perception. Here we take advantage of the intrinsic thermal sensitivity
of metal-insulator transition in VO2 and its combination
with a pressure sensor has led to multiple sensory perception.