UltrasonicSuperHearing
Plug-in description for the UltrasonicSuperHearing.
Plug-in description
Ultrasonic sound sources are inaudible to humans. While there do exist devices that can capture, pitch shift and reproduce ultrasonic sound sources for humans to listen to, they generally only employ a single microphone and loudspeaker. Therefore, the listener is unable to localise where these pitch shifted ultrasonic sound sources are located.
However, by instead capturing the ultrasonic sound sources using an array of microphones, it is possible to estimate their direction of arrival (DoA), and use this information to spatialise a pitch shifted signal in this direction. Therefore, not only is the listener able to perceive these ultrasonic sound sources, but they are also able to localise them in the correct direction.
This plug-in requires 192kHz sampling rate and relies on constructing the ultrasonic microphone array described below. Although, it is noted that the source code could be altered to support other arrays without too much difficulty.
Building a suitable ultrasonic microphone array
The CAD files and drawings used for 3D printing the 6 sensor proof-of-concept array, which was employed for the study detailed in [1], can be found in the companion git repository. Further details regarding its construction can also be found in the paper [1].
A recent portable alternative hardware solution is also detailed in [2].
Auralising the captured ultrasonic sound sources
The audio plugin renders the 6 microphone signals to the 2 binaural channels. It employs the DoA estimation, pitch-shifting, and binauralisation, as described and used for the listening tests conducted in [1].
Demo
The proposed ultrasonic super-hearing system is demonstrated in the following video:
(Please wear headphones)
About the authors
- Leo McCormack: a doctoral candidate at Aalto University.
- Ville Pulkki: Professor at Aalto University, known for VBAP, SIRR, DirAC and eccentric behaviour.
- Raimundo Gonzalez: a doctoral candidate at Aalto University.
- Ossi Miikkulainen: a MSc. student at Aalto University.
License
This application may be used for academic, personal, and/or commercial use. The source code may also be used for commercial purposes, provided that the terms of the GPLv3 license are fulfilled. This requires that the original code and/or any derived works must also be open-sourced and made available under the same GPLv3 license, if it is to be used for commercial purposes.
References
[1] Pulkki, V., McCormack, L. and Gonzalez, R. 2021. Superhuman spatial hearing technology for ultrasonic frequencies. Scientific Reports 11, 11608 (2021). https://doi.org/10.1038/s41598-021-90829-9
[2] Pulkki, V., Miikkulainen, O., and McCormack, L. 2022. Portable Spatial Super-Hearing Technology For Ultrasound Range. in 24th International Congress on Acoustics (ICA).