The immersive experience allowed by VR headsets holds much promise for hearing assessment within more realistic listening environments than are usually found in audiological tests - ones that include visual information, spatialised sound and everyday settings. However, realising the promise of VR-based assessment requires both new audiovisual materials compatible with the headsets and software tools for composition and running of tests. Existing resources are still rather limited in scope. Hear-VR is a new corpus of audiovisual materials designed for intelligibility testing in VR.
British English matrix sentences were recorded in our anechoic chamber against a green screen, using a 360° camera and a high-quality condenser microphone. These sentences contain 5 slots with 10 choices in each slot. Different sets of 200 sentences were recorded by 5 male and 5 female talkers sitting at a table. The individual talkers have been cropped from the videos and the green screen and table replaced by a transparent background to allow compositing of speakers in new 360° scenes. The individual matrix sentences have been extracted as 11s videos together with level-normalised monophonic audio. We have developed scripts that build multi-talker videos from these elements in a format that can be played on VR Headsets such as the Oculus 2. We have also developed a Unity application for the headsets, which will play the 360° composite videos and create spatialised sound sources for the talkers together with background noise delivered from multiple virtual loudspeakers.
The HearVR database is very large, with approximately 30GB of files per talker. The only practical means for dissemination is by mail, using a 500GB USB storage device.
Please contact Mark Huckvale for mailing instructions.
The work described here was supported in part by the UK Engineering and Physical Sciences Research Council [grants: EP/S03580X/1 and EP/S035842/1]
M.Huckvale, G.Hilkhuysen, T.Green, "Video database for intelligibility testing in Virtual Reality", SPIN Workshop 2023, Split, Croatia.