Ear-like Auditory Technology
Our auditory technology was conceived by John Bates, who has parlayed his twenty years of experience in system design and testing airborne electronic intercept systems into a new model of auditory perception. His early work involved acquiring and processing the mixture of intercept information of a radar environment. Analysis of the data also involved dealing with human perception factors.
Bates, an electrical engineer, saw that the complexities of intercept analysis strongly resembled the unsolved cocktail party effect that remains today as the “holy grail” problem of auditory perception. In both radar interception and human audition, the object is to discover, separate, and understand meaningful patterns in perceived information. His interest in radar’s similarity to the cocktail party effect eventually led him to transfer into the auditory domain the technical concepts he had used successfully in the electromagnetic domain.
For example, he had solved a major problem by inventing a way to instantaneously separate the information from intermixed radar pulse trains. Logically, this is the same as the ear’s ability to detect and track human voice pitch. So he applied his invention, the periodicity sorting matrix (PSM), to the problem with great success. No one has yet found a better way to do this.
Despite this success, he realized that his radically different auditory theory needed additional credibility to get support. So he carried out experiments using the PSM to show that time domain methods had solutions for auditory anomalies that had baffled auditory researchers since the 19th century.
For example, he showed that the PSM could detect the deep pitch of men’s voices on the telephone even though conventional theory says that it has been removed in transmission. Similarly, like the ear, it detects the low notes of organ music from tinny little computer loudspeakers. In scientific psychoacoustics he ran experiments in two-tone interference that showed how the ear can hear “combination tones” and could be the source of the mysterious “critical band.”
Now, consider how a realistic auditory model should be built. First, it should be have a processing algorithm that is compatible with nature’s neuron-powered methods. And it should have instantaneous awareness of the entire environment. But at the same time, it should be able to select any source for attention. Our processing method does these things by translating the sound waveform, with its multiple sound sources and reverberations, into a sequential stream of fine-grained particles called “Waveform Information Vectors” (WIVs.)
Each waveform cycle, determined by its zero crossings, is treated as an independent event and is encoded as a WIV by its fundamental dimensions of time and energy. Thus, each WIV particle independently carries information of its source and, because of its short duration, makes it possible to extract its information separately from other WIVs. When used binaurally a spatial vector dimension provides its direction of arrival.
Meaning is extracted from the stream of WIVs by both learning and recognizing patterns in the time sequences. This is done by a unique instantaneous pattern recognizer that is based on the same successful real-time method used by the PSM. Thus, all WIV processing is locked, or synchronized with the input waveform and will output a result each time a WIV particle enters the system.
Physically, our WIV processor is structured as an analog computer that operates solely by using a hierarchy of matrix-oriented logic decisions. It does not use arithmetic computations. This means that the WIV algorithm should be capable of ear-like efficiency. Moreover, it could probably be burned into a microchip that could have ear-like size and power consumption.
We are confident that ultimately the WIV method, by directly decoding and parsing the acoustic waveform should make it possible to build a system that could implement and use the many aspects of auditory perception such as robust speech recognition. And ultimately, perhaps, build a working system that solves the mystery of the cocktail party effect.