Powered by KikBooks Widget

By on May 9, 2011, with 105 Comments

McGurk Effect Audio Video IllusionNow here is a great find, Dylan pointed out to me. I’m not sure if you’ve heard of the McGurk Effect before (I haven’t), but in short, it’s a perceptual phenomenon which demonstrates an interaction between hearing and vision in speech perception. In practice, this audio-video optical illusion effect may be experienced when a video of one phoneme’s production is dubbed with a sound-recording of a different phoneme being spoken.

Perhaps, it might be best if I just let you watch this short, yet professional video BBC produced. It explains the whole deal, and includes some real-life examples of McGurk effect in action. Have we just “discovered” a whole new field of optical illusions to be showcased on this site? Could be… BTW, it might take some time before the video loads. It’s somewhat large, comparing to our previous video illusions!

Comments

105 Responses
  1. genie says:

    Thats freaky!

    • Ed says:

      it is so funny.. when you cross your eyes making the 2 faces melt to a middle one when they are side by side, it works the other way around as well. I can’t get the middle picture to show the F movements of the mouth that way, while normally when overlapping 2 images I can switch to either one of them.

  2. bluepool says:

    OMG cool! ^^

  3. ohmygosh says:

    cool

  4. some random guy you'l never meet says:

    that is freaky!

  5. hysteria says:

    Wow! My ears are deceiving me!

  6. Myname says:

    This doesn’t really work for me, I hear pah pah pah all the time.

  7. Kaerlion says:

    Wow!

    That – is – a – ma – zing!

  8. Sushi says:

    Thats is so cool!

  9. Janelle McDermott says:

    This messes with my mind

  10. Mars says:

    wowow…. thats really interesting actually! And I am not saying that sarcastically. I mean it is generally interesting!!!

  11. Dew Silverdrop says:

    SSSSSOOOOOO CCOOOOOOOOOOOOL! very confusing! a great optical illusion! ps first comment!!!

    • mike says:

      yours wasnt the first comment
      it was mine or maybe someone elses
      mine didnt show up because it was waiting ”moderation”

  12. roarshack says:

    holy crap! does this work for distance perception too? when they showed him at different distances the lady was talking so i couldnt tell..

    this is really interesting ‘cuz i’m studying psychology right now.

    they really did an excellent job of explaining sensation and perception.. something my class couldnt do for 2 weeks xD

  13. Nixygirl says:

    Very very cool!!! More plz :)

  14. Kimon Frousios says:

    That’s brilliant!

    The sounds for “P”, “V” and “F” form the group of consonants produced by the lips. There are other groups of related consonants as well.
    If you look at related languages/dialects, you’ll see that sounds from the same group are often interchangeable.

    I wonder if the illusion would work with a consonant outside the P/V/F group, like L.

  15. Katie says:

    I’ve seen that entire documentary. It’s really amazing, you guys should all check it out if you’re into documentaries! :)

  16. Andrea says:

    Awesome! Even when you know it’s the same sound, you still hear it differently!
    For me, as a speech-therapist, this is incredibly interesting, keep ‘m coming!

  17. margaret marker says:

    I heard immediately that it was ba ba ba and then changed to fa fa fa . did not see this illusion..

    • Dylan says:

      You did see the illusion- because it DOSNT change to FA FA FA- the only sound he ever makes in every shot is BA. It simply looks like he is saying FA and that overrides what your mind hears.

    • Maggie says:

      Ohhhh I see now.. or I hear now.. closed my eyes and the sound stayed the same.. open them and the sound changes.. cool

  18. Care Bear says:

    That was awesome! It’s another reason I don’t believe anything I hear and only half of what I see.

  19. moe says:

    it doesnt work on me!!!!!!!!!!!!!!!!!!
    In the second clip, i hear ‘faa’!
    is that normal?

    • Dylan says:

      If you hear a FA then it worked lol- he isnt saying FA- ever- its all BA. his mouth movements look like FA but the only sound playing is BA- on every shot.

  20. William Lim 'nkm' says:

    cool !!!, its heard like he said “ba” first, but next picture I heard he said “fa”, altough its was a same sound

  21. fossda says:

    ive seen this illusion before! its amazing!

  22. Amazed says:

    WHOAH! That’s spooky O.o

    (btw, first?)

  23. Steve says:

    Brilliant! This illusion has been one of my favorites since I first heard of the effect a couple years ago.

  24. Someone says:

    I don’t get it. It sounds exactly the same. At first I thought the illusion was the opposite of what it really was. I had to force hearing the F sound.

  25. Macy says:

    WOW THAT IS AMAZING

  26. josephine says:

    When I try really hard, I’m able to get like one or two bas during a fa

  27. U.N. Owen says:

    Wow! I’ve been coming to MOI for about 2 years (I just got an iPad, and am getting your app next week), and whenever I come here, I get stuck – looking at illusion-after-illusion.

    This is (for me) the first time I’ve learned something I’d never heard of before. It is bizzare, to say the least.

    I wonder if this McGurk effect can open a new door here – one to audio illusions. As I said, never heard of this effect, but, it’s fascinating.

    Thanks for sharing this.

  28. SDER says:

    Wow, that is pretty cool. No matter how much I tried, if I looked at the guy, all I could hear was “va” but if I closed my eyes, I would only hear the same sound over and over again. It never ceases to amaze me how our brain works. Thanks again for this wonderful illusion.

  29. Typeaux says:

    Amazing. I could not make myself hear “BAH” when it was visually clear that the mouth was saying “FAH,” even when I knew that the sounds I was hearing were identical. This is a very powerful effect. I only wonder what *practical* use it has in the real world and not in a study of effects.

  30. Phillip says:

    Huh? What’s the illusion? He said bah and then he said fah? It’s hardly misheard lyrics. Like “i’m not talking about millennium” of “”Excuse me, while I kiss this guy”.

    • Dylan says:

      He he- nope, he didnt say FA- that’s the point- he was still saying FA- so obviously the illusion worked very well :)

  31. notarealname says:

    I CAN DIFFER AND it sais fbaa fbaa wen he sais it at the same time the diferent pics
    but i can differ

  32. Anbonymous says:

    … Very nice. I had heard of this, but had forgotten what it was called.

  33. Eric says:

    Trippy

  34. Yusif Azari says:

    AWESOME!!!!!!!

    I closed my eyes and the sound changed

  35. Jackal says:

    That’s really awesome!! I can’t believe that our vision makes such a difference on our audio perception.

  36. jazz says:

    I swear he is actually saying “FA” and their just messing with us

    • SDER says:

      If you close your eyes while he says “fa” you’ll notice that he actually does say “ba”.

  37. shawn says:

    BAAAAAAAAAAAAAAAAAAA!!!!!!!!!

  38. Jeffrey says:

    I’ll be interested to see this taken to a new level. Right now, I’m mildly intrigued, but I didn’t think it was all that exciting.

  39. Chris says:

    I thought it was stupid. She said the picture changed but you can still hear “ba”. I was like, “BA?! I hear freakin’ ‘VA’.” Then after a while I got what they meant. Cool illusion.

  40. Mark Fain says:

    this one is pretty sweet.

  41. wandering_goat says:

    This is the first time the effect’s worked on me, but even so, I can only sometimes hear “fa”. It’s almost always “ba”, especially if I’m focusing on the video. I was trained in music, so possibly that makes a difference in how well it works.

  42. faith says:

    awesome

  43. morgan rowley says:

    hi

  44. morgan rowley says:

    its true you hear ba instead of va if you close your eyes but it goes back to va if you open them again.weird.

  45. me123987645 says:

    that is so cool! i closed my eyes when he started saying (what seemed to be) fa and i heard ba. this is a really cool illusion.

  46. kimbo says:

    OMG! THIS IS SOOOOOO COOL!!!!

  47. Ryan St. George says:

    Eye thing eye here “Baw” two! (Welcome to my world.)

  48. brad says:

    Heh, love this!

    Is the audio at the beginning of the video the same as at the end? the audio sample i mean with the theme park rides and screaming?

  49. miles says:

    thats fake close your eyes and listen

    • Dylan says:

      Sorry, but you probably didnt understand because its in no way faked- because you can control which you hear yourself by looking when you chose at one side or the other- or indeed closing your eyes when looking at the left image to stop hearing FA, and hear the correct BA whenever you choose to. And that IS the illusion.

  50. Worldy Guy says:

    Instead of “fa” I hear “va”. But that could be because I know a lot of Yiddish speakers.

  51. Adam Merrill says:

    Wow, this is great! I’d looked at this effect before, but this is the first one that actually worked! As an interesting sidenote, I noticed that if you look at the center of the two images (with one face saying “bah” and the other saying “fa” at the same time) and kind of view them both with your peripheral vision, it sounds like two people sboth saying each sound, and one (the “fa” I think) is slightly delayed! That’s amazing!!!

  52. DanG says:

    wow…….. not just neat… but scary awesome. Black magic or something.

  53. Dallas Valerian says:

    Wandering Goat’s mention of music’s influencing what is heard is funny as “fa.” Do, re, mi will do that I guess.

    My experience made me here “va” as the alternate, because I’ve noticed in Spanish speakers among South Americans that V as in Victor is pronounced Bictor as though it’s a B.

    A friend who spoke also fluent French confirmed this for me, asserting that in French his V was distinct from B but that in English, they were the same sound. Further, the confusion only goes in one direction. Not only does he say voulez, and not boulez, nobody ever asks the way either to el vaño or to the vathroom.

  54. Padma says:

    I do not get this at all!

    • fossda says:

      listen to the voiceover in the video

    • Rich Pro says:

      If you look at your right of the screen, it is saying baa but if you look at your left of the screen, it looks like its saying faa,However, if you close your eyes,you hear baa again.(They just changed the picture. The picture,but not the sound.)

    • Suor de Sapo says:

      Me too! I hear the same on both…Bahh!!!

  55. diggo says:

    It’s not all that dissimilar to the psychoacoustic phenomena used when designing lossy audio codecs such as MP3. Watch Poppy Crum’s segment in this video:
    http://www.youtube.com/watch?v=BYTlN6wjcvQ

  56. I love this! I saw the whole BBC show when it was on and was so amazed that I went to the website to show my husband and watched it over again!

  57. Zenbot says:

    In linguistics the only difference between an “f” and a “v” is that the “v” is vocalized, meaning you actually emit a sound from your throat when you say “vat” vs. just air from your throat when you say “fat.” Otherwise, the placement of the tongue and teeth against the lips are basically identical. I am not surprised some of you may perceive a “va” sound instead of a “fa” sound. I have also been told that when people are training to read lips they have to use a lot of contextual interpretation for issues like this, such as knowing the person probably wasn’t saying “It’s fairy nice to meet you.”

  58. Dylan says:

    Glad you found this video cool- I knew you and everyone here would like it if you’d not seen it before. I have noted that if you dont focus enough on the left side on the split screen shots, it will muddy the FA closer to the real BA.

  59. Jeannine says:

    this is one of the coolest illusions I have ever seen – I had to immediately email it to my friends! Almost scary…

  60. B_man says:

    HOLY CRAP!!!! THATS COOL!!

  61. sally says:

    Osama….obama….osama…..obama….osama….obama….Cool! Almost the same.

  62. wow says:

    try saying island view in the mirror silently..
    crazy how our minds work

  63. Eric says:

    Brendan, never get a tattoo

  64. Glen says:

    The most interesting part about this is stated in the video: No matter how much you know about it, it still works. Almost all other illusions go away once you know the mechanics behind it.

  65. Kyle says:

    Not exactly an illusion. I just spent a Semester on the Science of Speech sounds, it would take a while to explain what exactly is happening but essentially its a combination of human articulation sloppyness (lack of Invariance) and our brain trying to make sense of things contextually. Top Down preprocessing in action

  66. Bill says:

    If your teachers had done a better job of teaching you pronunciation, there would be no effect.

  67. Michael Carabini says:

    .This video demonstrates how looking at someones mouth movements affects the way we hear what they are saying. The man in the video is saying bah bah bah but when the same audio recording is played while he mouths out fah fah fah it sounds like he is saying fah fah fah. ..

  68. Penny says:

    Close your eyes people!!

  69. rajat says:

    stupid illusions!!!!!!!!!!!!!!!!!!!!!!!

  70. Meghan says:

    The narrator doesn’t stop talking long enough for me to hear it. I only heard ba. Also, I knew that the noise was ba even when the image changed, and I still heard ba. Maybe if I wasn’t being told about this effect the results would have been different.

  71. the man says:

    i didnt get any thing….wait!!! i think he is calling his father

  72. ASSFACE44 says:

    ID LIKE TO SEE THIS WITH THE SOUND SH BECAUSE BA AND VA ARE TOO SIMILAR.ITS COOL BUT A LITTLE SUSPECT

  73. dan says:

    if you close your eyes on the VA one…it will say BA. but what you see is what you hear. so if you don’t look you will hear correctly. when you close your eyes your hearing increases.

  74. EgRoJ says:

    Incredible, I saw the video without sound, I guess the word is “PAAAAM”, and the second “BIIIIRD”. lol

  75. jgfhjgf says:

    the narrator was too much talking!

  76. tiny says:

    It doesn’t work for me. I know that I am supposed to hear first ‘bah-bah’ and then ‘fah-fah’, but I keep hearing ‘bah’, even when I consciously try to hear ‘fah’.

  77. C-c-c-combo says:

    So what if you hear Bah? I figured out you can make a good enough -b- sound while still looking like you’re saying -f-. Since I figured that out all I can hear is bah.

  78. saman says:

    good web

  79. Zeb says:

    I think it would have looked better with a sexy girl’s lips doing it. When children are in speach class they are taught to look at the taechers lips to see the sound she is making, or some people learn to read lips. the sounds are almost the same, when you have one of those ABC toys that say the letter V and B sound the same.

  80. jake says:

    I heard the Yamaha “I wanna ride” ad, and silently I moved my mouth to say “I wanna Fly”

    then it sounded like the ad was saying “i wanna fly”

    COOL!

  81. Andy says:

    And this guy is a PROFESSOR?… It’s obvious for every phonetician that plosive consonants make unique “shock waves” recognized by our hearing, as opposed to the VAH. That’s not an illusion.

  82. Kip says:

    I find the narrator speaks only when he’s doing the Fah and never the Bah. Even if I don’t watch the video I hear an F. I’m not sure if it’s because I’ve already seen it, but it’s generally when the narrator has spoken over the other sound.

  83. Shane says:

    I’m deaf in one ear, so the hearing in my good ear is heightened, so it didn’t work for me. Although the theory is there, the practice didn’t work. It’s very similar to a prank my dad used to do on me. I’d be eating an orange or banana and he would come over and put an onion under my nose. after that, everything I ate tasted like onions. THANKS DAD!

  84. AmetuerAspirant says:

    this is just baffling, though after most of the video, and concentrating very fiercely on the sound while watching, i was actually able to make the “vah” mouth make a sound vaguely b’ish. then the narrator started talking over and i couldn’t hear the sound well enough to do it. thank you a lot annoying British narrator…

  85. Dem says:

    THIS WORKED FOR ME. I CLOSED AND OPENED MY EYES REPEATEDLY AFTER HE DID BAH WITH THE F ILLUSION.

    HOLY SHIET.

  86. Joex says:

    Reminds me of the seagulls in finding nemo…. lol…

  87. zoakera says:

    Why cant i download or hear it using my mobile phone???
    Can anybody helppp
    plzzzzzz. . . . .

  88. L.J says:

    I heard the “Vah” thing from the start with my eyes open. I have keen hearing and I had to lean Spanish. I’ve got used to “b” and “v” sounds.

  89. Natalia says:

    This seems so silly to me. The only example they could show was “ba” vs. “va,” which are such similar sounds anyway. If one’s teeth are close together and make a good seal, and one tries to make a sudden or hard “v,” the way the sounds are made, and thus the sounds too, are almost identical. Of course, you are going to choose the one it looks like he’s saying. Sheesh. They should show a lot more examples if they are going to use this to illustrate a whole concept.

  90. Chloe says:

    I can hear ‘PA’ and then ‘PVAH’ — ?!?

  91. Tommy G Starfire says:

    Have you any wool?

Speak Your Mind

You can add some images too.

Pinterest