Machine Learning Experts Have Found A Way To Edit Videos Of People Saying Words They've Never Said!
240,900
DESCRIPTION
Abstract:
Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis.
Ethical Considerations:
Our text-based editing approach lays the foundation for better editing tools for movie post production. Filmed dialogue scenes often require re-timing or editing based on small script changes, which currently requires tedious manual work. Our editing technique also enables easy adaptation of audio-visual video content to specific target audiences: e.g., instruction videos can be fine-tuned to audiences of different backgrounds, or a storyteller video can be adapted to children of different age groups purely based on textual script edits. In short, our work was developed for storytelling purposes.
However, the availability of such technology — at a quality that some might find indistinguishable from source material. Posted By Ghost
nextvideos
Work Smarter Not Harder: Dude Found A Way To Cheat The Claw Machine!
154,557
"I'm Going To Jail" Dude Said He Found A Way To Get Free Gas!
187,535
Ray J Confirms Sex Tape With Kim Kardashian Was A Deal Orchestrated By Her Mother Kris Jenner!
173,827
Georgia Football Trainer Gets Exposed For Being Racist ASF!
110,002
SMH: Couple Gave A Child Up After Learning They Can’t Include Them In Their Videos!
214,305
These Folks Found A Way To Gentrify Prison!
155,547
People Are Saying This Video Is How They Found 6ix9ine At The Gym!
177,963
People Will Find A Way To Whip Anything Moving... Never Seen A Fish Get Whooped In My Whole Entire Life!
135,991
22 Dead & Over 60 Injured In Mass Shooting In Maine By An Army Reservist With Documented Mental Issues... Still At Large!
179,801
Yung Dred - Found A Way
3,231,654
Somebody Getting Punished For This: 9,000 Pounds Of Cocaine Found In Semi-Submersible During Rescue Search Off Colombia!
54,763
‘World’s First’ Cloned Arctic Wolf Bred In A Chinese Lab!
82,885
The Size Of The Rim Tho: Korean Players Found A Way to Stop Steph Curry!
67,332
Quick Thinging Chick Found A Unique Way To Stop A Robber!
143,462
Piers Morgan Tells Kanye He's Being Racist After Saying Jewish Business People Have Raped His People!
75,730
Goals Or Nah? People Said They Would Never Work Out Because He's 5'2!
238,296
Dog Owner Solves The Mystery Of How Dog Was Able To Escape From Barred Gate!
144,434
Coolio Was Invited By Students To Hang Out After A Show And He Delivered This Powerful Version Of 'Gangsta's Paradise'
102,843
Bro Hit Harder Than Tyson: This Punch Machine Never Stood A Chance!
79,408
Agree Or Nah? Fans Are Speculating Eminem Is Taking A Shot At Kendrick Lamar On The Intro Of His Album!
72,867
The Internet Never Forgets: Clip Resurfaces Of Diddy Saying This To Meek Mill While At A Pool Party!
401,829
He Bout To Come Up Missing: Man Drains Family Bank Account, Throws $200k In Cash Out Of The Car Window On An Oregon Highway!
73,400
Jamaican Legend 'Sizzla' Burns DJ Khaled’s Platinum Plaques! "Yuh Insult Me DJ Khaled'
144,998
Spoke His Mind: Stephen A. Smith On Jake Paul's Boxing Career! "You're Not Fighting Boxers"
124,228
U.S. Coast Says Missing Submersible Imploded Near Titanic Wreckage!
139,306
Naw This Edit Is A Violation... Disrespecting Izzy After His Loss To Strickland!
95,138
Flexing Gone Wrong: Dude Made It Seem Like He Hopped Out The Chopper But They Forgot To Edit It The Right Way!
191,552
You Be The Judge: Man Found Dead After Uploading Videos Of Alleged Giant He Spotted On Top Of A Mountain In Canada!
370,071
The Future Scary: Engineered Arts Unveils Ameca Humanoid Robot!
154,051
Entrepreneur In The Making: Little Dude Found A Way To Hustle While Having The Time Of His Life!
65,613
"His Own People Said He's Never Made A Good Decision" Donald Trump Calls Out Biden & Global Elites!
64,829
Very Sus: This Is How They're Testing For Coronavirus Now In Some Hospitals In China!
241,679
Oh Nah: Man Suspected Of Cannibalism Detained At Airport After Police Found "Pieces Of Meat" In His Suitcase!
59,961
This Is Just Madness: Dude Found A Unique Way To Get Ultra High!
80,187
She Found A Unique Way To Get Revenge On Her Man For Cheating!
401,003
Lil Durk Look-A-Like "Perkio" Responds To 6ix9ine! "They Set Me Up, I Didn't Know 6ix9ine Would Be There"
113,337
If Men Acted Like 'Fitness Influencers' At The Gym!
123,406
Thoughts? Dude Speaks On People Going Out Of Their Way To Show Other People What Kind Of Person They Are!
181,491
So Casual With It: Boosters Walk Out With Trash Bags Full Of Merchandise From Work World In Sacramento, California!
58,547
People Are Saying Kanye Was Right After His Daughter Dressed Up Like Ice Spice For Tiktok!
186,692