This artificial intelligence turns a movie into a comic to read on paper
It’s time to change the tagline “I better wait for the movie”, when you don’t want to read a book or a comic. Now we can say “I better wait for the comic”, if we don’t want to see the movie …
Comics and movies or television often compete. There is no time for everything and sometimes you have to choose to read comics, or watch series and movies. Many comics Y sleeves they have been taken to the movies, and vice versa. But here we have an artificial intelligence that turns a movie or series into a comic. Actually, it works with any type of video.
Engineers and programmers from Dalian University of Technology in China, and Hong Kong University City, They have developed an artificial intelligence that is capable of converting moving images and dialogues into still photos and text bubbles that tell the story.
The only condition for the AI to work is that the video has subtitles, since he is not able to process the voice, but he is able to understand texts.
As we see in this scheme, the AI analyzes the image and the subtitles and creates a summary of the plot, extracting the most relevant frames and dialogue for the story.
Then group the images according to the scene, and crop faces or select panoramic shots, to create vignettes of different sizes of each scene, which can be interspersed with others.
Depending on the size of each group, choose the bullet template for each page, alternating close-ups and more open shots.
Finally, artificial intelligence adds the speech bubbles with the texts.
There are several comic generators quality in the market, which even perform professional work. But the contribution of this software is that add the emotion to the dialogues. Depending on the tone of voice of the audio, it knows if a text is expressed with anger, joy, fear, etc., and reflects it in the comic using speech bubbles of different sizes, and larger or smaller font sizes.
Despite everything, it is not perfect. Their creators recognize that sometimes creates vignettes very similar to each other, and the comic is somewhat monotonous. But they continue to refine it, because their intention is that content creators can use it to create comics and manga automatically.