$\"Runway$ <\/p>\n<\/p><\/div>\n

\n<\/p>\n

Runway has shouldered aside Midjourney and Stable Diffusion, introducing the first clips of text-to-video AI art that the company says is completely generated by a text prompt.<\/p>\n

The company said that it\u2019s offering a waitlist to join what it calls \u201cGen 2\u201d of text-to-video AI, after offering a similar waitlist for its first, simpler text-to-video tools that use a real-world scene as a model.<\/p>\n

When AI art emerged last year<\/a>, it used a text-to-image model. A user would input a text prompt describing the scene, and the tool would attempt to create an image using what it knew of real-world \u201cseeds,\u201d artistic styles and so forth. Services like Midjourney<\/a> perform these tasks on a cloud server, while Stable Diffusion<\/a> and Stable Horde<\/a> take advantage of similar AI models running on home PCs. <\/p>\n

Text-to-video, however, is the next step. There are various ways of accomplishing this: Pollinations.ai has accumulated a few models which you can try out, simply takes a few related scenes and constructs an animation stringing them together. Another and allows you to zoom around. <\/p>\n

Runway takes a different approach. The company already offers : inpainting to remove objects from a video (as opposed to an image), AI-powered bokeh, transcripts and subtitles, and more. The first generation of its text-to-video tools allowed you to construct a real-world scene, then use it as a model to overlay a text-generated video on top of it. This is normally done as an image, where you could take a photo of a Golden Retriever and use AI to transform the photo into a photo of a Doberman, for example.<\/p>\n

That was Gen 1. Runway\u2019s Gen 2, as the company tweeted, can use existing images or videos as a base. But the technology can also completely auto-generate a short video clip from a text prompt and nothing more.<\/p>\n

<\/figure>\n

As Runway\u2019s tweet indicates, the clips are both short (just a few seconds at most), awfully grainy, and suffers from a low frame rate. It\u2019s not clear when Runway will release the model for early access or general access, either. But the examples on the Runway Gen 2 page do show a wide variety of video prompts: pure text-to-video AI, text+image to video, and so on. It appears that the more input you give the model, the better your luck. Applying a video \u201coverlay\u201d over an existing object or scene seemed to offer the smoothest video and highest resolution.<\/p>\n

Runway already offers a $12\/mo \u201cStandard\u201d plan that allows for unlimited video projects. But certain tools, such as actually training your own portrait or animal generator, require an additional $10 fee. It\u2019s unclear what Runway will charge for its new model.<\/p>\n

What Runway does <\/em>demonstrate, however, is that in a few short months, we\u2019ve moved from text-to-image AI art into text-to-video AI art\u2026 and all we can do is shake our heads in amazement. <\/p>\n

<\/body><\/div>\n

\n
\n\t\t
\n\t\tAuthor: Mark Hachman<\/a>, Senior Editor\t\t<\/h3>\n
\n
\n
\n\t\t\t\t<\/p>\n
As PCWorld’s senior editor, Mark focuses on Microsoft news and chip technology, among other beats. He has formerly written for PCMag, BYTE, Slashdot, eWEEK, and ReadWrite.<\/p>\n<\/p><\/div>\n
\n
\n\t\t\t\t\t\t\t\t
Read More<\/a>
\n Nancie Michaud<\/p>\n","protected":false},"excerpt":{"rendered":"
Runway has shouldered aside Midjourney and Stable Diffusion, introducing the first clips of text-to-video AI art that the company says is completely generated by a text prompt. The company said that it\u2019s offering a waitlist to join what it calls \u201cGen 2\u201d of text-to-video AI, after offering a similar waitlist for its first, simpler text-to-video<\/p>\n","protected":false},"author":1,"featured_media":620253,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27712,46,1870],"tags":[],"class_list":{"0":"post-620252","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-prompts","8":"category-technology","9":"category-turns"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/620252","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=620252"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/620252\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/620253"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=620252"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=620252"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=620252"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

\n\t\t\n\t\tAuthor: Mark Hachman<\/a>, Senior Editor\t\t<\/h3>\n\n\n\n\t\t\t\t<\/p>\nAs PCWorld’s senior editor, Mark focuses on Microsoft news and chip technology, among other beats. He has formerly written for PCMag, BYTE, Slashdot, eWEEK, and ReadWrite.<\/p>\n<\/p><\/div>\n