Luma AI Launches AI-Powered Text-to-Video Generation Platform Dream Machine

Luma AI launched its artificial intelligence (AI) text-to-video technology mannequin, Dream Machine, globally on Wednesday. The platform can generate as much as five-second-long movies from easy or descriptive textual content prompts. The AI mannequin can generate movies in varied types corresponding to cinematic, animation, reasonable, and extra. The AI agency claims that Dream Machine was skilled totally on movies, and is able to producing “bodily correct, constant and eventful photographs.” The platform is presently free to entry and use, however it’s prone to have a every day technology restrict.

Luma AI debuts Dream Machine

In line with the website, the Dream Machine AI mannequin is constructed on a transformer mannequin and was skilled instantly on movies. Often, massive language fashions (LLMs) are skilled on textual content and pictures after which are moved to movies, as they require a deeper spatial and movement understanding. “Dream Machine is our first step in direction of constructing a common creativeness engine,” the corporate added.

Dream Machine joins video technology platforms corresponding to Runway AI and Pika 1.0 that are additionally out there within the public area and supply video technology between three to 5 seconds. Devices 360 tried the platform and located that the immediate adherence of the platform is sub-par. It struggles with a number of characters or when the immediate is simply too advanced. Nevertheless, in comparison with the opposite two, it’s able to producing higher-quality cinematic movies.

The AI platform takes 120 seconds to generate a video which the corporate claims could have 120 completely different frames. Dream Machine can also be mentioned to know how folks, animals, and objects work together with the bodily world and create movies with correct physics and character consistency.

Nevertheless, Luma AI additionally highlighted a number of limitations within the present mode corresponding to motion, textual content, morphing, and the well-known Janus drawback, the place as an alternative of displaying a constant 3D output, the AI mannequin reveals a number of canonical views of an object in numerous instructions.

Luma AI’s Dream Machine creates a video about Stormtroopers
Photograph Credit score: Luma AI/Dream Machine

Luma AI has not shared any technical particulars in regards to the AI mannequin, so details about parameter dimension, benchmarks, structure, and coaching strategies will not be recognized. The corporate additionally didn’t share any particulars in regards to the procurement of the coaching knowledge. Notably, Devices 360 was capable of generate a number of movies with copyrighted characters in them.

To check out the platform, lovers can go to the web site and click on on the ‘Strive Now’ button. Customers could have to enroll in the platform earlier than they will generate movies.