Sakana AI Announces AI CUDA Engineer That Can Speed Up Model Development and Deployment

Sakana AI, a Tokyo-based artificial intelligence (AI) agency, launched a brand new synthetic intelligence (AI) agentic framework that may enhance the event and deployment speeds of enormous language fashions (LLMs). Introduced on Thursday, the corporate unveiled the AI CUDA Engineer that improves each the pre-training and inference speeds of an AI mannequin by optimising the codebase. The AI agency highlighted that all the course of is pushed by AI brokers and is end-to-end automated. Notably, Sakana AI introduced The AI Scientist final yr which may conduct scientific analysis.

Sakana AI Unveils AI CUDA Engineer

In a post, the Japanese AI agency said that after growing AI techniques that may create new fashions, and absolutely automate the AI analysis course of, it started engaged on methods to hurry up the deployment and inference speeds of an LLM.

The corporate mentioned that the analysis led to the event of the AI CUDA Engineer. It’s a absolutely automated, complete agent framework for CUDA (Compute Unified System Structure) kernel discovery and optimisation.

CUDA kernels will be understood as specialised features that run on Nvidia GPUs, permitting parallel execution of code throughout a number of threads. On account of parallelism, it’s extra optimised than conventional strategies and permits for the acceleration of computational duties, particularly these with massive datasets. As such, that is thought of an effective way to optimise AI fashions’ deployment and inference.

Sakana AI mentioned the AI CUDA Engineer can mechanically convert PyTorch modules into optimised CUDA kernels, to considerably enhance deployment speedups. It might probably generate kernels which might be mentioned to be 10-100 occasions sooner than its PyTorch counterpart.

The method contains 4 steps. First, the agent framework converts the PyTorch code into working kernels. Then, the agent implements optimisation strategies to make sure solely the perfect kernels are generated. Then, kernel crossover prompts are added, which mix a number of optimised kernels to create new kernels. Lastly, the AI agent preserves the high-performance CUDA kernels in an archive, that are used to ship efficiency enhancements. The corporate has additionally revealed a study that additional particulars the method.

Alongside the paper, Sakana AI can be publishing the AI CUDA Engineer Archive, which is a dataset consisting of greater than 30,000 kernels generated by the AI. These kernels are launched below the CC-By-4.0 license and will be accessed through Hugging Face.

Moreover, the Japanese agency additionally launched a web site that lets guests interactively discover 17,000 verified kernels and their profiles. The web site permits customers to discover these kernels throughout 230 duties, and in addition lets them evaluate CUDA kernels throughout particular person experiments.