RALEIGH, N.C.--(BUSINESS WIRE)--Red Hat, Inc., the world's leading provider of open source solutions, today announced that it has signed a definitive agreement to acquire Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads. Neural Magic’s expertise in inference performance engineering and commitment to open source aligns with Red Hat’s vision of high-performing AI workloads that directly map to customer-specific use cases and data, anywhere and everywhere across the hybrid cloud.
While the promise of gen AI dominates much of today’s technology landscape, the large language models (LLMs) underpinning these systems continue to increase in size. As a result, building cost-efficient and reliable LLM services requires significant computing power, energy resources and specialized operational skills. These challenges effectively put the benefits of customized, deployment-ready and more security-conscious AI out of reach for most organizations.
Red Hat intends to address these challenges by making gen AI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-driven open source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research and diverse hardware backends including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs and x86 CPUs. Neural Magic’s leadership in the vLLM project combined with Red Hat’s strong portfolio of hybrid cloud AI technologies will offer organizations an open pathway to building AI strategies that meet their unique needs, wherever their data lives.
Red Hat + Neural Magic: Enabling a future of hybrid cloud-ready gen AI
Neural Magic spun out of MIT in 2018 with the goal of building performant inference software for deep learning. With Neural Magic’s technology and performance engineering expertise, Red Hat aims to accelerate its vision for AI’s future, powered by the Red Hat AI technology portfolio. Built to break through the challenges of wide-scale enterprise AI, Red Hat uses open source innovation to further democratize access to AI’s transformative power via:
- Open source-licensed models, from the 1B to 405B parameter scale, that can run anywhere and everywhere needed across the hybrid cloud - in corporate data centers, on multiple clouds and at the edge.
- Fine-tuning capabilities that enable organizations to more easily customize LLMs to their private data and uses cases with a stronger security footprint;
- Inference performance engineering expertise, resulting in greater operational and infrastructure efficiencies; and
- A partner and open source ecosystem and support structures that enable broader customer choice, from LLMs and tooling to certified server hardware and underlying chip architectures.
vLLM leadership to enhance Red Hat AI
Neural Magic uses its expertise and knowledge in vLLM to build an enterprise-grade inference stack which enables customers to optimize, deploy and scale LLM workloads across hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle. Neural Magic also develops model optimization research, builds LLM Compressor (a unified library for optimizing LLMs with state-of-the-art sparsity and quantization algorithms) and maintains a repository of pre-optimized models ready to deploy with vLLM.
Red Hat AI aims to help customers lower AI’s costs and skill barriers with powerful technologies, including:
- Red Hat Enterprise Linux AI (RHEL AI), a foundation model platform to more seamlessly develop, test and run the IBM Granite family of open source LLMs for enterprise applications on Linux server deployments;
- Red Hat OpenShift AI, an AI platform that provides tools to rapidly develop, train, serve and monitor machine learning models across distributed Kubernetes environments on-site, in the public cloud or at the edge; and
- InstructLab, an approachable open source AI community project created by Red Hat and IBM that enables anyone to shape the future of gen AI via the collaborative improvement of open source-licensed Granite LLMs using InstructLab's fine-tuning technology.
Neural Magic’s technology leadership in vLLM will enhance Red Hat AI’s ability to support LLM deployments anywhere and everywhere across the hybrid cloud with a ready-made, highly-optimized and open inference stack.
The transaction is subject to applicable regulatory reviews and other customary closing conditions.
Supporting Quotes
Matt Hicks, president and CEO, Red Hat
“AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs. We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.”
Brian Stevens, CEO, Neural Magic
“Open source has proven time and again to drive innovation through the power of community collaboration. At Neural Magic, we’ve assembled some of the industry’s top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities. Joining Red Hat is not only a cultural match, but will benefit companies large and small in their AI transformation journeys.”
Dario Gil, IBM senior vice president and director of Research
“As our clients look to scale AI across their hybrid environments – virtualized, cloud-native LLMs built on open foundations will become the industry standard. Red Hat’s leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic’s offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise.”
Additional Resources
- Read a blog post from Red Hat president and CEO Matt Hicks on Red Hat’s vision for AI’s open source future
- See responses to frequently asked questions around this definitive agreement
- Learn more about Red Hat OpenShift AI
- Read more about RHEL AI
Connect with Red Hat
- Learn more about Red Hat
- Get more news in the Red Hat newsroom
- Read the Red Hat blog
- Follow Red Hat on X
- Follow Red Hat on Instagram
- Watch Red Hat videos on YouTube
- Follow Red Hat on LinkedIn
About Red Hat, Inc.
Red Hat is the world’s leading provider of enterprise open source software solutions, using a community-powered approach to deliver reliable and high-performing Linux, hybrid cloud, container, and Kubernetes technologies. Red Hat helps customers integrate new and existing IT applications, develop cloud-native applications, standardize on our industry-leading operating system, and automate, secure, and manage complex environments. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. As a strategic partner to cloud providers, system integrators, application vendors, customers, and open source communities, Red Hat can help organizations prepare for the digital future.
Forward-Looking Statements
Except for the historical information and discussions contained herein, statements contained in this press release may constitute forward-looking statements within the meaning of the Private Securities Litigation Reform Act of 1995. Forward-looking statements are based on the company’s current assumptions regarding future business and financial performance. These statements involve a number of risks, uncertainties and other factors that could cause actual results to differ materially. Any forward-looking statement in this press release speaks only as of the date on which it is made. Except as required by law, the company assumes no obligation to update or revise any forward-looking statements.
Red Hat, Red Hat Enterprise Linux, the Red Hat logo, and OpenShift are trademarks or registered trademarks of Red Hat, Inc. or its subsidiaries in the U.S. and other countries. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
