PALO ALTO, Calif.--(BUSINESS WIRE)--The four founders at startup NucleusAI just emerged from stealth and demonstrated they are able to build large language models (LLMs) like the Big Tech companies. NucleusAI just launched a 22-billion-parameter LLM that outperforms all the models of similar size.
Typically, foundational models are built by 30 to 100 people with skill sets that address the different aspects of LLMs. The fact Nucleus has achieved the same or better with just four people and the open source community is a testament to the team’s overall expertise.
There have been only a handful of companies to have open sourced and commercially licensed an LLM: Meta, Salesforce, Mosaic and Falcon by TII (UAE government funded) among them. With this release Nucleus will join that exclusive list. Nucleus is unique among those companies as an early stage startup.
Nucleus’s LLM
Nucleus’s 22-billion-parameter LLM was pre-trained on a context length of 2,048 tokens, and trained on a trillion tokens of data, from large scale deduplicated and cleaned web data, Wikipedia, Stack Exchange, arXiv, and code. This diverse training data ensures a well-rounded knowledge base, spanning general information to academic research and coding insights.
Today’s announcement will be followed by the release of two other models (3-billion parameters and 11-billion parameters) that were pre-trained on the larger context length of 4,096 tokens. The models were also trained with slightly different dimensional choices than the 22-billion-parameter model. The 22-billion-parameter model will be released in several versions, trained on one trillion tokens, 350 billion tokens, and 700 billion tokens, respectively.
Agriculture will be the first beneficiary
Nucleus will leverage AI to make agriculture failure proof. We have now reached a time in the world where all potential risks in agriculture (Including natural calamities) can be fully mitigated. The company is building an intelligent operating system (OS) for farming, optimizing supply and demand, just like Uber does.
Growing produce is one of the most important things for humanity, and it can be one of the most lucrative things to do. Agricultural challenges, such as optimizing and sharing yields, are top of mind for the founders who are familiar with and dedicated to tackling this challenge. More details will be announced In the second week of October.
About NucleusAI
NucleusAI is a team of 4 individuals, all of whom are former founders. Two were AI researchers at Amazon, one worked at Samsung Research. They have joined forces to pursue a long-held passion project of making agriculture one of the most lucrative things to do. The founders believe that their current network and skillset position them perfectly to take on this new challenge head-on.
