Russian tech company Yandex said it has made a large language model trained on 100bn parameters available for free to the public, in order to boost the growth of this form of AI technology.

Large language models are natural language processing (NLP) systems that are trained on a massive volume of text. Yandex said they have become the “pinnacle” of neural networks used in NLP tasks.

The Russian company said training these models requires millions of dollars, experts,and years of development, which means only major IT companies have access to “this state-of-the-art technology”.

“Researchers and developers all over the world need access to these solutions,” Yandex said in a blog post. “Without new research, their growth could wane. The only way to avoid this is by sharing best practices with the developer community.”

Yandex said its YaLM model is currently the world’s largest GPT-like neural network freely available for English. The Russian tech giant said it has published the model and training materials on GitHub, under a licence that permits both research and commercial use.

The company said on GitHub that it took 65 days to train the YaLM 100b model on a cluster of 800 A100 graphics cards and 1.7TB of online texts, books, and “countless other sources” in English and Russian.

Yandex’s reasons for sharing its large language model are similar to Meta’s statements last month, when it announced plans to share its large language model that has 175bn parameters trained on publicly available datasets, making it available for AI researchers.

“Meta AI believes that collaboration across research organisations is critical to the responsible development of AI technologies,” the company said.

Yandex and Meta are not the only companies looking into large language models. Last October, tech giants Microsoft and Nvidia teamed up to create a language model with 105 layers and 530bn parameters, three times as many parameters as OpenAI’s GPT-3 model.

10 things you need to know direct to your inbox every weekday. Sign up for  the Daily Brief, Silicon Republic’s digest of essential sci-tech news.

The post Russia’s Yandex gives public access to GPT-like AI large language model appeared first on Silicon Republic.