Meta opens access to its large language model for AI researchersLeigh Mc Gowranon May 4, 2022 at 07:08 Silicon RepublicSilicon Republic


Facebook’s parent company Meta said it is sharing its large language model trained with more than 175bn parameters, to be available for AI researchers.

The social media giant said it is sharing access to both the pretrained models and the code needed to train and use them. Meta said it is sharing the large language model to allow for “more community engagement in understanding this foundational new technology”.

“Access to the model will be granted to academic researchers, those affiliated with organisations in government, civil society and academia, along with industry research laboratories around the world,” Meta AI said in a blog post yesterday (3 May)

Large language models are natural language processing (NLP) systems which are trained on a massive volume of text. These models are able to answer reading comprehension questions, solve basic math problems and generate text.

Meta said full research access to large language models is usually restricted to a “few highly resources labs”, which hinders efforts to increase their “robustness” and remove issues such as bias and toxicity within the models.

“For AI research to advance, the broader scientific community must be able to work together with cutting-edge models to effectively explore their potential while also probing for their vulnerabilities at the same time,” Meta AI said.

“Meta AI believes that collaboration across research organisations is critical to the responsible development of AI technologies.”

The social media company said it designed its model – called OPT-175B – to be energy efficient, as it was trained using roughly 14pc of the carbon footprint used to train OpenAI’s GPT-3.

Meta also said it is releasing a suite of “smaller-scale baseline models”, trained on the same data set and using similar settings as OPT-175B.

Meta has been investing in AI research for some time. In February the company shared some of the AI research projects it is focused on, including universal speech translation, AI that can learn like a human and a more conversational AI assistant.

Last January, Meta said its AI research team has been working for years on a supercomputer that could be the world’s “largest and fastest” when fully built out, which it hopes to achieve by mid-2022.

Meta isn’t the only company looking into large language models. Last October, Tech giants Microsoft and Nvidia teamed up to create a language model with 105 layers and 530bn parameters, three times as many parameters as OpenAI’s GPT-3.

10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.

The post Meta opens access to its large language model for AI researchers appeared first on Silicon Republic.

Leave a Comment