In a recent announcement on Tuesday evening, Meta CEO Mark Zuckerberg took to his Facebook page to unveil a major collaboration between Meta and Microsoft.
The partnership aims to introduce Llama 2, an advanced iteration of Meta’s open source large language model.
Zuckerberg highlighted that Llama 2 will be available for both research and commercial use, emphasizing the company’s commitment to fostering innovation and progress across the industry.
Meta, formerly known as Facebook, has a notable history of open sourcing its infrastructure and artificial intelligence (AI) work.
From popular machine learning framework PyTorch to groundbreaking models like Segment Anything, ImageBind, and Dino, Meta’s open source initiatives have consistently driven technological advancements and product development.
Zuckerberg emphasized the power of open source in propelling innovation, citing its ability to enable a broader community of developers to build with cutting-edge technology. Moreover, he emphasized that the transparency brought about by open source projects leads to enhanced safety and security.
By allowing more individuals to scrutinize the software, potential issues can be identified and resolved promptly. Consequently, Meta’s decision to open source Llama 2 aims to unlock further progress and foster a more collaborative ecosystem.
The release of Llama 2 includes pretrained and fine-tuned models with varying parameter sizes. The models come in 7 billion, 13 billion, and 70 billion parameters, showcasing the significant improvements made in comparison to its predecessor, Llama 1. Notably, Llama 2 underwent pretraining on an expanded dataset that was 40% larger than the one used for Llama 1. In addition, to architectural enhancements, the fine-tuned models were subjected to supervised fine-tuning and reinforcement learning with human feedback (RLHF). These techniques leveraged over 1 million human annotations and have demonstrated leading results in terms of safety and quality.
To access Llama 2, users have multiple options available. They can directly download the models or utilize Microsoft’s Azure platform, thanks to the preferred partnership between Meta and Microsoft. This collaboration grants users access to Llama 2 through Azure, coupled with Microsoft’s robust safety and content tools. Furthermore, a locally optimized version of Llama 2 is available for users running Windows.
Zuckerberg expressed his excitement about the potential applications and developments that will arise from the use of Llama 2. As the model becomes more accessible to researchers and developers worldwide, Meta envisions a surge of groundbreaking innovations in various fields.
“The introduction of Llama 2 and the ongoing collaboration between Meta and Microsoft promise to bring new advancements in natural language processing and AI to the open source community, propelling the boundaries of technological possibilities forward.”
