A New Open-Source Large Language Model
Everyone in Artificial Intelligence is attempting to out-chat one another. The language models are rolling out one after another, with some companies competing against themselves and developing new AI within months of putting a model out. That’s the key to AI. It needs constant monitoring and updates to be successful.
What is Llama 2?
Meta announced LLaMA in February 2023. It stands for Large Language Model Meta Artificial Intelligence. This large language model, or LLM, has been trained on various model sizes, ranging from 7 billion to 65 billion parameters. The Llama models change due to parameter sizes. For instance:
- 7B parameters (trained on 1 trillion tokens)
- 13B parameters
- 33B parameters (trained on 1.4 trillion tokens)
- 65B parameters (trained on 1.4 trillion tokens)
The Llama 2 has been trained with even more data.
What did Meta Use to Train Llama 2?
- Webpages scraped by CommonCrawl
- Open-source repositories of source code from GitHub
- Wikipedia in 20 different languages
- Public domain books from Project Gutenberg
- The LaTeX source code for scientific papers uploaded to ArXiv
- Questions and answers from Stack Exchange websites
Large language models must be trained with a wealth of knowledge. The Llama 2 was designed to help developers and organizations create generative AI tools and experiences. With Llama 2, you can choose what models to develop and underwrite both open and frontier models.
Llama 2 Outperforms Its Predecessors
When you compare Llama 2 to its predecessors, its performance is above the board. According to Meta, “Llama 2 pre-trained models are trained on 2 trillion tokens and have double the context length than Llama 1. Its fine-tuned models have been trained on over 1 million human annotations. Llama 2 outperforms other open-source language models on many external benchmarks, including reasoning, coding, proficiency, and knowledge tests.”
How to Get Llama 2
Llama 2 is available for free for research and commercial use. However, you have to request access to the open-source model. Simply input some personal details such as name, email, and organization. Then, you’re added to the list. You’ll receive an email confirmation with how to download it. Llama 2 is also accessible through Amazon Web Services, Hugging Face, and other providers.
Meta explains, “Throughout Meta’s company’s history, they’ve experienced the benefits of an open-source approach when innovating in other business areas. Their engineers developed and shared frameworks that are now industry standards–like React, a leading framework for making web and mobile applications, and PyTorch, now the leading framework for AI. These became commonly used infrastructure for the entire technology industry. We believe that openly sharing today’s large language models will also support the development of helpful and safer generative AI.”
PCX also looks forward to seeing what the world builds with Llama 2. We provide our readers with current events in all things tech. You can read more here.