In developing language AI experiences for the enterprise, it is important to start with a foundational model that has a licensing agreement that allows you to fine-tune the model for full commercial use.
Meta did not exactly give us that with the v2 release of their Llama model, but the license does allow for commercial use up to 700,000,000 users of commercial solution without have to contact them and go under an agreement.
If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.
Llama access request form – Meta AI
Here is what we know from reading the 77 page paper :
Llama 2 was trained between January 2023 to July 2023, using an offline dataset and likely costs $20M+.
Llama 2, an updated version of Llama 1 from Feb 2023, trained on a new mix of publicly available data. Meta increased the size of the pretraining corpus by 40% (2 trillion tokens), doubled the context length of the model (~3500 words), and adopted grouped-query attention (Ainslie et al., 2023).
Meta is releasing pretrained and fine-tuned variants of Llama 2 with 7B, 13B, and 70B parameters. The fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases.
| Content Length | GQA | Tokens | LR | |
|---|---|---|---|---|
| 7B | 4k | ✗ | 2.0T | 3.0 x 10-4 |
| 13B | 4k | ✗ | 2.0T | 3.0 x 10-4 |
| 70B | 4k | ✔ | 2.0T | 1.5 x 10-4 |
Meta also has trained 34B variants but are not releasing them yet. I suspect these are the models that they will productize and are using the released variants to prime the pump in adoption. As a practitioner we know the best models are around 30B, so no surprise here.
Qualcomm has announced that it will make the Llama 2 model available on Snapdragon-powered mobile devices in early 2024 using methodologies like QLoRA (code).
Meta’s team did a human study on 4K prompts to evaluate Llama-2’s helpfulness. They use “win rate” as a metric to compare models, in similar spirit as the Vicuna benchmark.
Meta did a great job at providing a quick start guild, model card, setting up chat, implemented addition classifiers, and of course how to fine-tune.
The Llama 2 70B model roughly ties with GPT-3.5-0301, and performs noticeably stronger than Falcon, MPT, and Vicuna.
Llama 2 is not yet at a GPT-3.5 level, mainly because of its weak coding abilities. On “HumanEval” (Paper and Code), it isn’t nearly as good as StarCoder (Paper and Code) or many other models specifically designed for coding. I have little doubt that Llama 2 will improve significantly thanks to its open weights.
Meta’s team goes above and beyond in prioritizing AI safety, dedicating almost half of the paper to safety guardrails, red-teaming (paper), and evaluations. Their responsible efforts deserve applause! Unlike previous approaches, Meta resolves the tradeoff between helpfulness and safety by training two separate reward models. Though not currently open-source, these models hold significant potential value for the community.
Meta spelled out the entire recipe, including model details, training stages, hardware, data pipeline, and annotation process.
The Jivoo Foundry as an operationalized NLP pipeline within a larger ML Ops process that allows us to rapidly implement new models like this in our products such as Jivoo GRC, a language AI enabled governance risk, and compliance solution.
