9 Best 3B Local LLM Model (Open Source)

15 Min Read
New Project 1 - 9 Best 3B Local LLM Model (Open Source)

In the rapidly evolving landscape of language models, the power of compact yet efficient models cannot be underestimated. This article delves into the realm of 3 billion parameter language models (3B LLMs), debunking the notion that size is the sole determinant of performance.

These smaller-scale models have transcended expectations, showcasing their capability to rival and even surpass larger counterparts. By harnessing the potential of cutting-edge advancements and innovative approaches, this article uncovers the finest 3B LLMs that not only uphold performance benchmarks but also possess the unique capacity to be integrated into low-power devices, heralding a new era of personalized and private AI experiences.

Contact me if you think some other model should be on the list.

Phi-2 

Phi-2 is a language model developed by Microsoft Research. It’s part of the “Phi” series of small language models that aim to perform well compared to larger models.

Phi-2 has 2.7 billion parameters and was trained using various NLP synthetic texts and filtered websites for safety and educational value. It performs well on benchmarks testing common sense, language understanding, and logical reasoning.

Phi-2 is best suited for prompts using the QA format, the chat format, and the code format. It hasn’t been fine-tuned through reinforcement learning from human feedback. The goal of this model is to provide the research community with a small model to explore safety challenges, such as reducing toxicity, understanding societal biases, and enhancing controllability.

Phi-2 matches or outperforms models up to 25x larger on complex benchmarks, thanks to new innovations in model scaling and training data curation. With its compact size, Phi-2 is an ideal playground for researchers.

StableLM Zephyr 3B

The “StableLM Zephyr 3B” is a language model developed by Stability AI, designed for text generation tasks. It’s an auto-regressive language model based on the transformer decoder architecture, with a total of 3 billion parameters. This model is inspired by HuggingFaceH4’s Zephyr 7B training pipeline and has been trained on a mix of publicly available datasets and synthetic datasets using Direct Preference Optimization (DPO).

Performance Benchmarks

TaskValue (%)Description
ARC (25-shot)47.0ARC Challenge measures a model’s ability to answer science questions designed for school students.
HellaSwag (10-shot)74.2HellaSwag assesses a model’s understanding of commonsense scenarios.
MMLU (5-shot)46.3The MMLU (Massive Multitask Language Understanding) evaluates the model’s performance across a wide range of subjects and languages.
TruthfulQA (0-shot)46.5TruthfulQA checks how often a model provides factually accurate answers.
Winogrande (5-shot)65.5Winogrande measures a model’s ability to resolve ambiguous pronouns in sentences.
GSM8K (5-shot)42.3GSM8K evaluates a model’s problem-solving skills on grade-school level math problems.
BigBench (Avg)35.26BigBench is a broad benchmark covering many AI tasks to evaluate general-purpose language models.
AGI Benchmark (Avg)33.23AGI Benchmark measures a model’s performance across tasks deemed relevant for artificial general intelligence.

MiniChat-1.5-3B

MiniChat-1.5-3B, hosted on Hugging Face, is a distinguished language model notable for several characteristics:

  1. Origin and Development: MiniChat-1.5-3B is distilled and finetuned from an adapted version of LLaMA2-7B. This process follows the principles outlined in the publication “Towards the Law of Capacity Gap in Distilling Language Models”​​.
  2. Performance and Competitiveness: The model notably outperforms a broad spectrum of 3B competitors in GPT4 evaluations. It also competes effectively with several 7B chat models, demonstrating its robustness and efficiency in handling complex language tasks​​.
  3. Usage Example: To utilize MiniChat-1.5-3B, an example code snippet demonstrates the import of necessary modules and the setup for generating responses. This snippet showcases the model’s ability to engage in multiturn conversations, emphasizing its practical application in conversational AI scenarios​​.
  4. Evaluation Metrics: The model has been evaluated across various metrics, showcasing its performance in different language understanding tasks. These include an average score of 42.94, and notable scores in ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, and GSM8K benchmarks, reflecting its broad competence in language understanding​​.
  5. Technical Specifications: MiniChat-1.5-3B is built with a model size of 3.02 billion parameters and utilizes the BF16 tensor type, indicating its substantial computational resources and efficiency​​.

StableLM-3B-4E1T

StableLM-3B-4E1T is a powerful language model that boasts an impressive 3 billion parameters, making it well-suited for a wide range of natural language processing tasks. This decoder-only model was pre-trained on a massive dataset consisting of 1 trillion tokens of diverse English and code datasets, including Falcon RefinedWeb, RedPajama-Data, The Pile, and StarCoder. With such a large dataset, StableLM-3B-4E1T has been trained for 4 epochs, ensuring that it can handle even the most challenging language tasks with ease.

In addition to its impressive size and training, StableLM-3B-4E1T has also demonstrated exceptional performance in various benchmarks and leaderboards. It has achieved top scores in benchmarks, which evaluates a model’s ability to perform a variety of natural language processing tasks, such as question answering, sentiment analysis, and text classification. Specifically, StableLM-3B-4E1T has achieved state-of-the-art results on the GLUE benchmark, outperforming other models in its class.

FeatureDescription
Number of Parameters3 billion
Pre-training Dataset1 trillion tokens of diverse English and code datasets
Training Epochs4 epochs
TokenizerNeoX
Vocabulary Size50,257
OptimizerAdamW
PrecisionBfloat16
Benchmark PerformanceTop performer in GLUE and SuperGLUE benchmarks
Leaderboard RankingTop ranking in the 3B parameter category
RecommendationFine-tune for specific downstream tasks

Marx-3B-V2

Marx-3B-V2 stands as a testament to the remarkable capabilities of a 3 billion parameter language model. Derived from OpenLLaMA 3B V2 and refined through two epochs of fine-tuning on the expansive EverythingLM Data V2 (in ShareGPT format), this model’s potency is evident. Despite its modest scale, Marx-3B-V2 emerges as a stalwart contender, claiming a place among the upper echelons of the 3B LLM leaderboard. This LLM excels in its capacity to comprehend and generate text, making it a versatile tool for various applications. Its relatively compact size also ensures accessibility, allowing it to operate seamlessly on a range of consumer hardware setups. With Marx-3B-V2, the pursuit of linguistic excellence finds a fitting companion that marries efficiency with competence.

FeatureDescription
Model NameMarx-3B-V2
Parameter Count3 billion
Base ModelOpenLLaMA 3B V2
Fine-Tuning DataEverythingLM Data V2 (in ShareGPT format)
Leaderboard RankingUpper echelons of the 3B LLM leaderboard
Comprehension CapabilitiesExcels in comprehending and generating text
Generative AbilitiesVersatile tool for various applications
Compact SizeRelatively small size ensures accessibility on consumer hardware setups
EfficiencyMarries efficiency with competence

ReasonixPajama-3B-HF

This 3 billion parameter language model, while shrouded in mystery, holds a notable position on the LLM leaderboard for 3 billion parameter models. Despite the limited available information, its commendable rank speaks to its capabilities. With a compact size, this model offers the advantage of being well-suited for deployment on a range of consumer hardware. Its proficiency in understanding and generating text likely contributes to its high ranking. While specific details about its architecture and training data remain elusive, its performance underscores the potential for smaller-scale models to excel in natural language understanding and generation tasks.

FeatureDescription
Model NameMysterious 3B Parameter Language Model
Parameter Count3 billion
Leaderboard RankingNotable position on the LLM leaderboard for 3B parameter models
Compact SizeWell-suited for deployment on a range of consumer hardware
Text Understanding CapabilitiesProficient in understanding text
Text Generation CapabilitiesProficient in generating text
ArchitectureUnknown
Training DataUnknown
PerformanceHigh ranking on the LLM leaderboard
Potential ApplicationsNatural language understanding and generation tasks
AdvantagesCompact size, suitable for deployment on consumer hardware

BTLM-3B-8k-base

The BTLM-3B-8k-base, a cutting-edge Bittensor Language Model, redefines the capabilities of 3 billion parameter models. Trained on 627 billion tokens from SlimPajama, this model boasts an impressive 8k context length. Surpassing models trained on significantly larger datasets, BTLM-3B-8k-base achieves performance on par with open 7 billion parameter models. What’s truly groundbreaking is its adaptability: it can be quantized to a mere 4 bits, enabling deployment on devices with as little as 3GB of memory. This remarkable feat opens doors for personal AI assistants on mobile and IoT devices, ensuring local processing for enhanced privacy and independence from the cloud. With its Apache 2.0 license for commercial use, BTLM-3B-8k-base marks a significant stride towards a decentralized AI future.

FeatureDescription
Model NameBTLM-3B-8k-base
Parameter Count3 billion
Training Dataset627 billion tokens from SlimPajama
Context Length8k
PerformanceOn par with open 7 billion parameter models
QuantizationCan be quantized to 4 bits
DeploymentSuitable for devices with 3GB of memory or less
LicensingApache 2.0 license for commercial use
Key BenefitsAdaptability, decentralized AI future, local processing for enhanced privacy and independence from the cloud

mamba-gpt-3b-v3

Mamba-GPT-3b-v3 stands as a remarkable achievement among 3 billion-parameter LLM models, positioning itself as the premier choice on the Open LLM Leaderboard. This model’s prowess has transcended even the esteemed dolly-v2-12b, showcasing an exceptional leap in performance. Through meticulous fine-tuning of the open-lama model, Mamba-GPT-3b-v3 has achieved a remarkable feat by outperforming its progenitor across a spectrum of evaluation subtasks. Its current standing as the leading 3B model is further fortified by its ability to deliver performance akin to that of the larger llama-7b model. This triumph not only underscores the potential of compact LLMs but also paves the way for embedding potent AI assistants into resource-constrained devices, amplifying privacy and enabling local operations sans dependency on the cloud.

InformationDetails
Model NameMamba-GPT-3b-v3
PositionLeading 3B model on the Open LLM Leaderboard
PerformanceOutperforms dolly-v2-12b and delivers performance similar to llama-7b
DevelopmentAchieved through meticulous fine-tuning of the open-lama model
CapabilitiesEnables powerful AI assistants on resource-constrained devices
AdvantagesLocal processing, increased privacy, reduced reliance on cloud resources

open_llama_3b_v2

The “open_llama_3b_v2” is a significant step in the evolution of small-sized language models, demonstrating that a model with 3 billion parameters can excel. A notable achievement is its permissively licensed, open-source nature, allowing broader accessibility. Trained on diverse data mixtures, it serves as a seamless alternative to Meta AI’s LLaMA, offering compatibility with existing setups. This 3B model, trained on a massive 1 trillion tokens, showcases the potential of compact models to achieve impressive performance, paving the way for personal AI assistants on local devices, ensuring privacy and autonomy.

FeatureDescription
Model Size3 billion parameters
Training DataDiverse data mixtures
LicensingPermissively licensed, open-source
CompatibilitySeamless alternative to Meta AI’s LLaMA
PerformanceTrained on 1 trillion tokens, achieving impressive results
PrivacyEnables personal AI assistants on local devices, ensuring privacy and autonomy
Data UtilizationShowcases the potential of compact models to achieve high performance
AccessibilityBroader accessibility due to open-source nature

StableLM-Base-Alpha-3B-v2

StableLM-Base-Alpha-3B-v2 represents a significant advancement in compact language models. Building upon the original Alpha models, this iteration introduces architectural enhancements like SwiGLU (Shazeer, 2020) and relies on superior data sources for improved performance. With a context length of 4096 tokens, it offers a broader understanding of text.

Key enhancements include the use of high-quality data like RefinedWeb and C4 instead of The Pile v2 Common-Crawl scrape. Notably, the model’s ability to sample web text at an increased rate of 71% as opposed to 35% has led to noteworthy improvements in downstream performance, showcasing the potential of compact models to outperform in various applications.

FeatureDescription
Model Size3 billion parameters
ArchitectureEnhanced with SwiGLU (Shazeer, 2020)
Data SourcesUses high-quality data sources such as RefinedWeb and C4
Context Length4096 tokens, providing a broader understanding of text
Sampling Rate71%, an increase from 35%, resulting in improved downstream performance
PerformanceDemonstrates noteworthy improvements in various applications
CompactnessShowcases the potential of compact models to outperform larger models
Data UtilizationEffective utilization of superior data sources for better performance
Share This Article
Follow:
SK is a versatile writer deeply passionate about anime, evolution, storytelling, art, AI, game development, and VFX. His writings transcend genres, exploring these interests and more. Dive into his captivating world of words and explore the depths of his creative universe.