Phi-3 Mini Latest: Microsoft's Compact AI Revolution
Microsoft's Phi-3 Mini represents a groundbreaking shift in artificial intelligence, proving that smaller language models can deliver exceptional performance while maintaining efficiency and accessibility.
What is Phi-3 Mini?
Phi-3 Mini is a 3.8-billion parameter, lightweight, state-of-the-art open model trained with high-quality synthetic data and filtered publicly available websites data with a focus on reasoning-dense properties. Unlike traditional large language models that require massive computational resources, Phi-3 Mini challenges the conventional wisdom that bigger is always better in AI.
The model belongs to Microsoft's Phi-3 family and comes in two context-length variants: 4K and 128K tokens. It is the first model in its class to support a context window of up to 128K tokens, with little impact on quality.
Latest Update: Phi-3.5 Mini (August 2024)
In August 2024, Microsoft released Phi-3.5-mini, an update over the June 2024 instruction-tuned Phi-3 Mini release based on valuable user feedback. The model used additional post-training data leading to substantial gains on multilingual, multi-turn conversation quality, and reasoning capability.
Key improvements in Phi-3.5 Mini include:
• Enhanced multilingual support with 128K context length
• Improved reasoning capabilities across multiple languages
• Better multi-turn conversation quality
• Superior long-context understanding
• Upgraded instruction following and structured output generation
• Improved reasoning capabilities across multiple languages
• Better multi-turn conversation quality
• Superior long-context understanding
• Upgraded instruction following and structured output generation
Technical Specifications
| Parameter Count | 3.8 billion |
| Architecture | Dense decoder-only Transformer |
| Context Length | 128K tokens (also available in 4K variant) |
| Training Data | 3.4 trillion tokens |
| Training Time | 10 days on 512 H100-80G GPUs |
| Release Date | August 2024 (Phi-3.5 Mini) |
| License | MIT License |
Supported Languages
Phi-3.5 Mini supports 23 languages: Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, and Ukrainian.
Arabic, Dutch, Finnish, Polish, Thai and Ukrainian received the most boost from the new Phi version with 25-50% improvement in performance.
Performance and Benchmarks
Phi-3 models significantly outperform language models of the same and larger sizes on key benchmarks. Phi-3-mini does better than models twice its size.
Notable benchmark results:
• MMLU benchmark (5-shot): Phi-3 mini scores 69.7%, outperforming Gemma-7B (63.6%) and Mistral-7B (61.7%)
• AGI Eval benchmark (5-shot): Phi-3 mini achieves 39.5%, compared to 42.1% for Gemma-7B and 35.1% for Mistral-7B
• Phi-3.5-mini is competitive with other much larger open-weight models such as Llama-3.1-8B-instruct, Mistral-7B-instruct-v0.3, and Mistral-Nemo-12B-instruct-2407
• AGI Eval benchmark (5-shot): Phi-3 mini achieves 39.5%, compared to 42.1% for Gemma-7B and 35.1% for Mistral-7B
• Phi-3.5-mini is competitive with other much larger open-weight models such as Llama-3.1-8B-instruct, Mistral-7B-instruct-v0.3, and Mistral-Nemo-12B-instruct-2407
Revolutionary Training Approach
The breakthrough was enabled by a highly selective approach to training data. Instead of using massive amounts of data from the internet, Microsoft used carefully selected publicly-available data that was filtered based on educational value and content quality.
The training data includes:
• Synthetic "textbook-like" data for teaching math, coding, and reasoning
• High-quality chat format supervised data
• Filtered publicly available documents
• Code repositories
• Educational content focused on reasoning capabilities
• High-quality chat format supervised data
• Filtered publicly available documents
• Code repositories
• Educational content focused on reasoning capabilities
Key Capabilities and Use Cases
Long Context Tasks:
Phi-3.5-mini supports 128K context length, therefore the model is capable of several long context tasks including long document/meeting summarization, long document QA, long document information retrieval.
Ideal Applications:
• Memory and compute-constrained environments
• Latency-bound scenarios
• Strong reasoning tasks (especially math and logic)
• Code generation and analysis
• Multi-turn conversations
• Multilingual applications
• On-device AI implementations
• Latency-bound scenarios
• Strong reasoning tasks (especially math and logic)
• Code generation and analysis
• Multi-turn conversations
• Multilingual applications
• On-device AI implementations
Mobile and Edge Deployment
One of the most remarkable aspects of small language models like Phi-3 Mini is their ability to run locally on a smartphone. By quantizing the model to 4-bits, researchers were able to reduce its memory footprint to just 1.8GB.
This allowed them to deploy the model on an iPhone 14 with an A16 Bionic chip, running natively on-device and fully offline. Despite the constraints of mobile hardware, Phi-3 Mini managed to generate over 12 tokens per second.
Recent Improvements (June 2024 Update)
Microsoft released a significant update to their Phi-3 mini language models in July 2024, enhancing both the 4K and 128K context versions with substantial improvements across various aspects of the models' capabilities.
Major enhancements included:
• JSON output accuracy increased dramatically from 1.9% to 60.1%
• Improved code understanding and generation capabilities
• Better structured output generation
• Enhanced multi-turn conversation quality
• Explicit support for the system tag
• Significantly improved reasoning capability
• Improved code understanding and generation capabilities
• Better structured output generation
• Enhanced multi-turn conversation quality
• Explicit support for the system tag
• Significantly improved reasoning capability
Availability and Deployment Options
Phi-3 Mini and Phi-3.5 Mini are available through multiple platforms:
• Microsoft Azure AI Studio: Full deployment and fine-tuning capabilities
• Hugging Face: Easy access and integration
• Ollama: Local deployment and testing
• NVIDIA NIM: Microservice deployment with standard API
• ONNX Runtime: Cross-platform optimization including mobile
• Hugging Face: Easy access and integration
• Ollama: Local deployment and testing
• NVIDIA NIM: Microservice deployment with standard API
• ONNX Runtime: Cross-platform optimization including mobile
Limitations and Considerations
While Phi-3 Mini represents significant advancement, users should be aware of certain limitations:
• Phi-3 models do not perform as well on factual knowledge benchmarks (such as TriviaQA) as the smaller model size results in less capacity to retain facts
• Performance may vary across languages
• Limited scope for code: Majority of Phi-3 training data is based in Python and use common packages
• Not suitable for high-risk scenarios without additional assessment
• May generate fabricated content in some cases
• Performance may vary across languages
• Limited scope for code: Majority of Phi-3 training data is based in Python and use common packages
• Not suitable for high-risk scenarios without additional assessment
• May generate fabricated content in some cases
The Future of Small Language Models
Thanks to their smaller size, Phi-3 models can be used in compute-limited inference environments. The smaller size also makes fine-tuning or customization easier and more affordable, with lower computational needs making them a lower cost option with much better latency.
Microsoft's Phi-3 Mini demonstrates that the future of AI isn't just about building larger models, but about creating more efficient, accessible, and practical solutions that can run anywhere from data centers to smartphones. With continuous updates and improvements, Phi-3.5 Mini represents the cutting edge of compact AI technology.
Bottom Line: Phi-3.5 Mini proves that small language models can deliver exceptional performance while remaining accessible and efficient, making advanced AI capabilities available to a broader range of applications and developers than ever before.