Apple today released several open source Large Language Models (LLMs) designed to run on devices rather than cloud servers. The LLM programs are called OpenELM (Open Source Efficient Language Model) and are available here Hug mouth heartA community for sharing AI code.
As shown in the white paper [PDF]There are a total of eight OpenELM models, four of which are pre-trained using CoreNet libraries and four of which are instruction-tuned models. Apple uses a layer scaling technique that aims to improve accuracy and efficiency.
Apple has provided code, training protocols and multiple versions, rather than just the final training model, and the researchers behind the project hope it will lead to faster progress and “more reliable results” in the field of natural language AI.
OpenELM, an open, evolving language model. OpenELM uses a layer scaling technique to efficiently map parameters between each layer of the transformer model, resulting in improved accuracy. For example, with a parameter budget of about 1 billion parameters, OpenELM shows a 2.36% accuracy improvement over OLMO while requiring 2x fewer tokens for pre-training.
Our version moves away from previous practices that only provide model weights, inference codes, and pre-training on private datasets and includes a complete framework for language model training and evaluation on publicly available datasets, including training protocols, multiple checkpoints, and pre-protocols. training points. Training configuration.
Apple says it’s introducing OpenELM models to “strengthen and enrich the open research community” with modern language models Sharing open source models gives researchers the opportunity to explore risk, data and model biases. Developers and organizations can use the templates as they are or modify them.
Open data sharing has become an important tool for Apple to recruit top engineers, scientists, and experts because it provides opportunities for research that would not normally be disclosed under Apple’s privacy policy.
Apple has yet to bring such AI features to its devices, but iOS 18 is expected to include several new AI features, and there are rumors that Apple plans to run its own large language models on the device for privacy reasons.
