Package: gemma-llm
GemmaGemmais a family of open-weights Large Language
Model (LLM) byGoogle DeepMind, based on Gemini
research and technology.This repository contains an inference implementation and examples, based on theFlaxandJAX.Learn more about GemmaTheGemma technical reportdetails the models' capabilities.For tutorials, reference implementations in other ML frameworks, and more,
visithttps://ai.google.dev/gemma.Quick startInstallationTo install Gemma you need to use Python 3.10 or higher.Install JAX for CPU, GPU or TPU. Follow instructions atthe JAX website.Runpython -m venv gemma-demo
. gemma-demo/bin/activate
pip install git+https://github.com/google-deepmind/gemma.gitDownloading the modelsThe model checkpoints are available through Kaggle athttp://kaggle.com/models/google/gemma. Select one of theFlaxmodel
variations, click the ⤓ button to download the model archive, then extract the
contents to a local directory. The archive contains both the model weights and
the tokenizer, for example the 2b Flax variation contains:2b/              # Directory containing model weights
tokenizer.model  # TokenizerRunning the unit testsTo run the unit tests, install the optional[test]dependencies (e.g. usingpip install -e .[test]from the root of the source tree), then:pytest .Note that the tests insampler_test.pyare skipped by default since no
tokenizer is distributed with the Gemma sources. To run these tests, download a
tokenizer following the instructions above, and update the_VOCABconstant insampler_test.pywith the path totokenizer.model.ExamplesTo run the example sampling script, pass the paths to the weights directory and
tokenizer:python examples/sampling.py -- \
  --path_checkpoint=/path/to/archive/contents/2b/ \
  --path_tokenizer=/path/to/archive/contents/tokenizer.modelThere are also several Colab notebook tutorials:colabs/sampling_tutorial.ipynbcontains aColabnotebook with a sampling example.colabs/fine_tuning_tutorial.ipynbcontains aColabwith a basic tutorial on how to fine
tune Gemma for a task, such as English to French translation.colabs/gsm8k_eval.ipynbis aColabwith a reference GSM8K eval
implementation.To run these notebooks you will need to download a local copy of the weights and
tokenizer (see above), and update theckpt_pathandvocab_pathvariables
with the corresponding paths.System RequirementsGemma can run on a CPU, GPU and TPU. For GPU, we recommend a 8GB+ RAM on GPU for
the 2B checkpoint and 24GB+ RAM on GPU for the 7B checkpoint.ContributingWe are open to bug reports, pull requests (PR), and other contributions. Please
seeCONTRIBUTING.mdfor details on PRs.LicenseCopyright 2024 DeepMind Technologies LimitedThis code is licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. You may obtain
a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0.Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.DisclaimerThis is not an official Google product.