Mayank Mishra commited on
Commit
2ab68a5
1 Parent(s): 64765e9

update readme

Browse files
Files changed (1) hide show
  1. README.md +18 -21
README.md CHANGED
@@ -369,7 +369,7 @@ Citation
369
  <!-- ![granite](https://github.com/ibm-granite/granite-code-models/blob/main/figures/granite.png) -->
370
 
371
  ## Model Summary
372
- **Granite 3B Code Base** model is a decoder-only code model designed for code generative tasks (e.g., code generation, code explanation, code fixing). It was trained from scratch on 4 trillion tokens sourced from 116 programming languages, ensuring a comprehensive understanding of programming languages and syntax.
373
 
374
  - **Developers:** IBM Research
375
  - **GitHub Repository:** [ibm-granite/granite-code-models](https://github.com/ibm-granite/granite-code-models)
@@ -383,44 +383,41 @@ for Code Intelligence](https://)
383
  Prominent enterprise use cases of LLMs in software engineering productivity include code generation, code explanation, code fixing, generating unit tests, generating documentation, addressing technical debt issues, vulnerability detection, code translation, and more. All Granite Code Base models, including the **3B parameters model**, are able to handle these tasks as they were trained on a large amount of code data from 116 programming languages.
384
 
385
  ### Generation
386
- Before proceeding, you need to install the necessary dependencies. You can do this by running the following command:
387
- ```
388
- pip install -r requirements.txt
389
- ```
390
-
391
  This is a simple example of how to use Granite Code Base 3B model.
392
 
393
  ```python
394
  import torch
395
  from transformers import AutoModelForCausalLM, AutoTokenizer
396
 
 
397
  model_path = "ibm-granite/granite-3b-code-base"
398
 
399
  tokenizer = AutoTokenizer.from_pretrained(model_path)
400
- model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cuda")
 
 
401
  model.eval()
402
 
 
403
  input_text = "def generate():"
404
 
 
405
  input_tokens = tokenizer(input_text, return_tensors="pt")
 
 
406
  for i in input_tokens:
407
- input_tokens[i] = input_tokens[i].cuda()
408
 
 
409
  output = model.generate(**input_tokens)
 
 
410
  output = tokenizer.batch_decode(output)
 
 
411
  for i in output:
412
  print(output)
413
  ```
414
- ### Fill-in-the-middle
415
-
416
- Fill-in-the-middle uses special tokens to identify the prefix/middle/suffix part of the input and output:
417
-
418
- ```python
419
- input_text = "<fim_prefix>def print_hello_world():\n <fim_suffix>\n print('Hello world!')<fim_middle>"
420
- inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
421
- outputs = model.generate(inputs)
422
- print(tokenizer.decode(outputs[0]))
423
- ```
424
 
425
  ## Training Data
426
  - **Data Collection and Filtering:** Pretraining code data is sourced from a combination of publicly available datasets (e.g., [GitHub Code Clean](https://huggingface.co/datasets/codeparrot/github-code-clean), [Starcoder data](https://huggingface.co/datasets/bigcode/starcoderdata)), and additional public code repositories and issues from GitHub. We filter raw data to retain a list of 116 programming languages. After language filtering, we also filter out low-quality code.
@@ -429,10 +426,10 @@ print(tokenizer.decode(outputs[0]))
429
  - **Natural Language Datasets:** In addition to collecting code data for model training, we curate several publicly available high-quality natural language datasets to improve models' proficiency in language understanding and mathematical reasoning. Unlike the code data, we do not deduplicate these datasets.
430
 
431
  ## Infrastructure
432
- We train the Granite Code models using two of IBMs super computing clusters, namely Vela and Blue Vela, both outfitted with NVIDIA A100 and H100 GPUs respectively. These clusters provide a scalable and efficient infrastructure for training our models over thousands of GPUs.
433
 
434
  ## Limitations
435
- Large Language Models are often prone to generating incorrect information, typically referred to as hallucinations. **Granite 3B Code Base** model is not the exception in this regard. Even though this model is suited for code-related tasks as it is trained on source code from 116 programming languages, the generated code is not guaranteed to work as intended. It can be inefficient and contain bugs or exploits. Moreover, Granite Code Base models are _not_ instruction-following models. Thus, commands like *"Write a function that computes the square root"* may not work well.
436
 
437
  ## Citation
438
  ```
@@ -444,4 +441,4 @@ Large Language Models are often prone to generating incorrect information, typic
444
  year = {2024},
445
  url = {https://arxiv.org/abs/0000.00000},
446
  }
447
- ```
 
369
  <!-- ![granite](https://github.com/ibm-granite/granite-code-models/blob/main/figures/granite.png) -->
370
 
371
  ## Model Summary
372
+ **Granite 3B Code Base** is a decoder-only code model designed for code generative tasks (e.g., code generation, code explanation, code fixing). It was trained from scratch on 4 trillion tokens sourced from 116 programming languages, ensuring a comprehensive understanding of programming languages and syntax.
373
 
374
  - **Developers:** IBM Research
375
  - **GitHub Repository:** [ibm-granite/granite-code-models](https://github.com/ibm-granite/granite-code-models)
 
383
  Prominent enterprise use cases of LLMs in software engineering productivity include code generation, code explanation, code fixing, generating unit tests, generating documentation, addressing technical debt issues, vulnerability detection, code translation, and more. All Granite Code Base models, including the **3B parameters model**, are able to handle these tasks as they were trained on a large amount of code data from 116 programming languages.
384
 
385
  ### Generation
 
 
 
 
 
386
  This is a simple example of how to use Granite Code Base 3B model.
387
 
388
  ```python
389
  import torch
390
  from transformers import AutoModelForCausalLM, AutoTokenizer
391
 
392
+ device = "cuda" # or "cpu"
393
  model_path = "ibm-granite/granite-3b-code-base"
394
 
395
  tokenizer = AutoTokenizer.from_pretrained(model_path)
396
+
397
+ # drop device_map if running on CPU
398
+ model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
399
  model.eval()
400
 
401
+ # change input text as desired
402
  input_text = "def generate():"
403
 
404
+ # tokenize the text
405
  input_tokens = tokenizer(input_text, return_tensors="pt")
406
+
407
+ # transfer tokenized inputs to the device
408
  for i in input_tokens:
409
+ input_tokens[i] = input_tokens[i].to(device)
410
 
411
+ # generate output tokens
412
  output = model.generate(**input_tokens)
413
+
414
+ # decode output tokens into text
415
  output = tokenizer.batch_decode(output)
416
+
417
+ # loop over the batch to print, in this example the batch size is 1
418
  for i in output:
419
  print(output)
420
  ```
 
 
 
 
 
 
 
 
 
 
421
 
422
  ## Training Data
423
  - **Data Collection and Filtering:** Pretraining code data is sourced from a combination of publicly available datasets (e.g., [GitHub Code Clean](https://huggingface.co/datasets/codeparrot/github-code-clean), [Starcoder data](https://huggingface.co/datasets/bigcode/starcoderdata)), and additional public code repositories and issues from GitHub. We filter raw data to retain a list of 116 programming languages. After language filtering, we also filter out low-quality code.
 
426
  - **Natural Language Datasets:** In addition to collecting code data for model training, we curate several publicly available high-quality natural language datasets to improve models' proficiency in language understanding and mathematical reasoning. Unlike the code data, we do not deduplicate these datasets.
427
 
428
  ## Infrastructure
429
+ We train the Granite Code models using two of IBM's super computing clusters, namely Vela and Blue Vela, both outfitted with NVIDIA A100 and H100 GPUs respectively. These clusters provide a scalable and efficient infrastructure for training our models over thousands of GPUs.
430
 
431
  ## Limitations
432
+ Large Language Models are often prone to generating incorrect information, typically referred to as hallucinations. **Granite 3B Code Base** model is not the exception in this regard. Even though this model is suited for code-related tasks as it is trained on source code from 116 programming languages, the generated code is not guaranteed to work as intended. It can be inefficient and can also contain bugs or exploits. Moreover, Granite Code Base models are **NOT** instruction-following models. Thus, commands like *"Write a function that computes the square root"* may not work well. The model is best treated as a code completion or code infilling model.
433
 
434
  ## Citation
435
  ```
 
441
  year = {2024},
442
  url = {https://arxiv.org/abs/0000.00000},
443
  }
444
+ ```