Hector Salvador [Fisharp] commited on
Commit
da23173
1 Parent(s): 4f9e345

Use of a proper .js file for click action scripts

Browse files

(instead of the python file with js scripts in variables)
Including changes to fuctions for loading files

README.md CHANGED
@@ -21,8 +21,6 @@ This is a demo playground to generate code with the power of ⭐[StarCoder](http
21
 
22
  🗣️For instruction and chatting you can chat with a prompted version of the model directly at the [HuggingFace🤗Chat💬(hf.co/chat)](https://huggingface.co/chat/?model=starcoder)
23
 
24
- ![StarCoder](https://huggingface.co/datasets/bigcode/admin/resolve/main/StarCoderBanner.png)
25
-
26
  ---
27
 
28
  **Intended Use**: this app and its [supporting model](https://huggingface.co/bigcode/starcoder) are provided for demonstration purposes only; not to serve as a replacement for human expertise. For more details on the model's limitations in terms of factuality and biases, please refer to the source [model card](hf.co/bigcode)
@@ -30,3 +28,51 @@ This is a demo playground to generate code with the power of ⭐[StarCoder](http
30
  ⚠️ Any use or sharing of this demo constitutes your acceptance of the BigCode [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) License Agreement and the use restrictions included within.
31
 
32
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  🗣️For instruction and chatting you can chat with a prompted version of the model directly at the [HuggingFace🤗Chat💬(hf.co/chat)](https://huggingface.co/chat/?model=starcoder)
23
 
 
 
24
  ---
25
 
26
  **Intended Use**: this app and its [supporting model](https://huggingface.co/bigcode/starcoder) are provided for demonstration purposes only; not to serve as a replacement for human expertise. For more details on the model's limitations in terms of factuality and biases, please refer to the source [model card](hf.co/bigcode)
 
28
  ⚠️ Any use or sharing of this demo constitutes your acceptance of the BigCode [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) License Agreement and the use restrictions included within.
29
 
30
  ---
31
+
32
+ ## Model Formats
33
+
34
+ The model is pretrained on code and is formatted with special tokens in addition to the pure code data,\
35
+ such as prefixes specifying the source of the file or tokens separating code from a commit message.\
36
+ Use these templates to explore the model's capacities:
37
+
38
+ ### 1. Prefixes 🏷️
39
+
40
+ For pure code files, use any combination of the following prefixes:
41
+
42
+ ```xml
43
+ <reponame>REPONAME<filename>FILENAME<gh_stars>STARS\ncode<|endoftext|>
44
+ ```
45
+
46
+ STARS can be one of: 0, 1-10, 10-100, 100-1000, 1000+
47
+
48
+ ### 2. Commits 💾
49
+
50
+ The commits data is formatted as follows:
51
+
52
+ ```xml
53
+ <commit_before>code<commit_msg>text<commit_after>code<|endoftext|>
54
+ ```
55
+
56
+ ### 3. Jupyter Notebooks 📓
57
+
58
+ The model is trained on Jupyter notebooks as Python scripts and structured formats like:
59
+
60
+ ```xml
61
+ <start_jupyter><jupyter_text>text<jupyter_code>code<jupyter_output>output<jupyter_text>
62
+ ```
63
+
64
+ ### 4. Issues 🐛
65
+
66
+ We also trained on GitHub issues using the following formatting:
67
+
68
+ ```xml
69
+ <issue_start><issue_comment>text<issue_comment>...<issue_closed>
70
+ ```
71
+
72
+ ### 5. Fill-in-the-middle 🧩
73
+
74
+ Fill in the middle requires rearranging the model inputs. The playground handles this for you - all you need is to specify where to fill:
75
+
76
+ ```xml
77
+ code before<FILL_HERE>code after
78
+ ```
app.py CHANGED
@@ -8,8 +8,6 @@ from gradio.themes.utils import sizes
8
  from text_generation import Client
9
  from src.request import StarCoderRequest, StarCoderRequestConfig
10
 
11
- # todo: remove and replace by the actual js file instead
12
- from src.share_btn import (share_js)
13
  from src.utils import (
14
  get_file_as_string,
15
  get_sections,
@@ -50,21 +48,20 @@ preview("StarCoder Model URL", API_URL_STAR)
50
  preview("StarCoderBase Model URL", API_URL_BASE)
51
  preview("HF Token", HF_TOKEN, ofuscate=True)
52
 
53
- # Loads the whole content of the formats.md file
54
- # and stores it into the FORMATS variable
55
- STATIC_PATH = "static"
56
- FORMATS = get_file_as_string("formats.md", path=STATIC_PATH)
57
- CSS = get_file_as_string("styles.css", path=STATIC_PATH)
58
- community_icon_svg = get_file_as_string("community_icon.svg", path=STATIC_PATH)
59
- loading_icon_svg = get_file_as_string("loading_icon.svg", path=STATIC_PATH)
60
-
61
- # todo: evaluate making STATIC_PATH the default path instead of the current one
62
- README = get_file_as_string("README.md")
63
-
64
- # Slicing the different sections from the README
65
- readme_sections = get_sections(README, "---")
66
-
67
- manifest, description, disclaimer = readme_sections[:3]
68
 
69
  theme = gr.themes.Monochrome(
70
  primary_hue="indigo",
@@ -72,7 +69,7 @@ theme = gr.themes.Monochrome(
72
  neutral_hue="slate",
73
  radius_size=sizes.radius_sm,
74
  font=[
75
- gr.themes.GoogleFont("Rubik"),
76
  "ui-sans-serif",
77
  "system-ui",
78
  "sans-serif",
@@ -159,7 +156,7 @@ examples = [
159
  "def alternating(list1, list2):\n results = []\n for i in range(min(len(list1), len(list2))):\n results.append(list1[i])\n results.append(list2[i])\n if len(list1) > len(list2):\n <FILL_HERE>\n else:\n results.extend(list2[i+1:])\n return results",
160
  ]
161
 
162
- with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
163
  with gr.Column():
164
  gr.Markdown(description)
165
  with gr.Row():
@@ -223,8 +220,8 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
223
  )
224
  gr.Markdown(disclaimer)
225
  with gr.Group(elem_id="share-btn-container"):
226
- community_icon = gr.HTML(community_icon_svg, visible=True)
227
- loading_icon = gr.HTML(loading_icon_svg, visible=True)
228
  share_button = gr.Button(
229
  "Share to community", elem_id="share-btn", visible=True
230
  )
@@ -235,7 +232,7 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
235
  fn=process_example,
236
  outputs=[output],
237
  )
238
- gr.Markdown(FORMATS)
239
 
240
  submit.click(
241
  generate,
@@ -245,6 +242,6 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
245
  max_batch_size=8,
246
  show_progress=True
247
  )
248
- share_button.click(None, [], [], _js=share_js)
249
 
250
  demo.queue(concurrency_count=16).launch(debug=True, server_port=DEFAULT_PORT)
 
8
  from text_generation import Client
9
  from src.request import StarCoderRequest, StarCoderRequestConfig
10
 
 
 
11
  from src.utils import (
12
  get_file_as_string,
13
  get_sections,
 
48
  preview("StarCoderBase Model URL", API_URL_BASE)
49
  preview("HF Token", HF_TOKEN, ofuscate=True)
50
 
51
+ _styles = get_file_as_string("styles.css")
52
+ _script = get_file_as_string("community-btn.js")
53
+ _sharing_icon_svg = get_file_as_string("community-icon.svg")
54
+ _loading_icon_svg = get_file_as_string("loading-icon.svg")
55
+
56
+ # Loads the whole content of the ./README.md file
57
+ # slicing/unpacking its different sections into their proper variables
58
+ readme_file_content = get_file_as_string("README.md", path='./')
59
+ (
60
+ manifest,
61
+ description,
62
+ disclaimer,
63
+ formats,
64
+ ) = get_sections(readme_file_content, "---", up_to=4)
 
65
 
66
  theme = gr.themes.Monochrome(
67
  primary_hue="indigo",
 
69
  neutral_hue="slate",
70
  radius_size=sizes.radius_sm,
71
  font=[
72
+ gr.themes.GoogleFont("IBM Plex Sans", [400, 600]),
73
  "ui-sans-serif",
74
  "system-ui",
75
  "sans-serif",
 
156
  "def alternating(list1, list2):\n results = []\n for i in range(min(len(list1), len(list2))):\n results.append(list1[i])\n results.append(list2[i])\n if len(list1) > len(list2):\n <FILL_HERE>\n else:\n results.extend(list2[i+1:])\n return results",
157
  ]
158
 
159
+ with gr.Blocks(theme=theme, analytics_enabled=False, css=_styles) as demo:
160
  with gr.Column():
161
  gr.Markdown(description)
162
  with gr.Row():
 
220
  )
221
  gr.Markdown(disclaimer)
222
  with gr.Group(elem_id="share-btn-container"):
223
+ community_icon = gr.HTML(_sharing_icon_svg, visible=True)
224
+ loading_icon = gr.HTML(_loading_icon_svg, visible=True)
225
  share_button = gr.Button(
226
  "Share to community", elem_id="share-btn", visible=True
227
  )
 
232
  fn=process_example,
233
  outputs=[output],
234
  )
235
+ gr.Markdown(formats)
236
 
237
  submit.click(
238
  generate,
 
242
  max_batch_size=8,
243
  show_progress=True
244
  )
245
+ share_button.click(None, [], [], _js=_script)
246
 
247
  demo.queue(concurrency_count=16).launch(debug=True, server_port=DEFAULT_PORT)
settings.py CHANGED
@@ -5,6 +5,8 @@ DEFAULT_STARCODER_BASE_API_PATH = "bigcode/starcoderbase/"
5
  FIM_INDICATOR = "<FILL_HERE>"
6
  DEFAULT_PORT = 7860
7
 
 
 
8
  DEFAULT_SETTINGS = dict(
9
  temperature = 0.9,
10
  max_new_tokens = 256,
 
5
  FIM_INDICATOR = "<FILL_HERE>"
6
  DEFAULT_PORT = 7860
7
 
8
+ STATIC_PATH = "static"
9
+
10
  DEFAULT_SETTINGS = dict(
11
  temperature = 0.9,
12
  max_new_tokens = 256,
src/utils.py CHANGED
@@ -2,8 +2,10 @@ import os
2
  from typing import List
3
  from urllib.parse import urljoin
4
 
5
- from settings import DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL
6
-
 
 
7
 
8
  def masked(value: str, n_shown: int, length: int = None) -> str:
9
  """Returns a string with the first and last n_shown characters
@@ -61,11 +63,11 @@ def get_url_from_env_or_default_path(env_name: str, api_path: str) -> str:
61
  DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL, api_path
62
  )
63
 
64
- def get_file_as_string(file_name, path='.') -> str:
65
  """Loads the content of a file given its name
66
  and returns all of its lines as a single string
67
  if a file path is given, it will be used
68
- instead of the current directory
69
 
70
  Args:
71
  file_name (_type_): The name of the file to load.
@@ -78,16 +80,18 @@ def get_file_as_string(file_name, path='.') -> str:
78
  return f.read()
79
 
80
 
81
- def get_sections(string: str, delimiter: str) -> List[str]:
82
  """Splits a string into sections given a delimiter
83
 
84
  Args:
85
  string (str): The string to split
86
  delimiter (str): The delimiter to use
 
 
87
 
88
  Returns:
89
- List[str]: The list of sections
90
  """
91
  return [section.strip()
92
  for section in string.split(delimiter)
93
- if (section and not section.isspace())]
 
2
  from typing import List
3
  from urllib.parse import urljoin
4
 
5
+ from settings import (
6
+ DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL,
7
+ STATIC_PATH,
8
+ )
9
 
10
  def masked(value: str, n_shown: int, length: int = None) -> str:
11
  """Returns a string with the first and last n_shown characters
 
63
  DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL, api_path
64
  )
65
 
66
+ def get_file_as_string(file_name, path=STATIC_PATH) -> str:
67
  """Loads the content of a file given its name
68
  and returns all of its lines as a single string
69
  if a file path is given, it will be used
70
+ instead of the default static path (from settings)
71
 
72
  Args:
73
  file_name (_type_): The name of the file to load.
 
80
  return f.read()
81
 
82
 
83
+ def get_sections(string: str, delimiter: str, up_to: int = None) -> List[str]:
84
  """Splits a string into sections given a delimiter
85
 
86
  Args:
87
  string (str): The string to split
88
  delimiter (str): The delimiter to use
89
+ up_to (int, optional): The maximum number of sections to return.
90
+ Defaults to None (which means all sections)
91
 
92
  Returns:
93
+ List[str]: The list of sections (up to the given limit, if any provided)
94
  """
95
  return [section.strip()
96
  for section in string.split(delimiter)
97
+ if (section and not section.isspace())][:up_to]
src/share_btn.py → static/community-btn.js RENAMED
@@ -1,4 +1,4 @@
1
- share_js = """async () => {
2
  async function uploadFile(file){
3
  const UPLOAD_URL = 'https://huggingface.co/uploads';
4
  const response = await fetch(UPLOAD_URL, {
@@ -72,4 +72,4 @@ ${outputTxt}`;
72
  shareBtnEl.style.removeProperty('pointer-events');
73
  shareIconEl.style.removeProperty('display');
74
  loadingIconEl.style.display = 'none';
75
- }"""
 
1
+ async () => {
2
  async function uploadFile(file){
3
  const UPLOAD_URL = 'https://huggingface.co/uploads';
4
  const response = await fetch(UPLOAD_URL, {
 
72
  shareBtnEl.style.removeProperty('pointer-events');
73
  shareIconEl.style.removeProperty('display');
74
  loadingIconEl.style.display = 'none';
75
+ }
static/{community_icon.svg → community-icon.svg} RENAMED
File without changes
static/formats.md DELETED
@@ -1,47 +0,0 @@
1
- ## Model Formats
2
-
3
- The model is pretrained on code and is formatted with special tokens in addition to the pure code data,\
4
- such as prefixes specifying the source of the file or tokens separating code from a commit message.\
5
- Use these templates to explore the model's capacities:
6
-
7
- ### 1. Prefixes 🏷️
8
-
9
- For pure code files, use any combination of the following prefixes:
10
-
11
- ```
12
- <reponame>REPONAME<filename>FILENAME<gh_stars>STARS\ncode<|endoftext|>
13
- ```
14
-
15
- STARS can be one of: 0, 1-10, 10-100, 100-1000, 1000+
16
-
17
- ### 2. Commits 💾
18
-
19
- The commits data is formatted as follows:
20
-
21
- ```
22
- <commit_before>code<commit_msg>text<commit_after>code<|endoftext|>
23
- ```
24
-
25
- ### 3. Jupyter Notebooks 📓
26
-
27
- The model is trained on Jupyter notebooks as Python scripts and structured formats like:
28
-
29
- ```
30
- <start_jupyter><jupyter_text>text<jupyter_code>code<jupyter_output>output<jupyter_text>
31
- ```
32
-
33
- ### 4. Issues 🐛
34
-
35
- We also trained on GitHub issues using the following formatting:
36
-
37
- ```
38
- <issue_start><issue_comment>text<issue_comment>...<issue_closed>
39
- ```
40
-
41
- ### 5. Fill-in-the-middle 🧩
42
-
43
- Fill in the middle requires rearranging the model inputs. The playground handles this for you - all you need is to specify where to fill:
44
-
45
- ```
46
- code before<FILL_HERE>code after
47
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
static/{loading_icon.svg → loading-icon.svg} RENAMED
File without changes
static/styles.css CHANGED
@@ -1,3 +1,9 @@
 
 
 
 
 
 
1
  .generating {
2
  visibility: hidden
3
  }
@@ -44,7 +50,7 @@ a {
44
  justify-content: center;
45
  align-items: center;
46
  border-radius: 9999px !important;
47
- width: 13rem;
48
  }
49
 
50
  #share-btn {
 
1
+ @import url('https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;600;700&display=swap');
2
+
3
+ h1, h2 {
4
+ font-family: 'IBM Plex Mono', sans-serif;
5
+ }
6
+
7
  .generating {
8
  visibility: hidden
9
  }
 
50
  justify-content: center;
51
  align-items: center;
52
  border-radius: 9999px !important;
53
+ width: 15rem;
54
  }
55
 
56
  #share-btn {