joey00072 commited on
Commit
2492040
1 Parent(s): 10c4986

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -8,10 +8,10 @@ ckpt will be out_* files trained 2 models 1 with regular softmax
8
 
9
  REPO: https://github.com/joey00072/llama2.c/tree/ghostmax
10
 
11
- softmax: https://wandb.ai/shubhamchoudhari00072/ghostmax/reports/loss-val-23-08-25-21-48-58---Vmlldzo1MjM0MjUw
12
- ghostmax: https://wandb.ai/shubhamchoudhari00072/ghostmax/reports/loss-val-23-08-25-21-50-15---Vmlldzo1MjM0MjYw
13
- <iframe src="https://wandb.ai/shubhamchoudhari00072/ghostmax/runs/cbeei9uh?workspace=user-shubhamchoudhari00072" style="border:none;height:1024px;width:100%">
14
-
15
  ```python
16
  def softmax(x, dim=None):
17
  e_x = torch.exp(x - torch.max(x, dim=dim, keepdim=True)[0])
@@ -22,5 +22,7 @@ def ghostmax(x, dim=None):
22
  return e_x / (1+e_x.sum(dim=dim, keepdim=True) )
23
 
24
  ```
25
-
 
 
26
 
 
8
 
9
  REPO: https://github.com/joey00072/llama2.c/tree/ghostmax
10
 
11
+ softmax: https://wandb.ai/shubhamchoudhari00072/ghostmax/reports/loss-val-23-08-25-21-48-58---Vmlldzo1MjM0MjUw <br>
12
+ ghostmax: https://wandb.ai/shubhamchoudhari00072/ghostmax/reports/loss-val-23-08-25-21-50-15---Vmlldzo1MjM0MjYw <br>
13
+ <br>
14
+
15
  ```python
16
  def softmax(x, dim=None):
17
  e_x = torch.exp(x - torch.max(x, dim=dim, keepdim=True)[0])
 
22
  return e_x / (1+e_x.sum(dim=dim, keepdim=True) )
23
 
24
  ```
25
+ <br>
26
+ <iframe src="https://wandb.ai/shubhamchoudhari00072/ghostmax/runs/cbeei9uh?workspace=user-shubhamchoudhari00072" style="border:none;height:1024px;width:100%">
27
+ <br>
28