Repository: sashabaranov/go-openai
Pull request #462 (2023-07-27 22:53:39)

docs: add Frequently Asked Questions to README.md
**Describe the change**
I've added a "Frequently Asked Questions" section to the README. The aim is to reduce duplicated issues and reduce  communication costs.

**Tests**
None

**Additional context**
Issue: #9, #62 


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/sashabaranov/go-openai/pull/462?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#462](https://app.codecov.io/gh/sashabaranov/go-openai/pull/462?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (9fbcc40) into [master](https://app.codecov.io/gh/sashabaranov/go-openai/commit/1876e0c20716afc4d012688bc393ccd5f28def79?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (1876e0c) will **not change** coverage.
> Report is 1 commits behind head on master.
> The diff coverage is `n/a`.

```diff
@@           Coverage Diff           @@
##           master     #462   +/-   ##
=======================================
  Coverage   97.03%   97.03%           
=======================================
  Files          17       17           
  Lines         709      709           
=======================================
  Hits          688      688           
  Misses         15       15           
  Partials        6        6           
```


[see 2 files with indirect coverage changes](https://app.codecov.io/gh/sashabaranov/go-openai/pull/462/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)


vvatanabe: @sashabaranov Please check it. If I made a mistake in answering the FAQ, I will correct it.

Patch:
diff --git a/README.md b/README.md
index 19aadde..d627a19 100644
--- a/README.md
+++ b/README.md
@@ -694,6 +694,37 @@ OPENAI_TOKEN=XXX go test -v -tags=integration ./api_integration_test.go
 
 If the `OPENAI_TOKEN` environment variable is not available, integration tests will be skipped.
 
+## Frequently Asked Questions
+
+### Why don't we get the same answer when specifying a temperature field of 0 and asking the same question?
+
+Even when specifying a temperature field of 0, it doesn't guarantee that you'll always get the same response. Several factors come into play.
+
+1. Go OpenAI Behavior: When you specify a temperature field of 0 in Go OpenAI, the omitempty tag causes that field to be removed from the request. Consequently, the OpenAI API applies the default value of 1.
+2. Token Count for Input/Output: If there's a large number of tokens in the input and output, setting the temperature to 0 can still result in non-deterministic behavior. In particular, when using around 32k tokens, the likelihood of non-deterministic behavior becomes highest even with a temperature of 0.
+
+Due to the factors mentioned above, different answers may be returned even for the same question.
+
+**Workarounds:**
+1. Using `math.SmallestNonzeroFloat32`: By specifying `math.SmallestNonzeroFloat32` in the temperature field instead of 0, you can mimic the behavior of setting it to 0.
+2. Limiting Token Count: By limiting the number of tokens in the input and output and especially avoiding large requests close to 32k tokens, you can reduce the risk of non-deterministic behavior.
+
+By adopting these strategies, you can expect more consistent results.
+
+**Related Issues:**  
+[omitempty option of request struct will generate incorrect request when parameter is 0.](https://github.com/sashabaranov/go-openai/issues/9)
+
+### Does Go OpenAI provide a method to count tokens?
+
+No, Go OpenAI does not offer a feature to count tokens, and there are no plans to provide such a feature in the future. However, if there's a way to implement a token counting feature with zero dependencies, it might be possible to merge that feature into Go OpenAI. Otherwise, it would be more appropriate to implement it in a dedicated library or repository.
+
+For counting tokens, you might find the following links helpful:  
+- [Counting Tokens For Chat API Calls](https://github.com/pkoukk/tiktoken-go#counting-tokens-for-chat-api-calls)
+- [How to count tokens with tiktoken](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb)
+
+**Related Issues:**  
+[Is it possible to join the implementation of GPT3 Tokenizer](https://github.com/sashabaranov/go-openai/issues/62)
+
 ## Thank you
 
 We want to take a moment to express our deepest gratitude to the [contributors](https://github.com/sashabaranov/go-openai/graphs/contributors) and sponsors of this project:
diff --git a/chat.go b/chat.go
index 514aaee..8d29b32 100644
--- a/chat.go
+++ b/chat.go
@@ -114,6 +114,13 @@ const (
 	FinishReasonNull          FinishReason = "null"
 )
 
+func (r FinishReason) MarshalJSON() ([]byte, error) {
+	if r == FinishReasonNull || r == "" {
+		return []byte("null"), nil
+	}
+	return []byte(`"` + string(r) + `"`), nil // best effort to not break future API changes
+}
+
 type ChatCompletionChoice struct {
 	Index   int                   `json:"index"`
 	Message ChatCompletionMessage `json:"message"`


Test patch:
diff --git a/chat_test.go b/chat_test.go
index 5723d6c..38d66fa 100644
--- a/chat_test.go
+++ b/chat_test.go
@@ -298,3 +298,34 @@ func getChatCompletionBody(r *http.Request) (ChatCompletionRequest, error) {
 	}
 	return completion, nil
 }
+
+func TestFinishReason(t *testing.T) {
+	c := &ChatCompletionChoice{
+		FinishReason: FinishReasonNull,
+	}
+	resBytes, _ := json.Marshal(c)
+	if !strings.Contains(string(resBytes), `"finish_reason":null`) {
+		t.Error("null should not be quoted")
+	}
+
+	c.FinishReason = ""
+
+	resBytes, _ = json.Marshal(c)
+	if !strings.Contains(string(resBytes), `"finish_reason":null`) {
+		t.Error("null should not be quoted")
+	}
+
+	otherReasons := []FinishReason{
+		FinishReasonStop,
+		FinishReasonLength,
+		FinishReasonFunctionCall,
+		FinishReasonContentFilter,
+	}
+	for _, r := range otherReasons {
+		c.FinishReason = r
+		resBytes, _ = json.Marshal(c)
+		if !strings.Contains(string(resBytes), fmt.Sprintf(`"finish_reason":"%s"`, r)) {
+			t.Errorf("%s should be quoted", r)
+		}
+	}
+}