Repository: miniflux/v2 (language: go)

Issue:
UTF8 issue with DaringFireball
For some reason I'm seeing this error now, I never had this before so I'm not sure where it could come from.

Using a self-hosted Miniflux instance:

```
Miniflux

Version: 2.0.18
Build Date: 2019-09-26T03:38:53+0000
```

```
There is a problem with this feed

Unable to create entry "https://daringfireball.net/linked/2019/10/11/rubin-playground" (feed #5): pq: invalid byte sequence for encoding "UTF8": 0xe2 0x80 0x2e
```


Discussion / hints:
<img width="776" alt="Screenshot 2019-10-15 at 21 39 45" src="https://user-images.githubusercontent.com/790262/66864026-91185500-ef94-11e9-87d2-eee6969a6c0c.png">

Screenshot for more context
You can use the XML feed in the meantime. I need to take a closer look to the JSON feed.
Thanks, didn't even realized that there were two feeds. Otherwise I would've tried that first, for now it works on the RSS feed.
The JSON feed is working now. The problematic entry is probably not in the feed anymore.
This error is still happening with https://jlelse.blog/.json. although the JSON Feed seems to be valid. 🤔 
I wonder if its the smart quotes in the summary for your first item @jlelse?

```
I’m riding my electric bike and came across this castle. It’s called “Schloss Richmond”. 🚴‍♂️
```

Both it and the Daring Fireball one have smart quotes of some description. Either `‘` or `“`
The error is "store: unable to update entry "https://jlelse.blog/micro/2021/05/2021-05-30-gipqz": pq: invalid byte sequence for encoding "UTF8": 0xf0 0x9f 0x9a 0x2e". So I suspect it's the emoji, but I can test it later.

Fix patch:
diff --git a/reader/json/json.go b/reader/json/json.go
index 18bbcaee226..65b44e58562 100644
--- a/reader/json/json.go
+++ b/reader/json/json.go
@@ -182,8 +182,11 @@ func getAuthor(author jsonAuthor) string {
 func truncate(str string) string {
 	max := 100
 	str = strings.TrimSpace(str)
-	if len(str) > max {
-		return str[:max] + "..."
+
+	// Convert to runes to be safe with unicode
+	runes := []rune(str)
+	if len(runes) > max {
+		return string(runes[:max]) + "…"
 	}
 
 	return str


Test patch:
diff --git a/reader/json/parser_test.go b/reader/json/parser_test.go
index 46bfba40cc7..81eaf497156 100644
--- a/reader/json/parser_test.go
+++ b/reader/json/parser_test.go
@@ -407,6 +407,41 @@ func TestParseTruncateItemTitle(t *testing.T) {
 	if len(feed.Entries[0].Title) != 103 {
 		t.Errorf("Incorrect entry title, got: %s", feed.Entries[0].Title)
 	}
+
+	if len([]rune(feed.Entries[0].Title)) != 101 {
+		t.Errorf("Incorrect entry title, got: %s", feed.Entries[0].Title)
+	}
+}
+
+func TestParseTruncateItemTitleUnicode(t *testing.T) {
+	data := `{
+		"version": "https://jsonfeed.org/version/1",
+		"title": "My Example Feed",
+		"home_page_url": "https://example.org/",
+		"feed_url": "https://example.org/feed.json",
+		"items": [
+			{
+				"title": "I’m riding my electric bike and came across this castle. It’s called “Schloss Richmond”. 🚴‍♂️"
+			}
+		]
+	}`
+
+	feed, err := Parse("https://example.org/feed.json", bytes.NewBufferString(data))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(feed.Entries) != 1 {
+		t.Errorf("Incorrect number of entries, got: %d", len(feed.Entries))
+	}
+
+	if len(feed.Entries[0].Title) != 110 {
+		t.Errorf("Incorrect entry title, got: %s", feed.Entries[0].Title)
+	}
+
+	if len([]rune(feed.Entries[0].Title)) != 93 {
+		t.Errorf("Incorrect entry title, got: %s", feed.Entries[0].Title)
+	}
 }
 
 func TestParseItemTitleWithXMLTags(t *testing.T) {