Knowledge bases for Amazon Bedrockのチャンキング戦略を確認する

 ナレッジベースのチャンキング戦略がいろいろ増えたので動きを確認していきます。
日本語文書では微妙だったので、英語の<a href="https://www.gutenberg.org/cache/epub/30017/pg30017.txt">エルマーの冒険</a>をS3に格納してチャンキングしていきます。

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3573242/1e1593a7-3ebc-bfad-81da-f1fb990189fa.png)


# デフォルトチャンキング

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3573242/6f63fa7b-0e2b-9da4-ab20-f9dde4f42c57.png)

ナレッジベースを同期後、OpenSearchのダッシュボードのDev Toolsで内容を確認してみます。

```
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt"}""",
```
"AMAZON_BEDROCK_METADATA"にはソースドキュメントの場所が格納されているようです。


<details><summary>冒頭幾つかのチャンクの内容を確認してみます（長いので畳みます）。</summary>

```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """
 The Project Gutenberg eBook of My Father's Dragon
     
 This ebook is for the use of anyone anywhere in the United States and
 most other parts of the world at no cost and with almost no restrictions
 whatsoever. You may copy it, give it away or re-use it under the terms
 of the Project Gutenberg License included with this ebook or online
 at www.gutenberg.org. If you are not located in the United States,
 you will have to check the laws of the country where you are located
 before using this eBook."""
        }
```
```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """
 Title: My Father's Dragon
 
 Author: Ruth Stiles Gannett
 
 Illustrator: Ruth Chrisman Gannett
 
 Release date: September 18, 2009 [eBook #30017]
 
 Language: English
 
 Credits: Produced by Sankar Viswanathan, Greg Weeks, and the Online
         Distributed Proofreading Team at http://www.pgdp.net
 
 
 *** START OF THE PROJECT GUTENBERG EBOOK MY FATHER'S DRAGON ***
 
 
 
 
 Produced by Sankar Viswanathan, Greg Weeks, and the Online
 Distributed Proofreading Team at http://www.pgdp.net
 
 
 
 
 
 
 
 
                          Transcriber's Note:
 
        Extensive research did not uncover any evidence that the
        U.S. copyright on this publication was renewed.
 
 
                           MY FATHER'S DRAGON
 
 
 
                                STORY BY
 
                          RUTH STILES GANNETT
 
 
 
                            ILLUSTRATIONS BY
 
                         RUTH CHRISMAN GANNETT
 
 
 
 
 
                      RANDOM HOUSE   .   NEW YORK
 
 
 
 
 
                  COPYRIGHT 1948 BY RANDOM HOUSE, INC."""
        }
```
```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """
 *       *       *       *       *
 
 
 
 
 For My
 
 FATHER
 
        *       *       *       *       *
 
 
 
 
 CONTENTS
 
 
     1.  My Father Meets the Cat          9
 
     2.  My Father Runs Away             15
 
     3.  My Father Finds the Island      22
 
     4.  My Father Finds the River       31
 
     5.  My Father Meets Some Tigers     39
 
     6.  My Father Meets A Rhinoceros    48
 
     7.  My Father Meets A Lion          56
 
     8.  My Father Meets A Gorilla       63
 
     9.  My Father Makes A Bridge        73
 
    10.  My Father Finds the Dragon      79
 
        *       *       *       *       *
 
 
 
 
 _Chapter One_
 
 MY FATHER MEETS THE CAT
 
 
 One cold rainy day when my father was a little boy, he met an old
 alley cat on his street."""
        }
```
```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """
 My Father Makes A Bridge        73
 
    10.  My Father Finds the Dragon      79
 
        *       *       *       *       *
 
 
 
 
 _Chapter One_
 
 MY FATHER MEETS THE CAT
 
 
 One cold rainy day when my father was a little boy, he met an old
 alley cat on his street. The cat was very drippy and uncomfortable so
 my father said, "Wouldn't you like to come home with me?"
 
 This surprised the cat--she had never before met anyone who cared
 about old alley cats--but she said, "I'd be very much obliged if I
 could sit by a warm furnace, and perhaps have a saucer of milk."
 
 "We have a very nice furnace to sit by," said my father, "and I'm sure
 my mother has an extra saucer of milk."
 
 [Illustration]
 
 My father and the cat became good friends but my father's mother was
 very upset about the cat. She hated cats, particularly ugly old alley
 cats. "Elmer Elevator," she said to my father, "if you think I'm going
 to give that cat a saucer of milk, you're very wrong."""
        }
```

</details>

# Semanticチャンキング

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3573242/320dff7b-9113-a8dd-8bab-9fadd6ac46bd.png)

こちらもOpenSearchのダッシュボードのDev Toolsで内容を確認してみます。
```
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt"}""",
```
"AMAZON_BEDROCK_METADATA"には同様にソースドキュメントの場所が格納されているようです。

<details><summary>こちらも冒頭幾つかのチャンクの内容を確認してみます（長いので畳みます）。</summary>

```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """﻿
The Project Gutenberg eBook of My Father's Dragon
    
This ebook is for the use of anyone anywhere in the United States and
most other parts of the world at no cost and with almost no restrictions
whatsoever. You may copy it, give it away or re-use it under the terms
of the Project Gutenberg License included with this ebook or online
at www.gutenberg.org. If you are not located in the United States,
you will have to check the laws of the country where you are located
before using this eBook.

Title: My Father's Dragon

Author: Ruth Stiles Gannett

Illustrator: Ruth Chrisman Gannett

Release date: September 18, 2009 [eBook #30017]

Language: English

Credits: Produced by Sankar Viswanathan, Greg Weeks, and the Online
        Distributed Proofreading Team at http://www.pgdp.net


*** START OF THE PROJECT GUTENBERG EBOOK MY FATHER'S DRAGON ***




Produced by Sankar Viswanathan, Greg Weeks, and the Online
Distributed Proofreading Team at http://www.pgdp.net








                         Transcriber's Note:

       Extensive research did not uncover any evidence that the
       U.S. copyright on this publication was renewed."""
        }
```
```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": "
MY FATHER'S DRAGON



                    
           STORY BY

                         RUTH STILES GANNETT



                           ILLUSTRATIONS BY

                        RUTH CHRISMAN GANNETT





                     RANDOM HOUSE   .   NEW YORK





                 COPYRIGHT 1948 BY RANDOM HOUSE, INC.

       *       *       *       *       *




For My

FATHER

       *       *       *       *       *




CONTENTS


    1.  My Father Meets the Cat          9

    2.  My Father Runs Away             15

    3.  My Father Finds the Island      22

    4.  My Father Finds the River       31

    5.  My Father Meets Some Tigers     39

    6.  My Father Meets A Rhinoceros    48

    7.  My Father Meets A Lion          56

    8.  My Father Meets A Gorilla       63

    9.  My Father Makes A Bridge        73

   10.  My Father Finds the Dragon      79

       *       *       *       *       *




_Chapter One_

MY FATHER MEETS THE CAT


One cold rainy day when my father was a little boy, he met an old
alley cat on his street. The cat was very drippy and uncomfortable so
my father said, \"Wouldn't you like to come home with me?\""
        }
```
```
        "_source": {
          "AMAZON_BEDROCK_TEXT_CHUNK": """
This surprised the cat--she had never before met anyone who cared
about old alley cats--but she said, "I'd be very much obliged if I
could sit by a warm furnace, and perhaps have a saucer of milk."

"We have a very nice furnace to sit by," said my father, "and I'm sure
my mother has an extra saucer of milk."

[Illustration]

My father and the cat became good friends but my father's mother was
very upset about the cat. She hated cats, particularly ugly old alley
cats. "Elmer Elevator," she said to my father, "if you think I'm going
to give that cat a saucer of milk, you're very wrong. Once you start
feeding stray alley cats you might as well expect to feed every stray
in town, and I am _not_ going to do it!"

This made my father very sad, and he apologized to the cat because his
mother had been so rude. He told the cat to stay anyway, and that
somehow he would bring her a saucer of milk each day. My father fed
the cat for three weeks, but one day his mother found the cat's saucer
in the cellar and she was extremely angry. She whipped my father and
threw the cat out the door, but later on my father sneaked out and
found the cat."""
        }
```
</details>

これが良い感じでチャンクしてくれたのかどうかは何とも言えないですが、冒頭部分についてはデフォルトチャンキングよりも大きく切られていました。但し全体でみると、チャンク数はデフォルト：67チャンク、Semantic：64チャンクとさほど違いはありませんでした。

# Hierarchicalチャンキング

一番確認したかったHierarchicalチャンキングを確認します。

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/3573242/c20f5518-4ce1-46cd-17a2-e73726ebc296.png)

同様にOpenSearchのダッシュボードのDev Toolsで内容を確認したところ、デフォルトチャンキングやSemanticチャンキングと異なり、
"AMAZON_BEDROCK_METADATA"にはソースドキュメントの場所だけでなく、"parentText"にドキュメント自体が格納されています。

```:抜粋
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt",
 "parentText":"﻿
 The Project Gutenberg eBook of My Father's Dragon
 ...
```

<details><summary>全体は長いので畳みます</summary>

```
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt",
 "parentText":"﻿
 The Project Gutenberg eBook of My Father's Dragon
     
 This ebook is for the use of anyone anywhere in the United States and
 most other parts of the world at no cost and with almost no restrictions
 whatsoever. You may copy it, give it away or re-use it under the terms
 of the Project Gutenberg License included with this ebook or online
 at www.gutenberg.org. If you are not located in the United States,
 you will have to check the laws of the country where you are located
 before using this eBook.
 
 Title: My Father's Dragon
 
 Author: Ruth Stiles Gannett
 
 Illustrator: Ruth Chrisman Gannett
 
 Release date: September 18, 2009 [eBook #30017]
 
 Language: English
 
 Credits: Produced by Sankar Viswanathan, Greg Weeks, and the Online
         Distributed Proofreading Team at http://www.pgdp.net
 
 
 *** START OF THE PROJECT GUTENBERG EBOOK MY FATHER'S DRAGON ***
 
 
 
 
 Produced by Sankar Viswanathan, Greg Weeks, and the Online
 Distributed Proofreading Team at http://www.pgdp.net
 
 
 
 
 
 
 
 
                          Transcriber's Note:
 
        Extensive research did not uncover any evidence that the
        U.S. copyright on this publication was renewed.
 
 
                           MY FATHER'S DRAGON
 
 
 
                                STORY BY
 
                          RUTH STILES GANNETT
 
 
 
                            ILLUSTRATIONS BY
 
                         RUTH CHRISMAN GANNETT
 
 
 
 
 
                      RANDOM HOUSE   .   NEW YORK
 
 
 
 
 
                  COPYRIGHT 1948 BY RANDOM HOUSE, INC.
 
        *       *       *       *       *
 
 
 
 
 For My
 
 FATHER
 
        *       *       *       *       *
 
 
 
 
 CONTENTS
 
 
     1.  My Father Meets the Cat          9
 
     2.  My Father Runs Away             15
 
     3.  My Father Finds the Island      22
 
     4.  My Father Finds the River       31
 
     5.  My Father Meets Some Tigers     39
 
     6.  My Father Meets A Rhinoceros    48
 
     7.  My Father Meets A Lion          56
 
     8.  My Father Meets A Gorilla       63
 
     9.  My Father Makes A Bridge        73
 
    10.  My Father Finds the Dragon      79
 
        *       *       *       *       *
 
 
 
 
 _Chapter One_
 
 MY FATHER MEETS THE CAT
 
 
 One cold rainy day when my father was a little boy, he met an old
 alley cat on his street. The cat was very drippy and uncomfortable so
 my father said, \"Wouldn't you like to come home with me?\"
 
 This surprised the cat--she had never before met anyone who cared
 about old alley cats--but she said, \"I'd be very much obliged if I
 could sit by a warm furnace, and perhaps have a saucer of milk.\"
 
 \"We have a very nice furnace to sit by,\" said my father, \"and I'm sure
 my mother has an extra saucer of milk.\"
 
 [Illustration]
 
 My father and the cat became good friends but my father's mother was
 very upset about the cat. She hated cats, particularly ugly old alley
 cats. \"Elmer Elevator,\" she said to my father, \"if you think I'm going
 to give that cat a saucer of milk, you're very wrong. Once you start
 feeding stray alley cats you might as well expect to feed every stray
 in town, and I am _not_ going to do it!\"
 
 This made my father very sad, and he apologized to the cat because his
 mother had been so rude. He told the cat to stay anyway, and that
 somehow he would bring her a saucer of milk each day. My father fed
 the cat for three weeks, but one day his mother found the cat's saucer
 in the cellar and she was extremely angry. She whipped my father and
 threw the cat out the door, but later on my father sneaked out and
 found the cat. Together they went for a walk in the park and tried to
 think of nice things to talk about. My father said, \"When I grow up
 I'm going to have an airplane. Wouldn't it be wonderful to fly just
 anywhere you might think of!\"
 
 \"Would you like to fly very, very much?\" asked the cat.
 
 \"I certainly would. I'd do anything if I could fly.\"
 
 [Illustration]
 
 \"Well,\" said the cat, \"If you'd really like to fly that much, I think
 I know of a sort of a way you might get to fly while you're still a
 little boy.\"
 
 \"You mean you know where I could get an airplane?\"
 
 \"Well, not exactly an airplane, but something even better. As you can
 see, I'm an old cat now, but in my younger days I was quite a
 traveler. My traveling days are over but last spring I took just one
 more trip and sailed to the Island of Tangerina, stopping at the port
 of Cranberry. Well, it just so happened that I missed the boat, and
 while waiting for the next I thought I'd look around a bit. I was
 particularly interested in a place called Wild Island, which we had
 passed on our way to Tangerina. Wild Island and Tangerina are joined
 together by a long string of rocks, but people never go to Wild Island
 because it's mostly jungle and inhabited by very wild animals. So, I
 decided to go across the rocks and explore it for myself. It certainly
 is an interesting place, but I saw something there that made me want
 to weep.\"
 
 [Illustration]
 
 [Illustration]
 
 
 
 
 _Chapter Two_
 
 MY FATHER RUNS AWAY
 
 
 \"Wild Island is practically cut in two by a very wide and muddy
 river,\" continued the cat. \"This river begins near one end of the
 island and flows into the ocean at the other. Now the animals there
 are very lazy, and they used to hate having to go all the way around
 the beginning of this river to get to the other side of the island.
 It made visiting inconvenient and mail deliveries slow, particularly
 during the Christmas rush. Crocodiles could have carried passengers
 and mail across the river, but crocodiles are very moody, and not the
 least bit dependable, and are always looking for something to eat.
 They don't care if the animals have to walk around the river, so
 that's just what the animals did for many years.\"
 
 \"But what does all this have to do with airplanes?\" asked my father,
 who thought the cat was taking an awfully long time to explain.
 
 \"Be patient, Elmer,\" said the cat, and she went on with the story.
 \"One day about four months before I arrived on Wild Island a baby
 dragon fell from a low-flying cloud onto the bank of the river. He was
 too young to fly very well, and besides, he had bruised one wing quite
 badly, so he couldn't get back to his cloud. "}""",


```
</details>

今までのチャンクと比べるとかなり長く、設定したMax parent token size（デフォルトでは1500トークン）まで、
"AMAZON_BEDROCK_METADATA"の"parentText"に格納されているようです。

ベクトル検索に使用するチャンク自体は、
"AMAZON_BEDROCK_TEXT_CHUNK"に格納されています。

<details><summary>長いので畳みます</summary>

```
          "AMAZON_BEDROCK_TEXT_CHUNK": """﻿
 The Project Gutenberg eBook of My Father's Dragon
     
 This ebook is for the use of anyone anywhere in the United States and
 most other parts of the world at no cost and with almost no restrictions
 whatsoever. You may copy it, give it away or re-use it under the terms
 of the Project Gutenberg License included with this ebook or online
 at www.gutenberg.org. If you are not located in the United States,
 you will have to check the laws of the country where you are located
 before using this eBook.
 
 Title: My Father's Dragon
 
 Author: Ruth Stiles Gannett
 
 Illustrator: Ruth Chrisman Gannett
 
 Release date: September 18, 2009 [eBook #30017]
 
 Language: English
 
 Credits: Produced by Sankar Viswanathan, Greg Weeks, and the Online
         Distributed Proofreading Team at http://www.pgdp.net
 
 
 *** START OF THE PROJECT GUTENBERG EBOOK MY FATHER'S DRAGON ***
 
 
 
 
 Produced by Sankar Viswanathan, Greg Weeks, and the Online
 Distributed Proofreading Team at http://www.pgdp.net
 
 
 
 
 
 
 
 
                          Transcriber's Note:
 
        Extensive research did not uncover any evidence that the
        U.S. copyright on this publication was renewed.
 
 
                           MY FATHER'S DRAGON
 
 
 
                                STORY BY
 
                          RUTH STILES GANNETT
 
 
 
                            ILLUSTRATIONS BY
 
                         RUTH CHRISMAN GANNETT
 
 
 
 
 
                      RANDOM HOUSE   .   NEW YORK
 
 
 
 
 
                  COPYRIGHT 1948 BY RANDOM HOUSE, INC.
 
        *       *       *       *       *
 
 
 
 
 For My
 
 FATHER
 
        *       *       *       *       *
 
 
 
 
 CONTENTS
 
 
     1.  My Father Meets the Cat          9
 
     2.  My Father Runs Away             15
 
     3.  My Father Finds the Island      22
 
     4."""
        }
```

</details>

"parentText"よりはかなり短く、子チャンク（デフォルト300トークン）の長さで格納されているようです。

次のチャンクも確認してみます。

"AMAZON_BEDROCK_METADATA"は一つ目のチャンクと同じ内容が格納されています。

```:抜粋
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt",
 "parentText":"﻿
 The Project Gutenberg eBook of My Father's Dragon
 ...
```

<details><summary>全体は長いので畳みます</summary>

```
        "_source": {
          "x-amz-bedrock-kb-source-uri": "s3://***/pg30017.txt",
          "AMAZON_BEDROCK_METADATA": """{"source":"s3://***/pg30017.txt",
 "parentText":"﻿
 The Project Gutenberg eBook of My Father's Dragon
     
 This ebook is for the use of anyone anywhere in the United States and
 most other parts of the world at no cost and with almost no restrictions
 whatsoever. You may copy it, give it away or re-use it under the terms
 of the Project Gutenberg License included with this ebook or online
 at www.gutenberg.org. If you are not located in the United States,
 you will have to check the laws of the country where you are located
 before using this eBook.
 
 Title: My Father's Dragon
 
 Author: Ruth Stiles Gannett
 
 Illustrator: Ruth Chrisman Gannett
 
 Release date: September 18, 2009 [eBook #30017]
 
 Language: English
 
 Credits: Produced by Sankar Viswanathan, Greg Weeks, and the Online
         Distributed Proofreading Team at http://www.pgdp.net
 
 
 *** START OF THE PROJECT GUTENBERG EBOOK MY FATHER'S DRAGON ***
 
 
 
 
 Produced by Sankar Viswanathan, Greg Weeks, and the Online
 Distributed Proofreading Team at http://www.pgdp.net
 
 
 
 
 
 
 
 
                          Transcriber's Note:
 
        Extensive research did not uncover any evidence that the
        U.S. copyright on this publication was renewed.
 
 
                           MY FATHER'S DRAGON
 
 
 
                                STORY BY
 
                          RUTH STILES GANNETT
 
 
 
                            ILLUSTRATIONS BY
 
                         RUTH CHRISMAN GANNETT
 
 
 
 
 
                      RANDOM HOUSE   .   NEW YORK
 
 
 
 
 
                  COPYRIGHT 1948 BY RANDOM HOUSE, INC.
 
        *       *       *       *       *
 
 
 
 
 For My
 
 FATHER
 
        *       *       *       *       *
 
 
 
 
 CONTENTS
 
 
     1.  My Father Meets the Cat          9
 
     2.  My Father Runs Away             15
 
     3.  My Father Finds the Island      22
 
     4.  My Father Finds the River       31
 
     5.  My Father Meets Some Tigers     39
 
     6.  My Father Meets A Rhinoceros    48
 
     7.  My Father Meets A Lion          56
 
     8.  My Father Meets A Gorilla       63
 
     9.  My Father Makes A Bridge        73
 
    10.  My Father Finds the Dragon      79
 
        *       *       *       *       *
 
 
 
 
 _Chapter One_
 
 MY FATHER MEETS THE CAT
 
 
 One cold rainy day when my father was a little boy, he met an old
 alley cat on his street. The cat was very drippy and uncomfortable so
 my father said, \"Wouldn't you like to come home with me?\"
 
 This surprised the cat--she had never before met anyone who cared
 about old alley cats--but she said, \"I'd be very much obliged if I
 could sit by a warm furnace, and perhaps have a saucer of milk.\"
 
 \"We have a very nice furnace to sit by,\" said my father, \"and I'm sure
 my mother has an extra saucer of milk.\"
 
 [Illustration]
 
 My father and the cat became good friends but my father's mother was
 very upset about the cat. She hated cats, particularly ugly old alley
 cats. \"Elmer Elevator,\" she said to my father, \"if you think I'm going
 to give that cat a saucer of milk, you're very wrong. Once you start
 feeding stray alley cats you might as well expect to feed every stray
 in town, and I am _not_ going to do it!\"
 
 This made my father very sad, and he apologized to the cat because his
 mother had been so rude. He told the cat to stay anyway, and that
 somehow he would bring her a saucer of milk each day. My father fed
 the cat for three weeks, but one day his mother found the cat's saucer
 in the cellar and she was extremely angry. She whipped my father and
 threw the cat out the door, but later on my father sneaked out and
 found the cat. Together they went for a walk in the park and tried to
 think of nice things to talk about. My father said, \"When I grow up
 I'm going to have an airplane. Wouldn't it be wonderful to fly just
 anywhere you might think of!\"
 
 \"Would you like to fly very, very much?\" asked the cat.
 
 \"I certainly would. I'd do anything if I could fly.\"
 
 [Illustration]
 
 \"Well,\" said the cat, \"If you'd really like to fly that much, I think
 I know of a sort of a way you might get to fly while you're still a
 little boy.\"
 
 \"You mean you know where I could get an airplane?\"
 
 \"Well, not exactly an airplane, but something even better. As you can
 see, I'm an old cat now, but in my younger days I was quite a
 traveler. My traveling days are over but last spring I took just one
 more trip and sailed to the Island of Tangerina, stopping at the port
 of Cranberry. Well, it just so happened that I missed the boat, and
 while waiting for the next I thought I'd look around a bit. I was
 particularly interested in a place called Wild Island, which we had
 passed on our way to Tangerina. Wild Island and Tangerina are joined
 together by a long string of rocks, but people never go to Wild Island
 because it's mostly jungle and inhabited by very wild animals. So, I
 decided to go across the rocks and explore it for myself. It certainly
 is an interesting place, but I saw something there that made me want
 to weep.\"
 
 [Illustration]
 
 [Illustration]
 
 
 
 
 _Chapter Two_
 
 MY FATHER RUNS AWAY
 
 
 \"Wild Island is practically cut in two by a very wide and muddy
 river,\" continued the cat. \"This river begins near one end of the
 island and flows into the ocean at the other. Now the animals there
 are very lazy, and they used to hate having to go all the way around
 the beginning of this river to get to the other side of the island.
 It made visiting inconvenient and mail deliveries slow, particularly
 during the Christmas rush. Crocodiles could have carried passengers
 and mail across the river, but crocodiles are very moody, and not the
 least bit dependable, and are always looking for something to eat.
 They don't care if the animals have to walk around the river, so
 that's just what the animals did for many years.\"
 
 \"But what does all this have to do with airplanes?\" asked my father,
 who thought the cat was taking an awfully long time to explain.
 
 \"Be patient, Elmer,\" said the cat, and she went on with the story.
 \"One day about four months before I arrived on Wild Island a baby
 dragon fell from a low-flying cloud onto the bank of the river. He was
 too young to fly very well, and besides, he had bruised one wing quite
 badly, so he couldn't get back to his cloud. "}""",


```

</details>

"AMAZON_BEDROCK_TEXT_CHUNK"も確認してみます。

<details><summary>長いので畳みます</summary>

```
          "AMAZON_BEDROCK_TEXT_CHUNK": """
My Father Runs Away             15
 
     3.  My Father Finds the Island      22
 
     4.  My Father Finds the River       31
 
     5.  My Father Meets Some Tigers     39
 
     6.  My Father Meets A Rhinoceros    48
 
     7.  My Father Meets A Lion          56
 
     8.  My Father Meets A Gorilla       63
 
     9.  My Father Makes A Bridge        73
 
    10.  My Father Finds the Dragon      79
 
        *       *       *       *       *
 
 
 
 
 _Chapter One_
 
 MY FATHER MEETS THE CAT
 
 
 One cold rainy day when my father was a little boy, he met an old
 alley cat on his street. The cat was very drippy and uncomfortable so
 my father said, "Wouldn't you like to come home with me?"
 
 This surprised the cat--she had never before met anyone who cared
 about old alley cats--but she said, "I'd be very much obliged if I
 could sit by a warm furnace, and perhaps have a saucer of milk."
 
 "We have a very nice furnace to sit by," said my father, "and I'm sure
 my mother has an extra saucer of milk."
 
 [Illustration]
 
 My father and the cat became good friends but my father's mother was
 very upset about the cat. She hated cats, particularly ugly old alley
 cats. "Elmer Elevator," she said to my father, "if you think I'm going
 to give that cat a saucer of milk, you're very wrong."""
        }
```

</details>

一つ目の"AMAZON_BEDROCK_TEXT_CHUNK"の続きが格納されています。

# まとめ

チャンキング戦略によって、（そりゃそうですが）チャンキングのされ方が変わる事が確認できました。

特にHierarchicalチャンキングについては、ベクトル検索に使用する子チャンクと検索結果として使用する親チャンクでチャンクの長さを変える事が出来るので、かなり便利に使えそうです。

今回はテキストファイルを使用した関係か日本語ではちょっと微妙で、英語のテキストを使ったのですが、日本語PDFを使って良い感じにした記事は以下

https://qiita.com/cyberBOSE/items/44515b0989b79ecfcda0