New Proposal：String Substring Search in Bitcoin Script - OP_ISSUBSTR

New Proposal：String Substring Search in Bitcoin Script - OP_ISSUBSTR
2025-03-17 16:54 ` Peter Todd
  
From: weichu deng @ 2025-03-17 16:14 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 4263 bytes --]



Dear fellow Bitcoin developers,

 

I am pleased to present a new BIP proposal. This proposal introduces a new 
opcode for Bitcoin scripts: OP_ISSUBSTR.


*Abstract*

This BIP introduces two string opcodes, OP_ISSUBSTR and OP_ISSUBSTRVERIFY 
(similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
determine whether one string is a substring of another. As these opcodes do 
not alter any blockchain state, they are secure.

 

*Specification*

These opcodes check if the second string on the stack is a substring of the 
first string. If the opcode is OP_ISSUBSTRVERIFY, it verifies the condition 
and throws an error if false, without retaining the result.

 

*Execution Process*

   1. Take the two strings at the top of the stack. 
   2. Use standard library functions to compare the two strings. 
   3. Pop the two strings from the stack and push the result onto the stack. 
   4. If the opcode is OP_ISSUBSTRVERIFY, do not push the result. 

*Motivation*

The absence of string operations in Bitcoin scripts restricts its 
applicability. When developers need string operations for applications, 
they must simulate these functions through off-chain preprocessing or 
complex scripts, increasing development difficulty and potentially 
introducing centralized dependencies.

 

Early Bitcoin versions supported some string operations, such as OP_SUBSTR, 
which extracted a substring of specified position and length from a string, 
replacing the original string. For security reasons, OP_SUBSTR was disabled 
in Bitcoin v0.3.10 and later versions due to a vulnerability 
(CVE-2010-5137) caused by OP_LSHIFT. To prevent similar overflow 
vulnerabilities, Bitcoin disabled several opcodes, including OP_SUBSTR. As 
Bitcoin adoption grows, the limitations of lacking string operations have 
become more evident. Our proposed OP_ISSUBSTR adds string search 
functionality to Bitcoin scripts without changing any state, making it safe.

 

*Advantages *

   1. *Enhanced Script Functionality and Flexibility* Developers can 
   process string logic on-chain without off-chain reliance. For example, in 
   multi-signature wallets, developers can verify specific signer information 
   or remarks directly in scripts using OP_ISSUBSTR to check transaction 
   comments or signature fields for particular substrings. 
   2. *Support for String Searching* In some scenarios, developers need to 
   verify if parts of a string match a format or contain specific data. For 
   example, checking if a payment transaction's payee name matches a preset 
   value. 
   3. *Conversion of Non-deterministic Algorithms to Deterministic Ones* 
   Some signature algorithms or hash functions produce non-deterministic 
   outputs. OP_ISSUBSTR allows developers to check if these outputs contain 
   known substrings in scripts, converting non-deterministic results to 
   deterministic ones. For example, verifying if a hash value contains a 
   specific hexadecimal sequence (like "0000") to trigger contract logic. 
   4. *Simplified Address Verification Logic* Bitcoin addresses typically 
   start with specific prefixes or suffixes. OP_ISSUBSTR enables direct 
   address format verification in scripts. For example, checking if a 
   transaction target address starts with "bc1" to ensure validity or detect 
   "address pollution" attacks. 
   5. *Integration with Modern Programming Languages* Modern languages 
   widely support string operations. OP_ISSUBSTR makes Bitcoin scripts more 
   aligned with these languages, lowering the barrier for developers. 

 

We have provided detailed documentation and a reference implementation in 
the BIP draft. You can read the full proposal here: 
https://github.com/Weichu-Deng/bips/blob/OP_ISSUBSTR/bip-yongdong%20wu-OP_ISSUBSTR.md


Thank you for your feedback! 

With respect,

Weichu Deng

 weichudeng@stu2024•jnu.edu.cn

 

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/f844a85e-8be8-4429-8687-bc19dd4b96ffn%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 6308 bytes --]

 permalink raw reply	[|nested]
2025-03-17 16:14 [bitcoindev] New Proposal：String Substring Search in Bitcoin Script - OP_ISSUBSTR weichu deng

  2025-03-18 15:32   ` weichu deng
  2025-03-18 16:41   ` Erik Aronesty
  
From: Peter Todd @ 2025-03-17 16:54 UTC (permalink / raw)
  To: weichu deng;  Bitcoin Development Mailing List

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote:
> 
> 
> Dear fellow Bitcoin developers,
> 
>  
> 
> I am pleased to present a new BIP proposal. This proposal introduces a new 
> opcode for Bitcoin scripts: OP_ISSUBSTR.
> 
> 
> *Abstract*
> 
> This BIP introduces two string opcodes, OP_ISSUBSTR and OP_ISSUBSTRVERIFY 
> (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
> determine whether one string is a substring of another. As these opcodes do 
> not alter any blockchain state, they are secure.

Bitcoin scripts are about validation. Not computation.

This means that substring search and concatenation are equivalent. For
every script that validates a substring search, you can instead
concatenate the substring with the rest of the string, and validate
equality instead.

Basically speaking:

	foobar foo IsSubStr

is equivalent to:

	foobar foo bar Cat Equal

A real-world example would be more complex. But I hope that illustrates
my point sufficiently.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/Z9hTu2TjMlLr5-Eg%40petertodd.org.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

 permalink raw reply	[|nested]
2025-03-17 16:54 ` Peter Todd

  2025-03-18 21:33     ` Rijndael
  2025-03-18 16:41   ` Erik Aronesty
  
From: weichu deng @ 2025-03-18 15:32 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 2445 bytes --]

Hi, Peter Todd
Thanks for your feedback. I agree that "Bitcoin scripts are about 
validation. Not computation."
String search and concatenation are equivalent in some cases, such as in 
the example you provided.
However, it is still necessary to introduce the OP_ISSUBSTR operation 
separately.
One example is converting a non-deterministic signature to a deterministic 
one.
Another case is when the substring in question is located in the middle of 
the checked string.
CAT cannot replace ISSUBSTR for the following reasons:

   1. The security of CAT is still controversial. It can easily generate 
   overly long strings, potentially causing a stack overflow. Additionally, 
   whether OP_CAT will be restored is still under discussion.
   2. The other substring (bar) must be known in advance.
   

With respect,

Weichu Deng

weichudeng@stu2024•jnu.edu.cn
在2025年3月18日星期二 UTC+8 01:01:16<Peter Todd> 写道：

On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote: 
> 
> 
> Dear fellow Bitcoin developers, 
> 
> 
> 
> I am pleased to present a new BIP proposal. This proposal introduces a 
new 
> opcode for Bitcoin scripts: OP_ISSUBSTR. 
> 
> 
> *Abstract* 
> 
> This BIP introduces two string opcodes, OP_ISSUBSTR and OP_ISSUBSTRVERIFY 
> (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
> determine whether one string is a substring of another. As these opcodes 
do 
> not alter any blockchain state, they are secure. 

Bitcoin scripts are about validation. Not computation. 

This means that substring search and concatenation are equivalent. For 
every script that validates a substring search, you can instead 
concatenate the substring with the rest of the string, and validate 
equality instead. 

Basically speaking: 

foobar foo IsSubStr 

is equivalent to: 

foobar foo bar Cat Equal 

A real-world example would be more complex. But I hope that illustrates 
my point sufficiently. 

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org 

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 12156 bytes --]

 permalink raw reply	[|nested]
2025-03-17 16:54 ` Peter Todd
  2025-03-18 15:32   ` weichu deng

  
From: Erik Aronesty @ 2025-03-18 16:41 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 524 bytes --]



foobar foo IsSubStr 

is equivalent to: 

foobar foo bar Cat Equal 


assuming bar is known (this excludes the nondeterministic example above) 

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/e1c28747-ccc9-4d14-a93f-77dbe08a0ba9n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 989 bytes --]

 permalink raw reply	[|nested]
2025-03-18 15:32   ` weichu deng

  2025-03-19  9:07       ` weichu deng
  
From: Rijndael @ 2025-03-18 21:33 UTC (permalink / raw)
  To: weichu deng;  Bitcoin Development Mailing List

[-- Attachment #1: Type: text/plain, Size: 3567 bytes --]

Stack elements in Taproot are limited to 520 bytes. The current proposal
for re-activating OP_CAT includes this restriction: creating a string
longer than 520 bytes with CAT will cause the script to fail.

With either CAT or ISSUBSTR, you can either commit to the substrings or
provide them at spend-time as witness data (and allow them to be unfixed in
the script).

Fixed: FOOBAR BAR ISSUBSTR == FOOBAR FOO BAR CAT EQ
Variable: [witness: FOOBAR] BAR ISSUBSTR == [witness: FOOBAR FOO] BAR CAT EQ


rijndael


On Mar 18, 2025, at 11:32 AM, weichu deng <weichudeng@stu2024•jnu.edu.cn>
wrote:

Hi, Peter Todd
Thanks for your feedback. I agree that "Bitcoin scripts are about
validation. Not computation."
String search and concatenation are equivalent in some cases, such as in
the example you provided.
However, it is still necessary to introduce the OP_ISSUBSTR operation
separately.
One example is converting a non-deterministic signature to a deterministic
one.
Another case is when the substring in question is located in the middle of
the checked string.
CAT cannot replace ISSUBSTR for the following reasons:

   1. The security of CAT is still controversial. It can easily generate
   overly long strings, potentially causing a stack overflow. Additionally,
   whether OP_CAT will be restored is still under discussion.
   2. The other substring (bar) must be known in advance.


With respect,

Weichu Deng

weichudeng@stu2024•jnu.edu.cn
在2025年3月18日星期二 UTC+8 01:01:16<Peter Todd> 写道：

On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote:
>
>
> Dear fellow Bitcoin developers,
>
>
>
> I am pleased to present a new BIP proposal. This proposal introduces a new

> opcode for Bitcoin scripts: OP_ISSUBSTR.
>
>
> *Abstract*
>
> This BIP introduces two string opcodes, OP_ISSUBSTR and OP_ISSUBSTRVERIFY
> (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to
> determine whether one string is a substring of another. As these opcodes
do
> not alter any blockchain state, they are secure.

Bitcoin scripts are about validation. Not computation.

This means that substring search and concatenation are equivalent. For
every script that validates a substring search, you can instead
concatenate the substring with the rest of the string, and validate
equality instead.

Basically speaking:

foobar foo IsSubStr

is equivalent to:

foobar foo bar Cat Equal

A real-world example would be more complex. But I hope that illustrates
my point sufficiently.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org


-- 
You received this message because you are subscribed to the Google Groups
"Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit
https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com
<https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com?utm_medium=email&utm_source=footer>
.

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/CAD2YOAp%3Dft%2BpApAwh6CQvwMpujfZ0ysSK%2B%3D7zKA64nBeB5w3nA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 17439 bytes --]

 permalink raw reply	[|nested]
2025-03-18 21:33     ` Rijndael

  2025-03-20  0:23         ` Vojtěch Strnad
  
From: weichu deng @ 2025-03-19  9:07 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 4645 bytes --]



Hi Rijndael,

 

Thanks for your example 

[witness: foobar  foo] bar CAT EQ

 

Yes, the unfixed string can be checked against a target substring in your 
example. However, if the target substring is located in the middle of the 
unfixed string, how to check it? In other words, how to have the same 
function as “foobar ob ISSUBSTR” with CAT if “foobar” is unfixed?

 

For example, suppose that a lucky draw game has the rule: if anyone has a 
publicKey which includes a special substring "goodluck", he/she will be 
awarded. 

This game can be easily implemented with OP_ISSUBSTR as follow.

- LockScript: OP_DUP goodluck OP_ISSUBSTR...

- UnlockScript: signature publicKey

How to implement it with OP_CAT?

 

*Regards*

Weichu deng

weichudeng@stu2024•jnu.edu.cn

 

在2025年3月19日星期三 UTC+8 10:28:25<Rijndael> 写道：

> Stack elements in Taproot are limited to 520 bytes. The current proposal 
> for re-activating OP_CAT includes this restriction: creating a string 
> longer than 520 bytes with CAT will cause the script to fail.
>
> With either CAT or ISSUBSTR, you can either commit to the substrings or 
> provide them at spend-time as witness data (and allow them to be unfixed in 
> the script).
>
> Fixed: FOOBAR BAR ISSUBSTR == FOOBAR FOO BAR CAT EQ
> Variable: [witness: FOOBAR] BAR ISSUBSTR == [witness: FOOBAR FOO] BAR CAT 
> EQ
>
>
> rijndael 
>
>
> On Mar 18, 2025, at 11:32 AM, weichu deng <weich...@stu2024•jnu.edu.cn> 
> wrote:
>
> Hi, Peter Todd
> Thanks for your feedback. I agree that "Bitcoin scripts are about 
> validation. Not computation."
> String search and concatenation are equivalent in some cases, such as in 
> the example you provided.
> However, it is still necessary to introduce the OP_ISSUBSTR operation 
> separately.
> One example is converting a non-deterministic signature to a deterministic 
> one.
> Another case is when the substring in question is located in the middle of 
> the checked string.
> CAT cannot replace ISSUBSTR for the following reasons:
>
>    1. The security of CAT is still controversial. It can easily generate 
>    overly long strings, potentially causing a stack overflow. Additionally, 
>    whether OP_CAT will be restored is still under discussion.
>    2. The other substring (bar) must be known in advance.
>    
>
> With respect,
>
> Weichu Deng
>
> weich...@stu2024•jnu.edu.cn
> 在2025年3月18日星期二 UTC+8 01:01:16<Peter Todd> 写道：
>
> On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote:
> > 
> > 
> > Dear fellow Bitcoin developers,
> > 
> > 
> > 
> > I am pleased to present a new BIP proposal. This proposal introduces a 
> new 
> > opcode for Bitcoin scripts: OP_ISSUBSTR.
> > 
> > 
> > *Abstract*
> > 
> > This BIP introduces two string opcodes, OP_ISSUBSTR and OP_ISSUBSTRVERIFY
>  
> > (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
> > determine whether one string is a substring of another. As these opcodes 
> do 
> > not alter any blockchain state, they are secure.
>
> Bitcoin scripts are about validation. Not computation.
>
> This means that substring search and concatenation are equivalent. For
> every script that validates a substring search, you can instead
> concatenate the substring with the rest of the string, and validate
> equality instead.
>
> Basically speaking:
>
> foobar foo IsSubStr
>
> is equivalent to:
>
> foobar foo bar Cat Equal
>
> A real-world example would be more complex. But I hope that illustrates
> my point sufficiently.
>
> -- 
> https://petertodd.org 'peter'[:-1]@petertodd.org
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Bitcoin Development Mailing List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to bitcoindev+...@googlegroups•com.
> To view this discussion visit 
> https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com 
> <https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/8c823e50-197e-479c-8651-9e0407a4168en%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 20248 bytes --]

 permalink raw reply	[|nested]
2025-03-19  9:07       ` weichu deng

  2025-03-28 22:40           ` Javier Mateos
  
From: Vojtěch Strnad @ 2025-03-20  0:23 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 5240 bytes --]

Hi Weichu,

You can implement this game by having the user supply in the initial stack 
the two parts of their public key with the middle "goodluck" removed, and 
inserting the "goodluck" as part of the script:

   - script: "goodluck" OP_SWAP OP_CAT OP_CAT OP_CHECKSIG
   - initial stack: signature pubkey_left pubkey_right

Hope this helps.

Vojtěch
On Thursday, March 20, 2025 at 1:06:41 AM UTC+1 weichu deng wrote:

> Hi Rijndael,
>
>  
>
> Thanks for your example 
>
> [witness: foobar  foo] bar CAT EQ
>
>  
>
> Yes, the unfixed string can be checked against a target substring in your 
> example. However, if the target substring is located in the middle of the 
> unfixed string, how to check it? In other words, how to have the same 
> function as “foobar ob ISSUBSTR” with CAT if “foobar” is unfixed?
>
>  
>
> For example, suppose that a lucky draw game has the rule: if anyone has a 
> publicKey which includes a special substring "goodluck", he/she will be 
> awarded. 
>
> This game can be easily implemented with OP_ISSUBSTR as follow.
>
> - LockScript: OP_DUP goodluck OP_ISSUBSTR...
>
> - UnlockScript: signature publicKey
>
> How to implement it with OP_CAT?
>
>  
>
> *Regards*
>
> Weichu deng
>
> weich...@stu2024•jnu.edu.cn
>
>  
>
> 在2025年3月19日星期三 UTC+8 10:28:25<Rijndael> 写道：
>
>> Stack elements in Taproot are limited to 520 bytes. The current proposal 
>> for re-activating OP_CAT includes this restriction: creating a string 
>> longer than 520 bytes with CAT will cause the script to fail.
>>
>> With either CAT or ISSUBSTR, you can either commit to the substrings or 
>> provide them at spend-time as witness data (and allow them to be unfixed in 
>> the script).
>>
>> Fixed: FOOBAR BAR ISSUBSTR == FOOBAR FOO BAR CAT EQ
>> Variable: [witness: FOOBAR] BAR ISSUBSTR == [witness: FOOBAR FOO] BAR CAT 
>> EQ
>>
>>
>> rijndael 
>>
>>
>> On Mar 18, 2025, at 11:32 AM, weichu deng <weich...@stu2024•jnu.edu.cn> 
>> wrote:
>>
>> Hi, Peter Todd
>> Thanks for your feedback. I agree that "Bitcoin scripts are about 
>> validation. Not computation."
>> String search and concatenation are equivalent in some cases, such as in 
>> the example you provided.
>> However, it is still necessary to introduce the OP_ISSUBSTR operation 
>> separately.
>> One example is converting a non-deterministic signature to a 
>> deterministic one.
>> Another case is when the substring in question is located in the middle 
>> of the checked string.
>> CAT cannot replace ISSUBSTR for the following reasons:
>>
>>    1. The security of CAT is still controversial. It can easily generate 
>>    overly long strings, potentially causing a stack overflow. Additionally, 
>>    whether OP_CAT will be restored is still under discussion.
>>    2. The other substring (bar) must be known in advance.
>>    
>>
>> With respect,
>>
>> Weichu Deng
>>
>> weich...@stu2024•jnu.edu.cn
>> 在2025年3月18日星期二 UTC+8 01:01:16<Peter Todd> 写道：
>>
>> On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote:
>> > 
>> > 
>> > Dear fellow Bitcoin developers,
>> > 
>> > 
>> > 
>> > I am pleased to present a new BIP proposal. This proposal introduces a 
>> new 
>> > opcode for Bitcoin scripts: OP_ISSUBSTR.
>> > 
>> > 
>> > *Abstract*
>> > 
>> > This BIP introduces two string opcodes, OP_ISSUBSTR and 
>> OP_ISSUBSTRVERIFY 
>> > (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
>> > determine whether one string is a substring of another. As these 
>> opcodes do 
>> > not alter any blockchain state, they are secure.
>>
>> Bitcoin scripts are about validation. Not computation.
>>
>> This means that substring search and concatenation are equivalent. For
>> every script that validates a substring search, you can instead
>> concatenate the substring with the rest of the string, and validate
>> equality instead.
>>
>> Basically speaking:
>>
>> foobar foo IsSubStr
>>
>> is equivalent to:
>>
>> foobar foo bar Cat Equal
>>
>> A real-world example would be more complex. But I hope that illustrates
>> my point sufficiently.
>>
>> -- 
>> https://petertodd.org 'peter'[:-1]@petertodd.org
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Bitcoin Development Mailing List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to bitcoindev+...@googlegroups•com.
>> To view this discussion visit 
>> https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com 
>> <https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/b4594133-e6e5-438b-aa56-98d6171296fan%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 20870 bytes --]

 permalink raw reply	[|nested]
2025-03-20  0:23         ` Vojtěch Strnad

  2025-04-01 12:25             ` Pieter Wuille
  
From: Javier Mateos @ 2025-03-28 22:40 UTC (permalink / raw)
  To: Bitcoin Development Mailing List


[-- Attachment #1.1: Type: text/plain, Size: 5692 bytes --]

  The solution of splitting the string and using OP_CAT only works if the 
exact position of the substring is known. How would a case be handled where 
the substring could be in any position?  

El miércoles, 19 de marzo de 2025 a las 22:02:05 UTC-3, Vojtěch Strnad 
escribió:

> Hi Weichu,
>
> You can implement this game by having the user supply in the initial stack 
> the two parts of their public key with the middle "goodluck" removed, and 
> inserting the "goodluck" as part of the script:
>
>    - script: "goodluck" OP_SWAP OP_CAT OP_CAT OP_CHECKSIG
>    - initial stack: signature pubkey_left pubkey_right
>
> Hope this helps.
>
> Vojtěch
> On Thursday, March 20, 2025 at 1:06:41 AM UTC+1 weichu deng wrote:
>
>> Hi Rijndael,
>>
>>  
>>
>> Thanks for your example 
>>
>> [witness: foobar  foo] bar CAT EQ
>>
>>  
>>
>> Yes, the unfixed string can be checked against a target substring in your 
>> example. However, if the target substring is located in the middle of the 
>> unfixed string, how to check it? In other words, how to have the same 
>> function as “foobar ob ISSUBSTR” with CAT if “foobar” is unfixed?
>>
>>  
>>
>> For example, suppose that a lucky draw game has the rule: if anyone has a 
>> publicKey which includes a special substring "goodluck", he/she will be 
>> awarded. 
>>
>> This game can be easily implemented with OP_ISSUBSTR as follow.
>>
>> - LockScript: OP_DUP goodluck OP_ISSUBSTR...
>>
>> - UnlockScript: signature publicKey
>>
>> How to implement it with OP_CAT?
>>
>>  
>>
>> *Regards*
>>
>> Weichu deng
>>
>> weich...@stu2024•jnu.edu.cn
>>
>>  
>>
>> 在2025年3月19日星期三 UTC+8 10:28:25<Rijndael> 写道：
>>
>>> Stack elements in Taproot are limited to 520 bytes. The current proposal 
>>> for re-activating OP_CAT includes this restriction: creating a string 
>>> longer than 520 bytes with CAT will cause the script to fail.
>>>
>>> With either CAT or ISSUBSTR, you can either commit to the substrings or 
>>> provide them at spend-time as witness data (and allow them to be unfixed in 
>>> the script).
>>>
>>> Fixed: FOOBAR BAR ISSUBSTR == FOOBAR FOO BAR CAT EQ
>>> Variable: [witness: FOOBAR] BAR ISSUBSTR == [witness: FOOBAR FOO] BAR 
>>> CAT EQ
>>>
>>>
>>> rijndael 
>>>
>>>
>>> On Mar 18, 2025, at 11:32 AM, weichu deng <weich...@stu2024•jnu.edu.cn> 
>>> wrote:
>>>
>>> Hi, Peter Todd
>>> Thanks for your feedback. I agree that "Bitcoin scripts are about 
>>> validation. Not computation."
>>> String search and concatenation are equivalent in some cases, such as in 
>>> the example you provided.
>>> However, it is still necessary to introduce the OP_ISSUBSTR operation 
>>> separately.
>>> One example is converting a non-deterministic signature to a 
>>> deterministic one.
>>> Another case is when the substring in question is located in the middle 
>>> of the checked string.
>>> CAT cannot replace ISSUBSTR for the following reasons:
>>>
>>>    1. The security of CAT is still controversial. It can easily 
>>>    generate overly long strings, potentially causing a stack overflow. 
>>>    Additionally, whether OP_CAT will be restored is still under discussion.
>>>    2. The other substring (bar) must be known in advance.
>>>    
>>>
>>> With respect,
>>>
>>> Weichu Deng
>>>
>>> weich...@stu2024•jnu.edu.cn
>>> 在2025年3月18日星期二 UTC+8 01:01:16<Peter Todd> 写道：
>>>
>>> On Mon, Mar 17, 2025 at 09:14:05AM -0700, weichu deng wrote:
>>> > 
>>> > 
>>> > Dear fellow Bitcoin developers,
>>> > 
>>> > 
>>> > 
>>> > I am pleased to present a new BIP proposal. This proposal introduces a 
>>> new 
>>> > opcode for Bitcoin scripts: OP_ISSUBSTR.
>>> > 
>>> > 
>>> > *Abstract*
>>> > 
>>> > This BIP introduces two string opcodes, OP_ISSUBSTR and 
>>> OP_ISSUBSTRVERIFY 
>>> > (similar to the relationship between OP_EQUAL and OP_EQUALVERIFY), to 
>>> > determine whether one string is a substring of another. As these 
>>> opcodes do 
>>> > not alter any blockchain state, they are secure.
>>>
>>> Bitcoin scripts are about validation. Not computation.
>>>
>>> This means that substring search and concatenation are equivalent. For
>>> every script that validates a substring search, you can instead
>>> concatenate the substring with the rest of the string, and validate
>>> equality instead.
>>>
>>> Basically speaking:
>>>
>>> foobar foo IsSubStr
>>>
>>> is equivalent to:
>>>
>>> foobar foo bar Cat Equal
>>>
>>> A real-world example would be more complex. But I hope that illustrates
>>> my point sufficiently.
>>>
>>> -- 
>>> https://petertodd.org 'peter'[:-1]@petertodd.org
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Bitcoin Development Mailing List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to bitcoindev+...@googlegroups•com.
>>> To view this discussion visit 
>>> https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com 
>>> <https://groups.google.com/d/msgid/bitcoindev/678d40e3-3e22-4d55-82c0-b25ccafb87ecn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/e8c30db7-b509-4fbd-93ef-0bca0313003cn%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 21366 bytes --]

 permalink raw reply	[|nested]
2025-03-28 22:40           ` Javier Mateos

  2025-04-01 15:35               ` Martin Habovštiak
  2025-04-08 18:04               ` Anthony Towns
  
From: Pieter Wuille @ 2025-04-01 12:25 UTC (permalink / raw)
  To: Javier Mateos;  Bitcoin Development Mailing List

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Monday, March 31st, 2025 at 4:41 PM, Javier Mateos <javierpmateos@gmail•com> wrote:

> The solution of splitting the string and using OP_CAT only works if the exact position of the substring is known. How would a case be handled where the substring could be in any position

Whoever produces the signature/witness for spending the coin always knows the position already, so the script can always be modified to instead take that position as an additional input.

This is a general principle: the point of scripts is verifying provided information, not computing it. As another example, this means that there is no need for a division or square root opcode if one has a multiplication opcode.

--
Pieter

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/frERrHlzWhpskJw74fBQorSrXEaBP1d4XBUgM-Nkww_2ulhc7i2Lqmu2kcAlvh5fd7LzYiBmX5HNBtg7Ownbsa0KZ26ihfJjri6R01kuozA%3D%40wuille.net.

[-- Attachment #2: Type: text/html, Size: 1770 bytes --]

 permalink raw reply	[|nested]
2025-04-01 12:25             ` Pieter Wuille

  2025-04-08 18:04               ` Anthony Towns
  
From: Martin Habovštiak @ 2025-04-01 15:35 UTC (permalink / raw)
  To: Pieter Wuille;  Javier Mateos, Bitcoin Development Mailing List

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

Hi,

I was dismissing the proposal for the same reason you do but it just
occurred to me that substrings might be better than OP_CAT because it's
possible to make them unabusable without any arbitrary limit on item size.

The idea is to store stack elements on the heap inside struct { ref_count,
length, data[] } and put struct { pointer_to_item, position, length } on
the stack. (Rust developers may be familiar with the `bytes` crate that
does this.)
Substring operations would only duplicate the pointers with adjusted
position and length so there's no way to blow up the stack using them.

Of course there's an exception if OP_SHA256 is used on a shorter slice but
the same is true today - you can already write OP_ZERO OP_SHA256 OP_DUP
OP_DUP...

Funnily, this can be used to optimize OP_DUP as well which would now add
constant amount of memory, so the "exploit" above would need to use two
bytes per every large object.

Anyway, while I would personally prefer not having arbitrary limits on item
sizes, since the limit is already there, it might not matter. I guess
something worth considering if any other future soft fork somehow enables
larger items.

Have a nice day!

Martin


Dňa ut 1. 4. 2025, 16:49 Pieter Wuille <bitcoin-dev@wuille•net> napísal(a):

> On Monday, March 31st, 2025 at 4:41 PM, Javier Mateos <
> javierpmateos@gmail•com> wrote:
>
> The solution of splitting the string and using OP_CAT only works if the
> exact position of the substring is known. How would a case be handled where
> the substring could be in any position
>
>
> Whoever produces the signature/witness for spending the coin always knows
> the position already, so the script can always be modified to instead take
> that position as an additional input.
>
> This is a general principle: the point of scripts is verifying provided
> information, not computing it. As another example, this means that there is
> no need for a division or square root opcode if one has a multiplication
> opcode.
>
> --
> Pieter
>
> --
> You received this message because you are subscribed to the Google Groups
> "Bitcoin Development Mailing List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to bitcoindev+unsubscribe@googlegroups•com.
> To view this discussion visit
> https://groups.google.com/d/msgid/bitcoindev/frERrHlzWhpskJw74fBQorSrXEaBP1d4XBUgM-Nkww_2ulhc7i2Lqmu2kcAlvh5fd7LzYiBmX5HNBtg7Ownbsa0KZ26ihfJjri6R01kuozA%3D%40wuille.net
> <https://groups.google.com/d/msgid/bitcoindev/frERrHlzWhpskJw74fBQorSrXEaBP1d4XBUgM-Nkww_2ulhc7i2Lqmu2kcAlvh5fd7LzYiBmX5HNBtg7Ownbsa0KZ26ihfJjri6R01kuozA%3D%40wuille.net?utm_medium=email&utm_source=footer>
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/CALkkCJb1RdC646q1QrOP%2ByQut7EG4NLJNm3cyxW4S1%3DEx1NBmQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4516 bytes --]

 permalink raw reply	[|nested]
2025-04-01 12:25             ` Pieter Wuille
  2025-04-01 15:35               ` Martin Habovštiak

  
From: Anthony Towns @ 2025-04-08 18:04 UTC (permalink / raw)
  To: Pieter Wuille;  Javier Mateos, Bitcoin Development Mailing List

On Tue, Apr 01, 2025 at 12:25:26PM +0000, Pieter Wuille wrote:
> On Monday, March 31st, 2025 at 4:41 PM, Javier Mateos <javierpmateos@gmail•com> wrote:
> > The solution of splitting the string and using OP_CAT only works if the exact position of the substring is known. How would a case be handled where the substring could be in any position
> Whoever produces the signature/witness for spending the coin always knows the position already, so the script can always be modified to instead take that position as an additional input.
> This is a general principle: the point of scripts is verifying provided information, not computing it. As another example, this means that there is no need for a division or square root opcode if one has a multiplication opcode.

I somewhat disagree with this: there are some concerns that are *easier*
to express with different opcodes, and I think that's a factor worth
considering.

This came up with the OP_CAT based proof-of-work faucet [0] -- the
idea there is that you provide a signature and some nonce data, and
when you combine the two and hash the result, that result begins with
some sufficient number of 0 bits (that then gets related back to a
CHECKSEQUENCEVERIFY delay).

OP_CAT is *sufficient* for testing this, because you just CAT the
signature and nonce together and hash them, and can then again CAT the
the 0-bits you expect together with some other data and check that all
of those combined match the hash you calculated earlier.

But it would be more efficient, and a little easier to code, if you
could instead have used SUBSTR/LEFT to pull the initial bytes from
the calculated hash and check that those have the expected number of
leading 0-bits. More efficient, because you don't have to supply all
the trailing bytes of the hash in the witness, and easier to code,
because it's a bit more natural to think of manipulating the hash you
calculated, rather than having to put user-provided data together and
check that that actually matched what you would have extracted from
the hash had you been able to do it that way.

So just like we have unnecessary opcodes like "CHECKSIGVERIFY" (versus
"CHECKSIG VERIFY"), I think it's still worth evaluating "SUBSTR" and
"DIV" on questions of efficiency, even if they're not providing any
additional expressivity -- ie, does it provide a meaningful improvement
either for on-chain validation or when writing/maintaining scripts
compared to other ways of achieving the same goal?

(That said, I don't think "ISSUBSTR" is a great opcode; the original
SUBSTR that specifies exactly where in the original string the substring
appears seems more useful to me)

Cheers,
aj

[0] https://delvingbitcoin.org/t/proof-of-work-based-signet-faucet/937

-- 
You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups•com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/Z_VlNPeDS8Hk3Dyy%40erisian.com.au.


 permalink raw reply	[|nested]