Repository: rsgalloway/pyseq

Issue:
maybe versions like _v### should be excluded by default?
sometimes pyseq sees my versions ie file_v001.png file_v002.png as sequences which i imagine would be wrong in almost all cases (at least in VFX).

i wonder if this should be filtered out by default in `pyseq.seq.digits_re`?

one way could be do have a couple of default regex templates that the user can set based on their environment
- everything can be a sequence including v001
- everything can be a sequence excluding v001
- sequences have to start and end with [._-] 
 - ie file_v001.1001.exr file_v001_1001.exr file_v001-1001.exr, file_v001.1001_highres.exr 

Thanks,
Patrick

Discussion:
@rsgalloway: I have seen this sometimes, too. pyseq is intentionally very naive (flexible?) in it's implementation of what it considers a "sequence" because I wanted to work regardless of naming conventions. this can sometime backfire though. 

> default regex templates that the user can set based on their environment

I think I favor this approach. if it's set in the environment then it can even be contextual (different shows/shots/etc may have different conventions). need to give it a little thought.@rsgalloway: FYI this has been merged and uploaded to pypi. 

```bash
$ pip install -U pyseq
```

You can control the frame regex using the env var ${PYSEQ_FRAME_PATTERN}, or you can set the same in the pyseq.env file if using envstack, for example:

```bash
$ pip install -U envstack
$ envstack -s PYSEQ_FRAME_PATTERN:"_\d+" -o pyseq.env
```

Hope that helps, let me know if there are any issues ;)@patrickwolf: Great thank you. By the way did you consider providing some default regex templates? 
ie 
${PYSEQ_FRAME_PATTERN} = pyseq.RegEx_withVersion

> * everything can be a sequence including v001
> * everything can be a sequence excluding v001
> * sequences have to start and end with [._-]
> * ie file_v001.1001.exr file_v001_1001.exr file_v001-1001.exr, file_v001.1001_highres.exr

I imagine it would help people if there are a few tested and supported regex that can also be mentioned in the docs@rsgalloway: good idea, I'll give this some thought@rsgalloway: @patrickwolf I'm including some examples in the v0.9.0 branch, but try these:

```yaml
# matches all numbers, the most flexible
PYSEQ_FRAME_PATTERN: \d+
# excludes version numbers, e.g. file_v001.1001.exr
PYSEQ_FRAME_PATTERN: ([^v\d])\d+
# frame numbers are dot-delimited, e.g. file.v1.1001.exr
PYSEQ_FRAME_PATTERN: \.\d+\.
# frame numbers start with an underscore, e.g. file_v1_1001.exr
PYSEQ_FRAME_PATTERN: _\d+
```

these are meant to be put into the pyseq.env file for convenience. I tested the one that excludes version numbers and it seemed to do the right thing. try creating pyseq.env file and putting this in it:

```bash
$ pip install -U envstack
$ envstack -s PYSEQ_FRAME_PATTERN:"([^v\d])\d+" -o pyseq.env
```

your env file should look like this:

```yaml
#!/usr/bin/env envstack
include: []
all: &all
  <<: *all
  PYSEQ_FRAME_PATTERN: ([^v\d])\d+
darwin:
  <<: *all
linux:
  <<: *all
windows:
  <<: *all
```

and you should get results like this:

```bash
$ lss tests/files/
  10 012_vb_110_v001.%04d.png [1-10]
  10 012_vb_110_v002.%04d.png [1-10]
   7 a.%03d.tga [1-3, 10, 12-14]
   1 alpha.txt 
   5 bnc01_TinkSO_tx_0_ty_0.%04d.tif [101-105]
   5 bnc01_TinkSO_tx_0_ty_1.%04d.tif [101-105]
   5 bnc01_TinkSO_tx_1_ty_0.%04d.tif [101-105]
   5 bnc01_TinkSO_tx_1_ty_1.%04d.tif [101-105]
   4 file.%02d.tif [1-2, 98-99]
   1 file.info.03.rgb 
   3 file01.%03d.j2k [1-2, 4]
   4 file01_%04d.rgb [40-43]
   4 file02_%04d.rgb [44-47]
   4 file%d.03.rgb [1-4]
   3 fileA.%04d.jpg [1-3]
   3 fileA.%04d.png [1-3]
   1 file_02.tif 
   4 z1_001_v1.%d.png [1-4]
   4 z1_002_v1.%d.png [1-4]
   4 z1_002_v2.%d.png [9-12]
$ lss tmp/issue-83/
   1 file_v001.1001.png 
   1 file_v001.png 
   1 file_v002.1001.png 
   1 file_v002.png 
   1 file_v003.1001.png 
   1 file_v003.png 
   1 file_v004.1001.png 
   1 file_v004.png 
   2 file_v005.%d.png [1001-1002]
   1 file_v005.png 
```@patrickwolf: > [@patrickwolf](https://github.com/patrickwolf) I'm including some examples in the v0.9.0 branch, but try these:

thank its actually less for me as I know how to write regex and have been overwriting your default regex by using pyseq.seq.digits_re = xyz for a while. 

I just thought it would help your library to be used by more as it deserves adoption :)@rsgalloway: thanks! appreciate that ;)

Solution patch:
diff --git a/README.md b/README.md
index 03f54d3..8b3bcaf 100644
--- a/README.md
+++ b/README.md
@@ -16,30 +16,32 @@ The easiest way to install pyseq:
 $ pip install -U pyseq
 ```
 
-#### distman
+#### Environment
 
-If installing from source you can use [distman](https://github.com/rsgalloway/distman)
-to install pyseq using the provided `dist.json` file:
+PySeq uses [envstack](https://github.com/rsgalloway/envstack) to externalize
+settings and looks for a `pyseq.env` file to source environment variables:
 
 ```bash
-$ distman [-d]
+$ pip install -U envstack
+$ ./pyseq.env -r
+PYSEQ_FRAME_PATTERN=\d+
+PYSEQ_RANGE_SEP=, 
+PYSEQ_STRICT_PAD=0
 ```
 
-Using distman will deploy the targets defined in the `dist.json` file to the
-root folder defined by `$DEPLOY_ROOT`.
-
-#### envstack
+#### Distribution
 
-PySeq uses [envstack](https://github.com/rsgalloway/envstack) to manage configs
-via environment variables.
+If installing from source you can use [distman](https://github.com/rsgalloway/distman)
+to install PySeq using the provided `dist.json` file:
 
 ```bash
-$ envstack pyseq
-PYSEQ_RANGE_SEP=, 
-PYSEQ_STRICT_PAD=0
-STACK=pyseq
+$ pip install -U distman
+$ distman [-d]
 ```
 
+Using distman will deploy the targets defined in the `dist.json` file to the
+root folder defined by `${DEPLOY_ROOT}`:
+
 ## Basic Usage
 
 Using the "z1" file sequence example in the "tests" directory, we start by
@@ -195,6 +197,24 @@ a.1-14.tga
 7 a.1-14.tga [4-9, 11]
 ```
 
+## Frame Patterns
+
+The environment var `${PYSEQ_FRAME_PATTERN}` can be used to define custom regex
+patterns for identifying frame numbers. For example if frames are always preceded
+with an _, you might use:
+
+```bash
+$ export PYSEQ_FRAME_PATTERN="_\d+"
+```
+
+Environment vars can be defined anywhere in your environment, or if using
+[envstack](https://github.com/rsgalloway/envstack) add it to the
+`pyseq.env` file and make sure it's found in `${ENVPATH}`:
+
+```bash
+$ export ENVPATH=/path/to/env/files
+```
+
 ## Testing
 
 To run the unit tests, simply run `pytest` in a shell:
diff --git a/lib/pyseq/__init__.py b/lib/pyseq/__init__.py
index 8741197..81df04d 100644
--- a/lib/pyseq/__init__.py
+++ b/lib/pyseq/__init__.py
@@ -48,7 +48,7 @@ Docs and latest version available for download at
 """
 
 __author__ = "Ryan Galloway"
-__version__ = "0.8.3"
+__version__ = "0.8.4"
 
 try:
     import envstack
diff --git a/lib/pyseq/config.py b/lib/pyseq/config.py
new file mode 100644
index 0000000..03075bc
--- /dev/null
+++ b/lib/pyseq/config.py
@@ -0,0 +1,66 @@
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2025, Ryan Galloway (ryan@rsgalloway.com)
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+#  - Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+#
+#  - Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+#
+#  - Neither the name of the software nor the names of its contributors
+#    may be used to endorse or promote products derived from this software
+#    without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+# -----------------------------------------------------------------------------
+
+__doc__ = """
+Contains pyseq configs and default settings.
+"""
+
+import os
+import re
+
+# default serialization format string
+DEFAULT_FORMAT = "%h%r%t"
+default_format = os.getenv("PYSEQ_DEFAULT_FORMAT", DEFAULT_FORMAT)
+
+# default serialization format string for global sequences
+DEFAULT_GLOBAL_FORMAT = "%4l %h%p%t %R"
+global_format = os.getenv("PYSEQ_GLOBAL_FORMAT", DEFAULT_GLOBAL_FORMAT)
+
+# use strict padding on sequences (pad length must match)
+PYSEQ_STRICT_PAD = os.getenv("PYSEQ_STRICT_PAD", 0)
+PYSEQ_NOT_STRICT = os.getenv("PYSEQ_NOT_STRICT", 1)
+strict_pad = int(PYSEQ_STRICT_PAD) == 1 or int(PYSEQ_NOT_STRICT) == 0
+
+# regex pattern for matching all numbers in a filename
+digits_re = re.compile(r"\d+")
+
+# regex pattern for matching frame numbers only
+# the default is \d+ for maximum compatibility
+DEFAULT_FRAME_PATTERN = r"\d+"
+PYSEQ_FRAME_PATTERN = os.getenv("PYSEQ_FRAME_PATTERN", DEFAULT_FRAME_PATTERN)
+frames_re = re.compile(PYSEQ_FRAME_PATTERN)
+
+# regex for matching format directives
+format_re = re.compile(r"%(?P<pad>\d+)?(?P<var>\w+)")
+
+# character to join explicit frame ranges on
+DEFAULT_RANGE_SEP = ", "
+range_join = os.getenv("PYSEQ_RANGE_SEP", DEFAULT_RANGE_SEP)
diff --git a/lib/pyseq/seq.py b/lib/pyseq/seq.py
index 1a548d8..0ec5035 100755
--- a/lib/pyseq/seq.py
+++ b/lib/pyseq/seq.py
@@ -41,30 +41,16 @@ import warnings
 from collections import deque
 from glob import glob, iglob
 
+from pyseq import config
 from pyseq.util import _ext_key
-
-# default serialization format string
-global_format = "%4l %h%p%t %R"
-default_format = "%h%r%t"
-
-# use strict padding on sequences (pad length must match)
-#    $ export PYSEQ_STRICT_PAD=1 or
-#    $ export PYSEQ_NOT_STRICT=1
-# to enable/disable. disabled by default.
-strict_pad = (
-    int(os.getenv("PYSEQ_STRICT_PAD", 0)) == 1
-    or int(os.getenv("PYSEQ_NOT_STRICT", 1)) == 0
+from pyseq.config import (
+    default_format,
+    format_re,
+    global_format,
+    range_join,
+    strict_pad,
 )
 
-# regex for matching numerical characters
-digits_re = re.compile(r"\d+")
-
-# regex for matching format directives
-format_re = re.compile(r"%(?P<pad>\d+)?(?P<var>\w+)")
-
-# character to join explicit frame ranges on
-range_join = os.getenv("PYSEQ_RANGE_SEP", ", ")
-
 
 class SequenceError(Exception):
     """Special exception for Sequence errors."""
@@ -120,7 +106,7 @@ class Item(str):
             self.__path = str(item)
         self.__filename = os.path.basename(self.__path)
         self.__number_matches = []
-        self.__parts = digits_re.split(self.name)
+        self.__parts = config.frames_re.split(self.name)
         self.__stat = None
 
         # modified by self.is_sibling()
@@ -250,7 +236,7 @@ class Item(str):
 
         :return: The numerical components.
         """
-        return digits_re.findall(self.__filename)
+        return config.frames_re.findall(self.__filename)
 
     @property
     def number_matches(self):
@@ -260,7 +246,7 @@ class Item(str):
         :return: The numerical components.
         """
         if not self.__number_matches:
-            self.__number_matches = list(digits_re.finditer(self.__filename))
+            self.__number_matches = list(config.digits_re.finditer(self.__filename))
         return self.__number_matches
 
     @property
@@ -1055,7 +1041,7 @@ def uncompress(seq_string, fmt=global_format):
     return seqs
 
 
-def get_sequences(source):
+def get_sequences(source, frame_pattern=config.PYSEQ_FRAME_PATTERN):
     """Returns a list of Sequence objects given a directory or list that contain
     sequential members.
 
@@ -1091,11 +1077,14 @@ def get_sequences(source):
         fileB.1.rgb
 
     :param source: Can be directory path, list of strings, or sortable list of objects.
+    :param frame_pattern: Regular expression pattern for frame matching.
     :return: List of pyseq.Sequence class objects.
     """
 
     seqs = []
 
+    config.frames_re = re.compile(frame_pattern)
+
     if isinstance(source, list):
         items = sorted(source, key=lambda x: str(x))
 
@@ -1126,7 +1115,7 @@ def get_sequences(source):
     return seqs
 
 
-def iget_sequences(source):
+def iget_sequences(source, frame_pattern=config.PYSEQ_FRAME_PATTERN):
     """Generator version of get_sequences.  Creates Sequences from a various
     source files.  A notable difference is the sort order of iget_sequences
     versus get_sequences.  iget_sequences uses an adaption of natural sorting
@@ -1170,9 +1159,12 @@ def iget_sequences(source):
         fileB.1.rgb
 
     :param source: Can be directory path, list of strings, or sortable list of objects.
+    :param frame_pattern: Regular expression pattern for frame matching.
     :return: List of pyseq.Sequence class objects.
     """
 
+    config.frames_re = re.compile(frame_pattern)
+
     if isinstance(source, list):
         items = source
     elif isinstance(source, str):
diff --git a/setup.py b/setup.py
index 4727438..c636508 100644
--- a/setup.py
+++ b/setup.py
@@ -57,7 +57,6 @@ setup(
             "lss = pyseq.lss:main",
         ],
     },
-    install_requires=["envstack>=0.8.3"],
     python_requires=">=3.6",
     zip_safe=False,
 )


Test patch:
diff --git a/tests/test_pyseq.py b/tests/test_pyseq.py
index a1ace64..de8e140 100644
--- a/tests/test_pyseq.py
+++ b/tests/test_pyseq.py
@@ -35,6 +35,7 @@ import random
 import unittest
 import subprocess
 import sys
+import time
 
 sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from pyseq import Item, Sequence, diff, uncompress, get_sequences
@@ -599,6 +600,22 @@ class LSSTestCase(unittest.TestCase):
         )
 
 
+class PerformanceTests(unittest.TestCase):
+    """tests the performance of pyseq"""
+
+    def test_performance_1(self):
+        """tests performance for single 10k frame sequence"""
+        files = ["file.%03d.jpg" % i for i in range(1, 10000)]
+        s = time.time()
+        seq = Sequence(files)
+        e = time.time()
+        total_time = e - s
+        print("time taken to create sequence: %s" % (total_time))
+        self.assertEqual(str(seq), "file.1-9999.jpg")
+        self.assertEqual(len(seq), 9999)
+        self.assertTrue(total_time < 0.1)
+
+
 class TestIssues(unittest.TestCase):
     """tests reported issues on github"""
 
@@ -859,6 +876,47 @@ class TestIssues(unittest.TestCase):
         s = uncompress("file.1-150.jpg", fmt="%h%r%t")
         self.assertEqual(len(s), 150)
 
+    def test_issue_83(self):
+        """tests issue 83. externalize frame pattern."""
+
+        filenames = [
+            "file_v001.jpg",
+            "file_v002.jpg",
+            "file_v003.jpg",
+            "file_v004.jpg",
+        ]
+
+        # test using default frame pattern
+        seqs1 = pyseq.get_sequences(filenames)
+        self.assertEqual(len(seqs1), 1)
+
+        # test if a new file in the sequence is included
+        item = Item("file_v005.jpg")
+        self.assertTrue(seqs1[0].includes(item))
+
+        # test using custom frame pattern, different from first sequence
+        seqs2 = pyseq.get_sequences(filenames, frame_pattern="_%d")
+
+        # should have 4 sequences, with one file each
+        self.assertEqual(len(seqs2), len(filenames))
+
+        # test that items from sequences 1 and 2 are not siblings
+        seq1item1 = seqs1[0][0]
+        seq2item1 = seqs2[0][0]
+        self.assertFalse(seq1item1.is_sibling(seq2item1))
+
+        # test that 2 items in the sequence 1 are still siblings
+        seq1item2 = seqs1[0][1]
+        self.assertTrue(seq1item1.is_sibling(seq1item2))
+
+        # test items in sequences 1 and 2 are not siblings
+        self.assertFalse(seq1item1.is_sibling(seq2item1))
+
+        # test that the new item is still included in the first sequence,
+        # and excluded from the second sequence
+        self.assertTrue(seqs1[0].includes(item))
+        self.assertFalse(seqs2[0].includes(item))
+
 
 if __name__ == "__main__":
     unittest.main()