Repository: nvim-lua/plenary.nvim
Pull request #96 (2021-03-13 01:43:36)

Add `strings` module
ref #95
ref https://github.com/nvim-telescope/telescope.nvim/pull/464

### TODO

* [x] need to transfer another funcs from telescope.nvim?
* [x] doc

elianiva: could probably add this to align multibyte string as well. The lua `string.format("%-50")` freaks out when it receives a multibyte character.
 
https://github.com/nvim-telescope/telescope.nvim/blob/c8cc024ebf86d58dc37913e1e9ec7e465d687d68/lua/telescope/utils.lua#L334
Conni2461: Thanks for porting this :)

Do you mind also adding border title truncate?
Should happen here: https://github.com/nvim-lua/plenary.nvim/blob/abdc34a806921de436654adb7f68186dd8dcf796/lua/plenary/window/border.lua#L36

Max width is: `content_win_options.width`
Conni2461: Do you also wanna add this `dedent` function? We use it in `telescope` (docs) and `tree-sitter-lua docgen` (tests) and its easier if it would be part of plenary.

```lua
local dedent = function(str, leave_indent)
  -- find minimum common indent across lines
  local indent = nil
  for line in str:gmatch('[^\n]+') do
    local line_indent = line:match('^%s+') or ''
    if indent == nil or #line_indent < #indent then
      indent = line_indent
    end
  end
  if indent == nil or #indent == 0 then
    -- no minimum common indent
    return str
  end
  local left_indent = (' '):rep(leave_indent or 0)
  -- create a pattern for the indent
  indent = indent:gsub('%s', '[ \t]')
  -- strip it from the first line
  str = str:gsub('^'..indent, left_indent)
  -- strip it from the remaining lines
  str = str:gsub('[\n]'..indent, '\n' .. left_indent)
  return str
end
```
delphinus: @Conni2461 I added `dedent()`. The original logic cannot deal with `'tabstop'`, so I rewrote all with using `strdisplaywidth()`.
Conni2461: > The original logic cannot deal with 'tabstop', so I rewrote all with using strdisplaywidth()

The original was just some hacky thing tj wrote so we can write tests like this:
https://github.com/tjdevries/tree-sitter-lua/blob/ca38705ab4551db996428ebce6bed9bdc9b77780/lua/tests/docgen/renderer_spec.lua#L124-L146

Its not really used in production but thanks anyways :)
tjdevries: One other thing as well is that if there are any string functions that exist within neovim that would be useful but just aren't exposed, we can expose them over the `vim.str_*` functions via the C api in neovim core. I haven't looked much at the code here yet to know if that's true (and also I don't know anything about non-ascii characters :laughing: )(
delphinus: @Conni2461
I tested the case for it. ( https://github.com/nvim-lua/plenary.nvim/commit/d46e342e1828295832895c476e482368d860ffd9 ) I think the current implementaation can deal with the one you pointed. How do you think?

@tjdevries
I looked into the code for `vim.str_*` and it seems possible for `strdisplaywidth()` because the logic exists in charset.c ([here][1]). But the logic for `strcharpart()` exists in func.c ([here][2]) and I cannot find the method to call it from executor.c. How can I do this?

[1]: https://github.com/neovim/neovim/blob/2e156a3b7d7e25e56b03683cc6228c531f4c91ef/src/nvim/charset.c#L773-L788
[2]: https://github.com/neovim/neovim/blob/2e156a3b7d7e25e56b03683cc6228c531f4c91ef/src/nvim/eval/funcs.c#L10004-L100056

```c
static int nlua_str_charpart(lua_State *const lstate) FUNC_ATTR_NONNULL_ALL
{
  f_strcharpart(...); // What should be here for arguments?
  return 1;
}
```

tjdevries: Hey @delphinus , are you asking what code you would need to add to neovim core to be able to do this?

If so, I can guide you through the PR that you'd probably want to be able to do that.
Conni2461: @delphinus Whats the progress here? Some people are waiting for a telescope PR https://github.com/nvim-telescope/telescope.nvim/pull/528#issuecomment-823657075 that is already done but requires the truncating border titles.

I would be fine with merging a not done version here. (for example everything besides `dedent` if you have trouble with removing the `goto`)
delphinus: @tjdevries @Conni2461 

Sorry for too large delay! I have been too busy to write code here, so I (or some person) will improve in the future when I have time ;( Can you merge this current code?
Conni2461: Somehow the `goto` is back. I have no idea why :rofl: 

Can you also rebase https://github.com/nvim-telescope/telescope.nvim/pull/690 so i can test it in combination? After that i think we can merge.

Thanks :)
delphinus: Fixed again. and https://github.com/nvim-telescope/telescope.nvim/pull/690 is also ready.
VVKot: @tjdevries @Conni2461 any chance you can take a look at this PR? As far as I can tell it is blocking https://github.com/nvim-telescope/telescope.nvim/pull/528 which I quite fancy.

Patch:
diff --git a/README.md b/README.md
index 1b671a3..81ad0ee 100644
--- a/README.md
+++ b/README.md
@@ -24,6 +24,7 @@ Plug 'nvim-lua/plenary.nvim'
 - `plenary.context_manager`
 - `plenary.test_harness`
 - `plenary.filetype`
+- `plenary.strings`
 
 ### plenary.job
 
@@ -179,6 +180,19 @@ return {
 }
 ```
 
+### plenary.strings
+
+Re-implement VimL funcs to use them in Lua loop.
+
+* `strings.strdisplaywidth`
+* `strings.strcharpart`
+
+And some other funcs are here to deal with common problems.
+
+* `strings.truncate`
+* `strings.align_str`
+* `strings.dedent`
+
 ### plenary.popup
 
 `popup_*` clone of Vim's commands. If it gets good enough, will submit PR to Neovim and write C wrappers
@@ -205,4 +219,3 @@ This will enable debuggin for the plugin.
 ### plenary.neorocks
 
 DELETED: Please use packer.nvim or other lua-rocks wrapper instead. This no longer exists.
-
diff --git a/lua/plenary/strings.lua b/lua/plenary/strings.lua
new file mode 100644
index 0000000..2b14f62
--- /dev/null
+++ b/lua/plenary/strings.lua
@@ -0,0 +1,167 @@
+local path = require('plenary.path').path
+
+local M = {}
+
+M.strdisplaywidth = (function()
+  if jit and path.sep ~= [[\]] then
+    local ffi = require('ffi')
+    ffi.cdef[[
+      typedef unsigned char char_u;
+      int linetabsize_col(int startcol, char_u *s);
+    ]]
+
+    return function(str, col)
+      str = tostring(str)
+      local startcol = col or 0
+      local s = ffi.new('char[?]', #str + 1)
+      ffi.copy(s, str)
+      return ffi.C.linetabsize_col(startcol, s) - startcol
+    end
+  else
+    return function(str, col)
+      str = tostring(str)
+      if vim.in_fast_loop() then
+        return #str - (col or 0)
+      end
+      return vim.fn.strdisplaywidth(str, col)
+    end
+  end
+end)()
+
+M.strcharpart = (function()
+  if jit and path.sep ~= [[\]] then
+    local ffi = require('ffi')
+    ffi.cdef[[
+      typedef unsigned char char_u;
+      int utf_ptr2len(const char_u *const p);
+    ]]
+
+    local function utf_ptr2len(str)
+      local c_str = ffi.new('char[?]', #str + 1)
+      ffi.copy(c_str, str)
+      return ffi.C.utf_ptr2len(c_str)
+    end
+
+    return function(str, nchar, charlen)
+      local nbyte = 0
+      if nchar > 0 then
+        while nchar > 0 and nbyte < #str do
+          nbyte = nbyte + utf_ptr2len(str:sub(nbyte + 1))
+          nchar = nchar - 1
+        end
+      else
+        nbyte = nchar
+      end
+
+      local len = 0
+      if charlen then
+        while charlen > 0 and nbyte + len < #str do
+          local off = nbyte + len
+          if off < 0 then
+            len = len + 1
+          else
+            len = len + utf_ptr2len(str:sub(off + 1))
+          end
+          charlen = charlen - 1
+        end
+      else
+        len = #str - nbyte
+      end
+
+      if nbyte < 0 then
+        len = len + nbyte
+        nbyte = 0
+      elseif nbyte > #str then
+        nbyte = #str
+      end
+      if len < 0 then
+        len = 0
+      elseif nbyte + len > #str then
+        len = #str - nbyte
+      end
+
+      return str:sub(nbyte + 1, nbyte + len)
+    end
+  else
+    return function(str, nchar, charlen)
+      if vim.in_fast_loop() then
+        return str:sub(nchar + 1, charlen)
+      end
+      return vim.fn.strcharpart(str, nchar, charlen)
+    end
+  end
+end)()
+
+M.truncate = function(str, len, dots)
+  str = tostring(str) -- We need to make sure its an actually a string and not a number
+  dots = dots or '…'
+  if M.strdisplaywidth(str) <= len then
+    return str
+  end
+  local start = 0
+  local current = 0
+  local result = ''
+  local len_of_dots = M.strdisplaywidth(dots)
+  while true do
+    local part = M.strcharpart(str, start, 1)
+    current = current + M.strdisplaywidth(part)
+    if (current + len_of_dots) > len then
+      result = result .. dots
+      break
+    end
+    result = result .. part
+    start = start + 1
+  end
+  return result
+end
+
+M.align_str = function(string, width, right_justify)
+  local str_len = M.strdisplaywidth(string)
+  return right_justify
+    and string.rep(" ", width - str_len)..string
+    or string..string.rep(" ", width - str_len)
+end
+
+M.dedent = function(str, leave_indent)
+  -- Check each line and detect the minimum indent.
+  local indent
+  local info = {}
+  for line in str:gmatch('[^\n]*\n?') do
+    -- It matches '' for the last line.
+    if line ~= '' then
+      local chars, width
+      local line_indent = line:match('^[ \t]+')
+      if line_indent then
+        chars = #line_indent
+        width = M.strdisplaywidth(line_indent)
+        if not indent or width < indent then
+          indent = width
+        end
+      -- Ignore empty lines
+      elseif line ~= '\n' then
+        indent = 0
+      end
+      table.insert(info, {line = line, chars = chars, width = width})
+    end
+  end
+
+  -- Build up the result
+  leave_indent = leave_indent or 0
+  local result = {}
+  for _, i in ipairs(info) do
+    local line
+    if i.chars then
+      local content = i.line:sub(i.chars + 1)
+      local indent_width = i.width - indent + leave_indent
+      line = (' '):rep(indent_width) .. content
+    elseif i.line == '\n' then
+      line = '\n'
+    else
+      line = (' '):rep(leave_indent) .. i.line
+    end
+    table.insert(result, line)
+  end
+  return table.concat(result)
+end
+
+return M
diff --git a/lua/plenary/window/border.lua b/lua/plenary/window/border.lua
index f8db15b..c447a12 100644
--- a/lua/plenary/window/border.lua
+++ b/lua/plenary/window/border.lua
@@ -1,4 +1,5 @@
 local tbl = require('plenary.tbl')
+local strings = require('plenary.strings')
 
 local Border = {}
 
@@ -29,14 +30,23 @@ function Border._create_lines(content_win_options, border_win_options)
 
   if content_win_options.row > 0 then
     if border_win_options.title then
+      local title_len
       local title = border_win_options.title
-      if title ~= '' then
+      if title == '' then
+        title_len = 0
+      else
+        local len = strings.strdisplaywidth(title)
+        local max_title_width = content_win_options.width - 2
+        if len > max_title_width then
+          title = strings.truncate(title, max_title_width)
+          len = strings.strdisplaywidth(title)
+        end
         title = string.format(" %s ", title)
+        title_len = len + 2
       end
-      local title_len = string.len(title)
 
       local midpoint = math.floor(content_win_options.width / 2)
-      local left_start = midpoint - math.floor(title_len / 2)
+      local left_start = midpoint - math.ceil(title_len / 2)
 
       topline = string.format("%s%s%s%s%s",
         topleft,


Test patch:
diff --git a/tests/plenary/strings_spec.lua b/tests/plenary/strings_spec.lua
new file mode 100644
index 0000000..a528414
--- /dev/null
+++ b/tests/plenary/strings_spec.lua
@@ -0,0 +1,248 @@
+local strings = require('plenary.strings')
+local eq = assert.are.same
+
+describe('strings', function()
+  describe('strdisplaywidth', function()
+    for _, case in ipairs{
+      {str = 'abcde', expected = {single = 5, double = 5}},
+      -- This space below is a tab (U+0009)
+      {str = 'abc	de', expected = {single = 10, double = 10}},
+      {str = 'アイウエオ', expected = {single = 10, double = 10}},
+      {str = '├─┤', expected = {single = 3, double = 6}},
+      {str = 123, expected = {single = 3, double = 3}},
+    } do
+      for _, ambiwidth in ipairs{'single', 'double'} do
+        local item = type(case.str) == 'string' and '"%s"' or '%s'
+        local msg = ('ambiwidth = %s, '..item..' -> %d'):format(ambiwidth, case.str, case.expected[ambiwidth])
+        local original = vim.o.ambiwidth
+        vim.o.ambiwidth = ambiwidth
+        it('lua: '..msg, function()
+          eq(case.expected[ambiwidth], strings.strdisplaywidth(case.str))
+        end)
+        it('vim: '..msg, function()
+          eq(case.expected[ambiwidth], vim.fn.strdisplaywidth(case.str))
+        end)
+        vim.o.ambiwidth = original
+      end
+    end
+  end)
+
+  describe('strcharpart', function()
+    for _, case in ipairs{
+      {args = {'abcde', 2}, expected = 'cde'},
+      {args = {'abcde', 2, 2}, expected = 'cd'},
+      {args = {'アイウエオ', 2, 2}, expected = 'ウエ'},
+      {args = {'├───┤', 2, 2}, expected = '──'},
+    } do
+      local msg = ('("%s", %d, %s) -> "%s"'):format(case.args[1], case.args[2], tostring(case.args[3]), case.expected)
+      it('lua: '..msg, function()
+        eq(case.expected, strings.strcharpart(unpack(case.args)))
+      end)
+      it('vim: '..msg, function()
+        eq(case.expected, vim.fn.strcharpart(unpack(case.args)))
+      end)
+    end
+  end)
+
+  describe('truncate', function()
+    for _, case in ipairs{
+      {args = {'abcde', 6}, expected = {single = 'abcde', double = 'abcde'}},
+      {args = {'abcde', 5}, expected = {single = 'abcde', double = 'abcde'}},
+      {args = {'abcde', 4}, expected = {single = 'abc…', double = 'ab…'}},
+      {args = {'アイウエオ', 11}, expected = {single = 'アイウエオ', double = 'アイウエオ'}},
+      {args = {'アイウエオ', 10}, expected = {single = 'アイウエオ', double = 'アイウエオ'}},
+      {args = {'アイウエオ', 9}, expected = {single = 'アイウエ…', double = 'アイウ…'}},
+      {args = {'アイウエオ', 8}, expected = {single = 'アイウ…', double = 'アイウ…'}},
+      {args = {'├─┤', 7}, expected = {single = '├─┤', double = '├─┤'}},
+      {args = {'├─┤', 6}, expected = {single = '├─┤', double = '├─┤'}},
+      {args = {'├─┤', 5}, expected = {single = '├─┤', double = '├…'}},
+      {args = {'├─┤', 4}, expected = {single = '├─┤', double = '├…'}},
+      {args = {'├─┤', 3}, expected = {single = '├─┤', double = '…'}},
+      {args = {'├─┤', 2}, expected = {single = '├…', double = '…'}},
+    } do
+      for _, ambiwidth in ipairs{'single', 'double'} do
+        local msg = ('ambiwidth = %s, [%s, %d] -> %s'):format(
+          ambiwidth,
+          case.args[1],
+          case.args[2],
+          case.expected[ambiwidth]
+        )
+        it(msg, function()
+          local original = vim.o.ambiwidth
+          vim.o.ambiwidth = ambiwidth
+          eq(case.expected[ambiwidth], strings.truncate(unpack(case.args)))
+          vim.o.ambiwidth = original
+        end)
+      end
+    end
+  end)
+
+  describe('align_str', function()
+    for _, case in ipairs{
+      {args = {'abcde', 8}, expected = {single = 'abcde   ', double = 'abcde   '}},
+      {args = {'アイウ', 8}, expected = {single = 'アイウ  ', double = 'アイウ  '}},
+      {args = {'├─┤', 8}, expected = {single = '├─┤     ', double = '├─┤  '}},
+      {args = {'abcde', 8, true}, expected = {single = '   abcde', double = '   abcde'}},
+      {args = {'アイウ', 8, true}, expected = {single = '  アイウ', double = '  アイウ'}},
+      {args = {'├─┤', 8, true}, expected = {single = '     ├─┤', double = '  ├─┤'}},
+    } do
+      for _, ambiwidth in ipairs{'single', 'double'} do
+        local msg = ('ambiwidth = %s, [%s, %d, %s] -> "%s"'):format(
+          ambiwidth,
+          case.args[1],
+          case.args[2],
+          tostring(case.args[3]),
+          case.expected[ambiwidth]
+        )
+        it(msg, function()
+          local original = vim.o.ambiwidth
+          vim.o.ambiwidth = ambiwidth
+          eq(case.expected[ambiwidth], strings.align_str(unpack(case.args)))
+          vim.o.ambiwidth = original
+        end)
+      end
+    end
+  end)
+
+  describe('dedent', function()
+    local function lines(t)
+      return table.concat(t, '\n')
+    end
+    for _, case in ipairs{
+      {
+        msg = 'empty string',
+        tabstop = 8,
+        args = {''},
+        expected = '',
+      },
+      {
+        msg = 'in case tabs are longer than spaces',
+        tabstop = 8,
+        args = {
+          lines{
+            '		<Tab><Tab> -> 13 spaces',
+            '     5 spaces -> 0 space',
+          },
+        },
+        expected = lines{
+          '           <Tab><Tab> -> 13 spaces',
+          '5 spaces -> 0 space',
+        },
+      },
+      {
+        msg = 'in case tabs are shorter than spaces',
+        tabstop = 2,
+        args = {
+          lines{
+            '		<Tab><Tab> -> 0 space',
+            '     5spaces -> 1 space',
+          },
+        },
+        expected = lines{
+          '<Tab><Tab> -> 0 space',
+          ' 5spaces -> 1 space',
+        },
+      },
+      {
+        msg = 'ignores empty lines',
+        tabstop = 2,
+        args = {
+          lines{
+            '',
+            '',
+            '',
+            '        8 spaces -> 3 spaces',
+            '',
+            '',
+            '     5 spaces -> 0 space',
+            '',
+            '',
+            '',
+          },
+        },
+        expected = lines{
+          '',
+          '',
+          '',
+          '   8 spaces -> 3 spaces',
+          '',
+          '',
+          '5 spaces -> 0 space',
+          '',
+          '',
+          '',
+        },
+      },
+      {
+        msg = 'no indent',
+        tabstop = 2,
+        args = {
+          lines{
+            '	<Tab> -> 2 spaces',
+            'Here is no indent.',
+            '    4 spaces will remain',
+          },
+        },
+        expected = lines{
+          '  <Tab> -> 2 spaces',
+          'Here is no indent.',
+          '    4 spaces will remain',
+        },
+      },
+      {
+        msg = 'leave_indent = 4',
+        tabstop = 2,
+        args = {
+          lines{
+            '	<Tab> -> 6 spaces',
+            '0 indent -> 4 spaces',
+            '    4 spaces -> 8 spaces',
+          },
+          4,
+        },
+        expected = lines{
+          '      <Tab> -> 6 spaces',
+          '    0 indent -> 4 spaces',
+          '        4 spaces -> 8 spaces',
+        },
+      },
+      {
+        msg = 'typical usecase: <Tab> to 5 spaces',
+        tabstop = 4,
+        args = {
+          lines{
+            '',
+            '		Chapter 1',
+            '',
+            '	  Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed',
+            '	do eiusmod tempor incididunt ut labore et dolore magna aliqua.',
+            '',
+            '	  Ut enim ad minim veniam, quis nostrud exercitation ullamco',
+            '	laboris nisi ut aliquip ex ea commodo consequat.',
+            '',
+          },
+          5,
+        },
+        expected = lines{
+          '',
+          '         Chapter 1',
+          '',
+          '       Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed',
+          '     do eiusmod tempor incididunt ut labore et dolore magna aliqua.',
+          '',
+          '       Ut enim ad minim veniam, quis nostrud exercitation ullamco',
+          '     laboris nisi ut aliquip ex ea commodo consequat.',
+          '',
+        },
+      },
+    } do
+      local msg = ('tabstop = %d, %s'):format(case.tabstop, case.msg)
+      it(msg, function()
+        local original = vim.bo.tabstop
+        vim.bo.tabstop = case.tabstop
+        eq(case.expected, strings.dedent(unpack(case.args)))
+        vim.bo.tabstop = original
+      end)
+    end
+  end)
+end)