Package: img2unicode
img2unicodeA tool to display images as Unicode in your terminal.
The library is currently optimized for Ubuntu Mono font rendered by libvte (Gnome Terminal, Terminator etc.) i.e. it works on stock Ubuntu.pipinstallimg2unicode
imgcatimage.jpgInstallationYou can install img2unicode via pip from PyPI:$pipinstallimg2unicodeIf you want to utilizeFastGammaOptimizer,
install the optional n2 dependency:$pipinstall'img2unicode[n2]'Note thatExactGammaOptimizeris more portable, yet much slower.
UseBestGammaOptimizeralias to choose automatically between these two.Usageimportimg2unicode# Use Unicode Block Elementsoptimizer=img2unicode.FastGenericDualOptimizer("block")renderer=img2unicode.Renderer(default_optimizer=optimizer,max_h=60,max_w=160)renderer.render_terminal('examples/obama.jpg','obama-block.txt')# Pair Renderer to Optimizeroptimizer=img2unicode.FastGammaOptimizer("no_block")# Or ExactGammaOptimizerrenderer=img2unicode.GammaRenderer(default_optimizer=optimizer,max_h=60,max_w=160)renderer.render_terminal('examples/obama.jpg','obama-noblock.txt')pil_image=renderer.prerender('examples/obama.jpg')pil_image.save('obama-rendered.png')# Use own mask: it may be name from common masks (see source), slice or numpy bool array.ascii_optimizer=img2unicode.FastGammaOptimizer(slice(32,127),use_color=False)# Get the characters, foreground and background colors. Use non-default optimizer.chars,fores,backs=renderer.render_numpy('examples/obama.jpg',optimizer=ascii_optimizer)OptimizersHere is a quick comparison of the most usable optimizers:FastQuadDualOptimizer()FastGenericDualOptimizer ("block")FastGammaOptimizer (charmask="no_block")FastGammaOptimizer (charmask="no_block", use_color=False)Choses from 4-pixel characters like ▚Optimizes foreground/background for wholeUnicode Block Elements.Optimizes foreground color for all Unicode rendered in single cell.Same, but does't use terminal colors.Good color representationGood color and crisper imageCrisp edges with blackPure art, no color.Foreground & backgroundForeground & backgroundJust foregroundNo color~5Hz~4Hz~1Hz~2HzO(S*T), T=7O(S*T), T=24O(S*log(T)), T=5553O(S*log(T)), T=5553WhereSis the number of 16x8 pixel samples to optimize for andTis the number of templates.See it yourselfUse the includedtermviewscript to browse images with all renderers. First, install the optional dependency:$pipinstall'img2unicode[termview]'then execute:termviewexamples/obama.jpgor useimgcatto display an image in your terminal:imgcatexamples/obama.jpgMore samplesTo see how other optimizers compare to each other, seeexamples/README.md.To see more eyecandy of photos, videos and plots, see thematrach/img2unicode-demos repoHow FastGammaOptimizer worksimg2unicodeemploys optimization with (Approximate) Nearest Neighbors. For each chunk of an image (i.e. 16x32 px), the tool basically selects a glyph (from a prerendered dataset) that optimizes both:perceptual similarity (implemented as a pixel-by-pixel Euclidean distance between the blurred glyph template and the chunk),visually matching edges (as you can see in the Obama example).The need to use ML arose from the need to support arbitrary Unicode glyphs. This is not easily portable while maintaining the rendering quality, because there is a lot of variability between rendering by different terminal backends (e.g., libvte, kitty, etc.) and fonts.Details of FastGenericDualOptimizerThe algorithm select characters when we control both background and foreground of each piece.In a general case, when using characters with a grayscale channel and awell-behaved color space such as LAB or HCL, the algorithm should minimize the total deviation of each pixel from the selected template.Letcsbe a matrix of char template luma values -- an(N, H*W)float matrix called a mask, whereNis the number of templates andH, Ware the image chunk dimensions.
By $\lVert c \rVert^2$ I will indicate the error in pixel color rendering (squared euclidean distance -- $L_2$ loss function), and $\cdot$ will designate a mask-color combination.
Then, ifsis an image chunk for rendering --(H*W, 3)shaped matrix, my tool would optimize the following:
$$argmin_{i \in 0,\ldots,N} \sum_{p=0}^{WH} \lVert cs_{i,p} \cdot fg_i + (1-cs_{i,p}) \cdot bg_i - s_p \rVert^2$$where $fg$ and $bg$ are the calculated average foreground and background colors, respectively, as follows$$fg_i = \frac{\sum_{p=0}^{WH} (cs_{i,p} \cdot s_p)}{\sum_{p=0}^{WH} cs_{i,p}}$$
$$bg_i = \frac{\sum_{p=0}^{WH} (1-cs_{i,p}) \cdot s_p)}{\sum_{p=0}^{WH} 1-cs_{i,p}}$$
with the assumption, that $0/0$ is $0$. (For some reason, the $\Sigma$ is weirdly rendered by GitHub's MathJax).In the special case that $cs_{i,p}$ is binary (either 0 or 1 - as in the case of Unicode block characters), the first formula may be written equivalently as
$$argmin_{i \in 0,\ldots,N}~ \sum_{p=0}^{WH} \lVert cs_{i,p} \cdot fg_i  - s_p \rVert^2 + \sum_{p=0}^{WH} \lVert (1-cs_{i,p}) \cdot bg_i - s_p \rVert^2$$But in this version, we may convert the initial optimization problem into just:argmax_i~ \lVert
\frac{
  \sum_p cs_{i,p}
}{
  \sqrt{\sum_p cs_{i,p}}
} \cdot s_p \rVert^2
+ \lVert
\frac{\sum_p (1-cs_{i,p})}{\sqrt{\sum_p 1-cs_{i,p}}}  \cdot s_p
\rVert^2This formula allows us to precompute theC_{i,p} \equiv \frac{
  \sum_p cs_{i,p}
}{
  \sqrt{\sum_p cs_{i,p}}
}part as a matrix, (same $C'$ for $1-cs$).
And now we can rewrite the above formula as (using dot-products):
$$argmax_i \lVert C_i \cdot s \rVert ^2 + \lVert C'_i \cdot s \rVert ^2$$
This can be further simplified to matrix multiplication if we have multiple samples to optimize at the same time (the $S$ tensor has shapes(chunks, W*H, 3)).
Then, the optimal characters are selected by calculating just:best\_char(j, S) = (argmax_i  \lVert (C_i \cdot S \rVert^2+ \lVert C'_i \cdot S \rVert^2)_{j}With a bit of juggling, this may be implemented as:$Q$ = query matrix of image chunks(chunks, W*H, 3)$C$ = foreground template tensor with shape extended to(N, W*H, 3). $C'$ - same for background.foreground_match = C.mat_mult(Q)**2- shaped(chunks, N, W*H)background_match = C'.mat_mul(Q)**2- shaped(chunks, N, W*H)scores = (foreground_match + background_match).sum(axis=2)shaped(chunks, N)best_char = scores.argmax(axis=1)-- highest scoring template -- a vector (list) ofchunksintegers of0...N.All that remains is to average the pixels over the mask to compute the appropriate color for the background and foreground.TODOwrite more docs, document codedescribe how dual optimizer works in readmedescribe how gamma optimizer works in readmesolve foreground & background optimization sublinear in templatesadd support for Unicode 13 Legacy Computing blockadd support for edges optimization in FastDualGenericOptimizer