Repository: ehb54/ultrascan3
Pull request #209: Ehb issue 466
Author: login: ehb54
Created: 2025-04-19T20:49:57Z | Merged: 2025-04-25T15:43:09Z

Updates to release build & dockerfiles
better manual building
utilization of qmake && make for faster ultrascan build
ubuntu 24 support



Diff:
diff --git a/admin/release/buildpkg.pl b/admin/release/buildpkg.pl
index 5518ab22e..8ccfade78 100755
--- a/admin/release/buildpkg.pl
+++ b/admin/release/buildpkg.pl
@@ -16,6 +16,7 @@
      "ubuntu:18.04"  => "python"
      ,"ubuntu:20.04" => "python"
      ,"ubuntu:22.04" => "python3"
+     ,"ubuntu:24.04" => "python3"
      ,"redhat:8.10"   => "python3"
     );
 
@@ -59,7 +60,7 @@
 $name = "us3-$image";
 $name =~ s/:/-/;
 $bimage = $image;
-$bimage =~ s/^redhat/rockylinux/;
+$bimage =~ s/^redhat/rockylinux\/rockylinux/;
 
 $cmd =
     "cd $from"
diff --git a/admin/release/redhat/Dockerfile b/admin/release/redhat/Dockerfile
index b02028b5f..90b8559ce 100644
--- a/admin/release/redhat/Dockerfile
+++ b/admin/release/redhat/Dockerfile
@@ -115,10 +115,18 @@ RUN cd /ultrascan3/us_somo/develop && env ULTRASCAN=/ultrascan3 ./revision.sh &&
 # install dependencies for building manual
 RUN dnf install -y doxygen texlive-*
 RUN dnf install -y perl-CPAN && cpan Template
+RUN dnf install -y graphviz
+## upgrade doxygen as the file is often not current in repo
+RUN cd $ULTRASCAN && doxygen -u
 
 # make ultrascan
+## faster qmake, make first
+RUN cd $ULTRASCAN && export PATH=$QTDIR/bin:$PATH && qmake && make -j$parallel_compile
 RUN cd $ULTRASCAN && export PATH=$QTDIR/bin:$PATH && ./makeall.sh -j$parallel_compile
 
+## old versions of makeall.sh fail to build manual as it is doing in parallel, make sure we build the manual
+RUN cd $ULTRASCAN/doc/manual && export PATH=$QTDIR/bin:$PATH && make -j1
+
 # make sure manual exists
 RUN cd $ULTRASCAN && ls -l bin/manual*
 
@@ -141,4 +149,4 @@ RUN cp $QTDIR/bin/assistant /ultrascan3/bin
 
 # make package
 ARG image
-RUN IMAGE=`echo $image | sed s/rockylinux/redhat/ | sed s/:/-/` && echo $IMAGE  && LD_LIBRARY_PATH=/ultrascan3/lib:$LD_LIBRARY_PATH /ultrascan3/admin/release/linux.pl $IMAGE && ls -1 us3-*.tar.xz > /lastpkgname
+RUN IMAGE=`echo $image | sed 's/rockylinux\/rockylinux/redhat/' | sed s/:/-/` && echo $IMAGE  && LD_LIBRARY_PATH=/ultrascan3/lib:$LD_LIBRARY_PATH /ultrascan3/admin/release/linux.pl $IMAGE && ls -1 us3-*.tar.xz > /lastpkgname
diff --git a/admin/release/ubuntu/Dockerfile b/admin/release/ubuntu/Dockerfile
index 18fd875a2..2fe3e889b 100644
--- a/admin/release/ubuntu/Dockerfile
+++ b/admin/release/ubuntu/Dockerfile
@@ -63,7 +63,10 @@ RUN echo 'ls -sxF $*' > /usr/bin/s && chmod +x /usr/bin/s
 RUN echo 'ls -lth $* | head' > /usr/bin/lth && chmod +x /usr/bin/lth
 
 # qt
-RUN sed -Ei 's/^# deb-src /deb-src /' /etc/apt/sources.list
+## the echo at the end of the two lines below allow them to fail, the first is for ubuntu <=22.04, the second for ubuntu >= 24.04
+RUN sed -Ei 's/^# deb-src /deb-src /' /etc/apt/sources.list; echo
+RUN sed -i 's/^Types: deb$/Types: deb deb-src/' /etc/apt/sources.list.d/ubuntu.sources; echo 
+
 RUN apt-get update
 RUN apt-get build-dep -y qtbase5-dev
 #RUN apt-get build-dep -y qt5-default
@@ -77,10 +80,44 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
-RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## if QtWeb needs to be built
+## try to install python2 & put in path
+# RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+# RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+# ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu -skip webengine -skip location
+# qtweb engine requries these changes:
+# COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
+# RUN sed -i '6i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/geometry.hpp
+# RUN sed -i '7i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/string.hpp
+# RUN sed -i '4i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/src/mbgl/gl/stencil_mode.hpp
+
+# RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -k ; echo 0
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -j$parellel_compile install
 RUN cd qt-everywhere-src-$qt_version && nohup make -j$parallel_compile 2>&1 > make.out
 RUN cd qt-everywhere-src-$qt_version && nohup make install -j1 2>&1 > makeinstall.out
-# RUN ls -l /qt-$qt_version/bin /qt-$qt_version/lib
+RUN rm -r qt-everywhere-src-$qt_version
 
 # install qwt
 RUN wget https://versaweb.dl.sourceforge.net/project/qwt/qwt/$qwt_version/qwt-$qwt_version.tar.bz2 && tar jxf qwt-$qwt_version.tar.bz2 && cd qwt-$qwt_version && /qt-$qt_version/bin/qmake && make -j$parallel_compile
@@ -106,12 +143,21 @@ RUN cd /ultrascan3/programs/us && env ULTRASCAN=/ultrascan3 ./revision.sh
 RUN cd /ultrascan3/us_somo/develop && env ULTRASCAN=/ultrascan3 ./revision.sh && env us3=/ultrascan3 ./version.sh
 
 # install dependencies for building manual
-RUN apt-get install -y doxygen texlive-full
+RUN apt-get install -y doxygen
+RUN apt-get install -y texlive-full
 RUN cpan Template
+RUN apt-get install -y graphviz
+## upgrade doxygen as the file is often not current in repo
+RUN cd $ULTRASCAN && doxygen -u
 
 # make ultrascan
+## faster qmake, make first
+RUN cd $ULTRASCAN && export PATH=$QTDIR/bin:$PATH && qmake && make -j$parallel_compile
 RUN cd $ULTRASCAN && export PATH=$QTDIR/bin:$PATH && ./makeall.sh -j$parallel_compile
 
+## old versions of makeall.sh fail to build manual as it is doing in parallel, make sure we build the manual
+RUN cd $ULTRASCAN/doc/manual && export PATH=$QTDIR/bin:$PATH && make -j1
+
 # make sure manual exists
 RUN cd $ULTRASCAN && ls -l bin/manual*
 
@@ -139,3 +185,4 @@ RUN cp $QTDIR/bin/assistant /ultrascan3/bin
 # make package
 ARG image
 RUN LD_LIBRARY_PATH=/ultrascan3/lib:$LD_LIBRARY_PATH /ultrascan3/admin/release/linux.pl `echo $image | sed s/:/-/` && ls -1 us3-ubuntu*.tar.xz > /lastpkgname
+
diff --git a/admin/release/ubuntu/mathops.h b/admin/release/ubuntu/mathops.h
new file mode 100644
index 000000000..ca7e2dffc
--- /dev/null
+++ b/admin/release/ubuntu/mathops.h
@@ -0,0 +1,153 @@
+/*
+ * simple math operations
+ * Copyright (c) 2006 Michael Niedermayer <michaelni@gmx.at> et al
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVCODEC_X86_MATHOPS_H
+#define AVCODEC_X86_MATHOPS_H
+
+#include "config.h"
+
+#include "libavutil/common.h"
+#include "libavutil/x86/asm.h"
+
+#if HAVE_INLINE_ASM
+
+#if ARCH_X86_32
+
+#define MULL MULL
+static av_always_inline av_const int MULL(int a, int b, unsigned shift)
+{
+    int rt, dummy;
+    if (__builtin_constant_p(shift))
+    __asm__ (
+        "imull %3               \n\t"
+        "shrdl %4, %%edx, %%eax \n\t"
+        :"=a"(rt), "=d"(dummy)
+        :"a"(a), "rm"(b), "i"(shift & 0x1F)
+    );
+    else
+        __asm__ (
+            "imull %3               \n\t"
+            "shrdl %4, %%edx, %%eax \n\t"
+            :"=a"(rt), "=d"(dummy)
+            :"a"(a), "rm"(b), "c"((uint8_t)shift)
+        );
+    return rt;
+}
+
+#define MULH MULH
+static av_always_inline av_const int MULH(int a, int b)
+{
+    int rt, dummy;
+    __asm__ (
+        "imull %3"
+        :"=d"(rt), "=a"(dummy)
+        :"a"(a), "rm"(b)
+    );
+    return rt;
+}
+
+#define MUL64 MUL64
+static av_always_inline av_const int64_t MUL64(int a, int b)
+{
+    int64_t rt;
+    __asm__ (
+        "imull %2"
+        :"=A"(rt)
+        :"a"(a), "rm"(b)
+    );
+    return rt;
+}
+
+#endif /* ARCH_X86_32 */
+
+#if HAVE_I686
+/* median of 3 */
+#define mid_pred mid_pred
+static inline av_const int mid_pred(int a, int b, int c)
+{
+    int i=b;
+    __asm__ (
+        "cmp    %2, %1 \n\t"
+        "cmovg  %1, %0 \n\t"
+        "cmovg  %2, %1 \n\t"
+        "cmp    %3, %1 \n\t"
+        "cmovl  %3, %1 \n\t"
+        "cmp    %1, %0 \n\t"
+        "cmovg  %1, %0 \n\t"
+        :"+&r"(i), "+&r"(a)
+        :"r"(b), "r"(c)
+    );
+    return i;
+}
+
+#if HAVE_6REGS
+#define COPY3_IF_LT(x, y, a, b, c, d)\
+__asm__ volatile(\
+    "cmpl  %0, %3       \n\t"\
+    "cmovl %3, %0       \n\t"\
+    "cmovl %4, %1       \n\t"\
+    "cmovl %5, %2       \n\t"\
+    : "+&r" (x), "+&r" (a), "+r" (c)\
+    : "r" (y), "r" (b), "r" (d)\
+);
+#endif /* HAVE_6REGS */
+
+#endif /* HAVE_I686 */
+
+#define MASK_ABS(mask, level)                   \
+    __asm__ ("cdq                    \n\t"      \
+             "xorl %1, %0            \n\t"      \
+             "subl %1, %0            \n\t"      \
+             : "+a"(level), "=&d"(mask))
+
+// avoid +32 for shift optimization (gcc should do that ...)
+#define NEG_SSR32 NEG_SSR32
+static inline  int32_t NEG_SSR32( int32_t a, int8_t s){
+    if (__builtin_constant_p(s))
+    __asm__ ("sarl %1, %0\n\t"
+         : "+r" (a)
+         : "i" (-s & 0x1F)
+    );
+    else
+        __asm__ ("sarl %1, %0\n\t"
+               : "+r" (a)
+               : "c" ((uint8_t)(-s))
+        );
+    return a;
+}
+
+#define NEG_USR32 NEG_USR32
+static inline uint32_t NEG_USR32(uint32_t a, int8_t s){
+    if (__builtin_constant_p(s))
+    __asm__ ("shrl %1, %0\n\t"
+         : "+r" (a)
+         : "i" (-s & 0x1F)
+    );
+    else
+        __asm__ ("shrl %1, %0\n\t"
+               : "+r" (a)
+               : "c" ((uint8_t)(-s))
+        );
+    return a;
+}
+
+#endif /* HAVE_INLINE_ASM */
+#endif /* AVCODEC_X86_MATHOPS_H */
diff --git a/admin/release/ubuntu_qt6/Dockerfile b/admin/release/ubuntu_qt6/Dockerfile
index 4eb6251d5..435da55ff 100644
--- a/admin/release/ubuntu_qt6/Dockerfile
+++ b/admin/release/ubuntu_qt6/Dockerfile
@@ -3,7 +3,7 @@
 
 ### exposed ARGs override defaults with --build-arg <varname>=<value>
 
-ARG image=ubuntu:23.10
+ARG image=ubuntu:24.04
 ## ARGs are scoped FROM
 FROM $image
 


Reviews:
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-19T20:52:57Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-19T20:53:47Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-19T20:55:16Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-19T20:56:25Z
user: login: doluk
body: building any ubuntu docker using `buildpkg.pl` failed with the error below
```
=> ERROR [50/86] COPY mathops.h /qt-everywhere-src-5.15.16/qtwebengine/src/3rdparty/chromium/third_par  0.0s
------
 > [50/86] COPY mathops.h /qt-everywhere-src-5.15.16/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h:
------
Dockerfile:108
--------------------
 106 |
 107 |     RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
 108 | >>> COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
 109 |     RUN sed -i '6i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/geometry.hpp
 110 |     RUN sed -i '7i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/string.hpp
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 6533fe9b-3957-458f-9054-650c8e9b4583::v0paexk3077qjz2iwdqxkzlls: "/mathops.h": not found
--------------------------------------------------------------------------------
```
redhat build worked fine
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-04-23T19:32:17Z
user: login: doluk
body: I made a while back some docker images for building ultrascan3 with debian (started for ubuntu too, but never finished) the resulting images were a lot smaller than those (around 3 GB compared to 30 GB). You can find the dockerfiles for that on github https://github.com/doluk/us3docker/blob/main/Dockerfile.base-ubuntu-latest 
Not saying that mine is production ready, but there has to be someway to reduce the size of those images and the build time
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-04-24T12:38:48Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-24T12:50:18Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-24T12:52:18Z
user: login: ehb54
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-24T12:54:04Z
user: login: doluk
body: lgtm
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-24T16:23:14Z

Review comments:
diff_hunk: @@ -115,10 +115,18 @@ RUN cd /ultrascan3/us_somo/develop && env ULTRASCAN=/ultrascan3 ./revision.sh &&
 # install dependencies for building manual
 RUN dnf install -y doxygen texlive-*
 RUN dnf install -y perl-CPAN && cpan Template
+RUN dnf install -y graphviz
path: admin/release/redhat/Dockerfile
user: login: ehb54
body: doxygen builds require dot from graphviz
created_at: 2025-04-19T20:52:56Z
author_association: OWNER
diff_hunk: @@ -141,4 +149,4 @@ RUN cp $QTDIR/bin/assistant /ultrascan3/bin
 
 # make package
 ARG image
-RUN IMAGE=`echo $image | sed s/rockylinux/redhat/ | sed s/:/-/` && echo $IMAGE  && LD_LIBRARY_PATH=/ultrascan3/lib:$LD_LIBRARY_PATH /ultrascan3/admin/release/linux.pl $IMAGE && ls -1 us3-*.tar.xz > /lastpkgname
+RUN IMAGE=`echo $image | sed 's/rockylinux\/rockylinux/redhat/' | sed s/:/-/` && echo $IMAGE  && LD_LIBRARY_PATH=/ultrascan3/lib:$LD_LIBRARY_PATH /ultrascan3/admin/release/linux.pl $IMAGE && ls -1 us3-*.tar.xz > /lastpkgname
path: admin/release/redhat/Dockerfile
user: login: ehb54
body: rockylinux docker became rockylinux/rockylinux for 8.10
created_at: 2025-04-19T20:53:47Z
author_association: OWNER
path: admin/release/ubuntu_qt6/Dockerfile
user: login: ehb54
body: this is just a test qt6 Dockerfile, in no way production
created_at: 2025-04-19T20:55:16Z
author_association: OWNER
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
path: admin/release/ubuntu/Dockerfile
user: login: ehb54
body: Kept this commented section for reference in case I want to use conda to install older versions of gcc
created_at: 2025-04-19T20:56:25Z
author_association: OWNER
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
path: admin/release/ubuntu/Dockerfile
user: login: doluk
body: For what do we need the anaconda2?
created_at: 2025-04-24T11:41:01Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
path: admin/release/ubuntu/Dockerfile
user: login: doluk
body: You said, we need this to build QtWebEngine, but we don't use the Webengine at all. We onyl Use the WebChannel and WebSocket. But even those are only copied as lib file during packaging. At the end is all webstuff we need part of QtNetwork. Further more QtWebEnginge contains a Chromium instance which consumes quite some disk space.
created_at: 2025-04-24T11:52:58Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
+RUN sed -i '6i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/geometry.hpp
+RUN sed -i '7i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/string.hpp
+RUN sed -i '4i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/src/mbgl/gl/stencil_mode.hpp
+
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -k ; echo 0
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -j$parellel_compile install
 RUN cd qt-everywhere-src-$qt_version && nohup make -j$parallel_compile 2>&1 > make.out
 RUN cd qt-everywhere-src-$qt_version && nohup make install -j1 2>&1 > makeinstall.out
path: admin/release/ubuntu/Dockerfile
user: login: doluk
body: We have now 4 lines of make commands for qt. some of them have to be duplicates
created_at: 2025-04-24T12:04:45Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
+RUN sed -i '6i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/geometry.hpp
+RUN sed -i '7i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/string.hpp
+RUN sed -i '4i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/src/mbgl/gl/stencil_mode.hpp
+
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -k ; echo 0
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -j$parellel_compile install
 RUN cd qt-everywhere-src-$qt_version && nohup make -j$parallel_compile 2>&1 > make.out
 RUN cd qt-everywhere-src-$qt_version && nohup make install -j1 2>&1 > makeinstall.out
 # RUN ls -l /qt-$qt_version/bin /qt-$qt_version/lib
path: admin/release/ubuntu/Dockerfile
user: login: doluk
body: ```suggestion
RUN rm -r qt-everywhere-src-$qt_version
```
We don't need the source code of qt anymore after compiling it. That makes the resulting image 15 GB smalller.
created_at: 2025-04-24T12:27:51Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
path: admin/release/ubuntu/Dockerfile
user: login: doluk
body: Compiling it without QtWebEngine and QtLocation (we don't need either), saves around 4 GB without the deletion of the source and build directory after installation as suggested further down.
created_at: 2025-04-24T12:29:54Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
+RUN sed -i '6i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/geometry.hpp
+RUN sed -i '7i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/include/mbgl/util/string.hpp
+RUN sed -i '4i #include \<cstdint\>' /qt-everywhere-src-$qt_version/qtlocation/src/3rdparty/mapbox-gl-native/src/mbgl/gl/stencil_mode.hpp
+
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -k ; echo 0
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make
+RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile make -j$parellel_compile install
 RUN cd qt-everywhere-src-$qt_version && nohup make -j$parallel_compile 2>&1 > make.out
 RUN cd qt-everywhere-src-$qt_version && nohup make install -j1 2>&1 > makeinstall.out
path: admin/release/ubuntu/Dockerfile
user: login: ehb54
body: Ideally. Have seen cases where the parallel make has some dependency race conditions. Extra make makes sure. This is just to build packages. Not worried.

created_at: 2025-04-24T12:50:18Z
author_association: OWNER
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
path: admin/release/ubuntu/Dockerfile
user: login: ehb54
body: For python2, not available by default in ubuntu 24
created_at: 2025-04-24T12:52:18Z
author_association: OWNER
diff_hunk: @@ -77,7 +80,39 @@ RUN mkdir /usr/X11R6
 
 ## required for qt to find xcb-xlib deps
 RUN ln -s /usr/lib/x86_64-linux-gnu /usr/X11R6/lib64
+
+#RUN wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
+#RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b -p /opt/anaconda3
+#RUN ls -l /opt/anaconda3
+#RUN /opt/anaconda3/bin/conda install -y -v -c conda-forge gcc=12.4.0
+#RUN /opt/anaconda3/bin/gcc --version
+#ENV PATH="/opt/anaconda3/bin:${PATH}"
+RUN gcc --version
+#RUN apt-get install -y clang
+RUN apt-get install -y libclang-dev
+RUN apt-get install -y gperf bison flex
+RUN apt-get install -y llvm
+RUN which llvm-config 
+
+## try to install python2 & put in path
+RUN wget https://repo.anaconda.com/archive/Anaconda2-2019.10-Linux-x86_64.sh
+RUN bash Anaconda2-2019.10-Linux-x86_64.sh -b -p /opt/anaconda2
+#RUN ls -l /opt/anaconda2/bin
+ENV PATH="/opt/anaconda2/bin:${PATH}"
+
+RUN apt-get install -y nodejs
+RUN apt-get install -y libnss3-dev
+RUN apt-get install -y libxkbfile-dev
+
 RUN cd qt-everywhere-src-$qt_version && env MAKEFLAGS=-j$parallel_compile ./configure -prefix /qt-$qt_version -release -opensource -confirm-license -platform linux-g++-64 -nomake tests -nomake examples -xcb -xcb-xlib -D QT_SHAREDMEMORY -D QT_SYSTEMSEMAPHORE -no-icu
+COPY mathops.h /qt-everywhere-src-$qt_version/qtwebengine/src/3rdparty/chromium/third_party/ffmpeg/libavcodec/x86/mathops.h
path: admin/release/ubuntu/Dockerfile
user: login: ehb54
body: If our goal is to make an distributable docker image, then perhaps. Never know what new dependency requirement might come up in future, so don't want to have revisit this because we added code using a Qt feature.

created_at: 2025-04-24T12:54:04Z
author_association: OWNER

PR comments:
user: login: ehb54
created_at: 2025-04-23T17:34:28Z
body: @samo38 @doluk 
added you both as reviewers.
at least perhaps one can test actually running the build of a package and/or testing the package i built in downloads.
 
author_association: OWNER
user: login: doluk
created_at: 2025-04-23T20:46:28Z
body: commenting the line out let me build ubuntu:18.04, but ubuntu:24.04 still fails

author_association: COLLABORATOR
user: login: ehb54
created_at: 2025-04-23T21:22:56Z
body: @doluk thanks for finding this issue. Added "mathops.h" to branch required for Ubuntu 24 build.

author_association: OWNER
user: login: doluk
created_at: 2025-04-24T06:05:55Z
body: it is required by the dockerfile for all versions for ubuntu. I was actually wondering if we need all the different ubuntu versions, as long as the gcc version stays the same, is a complete compile from zero needed? I see that the approach is to be better safe than sorry, but in case we would want to cover more distributions at some point this would be a good thing to reduce the packaging times.


And when we are already having a full dockerimage anyway, wouldn't it make sense to publish that as well? So the people could use "ultrascan in a docker"? I have seen multiple cases of software being used like that to provide a consistent environment without having to deal with platform/os specific stuff. And it would offer a great isolation/security option for commercial users
https://www.geeksforgeeks.org/how-to-run-gui-based-applications-inside-docker/
https://github.com/mviereck/x11docker
https://medium.com/geekculture/run-a-gui-software-inside-a-docker-container-dce61771f9#:~:text=Displaying%20a%20GUI%2Dbased%20application,source%20file%20of%20many%20locales.
to give some references 
author_association: COLLABORATOR
user: login: ehb54
created_at: 2025-04-24T13:09:41Z
body: The purpose of these Dockerfiles is simply to build packages.
If we wish to build docker images to distribute, I agree further modifications 
would be needed & I suggest a separate Dockerfile and wrapping script to build (or simply docker buildx ...). In that case, perhaps a Debain minimal base image would be preferable and then also remove everything that is not essential (e.g. don't even need the compiler after building). Completely different purpose. Of course, if we were going to offer this to users, they would need to install docker, which is commercial so some possible complications for non-academic users there. Linux docker images can run on linux, windows and mac, but running linux intel containers on ARM macs isn't perfect, so we would also need to build ARM mac containers. Lots of testing in different environments to validate.

I don't know if anyone is even using the linux binary versions.
I build & run gui apps from docker on linux & macs frequently. 
I have had requests from US-SOMO users to get a package for their specific linux version, but not recently.

The time to build the images will likely decrease significantly once we move to qt6, as I'm hoping we can just pull prebuilt Qt devel packages instead of building qt from source (TBD).


author_association: OWNER
user: login: doluk
created_at: 2025-04-24T14:59:54Z
body: I checked the redhat package, which worked fine. Even the help worked properly for me.
author_association: COLLABORATOR
user: login: doluk
created_at: 2025-04-24T16:04:48Z
body: The build went through with the latest changes.
author_association: COLLABORATOR
user: login: ehb54
created_at: 2025-04-24T16:17:22Z
body: @doluk merging is blocked by your requested changes, can you resolve this?

author_association: OWNER