Repository: LadybirdBrowser/ladybird
Pull request #4830: LibCore+LibGfx: Rely on fontconfig(if enabled) for font directory resolution
Author: login: blukai
Created: 2025-05-21T01:18:55Z | Merged: 2025-05-26T18:14:29Z

first timer here.

i run a rather obscure linux and none of the fallback fonts that are listed in update_generic_fonts are present on my system. my initial clue was to try to enable fontconfig, but result was fruitless. eventually after a bunch of printf (or in this case outln) debugging i figured out that the issues was that ladybird was not able to load the fonts that fontconfig was resolving because ladybird was not trying look into directories in which fontconfig has found fonts.

this might be related to #4601

i moved font_directories from StandardPaths to FontDatabase because it didn't seem like it would be appropriate to call to fontconfig funcs from within the LibCore (based on my brief observations of how everything is structured).
i'm not 100% confident that FontDatabase is the correct place.. but seems okay?

besides issue with fonts it was a rather straightforward sailing, getting to build the whole thing. thanks for your efforts.


Diff:
diff --git a/Libraries/LibCore/StandardPaths.cpp b/Libraries/LibCore/StandardPaths.cpp
index 2b454a6486cea..8ba738e8d02bb 100644
--- a/Libraries/LibCore/StandardPaths.cpp
+++ b/Libraries/LibCore/StandardPaths.cpp
@@ -22,10 +22,6 @@
 #    include <unistd.h>
 #endif
 
-#if defined(AK_OS_HAIKU)
-#    include <FindDirectory.h>
-#endif
-
 namespace Core {
 
 static Optional<StringView> get_environment_if_not_empty(StringView name)
@@ -237,49 +233,4 @@ ByteString StandardPaths::tempfile_directory()
 #endif
 }
 
-ErrorOr<Vector<String>> StandardPaths::font_directories()
-{
-#if defined(AK_OS_HAIKU)
-    Vector<String> paths_vector;
-    char** paths;
-    size_t paths_count;
-    if (find_paths(B_FIND_PATH_FONTS_DIRECTORY, NULL, &paths, &paths_count) == B_OK) {
-        for (size_t i = 0; i < paths_count; ++i) {
-            StringBuilder builder;
-            builder.append(paths[i], strlen(paths[i]));
-            paths_vector.append(TRY(builder.to_string()));
-        }
-    }
-    return paths_vector;
-#else
-    auto paths = Vector { {
-#    if defined(AK_OS_SERENITY)
-        "/res/fonts"_string,
-#    elif defined(AK_OS_MACOS)
-        "/System/Library/Fonts"_string,
-        "/Library/Fonts"_string,
-        TRY(String::formatted("{}/Library/Fonts"sv, home_directory())),
-#    elif defined(AK_OS_ANDROID)
-        // FIXME: We should be using the ASystemFontIterator NDK API here.
-        // There is no guarantee that this will continue to exist on future versions of Android.
-        "/system/fonts"_string,
-#    elif defined(AK_OS_WINDOWS)
-        TRY(String::formatted(R"({}\Fonts)"sv, getenv("WINDIR"))),
-        TRY(String::formatted(R"({}\Microsoft\Windows\Fonts)"sv, getenv("LOCALAPPDATA"))),
-#    else
-        TRY(String::formatted("{}/fonts", user_data_directory())),
-        TRY(String::formatted("{}/X11/fonts", user_data_directory())),
-#    endif
-    } };
-#    if !(defined(AK_OS_SERENITY) || defined(AK_OS_MACOS) || defined(AK_OS_WINDOWS))
-    auto data_directories = system_data_directories();
-    for (auto& data_directory : data_directories) {
-        paths.append(TRY(String::formatted("{}/fonts", data_directory)));
-        paths.append(TRY(String::formatted("{}/X11/fonts", data_directory)));
-    }
-#    endif
-    return paths;
-#endif
-}
-
 }
diff --git a/Libraries/LibCore/StandardPaths.h b/Libraries/LibCore/StandardPaths.h
index e018334014297..028f860027a8b 100644
--- a/Libraries/LibCore/StandardPaths.h
+++ b/Libraries/LibCore/StandardPaths.h
@@ -26,7 +26,6 @@ class StandardPaths {
     static ByteString user_data_directory();
     static Vector<ByteString> system_data_directories();
     static ErrorOr<ByteString> runtime_directory();
-    static ErrorOr<Vector<String>> font_directories();
 };
 
 }
diff --git a/Libraries/LibGfx/CMakeLists.txt b/Libraries/LibGfx/CMakeLists.txt
index aa736e7b98d6c..7cd9beba6db4e 100644
--- a/Libraries/LibGfx/CMakeLists.txt
+++ b/Libraries/LibGfx/CMakeLists.txt
@@ -72,6 +72,10 @@ if (HAS_VULKAN)
     list(APPEND SOURCES VulkanContext.cpp)
 endif()
 
+if (HAS_FONTCONFIG)
+    list(APPEND SOURCES Font/GlobalFontConfig.cpp)
+endif()
+
 serenity_lib(LibGfx gfx)
 
 target_link_libraries(LibGfx PRIVATE LibCompress LibCore LibCrypto LibFileSystem LibRIFF LibTextCodec LibIPC LibUnicode LibURL)
diff --git a/Libraries/LibGfx/Font/FontDatabase.cpp b/Libraries/LibGfx/Font/FontDatabase.cpp
index 9317b7eabc362..fa302cd1ffd87 100644
--- a/Libraries/LibGfx/Font/FontDatabase.cpp
+++ b/Libraries/LibGfx/Font/FontDatabase.cpp
@@ -5,9 +5,18 @@
  */
 
 #include <AK/FlyString.h>
+#include <LibCore/StandardPaths.h>
 #include <LibGfx/Font/Font.h>
 #include <LibGfx/Font/FontDatabase.h>
 
+#if defined(AK_OS_HAIKU)
+#    include <FindDirectory.h>
+#endif
+
+#ifdef USE_FONTCONFIG
+#    include <LibGfx/Font/GlobalFontConfig.h>
+#endif
+
 namespace Gfx {
 
 // Key function for SystemFontProvider to emit the vtable here
@@ -44,4 +53,73 @@ void FontDatabase::for_each_typeface_with_family_name(FlyString const& family_na
     m_system_font_provider->for_each_typeface_with_family_name(family_name, move(callback));
 }
 
+ErrorOr<Vector<String>> FontDatabase::font_directories()
+{
+#if defined(USE_FONTCONFIG)
+    Vector<String> paths;
+    FcConfig* config = Gfx::GlobalFontConfig::the().get();
+    FcStrList* dirs = FcConfigGetFontDirs(config);
+    while (FcChar8* dir = FcStrListNext(dirs)) {
+        char const* dir_cstring = reinterpret_cast<char const*>(dir);
+        paths.append(TRY(String::from_utf8(StringView { dir_cstring, strlen(dir_cstring) })));
+    }
+    return paths;
+
+#elif defined(AK_OS_HAIKU)
+    Vector<String> paths_vector;
+    char** paths;
+    size_t paths_count;
+    if (find_paths(B_FIND_PATH_FONTS_DIRECTORY, NULL, &paths, &paths_count) == B_OK) {
+        for (size_t i = 0; i < paths_count; ++i) {
+            StringBuilder builder;
+            builder.append(paths[i], strlen(paths[i]));
+            paths_vector.append(TRY(builder.to_string()));
+        }
+    }
+    return paths_vector;
+
+#else
+#    if defined(AK_OS_SERENITY)
+    return Vector<String> { {
+        "/res/fonts"_string,
+    } };
+
+#    elif defined(AK_OS_MACOS)
+    return Vector<String> { {
+        "/System/Library/Fonts"_string,
+        "/Library/Fonts"_string,
+        TRY(String::formatted("{}/Library/Fonts"sv, Core::StandardPaths::home_directory())),
+    } };
+
+#    elif defined(AK_OS_ANDROID)
+    return Vector<String> { {
+        // FIXME: We should be using the ASystemFontIterator NDK API here.
+        // There is no guarantee that this will continue to exist on future versions of Android.
+        "/system/fonts"_string,
+    } };
+
+#    elif defined(AK_OS_WINDOWS)
+    return Vector<String> { {
+        TRY(String::formatted(R"({}\Fonts)"sv, getenv("WINDIR"))),
+        TRY(String::formatted(R"({}\Microsoft\Windows\Fonts)"sv, getenv("LOCALAPPDATA"))),
+    } };
+
+#    else
+    Vector<String> paths;
+
+    auto user_data_directory = Core::StandardPaths::user_data_directory();
+    paths.append(TRY(String::formatted("{}/fonts", user_data_directory)));
+    paths.append(TRY(String::formatted("{}/X11/fonts", user_data_directory)));
+
+    auto data_directories = Core::StandardPaths::system_data_directories();
+    for (auto& data_directory : data_directories) {
+        paths.append(TRY(String::formatted("{}/fonts", data_directory)));
+        paths.append(TRY(String::formatted("{}/X11/fonts", data_directory)));
+    }
+
+    return paths;
+#    endif
+#endif
+}
+
 }
diff --git a/Libraries/LibGfx/Font/FontDatabase.h b/Libraries/LibGfx/Font/FontDatabase.h
index 2787a2a1a9a57..a4bd849640970 100644
--- a/Libraries/LibGfx/Font/FontDatabase.h
+++ b/Libraries/LibGfx/Font/FontDatabase.h
@@ -32,6 +32,8 @@ class FontDatabase {
     void for_each_typeface_with_family_name(FlyString const& family_name, Function<void(Typeface const&)>);
     [[nodiscard]] StringView system_font_provider_name() const;
 
+    static ErrorOr<Vector<String>> font_directories();
+
 private:
     FontDatabase();
     ~FontDatabase() = default;
diff --git a/Libraries/LibGfx/Font/GlobalFontConfig.cpp b/Libraries/LibGfx/Font/GlobalFontConfig.cpp
new file mode 100644
index 0000000000000..85c27dbe51821
--- /dev/null
+++ b/Libraries/LibGfx/Font/GlobalFontConfig.cpp
@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2025, blukai <init1@protonmail.com>
+ *
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#include <AK/Assertions.h>
+#include <LibGfx/Font/GlobalFontConfig.h>
+#include <fontconfig/fontconfig.h>
+
+namespace Gfx {
+
+GlobalFontConfig::GlobalFontConfig()
+{
+    FcBool inited = FcInit();
+    VERIFY(inited);
+
+    m_config = FcConfigGetCurrent();
+    FcConfigReference(m_config);
+}
+
+GlobalFontConfig::~GlobalFontConfig()
+{
+    FcConfigDestroy(m_config);
+}
+
+GlobalFontConfig& GlobalFontConfig::the()
+{
+    static GlobalFontConfig s_the;
+    return s_the;
+}
+
+FcConfig* GlobalFontConfig::get()
+{
+    return m_config;
+}
+
+}
diff --git a/Libraries/LibGfx/Font/GlobalFontConfig.h b/Libraries/LibGfx/Font/GlobalFontConfig.h
new file mode 100644
index 0000000000000..4b6920715656a
--- /dev/null
+++ b/Libraries/LibGfx/Font/GlobalFontConfig.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright (c) 2025, blukai <init1@protonmail.com>
+ *
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#pragma once
+
+#include <fontconfig/fontconfig.h>
+
+namespace Gfx {
+
+class GlobalFontConfig {
+public:
+    static GlobalFontConfig& the();
+    FcConfig* get();
+
+private:
+    GlobalFontConfig();
+    ~GlobalFontConfig();
+
+    GlobalFontConfig(GlobalFontConfig const&) = delete;
+    GlobalFontConfig& operator=(GlobalFontConfig const&) = delete;
+
+    FcConfig* m_config;
+};
+
+}
diff --git a/Libraries/LibGfx/Font/PathFontProvider.cpp b/Libraries/LibGfx/Font/PathFontProvider.cpp
index 4886897e9c48a..f9c3e6181bb41 100644
--- a/Libraries/LibGfx/Font/PathFontProvider.cpp
+++ b/Libraries/LibGfx/Font/PathFontProvider.cpp
@@ -31,8 +31,7 @@ void PathFontProvider::load_all_fonts_from_uri(StringView uri)
     root->for_each_descendant_file([this](Core::Resource const& resource) -> IterationDecision {
         auto uri = resource.uri();
         auto path = LexicalPath(uri.bytes_as_string_view());
-        if (path.has_extension(".ttf"sv) || path.has_extension(".ttc"sv)) {
-            // FIXME: What about .otf
+        if (path.has_extension(".ttf"sv) || path.has_extension(".ttc"sv) || path.has_extension(".otf"sv)) {
             if (auto font_or_error = Typeface::try_load_from_resource(resource); !font_or_error.is_error()) {
                 auto font = font_or_error.release_value();
                 auto& family = m_typeface_by_family.ensure(font->family(), [] {
diff --git a/Libraries/LibWebView/Plugins/FontPlugin.cpp b/Libraries/LibWebView/Plugins/FontPlugin.cpp
index efb5aaf106056..33f47516d0122 100644
--- a/Libraries/LibWebView/Plugins/FontPlugin.cpp
+++ b/Libraries/LibWebView/Plugins/FontPlugin.cpp
@@ -16,7 +16,7 @@
 #include <LibWebView/Plugins/FontPlugin.h>
 
 #ifdef USE_FONTCONFIG
-#    include <fontconfig/fontconfig.h>
+#    include <LibGfx/Font/GlobalFontConfig.h>
 #endif
 
 namespace WebView {
@@ -24,19 +24,12 @@ namespace WebView {
 FontPlugin::FontPlugin(bool is_layout_test_mode, Gfx::SystemFontProvider* font_provider)
     : m_is_layout_test_mode(is_layout_test_mode)
 {
-#ifdef USE_FONTCONFIG
-    {
-        auto fontconfig_initialized = FcInit();
-        VERIFY(fontconfig_initialized);
-    }
-#endif
-
     if (!font_provider)
         font_provider = &static_cast<Gfx::PathFontProvider&>(Gfx::FontDatabase::the().install_system_font_provider(make<Gfx::PathFontProvider>()));
     if (is<Gfx::PathFontProvider>(*font_provider)) {
         auto& path_font_provider = static_cast<Gfx::PathFontProvider&>(*font_provider);
         // Load anything we can find in the system's font directories
-        for (auto const& path : Core::StandardPaths::font_directories().release_value_but_fixme_should_propagate_errors())
+        for (auto const& path : Gfx::FontDatabase::font_directories().release_value_but_fixme_should_propagate_errors())
             path_font_provider.load_all_fonts_from_uri(MUST(String::formatted("file://{}", path)));
     }
 
@@ -114,7 +107,7 @@ static Optional<String> query_fontconfig_for_generic_family(Web::Platform::Gener
         VERIFY_NOT_REACHED();
     }
 
-    auto* config = FcConfigGetCurrent();
+    auto* config = Gfx::GlobalFontConfig::the().get();
     VERIFY(config);
 
     FcPattern* pattern = FcNameParse(reinterpret_cast<FcChar8 const*>(pattern_string));


Reviews:
user: login: ADKaster
body: I think if we make these two changes, we can get this in and unbreak font loading on a lot of systems 
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-21T22:38:21Z
user: login: blukai
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T15:30:45Z
user: login: ADKaster
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-22T15:37:22Z
user: login: ADKaster
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-26T18:14:22Z

Review comments:
diff_hunk: @@ -237,49 +237,4 @@ ByteString StandardPaths::tempfile_directory()
 #endif
 }
 
-ErrorOr<Vector<String>> StandardPaths::font_directories()
path: Libraries/LibCore/StandardPaths.cpp
user: login: ADKaster
body: this commit should be able to remove FontDirectory.h for Haiku from StandardPaths.cpp
created_at: 2025-05-21T22:22:17Z
author_association: MEMBER
diff_hunk: @@ -82,7 +82,7 @@ ErrorOr<Vector<String>> FontDatabase::font_directories()
 #    elif defined(AK_OS_WINDOWS)
         TRY(String::formatted(R"({}\Fonts)"sv, getenv("WINDIR"))),
         TRY(String::formatted(R"({}\Microsoft\Windows\Fonts)"sv, getenv("LOCALAPPDATA"))),
-#    else
+#    elif !defined(USE_FONTCONFIG)
path: Libraries/LibGfx/Font/FontDatabase.cpp
user: login: ADKaster
body: I believe this will also double search /usr/share/fonts and /usr/share/X11/fonts due to the system_data_directories goo below.

This method is such a giant mess of ifdefs, it might make more sense to frontload this new logic and do ...

```
#if defined(USE_FONTCONFIG)
Vector<String> paths;
// your new code
return paths;
#elif defined(AK_OS_HAIKU)
// existing haiku path
# else
// existing else that scans user_data_directory and system_data_directories
#endif
```
back in the commit that added your USE_FONTCONFIG path to start with.
created_at: 2025-05-21T22:38:02Z
author_association: MEMBER
diff_hunk: @@ -82,7 +82,7 @@ ErrorOr<Vector<String>> FontDatabase::font_directories()
 #    elif defined(AK_OS_WINDOWS)
         TRY(String::formatted(R"({}\Fonts)"sv, getenv("WINDIR"))),
         TRY(String::formatted(R"({}\Microsoft\Windows\Fonts)"sv, getenv("LOCALAPPDATA"))),
-#    else
+#    elif !defined(USE_FONTCONFIG)
path: Libraries/LibGfx/Font/FontDatabase.cpp
user: login: blukai
body: done!
additionally added some more logical separation and breathing space between ifdefs. things feel a bit nicer now.
created_at: 2025-05-22T15:30:45Z
author_association: CONTRIBUTOR

PR comments:
user: login: ADKaster
created_at: 2025-05-21T18:18:07Z
body: Thanks for the PR! I tried to tackle this issue more fully in https://github.com/LadybirdBrowser/ladybird/pull/4291, where the idea is to create a FontConfigFontProvider that will lazily cache the fonts needed by the WebContent process by loading the bytes directly from fontconfig.

Simply adding the fontconfig global directories didn't occur to me as a partial solution. I think if we accept this partial solution, we probably want to only look in fontconfig's directories if we're using fontconfig? Rather than looking there *and* in $XDG_DATA_HOME/fonts.

This still has the issue that once we know about these directories, we try to load every single font into memory at once. That's what 4291 was really trying to solve.
author_association: MEMBER
user: login: blukai
created_at: 2025-05-21T19:38:25Z
body: > I tried to tackle this issue more fully in https://github.com/LadybirdBrowser/ladybird/pull/4291, where the idea is to create a FontConfigFontProvider that will lazily cache the fonts needed by the WebContent process by loading the bytes directly from fontconfig.

nice to see an the effort in that direction!
it doesn't sit quite right that ladybird is greedily loading every font it can discover, but at the moment it is what it is i suppose.

> This still has the issue that once we know about these directories, we try to load every single font into memory at once. That's what 4291 was really trying to solve.

indeed! this pr does not address that issue. if you don't mind sharing - what was the challenge for finalizing the changes that you were making / what were the problems that you faced?

---

> we probably want to only look in fontconfig's directories if we're using fontconfig? Rather than looking there and in $XDG_DATA_HOME/fonts.

right, absolutely.
is there anything else you think needs to be changed here?
author_association: CONTRIBUTOR
user: login: ADKaster
created_at: 2025-05-21T22:57:21Z
body: > indeed! this pr does not address that issue. if you don't mind sharing - what was the challenge for finalizing the changes that you were making / what were the problems that you faced?

We have three places that define what the enum values for slope/weight/etc are: LibGfx, Skia, and fontconfig. I'm not convinced that I got it remotely correct in how to translate between them. It doesn't help that all our APIs use `int` instead of actual enum types.

When I plugged in the FontConfigFontProvider, I found that the CSS StyleComputer was never hitting my cache. It would always pass through to directly call fontconfig for every font lookup.

And a few tests started failing.
author_association: MEMBER
user: login: ladybird-bot
created_at: 2025-05-22T15:27:46Z
body: Hello!

One or more of the commit messages in this PR do not match the Ladybird [code submission policy](https://github.com/LadybirdBrowser/ladybird/blob/master/CONTRIBUTING.md#code-submission-policy), please check the `lint_commits` CI job for more details on which commits were flagged and why.
Please do not close this PR and open another, instead modify your commit message(s) with [git commit --amend](https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/changing-a-commit-message) and force push those changes to update this PR.
author_association: COLLABORATOR
user: login: blukai
created_at: 2025-05-22T15:36:25Z
body: re https://github.com/LadybirdBrowser/ladybird/pull/4830#issuecomment-2899449595

I see, interesting.. thanks for the info!
author_association: CONTRIBUTOR
user: login: blukai
created_at: 2025-05-25T22:12:51Z
body: heads up, ci failure appears to be caused by timeout or something:

> The hosted runner lost communication with the server. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
author_association: CONTRIBUTOR