Repository: Colin-b/logging_json (language: python)

Issue:
hmm why does not work encoding in setting?
python version is 3.11.5

my logger setting is here


```
        logger = logging.getLogger(type(self).__name__)

        # set Log Level
        logger.setLevel(logging.INFO)
        # set Stream Handle
        ch = logging.StreamHandler()
        ch.encoding = 'utf-8'
        ch.setLevel(logging.INFO)
        stream_formatter = logging_json.JSONFormatter(fields={
            "name": "name",
            "level_name": "levelname",
            "thread": "thread",
            "process": "process",
            "response": "response"
        })
        ch.setFormatter(stream_formatter)
        logger.addHandler(ch)
        # set Formatter
        file_formatter = logging_json.JSONFormatter(fields={
            "name": "name",
            "level_name": "levelname",
            "thread": "thread",
            "thread_name": "threadName",
            "process": "process",
            "process_name": "processName"
        })

        # set File Handler

        fh = logging.FileHandler(filename=f"{os.getcwd()}/logs/dummy.log",encoding='utf-8')
        fh.setFormatter(file_formatter)
        fh.setLevel(logging.INFO)
        logger.addHandler(fh)
        return logger
```

logging.info("테스트") 

result is below

```
{"name": "BaseLogger", "level_name": "INFO", "thread": 139637689555784, "thread_name": "MainThread", "process": 314, "process_name": "SpawnProcess-32", "message": "\uc548\ub155"}
```

encoding utf-8 does not work


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index d5fd0b8..869264c 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -11,12 +11,12 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']
+        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
@@ -28,8 +28,8 @@ jobs:
         pytest --cov=logging_json --cov-fail-under=100 --cov-report=term-missing
     - name: Create packages
       run: |
-        python -m pip install wheel
-        python setup.py sdist bdist_wheel
+        python -m pip install build
+        python -m build .
     - name: Publish packages
       run: |
         python -m pip install twine
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index ad037fe..6f108d1 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -1,5 +1,5 @@
 repos:
   - repo: https://github.com/psf/black
-    rev: 22.12.0
+    rev: 23.12.1
     hooks:
       - id: black
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f6b355e..9b00549 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,17 +1,26 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.5.0] - 2024-01-17
+### Added
+- Explicit support for python `3.12`. Meaning `taskName` is now considered a reserved keyword where value is supposed to be contained in the record itself (otherwise value will be `taskName` for python < 3.12 when specified).
+- New parameters `default_time_format` (default to `%Y-%m-%d %H:%M:%S`) and `default_msec_format` (default to `%s,%03d`) allowing to change the formatting of `asctime`.
+  - More details can be found in the documentation on what the impact is when changing those values.
+
+### Fixed
+- non-ASCII but valid UTF-8 values and field names will now be output as provided (they will not be escaped anymore).
+
 ## [0.4.0] - 2023-01-09
 ### Changed
 - Default message key is now `message` instead of `msg` to stay in line with python default. If you still want previous behavior, set `message_field_name` to `msg` at formatter creation.
 
 ### Removed
-- Drop support for python 3.6.
+- Drop support for python `3.6`.
 
 ## [0.3.0] - 2022-12-02
 ### Added
@@ -34,7 +43,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 - Public release.
 
-[Unreleased]: https://github.com/Colin-b/logging_json/compare/v0.4.0...HEAD
+[Unreleased]: https://github.com/Colin-b/logging_json/compare/v0.5.0...HEAD
+[0.5.0]: https://github.com/Colin-b/logging_json/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/Colin-b/logging_json/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/Colin-b/logging_json/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/Colin-b/logging_json/compare/v0.2.0...v0.2.1
diff --git a/LICENSE b/LICENSE
index 3738ea5..cab2d30 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2023 Colin Bounouar
+Copyright (c) 2024 Colin Bounouar
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
diff --git a/README.md b/README.md
index bbc8091..8b99128 100644
--- a/README.md
+++ b/README.md
@@ -5,7 +5,7 @@
 <a href="https://github.com/Colin-b/logging_json/actions"><img alt="Build status" src="https://github.com/Colin-b/logging_json/workflows/Release/badge.svg"></a>
 <a href="https://github.com/Colin-b/logging_json/actions"><img alt="Coverage" src="https://img.shields.io/badge/coverage-100%25-brightgreen"></a>
 <a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
-<a href="https://github.com/Colin-b/logging_json/actions"><img alt="Number of tests" src="https://img.shields.io/badge/tests-21 passed-blue"></a>
+<a href="https://github.com/Colin-b/logging_json/actions"><img alt="Number of tests" src="https://img.shields.io/badge/tests-25 passed-blue"></a>
 <a href="https://pypi.org/project/logging_json/"><img alt="Number of downloads" src="https://img.shields.io/pypi/dm/logging_json"></a>
 </p>
 
@@ -70,6 +70,24 @@ import logging
 logging.info("This is my message")
 ```
 
+### Changing asctime representation
+
+You can override the default representation of asctime (`2003-07-08 16:49:45,896`) based on two different scenarii:
+
+#### Without milliseconds
+
+Set `datefmt` parameter.
+
+Setting `datefmt` to `%Y-%m-%dT%H:%M:%S` would result in `2003-07-08T16:49:45`.
+
+#### With milliseconds
+
+Set `default_time_format` to something else than `%Y-%m-%d %H:%M:%S` to change the representation part without milliseconds.
+Set `default_msec_format` to something else than `%s,%03d` to change the representation milliseconds.
+Note that `%s` in `default_msec_format` is going to be replaced by the representation without milliseconds.
+
+Setting `default_time_format` to `%Y-%m-%dT%H:%M:%S` and `default_msec_format` to `%s.%03d` would result in `2003-07-08T16:49:45.896`.
+
 ## Configuration
 
 You can create a formatter instance yourself as in the following, or you can use a logging configuration.
diff --git a/logging_json/_formatter.py b/logging_json/_formatter.py
index 3137e15..509ba77 100644
--- a/logging_json/_formatter.py
+++ b/logging_json/_formatter.py
@@ -27,6 +27,7 @@
     "process",
     "message",
     "asctime",
+    "taskName",
 )
 
 
@@ -62,14 +63,19 @@ def __init__(
         fields: Dict[str, Any] = None,
         message_field_name: str = "message",
         exception_field_name: Optional[str] = "exception",
+        default_time_format: str = "%Y-%m-%d %H:%M:%S",
+        default_msec_format: str = "%s,%03d",
         **kwargs,
     ):
         # Allow to provide any formatter setting (useful to provide a custom date format)
         super().__init__(*args, **kwargs)
         self.fields = fields or {}
-        self.usesTime = lambda: "asctime" in self.fields.values()
         self.message_field_name = message_field_name
         self.exception_field_name = exception_field_name
+        # (Allows to) overrides logging.Formatter default behavior
+        self.usesTime = lambda: "asctime" in self.fields.values()
+        self.default_time_format = default_time_format
+        self.default_msec_format = default_msec_format
 
     def format(self, record: logging.LogRecord):
         # Let python set every additional record field
@@ -96,7 +102,7 @@ def format(self, record: logging.LogRecord):
         if len(message) == 1 and self.message_field_name in message:
             return super().formatMessage(record)
 
-        return json.dumps(message, default=default_converter)
+        return json.dumps(message, ensure_ascii=False, default=default_converter)
 
     def formatMessage(self, record: logging.LogRecord) -> str:
         # Speed up this step by doing nothing
diff --git a/logging_json/py.typed b/logging_json/py.typed
new file mode 100644
index 0000000..e69de29
diff --git a/logging_json/version.py b/logging_json/version.py
index 151e68f..c4b9ff7 100644
--- a/logging_json/version.py
+++ b/logging_json/version.py
@@ -3,4 +3,4 @@
 # Major should be incremented in case there is a breaking change. (eg: 2.5.8 -> 3.0.0)
 # Minor should be incremented in case there is an enhancement. (eg: 2.5.8 -> 2.6.0)
 # Patch should be incremented in case there is a bug fix. (eg: 2.5.8 -> 2.5.9)
-__version__ = "0.4.0"
+__version__ = "0.5.0"
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..25d855a
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,55 @@
+[build-system]
+requires = ["setuptools", "setuptools_scm"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "logging_json"
+description = "JSON formatter for python logging"
+readme = "README.md"
+requires-python = ">=3.7"
+license = {file = "LICENSE"}
+authors = [
+    {name = "Colin Bounouar", email = "colin.bounouar.dev@gmail.com" }
+]
+maintainers = [
+    {name = "Colin Bounouar", email = "colin.bounouar.dev@gmail.com" }
+]
+keywords = ["logging", "json"]
+classifiers=[
+    "Development Status :: 5 - Production/Stable",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Natural Language :: English",
+    "Typing :: Typed",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.7",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Build Tools",
+]
+dependencies = []
+dynamic = ["version"]
+
+[project.urls]
+documentation = "https://colin-b.github.io/logging_json/"
+repository = "https://github.com/Colin-b/logging_json"
+changelog = "https://github.com/Colin-b/logging_json/blob/master/CHANGELOG.md"
+issues = "https://github.com/Colin-b/logging_json/issues"
+
+[project.optional-dependencies]
+testing = [
+    # Used to freeze time
+    "time-machine==2.*",
+    # Used to check coverage
+    "pytest-cov==4.*",
+]
+
+[tool.setuptools.packages.find]
+exclude = ["tests*"]
+
+[tool.setuptools.dynamic]
+version = {attr = "logging_json.version.__version__"}
diff --git a/setup.py b/setup.py
deleted file mode 100644
index 6bd2d59..0000000
--- a/setup.py
+++ /dev/null
@@ -1,51 +0,0 @@
-import os
-from setuptools import setup, find_packages
-
-this_dir = os.path.abspath(os.path.dirname(__file__))
-with open(os.path.join(this_dir, "README.md"), "r") as f:
-    long_description = f.read()
-
-setup(
-    name="logging_json",
-    version=open("logging_json/version.py").readlines()[-1].split()[-1].strip("\"'"),
-    author="Colin Bounouar",
-    author_email="colin.bounouar.dev@gmail.com",
-    maintainer="Colin Bounouar",
-    maintainer_email="colin.bounouar.dev@gmail.com",
-    url="https://colin-b.github.io/logging_json/",
-    description="JSON formatter for python logging",
-    long_description=long_description,
-    long_description_content_type="text/markdown",
-    download_url="https://pypi.org/project/logging_json/",
-    license="MIT",
-    classifiers=[
-        "Development Status :: 5 - Production/Stable",
-        "Intended Audience :: Developers",
-        "License :: OSI Approved :: MIT License",
-        "Natural Language :: English",
-        "Programming Language :: Python",
-        "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.7",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: 3.9",
-        "Programming Language :: Python :: 3.10",
-        "Programming Language :: Python :: 3.11",
-        "Topic :: Software Development :: Build Tools",
-    ],
-    keywords=["logging", "json"],
-    packages=find_packages(exclude=["tests*"]),
-    install_requires=[],
-    extras_require={
-        "testing": [
-            # Used to check coverage
-            "pytest-cov==4.*",
-        ]
-    },
-    python_requires=">=3.7",
-    project_urls={
-        "GitHub": "https://github.com/Colin-b/logging_json",
-        "Changelog": "https://github.com/Colin-b/logging_json/blob/master/CHANGELOG.md",
-        "Issues": "https://github.com/Colin-b/logging_json/issues",
-    },
-    platforms=["Windows", "Linux"],
-)


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 1ab43ce..9805052 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -8,12 +8,12 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']
+        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
@@ -22,4 +22,8 @@ jobs:
         python -m pip install .[testing]
     - name: Test with pytest
       run: |
-        pytest --cov=logging_json --cov-fail-under=100 --cov-report=term-missing
\ No newline at end of file
+        pytest --cov=logging_json --cov-fail-under=100 --cov-report=term-missing
+    - name: Create packages
+      run: |
+        python -m pip install build
+        python -m build .
\ No newline at end of file
diff --git a/tests/test_formatter.py b/tests/test_formatter.py
index 85b6824..d904066 100644
--- a/tests/test_formatter.py
+++ b/tests/test_formatter.py
@@ -1,9 +1,9 @@
 import logging
 import json
-import time
 import datetime
 import pytest
 import sys
+import time_machine
 
 import logging_json
 
@@ -62,39 +62,39 @@ def test_str_with_args_and_extra_message(caplog):
     assert dict_fmt(caplog) == {"message": "message 1", "key1": "value 1"}
 
 
-def test_dict_message_with_asctime(caplog, monkeypatch):
-    monkeypatch.setattr(time, "time", lambda: 1599736353.0076675)
+@time_machine.travel("2020-09-10 13:12:33.007667")
+def test_dict_message_with_asctime(caplog):
     caplog.set_level("INFO")
     logging.info({"key 1": "value 1", "key 2": 2})
     actual = dict_fmt(caplog, fields={"date_time": "asctime"})
-    assert time.strptime(actual.pop("date_time"), "%Y-%m-%d %H:%M:%S,007")
+    assert actual.pop("date_time") == "2020-09-10 13:12:33,007"
     assert actual == {"key 1": "value 1", "key 2": 2}
 
 
-def test_str_message_with_asctime(caplog, monkeypatch):
-    monkeypatch.setattr(time, "time", lambda: 1599736353.0076675)
+@time_machine.travel("2020-09-10 13:12:33.007667")
+def test_str_message_with_asctime(caplog):
     caplog.set_level("INFO")
     logging.info("message 1")
     actual = dict_fmt(caplog, fields={"date_time": "asctime"})
-    assert time.strptime(actual.pop("date_time"), "%Y-%m-%d %H:%M:%S,007")
+    assert actual.pop("date_time") == "2020-09-10 13:12:33,007"
     assert actual == {"message": "message 1"}
 
 
-def test_str_with_args_message_with_asctime(caplog, monkeypatch):
-    monkeypatch.setattr(time, "time", lambda: 1599736353.0076675)
+@time_machine.travel("2020-09-10 13:12:33.007667")
+def test_str_with_args_message_with_asctime(caplog):
     caplog.set_level("INFO")
     logging.info("message %s", "1")
     actual = dict_fmt(caplog, fields={"date_time": "asctime"})
-    assert time.strptime(actual.pop("date_time"), "%Y-%m-%d %H:%M:%S,007")
+    assert actual.pop("date_time") == "2020-09-10 13:12:33,007"
     assert actual == {"message": "message 1"}
 
 
-def test_str_with_args_and_extra_message_with_asctime(caplog, monkeypatch):
-    monkeypatch.setattr(time, "time", lambda: 1599736353.0076675)
+@time_machine.travel("2020-09-10 13:12:33.007667")
+def test_str_with_args_and_extra_message_with_asctime(caplog):
     caplog.set_level("INFO")
     logging.info("message %s", "1", extra={"key1": "value 1"})
     actual = dict_fmt(caplog, fields={"date_time": "asctime"})
-    assert time.strptime(actual.pop("date_time"), "%Y-%m-%d %H:%M:%S,007")
+    assert actual.pop("date_time") == "2020-09-10 13:12:33,007"
     assert actual == {"message": "message 1", "key1": "value 1"}
 
 
@@ -182,8 +182,8 @@ def test_str_with_args_message_at_exception_level(caplog):
     }
 
 
-def test_documented_record_attributes(caplog, monkeypatch):
-    monkeypatch.setattr(time, "time", lambda: 1599736353.0076675)
+@time_machine.travel("2020-09-10T13:12:33.007667+00:00", tick=False)
+def test_documented_record_attributes(caplog):
     caplog.set_level("INFO")
     logging.info({})
     actual = dict_fmt(
@@ -204,6 +204,7 @@ def test_documented_record_attributes(caplog, monkeypatch):
             "thread_name": "threadName",
             "process_id": "process",
             "process_name": "processName",
+            "task_name": "taskName",
             "record_message": "message",
             "extra": "this is a value",
         },
@@ -213,7 +214,7 @@ def test_documented_record_attributes(caplog, monkeypatch):
     actual.pop("process_id")
     actual.pop("relative_timestamp")
     actual.pop("line_number")
-    python310 = sys.version_info.minor >= 10
+    python_minor = sys.version_info.minor
     assert actual == {
         "extra": "this is a value",
         "file_name": "test_formatter.py",
@@ -225,8 +226,9 @@ def test_documented_record_attributes(caplog, monkeypatch):
         "process_name": "MainProcess",
         "record_message": "{}",
         "thread_name": "MainThread",
-        "timestamp": 1599736353.0076675,
-        "timestamp_milliseconds": 7.0 if python310 else 7.66754150390625,
+        "task_name": None if python_minor >= 12 else "taskName",
+        "timestamp": 1599743553.0076668,
+        "timestamp_milliseconds": 7.0 if python_minor >= 10 else 7.666826248168945,
     }
 
 
@@ -241,6 +243,38 @@ def test_with_extra_in_fields_and_message(caplog):
     }
 
 
+@time_machine.travel("2020-09-10 13:12:33.0076675")
+def test_asctime_with_datefmt(caplog):
+    caplog.set_level("INFO")
+    logging.info("message 1")
+    actual = dict_fmt(caplog, fields={"date_time": "asctime"}, datefmt="%Y-%m-%dT%H:%M:%S")
+    assert actual.pop("date_time") == "2020-09-10T13:12:33"
+    assert actual == {"message": "message 1"}
+
+
+@time_machine.travel("2020-09-10 13:12:33.0076675")
+def test_asctime_with_default_time_format_and_default_msec_format(caplog):
+    caplog.set_level("INFO")
+    logging.info("message 1")
+    actual = dict_fmt(caplog, fields={"date_time": "asctime"}, default_time_format="%Y-%m-%dT%H:%M:%S", default_msec_format="%s.%03d")
+    assert actual.pop("date_time") == "2020-09-10T13:12:33.007"
+    assert actual == {"message": "message 1"}
+
+
+def test_encoding_with_str(caplog):
+    caplog.set_level("INFO")
+    logging.info("테스트")
+    actual = fmt(caplog)
+    assert actual == "테스트"
+
+
+def test_encoding_with_dict(caplog):
+    caplog.set_level("INFO")
+    logging.info("테스트", extra={"테스트": "테스트"})
+    actual = fmt(caplog)
+    assert actual == '{"message": "테스트", "테스트": "테스트"}'
+
+
 def test_json_dumps_error(caplog):
     class CustomWithoutStr:
         pass