Repository: borgbackup/borg

Issue:
To UTC or not to UTC...
#7282 fixed the issue that our tests fail when they are run in some timezones (e.g. `TZ=UTC+12`) by changing the timezone in which borg prunes to UTC. Update: #7284 reverts #7282 and fixes the tests differently.

This corresponds to borg2 currently using UTC for about everything (only exception is when giving `--timestamp=...` on the commandline **without** specifying a tz offset like `+01:00`, then it will use local time).

Some IRC conversation related to this:
```
18:32  ThomasWaldmann$ guess this might give slightly different behaviour from user expectations, but it is more consistent with borg2 having utc archive timestamps.
18:33  ThomasWaldmann$ also, our unit tests failed when running them in some timezones like utc+12 and with that change, they now work in any TZ.
18:35  ThomasWaldmann$ borg2 list now outputs archive listing like:
18:35  ThomasWaldmann$ arch                                 Wed, 2023-01-18 17:34:40 +0000  [592...0d12c]
18:35  ThomasWaldmann$ so that is also utc
18:36  tao$ is it always UTC? or is that just because you set the TZ to UTC
18:36  ThomasWaldmann$ for local users maybe needs a bit getting used to it. for internationally dispersed users with common repos it might be simpler now.
18:36  tao$ hmmm. i'd not like that. i'd rather see local time in every time stamp
18:36  ThomasWaldmann$ no, the archive list is always utc.
18:37  ThomasWaldmann$ yeah, that's what i mean with "getting used to it". :-)
18:38  ThomasWaldmann$ but just imagine you have that international context. having people in different TZs making backups and later looking at them.
18:38  ThomasWaldmann$ then you'ld have all sorts of hard-to-compare timestamps.
18:38  tao$ i don't want to get used to that though. if i make a backup at 19:00 and list the archives i want to see one that says 19:00. setting it to UTC is fine, but should be some way of setting it to localtime too surely?
18:39  ThomasWaldmann$ the ts in the archive is in utc. when running borg list, borg could recompute that to local time.
18:40  tao$ yes. i only mean when i display the list using rlist. internally in the repo metadata i don't care about
18:40  ThomasWaldmann$ but then, if your friend in london made a backup at 18:00 local time there, you'ld get it displayed as being from 19:00 (in germany)
18:41  ThomasWaldmann$ which refers to the same point in time, but might make discussion about it a bit confusing.
18:42  tao$ way i would see it working would be to internally store the timestamp in UTC and then when i do borg rlist it uses the local TZ setting to display it in the local timezone, for UK, Germany, USA whatever
18:42  ThomasWaldmann$ like e.g. if he says "restore the backup of 18:00"
18:43  ThomasWaldmann$ then you might restore the wrong one.
18:43  ThomasWaldmann$ (assuming there is one per hour, for example)
18:43  tao$ i might be wrong about this, but most users i would expect only have one person using that repo in one single timezone. multiple users in multiple timezones seems less likely
18:44  ThomasWaldmann$ that might be, yes. but there is a lot of remote work and there are international companies and organisations.
18:45  tao$ could it be a switch/environment variable at least so you can choose UTC or localtime in operations? or is that too complicated
18:46  ThomasWaldmann$ that's something we could think about and someone would need to write the code.
18:46  ThomasWaldmann$ what i am changing above (url) is not the list behaviour btw (that was done quite a while ago), but how prune looks at dates/times.
18:46  tao$ it's feeling a bit like the --iec thing where iB stats are the only thing that makes sense to me but it's not the default :)
18:47  ThomasWaldmann$ yeah. ISO vs. IEC.
18:47  ThomasWaldmann$ HDD manufacturer like ISO better (because then their HDDs are "bigger")
18:49  ThomasWaldmann$ at least borg2 will be much more precise no matter what ones chooses, as it is always giving the tz offset (like +00:00 for utc tz)
18:50  ThomasWaldmann$ in some contexts (e.g. when looking at log files) that can be quite important
18:51  ThomasWaldmann$ you don't want to get into prison just because somebody confused the datetime when determining who had the IP address in question 
                       at a specific time.
18:53  tao$ yes. i absolutely agree when it's got international use. just for the use case of a one person personal backup, UTC seems wrong
18:54  tao$ it's not so bad for me because i'm in the UK so UTC == GMT, but it will still be an hour out during DST. will be really weird though for people in USA, Japan etc
18:59  ThomasWaldmann$ besides the display issue, there would be also the "pruning behaviour" issue.
```


Discussion:
When discussing this, we have to differentiate:
- what borg is internally storing (borg2 uses UTC timezone everywhere in borg archives / file items)
- what borg is displaying to the user
- how borg is behaving (e.g. `borg prune` rules keeping/deleting archives based on their timestamp)
What would `ls` and `tar` do?
```
% TZ=UTC ls -l setup.py
-rw-r--r--  1 tw  staff  8787 Jan 16 16:45 setup.py
% TZ=UTC+1 ls -l setup.py
-rw-r--r--  1 tw  staff  8787 Jan 16 15:45 setup.py
% tar cf test.tar setup.py
% TZ=UTC tar tvf test.tar         
-rw-r--r--  0 tw     staff    8787 Jan 16 16:45 setup.py
% TZ=UTC+1 tar tvf test.tar
-rw-r--r--  0 tw     staff    8787 Jan 16 15:45 setup.py
```
So it is "same point-in-time, display that in localtime" behaviour.

I think borg should by default display local time.  There could be a --utc switch to get it to display UTC instead (or the `TZ=UTC` trick could be documented).  Most programs and websites display local time.  I agree that including the offset is important to make this unambiguous.

For pruning, I also think most users will be less confused if "day" is defined using local time.  First, that's long been the borg 1.x default, and is documented.  Second, we already get lots of issues raised by misunderstanding pruning, and I think it would be worse if the last backup on (local) Tuesday was deleted because in UTC that backup was the first on Wednesday.  That all said, I'm ok with changing this if it is well-documented and clearly announced.  I don't really care which backups get pruned, as long as they get gradually thinned out.
(BTW, I was the last person to make significant changes to the pruning code in attic, including documenting that it uses localtime, way back in 2014.)
I don't think the test failures should be a factor in this decision.  One could just use the local timezone to adjust things in the tests to make them work.

Resolving patch:
diff --git a/src/borg/helpers/time.py b/src/borg/helpers/time.py
--- a/src/borg/helpers/time.py
+++ b/src/borg/helpers/time.py
@@ -114,13 +114,15 @@ def __init__(self, ts: datetime):
         self.ts = ts
 
     def __format__(self, format_spec):
-        return format_time(self.ts, format_spec=format_spec)
+        # we want to output a timestamp in the user's local timezone
+        return format_time(self.ts.astimezone(), format_spec=format_spec)
 
     def __str__(self):
         return f"{self}"
 
     def isoformat(self):
-        return self.ts.isoformat(timespec="microseconds")
+        # we want to output a timestamp in the user's local timezone
+        return self.ts.astimezone().isoformat(timespec="microseconds")
 
     to_json = isoformat
 
diff --git a/src/borg/testsuite/archiver/transfer_cmd.py b/src/borg/testsuite/archiver/transfer_cmd.py
--- a/src/borg/testsuite/archiver/transfer_cmd.py
+++ b/src/borg/testsuite/archiver/transfer_cmd.py
@@ -2,7 +2,6 @@
 import os
 import stat
 import tarfile
-from datetime import timezone
 import unittest
 
 from ...constants import *  # NOQA
@@ -48,9 +47,10 @@ def test_transfer_upgrade(self):
         repo12_tar = os.path.join(os.path.dirname(__file__), "repo12.tar.gz")
         repo12_tzoffset = "+01:00"  # timezone used to create the repo/archives/json dumps inside the tar file
 
-        def local_to_utc(local_naive, tzoffset, tzinfo):
+        def convert_tz(local_naive, tzoffset, tzinfo):
             # local_naive was meant to be in tzoffset timezone (e.g. "+01:00"),
-            # but we want it non-naive in tzinfo time zone (e.g. timezone.utc).
+            # but we want it non-naive in tzinfo time zone (e.g. timezone.utc
+            # or None if local timezone is desired).
             ts = parse_timestamp(local_naive + tzoffset)
             return ts.astimezone(tzinfo).isoformat(timespec="microseconds")
 
@@ -85,10 +85,10 @@ def local_to_utc(local_naive, tzoffset, tzinfo):
             del expected_archive["barchive"]
             # timestamps:
             # borg 1.2 transformed to local time and had microseconds = 0, no tzoffset
-            # borg 2 uses an utc timestamp, with microseconds and with tzoffset
+            # borg 2 uses local time, with microseconds and with tzoffset
             for key in "start", "time":
-                # fix expectation: local time meant +01:00, so we convert that to utc +00:00
-                expected_archive[key] = local_to_utc(expected_archive[key], repo12_tzoffset, timezone.utc)
+                # fix expectation: local time meant +01:00, so we convert that to whatever local tz is here.
+                expected_archive[key] = convert_tz(expected_archive[key], repo12_tzoffset, None)
                 # set microseconds to 0, so we can compare got with expected.
                 got_ts = parse_timestamp(got_archive[key])
                 got_archive[key] = got_ts.replace(microsecond=0).isoformat(timespec="microseconds")
@@ -125,7 +125,7 @@ def local_to_utc(local_naive, tzoffset, tzinfo):
 
                 for key in "mtime", "ctime", "atime":
                     if key in e:
-                        e[key] = local_to_utc(e[key], repo12_tzoffset, timezone.utc)
+                        e[key] = convert_tz(e[key], repo12_tzoffset, None)
 
                 # borg 1 used hardlink slaves linking back to their hardlink masters.
                 # borg 2 uses symmetric approach: just normal items. if they are hardlinks,


Test patch: