Repository: minio/minio (language: go)

Issue:
Tier info cannot be displayed correctly
Tier info cannot be displayed correctly.But the event log and destination bucket can see that the file has been transferred.
![image](https://github.com/minio/minio/assets/16344059/16eae9dc-c8d6-4465-aad2-07a42de94523)
![image](https://github.com/minio/minio/assets/16344059/0f0d7bc5-0ce3-433b-86b3-d0612a090f41)

![image](https://github.com/minio/minio/assets/16344059/0febbb48-963d-478d-9bbe-9774d519a932)
![image](https://github.com/minio/minio/assets/16344059/dc21d1a5-c502-4c47-a3ce-41173895ae0a)



## Expected Behavior
Tier info can be displayed correctly

## Current Behavior
Tier info cannot be displayed correctly

## Possible Solution


## Steps to Reproduce (for bugs)
1.Add tier settings
2.Set bucket lifecycle
3.Wait until the life cycle comes

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
minio version RELEASE.2023-09-07T02-05-02Z (commit-id=703ed46d79bba51d5d781e2297ca0216c99c8536)
Runtime: go1.21.1 linux/amd64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2023 MinIO, Inc.
* Server setup and configuration:
Docker version 24.0.2, build cb74dfc
* Operating System and version (`uname -a`):
Linux 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux



Discussion / hints:
Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.


> 层信息不是即时的，它是基于循环运行的扫描程序例程自行构建的。



> Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.

The life cycle I set is one day, but now it has been tree days
> > 层信息不是即时的，它是基于循环运行的扫描程序例程自行构建的。
> 
> > Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.
> 
> The life cycle I set is one day, but now it has been tree days

Lifecycle even if set for a day the actual transition happens in 3 days.

The scanner would update this in a bit.
> > > 层信息不是即时的，它是基于循环运行的扫描程序例程自行构建的。
> > 
> > 
> > > Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.
> > 
> > 
> > The life cycle I set is one day, but now it has been tree days
> 
> Lifecycle even if set for a day the actual transition happens in 3 days.
> 
> The scanner would update this in a bit.

Okay, I'll observe it for a while . How long can the minimum life cycle be set to?
> > > > 层信息不是即时的，它是基于循环运行的扫描程序例程自行构建的。
> > > 
> > > 
> > > > Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.
> > > 
> > > 
> > > The life cycle I set is one day, but now it has been tree days
> > 
> > 
> > Lifecycle even if set for a day the actual transition happens in 3 days.
> > The scanner would update this in a bit.
> 
> Okay, I'll observe it for a while . How long can the minimum life cycle be set to?

1 day but it doesn't run in 24hrs it runs after the object has expired for a day.
> > > > > 层信息不是即时的，它是基于循环运行的扫描程序例程自行构建的。
> > > > 
> > > > 
> > > > > Tier info is not instantaneous, it builds itself based on the scanner routine which runs in cycles.
> > > > 
> > > > 
> > > > The life cycle I set is one day, but now it has been tree days
> > > 
> > > 
> > > Lifecycle even if set for a day the actual transition happens in 3 days.
> > > The scanner would update this in a bit.
> > 
> > 
> > Okay, I'll observe it for a while . How long can the minimum life cycle be set to?
> 
> 1 day but it doesn't run in 24hrs it runs after the object has expired for a day.

Okay,thank you very much
@aaa930811 can you explain how you setup the tiering? for some reason, it looks like you tiered to yourself. 
> @aaa930811 can you explain how you setup the tiering? for some reason, it looks like you tiered to yourself.

![image](https://github.com/minio/minio/assets/16344059/df060849-d5e0-4670-9fd8-932be31e738f)
just like this
> @aaa930811 can you explain how you setup the tiering? for some reason, it looks like you tiered to yourself.

I have deployed 2 clusters on one machine, one of which uses NVME  storage as hot storage, the another uses SAS hard drive for long-term storage. Just for faster upload speed
@aaa930811 we will get back.
@harshavardhana 
Hi, I have configured Lifecycle for the bucket using `minio-go/v7`, set the data to transition to the target tier after 1 day, and currently after 7 days the data has not transitioned to the target tier.   How do I check the current minio transition logs?
![image](https://github.com/minio/minio/assets/49969527/8caffa7f-886b-4d55-8ce8-5d71db610ca1)

Current minio metrics Info:
- Objects: 193M
- Reported Usage: 76TiB

`upload` bucket metrics Info:
- Objects: 5693
- Usage: 2.1TiB

Current minio version: 
```bash
minio version RELEASE.2022-08-25T07-17-05Z (commit-id=18dffb26e7aba09074704d169517285ac3870e86)
Runtime: go1.18.5 linux/amd64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2022 MinIO, Inc.
```
Target minio tier version:
```bash
minio version RELEASE.2023-09-23T03-47-50Z (commit-id=22041bbcc469f5ac84207d09b4c4db326ec5fd57)
Runtime: go1.21.1 linux/amd64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2023 MinIO, Inc.
```

@rensongqi Please don't hijack issues with unrelated stuff. Use the latest version - we only accept issues on latest version. Open an issue if still a problem. Show how you determined that tiering isn't happening.

PS. Tiering will not remove objects locally, it will keep the metadata and move the *data* to second tier. This is mostly transparent.

Fix patch:
diff --git a/cmd/data-scanner.go b/cmd/data-scanner.go
index 2ed4b72c3bdfa..7ac982f7db2fa 100644
--- a/cmd/data-scanner.go
+++ b/cmd/data-scanner.go
@@ -343,7 +343,6 @@ func scanDataFolder(ctx context.Context, disks []StorageAPI, basePath string, ca
 		// No useful information...
 		return cache, err
 	}
-
 	s.newCache.Info.LastUpdate = UTCNow()
 	s.newCache.Info.NextCycle = cache.Info.NextCycle
 	return s.newCache, nil
diff --git a/cmd/data-usage-cache.go b/cmd/data-usage-cache.go
index ff1d1d280f32e..c97fb57f6dd8f 100644
--- a/cmd/data-usage-cache.go
+++ b/cmd/data-usage-cache.go
@@ -78,8 +78,8 @@ func newAllTierStats() *allTierStats {
 	}
 }
 
-func (ats *allTierStats) addSizes(sz sizeSummary) {
-	for tier, st := range sz.tiers {
+func (ats *allTierStats) addSizes(tiers map[string]tierStats) {
+	for tier, st := range tiers {
 		ats.Tiers[tier] = ats.Tiers[tier].add(st)
 	}
 }
@@ -95,18 +95,16 @@ func (ats *allTierStats) clone() *allTierStats {
 		return nil
 	}
 	dst := *ats
-	if dst.Tiers != nil {
-		dst.Tiers = make(map[string]tierStats, len(dst.Tiers))
-		for tier, st := range dst.Tiers {
-			dst.Tiers[tier] = st
-		}
+	dst.Tiers = make(map[string]tierStats, len(ats.Tiers))
+	for tier, st := range ats.Tiers {
+		dst.Tiers[tier] = st
 	}
 	return &dst
 }
 
-func (ats *allTierStats) adminStats(stats map[string]madmin.TierStats) map[string]madmin.TierStats {
+func (ats *allTierStats) populateStats(stats map[string]madmin.TierStats) {
 	if ats == nil {
-		return stats
+		return
 	}
 
 	// Update stats for tiers as they become available.
@@ -117,7 +115,7 @@ func (ats *allTierStats) adminStats(stats map[string]madmin.TierStats) map[strin
 			NumObjects:  st.NumObjects,
 		}
 	}
-	return stats
+	return
 }
 
 // tierStats holds per-tier stats of a remote tier.
@@ -128,10 +126,11 @@ type tierStats struct {
 }
 
 func (ts tierStats) add(u tierStats) tierStats {
-	ts.TotalSize += u.TotalSize
-	ts.NumVersions += u.NumVersions
-	ts.NumObjects += u.NumObjects
-	return ts
+	return tierStats{
+		TotalSize:   ts.TotalSize + u.TotalSize,
+		NumVersions: ts.NumVersions + u.NumVersions,
+		NumObjects:  ts.NumObjects + u.NumObjects,
+	}
 }
 
 //msgp:tuple replicationStatsV1
@@ -197,12 +196,11 @@ func (r *replicationAllStats) clone() *replicationAllStats {
 	dst := *r
 
 	// Copy individual targets.
-	if dst.Targets != nil {
-		dst.Targets = make(map[string]replicationStats, len(dst.Targets))
-		for k, v := range r.Targets {
-			dst.Targets[k] = v
-		}
+	dst.Targets = make(map[string]replicationStats, len(r.Targets))
+	for k, v := range r.Targets {
+		dst.Targets[k] = v
 	}
+
 	return &dst
 }
 
@@ -360,11 +358,11 @@ func (e *dataUsageEntry) addSizes(summary sizeSummary) {
 		tgtStat.PendingCount += st.pendingCount
 		e.ReplicationStats.Targets[arn] = tgtStat
 	}
-	if summary.tiers != nil {
+	if len(summary.tiers) != 0 {
 		if e.AllTierStats == nil {
 			e.AllTierStats = newAllTierStats()
 		}
-		e.AllTierStats.addSizes(summary)
+		e.AllTierStats.addSizes(summary.tiers)
 	}
 }
 
@@ -403,7 +401,7 @@ func (e *dataUsageEntry) merge(other dataUsageEntry) {
 		e.ObjVersions[i] += v
 	}
 
-	if other.AllTierStats != nil {
+	if other.AllTierStats != nil && len(other.AllTierStats.Tiers) != 0 {
 		if e.AllTierStats == nil {
 			e.AllTierStats = newAllTierStats()
 		}
diff --git a/cmd/data-usage-utils.go b/cmd/data-usage-utils.go
index aed9186e9352d..24c139fb880e7 100644
--- a/cmd/data-usage-utils.go
+++ b/cmd/data-usage-utils.go
@@ -107,35 +107,22 @@ func (dui DataUsageInfo) tierStats() []madmin.TierInfo {
 		return nil
 	}
 
-	cfgs := globalTierConfigMgr.ListTiers()
-	if len(cfgs) == 0 {
+	if globalTierConfigMgr.Empty() {
 		return nil
 	}
 
-	ts := make(map[string]madmin.TierStats, len(cfgs)+1)
-	infos := make([]madmin.TierInfo, 0, len(ts))
+	ts := make(map[string]madmin.TierStats)
+	dui.TierStats.populateStats(ts)
 
-	// Add STANDARD (hot-tier)
-	ts[minioHotTier] = madmin.TierStats{}
-	infos = append(infos, madmin.TierInfo{
-		Name: minioHotTier,
-		Type: "internal",
-	})
-	// Add configured remote tiers
-	for _, cfg := range cfgs {
-		ts[cfg.Name] = madmin.TierStats{}
+	infos := make([]madmin.TierInfo, 0, len(ts))
+	for tier, stats := range ts {
 		infos = append(infos, madmin.TierInfo{
-			Name: cfg.Name,
-			Type: cfg.Type.String(),
+			Name:  tier,
+			Type:  globalTierConfigMgr.TierType(tier),
+			Stats: stats,
 		})
 	}
 
-	ts = dui.TierStats.adminStats(ts)
-	for i := range infos {
-		info := infos[i]
-		infos[i].Stats = ts[info.Name]
-	}
-
 	sort.Slice(infos, func(i, j int) bool {
 		if infos[i].Type == "internal" {
 			return true
diff --git a/cmd/erasure.go b/cmd/erasure.go
index 84d7c8b102957..9918c53971acb 100644
--- a/cmd/erasure.go
+++ b/cmd/erasure.go
@@ -433,6 +433,7 @@ func (er erasureObjects) nsScanner(ctx context.Context, buckets []BucketInfo, wa
 				}
 				logger.LogOnceIf(ctx, cache.save(ctx, er, dataUsageCacheName), "nsscanner-cache-update")
 				updates <- cache.clone()
+
 				lastSave = cache.Info.LastUpdate
 			case v, ok := <-bucketResults:
 				if !ok {
@@ -440,7 +441,7 @@ func (er erasureObjects) nsScanner(ctx context.Context, buckets []BucketInfo, wa
 					cache.Info.NextCycle = wantCycle
 					cache.Info.LastUpdate = time.Now()
 					logger.LogOnceIf(ctx, cache.save(ctx, er, dataUsageCacheName), "nsscanner-channel-closed")
-					updates <- cache
+					updates <- cache.clone()
 					return
 				}
 				cache.replace(v.Name, v.Parent, v.Entry)
diff --git a/cmd/tier-last-day-stats.go b/cmd/tier-last-day-stats.go
index 0c984a67d776c..8e2dc61aea80f 100644
--- a/cmd/tier-last-day-stats.go
+++ b/cmd/tier-last-day-stats.go
@@ -101,9 +101,8 @@ func (l DailyAllTierStats) merge(m DailyAllTierStats) {
 
 func (l DailyAllTierStats) addToTierInfo(tierInfos []madmin.TierInfo) []madmin.TierInfo {
 	for i := range tierInfos {
-		var lst lastDayTierStats
-		var ok bool
-		if lst, ok = l[tierInfos[i].Name]; !ok {
+		lst, ok := l[tierInfos[i].Name]
+		if !ok {
 			continue
 		}
 		for hr, st := range lst.Bins {
diff --git a/cmd/tier.go b/cmd/tier.go
index 498285a4f8801..67f69cbfc2c7b 100644
--- a/cmd/tier.go
+++ b/cmd/tier.go
@@ -177,8 +177,24 @@ func (config *TierConfigMgr) Empty() bool {
 	return len(config.ListTiers()) == 0
 }
 
+// TierType returns the type of tier
+func (config *TierConfigMgr) TierType(name string) string {
+	config.RLock()
+	defer config.RUnlock()
+
+	cfg, ok := config.Tiers[name]
+	if !ok {
+		return "internal"
+	}
+	return cfg.Type.String()
+}
+
 // ListTiers lists remote tiers configured in this deployment.
 func (config *TierConfigMgr) ListTiers() []madmin.TierConfig {
+	if config == nil {
+		return nil
+	}
+
 	config.RLock()
 	defer config.RUnlock()
 
diff --git a/cmd/xl-storage.go b/cmd/xl-storage.go
index f08b8f2e201d6..a3b52e5ba357c 100644
--- a/cmd/xl-storage.go
+++ b/cmd/xl-storage.go
@@ -544,9 +544,11 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
 		}
 
 		sizeS := sizeSummary{}
-		var noTiers bool
-		if noTiers = globalTierConfigMgr.Empty(); !noTiers {
-			sizeS.tiers = make(map[string]tierStats)
+		for _, tier := range globalTierConfigMgr.ListTiers() {
+			if sizeS.tiers == nil {
+				sizeS.tiers = make(map[string]tierStats)
+			}
+			sizeS.tiers[tier.Name] = tierStats{}
 		}
 
 		done := globalScannerMetrics.time(scannerMetricApplyAll)
@@ -578,25 +580,27 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
 			}
 			sizeS.totalSize += sz
 
-			// Skip tier accounting if,
-			// 1. no tiers configured
-			// 2. object version is a delete-marker or a free-version
-			//    tracking deleted transitioned objects
+			// Skip tier accounting if object version is a delete-marker or a free-version
+			// tracking deleted transitioned objects
 			switch {
-			case noTiers, oi.DeleteMarker, oi.TransitionedObject.FreeVersion:
+			case oi.DeleteMarker, oi.TransitionedObject.FreeVersion:
 				continue
 			}
-			tier := minioHotTier
+			tier := oi.StorageClass
+			if tier == "" {
+				tier = minioHotTier
+			}
 			if oi.TransitionedObject.Status == lifecycle.TransitionComplete {
 				tier = oi.TransitionedObject.Tier
 			}
-			sizeS.tiers[tier] = sizeS.tiers[tier].add(oi.tierStats())
+			if sizeS.tiers != nil {
+				if st, ok := sizeS.tiers[tier]; ok {
+					sizeS.tiers[tier] = st.add(oi.tierStats())
+				}
+			}
 		}
 
 		// apply tier sweep action on free versions
-		if len(fivs.FreeVersions) > 0 {
-			res["free-versions"] = strconv.Itoa(len(fivs.FreeVersions))
-		}
 		for _, freeVersion := range fivs.FreeVersions {
 			oi := freeVersion.ToObjectInfo(item.bucket, item.objectPath(), versioned)
 			done = globalScannerMetrics.time(scannerMetricTierObjSweep)
@@ -606,14 +610,18 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
 
 		// These are rather expensive. Skip if nobody listens.
 		if globalTrace.NumSubscribers(madmin.TraceScanner) > 0 {
+			if len(fivs.FreeVersions) > 0 {
+				res["free-versions"] = strconv.Itoa(len(fivs.FreeVersions))
+			}
+
 			if sizeS.versions > 0 {
 				res["versions"] = strconv.FormatUint(sizeS.versions, 10)
 			}
 			res["size"] = strconv.FormatInt(sizeS.totalSize, 10)
 			if len(sizeS.tiers) > 0 {
 				for name, tier := range sizeS.tiers {
-					res["size-"+name] = strconv.FormatUint(tier.TotalSize, 10)
-					res["versions-"+name] = strconv.Itoa(tier.NumVersions)
+					res["tier-size-"+name] = strconv.FormatUint(tier.TotalSize, 10)
+					res["tier-versions-"+name] = strconv.Itoa(tier.NumVersions)
 				}
 			}
 			if sizeS.failedCount > 0 {


Test patch: