SlowGuess/NeurIPS_2023_OCR huggingface dataset:
normalization(BN) statistics to generate fake data. Inspired by mixup data, Cronus [49] transmits the logits information while CCVR [10] collects statistical information of logits to sample fake data. FedFTG [67] use a generator to explore the input space of the local model and transfer local knowledge to the global model. FedDF [66] utilizes knowledge distillation based on unlabeled data or a generator and then conducts AVGLOGITS. We summarize existing information-sharing methods in Table 17 to compare the dissimilarity with FedFed. The main difference between FedDF and FedFed is that our method distil raw data into two parts (performance-sensitive features  $x_{e}$  and performance-robust features  $x_{r}$ ) rather than transferring distilled knowledge. We provide hierarchical protections to preserve information privacy while overcoming the privacy-performance dilemma.
G.2 Differential Privacy with Federated Learning
Carlini et.al [72] found that memorizing sensitive data occurs in early training, regardless of data rarity and model.
Training with differential privacy [73, 74] is a feasible solution to avoid its risk, albeit at some loss in utility. Differential privacy guarantees that an adversary should not discern whether a client's data was used.
Huang et al [75] and Wei et al [76] are the first (to their knowledge) to analyze the relation between convergence and utility in FL. Andrew et al [77] explore setting an adaptive clipping norm in the federated setting rather than using a fixed one. Andrew et al [77] explore setting an adaptive clipping norm in the federated setting rather than using a fixed one. They show that adaptive clipping to gradients can perform as well as any fixed clip chosen by hand. Hoeven et al [78] introduce data-dependent bounds and apply symmetric noise in online learning, which allows data providers to pick noise distribution. Sun et al [79] explicitly vary ranges of weights at different layers in a DNN and shuffle high-dimensional parameters at an aggregation for easing explodes of privacy budgets. Peng et al [80] study the knowledge embedding problem using DP protection in FL. Applying differential privacy and its variants to the federated setting become more prevailing nowadays.