Few-Shot Domain Adaptation For End-to-End Communication
Authors: Jayaram Raghuram
Yijing Zeng
Dolores Garcia
Rafael Ruiz
Somesh Jha
Joerg Widmer
Suman Banerjee
Venue: ICLR 2023 notable top 25% (2023)
Keywords: domain adaptation
end-to-end communication
autoencoders
Gaussian mixtures
mixture density networks
few-shot
wireless channel

Abstract: The problem of end-to-end learning of a communication system using an autoencoder -- consisting of an encoder, channel, and decoder modeled using neural networks -- has recently been shown to be an effective approach. A challenge faced in the practical adoption of this learning approach is that under changing channel conditions (e.g. a wireless link), it requires frequent retraining of the autoencoder in order to maintain a low decoding error rate. Since retraining is both time consuming and requires a large number of samples, it becomes impractical when the channel distribution is changing quickly. We propose to address this problem using a fast and sample-efficient (few-shot) domain adaptation method that does not change the encoder and decoder networks. Different from conventional training-time unsupervised or semi-supervised domain adaptation, here we have a trained autoencoder from a source distribution that we want to adapt (at test time) to a target distribution using only a small labeled dataset, and no unlabeled data. We focus on a generative channel model based on the Gaussian mixture density network (MDN), and propose a regularized, parameter-efficient adaptation of the MDN using a set of affine transformations. The learned affine transformations are then used to design an optimal transformation at the decoder input to compensate for the distribution shift, and effectively present to the decoder inputs close to the source distribution. Experiments on many simulated distribution changes common to the wireless setting, and a real mmWave FPGA testbed demonstrate the effectiveness of our method at adaptation using very few target domain samples~\footnote{Code for our work: \url{https://github.com/jayaram-r/domain-adaptation-autoencoder}}.

stochas- The channel (i.e., propagation medium and transceiver imperfections) can be represented tic transfer function that transforms its input z Rdd to an output x E Rd. It can be regarded a black-box that is typically non-linear and non-differentiable due to hardware imperfections Published as a conference paper at ICLR 2023
FEW-SHOT DOMAIN ADAPTATION FOR END-TO-END COMMUNICATION
Jayaram Raghuram Yijing Zeng 1 Dolores García Martí Rafael Ruiz Ortiz 2, Somesh Jha 1.3 Joerg Widmer 2, Suman Banerjee
University of Wisconsin Madison IMDEA Networks Institute, Madrid XaiPient (jayarane ramr, yijingzeng, jha, suman (cs edu (dolores. garcia, rafael ruiz, joerg widmer y simdea. org
ABSTRACT
The problem of end-to-end learning of communication system using an autoen- coder consisting of an encoder, channel, and decoder modeled using neural networks - has recently been shown to be an effective approach. A challenge faced in the practical adoption of this learning approach is that under changing channel conditions (e.g. wireless link), requires frequent retraining of the autoencoder in order to maintain low decoding error rate. Since retraining is both time consuming and requires large number of samples, it becomes impractical when the channel distribution is changing quickly. We propose to address this problem using a fast and sample-efficient (few-shot) domain adaptation method that does not change the encoder and decoder networks. Different from conventional training-time unsupervised or semi-supervised domain adaptation, here we have a trained autoencoder from a source distribution that we want to adapt test time) to a target distribution using only small labeled dataset, and no unlabeled data. We focus on a generative channel model based on the Gaussian mixture density network (MDN), and propose a regularized, parameter-efficient adaptation of the MDN using a set of affine transformations. The learned affine transformations are then used to design an optimal transformation at the decoder input compensate for the distribution shift, and effectively present to the decoder inputs close to the source distribution. Experiments mon to the wireless setting, and a real mm Wave FPGA testbed demonstrate the effectiveness of our method adaptation using very few target domain samples
many simulated distribution changes com-
INTRODUCTION
End-to-end (e2e) learning of a communication system using autoencoder has been recently shown to be a promising approach for designing the next generation of wireless networks (O Shea & Hoydis 2017 Dörner et al. 2017). This new paradigm is a viable alternative for optimizing communication in diverse applications, hardware. and environments (Hoydis et al., 202I). is particularly promising for dense deployments of low-cost transceivers, where there is interference between the devices and hardware imperfections that are difficult model analytically. The key idea of e2e learning for communication system is to use an autoencoder architecture to model and learn the transmitter and receiver jointly using neural networks in order to minimize the e2e symbol error rate (SER).
2018 Aoudia & Hoydis 2019 O'Shea et al. 2019 Ye et al. 2018 Wang
(e.g., quantization and amplifiers). Since autoencoders are trained using stochastic gradient descent (Ssd)-baaedd optimization (O'Shea & Hoydis 2017). it is challenging to work with black-box channel that is not differentiable. One approach to address this problem is to use known mathemat-
Code for our work: httptttt////ttt httpp::/gittub..oo http://////ttthhoooo/o////aaaar/ adaptat on autoencoder

---

Published as a conference paper at ICLR 2023
ical model of the channel (e.g.. additive Gaussian noise), which would enable the computation of gradients with respect to the autoencoder parameters via backpropagation, However, such standard channel models do not capture well the realistic channel effects shown in Aoudia & Hoydis (2018). Alternatively, recent works have proposed to learn the channel using deep generative models that approximate p(x zz). the conditional probability density of the channel, using Generative Adversarial Networks (GANs) (O O'Seaa et al., 2019; Ye et al., 2018), Mixture Density Networks (MDNs) (Gar cía Martí et al. 2020), and conditional Variational Autoencoders (VAEs) (Xia et al.) 2020). The use of a differentiable generative model of the channel enables SGD-base training of the autoencoder, while also capturing realistic channel effects better than standard models.
Although this e2e optimization with generative channel model learned from data can improve the physical-layer design for communication systems, in reality, channels often change, requiring collection of a large number of samples and frequent retraining of the channel model and autoencoder. For this reason, adapting the generative channel model and the autoencoder as often as possible, using only small number of samples is required for good communication performance. Prior works have (to be best of our knowledge) not addressed the adaptation problem for autoencoder-basse e2e learning, which is crucial for real-time deployment of such a system under beequentlychanngee channel conditions. In this paper, we study the problem of domain adaptation (DA) of autoencoders using an MDN the channel model. In contrast to conventional DA, where the target domain has large unlabeled dataset and sometimes also a small labeled dataset (semi-supervised DA) (Ben- David ct al., 2006), here we consider a few-shor DA setting where the target domain has only small labeled dataset, and no unlabeled data. This setting applies to our problem since we only get to collect a small number of labeled samples at time from the changing target domain (here the channel) 2
Towards addressing this important practical problem, we make the following contributions:
We propose a parameter- and sample-efficient method for adapting a generative MDN (used for modeling the channel) based the properties of Gaussian mixtures ($ 3.1 and $ (.22p
Based on the MDN adaptation, we propose an optimal imput-transformmtion method at the decoder that compensates for changes in the channel distribution, and decreases maintains the error rate of the autoencoder without any modification to the encoder and decoder networks ($ 3.3).
Experiments on a mmWave FPGA platform and number of simulated distribution changes show strong performance improvements for our method. For instance, in the FPGA experiment, our method improves the SER by 69% with only 10 samples per class from the target distribution ($
Related Work. Recent approaches for DA such as DANN (Ganin et al. 2016). based on adversarial learning of a shared representation between the source and target domains GGanin & Lempitsky
2015; Ganin et al., 2016: Long et al., 2018: Saito et al. 2018 Zhao et al. 2019 Johanssonn et al. 2019), have achieved much success on computer vision and natural language processing. Their high-level idea is to adversarially learn a shared feature representation for which inputs from the source and target distributions are nearly indistinguishable to domain discrininator DNN, such that a label predictor DNN using this representation and trained using labeled data from only the source domain also generalizes well to the target domain. Adversarial DA methods are not suitable for our problem, which requires fast and frequent test-time DA, because of their high computational and sample complexity and the imbalance in the number of source and target domain samples.
Related frameworks such transfer learning (Long al. 2013 2016), model-agnostic meta- learning (Finn et al. (2017), addain-adaptive few-shot learning (Zhao et al. [2021] (Sun et al.) (201)). and supervised DA Mootian et al.] 2017a b) also deal with the problem of adaptation using a small number of samples. Most of them are not applicable to our problem because they primarily address novel classes (with potentially different distributions), and knowledge transfer from existing to novel tasks. MMotiaa et al. (2017a) is closely related since they also deal with a target domain that only has small labeled dataset and has the same label space. The key difference is that Mootian et al.) (2017a) address the training-time few-shot DA problem, while we focus on test-time few-shot DA. Specifically, their adversarial DA method requires both the source and target domain datasets at training time, and can be computationally expensive to retrain for every new batch of target domain data (a key motivation for this work is to avoid frequent retraining).
II our prohlem, labels correspond to the transmitted messages and are essentially obtained for free (see
2

---

Published as a conference paper at ICLR 2023
2 PRIMER ON AUTOENCODER-BAN END-TO-END COMMUNICATION
Notations. We denote vectors and matrices with boldface symbols. We define the indicator function (c) that takes value (0) when the condition is true (false). For any integer 2 We denote the one-hot-coded vector with at index i and the rest zeros by 1, The probability density of multivariate Gaussian with mean μa and covariance matrix Σ is denoted by ((x p.. E). We use the superseripts and t to denote quantities corresponding to the source and target domain respectively. Table in the Appendix provides quick reference for the notations.
define
[n]
Following (O'Shea & Hoydis 2017
Dörnrr et al. 2018), consider a single- input, single-output (SISO) communi- cation system shown in Fig. | consist- ing of transmitter (or encoder), chan- nel, and receiver (or decoder). The encoder Eg, (.) is multi- -laer feed- forward neural network (NN) with pa- rameters 0e, that maps input mes- sage y E y := (1,... ...... m} into an encoded symbol z E ıtd. The input message is mapped into a one-hot-coded vector ly prior to being processed by the encoder The message y is equivalent to a class label in machine learning terms, and the encoded symbol z Eg. (1y) is like a representative vector for the class y. We note that the dimension of the encoding d is small (less than 10), and is typically used to coincide with traditional modula- tion techniques (O' (O'ahea & Hoydis 2017 Goldsmith 2005). The set of distinct encoded symbols 2 (Eo_ (1)),-- ... (1m)) is referred to as the constellation of the autoencoder.
Pg0 x]
roelage VED
Vhsina charnd Pixl
Pooiared
Pyti xi
Gurecation 1a R)
Droder Theorda:
Deocder Reokier [x(s)
(2)
Th (6.1))
Figure I: Autoencoder- boaeee end-to-end communication system with a generative channel model.
The symbol z is transmitted (via the custom modulation learned by the encoder) over communication channel, represented by an unknown conditional probability density p(x z). and is received at the output of the channel as noisy, distorted symbol E Rdd The decoder Do, (-) is also a multi- layer, feed-forward NN with parameters 0₂ that predicts the class plast-posterior probabilities over the m messages based on the distorted channel output x. The decoder is essentially a classifier whose Po, (m xx), where Po. (y x) is the predicted probability of class y given x. The class with the highest predicted probability is the
input-output mapping is defined by Do, (x) := [Po, (1 x),.
decoded message π(x) = argmaxyey yEy Pe. (y x). As in standard classification, the performance metric of the autoencoder is the symbol error rate (SER), defined as Exx.,) (7(() * y)]-
Generative Channel Model. In order learn the encoder and decoder networks using SGD- based optimization, it is necessary to have a differentiable backward path from the decoder to the encoder through the channel. We address this by learning parametric generative model of the channel Pa. (x z) (with parameters 0.) that closely approximates the true channel conditional density p(x z). There exists a stochastic data generation or sampling function x = he. (z, u) corresponding to the generative model, where u captures the random aspects of the channel (e.g., noise and phase offsets; details in Appendix(E] [E. In this work, we model the conditional density of the channel using a set of m Gaussian mixtures, one per input message (or class) y E )::
Po. (x(2)
Σ (x) (z), E.). E {Ee, (11), ,Eo, (1m))-
(1)
Here, & is the number of components, (z) Rd is the mean vector, (z) Rdxd is the (symmetric, positive-defiiit) covariance matrix, and (z) E 10 is the prior probability of component i. It is convenient to express the component prior probability in terms of the softmax function as (z) = <<)(x) / Σ e0)(2), Vi E [k]. where a, (z) E R are the component prior logits. We define the parameter vector of component as (z(2) [a,(2)) μ4 (2) vec E, (2)) where vee(-) is the vector representation of the unique entries of the covariance matrix. We also define the combined parameter vector from all components by <bz))" ( (z)T. (z))).
An MDN can model complex conditional distributions by combining a feed-forwart network with a parametric mixture density (Bishop) 1994 2007). We use the MDN predict the parameters of the
'The encoder has normalization layer that constrains the average power of the symbols (see Appendix D
3

---

Published as a conference paper at ICLR 2023
Gaussian mixtures (zz) as function of its input symbol z, i.e. ((z) Me. (z), where θ. are the parameters of the MDN network. The MDN output with all the mixture parameters has dimension p p= k (d(d + 1)/2 + d + 1). While there are competing methods for generative modeling of the channel such as conditional GANs (Ye et al.) 2018) and VAFS (Xia et al., 2020), we choose the Gaussian MDN based on i) the strong approximation properties of Gaussian mixtures (Kostantinos)
2000) for learning probability distributions; and ii) the analytical and computational tractability it
lends to our domain adaptation formulation. The effectiveness of Gaussian MDN for wireless
channel modeling has also been shown in García Martí et al. (2020).
The input-output function of the autoencoder is given by f1yy Do, (he, (Es, (1y), u)), and the goal of autoencoder learning is to minimize the symbol error rate. Since the sampling function he of a Gaussian mixture channel not directly differentiable, we apply the Gumbel-Softmax reparametrization (Jang et al.) (207)) to obtain a differentiable sampling function (details in Ap- pendix E. More background, including the training algorithm of the autoencoder, is in Appendix [
PROPOSED METHOD
Problem Setup. Let x, y, z denote a realization of the channel output, message (class label), and channel input (symbol) distributed according to the joint distribution p(x, y, z). We first establish the following result about the joint distribution.
Proposition 1. The joint distributions p(x, y; 2) and p(x, y) can be expressed in the following form:
p(x, y. z) = =p(x Eo. (1y)) ply) 8(z Eo, (1y); Vx, E Rd, Ey
p(x,u) = p(x Eo, (1y)) p(U), VX TRd y y
(2)
where 8(.) is the Dirac delta (or Impulse) function, functiin, and we define p(x v) = p(x Eo, (1y)) as the conditional distribution of x given the class y.
The proof is simple and given in Appendix A Let Let D= ((x) ,2))), i= 1= 1,.... No } be a large dataset from source distribution p" (x. y, z
(x(x ())p)p(0 (y) 8(z Eg, (1y)). The data collection involves sending multiple copies of each of the m messages through the channel (e.g., over the air from the transmitter to receiver) by using standard modulation technique (encoding) for (e.g., M-QAM (Goldsmith (005)), and observing the corresponding channel output Different from conventional machine learning, where class labeling is expensive, this setting the class label is simply the message transmitted, which is obtained for free while collecting the data. The MDN channel model and autoencoder are trained on De according to Algorithm [ (see Appendix DD3)
Due to changes in the channel condition and environmental factors (e.g., moving obsta- cles), suppose the data distribution changes to p'(x. y. z) = (x v) p' (v) 6(z Eg. (1,)). While the distribution change may cause drop in the autoencoder's performance, we assume that it is gradual enough that domain adaptation is possible (David et al. (010)) (by domain, here we mean the state of the communication channel during the time period when the MDN and autoen- coder are trained). As discussed in S S the main challenge in this setting is to collect a sufficiently large dataset to retrain the MDN and autoencoder under the distribution shift. Therefore, suppose we collect small dataset from the target distribution DD' = {(x{, vf. zt), i = 1,... N where N < N Our goal is to design a few-shot domain adaptation for the MDN and autoencoder in order to maintain or improve the symbol error rale.
Mixture paramellers
Adaptaaion Layer
Adapted mixture parametors
MDN channel model
Ma.
(xx)
3(x)
M2 (+)
Figure& 2: Proposed MDN adaptation method.
Distribution Change. Referring to the joint distribution Eq. 2 the class prior p(y) is the prior probability of message y transmitted through the system. In this work, we make reasonable practical assumption that this prior probability does not change, i.e., p'()) s p²(y), Vy E Y. However, the class-conditional distribution of channel output p(x y) changes, and therefore the class-posterior distribution p(y x) also changes. This is commonly referred to as the conditional shift assumption (Zhang al. 2013) (different from covariate shift (Sugiyama al. 2007))
Overview of the Proposed Method. Recall from Eqn. IT] that we model the channel distribution p(x (2) as Gaussian mixture Po. (x| z)), whose parameters are predicted by the MDN, ie., ((z)=
4

---

Published as a conference paper at ICLR 2023
Mo. (z). From Proposition 1 the m class caas-conditional distributions of x are given by p(x y) p(x Eg, (1y)), Vy E Therefore, in our setting, adaptation of the coass-condiional distributions is equivalent adaptation of the Gaussian mixtures in Eqn. []] Adaptation of the Gaussian mixtures can be directly accomplished by adapting the MDN (i.e., the parameters f.) using the small target-domain dataset D'. Our proposed adaptation of the autoencoder consists of two key steps:
1. A light- parameter-efficien adaptation of the MDN using the small target dataset Dt.
2. An efficient feature transformation at the input of the decoder (based on the MDN adaptation) that compensates for changes in the dlas--conditioall distributions.
Our method requires adaptation of only the MDN (channel model), while the encoder and decoder networks (0, and 0z) remain unchanged, making it amenable to fast and frequent adaptation that requires collecting only small target dataset each time (few-shot setting).
3.1 MDN CHANNEL MODEL ADAPTATION
Our goal is to adapt the m Gaussian mixtures in Eqn 4T that model the source class-conditional distributions. Suppose the m adapted Gaussian mixtures corresponding to the (unknown) target class-conditional distributions are
0. (x|2) = Σ (()
E, (z)). z E [E, (11), ... Ee, (1m)):
(3)
where Og are parameters of the adapted (target) MDN, and the component means, covariances, and prior probabilities with a hat notation are defined as in $ The adapted MDN predicts all the parameters of the target Gaussian mixture as ((z) = M (z) as shown in Fig. where ((2) is defined in the same way as ((z). Instead of naively fine-tuning all the MDN parameters 9c. or even just the final fully-connected layer" we propose a parameter-efficient adaptation of the MDN based on the iifinettranffommatio property of the Gaussian distribution, i.e., one can transform between any two multivariate Gaussians through general affine transformation. First, state some basic assumptions required to make the proposed adaptation tractable.
A1) The source and target Gaussian mixtures per class have the same number of components k. A2) The source and target Gaussian mixtures (from each class) have one-to-one correspondence between their components.
Assumption Al is made in order to not have to change the architecture of the MDN during adaptation due to adding removing of components. Both assumptions Al and A2 make it tractable to find the closed- form expression for a simplified KL-divergence between the source and target Gaussian mixtures per class (see Proposition 2
Parameter Transformations. As shown in Appendix B.2 the transformations between the source and target Gaussian mixture parameters, for any symbol Z and component i EE [k]], are given by
(4) The affine transformation parameters A, E Rdxd and b, E Rd transform the means, C, E Rdxd transforms the covariance matrix, and 8 R transform the prior logits. The vector of all = adaptation parameters to be optimized is defined by +² = -
(z) = A, (z) + b,, 5,(z) C, CT, and (.(z) B (z) %
v. where 3. contains all the affine-transformation parameters from component i. The number of adaptation parameters is given by 1 d2 + d + 2). This is typically much smaller than the number of MDN parameters (weights and biases from all layers), even if we consider only the final fully-connected layer for fine-tuning (see
Table In Fig. 2 the udaptation layer mapping ΦZz) to ((z) basically implements the parameter transformations defined in Eqn. (4). We observe that the triinettranfformation parameters are not dependent on the symbol z (or the class), which is constraint we impose in order to keep the number of adaptation parameters small. This is also consistent with the MDN parameters O, being independent of the symbol z. Allowing the affine transformations to depend of z would provide more flexibility, but at the same time require more target domain data for successful adaptation.
Wee show in our experiments that both the fine-tuning approaches to adapt well.
'We perform ablation experiments (Appendix C.4) that evaluate our method under random Gaussian mixtures with mismatched components. We find that our method robust even when these assumptions violated.
5

---

Published as a conference paper at ICLR 2023
Proposition 2. Given m Gaussian mixtures from the source domain and m Gaussian mixtures from the target domain (one each per class), which satisfy Assumptions Al and A2, the KL-vieegeence between Po. (x. K z) and Pa (x, KK] |z) can be computed in closed-form, and is given by:
0.
Po. (x (x,)
Ty (2)
( Po.: Po.) = Brog
log
= p(z) ZE2
my
log
P
(x, K [2)
(Z)ZZ)
8.
Σ p(Z) Σ (i(z) DKT. NN( (2,)):) Ei()),, N(. (z)))), E, Ex) ZE2
(5)
where K is the mixture component random variable. The first term is the KL-divergence between the component prior probabilities, which simplifies into a function of the parameters [81, ',.-.. .kg, 7k. The second term involves the KL-divergence between two multivariate Gaussians (a standard result), which also simplifies into a function of up.
The proof and the final expression for the KL-divergence as a function of sb are given in Appendix A The symbol priors {p(z), z E }} are estimated using the class proportions from the source dataset Dr. We note that this result is different from the KL-divergence between two arbitrary Gaussian mixtures, for which there is no closed-form expression (Hershey & Olsen 2007).
3.2 REGULARIZED ADAPTATION OBJECTIVE
From the above analysis, we can formulate the MDN adaptation as the equivalent problem of finding the optimal set of affine transformations (one per-component) mapping the source to the target Gaussian mixtures. To reduce the pos- sibility of the adaptation finding bad solutions due to the small-sample setting, we introduce regularization term based on the KL-divergence (defined earlier), which constrains the distribu- tion shift produced by the affine transformations. We consider two scenarios for adaptation: 1) transformations.
p saspe-loone donoo lapors
Pg (8')
Fastue panetomaatoo
Doncss layer
Wraiiss shareed
argmas Pu(v| RR)
Pix [)
e (x)
Decoder/ /Recaiver
(x(
Figure 3: Proposed decoder adaptation using feature
Generative udaptation of the MDN in isolation and 2) Discriminative adaptation of the MDN as part of the autoencoder. In the first case, the goal of adaptation is to find good generative model for the target channel distribution, while in the second case the goal is to improve the classification accuracy of the autoencoder the target distribution. We focus on the discriminative adaptation here, and present the very similar generative adaptation in Appendix [B.
Since the goal of adaptation is to improving the decoder's accuracy in recovering the transmitted symbol z from the channel output x, we use the (negative) symbol posterior log- (PLL) as the first data-dependent term of the adaptation objective. The second term is the simplified KL- divergence between the source and target Gaussian mixtures, which does not depend on the data.
J AA)
N Σ log Pa (z), xxx) + DDD Po.))
(6)
The symbol posterior Pg. (z [x) is computed from the conditional Po. (x zz) and the symbol priors {p(z), z E }} using Bayes rule. We observe that the adaptation objective is a smooth and nonconvex function of vb Also, computation of the objective and its gradient (w.r.t vp) are inexpensive operations since i) they do not require forward and pack-propagation through the layers of the MDN and ii) both N and the dimension of 2/ are small. Therefore, we use the BFGS Quasi-Newtor method (Nocedal & Wright 2006) for minimization, instead of SGD-based large-scale optimization (e.g., Adam). The regularization constant is a hyper-parameter of the proposed method, and we propose a validation metric in Appendix B.4) to set its value automatically.
3.3 DECODER ADAPTATION USING FEATURE TRANSFORMATIONS
We propose a computationallyy-effttii efficient feature transformation g Rd + Rd at the decoder such that the transformed inputs X g (x²) are closely aligned to the source distribution on which

---

Published as a conference paper at ICLR 2023
the decoder was trained (see Fig. This is based the optimal affine- transformations 2b of the MDN found by minimizing Equ. ( This method does not require any change to the trained encoder and decoder networks, making it well suited for the few-shot DA setting.
Consider a test input x² at the decoder from the target-domain marginal distribution p'(x) ExEz p(z) Σ 1= (z) (x (z) Σ, (x)). As shown in Appendix B.2 conditioned on given symbol z E Z and component E [k]] the affine transformation that maps from the tar- get Gaussian distribution x" z, [(x (z).
Ez)z to the source Gaussian distribution
x² z, ~ [(x p. (z), [z) is given by
X Szi (xt) C, (x'
A. (z b,) (.(z).
However, this transformation requires knowledge both the transmitted symbol z and the mixture component which are not observed at the decoder (the decoder only observes the channel output x"). We address this by taking the expected affine transformation from target to source, where the expectation is with respect the joint posterior over the symbol and component given the channel output x". This posterior distribution based on the target Gaussian mixture is:
p(z) (z) N (x' (z). Ex(z))
Po. (2,
Ex E, p(z))())))))))) F (z') N(x* [z) [z()
The expected inverse-affine feature transformation at the decoder is then defined as
(x') == Brg (x.)
[g (xx)
x²] =
ΣΣ
Po. (z, x²) Szi (x').
(8)
XE21E
We show that this conditional expectation is the optimal transformation from the standpoint of mean- squared-error estimation (Kay) 1993) in Appendix A.2 The adapted decoder based on this feature transformation illustrated Fig. and defined as Do, (x'; == Do, (x- For small to moderate number of symbols and number of components k, this transformation is computationally efficient and easy to implement at the receiver of a communication system. A discussion of the computational complexity of the proposed method is given in Appendix B.5
EXPERIMENTS
We perform experiments evaluate the proposed adaptation method for the MDN and autoencoder. Our main findings are summarized as follows: 1) the proposed method adapts well to changes in the channel distribution using only a few samples per class, often leading strong improvement over the baselines; 2) our method performs well under multiple simulated distribution changes, and notably on our mm Wave FPGA experiments: 3) Extensive ablation studies show that the proposed KL-divergence based regularization and the validation metric for setting À are effective.
Setup. We implemented the MDN, autoencoder networks, and the adaptation methods in Python using TensorFlow (Abadi et al., 2015) and TensorFlow Probability. We used the following setting in our experiments. The size of the message set m is fixed to 16, corresponding to 4 bits. The dimension of the encoding (output of the encoder) d is set to 2, and the number of mixture components & is set to 5. More details on the experimental setup, neural network architecture, and the hyper-paramete are given in Appendix C.1
We compare the performance of our method with the following baselines: 1) No adaptation, which is the MDN and autoencoder from the source domain without adaptation. 2) Retrained MDN and autoencoder, which is like an "oracle method" that has access to a large dataset from the target domain. 3) Finetune- where the method optimizes all the MDN parameters for 200 epochs and optimizes the decoder for 20 epochs 6 4) Finetune last which follows the same approach "Finetune". but only optimizes the last layer of MDN (all the layers of the decoder are however optimized). We note that traditional domain adaptation methods are not suitable for this problem because it requires adaptation of both the MDN (generative model) and the decoder.
Baseline Methods.
The simulated channel variations are based on models commonly used for wireless communication, specifically: i) Additive white Gaussian noise (AWGN), ii) Ricean fading, and iii
Datasets.
We found no significant gains with larger number of epachs in this case.
7

---

Published as a conference paper at ICLR 2023
Uniform or flat fading (Goldsmith 2005). Details these channel models and calculation of the their signal-to-noise ratio (SNR) are provided in Appendix FF] We also created simulated distribution changes using random, class-conditions
Gaussian mixtures for both the source and target channels (we also include random phase shifts). The parameters of the source and target Gaussian mixtures generated in random but controlled manner detailed in Appendix C.3 We also evaluate the performance of the adaptation methods on real over-the-air wireless experiments. We use a recent nigh--pefformance mm wave testbed (Lacruz et al. (02T), featuring a high-end FPGA board with 2 GHz bandwidth per-channel and 60 GHz SIVERS antennas (SIVERSIMA) 2020). We introduce distribution changes via (In-phase and Quadrature-phan phase) IQ imbalance- distortions to the symbol constellation, and gradually increase the level of imbalance to the system More details on the FPGA experimental setup are given in Appendix C.2
Due to the space limit, we provide details of the evaluation protocol such as train, adaptation, and test sample sizes, and the number of random trials used to get averaged performance in Appendix C.1 We report the symbol error rate (SER) on large held-out test dataset (from the target domain) as function of the number of target-domain samples per class. The only hyper-parameter À of our method is set automatically using the validation metric proposed in B.4
Evaluation Protocol.
4.1
AUTOENCODER ADAPTATION ON SIMULATED DISTRIBUTION CHANGES
0.33
0.50
Symbed emor
0.25
20
0.15
30
40
10
20
20
stsiget complec paoecess
STorger outgui pen-taes
sranget cump perraass
(a) AWGN to Uniform fading.
(b) Ricean fading to Uniform fading.
(c) AWGN to Ricean fading.
Figure 4: Autoencoder adaptation on distribution shifts based on standard channel models.
10
0.18
0.12
ntt
0.14
10-1
0.10
0.12
0.10
8.6
002
0.04
0.02
10
etarget per-ciiss
(a) Uniform fading to Ricean fading.
(b) Random Gaussian mixtures.
(c) Random Gaussian mixtures with random phase shifts.
Figure 5: Autoencoder adaptation on distribution shifts based on standard channel models and random Gaussian mixtures. In figure (c), the target domain additionally includes random phase shifts.
The adaptation results under simulated distributions changes are given in Figs. and [ with the symbol error rates plotted as a function of the number of target samples per class. In Fig. 4 we consider standard channel distributions such as AWGN, Ricean fading, and Uniform fading. Fig. 5] we consider random Gaussian mixtures for both the source and the target distributions. We observe that the proposed adaptation leads to a strong improvement in SER in all cases, except in the case the failure of our method in this case in Appendix C.5 Note that the methods "No adapt" and "Retrained autoenc" have the same SER for all target sample sizes (i.e., a horizontal line). We find both the finetuning baselines to
of AWGN to Ricean fading (Fig. 4 c). We provide some insights
"QQ imbalance is common issue in RF communication that introduces distortions the final constellation.
8

---

Published as a conference paper at ICLR 2023
have very similar SER in all cases, and there is not much improvement compared to adaptation. This suggests that our approach of constraining the number adaptation parameters and using the Ki-divergence regularization are effective in the few-shot DA setting (see Table T)
4.2 AUTOENCODER ADAPTATION ON FPGA EXPERIMENTS
Symost ener R.tt
0.35
062
Symbol
Rerget
Rarget surgliss
(a) 20% IQ imbalance
(b) 25% IQ imbalance
(c) 30% IQ imbalance
Figure 6: Autoencoder adaptation on the FPGA experiments with distribution change based IQ imbalance. higher IQ imbalance results in more distribution change.
For this experiment, different levels of distribution change are introduced by varying the IQ imbalance over 20%, 25% and 30% (higher IQ imbalance corresponds to larger distribution change). From Fig. [ we observe that the proposed method achieves significant reduction in error rate compared to the (non-oracle) baselines. The relative improvement SER over the baselines is more pronounced under higher IQ imbalance. For instance, at 30% IQ imbalance, our method achieves relative SER improvement of around 69% over the fine-tuning baselines using only 10 samples per-class.
4.3 ADDITIONAL EXPERIMENTS
We have performed number of additional ex- periments including ablation studies, which are reported in Appendix C.4 through C.6 They in- clude: 1) evaluating the proposed validation met- ric for automatically setting the hyper-parameter X; 2) evaluating the importance of the KI- divergence regularization in the adaptation ob- jective; 3) performance of our method when the source and target Gaussian mixtures have mismatch in the components (addressing As- sumptions A1 and A2); 4) performance of our method when there is no distribution shift; and 5) performance of the generative adaptation of the MDN channel. To summarize the observations, found the validation metric to be effective at setting the value X, and that our method has good performance even when Assumptions Al and A2 are violated, when there is distribution shift. The generative MDN adaptation leads to increased log- log-liiclsooods with as low as samples per class.
Table Number of parameters being optimized by the MDN adaptation methods.
Adaptation method
parameters (specific)
parameters
(M,, + d
Finetune
I (2d (mp 11
12925
Finetune-last-laerr Proposed
(2d () +1)
2525
le (dd - 2d
50
5 CONCLUSIONS
In this work, we explore one of the first approaches for domain adaptation of autoencoder based e2e communication in the few-shot setting. We first propose a light-weight and parameter-efficient method for adapting Gaussian MDN with very small number of samples from the target distribution. Based on the MDN adaptation, we propose an optimal input transformation method at the decoder that attempts closely align the target domain inputs to the source domain. We demonstrate the effectiveness of the proposed methods through extensive experiments on both simulated channels and mm Wave FPGA testbed. A discussion of limitations and future directions is given in Appendix B.6

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
Banerjee, Raghuram, and Zeng were supported in part through the following grants US Na- tional Science Foundation's 's CNS-2112562. CNS-2107060, CNS-2003129, CNS-1838733, and CNS-1647152, and the US Department of Commerce's ' 7ONANB2IHO44 H043. Somesh Jha was partially supported by the DARPA-
problem under agreement number 885000. The authors from IMDEA Networks were sponsored by the Spanish Ministry Economic Affairs and Digital Trans- formation under European Union NextGeneration-FUU projects TSI-066300-2021--
RISC-6G and TSI-063000-20211-6 MAP-6G, and by the Regional Government of Madrid and the European Union through the European Regional Development Fund (ERDF) project REACT-CONTACCTCM22277
REFERENCES
Martín Abadi, Ashish Agarwal. Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro. Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, lan Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowiez, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Πya Sutskever, Kunal Talwar, Paul Tucker. Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorfiow.org.
URL https: : www. tensorflox
org/ /
Fayçal Ait Aoudia and Jakob Hoydis. End-to-end learning of communications systems without a channel model. In 52nd Asilomar Conference on Signals, Systems, and Computers (ACSSC), 298-303 IEEE, 2018. doi: 10.1 10110//SSCCC000 8645416. URL /oo1 org. 10. 11111119919SSC 2018. 8645416
Fayçal Ait Aoudia and Jakob Hoydis. Model-free training of end-to-end communication systems. JEEE Journal on Selected Areas in Communication, 37(1 1):2533322233 2516, 2019, doi: 10.110900SAC.2219929535 URL
h7 wr./10.......o.......... 1ww.11109JSCC 2019 2019.293389 2933891
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira.
Analysis of representa- tions for domain adaptation. In Advances in Neural Information Processing Systems 19, Pro- ceedings of the Twentieth Annual Conference Neural Information Processing Systems, pp. 137- 144. MIT Press, 2006. URL htcps://proceeddngs. www.ssnnurrp.... httpsc/ppppee///////eee/
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of lcarning from different domains. Machine Learning 79( 1155 2010. doi: 10.1007/
(0994-000-5112-4 URLLaasss: wttp:///...000///9/90000000
Christopher M. Bishop. Mixture density networks. Technical report, Aston University, 1994.
http://
publl cati cns. astann.tto....o ac. wttp:///////prt////3....
Christopher M. Bishop. Pattern recognition and machine learning, 5th Edition, chapter 5, pp. 272-277. Infor-
mation Science and Statistics. Springer, 2007. ISBN 9780387310732. URL http::://///// www. wor worldcat
orr/oclcc/1008143
John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman. Learning bounds for domain adaptation. In Advances in Neural Information Processing Systems 20, Proceedings of
the Twenty-First Annual Conference Neural Information Processing Systems, pp. 129 136. Cur- ran Associates, Inc., 2007. URL https: http:::://procceinng www.rieurss..... wwtpprppps...pppp//////.... /hash/
Shai Ben David, Tyler Lu, Teresa Luu, and Dávid Pál. Impossibility theorems for domain adaptation. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 129- 136. JMLR Workshop and Conference Proceedings, 2010.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using Real NVP. In 5th International Conference on Learning Representations (ICLR), Conference Track Proceedings. Open 2017. URL
net fonttt/oorune
Sebastian Dörner, Schastian Cammerer, Jakob Hoydis, and Stephan ten Brink. Deep learning based commu- nication over the air. IEEE Journal of Selected Topics Signal Processing, 1321):132-143, 2018. doi: 10.1109//STSP:011.. 1...224177.7888. URL htt.httts::/oo.orgg//// 1119////11.011111111 201.:///9SS...077777111113
Marthimus Christoffel Du Plessis and Masashi Sugiyama. Semi-supervised learning of class balance under class-prior change by distribution matching. Neural Nenworks, 50:110-119, 2014.
10

---

Published as a conference paper at ICLR 2023
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Doina Precup and Yoe Whye Tch (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, 1126 1 1135. PMLR, 2017.
Yaroslav Ganin and Victor S. Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the 32nd International Conference Machine Learning (ICML), volume 37 of JMLR Workshop and Conference Proceedings, pp. 1180-1189. JMI...rg, 2015. URI http::/procceedinngee ml press//
/37/ganin15.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor S. Lempitsky. Domain-adversarin training of neural networks. Journal of Machine Learning Research, 17:59:1-59:33 2016. URL
org/ http://prrrrrrrpppprssss/s////// wwwpapprssss//// 239. html
Dolores García Martí, Joan Palacios Beltrán, Jesús Omar Lacruz, and Joerg Widmer. A mixture density channel model for deep learning-bused wireless physical layer design. Proceedings the 23rd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp. 53-62, 2020.
Andrea Goldsmith. Wirelexs Communications, Cambridge University Press, 2005.
John R. Hershey and Peder A. Olsen. Approximating the Kullhack-Leibler divergence between Gaussian mixture models. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 317-320. IEEE, 2007. doi: 10.1 10.1009/CAASPP:201..66999 URL https:: //o:: org.l.rgg/i
#109/ICASSP 2007 366913
Jakob Hoydis, Fayçal Ait Aoudia, Alvaro Valcarce, and Harish Viswanathan. Toward 6g ai-native air interface.
IEEE Communications Magazine, 59(5):66-8, 2021.
Eric Jang, Shixiang Gu, and Ben Poolc. Categorical reparameterization with Gumbel-Softtax. In Sth Interna- tional Conference on Learning Representations (ICLR), Conference Truck Proceedings. OpenReview.net, 2017. BtRL:tt:::/// httttp::///penneveew..coo http://wwww..eeeeetteeeeeeeeeee.
Fredrik D. Johansson, David A. Sontag, and Rajesh Ranganath. Support and invertibility in domain-invariant representations. In The 22nd International Conference Artificial Intelligence and Statistics (AISTATS), volume 89 Proceedings of Machine Learning Research, pp. 527-536. PMLR, 2019, URL http::/
proceedings mlr. pwwss/ htt.e///aaaa.atapooooooooooooooa
jcwansscn19..
Steven M Kay. Fandamentals of statistical signal processing: Estimation theory. Prentice-Hall, Inc., 1993.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference Learning Representations (ICLR), Conference Track Proceedings, 2015. URL http://
arxiv ergg abs/ 1412. 6980
N Kostantinos. Gaussian mixtures and their applications signal processing. Advanced signal processing handbook: Theory and implementation, for radar, sonar, and medical imaging real time systems, pp. 3-1, 2000.
Jesus O. Lacruz, Rafael Ruiz, and Joerg Widmer. A real-time experimentation platform for sub-6 ghz and millimeter wave MIMO systems. In ACM MobiSys 2021. doi: 10.114553458884444666666
Jean Paul Linnartz. Ricean htwp:///wwwaareeeesssccooatootta http/rr////ieeee/rrieeee htm, 2001.
Mingsheng Long, Yue Cao. Jianmin Wang, and Michael I. Jordan. Learning transferable features with deep adaptation networks. Proceedings of the 32nd International Conference on Machine Learning (ICML). volume 37 of JMLR Workshop and Conference Proceedings, pp. 97-105, JMLR.org, 2015. URL http \/proceecings mir, wwwppp/wws...eeee wtt.://///////onn.......
Mingsheng Long, Han Zhu. Jianmin Wang, and Michael I. Jordan. Unsupervised domain adaptation with residual transfer networks. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, pp. 136- 144, 2016. URL hwllhtttpee//prreeiinn neurips
7ab http://ccccoccccccc.s.o. htt.://c...............o...000.2
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain
adaptation. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems, pp. 1647-1657, 2018. URL TRRIhttps. https: /wwpo/eeiiicicccccoccccccc ngs. neur r ips. cc/ 108dd Abst racsttratt..o hhtm
paper/201 B/wash/aaaaaeaee abaahh/aab8hbb 0tt.aaae//bb81113333333313303363

---

Published as a conference paper at ICLR 2023
Saeid Motiaan, Quinn Jones, Seyed Mehui Iranmanesh, and Gianfranco Doretto. Few-shot adversarial domain adaptation. In Advances in Neural Information Processing Systems 30: Anmual Conference on Neural Information Processing Systems, pp. 6670-6680, 2017a. URL
htwp://wdnnnnnnnnn.nieeeee cc/
Saeid Motiian, Marco Piccirilli,, Donald A. Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. /EEE International Conference on Computer Vision ICCV, pp. 37266 IEEE Computer Society, 2017b. doi: 10.1 10.119//CC222017..6000 URL https:///6ll.or//1000 11099
ICCV 2017 609
Jorge Nocedal and Stephen Wright. Numerical Optimization, chapter 5-7. Springer Science & Business Media, 2006.
Timothy O'Shea and Jakob Hoydis. An introduction to deep learning for the physical layer. IEEE Transactions on Cognitive Communications and Networking, 3(4):563-57) 575. 2017. doi: 10.1109/TCCN2201.2758333
URL BtLLhttss://cci ctt/10..rrr1/0. wtlpe/g/...11911111 1119/CCCCCC 2017.2758377 2758370
Timothy J. O' Shea, Tamoghna Roy, and Nathan West. Approximating the void: Learning stochastic channel
models from observation with variational generative adversarial networks. In International Conference on Computing. Networking and Communications (ICNC), 681 686. IEEE, 2019. doi: 10.1109/1C,,..20199 8685573. URL https:/ /oi. org/_0. -109 ICCNC. 2019. 8685573
Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of a classifier to new priori probabilities: A simple procedure. Neural Computation, 14(1):21-41, 2002. doi: 10.1162/
089976602753284144 URL/httss:/ httttp:///oi.org/ggg/ 100 1162 00//66//8999 6602666626000 76006632222222 753284446
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Put-
tem Recognition (CVPR), pp. 3723-3732. IEEE Computer Society, 2018. doi: 10.
2018.00392, URL htte::://ppcccccss.. theevf com/ coaa//oontttttccc cypr http__ccc__211//mm///aaaae
Maximum Classifier wtw.i//rrwiiccepcccccyeeeeee 201....2.1.....ooo.ttt
papor.
SIVERSIMA. EVK06002 Development Kit, 2020.
evk 06002-007
Masashi Sugiyama and Motoaki Kawanahe. Machine Learning in Non-stationary Environments: Introduction Covariate Shift Adaptation. MIT Press, 2012.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Rohert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8:985-1005, 2007. /tt/http: h/c1
a.cm.orge ii on. cfm?. d- 1393322
Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning.
In IEEE Conference on Computer Vision and Pattern Recognition CVPR, pp. 403 412. Computer Vision Foundation/ / IEEE, 2019. doi: 10.1 109/CVP..201990000
Tianqi Wang, Chao-Kai Wen, Hanqing Wang, Feifci Gao, Tao Jiang, and Shi Jin. Deep learning for wireless physical layer: Opportunities and challenges. CoRR, abs/1710.05311, 2017. URLphttp://arxiv org/
abs 2.8/17100 026//1100.000 05312
William Xia, Sundeep Rangan, Marco Mezzavilla, Angel Lozano, Giovanni Geraci, Vasilii Semkin, and Giuseppe Loianno. Millimeter wave channel modeling via generative neural networks. In IEEE Globecom Workshops (GC Wkshps), pp. 1-6. IEEE, 2020. doi: 10. 10111/////kWsss500000
2kkkssss50332.222093627777 2020.936440.0 URL
https: crg/ 10. 11t11119////M www.1////.//...0.0.. 30.. 9367420
Hao Ye, Geoffrey Ye Li. Büing-Hwang Fred Juang, and Kathiravetpillai Sivanesan. Channel agnostic end-to-end learning hased communication systems with conditional GAN. In IEEE Globecom Workshops (GC Wkshps),
pp. 1-5. IEEE, 2018 doi: 10. 109/LLCOMO 86442200. http://tpss:/o...rrrgggg
G GLOCCMM... 2018 8644250
Kun Zhang, Bernhard Schölkopf. Krikamol Muandet, and Zhikun Wang. Domain adaptation under target
and conditional shift. In Proceedings of the 30th International Conference on Machine Learning (ICMI., volume 28 of JMLR Workshop and Conference Proceedings, pp. 819-827. JMLR.org, 2013. URI. http
\/prozeed ngs. m1 ppress/v28/ zhang 1.d.htm1
An Zhao, Mingyu Ding, Zhiwu Lu, Tao Xiang, Yulei Niu, Jiechao Guan, and Ji-Rong Wen. Domain adaptive few-shot learning. In IEEE Winter Conference on Applications of Computer Vision, WACV, pp. 1389 1398. IEFF, 2021. doi: 10.1 109/VAACV86600220211 .00143. URI https: //6oi i .org 100 1099 WAC34463C3 2.02 00143
12

---

Published as a conference paper at ICLR 2023
Han Zhao, Remi Tachet des Combes, Kun Zhang, and Geoffrey J. Gordon. On learning invariant representations for domain adaptation. Proceedings of the 36th International Conference on Machine Learning (ICML), volume 97 Proceedings of Machine Learning Research, pp. 3532. PMLR, 2019. URL tp
//proseodings. mlr, prwps /0nsss/v99/2aae 9a.html
13

---

Published as a conference paper at ICLR 2023
Appendix
Table 2: Commonly used notations
Notation
Description
Ioput message class lahel. Usaally 2', where is the number hits, One-hot-coded representation of label (messsgge y, with at position y and acros elsewbere.
1, y
x 2
xx with [2]
Encoded representation symbol vector corrspendingg
input message.
x
Channel output that is festure vector be classifled by the decoder.
(1.
Encoder NN with parumeters mapping hot-coded message symbol vector Rt! with parameters mupping Class (message) prediction of the decoder. Conditional density (gencrative)
(x, (1 (x). (e xx]
outpet probabilities over message set.
8(x)
aeguanyry Po
channel parameters 6... Mixture demsity network wrkk that predicts the parameters Gaussian mixture. Transfer sampling function correspondngg the channel conditional density.
(ix) (x)
he. (x.11
G(1,)) = (hu (1)).
Input-output mapping of the autocacder with combined parameter voctor 8 .".0 (())). Affine transformation (adaptation) parameters per component used adapt the MDN.
10" IWT 1 N
Sa. and 5 2
Afline transfornations between the components the source-fo to target Gaussian mixtures and vice-verse. Kuilbback-I eihler divergence between the distributions pnd Multivariatc Gaussian density with Dime delta or impalse function centered at Xo- Categorical distribution with pi and Σ, P1 Indicator function mupping predicate l norm of a vector x.
(.)))
za covariance Σ.
8(x Xo)
false.
The appendices are organized as follows:
Appendix A
discusses the theoretical results from the main paper.
Appendix provides additional details on the proposed method including:
Discussion class labels and labeled data in the communication setting (Appendix [.
Feature and parameter transformation between multivariate Gaussians (Appendix B.2
Generative adaptation of the MDN channel (Appendix [B.3).
The validation metric used for setting the aaper-parameter À (Appendix B.4
Computational complexity analysis of the proposed method (Appendix B.5
Limitations and future work (Appendix(B.c) (B6)..
Appendix provides additional details the experiments and additional results, including ablation studies of the proposed method.
Appendix D provides additional background the following topics: 1) components of end- to-end autoencoder-basee communication system, 2) generative modeling using mixture density networks, 3) training algorithm of the autoencoder, and 4) primer domain adaptation.
Appendix E provides details on the MDN training and differentiable sampling using the Gumbel- softmax reparametrization.
Appendix provides details on the simulated channel distributions used in our experiments.
A THEORETICAL RESULTS
Propostion (restatement). The joint distributions p(x,y, y. 2) and p(x, y) be expressed in the following form:
yx y/, 2) p(x Egg (1₂)) ply) 8(z Eg, (1,));
RR Dey
p(x. v) = -((x Ee. (1,)) p(y), Vx E Rdd y E Y,
(9)
where 5(.) is the Dirac delta (or Impulse) function, and we define p(x (v) p(x Ee, (1n)) the conditional distribution of x given the class y-
Proof. It follows from the dependence y
defined by generative model that p(y) p(z y) p(x z. p(y) 8(z Eg. (1,) p(x Eg. (1,).
p(x, y.
p(y) 8(z E.. (1,)) p(x Eg, (1,)).
In the second step, the conditional p(z y) reduces to the Dirac delta since the symbol can only
take one of the values from the constellation Z (Ee. (11),... Eo, (1m)) (for fixed encoder mapping). The distribution p(x, y) in Eq. g is obtained from the third step by integrating p(x, y, over all z, and using the integration property of the Dirac delta.
14

---

Published as a conference paper at ICLR 2023
A.1 KL-DIVERGENCE BETWEEN THE SOURCE AND TARGET GAUSSIAN MIXTURES
Propostion 2 (restatement). Given m Gaussian mixtures from the source domain and m Gaussian mixtures from the target domain (one each per class), which satisfy Assumptions A1 and A2, the can be computed in closed-form, and is
Kl--divergence between Po. (x, K 2) and Pa (x, K|2 2)
given by:
Po. (x, K z)
Σ (z) log (z) Fy (z)
Dvllo. Paa)) EPoc
log
E xEZ
Po (x, K 8)
Σ ZEZ
p(z)
Σ (z) DKL (N( [z()) E,(z)), N(- ([.()) Ez))
(10)
1=1
where K' is the mixture component random variable. The first term is the Kl.-divergence between the component prior probabilities, which simplifies into a function of the parameters [81,71 "1."..... .e, Yk. The second term involves the KL--divergence between two multivariate Gaussians (a standard result), which also simplifies into a function of v/.
Proof. Referring to 3.1 we derive the closed-form KL-divergence between the source and target Gaussian mixtures under Assumptions and 2, i.e., the source and target Gaussian mixtures have the same number of components that have a one-to-one association. Recall that 6g and f. are the parameters of the original (source) and the adapted (target) MDN respectively. Let K E {1, k} denote the latent component random variable.
Po. (x, K 2)) log P (x, K z)
Dep (Po. Ps.) = Epoc
0.,
0.
Po. (x, K =i(2) dx
Σ p(Z)
Po. (x, k= =i(2) log
P 0.
1=1 Ru
k
Po. (K = i P) (x z, K =
= Σ p(z)
Po. (K = =ilz)
(x|2,, ii) log
z, K = i) dx
P (K = i z) P 0.
(x
XEZ
i=1
Rd
0,
log (2(2) (z)
N( 24
[1(2) (2): Ex)
Σ p(z)
(z(
N (2),
+ log
dx
N x (z), Ex(z)
XEZ
1=1
Rd
log (z) (z)
Σ
p(z)
(2(2))
xEZ
1=1
+
Σ ZEZ
p(z)
m (z) DKL NN (-) (()) N (. (())
(11)
The second term in the final expression involves the KL-divergence between two multivariate Gaussians (a standard result) given by
1 det Σ
DKL (N.
log
(rr(S-1)) Σ)
2 del E)
d
(ā u) E -
(ā 4)
2
For clarity, we further simplify Eq. 11] for the case of diagonal covariances by applying the above result. Recall that the Gaussian mixture parameters of the source and target domains are related by the parameter transformations in Eq. (4). The second term in Eq. [II involving the KL-divergence between multivariate Gaussians, simplifies to
DKL (N (- (z)), of())) N Tz(z),,²(z))
1
d
1
d
=
2
log 2 ci
+
+
2 (z)
(agg (2) + big - -m,(2))
2
(12)
15

---

Published as a conference paper at ICLR 2023
The first term in Eq. involving the KL-divergence between the component prior probabilties can
be expressed as a function of the adaptation parameters [31, 71,
Bk. 7k] as follows:
Ti(z) (z)
(z)(
(a((2)
e (2))
ni 2 log
log
log
q(z)
g(x)
((z)
k
k
= log
(2)())))) - log( e
(a(())
+
(ar(z) Be (.(((z) - 7b), (13)
1=1
where q(2) دド= (2) and ((x)
e (z(22) + are the normalization terms in the
softmax function. Substituting Eqs. (12 and (33 into the last step of Eq. (1] gives the KL-divergence
between the source and target Gaussian mixtures as a function of the adaptation parameters w/.
A.2 OPTIMALITY OF THE FEATURE TRANSFORMATION
We show that the proposed feature transformation at the decoder in 33.3 is optimal in the mimimum mean-squared error sense. The problem setting is that, at the decoder, we observe an input x" from the target domain marginal distribution, i.e.,
x p'(x) = ΣP(2) Σ ZE2
P(2))
(x) 1=1
(2), (z),,
where 2 = [E. (11), Eg, (1m)} is the encoder's constellation. Suppose we knew the symbol z Eg. (1y) that transmitted and the mixture component [k], then the transformation 8zi (x") in Eq. ( can map x' to the corresponding Gaussian component of the source distribution. However, since z and are not observed at the decoder, we propose to find the transformation g 11 Rdd + RRd (independent of z and i) that minimizes the following expected squared error:
(x")
PP (z.1)x)
[lg (x*) g (xx)111 x
(14)
This is the conditional expectation over (z, i) given x² with respect to the posterior distribution Pa (z, x). Since x' is fixed, the above objective is function of the vector w : g (xx) E R
and it can be simplified follows:
1
J(w)
Ep, (z.i)
[1gg (x')
w12 x*]
2
1
+ 1 ww
E Pp, (x,1)x)
8zii (x')
Bzi
(x*)
xx]
2
w" Erg (x,+)x)
(x*)
xx]]
Note that comes outside the expectation since does not depend on z or i, The minimum of this simple quadratic function can be found by setting the gradient of J with respect to w to 0, giving
w g (xx')
Epg (z.ixx)
[B (x') x²]
Σ
Σ
Po. (z, Bzi (x').
ZEZ kE[A]
This is the feature transformation at the decoder proposed in
3.3
B ADDITIONAL DETAILS ON THE PROPOSED METHOD
In this section we provide additional details on the proposed method that could not be discussed in of the main paper.
B. CLASS LABELS AND LABELED DATA
We would like to clarify that the statement "class labels are available for free" is made in Section in order to highlight the fact that class labels are easy obtain in this end-to-end communication
16

---

Published as a conference paper at ICLR 2023
setting, unlike other domains (e.g. computer vision) where labeling data could be expensive. Since the transmitted message is also the class label, is always available without additional effort during the data collection (from the packet preambles). However, note that it is still challenging/ expensive to collect a large number of samples for domain adaptation, as discussed in Section 1 In contrast, may be easy obtain plenty of unlabeled data in other domains such computer vision, where labeling is expensive.
In communication protocols, preambles are attached to the front of the packets for synchronization, carrier frequency offset correction, and other tasks. The preambles consist of sequences of known symbols (which have a one-to-one mapping the messages). Therefore, these sequences can be used as the labeled dataset since the receiver obtains the distorted symbol and knows the ground truth. The proposed MDN adaptation and input transformation the decoder do not incur any modifications to the encoder (transmitter side). The constellation learned by the autoencoder is kept fixed during adaptation. Therefore, using the preambles from small number of packets, our method performs adaptation at the receiver side and maintains the symbol error rate performance without communicating any information back to the encoder.
B.2 TRANSFORMATION BETWEEN MULTIVARIATE GAUSSIANS
We discuss the feature and parameter transformations between any two multivariate Gaussians. This result was applied to formulate the MDN adaptation in Eqs. ① and (7) Consider first the standard
transformation from X ~ N- p. Σ) to X ~ N(1 β2, Σ) given by the two-step process:
Apply a whitening transformation z D 1/2 U (x 11) such that z ~ N11)0,1)
Transform into the new Gaussian density using x ÚD μ.
We have denoted the eigen-decompositionn of the covariance matrices by Σ UDU and Σ ÜDÜT, where U and U are the orthonormal eigenvector matrices, and D and D are the diagonal eigenvalue matrices. Combining the two steps, the overall transformation from x to X is given by
x = 0D D 1/2 U (x - 11) + f..
(15)
Suppose we define the matrix C D U then it is easily verified that the covariance matrices are related by Σ = CTC In general, the mean vector and covariance matrix of any two Gaussians can be related by the following parameter transformations:
A b and Σ C.CT.
(16)
with parameters A E dxd b E Hd and C E dddd Substituting the above parameter transforma- tions into the feature transformation in Eq. (15), we get
X C (x p)) Ap + b.
From the above, we can also define the inverse feature transformation from x~ ~ N(- E) to X (-1 E):
x = C- (x - Ap - b) + μ.
B.3 GENERATIVE ADAPTATION OF THE MDN
In $ 3.22 we discussed the discriminative adaptation objective for the MDN, which is used when the MDN is adapted as part of the autoencoder in order to improve the end-to-end error rate. This adaptation approach was used for the experiments in $4 On the other hand, may be interested in adapting the MDN in isolation with the goal of improving its performance as a generative model of the channel. For this scenario, the adaptation objective Eq. 6 is modified as follows. The first (data-dependent) term is replaced with the negative conditional logelikelioood (CLL.) of the target dataset, while the second KL-divergence term remains the same:
-1 N'
(eb
Σ log P 0. (xx, 2/11 AD Pp.):
(17)
N
where (z), (z) and (z) as a function of 2b are given by Eq. 4). The parameters of the original Gaussian mixture (z),,, (z),), (z),, Vi are constants since they have no dependence on
17

---

Published as a conference paper at ICLR 2023
2.. The regularization constant À 0 controls the allowed KL-divergence between the source and target Gaussian mixtures. Small values of À weight the CLL term more, allowing more exploration in the adaptation, while large values of À impose strong regularization to constrain the space of target distributions. We evaluate the performance of this generative MDN adaptation in Appendix CC.6
B.4 VALIDATION METRIC FOR AUTOMATICALLY SETTING X
The choice of in the adaptation objectives Eqs. (6) and 17 is crucial it sets the right level of regularization suitable for the target domain distribution. Since the target domain dataset is very small, it is difficult to apply cross-validation type of methods select We propose a validation metric ((bb DD) that utilizes the feature-trane tranfrorer target domain dataset to evaluate the quality of the adapted solutions for different À values.
Let &b denote the adaptation parameters found by minimizing the objective Eq. ( for specific 2 The feature transformation (from target to source domain) at the decoder g -=( based on the adaptation parameters 2/ is given by Eq. (8). Recall that the target domain dataset is D' = {xx, Vn ).) n 1, 1. .. We define the feature-transformed target domain dataset as:
,N..
Dr Dra {g (xx,),
(xk,),ff vhn 2/,,, n..,...,, ,N'.
Suppose vp is good adaptation solution, then expect the decoder (trained the source do- main dataset) to have good classification performance on Dirans For a given tranure-transformed target domain sample, the decoder predicts the class posterior probabilities: Do, (xx)))))
[Po, (1 [ (=lgg(((x)) (x())). Lee (m (xx))] We define the validation metric the negative poste-
rior log- ligllikeldood of the decoder on D given by
DT) = N Σ
log Po (y Ig (x())
(18)
We expect smaller values of V( (D) to correspond to better adaptation solutions. The adaptation objective is minimized with À varied over range of values, and in each case the adapted solution & is evaluated using the validation metric. The pair of À and u resulting in the smallest validation metric is chosen the final adapted solution. The search set of À used in our experiments was (10-5, 10-4 10-3, 10-2, 0.1, 1, 10, 100). See Appendix C.4 for an ablation study on the choice of hyper-parameter using this validation metric.
Generative MDN Adaptation. The validation metric proposed above depends on the decoder, and cannot be used when the MDN is adapted generative model in isolation (Appendix [B.3). For this setting, we modify the validation metric based on the following idea. Suppose the adaptation finds a good solution, then we expect Pirass to have high conditional logeeiiiidood under the (original) source domain MDN. The validation metric is therefore given by
N
V(b:
Σ
log Po. (g (x()) 12,)
(19)
N
where Po. is the Gaussian mixture given by Eq.
B.5 COMPLEXITY ANALYSIS
We provide an analysis of the computational complexity of the proposed adaptation methods.
MDN Adaptation.
The number of free parameters being optimized in the adaptation objective (Eqs. 6 or 17 given by |g| = A: d2 + d + 2). This is much smaller than the number of parameters in typical MDN, even considering only the final fully-connected layer (see Table for a comparison). Each step the BFGS optimization involves computing the objective function, its gradient, and estimate of its inverse Hessian. The cost of one step of BFGS can thus be expressed as O(N' df |&b 2). Suppose BEGS runs for maximum of T iterations and the optimization is repeated for L values of X, then the overall cost of adaptation is given by O(L T NO Is d? (2)... Note that the optimization for different values can be easily solved in parallel.
18

---

Published as a conference paper at ICLR 2023
Test-time Adaptation at the Decoder.
We analyze the computational cost of the feature transformation based adaptation at the decoder proposed in $ 3.3 Consider single test input x' the decoder. The feature transformation method first computes the posterior distribution Pg_ (z, i x') over the set of symbols-componet pairs of size k m. Computation of each exponent factor in the posterior distribution requires (dd)) operations for the full-covariance case, and ((d) operations for the diagonal covariance case. This corresponds to calculation of the log of the Gaussian density. Therefore, computation of the posterior distribution for single (z, pair requires O(k m dd) operations for the full-covariance case (similarly for the diagonal case). Computation of the affine transformation gzi (x*) for single (z, i) pair requires (df) operations (the matrix C; only needs to be inverted once prior test-time adaptation). Since calculation of the posterior term dominates the computation, the overall cost of computing the transformation in Eq (8) over the k m symbol-component pairs will be Ok m k d²) Okk m² d²).
We note that in practical communication systems d small (typically d = 2). The number of symbols or messages m can vary from 1024 powers of 2. The number of mixture components can be any positive integer, but is usually not more than few tens to keep the size of the MDN practical. Therefore, the computational cost of test-time adaptation at the decoder based on the feature transformation method is relatively small, making our proposed adaptation very computationally efficient to implement at the receiver side of a communication system.
B.6 LIMITATIONS AND FUTURE WORK
The proposed work focuses mainly mixture density network (MDN) the generative channel model, which allows us to exploit some of their useful properties in our formulation. Generalizing the proposed few-shot domain adaptation to other types of generative channel models such as conditional GANs, VAEs, and normalizing flows (Dinh et al.) 2017) could be an interesting direction. These generative models can handle more nigh-dimensional structured inputs.
The proposed work does not adapt the encoder network, i.e., the autoencoder constellation is not adapted changes in the channel distribution. Adapting the encoder, decoder, and channel networks jointly would allow for more flexibility, but would likely be slower and require more data from the target distribution.
We focused on memoryless channels, where inter-symbol serrr-yymoo--innefeeeeee -intefference (ISI) is not problem. In practice, communication channels can have memory and ISI would have to addressed by the training and adaptation methods. Under changing channels, one would have to also adapt an Equalizer model (algorithm) in order to mitigate ISI.
C ADDITIONAL EXPERIMENTS
We provide additional details on the experiments $ and report additional results, including ablation studies on the proposed method.
C.I EXPERIMENTAL SETUP
We implemented the mixture density network and communication autoencoder models using Tensor- Flow (Abadi al., 2015) and TensorFlow Probability. We used the BFGS optimizer implementation available in TensorFlow Probability. The code base for our work has been submitted as supple- mentary material. All the experiments were run on Macbook Pro with GB memory and CPU cores. Table 3 summarizes the architecture of the encoder, MDN (channel model), and decoder neural networks. Note that the output layer of the MDN is concatenation (denoted by @) of three fully-connected layers predicting the means, variances, and mixing prior logit parameters of the Gaussian mixture. The following setting is used in all our experiments. The size of the message set m (also the number of classes) was fixed to 16, corresponding to 4 bits. The dimension of the encoding was set to 2, and the number of mixture components was set to 5. The size of the hidden layers n& to 100.
19

---

Published as a conference paper at ICLR 2023
Table 3: Architecture of the Encoder, MDN channel, and Decoder neural networks. FC fully connected (dense) layer; @ denotes layer concatenation; ELU exponential linear unit; m number of messages; d- encoding dimension; k number of mixture components; n4 size of hidden layer.
Network
Layer
Activation
FC, m X n4p
ReLU
Encoder
FC, nh x d
Linear
Normalization (avg. power)
None
FC, d x Y&a
ReLU
MDN
FC, nh 12
ReLU
FC, TE x kd (means)
Linear
+ FC, n. kd (variances)
ELU + +
@ FC, X & (prior logits)
Linear
FC, d n
ReLU
Decoder
FC, n2 x m
Softmax
The parameters ub of the proposed adaptation method are initialized as follows for each component i:
A, La. b; = 0, C; = Ia, B = 1, % 0,
where La is the d x d identity matrix. This initialization ensures that the target Gaussian mixtures (per class) are always initially equal to the source Gaussian mixtures. The regularization constant varied over 8 equally-spacee values on the log- log-scale with range 10-5
in the adaptation objective to 100, specifically (10-5, 10-4, 10-3 10-2,0.1110 0.1, 1, 10, 100}. The value and zb corresponding to the smallest validation metric are selected the final solution.
We used the Adam optimizer (Kingma & Ba 2015) with a fixed learning rate of 0.001, batch size of 128, and 100 epochs for training the MDN. For adaptation of the MDN using the baseline methods Finetune and Finetune last, we used Adam with the same learning rate for 200 epochs. The batch size is set as b = max(10 10, 0.1 N*), where N' is number of adaptation samples the target dataset. For training the autoencoder using Algorithm [ we found that stochastic gradient descent (SGD) with Nesterov momentum (constant 0.9), and an exponential learning rate schedule between 0.1 and 0.005 works better than Adam.
Finetuning Baselines. We provide additional details on the baselines Finetune and Finetune last. Both the methods first initialize the target domain MDN, encoder, and decoder networks with the corresponding parameters from the source domain. The method Finetune first finetunes all the MDN parameters to minimize the conditional log- log-likelihood of the target dataset using the Adam optimizer. After the MDN is finetuned, we freeze the parameters of the MDN and encoder, and train only the decoder using data generated from the updated MDN channel. The method Finetune last differs from Finetune in that it optimizes only the weights of the final MDN layer.
From the results in Figures 4 and 6 observe that the baselines Finetune and Finetune last have very similar performance compared to the case of no adaptation. We have investigated this carefully and verified that this is not due bug insufficient optimization (e.g.. by checking if the final weights of the MDN and decoder are different for both methods). For both methods, we tried range of learning rates for Adam and increased the number of epochs a large number (beyond 200 was not helpful). We have reported the best-case results for these methods, which suggests that they are not effective adaptation using small target domain datasets. As mentioned in Section 4.1 we hypothesize that using the KL-divergence based regularization and constraining the number of adaptation parameters leads to more effective performance of our method.
Uncertainty Estimation. Since there is inherent randomness in our experiments, especially with the small sample sizes of the target dataset, we always report average results from multiple trials. For the experiments on standard simulated channel variations (e.g., AWGN to Ricean fading), we report the results from 10 trials. For the random Gaussian mixtures experiment, we report the average and standard error over 50 random source/target dataset pairs. For the FPGA experiments, report the results from 20 random trials. The average metrics (symbol error rate and ligeliielhooo)) are reported in the plots.
20

---

Published as a conference paper at ICLR 2023
Training Data
Adaptation Data
Test Data
1.0
0.5
0.0
-0.5
-1.0
Figure 7: Example data plot for the case where the source and target domain class-conditional distributions are random Gaussian mixtures. The training data is from the source distribution, while the adaptation data (with 10 samples per class) and test data from the target distribution.
Evaluation Protocol. We create random class-stratifes 50-50 train-test split (each of size 300,000) for data from both the source and target domains. Performance on both domains is always evaluated on the held-out test split. The train split from the target domain dataset is sub-sampled adaptation datasets of different sizes, specifically with 5, 10, 20, 30, 40, and 50 samples per class (symbol). For the generative adaptation experiments adaptation samples from the target domain is reduced even further. We varied from 2 samples per- class 20 samples per-class in order to highlight the improvements obtained by the proposed method. The oracle baseline method, which retrains the autoencoder and MDN on the target distribution, uses the entire training dataset from the target domain.
create
the MDN (Appendix (C the number of
Choice of SNR. For the experiments on simulated channel distributions such as AWGN, Ricean fading. and Uniform fading, we set the signal-to- to-noss ratio (SNR) to 14 dB for the source distribution and 20 dB for the target distribution. The connection between the SNR and the distribution parameters given Appendix [
We have experimented with other combinations of SNR for the source and target channels and found similar trend in the adaptation performance.
In the simulated experiments, we focused on the SNR range dB to dB. Our process for selecting this SNR range was by first evaluating the symbol error rate (SER) vs. SNR curve of the autoencoder for the different simulated channel distributions. We found that going below 14 dB SNR results in degradation of the autoencoder's performance (except for the AWGN channel, which we do not use as a target distribution). Also, going above 20 dB SNR did not lead to a significant decrease in the SER. For the channels such as Ricean fading and Uniform fading, found that even a retrained autoencoder has a relatively high error rate for lower SNRs.
C.2 DETAILS THE FPGA EXPERIMENT
Referring to the experiment in 4.2 for the real and over-the-air traces we used the platform from Lacruz al. (202T). This ultra-wide-band mm-wave transceiver basehand memory-based design is developed on top of ZCU111 RFSoC FPGA. This evaluation board features Zynq Ultrascale + ZCU28DR. This FPGA is equipped with 8 AD/DA converters with Giga-sampling capabilities, which make it ideal for RF system development; the 4 GB DDR4 memories contain RF-ADCs with up to 4 GSPS of sampling rate, and RF-DACs with up to 6.544 GSPS. This board also includes a quad-core ARM Cortex-A53 and dual-core ARM Cortex-Re R5 real-time processor.
For the radio frequency, we used 60 GHz RF front-end antennas. These kits include a 16 + 16 TRX patch array antenna plus the RF module with up/down conversion from baseband to I/Q channels,
21

---

Published as a conference paper at ICLR 2023
and TX/RX local oscillator (LO) frequency control. The antennas use 57 71 GHz, range of frequencies that cover the unlicensed 60 GHz band for mm-wave channels, and are managed from a PC Host via USB.
We implemented a hardware on the loop training. For the experimentation on real traces, we use Matlab as central axis. The PC host running Matlab is connected the platform via Ethernet. The FPGA can transmit different custom waveforms like 16-QAM frames from the 802.1 lad and 802.1lay standards, with 2 GHz of bandwidth. The frames sent over-the-air via 60 GHz radio frequency kits, and the samples are stored at the FPGA DDR memory. We decode the received data from the transmission, removing the preamble and header fields and extracting the symbols train the MDN. We add preamble to the generated constellation from the MDN for packet detection purposes, and we transmit again the new waveforms over-the-air. Finally, the adaptation performed offline with the decoded symbols from the custom autoencoder-learned constellation.
Source and Target Domains.
For the experiment in $ 4.2. we introduced distribution changes via IQ imbalance sabslance-based distortions to the symbol constellation, and evaluated the adaptation performance as a function of the level of imbalance. The source domain would be the original channel, the over-the-air link between the transmitter and receiver on which the training data collected. This source domain data is used for training the MDN and the autocncoder. The target domain would be modification of the source domain where the symbols used by the transmitter are distorted by modifying the in-phase and quadrature- phase (IQ) components of the RF signal. This causes change in the distribution observed by the receiver (decoder), leading to drop in performance without any adaptation.
C.3 DETAILS ON THE RANDOM GAUSSIAN MIXTURE DATASETS
We created a simulated distribution shift setting where data from both the source and target domains are generated from class-conditional Gaussian mixtures whose parameters are modified between the two domains (e.g., see Fig. The parameters for the source and target Gaussian mixtures are generated as follows:
Source Domain. The source domain data is generated with standard 16- 16-QAM constellation ZQAM: which has 16 classes (messages). Let k2 be the number of components in the source Gaussian mixture.
For each z E ZQAM:
Calculate dmin the minimum distance from z to the remaining symbols in ZQAM. Let a, dmin 4 be constant standard deviation for this symbol.
Component priors: generate TT (2( Unif(0.05, 0.95), Vi E [k.]. Normalize the priors to sum to 1.
Component means: generate (z)z) ~ N- z, II), Vi [kg].
Component covariances: generate $1. Sd iid Unif(0.2 os σ,) and let Σ,(z) diag(sf, ... .,,,,)..)))) ,.)), Vi E [k:]] (the covariances are diagonal).
Generate N* / samples corresponding to symbol from the Gaussian mixture: x", (z))).
Eh= (z) N (x
Target Domain. The parameters of the target Gaussian mixture are generated in a very sim- ilar way. The MDN and autoencoder are trained on the source domain dataset. Let (Ee, (11), -.. Ee, (1m)} be the constellation learned by the autoencoder. Let ke be the num- ber of components in the target Gaussian mixture.
For each z E Z:
Calculate dmin- the minimum distance from z to the remaining symbols in Z. Let or = dmin / 4 be constant standard deviation for this symbol.
Component priors: generate के (z) Unif(0.05, 0.95), Vi E [kr]. Normalize the priors to sum 1.
Component means: generate (z(2)) N(: IT) Vi [ke]-
22

---

Published as a conference paper at ICLR 2023
Component covariances: generate 81."", Ad iid Unif(0.2a,, oe) and let [x(2) diag($i, "., ,8)), Vi E [Kx] (the covariances are diagonal).
Generate No m samples corresponding to symbol from the Gaussian mixture: x/,
Ei (2) [x (z) (2)))
We set k; k = 3, except for the experiment where the source and target Gaussian mixtures are mismatched. In this case, Kg and ky are randomly selected for each dataset from the range {3, 4, 5, 6).
Random Phase Shift. We allow the channel output x to be randomly phase shifted on top of other distribution changes. This is done by matrix multiplication of with rotation matrix, where the rotation angle for each sample is uniformly selected from [-Ø, øj. We set & to я /18 or 10 degrees. Results dataset with random phase shift applied top of random Gaussian mixture distribution shift can be found in Fig. 5e
C.4 ABLATION EXPERIMENTS
We perform ablation experiments to understand: 1) the choice of the hyper- syper-parameter X, 2) the importance of the divvereeee regularization in the adaptation objective, 3) performance of our method when the source and target Gaussian mixtures have mismatched components, and 4) the performance of our method when there is no distribution change.
2-acto
J-0.001
2=0.1
A-0.1
A-200
2-0.11
2000
1-0.01
Ae1
0.4
0.01
Symbol error rate
0.35
0.3
Symbol
0.002
0.0001
10
20
#target samples per-class
#target samples per-class
(a) AWGN Uniform fading.
(b) AWGN to Ricean fading.
0.5
0.4
Symbol error rate
3.4-
0.3
03.
Symbal
0.25
0.2
50
30
#target samples per-class
#target samples per-clas class
(c) Ricean fading to Uniform Fading.
(d) Random Gaussian Mixtures.
Figure 8: Evaluation the validation metric for selection the hyper-parameter under different distribution shifts. The setting À= auto corresponds to automatic choice using the validation metric.
Automatic Selection of Hyper-parameter X. We evaluate the proposed validation metric for automatically selecting the hyper-parameter À and report the results in Fig, [ We run the proposed method for different fixed values of À as well as the automatically-seletted X, and compare their
23

---

Published as a conference paper at ICLR 2023
performance the target domain test set. We consider both simulated channel variations and the random Gaussian mixture datasets. From the figure, we observe that in most cases performance based on the automatically set value of À is better than other fixed choices of A. The case of adaptation from AWGN to Ricean fading an exception, where our method does not learn a good adaptation solution (see Fig. 4cp. In this case, observe from Fig. that the setting 0.0001 has the best symbol error rate.
0.45
A=0
λ-0
0.40
Symbol emerr race
Symbol esror rate
10"2
0.35
30
40
#target samples per-class
atarget samples per-class
(a) AWGN Uniform fading.
(b) AWGN to Ricean fading.
A=auto
A-auto
A=
0.6
0.6
error rate
0.5
Symbel emor
0.4
0.3
0.1
10
20
50
10
30
40
starget samples per-class
#target samples per-class
Ricean fading to Uniform Fading.
(d) Random Gaussian Mixtures.
Figure 9: Evaluating the importance of the KL-divergence term in the adaptation objective. The setting X auto corresponds to automatic choice using the validation metric.
Performance Under Component Mismatch. We evaluate the symbol error rate performance of all the methods in the setting where the number of components in the source and target Gaussian mixtures is mismatched. The number of components in the source and target Gaussian mixtures is randomly selected from the range strong performance improvements even in this mismatched setting, suggesting that our method can perform well even when Assumptions Al and A2 are not satisfied.
From Fig. we observe that the proposed method has
Importance of the KL-divergence Regularization. Recall that the adaptation objectives Eqs. (6) and (17) include the KL-ddivergence term scaled by in order to avoid large distribution changes when there is not enough support from the small target-domain dataset. A natural question to ask whether this term is useful and helps improve the adaptation solution when this, we compare the performance of our method with À = 0 with that our method with À set automatically using the validation metric. The results of this comparison are given in Fig. 9 on four simulated channel variations. The results averaged setting À for our method leads to much higher symbol enror rates compared to setting to non-zero value using the validation metric, establishing the importance of the KL--divergence term.
0. answer
multiple trials before. It is clear that
24

---

Published as a conference paper at ICLR 2023
Performance Under No Distribution Change. We evaluate the symbol error rate performance of all the methods in the setting where there no distribution change. In this setting, the performance of the MDN and autoencoder should not change, and expect the proposed adaptation method to maintain similar performance (not lead to increased symbol error rate). In Fig. TO we report the results of this experiment when both the source and target channel distributions fading or Uniform fading. We consider medium SNR value of 14 dB and a high SNR value of 20 dB. We observe that our method is relatively stable even when there is distribution change, and there is only small increase in error rate. For instance, in Fig. T0c the error rate of our method increases from 0.015 0.018 for 5 samples per class.
either Ricean
We expect that practical system that frequently adapts to changes in the channel distribution should first have distribution change-detection algorithm that takes batch new samples from the channel and detects whether there is any change in the distribution. The actual domain adaptation algorithm then applied only when a distribution change is detected. In this way, any potential drop in the autoencoder's performance when there is no distribution change can be made less likely.
No stapt
Tinetune
Retrained Reddinedde autoenc
Affinw (oun)
Timetune
Frcttnn bat
0.27
0.150
0.145
Symbol error rate
0.26
0.140
0.25
0.24
0.130
0.23
0.125
0.22
10
10
30
50
#target samples class
#target samples per-class
(a) Uniform Fading, SNR 14 dBd
Uniform Fading, SNR 20 dB.
No adapt
Finetune
Bet mirsdd aineer
No adapt
Finstune
Retrained autoonc
Afficc adopt (ours)
Finctaneles
Afline wdeep [suis]
Finettame last
0.018
Symbol orror race
Symbol enerraa
0.017
0.016
0.015
0.014
10
20
30
40
50
10
20
30
40
50
#target samples per-class
#target samples per-class
(c) Ricean Fading, SNR = 14 dB.
(d) Ricean Fading, SNR = 20 dB.
Figure 10: Performance under no distribution change, i.e. the source and target distributions are the same. The type of channel distribution and the corresponding SNR common to both the source and target domain is indicated. In Fig. (d) note that the y-axis is scaled by 10
C.5 ANALYSIS THE FAILURE AWGN TO RICEAN FADING
Referring to Fig. in the main paper, we observe that method has worse symbol error rate compared to no adaptation and the other baselines for the adaptation setting from an AWGN channel at 14dB SNR to Ricean fading channel at 20dB SNR. In order to get intuition for this, we look into the data distribution plots from the source and target channels (see Fig. 12]. From the figure, we observe that the target distribution is actually very class-separable, but the distribution shift between
25

---

Published as a conference paper at ICLR 2023
No odapt
Rieture
Feeture Petrained aucoenc
Amne
0.14
0.12
0.10
Symbol error
0.08
0.06
0.04
0.02
10
20
50
#target samples per-class
Figure 11: Performance under distribution shift where the source and target distributions are random Gaussian mixtures with mismatched components.
the AWGN and Ricean fading channels is quite large, particularly because of the high SNR of Ricean fading channel. In this case, the classifier from the source domain is able to classify the target domain data reasonably well without any adaptation. The fine-tuning baselines ("Finetunc" and "Finetune last") do not lead to much improvement and have similar SER the case of adaptation.
Sinvlited cha trei cutpurs from the MDN sea-qjeer.
2
0-
-0.5
-1
-10
-3
-1.0
-0.5
0.0
0.5
1.0
-3
-2
(a) AWGN data 14dB SNR.
Ricean fading data 20dB SNR.
Figure 12: Data plots of the source domain (AWGN at 14dB SNR) and the target domain (Ricean fading at 20dB SNR). The black points are the channel inputs (i.e., the symbols). A 16-QAM constellation is used in the source domain (left), while the learned autoencoder constellation is used in the target domain (right).
On the other hand, the proposed method leads to worse SER on the target domain after adaptation. hypothesize that it fails in this case because the set of affine transformation are unable to capture the large distribution change well. In our current formulation, the affine transformations per component are shared by all the classes. From Eq. 4 the affine transformation parameters A, b; C₁ Br and ". i E [k]. Based on the data plots, think that allowing the affine transformations to be class-specific can handle this adaptation setting better. While allowing the affine transformations be class class-specific provides more flexibility, also introduces parameters which need to be optimized using a small dataset from the target distribution.
C.6 EXPERIMENTS GENERATIVE ADAPTATION THE MDN
We evaluate the generative adaptation method of the MDN discussed in Appendix B.3 where the goal is to achieve high conditional log- likeelilioood (CLL) under the target data distribution. As shown in Fig. 13 consider the standard simulated distribution changes including random Gaussian mixtures, and vary the number of target samples per-class from to 20. For each pair of source-target distributions and each target sample size, the experiment is repeated for 50 trials, and the average CL.L
26

---

Published as a conference paper at ICLR 2023
reported. The CLL is calculated held-out test dataset of size 25,000 from the target distribution. For the simulated channels, the SNR of the source and target distributions are each randomly chosen from the range dB to dB. From Fig. [3] observe that the proposed MDN adaptation has significantly higher CLIL compared to no adaptation and the finetuning baseline methods, especially for very small number of target samples per class.
-5
-10
Reellhod
-15
-15
Anetune
Aieeure ledt
-2
-20
Retroined MDN
-5
adapt
edeect (curs)
Fieeune
-30
Firetune last
Retrained MDN
12
16
20
14
18
#target samples per-class
#target samples per-class
AWGN Uniform fading.
(b) Ricean fading Uniform fading.
-2
Log-i Log-ikellhood
Log- heppiiieehhhodd
Affinw adap: (sun)
Rnetune
Anetune ast
Ratrained MON
adspt Affine adact (ours)
-10
-8
Pirstane
Firetune last
Reb Retrained NON
-10
2
8
10
12
16
18
20
8
10
12
14
16
18
20
#target samples class
#target samples
(c) AWGN Ricean fading.
(d) Random Gaussian mixtures.
Figure 13: Generative adaptation of the MDN channel to different types of distribution shifts. Larger values of conditional log-likeling likelihod correspond to better adaptation solutions.
C.7 EVALUATION WITH VERY FEW TARGET DOMAIN SAMPLES
To evaluate how the proposed method performs with fewer than 10 samples per class, we ran an experiment on some simulated distribution changes with 1,
and samples per class. We report these results in Fig. [4] for the following distribution changes: 1) Random Gaussian mixtures (Fig. 14ap, and 2) Ricean fading to Uniform fading (Fig. 14b). We follow the same protocol as in the main paper and report the average results from multiple random trials. From Fig, 14a, corresponding random Gaussian mixtures, we observe that the proposed method able improve performance (decrease SER) starting from sample per class.
However, in Fig. 44a (Ricean fading to Uniform fading), the proposed method increases the SER (worse) compared to no adaptation for and 2 samples per class. The SER starts to decrease significantly from samples per class. We hypothesize that the reason for failure of the proposed method with 1 and 2 samples per class could be the strong distribution change and complexity of the target uniform fading channel. While this may seem concerning, please note that we chose these strong simulated distribution changes in order to demonstrate the potential improvements of our method. In practical wireless channels, the distribution changes are likely to be more gradual, so that the proposed method usually adapt well with only few samples per class.
27

---

Published as a conference paper at ICLR 2023
No 20ag:
Rictune
Rctra not sucoenc
No asopt
Rectune
Actraincs sutoon:
Affic adbpt jours)
Rictune lest
Affice sdapt lours)
Anotune last
0.14
0.12
0.30
Symbol erterral rate
0.10
Symbol erforrate rate
0.25
0.08
0.20
0.04
0.02
5
10
40
50
10
20
50
@ @arrget samples per-class
Ntarget samples per-dass
(a) Random Gaussian mixtures.
(b) Ricean fading (14dB) to Uniform fading (20IB).
Figure 14: Results of autoencoder adaptation simulated distribution changes with very few target domain samples (starting from sample per class). For the case where both the source and target domain data are from random Gaussian mixtures, our method is able to improve the SER even with 1 sample per class. In the case adaptation from Ricean fading to Uniform fading, our method increases the SER (compared the SER after that.
adaptation) for and samples per class, but starts to decrease
C.8 GAP BETWEEN THE PROPOSED METHOD AND THE RETRAINED AUTOENCODER
In this section, we address the following question: Does the performance of the proposed method approach that of the oracle retrained method as the size of the adaptation dataset increases? We note that the oracle method "Retrained autoenc" retrains the channel model, encoder and decoder networks jointly using Algorithm|] [ in Appendix D.3 In our experiments, from the target (domain) distribution for training the MDN channel in between each epoch of the encoder/decoder update, and used 300, 000 samples for training the encoder/decoder networks with the MDN parameters fixed. The retrained autoencoder therefore learns custom optimal constellation (encoding) for the target distribution.
used 20, 000 samples
Table 4: Performance of the proposed adaptation method with large number of target domain samples, and comparison with an oracle retrained autoencoder for the target domain.
Proposed method
Proposed method
Source and target domains
Retrained autoencoder
(50 samples/class)
(1000 samples/class)
Random Gaussian mixtures
0.0766
0.0695
0.0154
AWGN Uniform fading
0.2370
0,2138
0. 1315
Ricean fading to Uniform fading
0.1945
0,1815
0.1345
AWGN to Ricean fading
6.38 13-4
2.531e-5
3. 3.125e-6
Uniform fading Ricean fading
6.484c-4
5.503e-4
3.125e-6
Our adaptation method, on the other hand, does not modify the constellation learned by the au- toencoder from the source domain data, which could be sub-optimal for the target domain. This restriction of not modifying the constellation is practically advantageous since there no need to communicate the new symbols back to the transmitter. Also, there is no need to change the transmitter side encoding frequently. Since the proposed method does not have the full flexibility the retrained autoencoder, we do not expect its performance (SER) converge to that of the latter under significant distribution change. For instance, Fig. 12b shows the learned autocncoder constellation for a Ricean fading channel of 20 dB SNR. This is quite different from the optimal constellation for AWGN channel of 14 dB SNR (which is closer to a 16-QAM constellation).
To understand this better, adaptation dataset, and compared it to the oracle retrained autoencoder. In Table 4 we report the SER of method with 1000 samples per class (so 16, 000 samples overall) subset of simulated
experiments evaluate our method with large (target)
28

---

Published as a conference paper at ICLR 2023
distribution changes. We follow the same protocol as the main paper, and report the average SER from multiple random trials on a large test set from the target distribution.
We also report the performance of method with 50 samples/class for comparison. From the table, we observe that there is a gap in the SER of our method (with 1000 samples/class) and that of the retrained autoencoder. We believe that allowing the encoder symbols to be optimized might be required order to bridge this gap in SER. In conclusion, when it is possible to collect sufficient (large) data from the target distribution, it might be better to allow the encoder network to also be optimized via retraining.
D DETAILED BACKGROUND
Expanding on $ 2 we provide additional background on the following topics: 1) components of an und-to-end autoencoder- based communication system, 2) generative modeling using mixture density networks, 3) training algorithm of the autoencoder, and 4) a primer on domain adaptation.
BASSED D.1 AUTOENCODER BASED END-TO-END LEARNING
Muttp lopors
Yoo moll botiion layer
cense tayers
P. x)
Demus loyer softmax stlvaation
massage
Omeeho empodkngg
Wireless channel
Predicted
E y
P(x x))
argmna Po
Pr, (m xx)
Generative Pg [s)
Encoder Transmillee network
Dacoder Reosiver neural newwrrk
B4 (1,)
ha, uz)
Do, (x)
Figure 15: An end-to-ema autoencoder- baseed communication system with a generative channel model.
Consider a single-input, single-output wireless communication system as shown in Fig. D where the transmitter encodes and transmits messages from the set y' 21,2,... .m] to the receiver through 2 2 discrete uses of the wireless channel. The receiver attempts accurately decode the transmitted message from the distorted and noisy channel output x. We discuss the end-to-end learning of such a system using the concept of autoencoders (O'Shea & Hoydis) 2017 Dörner et al.. 2018).
Transmitter Encoder Neural Network. The transmitter or encoder part of the autoencoder is modeled as multi-layer, feed-forward neural network (NN) that takes as input the one-hot-coded representation 1, of message y E V, and produces an encoded symbol vector z = Eg. (1,,) E IRd Here, O. are the parameters (weights and biases) of the encoder NN and is the encoding dimension. Due to hardware constraints present at the transmitter, a normalization layer used as the final layer of the encoder network in order to constrain the average power and/or the amplitude of the symbol vectors. The average power constraint is defined as | = [ (1 || S c, where the expectation is over the prior distribution of the input messages, and typically set 1. The amplitude constraint is defined as |z| < 1, Vi E [d]. The size of the message set is usually chosen to be a power of 2, i.e., the communication rate of this system is the number of bits transmitted per channel use, which in this case is R b/ d. An autoencoder transmitting bits over d uses of the channel is referred as (d, b) autoencoder. For example, (2, 4) autoencoder uses message set of size 16 and an encoding dimension of 2, with communication rate R bits/channel use.
2 representing bits of information. Following O'Shea & Hoydis (2017).
Receiver / Decoder Neural Network. The receiver or decoder component is also a multilayer, feedforward NN that takes the channel output x IR as its input and outputs a probability distri- bution over the m messages, The input-output mapping of the decoder NN can be expressed as De. (x) = [Po, (1 x).... Po. (m x))", where 0₂ are the parameters of the decoder NN. The softmax activation function is used at the final layer to ensure that the outputs valid probabilities. The message corresponding to the highest output probability is predicted as the decoded message, ie.. xxx argmax,ey Po, (y/ x). The decoder NN is essentially a discriminative classifier that
29

---

Published as a conference paper at ICLR 2023
learns accurately categorize the received (distorted) symbol vector into of the m message classes. This is in contrast to conventional autoencoders, where the decoder learns to accurately reconstruct nigh-dimensional tensor input from its low-dimensional representation learned by the encoder. In the case of communication autoencoders, the symbol or block error rate (BLER), defined as (x [1 (7(() * v)}, is used the end-to-end performance metric.
Channel Model. As discussed in $ the wireless channel can be represented by a conditional probability density of the channel output given its input p(x z). The channel can be equivalently characterized by a stochastic transfer function X = h(z, u) that transforms the encoded symbol vec- tor into the channel output, where u captures the stochastic components of the channel (e.g., random noise, phase offsets). For example, an additive white Gaussian noise (AWGN) channel is represented by x h(z, u) = z + u, with u ~ [N 0, o2 Ia) and p(x 2) N z, a² Iz). For realistic wireless channels, the transfer function and conditional probability density are usually unknown and hard to approximate well with standard mathematical models. Recently, a number of works have applied generative models such as conditional generative adversarial networks (GANs) (O ''Shea ctt al. 2019; Ye et al. 2018), MDNs (García Martí et al. 2020), and conditional variational autoen- coders (VAEs) (Xia et al. 2020) for modeling the wireless channel. To model wireless channel, generative methods learn a parametric model Po. (x z) (possibly a neural network) that closely approximates the true conditional density of the channel from dataset of channel input/output observations. Learning a generative model of the channel comes with important advantages. 1) Once the parameters of the channel model are learned from data, the model can be used generate any number of representative samples from the channel distribution. 2) A channel model with a differentiable transfer function makes it possible to backpropagate gradients of the autoencoder loss through the channel and train the autoencoder using stochastic gradient descent (GGD)-based optimization. 3) It allows for continuous adaptation of the generative channel model variations in the channel conditions, and thereby maintain a low error rate of the autoencoder.
D.2 GENERATIVE CHANNEL MODEL USING MIXTURE DENSITY NETWORK
As discussed in $ 2 we use an MDN (Bishop 1994 2007 with Gaussian components to model the conditional density of the channel. MDNs can model complex conditional densities by combining a (feed-forward) neural network with standard parametric mixture model (e.g., mixture of Gaussians). The MDN learns to predict the parameters of the mixture model <b(z) as a function the channel input 2. This can be expressed as Φbz)) Ma. (z), where 0g are the parameters of the neural network. The parameters of the mixture model defined by the MDN are a concatenation of the parameters from the A density components, i.e., "²z)" = (z)"" ..... (z)"), where (z)z) is the parameter vector of component i. Focusing on a Gaussian mixture, the channel conditional
density given each symbol z E Z is given by
&
k
Po. -K= = Po. (x 2x = Σ Po. (xx 1) = Σ
T
(20)
where (z) Rd is the mean vector, Ez(z) Raxd is the (symmetric, deffiie covariance matrix, and (z) E [0, 1] is the weight (prior probability) of component é. Also, K is the latent ran- dom variable denoting the mixture component of origin. The weights of the mixture are parameterized
using the softmax function as (z) (2) Σ (x)()), Vi in order to satisfy the probability
constraint. The MDN simply predicts the un mo-nommal weights (z( E R (also known as the prior logits). We define the parameter vector of component as (z(Z)) = [a,(z), p4 (z)", vec(E, (z))"), where vec(-) is the vector representation of the unique entries of the covariance matrix. Details on the conditional ligelikelhoood (CLL)) training objective and the transfer function of the MDN can be found in Appendix
D.3 TRAINING OF THE AUTOENCODER
In this section, we provide a formal discussion of the end-to-end training of the autoencoder. First, we define the input-output mapping of the autoencoder as f., Do, (he, (Ee, (1y), u)) (De, he. (.. u) E (1,), where 0T 10T 0. 0% is the combined vector of parameters from the encoder, channel, and decoder. Given an input message y e V. the autoencoder maps the one-hot- coded representation of y into an output probability vector over the message set. Note that, while the
30

---

Published as a conference paper at ICLR 2023
encoder and decoder neural networks are deterministic, forward pass through the autoencoder is stochastic due to the channel transfer function he.- The learning objective of the autoencoder is to accurately recover the input message at the decoder with high probability. The cross-entropy (CE) loss, which is commonly used for training classifiers, also suitable for end-to-end training of the autoencoder. For an input y with encoded representation
Eg, (Iy), channel output x he, (2, u),
and decoded output Do, (x) [Po, (1 x).... Po. (ml x)]". the CE loss is given by y log to (1g)
fcB (c,
1 log Do, (bo, (Eo, (Iy), u)) log Po, (y ho. (Eo, (1)):)).
(21)
which is always non-negative and takes the minimum value 0 when the correct message is decoded with probability 1. The autoencoder aims to minimize the following expected CE loss over the input message set and the channel output:
(1y: (I,)) = - p(u) / Pa. (x Eo, (1y)) log Pa (y x) dx.
(22)
Rd
Here {p(v), Vy E y is the prior probability over the input messages, which is usually taken to be uniform in the absence of prior knowledge. In practice, the autoencoder minimizes an empirical estimate of the expected CE loss function by generating large set of samples from the channel conditional density given each message. Let X = {x ho, (Eo, (1y), un), ), n 1..... ,N denote a set of independent and identically distributed (iid) samples from Po. (x Eo, (1,)). the channel conditional density given message y. Also, let X Up denote the combined set of samples from all messages. The empirical expectation the autoencoder CE loss (22) is given by
m >
(auto(0;X)
(0:X)
p(y)
Σ
log Pe. (y) ho, (Eo, (1y),
(23)
is clear from the above equation that the channel transfer function he should be differentiable in order to be able to backpropagate gradients through the channel to the encoder network. The transfer function defining sample generation for a Gaussian MDN channel is discussed Appendix L
Algorithm 1 End-to-end training of the autoencoder with an MDN channel
1: Inputs: Message size m; Encoding dimension d: Initial constellation (Eo(Ly); Vy E V): Number of optimization epochs for the autoencoder Nae and channel Noe-
2: Output: Trained network parameters O., 0., Oa-
3: Initialize the encoder, channel, and decoder network parameters.
4: Sample training data ! from the channel using the initial constellation.
5: Train the channel model for No: epochs to minimize (O, ) (Eq. 24h.
6: for epoch = .1............... Nas:
7: Freeze the channel model parameters 6..
8: Perform round of mini-batch SGD updates of 0. and O₂ with respect to Lanto (f; X).
9: Sample data e from the channel using the updated constellation (Ee, (1,), Vy E )).
Train the channel model for Noe epochs minimize Leh(O.; D(!) (Eq. 24h.
11: end for
12: Return O,, 0.,
The training algorithm for jointly learning the autoencoder and channel model (e.g., García Marti et al. (2020) given in Algorithm It an alternating (cyclic) optimization of the channel parameters and the autoencoder (encoder and decoder) parameters. The reason this type of alternating optimization is required is because the empirical expectation the CE loss Eq. (3) is valid only when the channel conditional density (i.e., 0.) is fixed. The training algorithm can be summarized as follows. First, the channel model is trained for Noe epochs using data sampled from the channel with an initial encoder constellation (e.g., M-QAM). With the channel model parameters fixed, the parameters of the encoder and decoder networks are optimized for one epoch of mini-batee SGD updates (using any adaptive learning rate algorithm e.g., Adam (Kingma & Ba 2015)). Since the channel model is longer
31

---

Published as a conference paper at ICLR 2023
optimal for the updated encoder constellation, is retrained for Nce epochs using data sampled from the channel with the updated constellation. This alternate training of the encoder/decoder and the channel networks is repeated for Nac epochs until convergence.
Finally, we observe some interesting nuances of the communication autoencoder learning task that is not common to other domains such as images. 1) The size of the input space is finite, equal to the number of distinct messages m. Because of the stochastic nature of the channel transfer function, the same input message results in a different autoencoder output each time. 2) There is theoretically no limit on the number of samples that can be generated for training and validating the autoencoder. These factors make the autoencoder learning less susceptible to overfitting, unlike in other domains.
D.4 A PRIMER ON DOMAIN ADAPTATION
We provide brief review of domain adaptation (DA) problem and literature. In the traditional learning setting, training and test data are assumed to be sampled independently from the same distribution P P(x,y) where x and y/ are the input vector and target respectively [ In many real world settings, it can be hard impractical to collect a large labeled dataset Df for a target domain where the machine learning model (e.g.. a DNN classifier) is to be deployed. On the other hand, it is common to have access to large unlabeled dataset Du from the target domain, and large labeled dataset D from a different but related source domain [ Both Dj and D.' are much larger than Dj, and in most cases there is no labeled data from the target domain (referred to unsupervised DA). For the target domain, the unlabeled dataset (and labeled dataset if any) are sampled from an unknown target distribution, i.e., Di (x(x) and (x,9) E Dj Px(x, y) For the source domain, the labeled dataset is sampled from an unknown source distribution, i.e., (x, y) E D' (x y) The goal of unsupervised DA to leverage the available labeled and unlabeled datasets from the two domains to learn predictor, denoted by the parametric function ÿ = fx()), such that the following risk function w.r.t the target distribution is minimized:
Re[fo] (.x.))-m,1(ffoxxx.... =
E Pi(x, 8) ((f(xx)))) y) dx,
where É(Ø. 3) is a loss function that penalizes the prediction ÿ for deviating from the true value (e.g., cross-entropy hinge loss). In similar way, we can define the risk function w.r.t the source distribution [ A number of seminal works in DA theory (Ben-David et al., 2006 Blitzer et al. 2007 Bev-D et al.. 2010) have studied this learning setting and provide bounds on Re in terms of fo and the divergence between source and target domain distributions. Motivated by this foundational theory, a number of recent works (Ganin & Lempitsky 2015 Ganin al., 2016
Long et al. 2018 Saito et al. 20188] ZZhao et al., Johansson et al. 2019) have proposed using DNNs for ad vdvrsariillyy learning a shared representation across the source and target domains such that a predictor using this representation and trained using labeled data from only the source domain also generalizes well to the target domain. An influential work in this line of DA is the domain adversarial neural network (DANN) proposed by Ganin & Lempitsky (2015) and later by Ganin et al. (2016). The key idea behind the DANN approach is to adversarially train a label predictor NN and a domain discriminator NN in order to learn a feature representation for which i) the source and target inputs are nearly indistigguishable to the domain discriminator, and ii) the label predictor has good generalization performance on the source domain inputs.
Special Cases of DA. While the general DA problem addresses the scenario where P, y) and (x y) are different, certain special cases of DA have also been explored. One such special case is covariate shift (Sugiyama et al.] 2007 Sugiyama & Kawanabe 2012), where only the marginal
distribution of the inputs changes (I.e., (x( + P.()), ), but the conditional distribution of the target given the input does not change ei.e.. (y x) R P. (y x)). Another special case is the so-called
label shift or class-prior mismatch (Saerens et al. 2002. Du Plessis & Sugiyama 2014), where only
), but the conditional distribution of the input given the target does not change (i.e., P. y) R3 Px g)). Prior works have proposed targeted theory and methods for these special cases of DA.
the marginal distribution of the label changes (i.e., (v() * Pa())),
'The notation used in this section is different from the rest of the paper, but consistent with the statistical learning literature. One could have multiple source domains in practice; we consider the single source domain setting.
32

---

Published as a conference paper at ICLR 2023
E MDN TRAINING AND SAMPLE GENERATION
In we briefly discussed how a Gaussian mixture density network (MDN) can be used to learn generative model for the channel. Here, we provide details the training algorithm for the MDN, followed by a discussion on the sampling function he of an MDN, and how to make the sampling function differentiable to enable SGD-based training of the autoencoder. Given dataset of input-output pairs sampled from the channel D.- = {(Z,, Xn); ; n = 1,,.. ... Neh. the MDN is trained minimize the negative conditional log- liglliillhhood (CLL) of the data given by
1 Nc
log Po. (Xn kn))
(24)
where the Gaussian mixture Po. (x,, ) is given by Eq. T With large Ne. the MDN can learn a sufficiently-comee compppex parametric density model of the channel. The negative CLL objective can be interpreted as the sample estimate of the Kullback-Leible divergence between the true (unknown) conditional density P(x z) and the conditional density modeled by the MDN Pa. (x [x|2). Therefore, minimizing the negative CLL finds the MDN parameters O, that lead to a close approximation of the true conditional density. Standard SGD-based optimization methods such as Adam (Kingma & Ba 2015) can be applied to find the MDN parameters O. that (locally) minimize the negative CLL.
After the MDN is trained, new simulated samples from the channel distribution can be generated from the Gaussian mixture using the following stochastic sampling method. We focus the diagonal covariance case for simplicity, where of (z) E IRC are the diagonal elements of the covariance matrix (z) for component
1. Randomly select an encoded symbol z from the constellation according to the prior distribution {p(z); 2)
10
2. Randomly select component K = according to the mixture weights { (z) ... (.,,,(())))))))))) (z))) Randomly sample from the isotropic d-dimensional Gaussian density u ~ (NN10001,
4. Generate the channel output as x (z O u + (z).
Here O refers the element-wise (Hadamard) product of vectors. The channel transfer or sampling function for a Gaussian MDN can thus be expressed as
k
x = ho. (z. = Σ 1(K = (o)()) (()))))+))))))).
(25)
where (a)(m1(z),... Te. (z)) and u N L2). Note that this transfer function is not differentiable w.r.t parameters (T, (2)} and the MDN weights predicting it, because of the indicator function. As such, is not directly suitable for SGD (backpropagation) based end-to-end training of the autoencoder (see Algorithm []. We next propose differentiable approximation of the MDN transfer function based on the Gumbel softmax reparametrization (Jang et al. 2017), which used in autoencoder implementation.
E. DIFFERENTIABLE MDN TRANSFER FUNCTION
Consider the transfer function of the MDN in Eq. (25). We would like to replace sampling from
the categorical mixture prior (Z), Te (z) with a differentiable function that closely approximates it. We apply the Gumbel-Sofiie this exact problem. Recall that the component prior probabilities can be written in terms of the prior logits as:
reparametrizationn (Jang et al. 2017) which solves
(x)
(z)
Vi [k].
(z)
Consider i iid standard Gumbel random variables G1, G iid that, for any z E Z, the random variable
Gumbel 1). It can be shown
S(z) argmax G, + (z(
(26)
TE[k]
This the the prior distrihution the input messages.
33

---

Published as a conference paper at ICLR 2023
follows the categorical distribution Cat( (z)..... Gumbel-max transformation. While Eq. (26) can be directly used inside the indicator function in Eq. (25). the argmax will still result in the transfer function being con-differentiable. Therefore, use the following temperature-scaler softmax function as a smooth approximation the argmax
(z)).
This standard result is known as the
expl(G: + (z)) /
(.(2;T) Ej=1 expl(G; az())) /r]
Vi E (k).
(27)
where r >> is temperature constant. For small values of r, the temperature- softmax will closely approximate the argmax, and the vector (z:T,,,, the one-hot vector [I()(z) = i., ((((z) = k)].
(2;z]]]] ..x;:)))) will closely approximate
Applying this Gumbel softmax reparametrization Eq. we define modified differentiable transfer function for the Gaussian MDN as
ho. (z, u) Σ S((2TT) o + (z())).
(28)
With this transfer function, it straightforward to compute gradients with respect the prior logits (z),) vi. Another neat outcome of this approach is that the stochastic components (Gumbel random variables G₂) are fully decoupled from the deterministic parameters a, (z) the gradient calculations with respect to (z In our experiments, we used this Gumbel-softmax based smooth transfer function while training the autoencoder, but during prediction (inference), we use the exact argmax based transfer function. We found T = 0.01 to be good choice for all the experiments.
F SIMULATED CHANNEL VARIATION MODELS
We provide details of the mathematical models used to create the simulated channel variations in our experiments. These models are frequently used in the study of wireless channels (Goldsmith, (2005).
F.1 ADDITIVE WHITE GAUSSIAN NOISE (AWGN) MODEL
This is the simplest type of channel model where the channel output is obtained by adding random Gaussian noise to the channel input i.e., n~ N O, of Ia) is independent of We find the signal-to-noise ratio (SNR) of this channel, and specify how to set the noise variance of in order to achieve a target SNR value.
+ n. It is assumed that the noise
The average power in the signal component of x is given by E ||| Pavg: The noise power in this case is given by |[][]]]]] o. The signal-to-noise ratio (SNR) for the AWGN model is therefore given by
E. No
E 2 |n
Pave 2
2RE
where is the communication rate of the system in bits/channel use (O' & Hoydis 2017). To simulate AWGN channel with target SNR of Eg/No, select the noise variance follows:
Pavg 2 (F./NN)))
σ
(29)
F.2
UNIFORM FADING MODEL
The channel output x E Rd for this model as a function of the channel input (symbol vector) z E IRd given by
x
Az +n
where A ~ Unif[O, a] is uniformly-distrien Gaussian noise vector. Both A and n are assumed to be independent each other and z. The average power in the signal component of x is given by
scale factor, and ~ (- 0, oi La) is an additive
Pave = Az] =
a²
3 Paygr
34

---

Published as a conference paper at ICLR 2023
where Pang denotes the average power in the channel input The noise power in this case is given by B og The signal-to-noise ratio (SNR) for this model is therefore given by
Eb E A 2
a² Pavg
No
2 P ||
6 o
where the communication rate of the system in bits/channel use. We select the fading factor such that the channel output has target SNR value using the following equation:
6 Roo (Es/No)
(30)
Pavg
F.3 RICEAN AND RAYLEIGH FADING MODELS
The channel output for the Ricean fading model is given by
Aztn,
where is a diagonal matrix with the diagonal elements a1, ag sid Rice(- v, o?) following a Rice distribution, and ~ (- o Ia) is additive Gaussian noise vector. It assumed that and A are independent of each other and of z. Note that Rayleigh fading is special case of Ricean fading when the parameter given by
For this model, the average power in the signal component of is
d
Pavg = R 1 = Σ E[a 2]] = Σ
B[a R
= (2 + 22) [[12 = (20 + +v²) Paug:
where Pavg denotes the average power in the channel input z. We used the fact that the second moment of the Rice distribution is given by E[aş] 2 +2² Is useful consider the derived parameters K = 22 /2/2222 /2 which corresponds to the ratio of power along the line-of-sight (LoS) path the power along the remaining paths, and power received along all the paths. The SNR for this model is given by
2 + 12 which corresponds the total
E
E
(20- + +2²)
No
2
Rog
For given input average power and target SNR, the parameters of the Rice distribution can be set using the equation
2 20 2 2 2Ro (Es/No)
Pavg
To create channel variations of different SNR, we fix the variance of and vary the power of the LoS component v² Suppose the smallest SNR value considered is Smin. we set o² using
20 R Smin
(31)
Pagg
and set v to achieve target SNR F using
2 R of (Eo/No Smin)
(32)
Povg
For this choice of parameters, the power ratio of LoS to non-LoS components is given by
F / No 1.
K
Smin
The K-factor for Rician fading in indoor channel environments with an unobstructed line-of-sight is typically in the range dB to 14 dB (Linnartz) 2001). Rayleigh fading is obtained for K = 0 (or # 0).
Finally, note that the vector z is composed of one or more pairs of in-phase and quadrature (IQ) components of the encoded signal (i.e., dimension = 2p). Since each IQ component is transmitted as single RF signal, the Ricean amplitude scale is kept the same for successive pairs of IQ components in z. other words, the amplitude scales are chosen to be ai a1 ... Opp ap. This does not change any of the above results.
35


Reviews:
[{"review_id": "r6tim9gHgw", "reviewer": "Reviewer_fua1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- The paper addresses the problem of handling domain-shifts that arises in generative learnt channel models in E2E communication systems in a few-shot setting.\n- The proposed domain adaptation approach is tailored around a Mixture Density Network (MDN) representing the channel model. In here, the approach:\n    - learns an adapter layer, which models an affine transform of the original conditional channel distribution\n    - introduces an additional regularization objective to ensure the adapter doesn't converge to bad/degenerate solutions\n    - presents a feature transformation formulation on the decoder side to aid learning on the domain-shifted distributions\n- The approach is evaluated extensively, covering multiple types of distribution changes in both synthethic settings as well on a high-resolution mmWave testbed.", "summary_of_the_review": "The paper tackles a relevant bottleneck in generative channel modelling for E2E communication systems (i.e., they are trained assuming a stationary distribution, but this isn't the typical case). The approach is novel and intuitive in my opinion, and is further evaluated extensively in both simulated and real conditions. While I have some minor concerns (e.g., can one really have a labelled dataset for this task in practise?), I don't think they significantly affect the paper's claims and contributions. \n", "strength_and_weaknesses": "### Strengths\n\n**1. Extensive evaluation**\n- The approach is evaluated rigorously with well-suited baselines and a range of scenarios (e.g., multiple types of domain shifts, real-world evaluation). I especially appreciate evaluations studying when the (reasonable) assumptions are violated.\n\n**2. Motivation and relevant problem**\n- While there has been a lot of attention on generative channel modelling recently, most works in my knowledge largely (and somewhat incorrectly) assume a stationary distribution. This paper takes a step in the right direction by addressing this pain-point.\n\n**3. Insightful approach**\n- The approach overall is insightful and makes sense. By learning an adapter network and learning parameters relevant for the domain shifts (e.g., like FiLM modules), it makes few-shot domain-adaptation more tractable.\n- Furthermore, I find the choice of the channel model representation (MDNs) to also be sufficiently appropriate for the task (as opposed to GANs) for this study. \n\n### Concerns\n\n**1. \"labeled set obtained for free\"**\n- The paper at multiple times claims that few-shot learning is especially possible since we can get labeled dataset for free -- I find this slightly confusing.\n- Wouldn't the labeled dataset be split between the encoder (transmitter) and decoder (receiver) devices? As a result, for a party to have the full labeled dataset, isn't a prerequisite communicating labels back to the other party?\n\n**2. Evaluation: Some observations unclear**\n- I found some patterns in the evaluation was somewhat unclear and would appreciate the authors' answers on the questions below:\n- (a) Oracle-approach gap in Figure 4/5: I'm slightly surprised that proposed approach's symbol error rate does not converge to the oracle with a reasonable number of additional examples (50 * 16-QAM classes = 800), given that there are 50 learnable parameters. Are the authors aware if convergence is possible with even higher examples? Morevover, what is the size of the source dataset?\n- (b) Unchanged error rates in Figure 4/5 for many baselines: Are the authors aware of why the error rates of many baselines do not improve at all in spite of more training examples? Were the \"finetune\" baselines finedtuned only on the new data or a combination? In the case of combination, are domain-invariant features learnt? \n- (nitpick) Please summarize the performance degradation discussions in Ricean fading experiments in the main paper.\n\n**3. Evaluation: Performance under no distribution change**\n- I appreciate that the authors also evaluate under a non-domain shifted dataset in Figure 10. Can the authors clarify why results drop in performance when there is no distribution change?\n- Specifically, it appears that the adapter layers' parameters are initialized such that it produces a identity mapping (page 18), so I'm surprised that this nonetheless degrades performance.\n\n**4. SNR=14-20 dB**\n- Can the authors comment whether a SNR of 14-20dB (which to me appears really large) is a reasonable setting? Did the authors also evaluate SNR vs. error rates for the approach and baselines? I wonder if the results shown here apply only in high SNR regimes.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "SAw8jL3qx2", "reviewer": "Reviewer_jGKn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors provide a few-shot domain adaptation method to address the channel distribution changes of communication systems. Specifically, using the properties of Gaussian mixtures, they propose a solid domain adaption process for the generative channel model (MDN). Besides, they propose a input-transformation method which transform the input of decoder from target domain into source domain, without modifying the encoder-decoder networks. They also derive experiments on a mmWave FPGA platform and show the strong performance improvements of the proposed method. ", "summary_of_the_review": "The paper is well organized and has rich details. The work is based on appropriate assumptions and the properties of Gaussian mixtures, and the effectiveness is demonstrated by experiments on both simulated and real experiments. ", "strength_and_weaknesses": "Strength\n1.\tThe paper is well organized and has rich details.\n2.\tThe advantage of the proposed method is clearly stated and demonstrated.\n3.\tThe adaptation approach is based on appropriate assumptions and is well supported by the properties of Gaussian mixtures.\n4.\tThe effectiveness of the method is evaluated by both simulated and real experiments, and there are also experiments when the assumptions could not hold.\n\nWeaknesses\n1.\tSome confusions.\nIn Parameter Transformation part, you state that “The number of adaptation parameters is given by k (2 d2 + d + 2). This is typically much smaller than the number of MDN parameters (weights and biases from all layers)”. \nIn previous part you state that “The MDN output with all the mixture parameters has dimension p = k (d(d + 1)/2 + d + 1).” \nWhy the adaptation parameters is much smaller than the number of MDN parameters?\n2.\tSome figures are not self-explanatory. For instance, in Figure 4, the line of No adapt or Finetune are covered by other lines, without additional explanation. \n3.\tMore experiments. How the unsupervised domain adaptation performs based on the baseline model and how it compares with the proposed approach?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2TsZcacoQn2", "reviewer": "Reviewer_91WK", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper models the changes of channel in a communication system as a few-shot domain adaptation problem. They employ the Gaussian mixture density network to specifically model the channel and propose a transformation to compensate for changes in the channel distribution. They perform experiments both on simulated channel distributions and FPGA.", "summary_of_the_review": "This paper aims to address the varying channels in communication with the help of few-shot DA. Their solution is simple and easy to follow, and the corresponding theoretical and empirical analysis are thorough. However, some experimental settings (see above) are need to be further optimized.", "strength_and_weaknesses": "Pros:\n\n1. This paper considers the frequent change of channel in communication systems. They treat the change of channel as distribution shift and link this practical problem with few-shot domain adaptation. I think this is novel and advanced enough in the field of communication.\n\n2. The proposed solution is easy to follow and can be used in more general few-shot DA scenarios. In addition, this method has better real-time performance compared with previous works.\n\nCons:\n\n1. This paper lacks the few-shot domain adaptation methods as baselines, e.g., [1]. Current baselines are all the basic FDA solutions, and I worry about their competitiveness. \n\n2. The evaluation metric they use is only the SER. As the application research article, the performance of the proposed method in practical communication problems is essential. They need to show the advantages of their learning-based method over conventional methods.\n\n3. I find that the number of target data per class is more than 10 in this paper, and maybe such an amount is beyond the scale of few-shot learning. Additional experiments with less than 7 samples per class are important.\n\n[1] Mottian et al. Few-Shot Adversarial Domain Adaptation. NeurIPS, 2017.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%