Paper: Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions (ICLR-2024)

Reviewer comment (Theory):
In equations (4), (5) and (6), the notations of expectation and variance do not follow the convention of probability theory. The reviewer fails to figure out what the expectation is taken of, and with respect to what. Even when the reviewer turns to the appendix, it is difficult to understand because $\theta$ and $\omega$ is not clearly defined. The reviewer understands it as the follows: $\omega$ is the model parameter, like the weight of a neural network? $\theta$ corresponds to the mean, $\mu$, and variance, \sigma, when the variational distribution takes Gaussian distribution form. But if it is the case, by variational inference, $\theta$ is a function of $\omega$ determined by the model (e.g., a neural network); then, how to understand $q(\omega|\theta_i)$. Maybe when your model (the neural network) is not a deterministic model, meaning, both $\omega$ and $\theta$ are random, the above seems right.

Author response:
Thank you for your helpful question. We will correct the notations. $E[q(\omega|\theta_i)]$ means $E_{q(\omega|\theta_i)}[\omega]$, and $VAR[q(\omega|\theta_i)]$ means $VAR_{q(\omega|\theta_i)}[\omega]$. If we use Gaussian distribution as the variational distribution on each domain, then $E[q(\omega|\theta_i)]=\mu_i$,$VAR[q(\omega|\theta_i)]=\sigma^2_i$. However, if the variational distribution is other distributions, Equation 4,5 still hold. $q(\omega|\theta_i)$ means the variational distribution, where $\theta_i$ is the parameter of this distribution. For Gaussian variational inference, $\theta_i=(\mu_i,\sigma^2_i)$ and $q(\omega|\theta_i) = \mathcal{N}(\mu_i,\sigma^2_i)$ (or $\mathcal{N}(\omega;\mu_i,\sigma^2_i)$ for better understanding). $\theta_i$ is deterministic, $\omega$ is random.

Reviewer comment (Experiments):
The proposed algorithm needs to train a featurizer for each domain. It is computationally expensive.

Author response:
Thank you for your helpful question. The computational costs are close to ERM, given that **PTG only need a few more iterations** (50 iterations, about 1.4 epochs) to aggregate the learned posteriors. **Despite training three models, each model is trained on one domain representing 1/3 of the entire training data, so the overall computationally expensive is still same as ERM**. The aggregation computations also have close forms. The training duration for ERM-Bayesian is approximately 44 minutes, followed by PTG training which spans approximately 20 minutes.

However, the GPU memory cost is expensive because PTG has to load the parameters of all models for training and computing the aggregated posteriors. We have addressed this phenomenon in Section 5. Simplified structure, parallel computation or simply making aggregations by CPU may be solutions, but they are future works.

Reviewer comment (Theory):
Although the idea is interesting, it is still unclear why directly inferring the domain invariant model posteriors is better than feature invariant learning. The authors argue that the proposed method can extract more invariant information from features but the reason is also not clear.

Author response:
Thank you for your insightful question. From Bayesian view, **our aim is to learn $p(\omega|\mathcal{D}^c)$, which means the posterior parameter distributions that use all domain invariant information for prediction.** There may be a misunderstanding that by "domain invariant posterior", we refer to $p(\omega|\mathcal{D}^c)$ rather than parameters that vary little among training domains. We made this description in Section 1 paragraph 4, and we will improve our expression for better clarity. We explain in Section 3.5 about the relationship between domain invariant parameters $p(\omega|\mathcal{D}^c)$ and its variation rate.

We also explain why **domain specific features may still contain domain invariant information** in Section 3.5. In Fig 2, there are stripes on feature maps. These strips are domain specific features because one is straight and one is wavy. If we use MSE to evaluate the distance between these features, they are far away (only few points overlap). We show by this toy model that this information can be extracted by parameters, but if we only want to extract domain invariant features, this information will be ignored. Fig 2 is a toy model that, for easy understanding, we assume the domain information exists in the feature map. However, we don't know exactly how the information is represented or how to get it, and we don't care. We only assume that $\mathcal{D}^c$ and $\mathcal{D}^v$ exist and are independent, and the domain distribution $p(\mathcal{D})$ can somehow be represented by them. One of our contributions is that we can estimate $p(\omega|\mathcal{D}^c)$ without access to $\mathcal{D}^c$.

Reviewer comment (Reproducibility):
Before pretrained on the source domains by ERM, is the BNN parameters initialized randomly or by ImageNet pretrained parameters?

Author response:
Thank you for your helpful question. We initialize a BNN by a ResNet pretrained on ImageNet, which is the common choice in Domain Generalization. The method to initialize a BNN by DNN is adopted from [1]

    [1]Krishnan R, Subedar M, Tickoo O. Specifying weight priors in bayesian deep neural networks with empirical bayes[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(04): 4477-4484.

Reviewer comment (Reproducibility):
Once obtained the mean and variance of the invariant posterior f_0, how to sample the parameters? By Monte-Carlo sampling?

Author response:
Thank you for your careful reading and question. We use Monte-Carlo sampling to sample from variational distributions. This is a common operation in variational inference BNN.[1]

    [1]Blundell C, Cornebise J, Kavukcuoglu, et al. Weight uncertainty in neural network[C]//International conference on machine learning. PMLR, 2015: 1613-1622.

Reviewer comment (Theory):
In the algorithm, it is found that in each training iteration, the source-specific posterior f_i is updated first, then the method aggregates them as the invariant posteriors and trains it together with the classifiers. However, in the next iterations, the f_i is updated again without considering any information in f_0, and f_0 is updated according to the new f_i, also without the f_0 in the previous iterations. If it is, why need to update f_0 by the cross-entropy loss functions?

Author response:
Thank you for your insightful question. First, we didn't update classifier on each domain, and after the change of $f_0$, we have to update classifier to adapt it to $f_0$. Second, in theory PTG don't need to update $f_0$ after the aggregation. However, we are afraid that after some iterations, $f_i$ may get diversified too much that most posteriors in $f_0$ may have too large variance. We are not sure that whether a good generalization ability can be achieved in this stage, but too large variance may make a BNN divergent. As a result, we further update $f_0$ by just one iteration, so $f_0$ will not change a lot, and it can still extract meaningful features.

Reviewer comment (Theory):
There is no need to present p(w|D^{c}) = E_{p(D^{v})}(p(w|D^{c}, D^{v})) as a theorem; it is a standard statistical technique known as marginalization. Routine operations in statistics, like this, do not necessitate proof.

Author response:
Thank you for your helpful suggestion. **Maybe there is a misunderstanding between $\int P(X|Y=y)P(Y=y)dy$ and $\int P(X|Y=y,Z)P(Y=y)dy$. **Marginalization usually means the process that requires summing over the possible values of one variable to determine the marginal contribution of another, to be specific, $P(X)=\int P(X,Y=y)dy=\int P(X|Y=y)P(Y=y)dy$. However, **our theorem, $p(\omega|\mathcal{D}^c)=\int p(\mathcal{D}^v)p(\omega|\mathcal{D}^c,\mathcal{D}^v)d\mathcal{D}^v$, follows another Bayesian principle.** The proof of Theorem 3.1 is shown in appendix B. Although $p(\omega|\mathcal{D}^c)=\mathbb{E}_{p(\mathcal{D}^v)}[p(\omega|\mathcal{D}^c,\mathcal{D}^v)]$ appears to be true, it only holds under the condition that $\mathcal{D}^c$ and $\mathcal{D}^v$ are independent. Besides, Theorem 3.1 shows that we can estimate $p(\omega|\mathcal{D}^c)$ by the posterior given full information, which plays a vital role in the following context.

Reviewer comment (Novelty):
I could not identify any specific contributions made by the authors in this paper. It reads more like a review of Bayesian methods in domain generalization. The claimed theorem appears to be about the marginalization of conditional distributions. I am interested in the authors' elaboration on this during the rebuttal.

Author response:
Thank you for your insightful question. One of our contributions is that **we are the first one to align parameter posteriors for domain generalization.** As introduced in Section 1 paragraph 3 and Section 2.3, all former works use BNN to show the distribution of latent features, and they align the feature distributions similar as other Non-Bayesian ways. We find that the alignment of parameter distributions can also bring improvements. Besides, **we propose a method to estimate the posterior given domain invariant information $p(\omega|\mathcal{D}^c)$,** which can achieve the best domain generalization ability in theory. There's a problem that $\mathcal{D}^c$ is unknown, but we can still estimate $p(\omega|\mathcal{D}^c)$ without access to $\mathcal{D}^c$. In addition, although we propose the way to estimate $p(\omega|\mathcal{D}^c)$, **we solve some difficulties in implementation**, such as disordered dimensions of parameters. We explained these difficulties in section 3.3 and 3.4. We propose two simple yet effective implementations to solve these problems, and experiments show their superiority. In conclusion, **we identify a new research direction to Domain Generalization**, the analysis of parameter posterior distributions, and we propose the basic framework for it.

Reviewer comment (Theory):
I'm sensing a concern about the assumption that \mathcal{D}_{c} ​ remains constant across the entire dataset \mathcal{D}, especially given the notation \mathcal{D}_{i}^{N} ​ mentioned by the authors. It seems the authors are questioning whether there can be a subset of data within one dataset that is fully domain-specific or domain-invariant, as opposed to only certain features (latent encoded outputs) being domain-invariant. Consequently, I find the motivation behind the proposed work to be lacking. Let us break it down. The authors seem to suggest that \mathcal{D}_{c} remains consistent throughout \mathcal{D}, where \mathcal{D}_{i}^{N} ​ represents subsets of data within the dataset. The authors point out that there is a subset of samples that are domain-specific or invariant rather than there is part of feature elements within entire data samples that are domain-specific or invariant (e.g., In the conventional understanding, we acknowledge that images of dogs often exhibit diverse background information, which can be seen as domain-specific. Simultaneously, the outlines or features of dogs themselves are considered domain-invariant. However, the authors introduce the idea that within the broader category of dogs, there should be a subset of images that are specifically tied to a domain or completely domain-invariant. This assertion does not seem to align with the typical variability observed in dog images, and I find it challenging to envision such a subset existing within this category). This leads to a potential misalignment in the assumption. If the proposed work relies on the idea that certain data samples are entirely domain-specific within a dataset, it might not be well-grounded. It might be more accurate to frame the motivation around features being domain-invariant rather than assuming entire data samples share this characteristic. In essence, it appears there's room for clarification or adjustment in the assumption to better reflect the reality of domain invariance within datasets. This refinement would strengthen the motivation behind the proposed approach.

Author response:
Thank you for your insightful questions. We are sorry that we caused some misunderstanding.

**The key misunderstanding is, $\mathcal{D}^c$ and $\mathcal{D}^v$ are not dataset! The second misunderstanding is, it is $p(\mathcal{D}^c)$ that remains constant, rather than $\mathcal{D}^c$.**

**$\mathcal{D}^c$ and $\mathcal{D}^v$ are two abstract concepts**: $\mathcal{D}^c$ means the collection of all domain invariant information from a domain $\mathcal{D}$, and $\mathcal{D}^v$ is the opposite. $\mathcal{D}$ refers to a domain rather than a dataset, $p(\mathcal{D}^v)$ is its distribution, and the corresponding dataset are samples from $p(\mathcal{D}^v)$. We use $\mathcal{D}_i$ to distinguish between different domains. We assume that $\mathcal{D}^c$ and $\mathcal{D}^v$ are two independent sufficient statistics of $\mathcal{D}$. To be specific, there exist a bijection between $p(\mathcal{D})$ and $p(\mathcal{D}^c)p(\mathcal{D}^v)$, and $p(\mathcal{D}^c)$ and $p(\mathcal{D}^v)$ can be inferred by the dataset of $\mathcal{D}$. However, we don't care about (1) the specific algebra expression of $\mathcal{D}^c$, (2) the specific bijection formula, (3) the specific formula to estimate $p(\mathcal{D}^c)$ from dataset. One of our contributions is to estimate $p(\omega|\mathcal{D}^c)$ without access to $\mathcal{D}^c$.

In your example, $\mathcal{D}$ means a domain of dogs, and $\mathcal{D}_i$ may refer to a specific kind of dog, like golden retriever. $\mathcal{D}^c$ means the common information that all dogs share, such as the overall body shape, the facial patterns, the teeth and so on. $\mathcal{D}^v_i$ means the specific information that golden retriever have, such as the color, the hair length. As stated before, we don't care how the information is represented or how to get it, we just assume that it exists. We can directly estimate the posterior parameter distributions that use the common information for recognition. This is why we claim that our assumption is more relaxed than domain invariant features.

We explained our idea in Section 1 paragraph 5, Section 3.1 and Appendix A. Thanks again for your helpful question, we will add these discussions and your example in the paper.

Reviewer comment (Experiments):
The literature compared in this study appears to be somewhat outdated, with the highlighted method, SOTA CORAL, dating back to 2017. Given that the proposed method relies on an existing domain generalization (DG) method for initialization, it becomes crucial to benchmark against more recent approaches. It would be particularly insightful to demonstrate how the proposed method not only aligns with but also enhances the state-of-the-art frameworks in domain generalization. Please consider conducting comparisons with the latest methods to provide a more comprehensive evaluation of your proposed approach in the context of contemporary DG frameworks.

Author response:
Thank you for your valuable suggestions. Our comparative analysis remains contemporary considering that more than half of the models(8/14) are introduced after 2021, and the rest are classical counterparts. We selected our competitors for a fair comparison. Model selection plays a vital role in domain generalization, so **we only compare with models available on DomainBed**, which is a fair test platform that automatically select models. **We did not compare PTG with many other SOTA methods, because they either use CLIP pretrained model, SWAD weight averaging strategy, large-scale ensemble learning or other strategies.**

We are sorry that we can't combine PTG with EoA, which is the best model in the benchmark NICO++. Because EoA is an ensemble model, while PTG requires a single model as its prior. However, we want to mention that although CORAL is old, it still gets the third performance in NICO++ and there is only a small distance.(EoA 78.88, MIRO 78.65, CORAL 78.38). As you suggested, we provide the result of the best two models on NICO++, but please note we still give results on DomainBed, and all models use ResNet50.  The results of MIRO and EoA are collected form [1]. Please refer to Section 4.1 paragraph 3 and Appendix G for more details.

| Algorithm | PACS | VLCS | Office-Home | TerraIncognita | Avg |
| :-------: | :--: | :--: | :---------: | :------------: | :--: |
|    PTG    | 86.7 | 79.4 |    69.4    |      48.5      | 71.0 |
|   CORAL   | 86.2 | 78.8 |    68.7    |      47.6      | 70.3 |
|   MIRO   | 85.4 | 79.0 |    70.5    |      50.4      | 71.3 |
|    EoA    | 88.6 | 79.1 |    72.5    |      52.3      | 73.1 |

We admit that EoA is better than PTG, but we want to remind again that EoA is an ensemble method while PTG just has one model in the testing stage. We already provide more combinations in Appendix H, and we are sorry that we can't provide the combination with MIRO due to the time limit.

    [1] Arpit D, Wang H, Zhou Y, et al. Ensemble of averages: Improving model selection and boosting performance in domain generalization[J]. Advances in Neural Information Processing Systems, 2022, 35: 8265-8277.

Reviewer comment (Experiments):
The authors should perform an ablation study to assess the performance of the proposed PTG without relying on other domain generalization (DG) methods for initialization. For instance, evaluating PTG using only the pre-trained ResNet-18 as the initialization for model training on each domain would provide insights into its standalone effectiveness.

Author response:
Thank you for your careful reading. We will modify these problems. The result of PTG on ResNet 18 without prior network is:

| Prior | PACS | VLCS | Office-Home | TerraIncognita | Avg |
| :---: | :--: | :--: | :---------: | :------------: | :--: |
|  w/  | 83.7 | 76.1 |    61.6    |      44.7      | 66.5 |
|  w/o  | 79.9 | 74.5 |    56.3    |      38.2      | 62.2 |

Reviewer comment (Reproducibility):
The authors in section 3.3 suggest initializing the BNN on each domain with a uniformly well-generalized model, such as a BNN trained using Empirical Risk Minimization (ERM). The confusion arises regarding the connection between initializing the model on each domain and selecting the best model through ERM. To clarify, the authors advocate training a BNN using ERM on a specific dataset before initializing it on different domains. The dataset used for this ERM training is not explicitly mentioned in this section, but it is the dataset that provides the well-generalized model used as the starting point for initialization across various domains. In essence, the ERM-trained BNN serves as a kind of starting point. It is trained on a particular dataset that captures general patterns. My understanding is that this well-generalized model is then employed as the initial configuration for the BNN when dealing with different domains. The emphasis is on using a model that has already demonstrated good generalization capabilities through ERM, ensuring a solid starting point for adaptation to diverse domains.

Author response:
Thank you for your insightful question. **We train the initial network by ERM just on all the given training domains. There's no additional training dataset.** The ERM initialization process is part of our method, and since ERM introduces no generalization techniques, we think the improvements comes from our work.

Reviewer comment (Writing):
The terminologies used in the paper are confusing. In the introduction, the authors equate domain-invariant features to domain-invariant parameters. In typical deep-learning contexts, parameters refer to learnable model weights and biases, while features denote the encoder's output. To prevent potential misinterpretations, it would be helpful if the authors could offer clear definitions for parameters and features within the scope of their work.

Author response:
We are sorry that we caused this confusion. Feature and parameters are just common defined. The domain invariant features are also same as common works. **Domain invariant parameters (domain invariant posteriors) denote the posterior distribution of parameters given domain invariant information $p(\omega|\mathcal{D}^c)$.** We give the definitions in Section 1 paragraph 4 and section 3.1, and we will add the definition by definition formula.

Reviewer comment (Reproducibility):
Could you provide clarity on the definition of the prior model in the proposed work? Is it initialized from the pretraining of ResNet, or is it initialized from the pretraining of another domain generalization (DG) method? Please elaborate on this aspect.

Author response:
Thank you for your helpful question. **There are two initialization procedure.** First, we initialize a BNN by a ResNet pretrained on ImageNet, which is the common choice in Domain Generalization. The method to initialize a BNN by DNN is adopted from [1]. Second, after the ERM training, we initialize the BNN on each domain by the ERM trained BNN. We are sorry that we didn't draw a clear distinction between these prior models. We will modify the expression.-

    [1]Specifying weight priors\mu in bayesian deep neural networks with empirical bayes.

Reviewer comment (Reproducibility):
In Algorithm 1, when the authors mention updating the model parameters with a Gaussian distribution, are they referring to sampling the model parameters from the Gaussian distribution outlined in Equation (4)? If so, could you provide clarification on the method used for this sampling process? Is it achieved through random sampling? Please provide additional details.

Author response:
There seems to by a misunderstanding of BNN. $\omega$ usually means the learnable parameters in DNN. However, in variational BNN, $\omega$ is the notation of the random-variable-sized parameter, $q(\omega|\theta)$ is the variational distribution of parameter, and $\theta$ is the true learnable object. For example, if we use Gaussian distribution as the variational distribution, then $\omega$ follows $\mathcal{N}(\mu,\sigma^2)$, and $\mu,\sigma^2$ are the true learnable variable. During training, we only learn $\mu,\sigma^2$ by Equation 4 and 5. During testing, we sample $\omega$ from $\mathcal{N}(\mu,\sigma^2)$ for 5 times to form 5 different models, and use the average of their predictions as the final prediction.

Reviewer comment (Theory):
In Figure 2, the authors mention extracting domain-invariant information from domain-specific features (Z^V). Given that these features reside in the encoded space (the output space of ResNet-18) if the training of ResNet-18 is indeed effective in extracting domain-invariant features, it raises a question: How can domain-invariant information also be present within features specifically identified as domain-specific? The term "information" is used without a clear definition from the authors. This ambiguity, coupled with an unclear definition of \mathcal{D}, restricts readers from fully grasping the paper's content. Could the authors provide more clarity on interpreting "information" and a more explicit definition of \mathcal{D}? My understanding is that the authors are trying to claim that the feature map might not effectively capture the domain-invariant information from the data. If my understanding is correct, then the claim is very ambiguous. The authors should at least provide some ways to quantify or visualize it. In the current version of the paper, this claim is not supported theoretically or empirically.

Author response:
Thank you for your insightful question. In Fig 2, there are stripes on feature maps. These strips are domain specific features because one is straight and one is wavy. If we use MSE to evaluate the distance between these features, they are far away (only few points overlap). We show by this toy model that this information can be extracted by parameters, but if we only want to extract domain invariant features, this information will be ignored. Fig 2 is a toy model that, for easy understanding, we assume the domain information exists in the feature map. However, we don't know exactly how the information is represented or how to get it, and we don't care. We only assume that $\mathcal{D}^c$ and $\mathcal{D}^v$ exist and are independent, and the domain distribution $p(\mathcal{D})$ can somehow be represented by them. One of our contributions is that we can estimate $p(\omega|\mathcal{D}^c)$ without access to $\mathcal{D}^c$.

Reviewer comment (Theory):
The paper is not very sound as it attempts to consider learning domain invariants using the Bayesian framework on the parameter space. This is different from the existing work where the feature space is considered. However, this may lead to challenging problems as the parameters of neural networks are extremely high-dimensional and unidentifiable. It is extremely difficult to align and aggregate the posteriors in the parameter space. Therefore, the authors propose some ad-hoc solutions in Section 3.3 such as neural networks for different domains should be initialized from the same point, the parameters of the last layer should be shared between neural networks, and the learning rate should be carefully decayed. These solutions are not well-motivated and do not guarantee that the posteriors of parameters are well-aligned and aggregated properly.

Author response:
Our major contribution is that we propose a method to estimate $p(\omega|\mathcal{D}^c)$ without $\mathcal{D}^c$. The analysis of parameter distributions has been widely studied so far, and variational inference is a simple way to align parameters. Besides Bayesian theorem, we also solve problems that may occur during implementation. Furthermore, our experiment can prove that our effort works.

Reviewer comment (Novelty):
The authors propose PTG-Lite, which is a lite version of the main method. Due to the difficulty of employing Bayesian treatment to the neural network’s parameters, PTG-Lite just uses maximum-a-posteriori MAP solutions. We can see from Table 1 that PTG-Lite performs comparably with PTG. This somehow downgrades the main narrative of the paper which is to propose a Bayesian framework for aggregating posteriors on domains and inferring domain invariant posteriors.

Author response:
Although PTG-Lite is not BNN, its principle is still Bayesian. It is Theorem 3.1, which shows how to get the domain invariant posterior $p(\omega|\mathcal{D}^c)$ with only access to $\mathcal{D}$, that matters, rather than way to approximate $p(\omega|\mathcal{D}^c)$. PTG-Lite uses deterministic values to approximate $p(\omega|\mathcal{D}^c)$, but the key idea is still to estimate the invariant posterior.

Reviewer comment (Theory):
One of the main contributions is the Theorem 3.1. However, it is trivial. Although the authors claim in the abstract that this theorem relies on relaxed assumptions. However, the assumption of full independence between the domain invariant and domain-specific information is quite strong.

Author response:
We explained why $\mathcal{D}^c$ and $\mathcal{D}^v$ should be independent in Appendix A. The proof of Theorem 3.1 is shown in Appendix B. Although Theorem 3.1 appears to be always true, in fact it holds under the condition that $\mathcal{D}^c$ and $\mathcal{D}^v$ are independent. Besides, Theorem 3.1 shows that we can estimate $p(\omega|\mathcal{D}^c)$ by the posterior given full information, which plays a vital role in the following context.

Reviewer comment (Experiments):
The authors should experimentally compare with other approaches using Bayesian neural networks such as Xiao et al. (2021) and follow-up works.

Author response:
We didn't report the mentioned approaches because they are not appliable on DomainBed. For fair comparison, we only compare with models on DomainBed.

Reviewer comment (Writing):
The writing should be improved. There are some citation and grammar typos and sentences that need further clarification. For example, in the second row from the bottom of page 3, what are the parameters?

Author response:
We will correct typos and improve the writing.

Reviewer comment (Theory):
In Section 3.1, the paper claims that “we do not need to specify how $\mathcal{D}^{c}$ and $\mathcal{D}^{v}$ are extracted from $\mathcal{D}$”. Is this useful as it would be important to learn which information from the data is invariant or not? Is it possible to interpret the inferred domain-invariant posterior?

Author response:
We don't need to specify which information from the data is invariant. In BNN, we care how to estimate $p(\omega|\mathcal{D})$ rather than what's the data distribution $p(\mathcal{D})$. Similarly, we care how to estimate $p(\omega|\mathcal{D}^c)$, rather than $p(\mathcal{D}^c)$. $\mathcal{D}^c$ means any information that keeps invariant between domains, and $p(\omega|\mathcal{D}^c)$ is parameter posterior distribution that use $\mathcal{D}^c$ for prediction.

Reviewer comment (Writing):
In Equations (5) and (7), what are the expectations with respect to?

Author response:
These are typos. $E[q(\omega|\theta_i)]$ means $E_{q(\omega|\theta_i)}[\omega]$, and $VAR[q(\omega|\theta_i)]$ means $VAR_{q(\omega|\theta_i)}[\omega]$.