In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
--------
Question: We conducted an experiment to determine whether we could maintain or improve classifier performances by applying the following three-tiered feature elimination approach:

Reduction We reduced the dataset encoded for each class by eliminating features that occur less than twice in the full dataset.

Selection We iteratively applied Chi-Square feature selection on the reduced dataset, selecting the top percentile of highest ranked features in increments of 5 percent to train and test the support vector model using a linear kernel and 5-fold, stratified cross-validation.

Rank We cumulatively plotted the average F1-score performances of each incrementally added percentile of top ranked features. We report the percentile and count of features resulting in the first occurrence of the highest average F1-score for each class.
Question: What are the three steps to feature elimination?

Answer: Reduction Selection Rank

Question: In the first one, shown in Figure FIGREF12, a special dedicated discriminator is added to the model to control that the latent representation does not contain stylistic information. The second extension of the baseline architecture does not use an adversarial component $D_z$ that is trying to eradicate information on $c$ from component $z$. Instead, the system, shown in Figure FIGREF16 feeds the "soft" generated sentence $ ilde{G}$ into encoder $E$ and checks how close is the representation $E( ilde{G} )$ to the original representation $z = E(x)$ in terms of the cosine distance. We further refer to it as shifted autoencoder or SAE. We also study a combination of both approaches described above, shown on Figure FIGREF17.
Question: What are three new proposed architectures?

Answer: special dedicated discriminator is added to the model to control that the latent representation does not contain stylistic information shifted autoencoder or SAE combination of both approaches

Question: The method is evaluated on the BERTbase model, which has 12 layers, 12 self-attention heads with a hidden size of 768.

To accelerate the training speed, two-phase training BIBREF1 is adopted. The first phase uses a maximal sentence length of 128, and 512 for the second phase. The numbers of training steps of two phases are 50K and 40K for the BERTBase model. We used AdamW BIBREF13 optimizer with a learning rate of 1e-4, a $eta _1$ of 0.9, a $eta _2$ of 0.999 and a L2 weight decay rate of $0.01$. The first 10% of the total steps are used for learning rate warming up, followed by the linear decay schema. We used a dropout probability of 0.1 on all layers. The data used for pre-training is the same as BERT, i.e., English Wikipedia (2500M words) and BookCorpus (800M words) BIBREF14.
Question: Do they train their model starting from a checkpoint?

Answer:
No