We propose novel model transfer-learning methods that refine a decision forest model @math learned within a “source” domain using a training set sampled from a “target” domain, assumed to be a variation of the source. We present two random forest transfer algorithms. The first algorithm searches greedily for locally optimal modifications of each tree structure by trying to locally expand or reduce the tree around individual nodes. The second algorithm does not modify structure, but only the parameter (thresholds) associated with decision nodes. We also propose to combine both methods by considering an ensemble that contains the union of the two forests. The proposed methods exhibit impressive experimental results over a range of problems.
Most early transfer learning methods were based on neural networks, and while SVMs and ensemble techniques have become prominent in this field, DT models are still under-explored in this setting. Of the few tree-based techniques researched, only the work by operates in our model transfer setting. Specifically, proposed a simple technique to update an existing tree trained only on source samples using target samples @cite_31 . Their approach resembled a batch iterative learning technique which relies solely on iterative expansion steps. This technique does not consider any refitting of numeric feature thresholds.
@cite_31
2095105645
This paper presents a novel technique for addressing domain adaptation (DA) problems with active learning (AL) in the classification of remote sensing images. DA models the important problem of adapting a supervised classifier trained on a given image (source domain) to the classification of another similar but not identical image (target domain) acquired on a different area. The main idea of the proposed approach is iteratively labeling and adding to the training set the minimum number of the most informative samples from the target domain, while removing the source-domain samples that do not fit with the distributions of the classes in the target domain. In this way, the classification system exploits already available information, i.e., the labeled samples of source domain, in order to minimize the number of target domain samples to be labeled, thus reducing the cost associated to the definition of the training set for the classification of the target domain. In addition, we define a convergence criterion that allows the technique to stop the iterative AL process on the target domain without relying on the availability of a test set for it. This is an important contribution, as in operational applications, it is not realistic to assume that a test set for the target domain is available. Experimental results obtained in the classification of very high resolution and hyperspectral images confirm the effectiveness of the proposed technique. Many machine learning models are vulnerable to adversarial examples: inputs that are specially crafted to cause a machine learning model to produce an incorrect output. Adversarial examples that affect one model often affect another model, even if the two models have different architectures or were trained on different training sets, so long as both models were trained to perform the same task. An attacker may therefore train their own substitute model, craft adversarial examples against the substitute, and transfer them to a victim model, with very little information about the victim. Recent work has further developed a technique that uses the victim model as an oracle to label a synthetic training set for the substitute, so the attacker need not even collect a training set to mount the attack. We extend these recent techniques using reservoir sampling to greatly enhance the efficiency of the training procedure for the substitute model. We introduce new transferability attacks between previously unexplored (substitute, victim) pairs of machine learning model classes, most notably SVMs and decision trees. We demonstrate our attacks on two commercial machine learning classification systems from Amazon (96.19 misclassification rate) and Google (88.94 ) using only 800 queries of the victim model, thereby showing that existing machine learning approaches are in general vulnerable to systematic black-box attacks regardless of their structure. Transfer learning is a vital technique that generalizes models trained for one setting or task to other settings or tasks. For example in speech recognition, an acoustic model trained for one language can be used to recognize speech in another language, with little or no re-training data. Transfer learning is closely related to multi-task learning (cross-lingual vs. multilingual), and is traditionally studied in the name of model adaptation'. Recent advance in deep learning shows that transfer learning becomes much easier and more effective with high-level abstract features learned by deep models, and the transfer' can be conducted not only between data distributions and data types, but also between model structures (e.g., shallow nets and deep nets) or even model types (e.g., Bayesian models and neural models). This review paper summarizes some recent prominent research towards this direction, particularly for speech and language processing. We also report some results from our group and highlight the potential of this very interesting research field. Most of the existing methods for pedestrian detection work well, only when the following assumption is satisfied: the features extracted from the training dataset and the testing dataset have very similar distributions in the feature space. However, in practice, this assumption does not hold because of the scene complexity and variation. In this paper, a new method is proposed for detecting pedestrians in various scenes based on the transfer learning technique. Our proposed method employs the following two strategies for improving the pedestrian detection performance. First, a new sample screening method based on manifold learning is proposed. The basic idea is to choose samples from the training set, which may be similar to the samples from the unseen scene, and then merge the selected samples into the unseen set. Second, a new classification model based on transfer learning is proposed. The advantage of the classification model is that only a small number of samples need to be used from the unseen scenes. Most of the training samples are still obtained from the training scene, which take up to 90 of the entire training samples. Compared to the traditional pedestrian detection methods, the proposed algorithm can adapt to different scenes for detecting pedestrians. Experiments on two pedestrian detection benchmark datasets, DC and NICTA, showed that the method can obtain better performance as compared to other previous methods.