Ref HAL: hal-00519610_v1
Abstract: Nous montrons qu’un ensemble d’arbres de décision avec une composante aléatoire permet de construire un noyau efficace destiné à l’apprentissage supervisé. Nous étudions théoriquement les propriétés d’un tel noyau et montrons que sous des conditions très souvent rencontrées en pratique, il existe une séparabilité linéaire entre exemples de classes distinctes dans l’espace induit par celui-ci. Parallèlement, nous observons également que le classique vote à la majorité d’un ensemble d’arbres est un hyperplan (sans garantie d’optimalité) dans l’espace induit par le noyau. Enfin, comme le montrent nos expérimentations,l’utilisation conjointe d’un ensemble d’arbres et d’un séparateur à vaste marge (SVM) aboutit à des résultats extrêmement encourageants.

Comments: egc10pjz

DOI: 10.1007/978-3-642-15939-8_5
Abstract: Ensembles of randomized trees such as Random Forests are among the most popular tools used in machine learning and data mining. Such algorithms work by introducing randomness in the induction of several decision trees before employing a voting scheme to give a prediction for unseen instances. In this paper, randomized trees ensembles are studied in the point of view of the basis functions they induce. We point out a connection with kernel target alignment, a measure of kernel quality, which suggests that randomization is a way to obtain a high alignment, leading to possibly low generalization error. The connection also suggests to post-process ensembles with sophisticated linear separators such as Support Vector Machines (SVM). Interestingly, post-processing gives experimentally better performances than a classical majority voting. We finish by comparing those results to an approximate innite ensemble classier very similar to the one introduced by Lin and Li. This methodology also shows strong learning abilities, comparable to ensemble post-processing.

DOI: 10.1007/978-3-642-04125-9_45
Abstract: We extend the framework of spatial autocorrelation analysis on Reproducing Kernel Hilbert Space (RKHS). Our results are based on the fact that some geometrical neighborhood structures vary when samples are mapped into a RKHS, while other neighborhood structures do not. These results allow us to design a new measure for measuring the goodness of a kernel and more generally a similarity matrix. Experiments on UCI datasets show the relevance of our methodology.

Comments: lcns09pz

Abstract: We study the problem of learning automatically a kernel matrix specially designed for supervised leaning tasks. We start by introducing how to measure the quality of a kernel matrix through a geometrical statistical test (RCEW) and we present interesting connections between this last and the well known Kernel Target Alignment (KTA). We show that ensemble of decision trees can produce very accurate kernels because they naturally tend to maximize KTA and RCEW. Experiments based on some UCI datasets demonstrate excellent performances of SVM and RLS based on kernels constructed with trees ensembles. Our experiments are compared with ones based on the "multiple kernel learning" paradigm (MKL).