. . Functional learning . Reproducing kernel spaces. how to build them . Xavier Mary and Stéphane Canu [email protected] asi.insa-rouen.fr/~scanu . INSA Rouen -Département ASI Laboratoire PSI . . Functional learning .Reproducing kernel spaces.how to build them. – p.1/12 ' & #$ " tanh is NOT a positive kernel : but it works - : ) ) *, ) +* or norms are NOT hilbertian () % " tanh $# ! % ! Motivations : is NOT a norm What do we need to learn ? . . Functional learning .Reproducing kernel spaces.how to build them. – p.2/12 roadmap Discrimination par SVM 1 0.8 0.6 0.2 0.4 0 −0.2 how to choose ? learning in functional space Reproducing Kernel Hilbert Space −0.4 −0.6 −0.8 −1 0 0.5 1 1.5 2 2.5 3 3.5 4 Multiscale Approx on Frame 2 how to build kernels ? injective operators exemple 1.5 1 y 0.5 how the solution looks like ? representer theorem 0 −0.5 Original Data SVM approximation Multi resolution semi parametric −1 −1.5 0 1 2 3 4 5 x 6 7 8 9 10 . . Functional learning .Reproducing kernel spaces.how to build them. – p.3/12 roadmap Discrimination par SVM 1 0.8 0.6 0.2 0.4 0 −0.2 how to choose ? learning in functional space Reproducing Kernel Space −0.4 −0.6 −0.8 −1 0 0.5 1 1.5 2 2.5 3 3.5 4 Multiscale Approx on Frame 2 how to build kernels ? injective operators exemple 1.5 1 y 0.5 how the solution looks like ? representer theorem 0 −0.5 Original Data SVM approximation Multi resolution semi parametric −1 −1.5 0 1 2 3 4 5 x 6 7 8 9 10 . . Functional learning .Reproducing kernel spaces.how to build them. – p.3/12 interpretation : local measure of the hypothesis in close to / _ _ _/ Measure at close to the measure describes the function .................... continuous we can measure the function ............................................ this is not the case of / : _ _ _/ Measure at coucou Continuity is NOT a mater of inner product . . Functional learning .Reproducing kernel spaces.how to build them. – p.4/12 exists & is continuous : and exists : Evaluation space Hilbert space - inner product ..................................Duality measures ..... functions helps me to measure the AND measures we need two spaces : hypothesis How the information I have got at point quality of the hypothesis at point . . Functional learning .Reproducing kernel spaces.how to build them. – p.5/12 its topological dual - ) ) , ) ) its topological dual , - The duality map : - - - - Duality map : an example - - - DD DD DD DD ! Dual is seen as a functional space - HH HH HH HH H# . . Functional learning .Reproducing kernel spaces.how to build them. – p.6/12 * + «good » properties Kernel existence general case / DD DD DD DD D" II II II II I$ Hilbertian case measures ..... functions Generalization of reproducing kernels . . Functional learning .Reproducing kernel spaces.how to build them. – p.7/12 * * * / FF FF FF FF F" HH HH HH HH H$ are in duality but NOT in - and using injective operators , , ( - ( - let’s map them to - how to build duality kernels . . Functional learning .Reproducing kernel spaces.how to build them. – p.8/12 are in duality but NOT in - and using injective operators , , , and ( - ( - let’s map them to - how to build duality kernels . . Functional learning .Reproducing kernel spaces.how to build them. – p.8/12 are in duality but NOT in - and using injective operators , , , define the duality map : and ( - ( - let’s map them to - how to build duality kernels . . Functional learning .Reproducing kernel spaces.how to build them. – p.8/12 how to build duality kernels : an illustration , +* +* , , +* * ( - ( - injective operators on . . Functional learning .Reproducing kernel spaces.how to build them. – p.9/12 how to build duality kernels : an illustration , +* +* with respect to , , , ) ) +* * +* ) ) with respect to ( - *+ *+ ( - injective operators on . . Functional learning .Reproducing kernel spaces.how to build them. – p.9/12 how to build duality kernels : an illustration , +* +* *+ , , , tanh " " to get and " further work : find ! ) ) +* * +* ) ) with respect to ( - with respect to *+ ( - injective operators on . . Functional learning .Reproducing kernel spaces.how to build them. – p.9/12 Back to the Hilbertian case ..(measures are hypothesis ans R.K.H.S.) ..(only one function) - *+ *+ (only one operator) , *+ *+ +* +* , +* * *+ ( Carleman operator . . Functional learning .Reproducing kernel spaces.how to build them. – p.10/12 Back to the Hilbertian case ..(measures are hypothesis ans R.K.H.S.) ..(only one function) - *+ *+ (only one operator) , *+ *+ +* ............................................................ to define you have to know with you define +* , +* * *+ ( Carleman operator . . Functional learning .Reproducing kernel spaces.how to build them. – p.10/12 Back to the Hilbertian case ..(measures are hypothesis ans R.K.H.S.) ..(only one function) - *+ *+ (only one operator) , *+ *+ +* : there exits a countable basis such that is separable so is - ............................................................ to define you have to know with you define +* , +* * *+ ( Carleman operator . . Functional learning .Reproducing kernel spaces.how to build them. – p.10/12 The representer theorem +* Assume " " *+ is a subduality of with kernel convex and differentiable ... denotes its Gateau derivative) such that . . Functional learning .Reproducing kernel spaces.how to build them. – p.11/12 The representer theorem +* Assume " " and (thanks to the convexity of ) Then for *+ is a subduality of with kernel convex and differentiable ... denotes its Gateau derivative) such that . . Functional learning .Reproducing kernel spaces.how to build them. – p.11/12 The representer theorem +* Assume " " and ) (thanks to the convexity of " " " Then for *+ is a subduality of with kernel convex and differentiable ... denotes its Gateau derivative) such that . . Functional learning .Reproducing kernel spaces.how to build them. – p.11/12 The representer theorem +* Assume " " and ) null space of " " " " (thanks to the convexity of Then for *+ is a subduality of with kernel convex and differentiable ... denotes its Gateau derivative) such that . . Functional learning .Reproducing kernel spaces.how to build them. – p.11/12 Conclusion because we want you can learn without kernel but there is one ! and the continuity of the evaluation functional a framework generalizing R.K.H.S to non hilbertian spaces build kernels thanks to simple operators and the regularizer " " " " we know the shape of the solution gives you the kernel How to learn the coefficients ? How to determine the cost function ? . . Functional learning .Reproducing kernel spaces.how to build them. – p.12/12

© Copyright 2019