moteur de recherche

Séminaire A3SI - François Malgouyres
Séminaire A3SI - François Malgouyres
24-May-2018 14:00
Age: 1 year

François Malgouyres (Institut de Mathématiques de Toulouse)

Multilinear compressive sensing and an application to convolutional linear networks

Seminar room B412 of the IMAGINE group (ENPC - Bat. Coriolis)

Abstract: We study a deep linear network expressed under the form of a matrix factorization problem. It takes as input a matrix X obtained by multiplying K matrices (called factors and corresponding to the action of a layer). Each factor is obtained by applying a fixed linear operator to a vector of parameters satisfying constraints. In machine learning, the error between the product of the estimated factors and X (i.e. the reconstruction error) relates to the statistical risk.

We first evaluate how the Segre embedding and its inverse distort distances. Then, we show that any deep matrix factorization can be cast in a generic multilinear problem (that uses the Segre embedding). We call this method tensorial lifting. Using the tensorial lifting, we provide necessary and sufficient conditions for the identifiability of the factors (up to a scale rearrangement). We also provide a necessary and sufficient condition called Deep Null Space Property (because of the analogy with the usual Null Space Property in the compressed sensing framework) which guarantees that even an inaccurate optimization algorithm for the factorization stably recovers the factors. In the machine learning context, the analysis provides sharp conditions on the network topology under which the error on the parameters defining the factors (i.e. the stability of the recovered parameters) scales linearly with the reconstruction error (i.e. the risk). Therefore, under these conditions on the network topology, any successful learning tasks leads to stably defined and therefore explainable layers.

We illustrate the theory with a practical example where the deep factorization is a convolutional linear network.

<- Back to: Accueil