An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...
Public Member Functions | |
LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda) | |
Set the parameters to LocalCoordinateCoding. | |
arma::mat & | Codes () |
Modify the codes. | |
const arma::mat & | Codes () const |
Accessor the codes. | |
const arma::mat & | Data () const |
Access the data. | |
arma::mat & | Dictionary () |
Mutator for dictionary. | |
const arma::mat & | Dictionary () const |
Accessor for dictionary. | |
void | Encode (const size_t maxIterations=0, const double objTolerance=0.01) |
Run local coordinate coding. | |
double | Objective (arma::uvec adjacencies) const |
Compute objective function given the list of adjacencies. | |
void | OptimizeCode () |
Code each point via distance-weighted LARS. | |
void | OptimizeDictionary (arma::uvec adjacencies) |
Learn dictionary by solving linear system. | |
Private Attributes | |
size_t | atoms |
Number of atoms in dictionary. | |
arma::mat | codes |
Codes (columns are points). | |
const arma::mat & | data |
Data matrix (columns are points). | |
arma::mat | dictionary |
Dictionary (columns are atoms). | |
double | lambda |
l1 regularization term. |
An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.
Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2 + lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.
This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).
The papers are listed below.
@incollection{NIPS2009_0719,
title = {Nonlinear Learning using Local Coordinate Coding},
author = {Kai Yu and Tong Zhang and Yihong Gong},
booktitle = {Advances in Neural Information Processing Systems 22},
editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
and A. Culotta},
pages = {2223--2231},
year = {2009}
}
@article{efron2004least, title={Least angle regression}, author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.}, journal={The Annals of statistics}, volume={32}, number={2}, pages={407--499}, year={2004}, publisher={Institute of Mathematical Statistics} }
Definition at line 91 of file lcc.hpp.
mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::LocalCoordinateCoding | ( | const arma::mat & | data, | |
const size_t | atoms, | |||
const double | lambda | |||
) |
Set the parameters to LocalCoordinateCoding.
data | Data matrix. | |
atoms | Number of atoms in dictionary. | |
lambda | Regularization parameter for weighted l1-norm penalty. |
arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes | ( | ) | [inline] |
Modify the codes.
Definition at line 146 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes | ( | ) | const [inline] |
Accessor the codes.
Definition at line 144 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Data | ( | ) | const [inline] |
arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary | ( | ) | [inline] |
Mutator for dictionary.
Definition at line 141 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary | ( | ) | const [inline] |
Accessor for dictionary.
Definition at line 139 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Encode | ( | const size_t | maxIterations = 0 , |
|
const double | objTolerance = 0.01 | |||
) |
Run local coordinate coding.
nIterations | Maximum number of iterations to run algorithm. | |
objTolerance | Tolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates. |
double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Objective | ( | arma::uvec | adjacencies | ) | const |
Compute objective function given the list of adjacencies.
void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeCode | ( | ) |
Code each point via distance-weighted LARS.
void mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::OptimizeDictionary | ( | arma::uvec | adjacencies | ) |
Learn dictionary by solving linear system.
adjacencies | Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms) |
size_t mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::atoms [private] |
arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes [private] |
Codes (columns are points).
Definition at line 159 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes().
const arma::mat& mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::data [private] |
arma::mat mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary [private] |
Dictionary (columns are atoms).
Definition at line 156 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary().
double mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::lambda [private] |