GENERALISED LINEAR MODELS: INFERENCE AND ESTIMATION Toolbox version 1.5
for GNU Octave 3.3.x and Matlab 7.x
Copyright (c) by Hannes Nickisch, 2013-08-31.
0) HOW TO READ
==============
If you want to get started without further delay, then read section 1) below
and jump right to the examples in doc/index.html.
1) ABOUT THESE PROGRAMS
=======================
This collection of matlab programs implements and demonstrates estimation
and inference algorithms for generalised linear models.
While (MAP) estimation or equivalently penalised least squares estimation
is a standard tool in statics, inference is less widely used and in general
more tricky.
The two approximate inference algorithms are described in the papers
[1] Large Scale Variational Inference and Experimental Design for Sparse Generalized Linear Models
by Matthias Seeger and Hannes Nickisch
SIAM Journal on Imaging Sciences, 2011
http://arxiv.org/abs/0810.0901 for a similar TR
and
[2] Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior
by Gerven, Cseske, Lange and Heskes
Neuroimage 2010
http://homepages.inf.ed.ac.uk/bcseke/webpubs/gerven2010.pdf.
The Monte Carlo marginal variance estimator is detailed in
[3] Efficient Variational Inference in Large-Scale Bayesian Compressed Sensing
by George Papandreou and Alan Yuille
ICCV Workshop, 2011
http://www.stat.ucla.edu/~gpapan/pubs/confr/PapandreouYuille_VariationalBayesCompressedSensing_ieee-c-iccvw11.pdf
There are 6 code subdirectories: doc, inf, mat, pen, pls and pot.
doc/ contains an index.html file providing documentation. This information
is also available from http://www.kyb.tue.mpg.de/bs/people/hn/glm-ie.
Usage examples reaching from probabilistic classification, robust
regression over nonblind deconvolution to MRI reconstruction can be
found here. Further details are provided in the developer
documentation manual.pdf.
inf/ contains the double loop inference engine dli.m and code to compute
Gaussian marginal variances i.e. the diagonal of a covariance matrix
=> see infEngine.m
mat/ contains the lazy matrix class; mat/@mat is the base class all other
classes are derived from and the two auxiliary functions cx2re.m and
re2cx.m allow to convert between real and complex numbers
pen/ contains the penalty functions pen*.m
=> see penFunctions.m
pls/ contains the penalised least squares (pls) solvers pls*.m
=> see plsSolvers.m
pot/ contains the potential functions pot*.m
=> see potFunctions.m
Before running the demos or any other part of the code, you should execute
startup.m to add all necessary directories to your path. In Octave, you can also
use the .octaverc file.
2) ABOUT MEX FILES
==================
Some of the programs make use of the MEX facility in matlab for more efficient
implementation. Besides the L-BFGS minimiser, the wavelet transform mat/@matWav
rely on C/C++ code. The wavelet code should compile right after the first call;
if there are problems, see the help in mat/@matWav/private/fwtn.m.
If you do not have a compiler installed on your system, you might want to
consult [2].
In order to use the L-BFGS minimiser , you have to compile Peter Carbonetto's
"Matlab interface for L-BFGS-B" [1]. The challenge here is the Fortran 77 code.
We provide a Makefile suitable for Linux 32/64 bit and Mac whenever you have
1) g77
or
2) gfortran
properly set up.
Under Ubuntu, you can achieve this by installing the packages fort77 and
gfortran, respectively.
Compilation is done by editing util/lbfgsb/Makefile. In any case, you need to
provide $MATLAB_HOME which can be found by the commands 'locate matlab' or
'find / -name "matlab"'. You can choose between two compilation modes:
a) using the mex utility by Matlab [default]
provide $MEX, then type 'make mex'
b) without mex utility by Matlab
provide $MEX_SUFFIX and $MATLAB_LIB, then type 'make nomex'
In Ubuntu 10.04 LTS, the libg2c library needed for both 1)+a) and 1)+b) is not
included per default. If 'ls /usr/lib/libg2c.*' does not list anything
this is the case on your machine. You then whant to install the packages
gcc-3.4-base and libg2c0 e.g. from
http://packages.ubuntu.com/hardy/gcc-3.4-base and
http://packages.ubuntu.com/hardy/libg2c0.
After installation, you have to create a symbolic link by 'cd /usr/lib' and
'ln -s libg2c.so.0 libg2c.so'.
Compilation under Windows is rather tricky but there is a way [3] of doing it
as pointed out by Guillaume Jacquenot. Steps involved:
* Install MinGW (including gfortran compiler) at C:\mingw
* http://www.mingw.org/
* If the target was Matlab 64-bit, we would probably need MinGW-w64 instead?
* Download http://gnumex.sourceforge.net/, unzip at C:\gnumex, and create
Fortran/C mexopts.bat files for use with MinGW.
* Add C:\mingw\bin and C:\gnumex in the PATH and reboot.
* Create and run pls/make_lbfgsb_gnumex_windows.m script (replace the
filenames for the mexopts.bat files accordingly).
Other useful stuff:
* MS Dependency walker (to debug missing DLL dependencies), links
to download at http://www.dependencywalker.com
[1] http://www.cs.ubc.ca/~pcarbo/lbfgsb-for-matlab.html
[2] http://www.mathworks.com/support/compilers/R2010a
[3] http://www.cs.ubc.ca/~pcarbo/Compile_LBFGSB_on_Windows.txt
3) CURRENT VERSION
==================
The current version of the programs is 1.4. Previous versions of the code are
available at http://hannes.nickisch.org/code/glm-ie/release/oldcode.html.
4) DIFFERENCES TO PREVIOUS VERSIONS
===================================
NEW in version 1.5, 2013-08-31
------------------------------
added inf/diaginv_factorial.m to allow for mean field inference
more general non-Gaussian potentials
* defined by affine instead of linear functions of the latent variables
* pls solver's interface has been changed
* inf/dli engine's interface has been changed
NEW in version 1.4, 2011-10-19
------------------------------
contributed by George Papandreou:
- preconditioning support in the inf/linsolve_lcg.m CG routine.
- @matConv2 and @matFD2 support different boundary conditions in deblurring
- various mat/@*/diagFAtAFt.m support circulant preconditioning
- bugfixes in nonnegativity option in pls/plsLBFGS.m and pen/penVBNorm.m when used together with EP
- inf/diag_inv_sample.m, a Monte Carlo estimator
gfortran support to pls/lbfgsb/Makefile (thanks to Ernst Kloppenburg)
slight modification to mat/@matFFTN/mvm.m to make it more consistent
simple gradient solver using Barzilai-Borwein step size pls/plsBB.m
NEW in version 1.3, 2010-11-12
------------------------------
extension of the matrix class; bugfixes
plsSB - split Bregmin PLS solver
new inference engine dli.m instead of vbidl.m
- support of parallel EP in addition to VB
- marginal likelihood computation
extended documentation and demo programs
NEW in version 1.2, 2010-08-25
------------------------------
refactored the matrix classes
added more penalty functions
added non-linear potential support for group sparsity
improved PLS solvers
updated complex number treatment
NEW in version 1.1, 2010-08-10
------------------------------
added demo examples and documentation
NEW in version 1.0, 2010-07-05
------------------------------
initial version of the glm-ie toolbox