GENERALISED LINEAR MODELS: INFERENCE AND ESTIMATION Toolbox version 1.5 for GNU Octave 3.3.x and Matlab 7.x Copyright (c) by Hannes Nickisch, 2013-08-31. 0) HOW TO READ ============== If you want to get started without further delay, then read section 1) below and jump right to the examples in doc/index.html. 1) ABOUT THESE PROGRAMS ======================= This collection of matlab programs implements and demonstrates estimation and inference algorithms for generalised linear models. While (MAP) estimation or equivalently penalised least squares estimation is a standard tool in statics, inference is less widely used and in general more tricky. The two approximate inference algorithms are described in the papers [1] Large Scale Variational Inference and Experimental Design for Sparse Generalized Linear Models by Matthias Seeger and Hannes Nickisch SIAM Journal on Imaging Sciences, 2011 http://arxiv.org/abs/0810.0901 for a similar TR and [2] Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior by Gerven, Cseske, Lange and Heskes Neuroimage 2010 http://homepages.inf.ed.ac.uk/bcseke/webpubs/gerven2010.pdf. The Monte Carlo marginal variance estimator is detailed in [3] Efficient Variational Inference in Large-Scale Bayesian Compressed Sensing by George Papandreou and Alan Yuille ICCV Workshop, 2011 http://www.stat.ucla.edu/~gpapan/pubs/confr/PapandreouYuille_VariationalBayesCompressedSensing_ieee-c-iccvw11.pdf There are 6 code subdirectories: doc, inf, mat, pen, pls and pot. doc/ contains an index.html file providing documentation. This information is also available from http://www.kyb.tue.mpg.de/bs/people/hn/glm-ie. Usage examples reaching from probabilistic classification, robust regression over nonblind deconvolution to MRI reconstruction can be found here. Further details are provided in the developer documentation manual.pdf. inf/ contains the double loop inference engine dli.m and code to compute Gaussian marginal variances i.e. the diagonal of a covariance matrix => see infEngine.m mat/ contains the lazy matrix class; mat/@mat is the base class all other classes are derived from and the two auxiliary functions cx2re.m and re2cx.m allow to convert between real and complex numbers pen/ contains the penalty functions pen*.m => see penFunctions.m pls/ contains the penalised least squares (pls) solvers pls*.m => see plsSolvers.m pot/ contains the potential functions pot*.m => see potFunctions.m Before running the demos or any other part of the code, you should execute startup.m to add all necessary directories to your path. In Octave, you can also use the .octaverc file. 2) ABOUT MEX FILES ================== Some of the programs make use of the MEX facility in matlab for more efficient implementation. Besides the L-BFGS minimiser, the wavelet transform mat/@matWav rely on C/C++ code. The wavelet code should compile right after the first call; if there are problems, see the help in mat/@matWav/private/fwtn.m. If you do not have a compiler installed on your system, you might want to consult [2]. In order to use the L-BFGS minimiser , you have to compile Peter Carbonetto's "Matlab interface for L-BFGS-B" [1]. The challenge here is the Fortran 77 code. We provide a Makefile suitable for Linux 32/64 bit and Mac whenever you have 1) g77 or 2) gfortran properly set up. Under Ubuntu, you can achieve this by installing the packages fort77 and gfortran, respectively. Compilation is done by editing util/lbfgsb/Makefile. In any case, you need to provide $MATLAB_HOME which can be found by the commands 'locate matlab' or 'find / -name "matlab"'. You can choose between two compilation modes: a) using the mex utility by Matlab [default] provide $MEX, then type 'make mex' b) without mex utility by Matlab provide $MEX_SUFFIX and $MATLAB_LIB, then type 'make nomex' In Ubuntu 10.04 LTS, the libg2c library needed for both 1)+a) and 1)+b) is not included per default. If 'ls /usr/lib/libg2c.*' does not list anything this is the case on your machine. You then whant to install the packages gcc-3.4-base and libg2c0 e.g. from http://packages.ubuntu.com/hardy/gcc-3.4-base and http://packages.ubuntu.com/hardy/libg2c0. After installation, you have to create a symbolic link by 'cd /usr/lib' and 'ln -s libg2c.so.0 libg2c.so'. Compilation under Windows is rather tricky but there is a way [3] of doing it as pointed out by Guillaume Jacquenot. Steps involved: * Install MinGW (including gfortran compiler) at C:\mingw * http://www.mingw.org/ * If the target was Matlab 64-bit, we would probably need MinGW-w64 instead? * Download http://gnumex.sourceforge.net/, unzip at C:\gnumex, and create Fortran/C mexopts.bat files for use with MinGW. * Add C:\mingw\bin and C:\gnumex in the PATH and reboot. * Create and run pls/make_lbfgsb_gnumex_windows.m script (replace the filenames for the mexopts.bat files accordingly). Other useful stuff: * MS Dependency walker (to debug missing DLL dependencies), links to download at http://www.dependencywalker.com [1] http://www.cs.ubc.ca/~pcarbo/lbfgsb-for-matlab.html [2] http://www.mathworks.com/support/compilers/R2010a [3] http://www.cs.ubc.ca/~pcarbo/Compile_LBFGSB_on_Windows.txt 3) CURRENT VERSION ================== The current version of the programs is 1.4. Previous versions of the code are available at http://hannes.nickisch.org/code/glm-ie/release/oldcode.html. 4) DIFFERENCES TO PREVIOUS VERSIONS =================================== NEW in version 1.5, 2013-08-31 ------------------------------ added inf/diaginv_factorial.m to allow for mean field inference more general non-Gaussian potentials * defined by affine instead of linear functions of the latent variables * pls solver's interface has been changed * inf/dli engine's interface has been changed NEW in version 1.4, 2011-10-19 ------------------------------ contributed by George Papandreou: - preconditioning support in the inf/linsolve_lcg.m CG routine. - @matConv2 and @matFD2 support different boundary conditions in deblurring - various mat/@*/diagFAtAFt.m support circulant preconditioning - bugfixes in nonnegativity option in pls/plsLBFGS.m and pen/penVBNorm.m when used together with EP - inf/diag_inv_sample.m, a Monte Carlo estimator gfortran support to pls/lbfgsb/Makefile (thanks to Ernst Kloppenburg) slight modification to mat/@matFFTN/mvm.m to make it more consistent simple gradient solver using Barzilai-Borwein step size pls/plsBB.m NEW in version 1.3, 2010-11-12 ------------------------------ extension of the matrix class; bugfixes plsSB - split Bregmin PLS solver new inference engine dli.m instead of vbidl.m - support of parallel EP in addition to VB - marginal likelihood computation extended documentation and demo programs NEW in version 1.2, 2010-08-25 ------------------------------ refactored the matrix classes added more penalty functions added non-linear potential support for group sparsity improved PLS solvers updated complex number treatment NEW in version 1.1, 2010-08-10 ------------------------------ added demo examples and documentation NEW in version 1.0, 2010-07-05 ------------------------------ initial version of the glm-ie toolbox