#### Last update: 10/22/2007

## Technical Notes on Linear Regression and Information Theory

#### Asari, H.

**Abstract**
To understand how the brain processes sensory information, it is important to
study the relationship between input stimuli and the output neural responses.
Neuroscientists have typically looked at two complementary aspects of neural
representations. The first, and best studied, is the encoding process by which
a stimulus is converted by the nervous system into neural activity.
Less studied is the decoding process, by which experimenters attempt to use
neural activity to reconstruct the stimulus that evoked it. To characterize
these processes, various methods have been developed to model the
stimulus-response functions and to test their performance.

Here we briefly overview the basics and logics of these methods.
The first part reviews linear regression methods with a certain regularization
to find the best linear models. In particular, we will go through how ridge
regression is related to the singular value decomposition. The second part shows
how to apply information theory to test the quality of linear filters. We will
discuss the connection of correlation functions to entropy and information, and
a way to compute information by exploiting singular value decomposition (SVD).

Preprint (107KB, PDF)

See also my dissertation (Appendix A).