DownloaD LecturE NoteS

There is considerable overlap between the fields of neural networks and statistics. Statistics is concerned with data analysis. In neural network terminology, statistical inference means learning to generalize […]

It is rarely useful to have a NN simply memorize a set of data, since memorization can be done much more efficiently by numerous algorithms for table look-up. Typically, you want the NN to be able to perform […]

A vector of values presented at one time to all the input units of a neural network is called a “case”, “example”, “pattern, “sample”, etc. The term “case” will be used in this FAQ because it is widely […]

How to count layers is a matter of considerable dispute.
Some people count layers of units. But of these people, some count the input layer and some don’t.
Some people count layers of weights. But I have no […]

Teuvo Kohonen is one of the most famous and prolific researchers in neurocomputing, and he has invented a variety of networks. But many people refer to “Kohonen networks” without specifying which kind of Kohonen […]

There are many many kinds of NNs by now. Nobody knows exactly how many. New ones (or at least variations of old ones) are invented every week. Below is a collection of some of the most well known methods, not […]

Neural Networks are interesting for quite a lot of very different people:
Computer scientists want to find out about the properties of non-symbolic information processing with neural nets and about learning […]

In principle, NNs can compute any computable function, i.e., they can do everything a normal digital computer can do (Valiant, 1988; Siegelmann and Sontag, 1999; Orponen, 2000; Sima and Orponen, 2001), or perhaps […]