KenSchutte.com

Demonstration of Neural Network Regression

This is demonstration of a two-layer neural network used to fit a function. You can use the sliders on the right to interactively see how individual parameters affect the output.

Number of hidden units:

Shown in the figure are,
- Blue curve : the "unknown" target function, t(x).
```
t(x) = 0.3  +  0.4*x  +  0.5*sin(2.7*x)  +  1.1/(1 + x^2)
```
- Black dots : samples from t(x) used to train the network (only showing 20 out of 100).
- Red curve : the output of the network, y(x), for the current values of parameters.
This is a two-layer network where the first layer has tanh() units and the second layer is linear. Thus, the network computes the function,
```
y(x) = W₂ tanh(W₁ x + b₁) + b₂
```
For example, for n=2 hidden units and 1 dimensional input and output, there are 7 parameters,
```
y(x) = w_1,2 tanh(w_1,1 x + b_1,1) + 
       w_2,2 tanh(w_1,2 x + b_1,2) +
       b₂
```
The data comes from an example provided by Radford Neal in his software for Flexible Bayesian Modeling. This package was also used to train the weights that are loaded here by default.
The locations of the training points were sampled from a zero-mean, unit-variance Gaussian, so the the function is only fit well for values of x near the origin. Zoom out on the graph, and you can see that the estimated function can differ greatly from the target.