Neural Networks (Geoffrey Hinton Course)

Some Simple Models or Neurons

Linear Neurons

$y = b + \sum_{i} x_i w_i$

$y$ output, $b$ bias, $x_i$ input, $w_i$ weight.

Binary Threshold Neurons

$z = \sum_{i} x_i w_i$

$y = 1$ if $z \geq \theta$, $0$ otherwise.

Or, equivalently,

$z = b + \sum_{i} x_i w_i$

$y = 1$ if $z \geq 0$, $0$ otherwise.