Neurals network, formerly known as parallel distributed system, is inspired by its biological counterpart. As the alias implies, input to the network is sent in parallel, though some networks are synchronously updated while others are asynchronously updated. Neural networks are used widely in computer science, engineering, and statistics, and its has many applications, from modelling neurological processes, mRNA splice site identification, to stock market price predictions.
While neural networks can perceived just as an abstract entity, it can also be considered in a biological context presented henceforth. The network consists of a collection of nodes, which representing neurons. The activity of a node represent the frequency of action potentials that is transduced. Activities are transmitted from one neuron to another through synapses, whose properties dictate the polarity and strength of the signal. Each input activity to a neuron is thus associated with a weight that describes the polarity and strength of the input. The weighted inputs from all the presynaptic neurons are summated and constitute the input to the postsynaptic neuron, which then produces an output activity based on a transfer function. For the sake of biological relevance, the choice the transfer function is usually a logistic funtion, which is ubiquitous in biological systems. (When neural network is first conceived in a non-biological context, the logistic function is used for its mathematical property of being differentiable.) Certain neurons in the network are designated as input neurons which receive sensory inputs, a few are motor neurons whose activities are the output of the network, and the remaining are interneurons.
Neural networks are often purely feed-forward, in the sense that the input is sent and processed without allowing it to traverse back. The network is arranged in layers, of which the bottom-most is the input layer and the top-most is the output layer, in addition to some number of hidden layers in between.
A variant of the aforementioned neural network is a recurrent neural network, particularly a fully recurrent neural network, in which all the neurons project to one another, including itself. Such a network (as well as any that permits recurrent projections) allows the system to be modeled as a dynamical system, as it can capture the dynamic interaction between sensory input and motor output. Intuitively, therefore, recurrent neural networks are physiologically recurrently and serves as a better for the brain.
