Random Variables

Backward propagation or more precisely Error Backward Propagation is a technique in neural networks used to optimize a neural network. An output error is propagated backward within the network and as the error propagates the network parameter are adjusted with the aim to reduce resulting output error. Our Model: Consider the following graph of our model of the neuron; graph LR A@{shape: circ, label: "w"} --> B@{shape: rect, label: "g(.)"} --> C@{shape: circ, label: "y"} --> D@{shape: rect, label: "h(.)"} --> E@{shape: circ, label: "f"} F@{shape: circ, label: "b"} --> B In this model; ...

Wikipedia defines Automatic Differentiation (AD) as a technique to evaluate partial derivative of a function. I will not repeat the definition, but I will quote an important point that’s relevant to what we are doing; … By applying the chain rule repeatedly to these operations, partial derivatives of arbitrary order can be computed automatically … The take away here is that with AD; we breaks down a long expression into small sub-expressions. ...

Computational graph is a directed graph used to represent a sequence of operations to some input data, until its eventual output. This sequence of operations could be any form of transformations or manipulation to the input data. In this article we will restrict ourselves to those operations that involve mathematical expressions. The key elements of a computational graph are; a node and an edge Since the graph is used to represent data flow along a sequence of operations, I will expand the concept of a computational graph to also incorporate an additional element; rules. The significance of this addition will become apparent as we proceed. Thus, for our purpose a computational graph will be composed of the following elements; ...

The standard formula for a cirlce is $\displaystyle x^2+y^2=r^2$. This can also be written as $\displaystyle x^2+y^2-r^2=0$ or more properly as $\displaystyle C(x,y)=x^2+y^2-r^2=0$. There are situations where the parametric form of the circle is useful. Here, I will show how to derive the parametric form of the circle using angles on the circle and a parameter $\displaystyle t$. Given a circle $\displaystyle C$, with radius $r=1$; we can derive its parametric form in 2 ways; ...