Andy Long
There are two fundamental ideas at work here. The first is finding a best-fit to an over-determined problem (over-determined means that we have too many constraints for our ability to fit a given model).
The second essential idea is that of using a linear approximation or method to attempt a solution to a non-linear problem. We will rely on linear approximations, and iteration, hoping to get a good approximation to the true solution of the non-linear problem.