Lagrange Multipliers

This notebook describes how to apply the method of Lagrange Multipliers to constrained optimization problems in Mathematica.  Specific tasks demonstrated are:
- Solve a constrained optimization problem exactly using Mathematica's Solve command.
- Solve a constrained optimization problem via estimations using Mathematica's FindRoot command.
The function in this notebook is set up to be easily cut-and-pasted so you can easily experiment with them.

TO SEE WHAT THIS NOTEBOOK PROVIDES:
- Select Kernel|Evaluation|Evaluate_Notebook to execute all the cell and produce the graphics.
- Then scroll through reading the commands and associated comments while you look at the commands' results.
- All input cells are set up such that they can be cut-and-pasted in their entirety for easy "re-use".

The material in this notebook was developed by Lt Col Steve Hadfield of the USAF Academy, Dept of Mathematical Sciences.	
Filename: Lagrange_Multipliers.nb

Lagrange Multipliers using the Solve command

The code below shows how to solve constrained optimization problems using the method of Lagrange Multipliers on a simple example problem where the objective function is f(x,y) = x*y and the constraint is g(x,y) = x + y = 100.  Use of the Solve command produces exact solutions using algebraic manipulation.  While this is the preferred method, it does not always work since not all systems of equations can be solved exactly.  Also it may only partially work and provide just some of the solutions.  In either case, Mathematica produces warning messages indicating it was not able to completely solve the system of equations.  When this occurs, you should look to the next section of this notebook that describes the use of the FindRoot command to approximate the solution to the system of equations.   Here's how the code does it:
Line L1 - Clears any previous symbol definitions to avoid conflicts.
Line L2 - Defines the objective function, f(x,y).
Line L3 - Defines the constraint function, g(x,y).
Line L4 - Defines the gradient of the objective function and calls it delf(x,y).
Line L5 - Defines the gradient of the constraint function and calls it delg(x,y).
Line L6 - Uses the Mathematica Solve command to solve the grad f = λ grad g and g(x,y)=c equations for x, y, and λ.  Give the solution the name "sol" for later reference.  The Solve command has two parameters.  The first is the set of equations to be solved and the second is the set of variables to solve for.  Both are given in list notation and one must take care to use the "==" notation to denote the forced equality in the equations.
Line L7 - Evaluates the objective function at the solution using the list extraction operator, [[-]], to exact parts of the solution from "sol".

Clear[f, g, x, y, delf, delg, sol]
   f[x_, y_] = 
        x*y;                       
   g[x_, y_] = 
        x + y;                      
   delf[x_, y_] = {D[f[x, y], x], 
          D[f[x, y], y]};       
   delg[x_, y_] = {D[g[x, y], x], 
          D[g[x, y], y]};       
    sol = 
      Solve[{delf[x, y] == λ*delg[x, y], 
          g[x, y] == 100}, {x, y, λ}]    
   
    f[sol[[1, 1, 2]], sol[[1, 2, 2]]]

Lagrange Multipliers using the FindRoot command

The code below shows how to solve constrained optimization problems using the method of Lagrange Multipliers on a simple example problem where the objective function is f(x,y) = x*y and the constraint is g(x,y) = x + y = 100.  This example used to the Mathematica FindRoot command to find an approximate solution to the system of equations based on an initial guess at the solution.  The drawback to this method is that, unlike the Solve command,  it only finds one solution and there may be others.  However, it applies to a much broader category of systems of equations than does the Solve command.  Here's how the code does it:
Line L1 - Clears any previous symbol definitions to avoid conflicts.
Line L2 - Defines the objective function, f(x,y).
Line L3 - Defines the constraint function, g(x,y).
Line L4 - Defines the set of equations that need to be solved and puts them into a list notation with the name "eqns".  One would think that we could just put the equations directly into the FindRoot command, however, due to order of evaluation problems, this does not work so we must define the equations separately.  Furthermore, we can not use the vector equation format that we did with the Solve command.
Line L5 - Uses the Mathematica FindRoot command to solve the equations for x, y, and λ given an initial starting point of 1 for each of the variables.  Typically, you'll want to look at the equations to determine an educated guess at the solution since the solution that you find (or even if you find a solution) will depend on the equations and how close the initial guess is to a solution.  We give the solution the name "sol" for later reference.
Line L6 - Evaluates the objective function at the solution using the list extraction operator, [[-]], to exact parts of the solution from "sol".

Clear[f, g, x, y, delf, delg, sol]
   f[x_, y_] = 
        x*y;                       
   g[x_, y_] = 
        x + y;                      
   
    eqns = {D[f[x, y], x] == λ*D[g[x, y], x], 
        D[f[x, y], y] == λ*D[g[x, y], y], 
        g[x, y] == 100}
    sol = 
      FindRoot[eqns, {x, 1}, {y, 1}, {λ, 1}]    
   
    f[sol[[1, 2]], sol[[2, 2]]]