how to use lagrange multipliers with inequalities

Daniel Parker

3 min read

·

19-01-2025

1

0

Share

Lagrange multipliers are a powerful tool for finding extrema (maximums and minimums) of a function subject to equality constraints. However, many real-world optimization problems involve inequality constraints. This article will guide you through the process of adapting the Lagrange multiplier method to handle these situations. We'll explore the concept of Karush-Kuhn-Tucker (KKT) conditions, which extend the Lagrange multiplier method to encompass inequality constraints.

Understanding the Challenge: Inequality Constraints

When dealing with equality constraints like g(x,y) = c, we know the solution lies on the constraint curve. With inequality constraints like g(x,y) ≤ c, the solution might lie on the constraint boundary (g(x,y) = c) or inside the feasible region (g(x,y) < c). This introduces complexity because we need a way to determine where the optimum lies.

Introducing the Karush-Kuhn-Tucker (KKT) Conditions

The KKT conditions provide a set of necessary conditions for a solution to be optimal when dealing with inequality constraints. They extend the method of Lagrange multipliers by introducing a new variable, often denoted as λ (lambda), which acts as a multiplier for each inequality constraint. These conditions are:

1. Stationarity: The gradient of the Lagrangian function is zero. The Lagrangian is formed by combining the objective function, equality constraints (if any), and inequality constraints. For a function f(x) subject to g(x) ≤ 0, the Lagrangian is:

   L(x, λ) = f(x) + λg(x)

   The stationarity condition is: ∇L(x, λ) = 0

2. Primal Feasibility: All inequality constraints must be satisfied: g(x) ≤ 0

3. Dual Feasibility: The Lagrange multipliers for inequality constraints must be non-negative: λ ≥ 0

4. Complementary Slackness: This is the crucial condition that distinguishes KKT from standard Lagrange multipliers. It states: λg(x) = 0

   This implies that either:

   • λ = 0 (the constraint is not binding; the solution lies in the interior of the feasible region)
   • g(x) = 0 (the constraint is binding; the solution lies on the boundary)

How to Apply the KKT Conditions: A Step-by-Step Approach

Let's illustrate with an example. Consider minimizing the function f(x, y) = x² + y² subject to the inequality constraint g(x, y) = x + y - 1 ≤ 0.

1. Form the Lagrangian:

   L(x, y, λ) = x² + y² + λ(x + y - 1)

2. Apply the Stationarity Condition: Take the partial derivatives and set them to zero:

   ∂L/∂x = 2x + λ = 0 ∂L/∂y = 2y + λ = 0 ∂L/∂λ = x + y - 1 ≤ 0

3. Consider Complementary Slackness: We have two cases:

   • Case 1: λ = 0: This implies 2x = 0 and 2y = 0, leading to x = 0 and y = 0. However, this violates the primal feasibility condition (0 + 0 - 1 ≤ 0 is false). Therefore, this case is invalid.

   • Case 2: x + y - 1 = 0: This means the constraint is binding. Substituting x = 1 - y into 2x + λ = 0 and 2y + λ = 0, we get:

     2(1 - y) + λ = 0 2y + λ = 0

     Solving this system gives x = 0.5, y = 0.5, and λ = -1. Note that λ is negative, which violates the dual feasibility condition. Thus, the solution must be on the boundary but the solution we derived is not valid. This means that we made an error in assuming that the solution is on the boundary.

     Let's reconsider the problem. If x + y - 1 = 0, then y = 1 - x. Substituting into the objective function, we get f(x) = x² + (1 - x)² = 2x² - 2x + 1. Taking the derivative and setting it to zero gives 4x - 2 = 0, so x = 0.5 and y = 0.5. This satisfies all KKT conditions.

4. Verify the Solution: The minimum value of f(x, y) is 0.5² + 0.5² = 0.5.

Multiple Inequality Constraints

The process extends to multiple inequality constraints. You'll have a separate Lagrange multiplier for each constraint, and the complementary slackness condition must hold for each one.

Conclusion

The KKT conditions provide a powerful framework for solving optimization problems with inequality constraints. While more complex than the standard Lagrange multiplier method, mastering this technique opens up a wider range of practical applications in various fields, from engineering to economics. Remember to always carefully consider the complementary slackness condition to determine the location of the optimum within the feasible region. Solving these problems often requires systematic checking of different scenarios.