Lagrange Multipliers

Lagrange multipliers find the extrema of a function $f(\mathbf{x})$ subject to a constraint $g(\mathbf{x}) = 0$: $$\nabla f = \lambda \nabla g$$ Solve this system along with $g(\mathbf{x}) = 0$ to find the optimal point(s) and the multiplier $\lambda$. Intuition: At a constrained optimum, the gradient of $f$ must be parallel to the gradient of $g$. Why? If they weren't parallel, you could move along the constraint surface in a direction that increases $f$ — meaning you haven't found the optimum yet. The condition $\nabla f = \lambda \nabla g$ says exactly "the only way to increase $f$ is to violate the constraint." Concrete example: Maximize $f(x,y) = xy$ subject to $x + y = 10$. Set $g(x,y) = x + y - 10 = 0$. Then $\nabla f = (y, x)$ and $\nabla g = (1, 1)$. System: $y = \lambda,\ x = \lambda,\ x + y = 10$. Solution: $x = y = 5$, $\lambda = 5$, maximum $f = 25$. The multiplier $\lambda$ has meaning: It measures the sensitivity of the optimum to the constraint. If the constraint were $x + y = 10 + \epsilon$, the optimal value would increase by approximately $\lambda \cdot \epsilon = 5\epsilon$. This is the "shadow price" in economics. When to use: Constrained optimization in portfolio theory (maximize return subject to risk budget), entropy maximization, and any "optimize subject to" problem. In interviews: "maximize/minimize X given that Y is fixed." Alternative approach: For simple two-variable problems, you can substitute the constraint directly (set $y = 10 - x$ and optimize $f(x) = x(10-x)$). Lagrange multipliers scale to higher dimensions and multiple constraints where substitution becomes unwieldy.