Math

Inequality

• Cauchy-Schwarz: \(E^2[XY] \le E(X^2) E(Y^2)\)

• Useful Inequalities

• Lipschitz continuous gradient

• \(\|(x+y)^+\|_2^2 \le \|x^+ + y\|_2^2\)

• \(\|(x+y)^+\|_2^2 \ge \|x^+\|_2^2 +2(x^+ \cdot y)\)

Optimization

• Yurii Nesterov: Lectures on Convex Optimization

• Lyapunov Optimization: An Introduction

Convexity

• If \(\forall y,f(\cdot,y)\) is convex, then \(x \mapsto \max_y f(x, y)\) is convex.

• If \((x,y) \mapsto f(x,y)\) is convex, then \(x \mapsto \min_y f(x, y)\) is convex.