MA3236 Non-linear Programming AY 2009/2010 Sem 1 NATIONAL UNIVERSITY OF SINGAPORE MATHEMATICS SOCIETY PAST YEAR PAPER SOLUTIONS with credits to Sean Lim Wei Xinq, Zheng Shaoxuan MA3236 Non-linear Programming AY 2009/2010 Sem 1 Throughout this paper, for any matrix A, A > 0 denotes A is positive definite, A ≥ 0 denotes A is positive semidefinite, A < 0 denotes A is negative definite, A ≤ 0 denotes A is negative semidefinite and A ≈ 0 denotes A is indefinite. Question 1 (a) We have 4x1 − 2x2 + 6x21 + 4x31 2x2 − 2x1 ∇f (x) = Then, setting ∇f (x) = 0 gives us 2x1 − x2 + 3x21 + 2x31 = 0 (1) x1 = x2 (2) From (1) we have 2x31 + 3x21 + x1 = 0 x1 (2x21 + 3x1 + 1) = 0 x1 (2x1 + 1)(x1 + 1) = 0 1 x1 = 0 or x1 = − or x1 = −1 2 −1/2 −1 0 (3) (1) (2) . and x∗ = So the stationary points are x∗ = , x∗ = −1/2 −1 0 (b) The Hessian of f is given by Hf (x) = 4 + 12x1 + 12x21 −2 −2 2 So we have (1) Hf (x∗ ) (2) Hf (x∗ ) (3) 4 −2 −2 2 1 −2 −2 2 4 −2 −2 2 = = Hf (x∗ ) = NUS Math LaTeXify Proj Team Page: 1 of 11 NUS Mathematics Society MA3236 Non-linear Programming (1) AY 2009/2010 Sem 1 (3) (1) (3) For Hf (x∗ ) and Hf (x∗ ), we have ∆1 = ∆2 = 4 > 0, and hence, Hf (x∗ ) = Hf (x∗ ) > 0, (1) (3) (2) which implies that x∗ and x∗ are strict local minimizers. Then, det(Hf (x∗ )) < 0 implies (2) (2) Hf (x∗ ) ≈ 0, which implies that x∗ is a saddle point. (c) We can write f as f (x) = xT Qx + x41 + 2x31 where Q= 2 −1 −1 1 Then, note that Q is symmetric positive definite. So if λmin (Q) > 0 is the smallest eigenvalue of Q, we have f (x) ≥ λmin (Q)||x||2 + x41 + 2x31 From here, we can observe that f is coercive. Hence, f has a global minimizer but no global maximizer. Question 2 Since f is continuous and coercive, it follows that for all K > 0, we can find a k > 0 such that if ||x|| > k, then f (x) > K. Let K > 0 and x0 ∈ D such that K > f (x0 ). Now consider the set S := {x ∈ D | ||x|| ≤ k} Note that S is non-empty and bounded. Also, S is closed by definition of S. Hence, by Weierstrass Theorem, there is a global minimizer in S, i.e. there is an x∗ ∈ S such that f (x∗ ) ≤ f (x) for all x ∈ S Now if x ∈ / S, then we have f (x∗ ) ≤ K < f (x) which shows that x∗ is also a global minimizer in D. Question 3 (a) For any x ∈ Rq we have, for any B ∈ Rn×q we have xT BT ABx = (Bx)T A(Bx) ≥ 0 since A is positive definite. This shows that BT AB is positive semidefinite. To have the fact that BT AB > 0, we require that Bx 6= 0 whenever x 6= 0. This will happen if N (B) = {0}, where N denotes the nullspace of B. Hence, Ak must be positive definite since Ak = ITn×k AIn×k , where In×k = (bij ) where ( 1 if i = j bij = 0 otherwise NUS Math LaTeXify Proj Team Page: 2 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 and N (In×k ) = {0} for all k = 1, · · · , n. (b) Let P be an orthogonal matrix such that A = PT DP for some diagonal matrix D with its diagonal entries λ1 , · · · , λn as the eigenvalues of A and Pk = PIn×k where In×k is defined as in part (a) so that Ak = PTk DPk . If µ is an eigenvalue of Ak and v its corresponding eigenvector, then we have µv = Ak v T v µv = vT Ak v = vT PTk DPk v = xT Dx for some vector x ∈ Rn . Then we have 2 µ||v|| = n X λi x2i i=1 ≤ n X λmax x2i = λmax ||x||2 = λmax ||v||2 i=1 Hence we have µ ≤ λmax . Similarly, we have µ||v||2 = n X λi x2i i=1 ≥ n X λmin x2i = λmin ||x||2 = λmin ||v||2 i=1 So µ ≥ λmin and this completes the proof. (c) By Taylor’s Theorem, there exists w ∈ [x, xk ] such that f (x) = f (xk ) + h∇f (xk ), x − xk i + 1 hHf (w), x − xk i 2 for all k = 1, · · · , p. Since f is convex, Hf (x) ≥ 0 and hence, hHf (w), x − xk i ≥ 0 and therefore f (x) ≥ f (xk ) + h∇f (xk ), x − xk i ∀x ∈ S ∀1 ≤ k ≤ p Hence, we have f (x) ≥ max{Lk (x) := f (xk ) + h∇f (xk ), x − xk i | 1 ≤ k ≤ p} ∀x ∈ S Question 4 (a) (Correction in paper during day of exam: “Width is twice the length” instead of “Length is twice the width”) Let x1 , x2 and x3 denote the length, width and height of the box respectively. Now since the width is twice the length, we have 2x1 = x2 . Then the volume of the box, V (x) is given by V (x) = x1 x2 x3 = 2x21 x3 NUS Math LaTeXify Proj Team Page: 3 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 Since it is not necessary to use all the wire and paper, we have inequality constraints instead of equality. For the wire, we have 4x1 + 4x2 + 4x3 − 20 = 4x1 + 8x1 + 4x3 − 20 = 12x1 + 4x3 − 20 ≤ 0 Dividing by 4 gives us h1 (x) := 3x1 + x3 − 5 ≤ 0 Next we consider the paper, which is the total surface area. We have 2x1 x2 + 2x1 x3 + 2x2 x3 − 16 = 2x1 (2x1 ) + 2x1 x3 + 2(2x1 )x3 − 16 = 4x21 + 6x1 x3 − 16 ≤ 0 Dividing by 2 gives us h2 (x) := 2x21 + 3x1 x3 − 8 ≤ 0 Next, since x1 and x3 represent the length and height of the box respectively, we require that they are non-negative, i.e. x1 , x3 ≥ 0. The objective function, if we are translating the problem into a minimization problem is the negation of the volume, i.e. f (x) := −V (x) = −2x21 x3 , x ∈ R2 Hence our optimization problem is min f (x) s.t. h1 (x) ≤ 0 h2 (x) ≤ 0 x1 , x3 ≥ 0 (b) The feasible set S is given by S := {x ∈ R2 | h1 (x) ≤ 0, h2 (x) ≤ 0, x1 , x3 ≥ 0} To show that all feasible points are regular, we consider 4 cases. Define the set R(x) by R(x) = {∇hi (x)} where hi (x) = 0, i = 1, 2. Now we have 3 1 4x1 + 3x3 3x1 ∇h1 (x) = ∇h2 (x) = Case 1: J(x) = ∅. Then R(x) = ∅ which is linearly independent by definition. Case 2: J(x) = {1}. Then R(x) = {∇h1 (x)}, and ∇h1 (x) 6= 0 for all x ∈ R2 . So the set is linearly independent. Case 3: J(x) = {2}. Then R(x) = {∇h2 (x)}, and ∇h2 (x) = 0 if and only if x = 0. But J(x) = {2} implies that h2 (x) = 0 ⇒ −8 = 0, a contradiction. So the set is always linearly independent. Case 4: J(x) = {1, 2}. Then R(x) = {∇h1 (x), ∇h2 (x)}. Now consider the matrix NUS Math LaTeXify Proj Team Page: 4 of 11 NUS Mathematics Society MA3236 Non-linear Programming R(x) := 3 4x1 + 3x3 1 3x1 AY 2009/2010 Sem 1 Then, the set R(x) is linearly independent if and only if the matrix R(x) is invertible. Now det(R(x)) = 5x1 − 3x3 = 0 if and only if x1 = 53 x3 . But this implies h1 (x) = 3 3 x3 5 + x3 − 5 = 0 14 x3 = 5 5 25 x3 = 14 Then, we have x1 = 15 14 . But this gives us h2 (x) = 2 15 14 2 +3 15 14 25 14 − 8 6= 0 which cannot happen since J(x) = {1, 2}. So in all cases, we have shown that x ∈ S must be regular. −4x1 x3 (c) We have ∇f (x) = . Hence the KKT conditions are −2x21 −4x1 x3 3 4x1 + 3x3 + µ2 + µ1 = 0, 1 −2x21 3x1 µ1 , µ2 ≥ 0 Case 1: J(x) = ∅. Then µ1 = µ2 = 0 and we have −4x1 x3 = 0 −2x21 = 0 But this implies x1 = x3 = 0, which contradicts our assumption that x1 , x3 > 0. Case 2: J(x) = {1}. Then µ2 = 0 and µ1 > 0 and we have −4x1 x3 + 3µ1 = 0 −2x21 (3) + µ1 = 0 (4) 3x1 + x3 − 5 = 0 (5) Now (4) implies µ1 = 2x21 . Then, from (3) we have −4x1 x3 + 6x21 = 0 ⇒ x1 (−4x3 + 6x1 ) = 0 ⇒ x1 = 0 or 3x1 = 2x3 . If x1 = 0, then from (3) we have µ1 = 0, a contradiction! If 3x1 = 2x3 , then from (5) we have x3 = 5/3, which then gives us x1 = 10/9. Then, µ1 = 200/81. But observe that 2 10 10 5 2 10/9 h2 =2 +3 −8= 6≤ 0 5/3 9 9 3 81 Hence it is NOT feasible. NUS Math LaTeXify Proj Team Page: 5 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 Case 3: J(x) = {2}. Then µ1 = 0 and µ2 > 0 and we have −4x1 x3 + 4µ2 x1 + 3µ2 x3 = 0 −2x21 + 3x1 µ2 2x21 + 3x1 x3 − 8 (6) =0 (7) =0 (8) Here, (7) implies x1 (−2x1 + 3µ2 ) = 0 which implies that either x1 = 0 or 3µ2 = 2x1 . But x1 = 0 implies −8 = 0, a contradiction. But 3µ2 = 2x1 implies from (6) that 8 −2x1 x3 + x21 = 0 3 (9) √ √ Then, 2(8) + 3(9) implies 12x21 = 16 ⇒ x21 = 4/3 ⇒ x1 = 2/ 3. This gives us µ2 = 4/3 3 and from (8) we have √ 8 + 4 3x3 − 8 = 0 3 16 1 4 √ = √ x3 = 3 4 3 3 3 Case 4: J(x) = {1, 2}. Then µ1 , µ2 > 0 and we have −4x1 x3 + 3µ1 + 4µ2 x1 + 3µ2 x3 = 0 (10) −2x21 + µ1 + 3µ2 x1 = 0 (11) 3x1 + x3 − 5 = 0 (12) + 3x1 x3 − 8 = 0 (13) 2x21 Now from (12) we have x3 = 5 − 3x1 , and from (13) we have 2x21 + 3x1 (5 − 3x1 ) − 8 = 0 −7x21 + 15x1 − 8 = 0 7x21 − 15x1 + 8 = 0 (7x1 − 8)(x1 − 1) = 0 8 x1 = , 1 7 If x1 = 1 then x3 = 2 and we have from (10) and (11) 3µ1 + 10µ2 = 8 µ1 + 3µ2 = 2 Solving the simultaneous equation yields µ1 = −4, µ2 = 2, a contradiction since µ1 > 0. If x1 = 8/7, then x3 = 11/7 and we have from (10) and (11) 65 352 µ2 = 7 49 24 128 µ1 + µ2 = 7 49 3µ1 + Solving the simultaneous equation yields µ1 = 128/343, µ2 = 32/49. Check again that is feasible. Hence the KKT points are NUS Math LaTeXify Proj Team x∗1 = 8/7 11/7 √ 2/ √3 8/7 ∗ and x2 = . 11/7 4/3 3 Page: 6 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 (d) Let the sets S1 and S2 be defined as S1 := {x ∈ R2 | h1 (x) ≤ 0, x1 ≥ 0, x3 ≥ 0} S2 := {x ∈ R2 | h2 (x) ≤ 0} Now observe that S1 is closed and bounded and S2 is closed since h2 is continuous, and that S = S1 ∩ S2 . Since S1 is bounded, S ⊆ S1 implies that S is also bounded, and S is closed since the intersection of closed sets is closed. So by Weierstrass Theorem, there is a global minimizer. Since all feasible points are regular, it follows that every global minimizer is a local minimizer, and by the KKT necessary conditions, every global min is a KKT point. Hence we simply evaluate 32 f (x∗1 ) = − √ = −2.053 9 3 1408 f (x∗2 ) = − = −4.105 343 So the global minimizer is x∗2 . Question 5 (a) Since it is a convex programming problem, a KKT point is also a global minimizer. Let h1 (x) := x21 + 4x22 − 4 + , h2 (x) = −x1 and h3 (x) = −x2 . Now we have −2 ∇f (x) = −4 2x1 ∇h1 (x) = 8x2 −1 ∇h2 (x) = 0 0 ∇h3 (x) = −1 So the KKT conditions are −1 0 2x1 −2 =0 + µ3 + µ1 + µ2 0 −1 8x2 −4 where µ1 , µ2 , µ3 ≥ 0. Case 1: J(x) = ∅. Then µ1 = µ2 = µ3 = 0 and hence, −2 −4 = 0, a contradiction. Case 2: J(x) = {1}. Then µ1 > 0 and µ2 = µ3 = 0 and hence, −2 + 2µ1 x1 = 0 (14) −4 + 8µ1 x2 = 0 (15) x21 + 4x22 − 4 + = 0 (16) x1 , x2 > 0 NUS Math LaTeXify Proj Team Page: 7 of 11 (17) NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 Now (14) implies that x1 = 1/µ1 and (15) implies that x2 = 1/2µ1 . Substituting them into (16) yields 1 1 + −4+=0 µ21 µ21 2 =4− µ21 2 µ21 = 4− r 2 µ1 = 4− Then we have r x1 = √ 4− 2 and x2 = 4− √ 2 2 From here, we have a KKT point and hence a global minimizer. If there is another KKT point, then their function values will be the same. So we have p (4 − )/2 ∗ √ √ x () = 4 − /2 2 with the multipliers µ∗1 () = p 2/(4 − ), µ∗2 () = µ∗3 () = 0. (b) Let F () = f (x∗ ()). Then r F () = −2 Then, r √ 4− 4− 4− − 4 √ = −4 2 2 2 2 √ 1 −2 2 ·− F () = √ 2 4− 0 by the chain rule. Simplifying gives us r 0 F () = 2 = µ∗1 () 4− as required. Question 6 (a) We are given f (x) = 4x21 + 4x22 − 2x1 x2 + x23 Observe that f (x) = xT where 1 1 Q x = xT Qx 2 2 8 −2 0 Q = −2 8 0 0 0 2 (You may want to check that Q is indeed symmetric positive definite) NUS Math LaTeXify Proj Team Page: 8 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 (b) By Proposition 5.1(a) we have h∇q(xk ), ∇q(xk )i2 E(xk+1 ) =1− E(xk ) h∇q(xk ), Q∇q(xk )i h∇q(xk ), Q−1 ∇q(xk )i Since f is a quadratic function, then its quadratic Taylor expansion, q is equal to the function itself. Also, by definition of E(xk ) and dk we have f (xk+1 ) − f (x∗ ) ||dk ||4 = 1 − f (xk ) − f (x∗ ) hdk , Qdk i (∇q(xk )T Q−1 ∇q(xk )) Now ∇f (x∗ ) = 0 implies x∗ = 0, and hence, f (x∗ ) = 0, and so we have f (xk+1 ) ||dk ||4 =1− f (xk ) hdk , Qdk i (∇q(xk )T Q−1 ∇q(xk )) Next, we multiply both sides by f (xk ) to obtain f (xk+1 ) = f (xk ) − ||dk ||4 (f (xk )) hdk , Qdk i (∇q(xk )T Q−1 ∇q(xk )) Then, by definition of f and ∇q(xk ) we have f (xk+1 ) = f (xk ) − ||dk ||4 (xTk Qxk ) 1 2 hdk , Qdk i xTk QT Q−1 Qxk Since Q is symmetric, it follows that QT = Q and canceling from numerator and denominator gives us 1 ||dk ||4 f (xk+1 ) = f (xk ) − 2 hdk , Qdk i as required. (c) We first determine the eigenvalues of Q. We have 8 − λ −2 0 −2 8 − λ 0 = 0 0 0 2 − λ 8 − λ −2 =0 (2 − λ) −2 8 − λ λ=2 or (8 − λ)2 − 4 = 0 λ = 2, 6, 10 Now since κ(Q) = λmax (Q)/λmin (Q) = 10/2 = 5 is large, the convergence rate, ρ is given by ρ(Q) = 1 − 1 4 = κ(Q) 5 Here, hence, in the worst case, the number of iterations, k is given by log k= +1 log ρ(Q) It is given that = 10−8 , so k = 12. NUS Math LaTeXify Proj Team Page: 9 of 11 NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 Question 7 (a) (i) Consider the figure below. (ii) Consider the figure below. (b) The Lagrangian function, L(x, µ) is given by L(x, µ) = −2x1 − x2 + µ(x21 + 4x22 − 2x1 x2 − 4) = µx21 + 4µx22 − 2µx1 x2 − 2x1 − x2 − 4µ where µ ≥ 0 and x ∈ R2 . Now we want to determine θ(µ) = inf L(x, µ) x∈R2 Hence we have 2µx1 − 2µx2 − 2 Lx (x, µ) = 8µx2 − 2µx1 − 1 Setting Lx = 0 gives us µx1 − µx2 − 1 = 0 (18) −2µx1 + 8µx2 − 1 = 0 (19) Taking (19) + 2(18) yields 6µx2 − 3 = 0 ⇒ x2 = NUS Math LaTeXify Proj Team Page: 10 of 11 1 2µ NUS Mathematics Society MA3236 Non-linear Programming AY 2009/2010 Sem 1 Note here that µ 6= 0 since that would imply that −1 = 0, a contradiction! Then, we have x1 = 3/2µ. Hence we obtain 9 1 3 3 1 + − − − − 4µ 4µ µ 2µ µ 2µ 7 =− − 4µ 4µ θ(µ) = So the dual problem is max θ(µ) = − 7 − 4µ 4µ s.t. µ > 0 Then, we have √ 7 7 7 2 ⇒ µ∗ = θ (µ∗ ) = 2 − 4 = 0 ⇒ µ∗ = 4µ∗ 16 4 0 (c) The Lagrangian function is given by 1 L(x, λ, µ) = xT Qx + cT x + λT (b − Ax) − µT x 2 Now we want to find the infimum of L for µ ≥ 0. Observe that L is convex, hence, the minimizer is obtained at the stationary point. So we have Lx = Qx + c − AT λ − µ Rewriting L gives us 1 L(x, λ, µ) = xT Qx + bT λ + (c − AT λ − µ)T x 2 Setting Lx = 0 gives us −Qx = c − AT λ − µ and hence we have θ(λ, µ) = inf L(x, λ, µ) x∈X 1 = xT Qx + bT λ − xT Qx 2 1 = − xT Qx + bT λ 2 So the dual problem is 1 − xT Qx + bT λ 2 s.t. Qx + c − AT λ − µ = 0 max µ≥0 as desired. NUS Math LaTeXify Proj Team Page: 11 of 11 NUS Mathematics Society

© Copyright 2018