Consider the problem maximize x_1 log(1/x_1) + ... + x_n log(1/x^n) subject to x

Question

Consider the problem maximize x_1 log(1/x_1) + ... + x_n log(1/x^n) subject to x_1 + ... x_n = 1 x_1, ..., x_n > 0, where "log" is the natural logarithm function. This is a well-known problem in statistics and information theory, where the variable x = [x_1, ..., x_n]^T represents a probability distribution and the objective function is its entropy. Because we cannot easily deal with problems with strict inequality constraints, we will ignore the constraints x_1, ..., x_n > 0 and see if we can obtain a solution that satisfies it. a. Are all feasible points for this problem (ignoring the strict inequality constraints) regular? b. Write down the Lagrange condition for this problem (ignoring the strict inequality constraints) and find all points satisfying it. c. For all points in part b, determine if the SONC is satisfied. d. For those points in part c satisfying the SONC, determine if the SOSC is satisfied. e. Is the original problem including the strict inequality constraints) a convex optimization problem? Justify fully.

Explanation / Answer

P{max(X,Y,Z) t} = P{X t and Y t and Z t} = P{X t}3 by
independence. Thus the distribution function of the maximum is (t
6)3 = t
18, and the
density is 18t
17, 0 t 1.
2. See Figure S1.1. We have
P{Z z} =

yzx
fXY (x, y) dx dy =

x=0
zx
y=0
exey dy dx
FZ(z) =
0
ex(1 ezx) dx = 1 1
1 + z
, z 0
fZ(z) = 1
(z + 1)2 , z 0
FZ(z) = fZ(z) = 0 for z < 0.
3. P{Y = y} = P{g(X) = y} = P{X g1(y)}, which is the number of xi’s that map to
y, divided by n. In particular, if g is one-to-one, then pY (g(xi)) = 1/n for i = 1,... ,n.
4. Since the area under the density function must be 1, we have ab3/3 = 1. Then (see
Figure S1.2) fY (y) = fX(y1/3)/|dy/dx| with y = x3, dy/dx = 3x2. In dy/dx we
substitute x = y1/3 to get
fY (y) = fX(y1/3)
3y2/3 = 3
b3
y2/3
3y2/3 = 1
b3
for 0 < y1/3 < b, i.e., 0 <y<b3.
5. Let Y = tan X where X is uniformly distributed between /2 and /2. Then (see
Figure S1.3)
fY (y) = fX(tan1 y)
|dy/dx|x=tan1 y
= 1/
sec2 x
with x = tan1 y, i.e., y = tan x. But sec2 x = 1 + tan2 x = 1+ y2, so fY (y) =
1/[(1 + y2)], the Cauchy density.
Le

Related Questions

Navigate

• Browse (All)
• Browse C
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.