UNIVERSITY OF CALIFORNIA AT BERKELEY
Department of Mechanical Engineering
ME233 Advanced Control Systems IISpring 2016

Homework #1

Assigned: Feb. 4 (Th)
Due:Feb. 11 (Th)

1. We wish to determine how to split a positive number L into N pieces, so that theproduct of the N pieces is maximized. The problem can be solved using dynamic
programming by formulating it as follows. Consider a first-order “pure integrator”
x(k + 1) = x(k) + u(k),x(0) = 0.

We wish to determine the optimal control sequence
U0o = {uo (0), uo (1), · · · , uo (N − 1)}such that:(a) uo (k) ≥ 0.
(b) x(N ) = L.
(c) The following cost function is maximized:
J=N−1
Yu(k) = u(0) u(1) · · · u(N − 1)

k=0

To use dynamic programming, it is convenient to define the following optimal valuefunctionoJm[x(m)] = maxUmN−1
Yu(k)

k=m

where Um = {u(m), u(m + 1), · · · , u(N − 1)} is the set of all feasible control sequencesfrom the instance m.
Hint: Notice that, because of the terminal condition x(N ) = L, and the state equation,
the optimal value function at x(N − 1) is given by
J o [x(N − 1)] = uo (N − 1) = L − x(N − 1)
Use the Bellman equation starting from this boundary condition to determine an optimal control law for uo (k).

1

2. Finite-Horizon Optimal Tracking Problem:
Consider the discrete time systemx(k + 1) = A x(k) + B u(k)y(k) = C x(k)where x ∈ Rn and y, u ∈ Rm . Assume the existence of a known reference output
sequence
yd (0), yd (1), . . . , yd (N )The optimal control is sought to minimize the finite-horizon quadratic performanceindex:J = [yd (N ) − y(N )]T Q̄f [yd (N ) − y(N )]
+N−1X



[yd (k) − y(k)]T Q̄ [yd (k) − y(k)] + uT (k) Ru(k)k=0where Q̄f , Q̄ and R are symmetric and positive definite matrices of the appropriate
dimensions. Find the optimal control law by applying dynamic programming andutilizing the Bellman equation
oJko [x(k)] = min L[x(k), u(k)] + Jk+1[x(k + 1)]u(k)whereL[x(k), u(k)] = [yd (k) − y(k)]T Q̄ [yd (k) − y(k)] + uT (k)Ru(k)JNo [x(N )] = [yd (N ) − y(N )]T Q̄f [yd (N ) − y(N )] .
Hint: Show that the optimal cost from state x(k) to the final state can be expressedas
Jko [x(k)] = xT (k)P (k)x(k) + xT (k)b(k) + c(k) .
Obtain recursive expressions for P (k), b(k), and c(k) (from k = N to k = 0).
3. A product is produced by three different factories: A, B, and C. Factories A, B, andC respectively produce 25%, 50%, and 25% of the total production. In factories A andB, 98% of the items produced are not defective, whereas in factory C, 99% are notdefective. Calculate: (a) The probability that a randomly-chosen item is defective and(b) the probability that a (randomly-chosen) non-defective item comes from factory C.
4. In the “Monty Hall” three-door problem, a contestant is asked to choose one of threedoors. One of the three doors conceals a prize while the other two do not. Afterthe contestant chooses, Monty Hall (the master of ceremonies of the Let’s Make a Deal
television show) opens one of the two doors the player did not choose to reveal one door
2that does not conceal the prize. The contestant is then permitted to either stay with
their original choice or switch to the other unopened door. Determine the contestant’s
probability of getting the prize if she switches. Assume that before Monty Hall opens
one of the doors, the prize is equally likely to be hidden behind each of the three doors.
Hint: Let the doors be called x, y, and z. Let Cx be the event that the prize is behinddoor x and so on. Let Hx be the event that the host opens door x and so on. Assumingthat you choose door x, the probability that you win a car if you then switch yourchoice is given byP ((Hz ∩ Cy ) ∪ (Hy ∩ Cz )) .
Notice that, by Bayes’ rule, P (Hz ∩ Cy ) = P (Hz |Cy )P (Cy ).
5. Consider a discrete random variable X with probability mass function pX (x1 ) = a,pX (xn ) = b, with pX (xi ) varying linearly between a and b for evenly spaced possiblevalues xi , i ∈ {1, . . . , n}.
(a) What are the conditions on a and b to make the above a valid probability mass(density) function? How do these conditions vary with n and the range of possible
values of x?
(b) What is the expected value and variance of X?
(c) What is the CDF FX (x) ?
(d) If we take the sum of two independent random variables that have this same
distribution, what is the probability mass (density) function of their sum?
(e) Write and submit code (in Matlab, Julia, or Python) that implements a function
to return a random variable with this distribution, with a, b, x1 (xmin ), xn (xmax ),and n as inputs. Verify the expected behavior by evaluating this function manytimes and plotting a histogram for several choices of parameter values - at least
one case you plot should have a 6= b. (Hint: start with the output of the standard
uniform rand() function.)
Repeat the above steps for a continuous random variable with probability density function pX (xmin ) = a, pX (xmax ) = b, and pX (x) varying linearly between a and b for
xmin ≤ x ≤ xmax . There is no n in the continuous case.

3