EE 2810, HW 3: Submit either one pdf file (preferred choice) or one word-documen

Question

EE 2810, HW 3: Submit either one pdf file (preferred choice) or one word-document for this HW.

Gradient descent is a numerical method for finding the minimum of a function y=f(x). In this HW we are going to write a function that performs the gradient descent method on y=f(x). Then we are going to see the method being applied to y=x2 by calling the function we wrote.

1. Open a script and write the following code in the script.

     a. Define a variable ‘x’ to be an array of numbers from -10 to 10 with step sizes of 0.1.

     b. Define the array ‘y’ to be y=x2, and plot ‘y’ as a function of ‘x’.

     c. From the plot created in (1.b), answer the following questions:

      i) Is ‘y’ a convex function of ‘x’?

     ii) Does ‘y’ have a minimum over the range of ‘x’? If so, what is the minimum value of ‘y’ and at what value of ‘x’ does that minimum occur? What is the gradient of y=x2 at that minimum point?

     d. Save your script under the name ‘myExample.m’

2. Open a new script to write a function called ‘myGradientDescent’ that takes ‘x’ and ‘y’ as an input and gives ‘min_y’ and ‘x_at_min_y’ as an output.

3. Now we want to implement the gradient descent method in the function ‘myGradientDescent’. The gradient descent method starts at an initial point ‘x0’ and keeps moving towards the minimum point with a step size of ‘a’ until it converges to the minimum. To move towards the minimum point means to move in the direction that reduces the gradient. To converge to the minimum point means that you reach a point where the gradient at that point converges to zero, i.e., the gradient at that point is within a distance of ‘e’ from zero.

     a. Define x0=10, a=0.1, e=0.01 and the condition cnd=1.

     b. Calculate the gradient of ‘y’ at ‘x0’ as grad_y_at_x0 = (y_at_x0 – y_at_x0_minus_a)/a where y_at_x0 is the value of ‘y’ at the point ‘x0’ and ‘y_at_x0_minus_a’ is the value of ‘y’ at the point ‘x0 - a’.

     c. Write a while loop which keeps executing until cnd=0. In the loop:

      i) The value stored in ‘x0’ is updated using x0 = x0 – (grad_y_at_x0 * a)

     ii)The gradient of ‘y’ is recalculated at the new point ‘x0’ using grad_y_at_x0 = (y_at_x0 – y_at_x0_minus_a)/a.

     iii) The value of the condition ‘cnd’ is updated using cnd= grad_y_at_x0<e.

     d. When the while loop is done executing, the variable ‘x0’ would contain the point where the minimum occurred and the variable ‘y_at_x0’ will contain the minimum value of ‘y’. Now, outside the while loop set min_y=y_at_x0 and x_at_min_y=x0.

4. Open your script ‘myExample.m’ and continue coding in there:

     a. Call the function ‘myGradientDescent’ using the input ‘x’ and ‘y’ and let the output of your function call appear on the command window by not placing a semi-colon at the end of your function call.

     i) Using ‘hold on’, plot the output of the function call as a star (i.e., ‘*’) on top of the plot created in part 1.b. Is the ‘*’ placed at the minimum of y=x2 where you expect it to be? If not, you have a mistake in your code and you need to debug your code!

Explanation / Answer

clear
clc
x=-10:0.1:10;
y=x.^2;
plot(x,y);

Yes it is a "y" is convex to "x" because the double dervative is positive

Now the function is coded below. I have made a change in the algorithm given in the question. That is in the while loop. The variable cnd is useless in my code. You can see how I have done the while loop. It works and hope it is ok

function [min_y,x_at_min_y]=myGradientDescent(x,y)
x0=10; a=0.1; e=0.01;cnd=1;
y = @(x) x.^2;

%y_at_x0=y(10);
%y_at_x0_minus_a=y(10-a);
grad_y_at_x0 = (y(x0) - y(x0-a))/a
while (grad_y_at_x0>e)
x0 = x0 - (grad_y_at_x0 * a)
grad_y_at_x0 = (y(x0) - y(x0-a))/a;
% cnd= grad_y_at_x0<e
end

min_y=y(x0)
x_at_min_y=x0
end

Related Questions

Navigate

• Browse (All)
• Browse E
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.