Cost Function, Linear Regression, trying to avoid hard coding theta. Octave.
Machine LearningOctaveLinear RegressionMachine Learning Problem Overview
I'm in the second week of Professor Andrew Ng's Machine Learning course through Coursera. We're working on linear regression and right now I'm dealing with coding the cost function.
The code I've written solves the problem correctly but does not pass the submission process and fails the unit test because I have hard coded the values of theta and not allowed for more than two values for theta.
Here's the code I've got so far
function J = computeCost(X, y, theta)
m = length(y);
J = 0;
for i = 1:m,
h = theta(1) + theta(2) * X(i)
a = h - y(i);
b = a^2;
J = J + b;
end;
J = J * (1 / (2 * m));
end
the unit test is
computeCost( [1 2 3; 1 3 4; 1 4 5; 1 5 6], [7;6;5;4], [0.1;0.2;0.3])
and should produce ans = 7.0175
So I need to add another for loop to iterate over theta, therefore allowing for any number of values for theta, but I'll be damned if I can wrap my head around how/where.
Can anyone suggest a way I can allow for any number of values for theta within this function?
If you need more information to understand what I'm trying to ask, I will try my best to provide it.
Machine Learning Solutions
Solution 1 - Machine Learning
You can use vectorize of operations in Octave/Matlab. Iterate over entire vector - it is really bad idea, if your programm language let you vectorize operations. R, Octave, Matlab, Python (numpy) allow this operation. For example, you can get scalar production, if theta = (t0, t1, t2, t3) and X = (x0, x1, x2, x3) in the next way: theta * X' = (t0, t1, t2, t3) * (x0, x1, x2, x3)' = t0x0 + t1x1 + t2x2 + t3x3 Result will be scalar.
For example, you can vectorize h in your code in the next way:
H = (theta'*X')';
S = sum((H - y) .^ 2);
J = S / (2*m);
Solution 2 - Machine Learning
Above answer is perfect but you can also do
H = (X*theta);
S = sum((H - y) .^ 2);
J = S / (2*m);
Rather than computing
(theta' * X')'
and then taking the transpose you can directly calculate
(X * theta)
It works perfectly.
Solution 3 - Machine Learning
The below line return the required 32.07 cost value while we run computeCost once using θ initialized to zeros:
J = (1/(2*m)) * (sum(((X * theta) - y).^2));
and is similar to the original formulas that is given below.
Solution 4 - Machine Learning
It can be also done in a line- m- # training sets
J=(1/(2*m)) * ((((X * theta) - y).^2)'* ones(m,1));
Solution 5 - Machine Learning
J = sum(((X*theta)-y).^2)/(2*m);
ans = 32.073
Above answer is perfect,I thought the problem deeply for a day and still unfamiliar with Octave,so,Just study together!
Solution 6 - Machine Learning
If you want to use only matrix, so:
temp = (X * theta - y); % h(x) - y
J = ((temp')*temp)/(2 * m);
clear temp;
Solution 7 - Machine Learning
This would work just fine for you -
J = sum((X*theta - y).^2)*(1/(2*m))
This directly follows from the Cost Function Equation
Solution 8 - Machine Learning
Python code for the same :
def computeCost(X, y, theta):
m = y.size # number of training examples
J = 0
H = (X.dot(theta))
S = sum((H - y)**2);
J = S / (2*m);
return J
Solution 9 - Machine Learning
function J = computeCost(X, y, theta)
m = length(y);
J = 0;
% Hypothesis h(x)
h = X * theta;
% Error function (h(x) - y) ^ 2
squaredError = (h-y).^2;
% Cost function
J = sum(squaredError)/(2*m);
end
Solution 10 - Machine Learning
I think we needed to use iteration for much general solution for cost rather one iteration, also the result shows in the PDF 32.07 may not be correct answer that grader is looking for reason being its a one case out of many training data.
I think it should loop through like this
for i in 1:iteration
theta = theta - alpha*(1/m)(theta'*x-y)*x
j = (1/(2*m))(theta'*x-y)^2