In this problem, we will formally examine how transforming the training data in

Question

In this problem, we will formally examine how transforming the training data in simple ways can affect the performance of common classifiers. Understanding the effect of transformations is important in practice, where we frequently have to combine multiple heterogeneous features. Suppose we are given a training data set S (ri,yi), ...,(rn,yn) where each feature vector ri lies in so coordinate j of ri is denoted by a d-dimensional space. Suppose each r For each zi, suppose we transform it to zi by rescaling each axis of the data by a fixed factor; that is, for n and every coordinate j 1 d, we write: every i Here als are real, non-zero and positive constants. Thus, our original training set S is transformed after rescaling to a new training set S' 1021, y1) (zn,yn). For example, if we have two features, and if a 3, and a 2, then, a feature vector z (zi,z2) gets transformed by rescaling to z (2,22) (3r1,2z2) A classifier C(z) in the original space (of r's) is said to be equal to a classifier C'(2) in the rescaled space (of z's) if for every z ERd, C(z) C'(z), where z is obtained by transforming by recaling. In our previous example, the classifier C in the original space C(z) Predict 0 ifris 1, else predict 1 is equal to the classifier Cr in the rescaled space C'(2): Predict 0 if zis 3, else predict 1. This is because if C(r) 0 for an z (ri, r2), then ar 1. This means that for the transformed vector 2) (3r1, 2r2) 3ris 3, and thus C'(z) 0 as well. Similarly, if C(r) 1, then zi 1 and 21 3 and thus C(z) 1. Now, answer the following questions: ad. Suppose we train a k-NN 1. First, suppose that all the a values are equal; that is, ai classifier C on S and a k-NN classifier C' on S'. Are these two classifiers equal? What if we trained C and C' on S and S' respectively using the ID3 Decision Tree algorithm? What if we trained C and C' on S and S" respectively using the Perceptron algorithm? If the classifiers are equal, provide a brief argument to justify why; if they are not equal, provide a counterexample. 2. Repeat your answers to the questions in part (1) when the ais are different. Provide a briefjustification for each answer if the classifiers are equal, and a counterexample if they are not. 3. From the results of parts (1) and (2), what can you conclude about how k-NN, decision trees and perceptrons behave under scaling transformations?

Explanation / Answer

I can do this but actually there are too many problems !! So I think this question surely need more than 3000 points and btw your response rate is low that's why waiting for your response!!

Related Questions

Navigate

• Browse (All)
• Browse I
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.