Answer the following questions questions as compactly as possible. Your answer s
ID: 669329 • Letter: A
Question
Answer the following questions questions as compactly as possible. Your answer should not be more
than 2-3 lines of explanation.
(a) What assumption is made in Naive Bayes algorithm? How does this assumption help in
making learning feasible?
(b) What modeling assumptions are made in Gaussian Naive Bayes classi er?
(c) In logistic regression algorithm, where exactly is the training data used? Explain your
(d) Describe the di erence between the generative and discriminative models in supervised
learning.
Explanation / Answer
a) Naive Bayes has been studied extensively since the 1950s. It was introduced under a different name into the text retrieval community in the early 1960s,[1]:488 and remains a popular (baseline) method for text categorization, the problem of judging documents as belonging to one category or the other (such as spam or legitimate, sports or politics, etc.)
Naive Bayes classifiers are highly scalable, requiring a number of parameters linear in the number of variables (features/predictors) in a learning problem. Maximum-likelihood training can be done by evaluating a closed-form expression,[1]:718 which takes linear time, rather than by expensive iterative approximation as used for many other types of classifiers.
b) More about the Gaussian Distribution I For a random variable X with pdf N(x; µ, ), the mean of the distribution is µ: E[X] = µ I The covariance of the random variable is : for all i, j E[(Xi µi)(Xj µj )] = i,j
Applying the Model I For a new test point x, the output of the classifier is f(x) = arg max y p(y|x; ) = arg max y p(x, y; ) P y p(x, y; ) = arg max y p(x, y; ) = arg max y q(y)N(x; µy , )
c) Logistic regression model where the dependent variable (DV) is categorical. This article covers the case of binary dependent variables—that is, where it can take only two values, such as pass/fail or win/lose.
Logistic regression was developed by statisticianDavid Cox in 1958[2][3] (although much work was done in the single independent variable case almost two decades earlier). The binary logistic model is used to estimate the probability of a binary response based on one or more predictor (or independent) variables (features). As such it is not a classification method. It could be called a qualitative response/discrete choice model in the terminology of economics.
d) Generative model: The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.
Discriminative model: Discriminative models, as opposed to generative models, do not allow one to generate samples from the joint distribution of and . However, for tasks such asclassification and regression that do not require the joint distribution, discriminative models can yield superior performance.[1][2][3] On the other hand, generative models are typically more flexible than discriminative models in expressing dependencies in complex learning tasks. In addition, most discriminative models are inherently supervised and cannot easily be extended to unsupervised learning. Application specific details ultimately dictate the suitability of selecting a discriminative versus generative model.
Related Questions
drjack9650@gmail.com
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.