When classifying data with logistic regression using the maximum likelihood method, the likelihood function represents the probability of observing the given data under the model parameters. The likelihood function is bounded between 0 and 1 because it is a product of probabilities of individual observations, each of which lies between 0 and 1
. The upper bound of the likelihood is 1 , which would correspond to the model perfectly predicting all observed data points. However, this perfect likelihood value of 1 is generally not attainable in practice because real-world data usually contain noise and overlap between classes, making perfect prediction impossible
. In logistic regression, the maximum likelihood estimate (MLE) is found by maximizing the likelihood function (or equivalently the log-likelihood) over the parameter space. While the likelihood can theoretically approach 1, it rarely reaches exactly 1. Instead, the MLE provides the parameter values that yield the highest likelihood achievable given the data, which is often less than 1 but as close as possible
. To summarize:
- The likelihood function in logistic classification is bounded between 0 and 1.
- The theoretical upper bound of the likelihood is 1.
- Achieving a likelihood of exactly 1 means perfect classification of all data points, which is uncommon.
- The maximum likelihood method seeks the parameter values that maximize the likelihood, often resulting in a value close to but less than 1
