Tag Archive: classification


Have you heard of k-Nearest Neighbors (k-NN) algorithm?

Filed under: Articles, Artificial Intelligence — Leave a comment
August 27, 2026

🤔 What is it?

The k-NN algorithm is a widely-used supervised learning technique that can handle classification and regression tasks. It is an instance-based learning method, which means that it stores the training instances and uses them at prediction time to make decisions. The algorithm uses proximity to determine the grouping of a new data point, based on the assumption that similar data points tend to cluster together. What makes it unique is its non-parametric nature, which means that it doesn’t make any assumptions about the underlying distribution of the data. As a result, it can be used to address non-linear separable data in classification tasks and non-linear regression problems. This property of k-NN makes it a valuable tool for various applications in machine learning.

🕵️‍♂️ How does it work?

The k-NN algorithm works by finding the k data points in the training set that are closest in distance (e.g. Euclidean, Manhattan, Cosine) to the new data point, and then assigning the new data point to the class that is most common among its k nearest neighbors in classification tasks or predicting the target value of the new data point based on the average of the target values of its k nearest neighbors in regression tasks. The value of k is a user-defined parameter that controls the number of neighbors considered for classification or regression.

💡 Why is it useful?

The k-NN algorithm is easy to understand and implement, making it a popular choice for beginners in machine learning. Since the k-NN algorithm has no assumptions about the underlying distribution of the data, it can be used for a wide range of data types and structures.

👨‍💻 My personal story

When I first started learning machine learning, the k-NN algorithm was one of the first algorithms I encountered. I remember feeling intimidated by some of the more complex algorithms, but k-NN seemed simple and intuitive. I used it to classify handwritten digits in the MNIST dataset and was impressed by its accuracy and speed. As I continued to study machine learning, I’ve moved on to more advanced techniques (e.g. Neural Networks). However, I still appreciate the simplicity and versatility of k-NN.

📚 Who should learn it?

The k-NN algorithm is a useful tool to have for anyone learning AI. It can help you -as it helped me personally- to understand later on even more complex and powerful AI algorithms and be a better machine learning engineer.

🗣️ Let’s talk!

Do you have any experience using the k-NN algorithm? What applications have you used it for? If anyone would like to know more or has questions, feel free to ask in the comments below.

A demonstration of the voting dynamics in k-NN classification, highlighting how the selection of the 'k' hyperparameter fundamentally reshapes the decision outcome by shifting the local majority influence.

A visualization of k-NN regression’s local averaging mechanism, demonstrating how a continuous output is predicted by triangulating the values of proximal data points within the feature space.
Tags: , , ,
Comment

The Magic of Generalized Linear Models.

Filed under: Articles, Artificial Intelligence — Leave a comment
August 24, 2026

A profound "red pill" moment for data scientists: stripping away the illusion of machine learning complexity to reveal that many advanced architectures are fundamentally rooted in the classic framework of Generalized Linear Models.
A profound “red pill” moment for data scientists: stripping away the illusion of machine learning complexity to reveal that many advanced architectures are fundamentally rooted in the classic framework of Generalized Linear Models.

Below are my personal thoughts and reflections on Andrew Ng’s Stanford Machine Learning lecture on Generalized Linear Models (GLMs), including the key ideas I found interesting and how I understood them.

There is something that feels almost like magic in the mathematics of GLMs.

By changing only the underlying assumed probability distribution of the hypothesis function, and expressing it in the form of an exponential family distribution, we can derive several well-known machine learning algorithms:

  • Ordinary Least Squares, using the Gaussian distribution
  • Logistic Regression, using the Bernoulli distribution
  • Softmax Regression, using the Multinomial distribution

This is a beautiful mathematical abstraction. We can essentially plug in a different probability distribution and derive a corresponding model. For example, the Gamma or Poisson distributions.

What makes this even more interesting is that these machine learning algorithms are quite different from one another. Each solves a different type of problem: regression, binary classification, or multiclass classification.

And yet, they can all be understood through the same underlying mathematical framework.

Isn’t that awesome?

The GLMs can be derived from the probabilistic interpretation of these machine learning algorithms. Once we have that probabilistic foundation, typical gradient descent or gradient ascent can be used to minimize the loss function J(x, y, θ) or maximize the log-likelihood, log(L(θ)), when fitting a GLM to a dataset.

There is another aspect I find particularly fascinating.

From the probabilistic interpretation of these regression and classification algorithms, the logistic function emerges naturally from the mathematics. It isn’t simply an arbitrary function that happens to work well. The probabilistic formulation helps answer the deeper “why?” behind its use.

And that is what makes GLMs so beautiful to me.

Different algorithms. Different probability distributions. Different problems.

Yet one elegant mathematical framework connects them all.

This is magic.

Tags: , , , , ,
Comment