Archive for August 26, 2026


Confidence Ellipses: Visualizing Data Distributions More Clearly.

Filed under: Articles, Artificial Intelligence — Leave a comment
August 26, 2026

Sometimes we want to visualize multiple data distributions in a 2D scatter plot. However, the points of different distributions can overlap a lot or even hide each other. To visualize the distributions more clearly and get a better understanding of our data, we can use confidence ellipses.

Confidence ellipses summarize the main information of a dataset using only four parameters: center, width, height, and angle. The center of the ellipse can be calculated using the mean of the data. The width and height can be determined based on the number of standard deviations from the mean, while the angle can be computed using the covariance matrix of the data.

Technically, to implement a confidence ellipse, we can first create a simple ellipse and then transform it using 2D affine transformations, such as scaling, rotation, and translation.

In the images below, we can see two distributions from a sentiment analysis project: positive and negative sentences. We can plot a confidence ellipse around each distribution to make the overall structure easier to understand.

In this example, the distributions are relatively easy to grasp even without the ellipses, but this is not always the case. When distributions overlap heavily or contain many data points, confidence ellipses can provide a simple and effective way to visualize their shape, spread, and orientation.

Confidence Ellipses for Positive and Negative Sentiment Distributions

Confidence Ellipses for Positive and Negative Sentiment Distributions
Tags: , ,
Comment

Using the Second Derivative to Estimate the Learning Rate.

Filed under: Articles, Artificial Intelligence — Leave a comment

Solid lines = loss functions; dotted lines = first derivatives; dashed lines = second derivatives. Curvature guides the learning rate: higher curvature → smaller α; lower curvature → larger α.
Solid lines = loss functions; dotted lines = first derivatives; dashed lines = second derivatives. Curvature guides the learning rate: higher curvature → smaller α; lower curvature → larger α.

Machine Learning friends! 💡

Let’s study something interesting! 📚

In this Desmos graph 📈, I would like to explain how the inverse of the second derivative of a loss function, (1 ÷ f”(x)), could be used as an initial estimate of the learning rate (α) when training ML models using gradient descent.

In the example, we have 3 simple convex quadratic functions (f(x)), (g(x)) and (h(x)) as solid lines (blue, red and green). Their first derivatives are the dotted lines, while their second derivatives (f”(x)), (g”(x)) and (h”(x)) are the dashed horizontal lines.

The second derivative tells us how quickly the first derivative changes. In our example:

1÷g′′(x)=0.051 \div g”(x)=0.05
1÷f′′(x)=0.51 \div f”(x)=0.5
1÷h′′(x)=51 \div h”(x)=5

So, the higher the curvature, the smaller the learning rate, and vice versa.

Higher curvature → smaller learning rate

Lower curvature → larger learning rate

This gives us a simple intuition for how the curvature of a loss function can help us choose an initial learning rate for gradient descent. 🚀

Tags: , , , ,
Comment