“Everything changes and nothing stands still.” ― Heraclitus.
Welcome to efxa.org, my personal website! I am Efstathios, a computer scientist and philosopher living in Greece, and I have a deep passion for Software Engineering and Computational Intelligence. Enjoy my posts!
A haunting architectural metaphor for the year 2046, depicting “The Layer” as a towering, translucent stack of probabilistic logic that has replaced the rigid certainty of the compiler age with a fluid, weather-like interface of constant drift and systemic uncertainty.
Year is 2046.
In the beginning, AI was never meant to be deterministic. It was celebrated for its ambiguity, its ability to surprise, improvise, and feel almost alive in its inconsistency.
People called it creativity. Engineers called it progress. Investors called it the future. But something fundamental was being lost in the background.
In early computing, there was an unspoken rule: same input, same output. That principle was the foundation of trust. The C compiler era proved it. Software civilizations were built on reproducibility.
Machines do not “guess”. They execute.
When large language models (LLMs) arrived, that assumption was quietly abandoned.
At first, it didn’t matter. These systems wrote emails, summarized documents, and generated ideas. Variability was even marketed as a feature, “look how human it is”.
Even when engineers tried to enforce stability, temperature at zero, greedy decoding, fixed seeds, randomness still leaked through versions, hardware, and deployment pipelines.
The illusion of control was enough. So we scaled it.
We embedded these systems into workflows, then companies, then governments. We wrapped them in APIs and called them abstractions, even when they were not stable enough to deserve the name.
Each layer built on another probabilistic layer, until the stack resembled engineering, but behaved like weather.
The breaking point was subtle. Not a collapse, but a drift.
A legal assistant gave different interpretations of the same law under different server loads. A medical triage system produced slightly different urgencies for identical symptoms across regions.
Financial systems began averaging decisions that were never meant to be averaged. No single output was wrong. That was the problem, nothing was consistently right.
By the time people noticed, it was already too late to roll back. Everything depended on everything else.
The real tragedy wasn’t power, it was that AI was never built to be a reliable abstraction layer.
We assumed intelligence would converge toward consistency. Instead, it stayed fluid. And we built rigid systems on top of fluid foundations.
Some engineers warned us early. They said determinism was engineering, not intelligence.
Without it, you don’t get systems, you get phenomena. But they were dismissed as nostalgic, stuck in the compiler age.
Now, no one calls it artificial intelligence anymore.
They call it “The Layer”.
A shifting interface between human intent and machine behavior, powerful, unpredictable, impossible to fully reproduce.
Every attempt to stabilize it creates new fractures. Every patch introduces new uncertainty.
And in documentation from 2026, now little more than historical footnote, there is a forgotten line:
“If the same input does not always produce the same output, you are not building an abstraction. You are observing phenomena and negotiating with uncertainty”.
I’m very proud of what we’ve achieved at Medoid AI with Reality Engine. 🎉
This recognition is a great reflection of the hard work, persistence, and creativity of the Reality Engine team at Medoid AI, together with our partners at K2view.
Winning Gold at the AI & Data Awards 2025 is an important milestone for us and a strong recognition of our work in advancing synthetic data for AI and data-driven innovation.
Proud of the team, proud of Reality Engine, and excited for what’s ahead! 🚀
Locality Sensitive Hashing (LSH) is an efficient technique used for approximate KNN search, which is a core function of several billion-dollar companies such as Amazon, Google, Netflix, Spotify, and Uber. What makes LSH so popular is its ability to perform efficient and fast similarity search. In simple terms, LSH maps similar data points to the same hash bucket with high probability, allowing for fast similarity search while maintaining a reasonable level of accuracy.
🕵️♂️ How does it work?
The LSH algorithm leverages a family of hash functions and random hyperplanes to partition the feature space into regions, allowing similar data points to be grouped together in the same hash bucket. Each hash bucket represents a region in the feature space. Once the hash buckets are constructed, we can perform the approximate KNN search by identifying candidate neighbors based on their hash values. The accuracy and efficiency of LSH depend on the quality of hash functions and the number of hash buckets used. Increasing the number of buckets improves accuracy but also increases computational cost.
👨💻 My personal story
As a Machine Learning Engineer, I can say that working with large datasets and high-dimensional feature spaces can be a challenging task. That’s where techniques like LSH come into play, allowing us to perform efficient similarity search and reduce the computational cost of the search process. LSH has been an essential tool for me when working on NLP problems, especially when dealing with large document datasets. It has allowed me to perform document retrieval tasks and save computational resources.
📚 Who should learn it?
LSH is a widely used algorithm that enables efficient similarity search on large datasets, making it an essential tool for machine learning professionals. Learning LSH can provide practical skills and deeper understanding of fundamental concepts for those interested in building efficient and scalable information retrieval systems. In general, LSH can be applied to various applications, including recommendation systems, content-based image retrieval, and document retrieval.
🗣️ Let’s talk!
Have you used LSH in any of your machine learning projects? What were your experiences with it? I am here to help if you have any questions or would like to know more.
Example of Locally Weighted Regression, containing in the upper graphic the set of data points (x,y) (blue dots), query point (green line), local linear model (red line) and prediction (black dot). The graphic in the middle shows the activation area of the model. The corresponding weighting kernel (receptive field) is shown in the bottom graphic.
🤔 What is it?
LWR is a non-parametric regression technique used in machine learning to model the relationship between a dependent variable and one or more independent variables. It is ideal for non-linear relationship modeling between variables as it does not assume a specific functional form. LWR is memory-based, storing the entire training dataset and making predictions based on the closest observations to the new data point. Model parameters are computed individually for each new data point, allowing for more customized and accurate predictions. It has applications in finance, economics, ecology, engineering, and can be used for time-series forecasting, data smoothing, and data imputation.
🕵️♂️ How does it work?
To estimate the dependent variable for a new data point, LWR fits a line to the nearest observations in the training data. The weight given to each observation is determined by its distance from the new data point, with closer observations receiving higher weights. For each new data point, LWR estimates a Gaussian distribution for the closest observations and uses it to fit a line for prediction. The Gaussian distribution is used to estimate the weights of examples in the cost function of LWR.
👨💻 My personal experience
As a machine learning engineer, I have found LWR to be an effective technique for modeling non-linear relationships between variables and handling noisy data, which has led to improved accuracy in my predictions. I believe that LWR is an essential technique to have in your toolkit as you learn and develop your skills in AI and machine learning.
🤔 Why is it useful?
LWR is valuable for modeling complex, non-linear relationships between variables and handling outliers and noise in data. Since it is memory-based, LWR does not require a pre-defined model structure and can adapt to new data easily. Additionally, it can be used for both single-variable and multi-variable regression problems, making it a versatile tool for a range of applications.
📚 Who should learn it?
LWR is a valuable technique for anyone working on regression problems, particularly those with non-linear relationships between variables or noisy data. It can help improve prediction accuracy and handle outliers and other sources of noise in data. If you are interested in machine learning and AI, I highly recommend learning LWR as part of your toolkit.
🗣️ Let’s start a conversation!
Have you used LWR in your work? What are your thoughts on this technique? Do you have any questions or comments? Let’s start a conversation in the comments below!
An iterative journey from unlabeled chaos to emergent structure, illustrating how the K-means algorithm dynamically repositions centroids and refines group memberships until the latent geometry of the data reaches a stable convergence.
🤔 What is it?
K-means is a popular unsupervised clustering algorithm used to partition data points into K clusters. It is a part of the centroid-based clustering family of algorithms and is particularly useful when the data is unstructured and has no predefined labels. K-means can be applied to a wide range of problems, such as customer segmentation, market research, document clustering, image segmentation, and more.
🕵️♂️ How does it work?
Here are the steps of the K-means algorithm:
Step 1: Randomly initialize K cluster centroids.
Step 2: Assign each data point to its nearest centroid cluster.
Step 3: Update the centroid of each cluster based on the mean of the data points in the cluster.
Step 4: Repeat steps 2 and 3 until the centroids no longer move, or until a maximum number of iterations is reached.
The choice of initial cluster centroids can have a significant impact on the final clustering result. Therefore, good initialization is crucial to obtain optimal results. One common technique for initialization is called K-means++, which aims to choose initial centroids that are well-spaced and representative of the dataset.
👨💻 My personal story
K-means was one of the first unsupervised learning algorithms I learned as a Machine Learning Engineer. It helped me understand how to group similar data points and paved the way for me to learn more complex techniques (such as hierarchical clustering and density-based clustering). By using K-means, I was able to efficiently analyze large datasets and gain valuable insights for my projects.
💡 Why is it useful?
K-means is a powerful algorithm for clustering and identifying patterns in large datasets. It is particularly useful when the data is unstructured and has no predefined labels. However, it is more suitable for numerical data and requires feature scaling since distances are used to measure distance between data points. The choice of K needs to be specified by the data scientist based on the problem at hand. The elbow method can be used to select the optimal number of clusters for a dataset. K-means can provide quick and accurate results, making it a versatile and valuable tool for machine learning engineers.
📚 Who should learn it?
K-means is a fundamental algorithm that any beginner in machine learning should learn. It is also useful for experienced machine learning engineers who want to further improve their clustering and pattern recognition skills. Additionally, anyone who wants to analyze and make sense of large datasets should learn this algorithm.
🗣️ Let’s talk!
Have you used K-means before? What was your experience with the algorithm? Do you have any questions or suggestions for using K-means in different fields?
The Kernel Perceptron is a variant of the perceptron algorithm used in machine learning. It uses a kernel function to map the input data from its original feature space to a higher-dimensional feature space. By doing so, the algorithm can capture nonlinear relationships between the input variables and improve the accuracy of the classification. This makes it a powerful tool for solving complex classification problems in machine learning.
🕵️♂️ How does it work?
The Kernel Perceptron algorithm works by creating a separating hyperplane in the high-dimensional feature space. It iteratively updates the weights of the hyperplane to minimize the classification error on the training data. The kernel function is used to transform the input data into a higher-dimensional space, where the data may become linearly separable. The dot product between the transformed input data and the weight vector of the hyperplane is then computed to determine the classification of the input data. The kernel function can be chosen based on the nature of the input data and the desired classification performance.
💡 Why is it useful?
The Kernel Perceptron algorithm is useful because it can handle complex classification problems in which the classes are not linearly separable in the input space. It can also improve the accuracy of the classification by capturing nonlinear relationships between the input variables. It’s a valuable tool for solving real-world problems in computer vision (e.g. image recognition) and natural language processing (e.g. text classification, sentiment analysis, and language identification).
👨💻 My personal story
As a Machine Learning Engineer, I have used the Kernel Perceptron algorithm in several projects. It has helped me to solve complex classification problems and improve the accuracy of the classification. By using a kernel function, I was able to capture nonlinear relationships between the input variables and obtain better results. Learning the Kernel Perceptron algorithm has also helped me to understand Support Vector Machines (SVMs) which is a similar algorithm.
📚 Who should learn it?
The Kernel Perceptron algorithm is a useful knowledge to have for anyone learning AI. It can help you, as it helped me too personally, to understand later on even more complex and powerful AI algorithms and be a better machine learning engineer. If you are interested in solving complex classification problems and improving the accuracy of the classification, then the Kernel Perceptron algorithm is a valuable tool to learn.
🗣️ Let’s talk!
Have you used the Kernel Perceptron algorithm in your projects? What kernel functions have you found to be effective? Share your thoughts and experiences in the comments below!
The k-NN algorithm is a widely-used supervised learning technique that can handle classification and regression tasks. It is an instance-based learning method, which means that it stores the training instances and uses them at prediction time to make decisions. The algorithm uses proximity to determine the grouping of a new data point, based on the assumption that similar data points tend to cluster together. What makes it unique is its non-parametric nature, which means that it doesn’t make any assumptions about the underlying distribution of the data. As a result, it can be used to address non-linear separable data in classification tasks and non-linear regression problems. This property of k-NN makes it a valuable tool for various applications in machine learning.
🕵️♂️ How does it work?
The k-NN algorithm works by finding the k data points in the training set that are closest in distance (e.g. Euclidean, Manhattan, Cosine) to the new data point, and then assigning the new data point to the class that is most common among its k nearest neighbors in classification tasks or predicting the target value of the new data point based on the average of the target values of its k nearest neighbors in regression tasks. The value of k is a user-defined parameter that controls the number of neighbors considered for classification or regression.
💡 Why is it useful?
The k-NN algorithm is easy to understand and implement, making it a popular choice for beginners in machine learning. Since the k-NN algorithm has no assumptions about the underlying distribution of the data, it can be used for a wide range of data types and structures.
👨💻 My personal story
When I first started learning machine learning, the k-NN algorithm was one of the first algorithms I encountered. I remember feeling intimidated by some of the more complex algorithms, but k-NN seemed simple and intuitive. I used it to classify handwritten digits in the MNIST dataset and was impressed by its accuracy and speed. As I continued to study machine learning, I’ve moved on to more advanced techniques (e.g. Neural Networks). However, I still appreciate the simplicity and versatility of k-NN.
📚 Who should learn it?
The k-NN algorithm is a useful tool to have for anyone learning AI. It can help you -as it helped me personally- to understand later on even more complex and powerful AI algorithms and be a better machine learning engineer.
🗣️ Let’s talk!
Do you have any experience using the k-NN algorithm? What applications have you used it for? If anyone would like to know more or has questions, feel free to ask in the comments below.
🧐 As a Machine Learning Engineer, I find autoencoders to be a valuable tool for dimensionality reduction and feature learning.
📉 The autoencoder is an unsupervised learning technique that can help you reduce the dimensionality of your data by mapping it to a lower-dimensional representation. This reduced representation can be used for various tasks, such as data compression, denoising, and anomaly detection.
🤓 An autoencoder consists of two neural networks: an encoder and a decoder. The encoder maps the input data to a lower-dimensional representation, while the decoder reconstructs the original input data from the reduced representation. These two networks are trained together by minimizing the reconstruction error between the input data and its reconstructed output.
🔀 Variants of autoencoders, such as denoising autoencoders and variational autoencoders, have been developed to address different challenges and applications.
👉 Denoising autoencoders are used to reconstruct the original input from a corrupted version of the input. By training an autoencoder to reconstruct the original input from a corrupted input, we can learn a more robust representation of the input that is less affected by noise and other forms of corruption.
👉 Variational autoencoders are used to generate new data samples that are similar to the training data. By learning the underlying distribution of the data in the latent space, we can sample new data points from this distribution and use the decoder network to generate new samples that are similar to the original data.
🕵️ When only unlabeled data is available, autoencoders can be utilized for unsupervised learning. Autoencoders can learn a compressed representation of the input data that can be used for clustering. By labeling the cluster representative examples, we can reduce the amount of manual labeling required and create a pseudo-labeled dataset with label propagation. This pseudo-labeled dataset can then be used to train a machine learning model for a supervised task.
🏷️ When we have a large amount of unlabeled data and limited labeled data, autoencoders can be used for unsupervised pre-training of machine learning models. Autoencoders can learn a compressed representation of the unlabeled data, and their encoder part can be transferred to a new model architecture for fine-tuning on the limited labeled dataset. This approach can reduce the amount of labeled data needed for supervised learning while improving the accuracy of the trained model.
💬 If you’re learning AI, understanding autoencoders is a must-have knowledge that will help you in the long run. And, if you have any questions about it, feel free to ask me in the comments below!
An elegant bridge between high-dimensional complexity and spatial intuition, illustrating how a Self-Organizing Map (SOM) translates abstract input vectors into a topologically preserved 2D hexagonal landscape where geometric proximity mirrors data similarity.
🧠 SOMs are a powerful unsupervised machine learning technique that can be used for a variety of tasks, including clustering, dimensionality reduction, and data visualization. As a machine learning engineer, understanding SOMs has been incredibly useful to me in my work with high-dimensional data.
🌐 In a nutshell, SOMs work by mapping input data to a two-dimensional grid, preserving the topology and similarity of the input space. This allows for complex data structures to be visualized in a more simple and intuitive way.
💡 One of the key benefits of using SOMs is the ability to identify patterns in data that may not be immediately apparent. This can lead to insights and discoveries that may not have been possible otherwise.
📈 Additionally, SOMs can be used for clustering, allowing for the identification of groups within large datasets. This can be particularly useful in applications such as market segmentation or anomaly detection.
👨💻 Understanding SOMs is a valuable skill for anyone learning AI or working with high-dimensional data. It has certainly helped me to understand even more complex and powerful AI algorithms and be a better machine learning engineer.
🤔 Do you have experience using SOMs in your work? I would love to hear about your experiences and answer any questions you may have. Let’s start a conversation!
A visual representation of the “surprisingness” gap in language generation, illustrating how beam search’s optimization for high-probability sequences results in a flat, predictable output that fails to capture the high-variance entropy and creative spontaneity inherent in human speech.
As a machine learning engineer working in the depths of Natural Language Processing (NLP), I find myself constantly pondering a fundamental question: Can AI truly imitate human language?
On a technical level, our current state-of-the-art models like GPT are essentially sophisticated engines trained to predict the next word in a sequence. They generate text using a variety of heuristic and stochastic search techniques. For example, Beam Search, Random Sampling, or Top-K.
But does this statistical “guessing” actually capture the true complexity and nuance of how we speak and think?
The human brain doesn’t just predict; it generates language in ways that are often unpredictable, emotional, and genuinely surprising. This leads to a deeper mystery: Where does language actually originate, and what kind of probability distributions (if any) govern the linguistic generation within the human brain?
Is our current technology truly replicating language, or just a very convincing echo of it?
What do you think? Join the discussion below and share your thoughts on whether we are approaching true linguistic imitation or if the human spark remains out of reach. 🧐
Ockham’s Razor, illustrated through William of Ockham’s principle that simpler explanations are generally preferable to more complex ones–a concept with important implications for machine learning and model complexity.
Machine learning friends! 💡
William of Ockham, a 14th-century friar and philosopher, had a deep appreciation for simplicity. His famous principle, Ockham’s Razor, suggests that when several explanations are possible, we should prefer the simpler one.
In machine learning terms, the idea is remarkably relevant: the less complex an ML model is, the more likely it is that a good empirical result reflects something real rather than simply the peculiarities of the sample.
This is one reason why model complexity matters. A highly sophisticated model may fit the training data extremely well, but that does not necessarily mean it will generalize to new, unseen data. Sometimes, the simplest model that explains the evidence is also the most useful.
And we should never forget that philosophy often speaks first about the interesting, new, and seemingly crazy ideas that later influence science and technology.
The real art in science, machine learning, and software engineering is not necessarily to build the most complicated solution. It is to find solutions that are simple, explainable, robust, and elegant.
A “Morpheus” moment for AI researchers, challenging the belief that complex Transformer architectures are impenetrable black boxes and suggesting they can be fully understood through rigorous study.
Hi AI folks! 👋
The Transformer has become the de-facto standard architecture in Natural Language Processing (NLP). It is designed to work with sequential data and has completely changed the way we approach many language-related problems.
In NLP, Transformers can be used to achieve state-of-the-art 📈 results in tasks such as text classification, named-entity recognition, text summarization, question answering, machine translation, conversational chatbots 🤖, and much more.
However, the Transformer is also a fairly complex piece of machinery, bringing together several powerful concepts from years of Deep Learning research 🧠.
For anyone who wants to understand how Transformers work and dive deeper into the topic 📖, here are some excellent resources:
If you’re getting started with NLP or want to better understand the technology behind today’s Generative AI systems, these are great places to begin. 🚀
Sometimes we want to visualize multiple data distributions in a 2D scatter plot. However, the points of different distributions can overlap a lot or even hide each other. To visualize the distributions more clearly and get a better understanding of our data, we can use confidence ellipses.
Confidence ellipses summarize the main information of a dataset using only four parameters: center, width, height, and angle. The center of the ellipse can be calculated using the mean of the data. The width and height can be determined based on the number of standard deviations from the mean, while the angle can be computed using the covariance matrix of the data.
Technically, to implement a confidence ellipse, we can first create a simple ellipse and then transform it using 2D affine transformations, such as scaling, rotation, and translation.
In the images below, we can see two distributions from a sentiment analysis project: positive and negative sentences. We can plot a confidence ellipse around each distribution to make the overall structure easier to understand.
In this example, the distributions are relatively easy to grasp even without the ellipses, but this is not always the case. When distributions overlap heavily or contain many data points, confidence ellipses can provide a simple and effective way to visualize their shape, spread, and orientation.
Solid lines = loss functions; dotted lines = first derivatives; dashed lines = second derivatives. Curvature guides the learning rate: higher curvature → smaller α; lower curvature → larger α.
Machine Learning friends! 💡
Let’s study something interesting! 📚
In this Desmos graph 📈, I would like to explain how the inverse of the second derivative of a loss function, (1 ÷ f”(x)), could be used as an initial estimate of the learning rate (α) when training ML models using gradient descent.
In the example, we have 3 simple convex quadratic functions (f(x)), (g(x)) and (h(x)) as solid lines (blue, red and green). Their first derivatives are the dotted lines, while their second derivatives (f”(x)), (g”(x)) and (h”(x)) are the dashed horizontal lines.
The second derivative tells us how quickly the first derivative changes. In our example:
So, the higher the curvature, the smaller the learning rate, and vice versa.
Higher curvature → smaller learning rate
Lower curvature → larger learning rate
This gives us a simple intuition for how the curvature of a loss function can help us choose an initial learning rate for gradient descent. 🚀
A profound “red pill” moment for data scientists: stripping away the illusion of machine learning complexity to reveal that many advanced architectures are fundamentally rooted in the classic framework of Generalized Linear Models.
Below are my personal thoughts and reflections on Andrew Ng’s Stanford Machine Learning lecture on Generalized Linear Models (GLMs), including the key ideas I found interesting and how I understood them.
There is something that feels almost like magic in the mathematics of GLMs.
By changing only the underlying assumed probability distribution of the hypothesis function, and expressing it in the form of an exponential family distribution, we can derive several well-known machine learning algorithms:
Ordinary Least Squares, using the Gaussian distribution
Logistic Regression, using the Bernoulli distribution
Softmax Regression, using the Multinomial distribution
This is a beautiful mathematical abstraction. We can essentially plug in a different probability distribution and derive a corresponding model. For example, the Gamma or Poisson distributions.
What makes this even more interesting is that these machine learning algorithms are quite different from one another. Each solves a different type of problem: regression, binary classification, or multiclass classification.
And yet, they can all be understood through the same underlying mathematical framework.
Isn’t that awesome?
The GLMs can be derived from the probabilistic interpretation of these machine learning algorithms. Once we have that probabilistic foundation, typical gradient descent or gradient ascent can be used to minimize the loss function J(x, y, θ) or maximize the log-likelihood, log(L(θ)), when fitting a GLM to a dataset.
There is another aspect I find particularly fascinating.
From the probabilistic interpretation of these regression and classification algorithms, the logistic function emerges naturally from the mathematics. It isn’t simply an arbitrary function that happens to work well. The probabilistic formulation helps answer the deeper “why?” behind its use.
And that is what makes GLMs so beautiful to me.
Different algorithms. Different probability distributions. Different problems.
Yet one elegant mathematical framework connects them all.
Marble portrait heads of four philosophers in the British Museum. From foreground: Socrates, Antisthenes, Chrysippos, Epicurus.
🤖 🏛️ Have you ever wondered about the connection between AI and Ancient Greek Philosophy?
🧔 📜 The ancient Greek philosophers, such as Aristotle, Plato, Socrates, Democritus, Epicurus and Heraclitus explored the nature of intelligence and consciousness thousands of years ago, and their ideas are still relevant today in the age of AI.
🧠 📚 Aristotle believed that there are different levels of intelligence, ranging from inanimate objects to human beings, with each level having a distinct form of intelligence. In the context of AI, this idea raises questions about the nature of machine intelligence and where it falls in the spectrum of intelligence. Meanwhile, Plato believed that knowledge is innate and can be discovered through reason and contemplation. This view has implications for AI, as it suggests that a machine could potentially have access to all knowledge, but it may not necessarily understand it in the same way that a human would.
💭 💡 Plato also believed in the concept of Platonic forms, which are abstract concepts or objects that exist independently of physical experience. In the context of machine learning, models can be thought of as trying to learn these Platonic forms from experience, such as recognizing patterns and relationships in data.
⚛️ 💫 Democritus is known for his work on atoms and his belief that everything in the world can be reduced to basic building blocks. In the context of AI, this idea of reducing complex systems to their fundamental components has inspired the development of bottom-up approaches, such as deep learning, reinforcement learning and evolutionary algorithms.
🔥 🌊 Heraclitus, emphasized the idea that “the only constant is change”. This philosophy can provide valuable insights into the field of machine learning and the challenges of building models that can effectively handle non-stationary data. It highlights the importance of designing algorithms that are flexible and can adapt to changing environments, rather than relying on fixed models that may become outdated over time.
🔁 🏆 What about reinforcement learning? Aristotle, believed in the power of habituation and reinforcement to shape behavior. According to him, repeated exposure to virtuous actions can lead to the formation of good habits, which in turn leads to virtuous behavior becoming second nature. This idea is similar to the concept of reinforcement learning in AI, where an agent learns to make better decisions through repeated exposure to rewards or punishments for its actions. The similarities between Aristotle’s ideas and reinforcement learning show that the fundamental principles of reinforcement and habituation have been recognized and explored for thousands of years (psychological theories) and are still relevant today in discussions of both human behavior and AI.
🕊️ 💛 The Greek concept of the soul was also tied to intelligence and consciousness. Some philosophers believed that the soul is what gives a person their unique identity and allows them to think, feel, and make decisions. In the context of AI, this raises questions about whether a machine can truly have a soul and be conscious in the same way that a human is.
💀 💔 Another Greek philosopher who explored the nature of intelligence and consciousness is Epicurus. He believed that the mind and soul were made up of atoms, just like the rest of the physical world. He also believed that the mind and soul were mortal and would cease to exist after death. In the context of AI, Epicurus’ ideas raise questions about the nature of machine consciousness and whether it is possible for machines to have a kind of “mind” or “soul” that is distinct from their physical components. It also raises questions about the possibility of creating machines that are mortal or that can “die” in some sense. Epicurus’ ideas about the nature of the mind and soul are still debated today, and their relevance to the field of AI is an ongoing topic of discussion.
👁️ 🗣️ With the advancements in computer vision, natural language understanding, and deep learning in general, it’s more important than ever to consider these philosophical questions. For example, deep neural networks can analyze vast amounts of data and generate human-like responses, but do they truly understand the meaning behind the words they generate? This is where the debate between connectionist AI and symbolic AI comes in.
🧠 🕸️ Connectionist AI approaches to artificial intelligence are based on the idea that intelligence arises from the interactions of simple, interconnected processing units. On the other hand, Symbolic AI approaches are based on the idea that intelligence arises from the manipulation of symbols and rules. In the context of ancient Greek philosophy, connectionist AI could be seen as aligning more with the idea of knowledge being discovered through experience and observation, while symbolic AI aligns more with the idea of knowledge being innate and discovered through reason and contemplation.
🗣️ 💬 Humans use natural language, which includes speech and written text, to communicate with the world. However, natural language is not structured and can be unpredictable because it emerges naturally rather than being designed. If it was designed, natural language processing would have been solved a long time ago. Nowadays, deep neural language models are used to learn language from text by predicting the next token in a sequence. However, natural languages are continually evolving and changing, which makes them a moving target.
🧠 ❤️ Plato expressed a negative view towards human languages because he believed they are unable to fully capture the breadth and depth of a person’s thoughts and emotions. He asserted that genuine human communication, such as that which is conveyed through body language, eye contact, speech, and physical touch, is necessary for the progression of the human mind and the creation of new ideas. In today’s digital age, we are observing a decline in face-to-face human interaction, underscoring the importance of acknowledging the limitations of language. Despite the challenges inherent in natural language, it is crucial that we persist in developing tools to better comprehend and employ it. In addition to embracing technological advancements, it is essential that we strive to maintain meaningful human connections to preserve the richness of the human experience.
🏛️ 🤖 Have you ever heard of Talos? It is the first robot in Greek mythology! According to legend, Talos was a bronze automaton created by the legendary inventor Hephaestus to protect the island of Crete. He was said to be invulnerable and possessed immense strength, making him a formidable guardian. This idea of a powerful, indestructible automaton has been a recurring theme in science fiction and continues to inspire new developments in the field of AI and robotics.
⚖️ 🌍 It’s fascinating to consider how the myth of Talos reflects the enduring human fascination with creating machines that can surpass our own capabilities. The notion of a powerful automaton created to guard a specific territory mirrors modern ideas around autonomous systems that are designed to perform specific tasks and operate independently. However, while Talos was depicted as a purely mechanical creation, contemporary AI and robotics are built upon complex algorithms and data sets that allow machines to learn, adapt, and improve over time. By embracing the potential of these technologies, we have the opportunity to develop new solutions to some of the most pressing challenges facing our world today, from climate change to healthcare. As we continue to push the boundaries of what machines are capable of, it’s important to consider the ethical implications of creating truly intelligent and conscious entities. Some believe that the development of advanced AI could lead to a future where machines are not only equal to humans, but even superior to us. Others argue that such a scenario is unlikely, as true consciousness may only be achievable through biological processes that cannot be replicated in machines. As we navigate these complex questions, it’s crucial that we prioritize ethical considerations and work to ensure that the development of AI and robotics benefits humanity as a whole.
🙏🏽🤝 When it comes to ethics and morality, Aristotle is one of the most influential figures in history. His emphasis on the importance of human virtues and the pursuit of a “good life” has been a guiding principle for philosophers, theologians, and thinkers across generations. Aristotle’s teachings offer valuable insights into how we should approach the ethical implications of creating intelligent agents. One crucial aspect of this is ensuring that we treat these agents with the same respect and dignity that we would afford to any sentient being. This means avoiding the temptation to view AI as mere tools or objects to be used for our own purposes, and recognizing their autonomy as intelligent entities. Another important consideration is the issue of bias and discrimination, which can be inadvertently built into AI systems through the data and algorithms used to create them. Aristotle’s emphasis on justice and fairness can serve as a guide to ensure that we address these issues and ensure that the development of AI is done in a way that is equitable and beneficial for all of humanity.
🧠 🔬 Ending this article, it would be remiss not to mention Socrates, widely considered to be the wisest of all Greek philosophers. Not because of his vast knowledge, but because he recognized the limits of his own knowledge. “I know one thing, and that is that I know nothing,” Socrates famously stated. This concert is particularly relevant to the field of AI, as we continue to push the boundaries of what machines can do and be. As we develop increasingly complex and intelligent AI, the question arises whether it will be possible to prove the existence of self-awareness in these machines.
💬 These are just a few examples of how the ideas of ancient Greek philosophers and mythology can inform our understanding of AI. I hope these thoughts spark further conversation and contemplation on this fascinating topic.
🤔 What do you think about the potential for machines to truly be intelligent and conscious?
I am so proud that Crowdspeak‘s alpha demo is up and running!
On Crowdspeak you will be able to make your own crowd-questions. For this demo, there is just a fixed example question “What are the biggest challenges of working from home?”.
Visit Crowdspeak and type your own response! Crowdspeak will then take your response into account, and a new collective response will be generated in just a few minutes.
This collective response is a machine generated text made to express the majority of the individual responses given by the people!
Don’t forget to give us your feedback and to subscribe to the Crowdspeak newsletter for more!
skrobot is an open-source Python module I have created at Medoid AI for automating Machine Learning (ML) tasks. It is built on top of scikit-learn framework and follows object-oriented design (OOD) for ML task creation, execution, and reproducibility. Multiple ML tasks can be combined together to implement an experiment. It also provides seamless tracking and logging of experiments (e.g. saving experiments’ parameters).
It can help Data Scientists and Machine Learning Engineers:
to keep track of modelling experiments / tasks
to automate the repetitive (and boring) stuff when designing ML modelling pipelines
to spend more time on the things that truly matter when solving a problem
The current release of skrobot (1.0.13) supports the following ML tasks for binary classification problems:
It is an open-source R/Shiny app for time-series segmentation and changepoint detection tasks. The app acts as a front-end for R packages such as the magnificent changepoint package.
I had the pleasure of working together with wonderful colleagues at Medoid AI to develop this application. My role in this small project was more organizational and consulting.
For more information on the parameters and algorithms currently included in the app please read the following paper by Killick et al. For an overview of changepoint packages that may be included in the future please see the following page.
By passing millions of ImageNet images through InceptionV1 (state-of-the-art deep convolutional neural network) we can extract the image patches that make specific neurons from various convolutional layers to activate mostly.
By projecting the image patches to 2D using UMAP we can see what the neural network “sees” at the various layers.
This is a great way for explaining how a computer vision model makes its classification decision.
However, the following part of the article was the reason for my post:
“….There is another phenomenon worth noting: not only are concepts being refined as you move from layer to layer, but new concepts seem to be appearing out of combinations of old ones….”
This is how a world of complexity works.
We know that deep neural networks perform hierarchical feature learning and combine simpler features to learn more complex ones. This is one of the reasons why we use deep learning for audio, visual and textual data.
Deep learning can decompose the complexity of data!
Have you ever asked why we randomly initialize the weights of a neural network?
After you read this post you will know why!
When the weights of the neurons in a neural network’s layer are initialized to the same value then all neurons of the layer produce the same output in the forward propagation.
Furthermore, when doing backpropagation the gradients of the loss w.r.t to the weights of the layer are also the same values. So, when training happens with gradient descent the weights change in the same way.
Lastly, when gradient descent converges, the weight matrix of the layer contains the same values for all neurons and thus the neurons have learned the same thing.
To break this symmetry and allow the neurons of the layer (and in general in all layers) to learn new and different features we randomly initialize the weights!
Despite the promise of Feature Learning in Deep Learning -where dense, low-dimensional and compressed representations can be learned automatically from high-dimensional raw data- usually Feature Engineering is the most important factor for the success of an ML project.
Among ML practitioners the best learning algorithms and models are well-known and most effort is done to transform the data in order to express as much as possible the useful parts that model best the underlying problem.
In other words, the success of an ML project depends mostly on the data representation and not model selection / tuning. When the features are not garbage usually even the simplest algorithms with default hyperparameter values can give good results.
Conditional Language Models are not used only in Text Summarization and Machine Translation. They can be used also for Image Captioning!
Here is a great example from Machine Learning Masteryof how we can connect the Feature Extraction component of a SOTA Computer Vision model (e.g., VGG, ResNet, Inception, Xception, etc) with the input of a Language Model in order to generate the caption of an image.
The whole deep learning architecture can be trained end-to-end. It is a simple encoder-decoder architecture but it can be extended and improved using an attention interface between encoder and decoder, or even using Transformer layers!
Adding attention not only enables the model to attend differently various parts of the input image but also explain its decisions. For each generated word in output caption we can visualize the attended visual part of input image.
In Machine Learning we use minimum-disturbance learning algorithms (e.g., LMS, Backprop) to tune the trainable parameters of topological architectures. Instead of programming a model we expose it to observations and let the algorithm find its optimal parameters. So, in this approach the solution to a task is the set of trainable parameters.
In biology, precocial species are those whose young already possess certain abilities from the moment of birth. There is evidence to show that lizard and snake hatchlings already possess behaviors to escape from predators. Shortly after hatching, ducks are able to swim and eat on their own, and turkeys can recognize predators. These species already have abilities before they are even exposed to the world.
Neuroevolution of augmenting topologies is a research field that aims to simulate exactly this behavior. Specifically, neural network architectures can be found that perform various machine learning tasks without weight training. So, in this approach the solution to a task is the topological architecture.
Natural languages (speech and text) are the way we communicate as species. They help us to express whatever is inside to the outer world.
Natural languages are not designed. They emerge. Thus, they are messy and semi-structured. If they were designed, NLP would be already solved, using context-free grammars and finite automata by linguists 50 years ago.
Today, we are trying to artificially “learn” language from text using state-of-the-art Deep Neural Language Models that behave probabilistically, predicting the next token in a sequence.
Moreover, natural languages are not static. They evolve and change. Different words can be used in different times with different meaning. It is a moving target.
Plato, the Greek philosopher was negative with “languages” -despite the fact the he has written so much- because a language cannot express the fullness of a human mind, of a person. Socrates and many philosophers from the Peripatetic school never wrote texts. The only way they were communicate was by real human communication (body language, eyes, speech, touch). Only with this way, a human mind and heart can evolve and create new worlds.
However, we are living in a century where everything is either digitalised or written and human communication goes to minimum.
Assume a train/validation/test split and an error metric for evaluating a machine learning model.
In case of high validation/test errors something is not working well and we can try to diagnose if it is a high-bias or high-variance problem.
When both the training and validation errors are high then we have a high-bias model (underfitting). When we have a high validation error and low training error we have a high-variance model (overfitting).
Also, there is a case where both validation and training errors are low but test error is high. This can happen either because we test the model in data from a different world (different data distribution) or we have overfitted to the hyperparameters on validation data, same as when overfitting the models’s parameters on training data.
Machine Learning practitioners always focus on finding in an unbiased way models where validation and training errors are low. When it happens, we say “this a good fit” and then we monitor the model on unseen test data to see how it performs in the real world.
Linear / Logistic regression learning algorithms can be used to learn surfaces to fit non-linear data (either for regression or classification).
It can be done by adding extra polynomial terms (quadratic, cubic, etc) in the dataset using the existing features. The approximated surfaces can be circles, ellipses, parabolas and hyperbolas.
However, (1) it is difficult to approximate very complex non-linear surfaces. Also, when the features are many then (2) it is very computationally expensive to calculate all polynomial terms (eg: for a 100×100 grayscale images we need 100000 quadratic terms).
Usually, due to the limitations (1) and (2) we either perform dimensionality reduction to reduce the number of features (eg: eigenfaces for face recognition) or use more complex non-linear approximators such as Support Vector Machines and Neural Networks.
In Linear Regression when using the loss function MSE it is always a bowl-shaped convex function and gradient descent can always find the global minima.
In Logistic Regression if we use the MSE then it will not be a convex function because the hypothesis function is non-linear (it uses a sigmoidal activation). Thus, it will be harder for gradient descent to find the global minima. However, if we use the cross-entropy loss it will be convex and gradient descent can easily converge to global minima!
Support Vector Machines have also convex loss function.
We should always use a convex loss function so that gradient descent can converge to the global minima (local optima free).
Neural Networks are very complex non-linear mathematical functions and the loss function most often is non-convex, thus it is usual to stuck in a local minima. However, most optimization problems in Neural Networks are due to long plateau and saddle points rather than local minima. For such problems advanced gradient descent optimization variants were invented (eg: Momentum, Adam, RMSprop).
How much AI and Machine Learning are related to ancient Greek Philosophy? More than you imagine!
In this article you will learn about the never ending debate between Platonic and Aristotelian ideas of the reality of the world.
In my opinion, I agree with Plato saying that the world we experience is just a shadow of the real one. One possible interpretation can come from modern physics with theories such a) multidimensional world (e.g. string theory, theory of relativity) b) quantum physics and c) non-Euclidean geometrical spaces. Things are not as they appear in our brains. We see only a projection of the real world and we are totally living in a matrix limited by our physiology.
However, I agree also with Aristotle saying that the forms of entities reside only in the physical world. So, the “ideal” forms are learned by our brains and do not pre-exist.
The article touches the issue of Universals and Particulars in Philosophy and tries to explain them also in terms of Machine Learning. Machine Learning models learn to separate the universal (signal) from the particulars (noise) from observed data of the world.
In this context a programmer creates a program to solve a specific problem using data structures and algorithms. After that, we can run the program with some input data to produce some output. Let’s call this Traditional Programming.
Nowadays, with AI we are “showing” computers.
In this context a programmer shows to a learning algorithm input and output data for a specific problem to be solved. The program is automatically created by the learning algorithm. After that, we can run the program with some new input data to produce its output. Let’s call this Machine Learning.
Nowadays, computer science departments in universities teach most the Traditional Programming mindset. For boosting an AI-powered society it would be wise to teach equally Machine Learning!
E. Chatzikyriakidis, C. Papaioannidis and I. Pitas, “Adversarial Face De-Identification,” 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 2019, pp. 684-688
This project aims to the development of an open-source experimental prototype for solving and generating Sudoku puzzles by using only the strength of Genetic Algorithms. This is not a general purpose GA framework but a specific GA implementation for solving and generating Sudoku puzzles. The mechanics of the GA are based on the theoretical scientific paper “Solving and Rating Sudoku Puzzles with Genetic Algorithms” of Timo Mantere and Janne Koljonen. From the first moment, I liked the paper. So, I implemented it in Python. Also, I have add some variations to the algorithm in order to be more efficient. This project can be used in order to solve or generate new NxN Sudoku puzzles with N sub-boxes (e.g. 4×4, 9×9, etc).
The project “PGASystem” (Parallel Genetic Algorithms System) is an under development system based on the client / server architecture and can be used to implement and study of parallel genetic algorithms.
Thoughts on Automatic Software Repairing and Genetic Programming
In the field of Software Engineering enough emphasis is given on the development of methodologies and mechanisms for the design of optimal software systems. Moreover, the quality of a software system can be assessed by carrying out appropriate metrics. Key features under study during the evaluation of a system are reliability, stability, security, portability and usability. The quality of a software system depends mainly on the time spent, expenses made, debugging and testing techniques used etc.
The Ant Colony System algorithm is an example of an Ant Colony Optimization method from the field of Swarm Intelligence, Metaheuristics and Computational Intelligence. Ant Colony System is an extension to the Ant System algorithm and is related to other Ant Colony Optimization methods such as Elite Ant System, and Rank-based Ant System.
The Cultural Algorithm is an extension to the field of Evolutionary Computation and may be considered a Meta-Evolutionary Algorithm. It more broadly belongs to the field of Computational Intelligence and Metaheuristics. It is related to other high-order extensions of Evolutionary Computation such as the Memetic Algorithm.
Memetic Algorithms have elements of Metaheuristics and Computational Intelligence. Although they have principles of Evolutionary Algorithms, they may not strictly be considered an Evolutionary Technique. Memetic Algorithms have functional similarities to Baldwinian Evolutionary Algorithms, Lamarckian Evolutionary Algorithms, Hybrid Evolutionary Algorithms, and Cultural Algorithms. Using ideas of memes and Memetic Algorithms in optimization may be referred to as Memetic Computing.
Within the framework of the course “Computer Networks III – Theory” (Department of Informatics and Communications, T.E.I. of Central Macedonia) we were asked to write a presentation related to the content of the course. The topic of my presentation was Genetic Routing.
I quote below a personal portable implementation (in C++) of a classic Differential Evolution algorithm used to maximize the function f(x) = sin(x) in the domain 0 <= x <= 2pi. You can compile the program with the g++ compiler.
I quote below a personal portable implementation (in C++) of a classic genetic algorithm (evolutionary algorithm) used to maximize the function f(x, y) = sin(x) * sin(y) in the domain 0 <= x, y <= 2pi. You can compile the program with the g++ compiler.
I quote below a personal portable implementation (in C++) of a classic genetic algorithm (evolutionary algorithm) used to maximize the function f(x) = sin(x) in the domain 0 <= x <= 2pi. You can compile the program with the g++ compiler.
I quote below a personal portable implementation (in C++) of a classic genetic algorithm (evolutionary algorithm) used to maximize the function f(x) = sin(x) in the domain 0 <= x <= 2pi. You can compile the program with the g++ compiler.
I quote below a personal portable implementation (in C++) of a classic genetic algorithm (evolutionary algorithm) used to maximize the function f(x) = sin(x) in the domain 0 <= x <= 2pi. You can compile the program with the g++ compiler.
The Artificial Immune Recognition System belongs to the field of Artificial Immune Systems, and more broadly to the field of Computational Intelligence. It was extended early to the canonical version called the Artificial Immune Recognition System 2 (AIRS2) and provides the basis for extensions such as the Parallel Artificial Immune Recognition System [Watkins2004]. It is related to other Artificial Immune System algorithms such as the Dendritic Cell Algorithm, the Clonal Selection Algorithm, and the Negative Selection Algorithm.
The Self-Organizing Map algorithm belongs to the field of Artificial Neural Networks and Neural Computation. More broadly it belongs to the field of Computational Intelligence. The Self-Organizing Map is an unsupervised neural network that uses a competitive (winner-take-all) learning strategy. It is related to other unsupervised neural networks such as the Adaptive Resonance Theory (ART) method. It is related to other competitive learning neural networks such as the the Neural Gas Algorithm, and the Learning Vector Quantization algorithm, which is a similar algorithm for classification without connections between the neurons. Additionally, SOM is a baseline technique that has inspired many variations and extensions, not limited to the Adaptive-Subspace Self-Organizing Map (ASSOM).
The Genetic Algorithm is an Adaptive Strategy and a Global Optimization technique. It is an Evolutionary Algorithm and belongs to the broader study of Evolutionary Computation. The Genetic Algorithm is a sibling of other Evolutionary Algorithms such as Genetic Programming, Evolution Strategies, Evolutionary Programming, and Learning Classifier Systems. The Genetic Algorithm is a parent of a large number of variant techniques and sub-fields too numerous to list.
Efstathios Chatzikyriakidis 