There are two kinds of truths: those of reasoning and those of fact. The truths of reasoning are necessary and their opposite is impossible; the truths of fact are contingent and their opposites are possible. (The Monadology of Leibniz)
The past few months have made me realize more and more about the sheer number of fundamental ideas that can be traced back, atleast in part to Gottfried Leibniz. The ones that I find most striking (other than his countless other contributions in calculus, geology, physics, philosophy, rationality, theology etc.) given what has been on my mind recently are his ideas in formal systems, symbolic logic and Kolmogorov Complexity.
It is not incorrect to think that Leibniz could be considered the first computer scientist to have lived. His philosophy centered around having a universal language of symbols combined with a calculus of reasoning, something from which modern symbolic logic and notation has directly descended from. An interest in mathematical logic also directly leads to an interest in the “mechanization of thought”, the same could be seen in Leibniz who was a prolific inventor of calculating devices.
His elucidation of what might be called the earliest ideas in Algorithmic Information Theory/Kolmogorov Complexity is equally intriguing. While he explicates them in depth, what he essentially talks about is the complexity of an “explanation” (basically Kolmogorov Complexity). And that an arbitrarily complex explanation is no explanation at all. I also find this idea similar to the bias-variance tradeoff in machine learning and the problem of overfitting. What I find striking is the clarity with which these ideas had been expressed and how little they have changed in essence in 3 centuries (though formalized).
In my intrigue, I have tried to read his very short works – Discours de métaphysique and The Monadology. While these have been debated over the centuries, their fundamental nature is unquestioned and are a recommended read. More recently I mentioned that I had been really intrigued by Leibniz for some months to my teacher from the undergraduate days. He was instrumental in getting me to read Cybernetics (by Norbert Wiener) and in Signal Processing in general. He was quick to point to this paragraph from Wiener’s book that I did not even remember reading:
Since Leibniz there has perhaps been no man who has had a full command of all the intellectual activity of his day. Since that time, science has been increasingly the task of specialists, in fields which show a tendency to grow progressively narrower. A century ago there may have been no Leibniz, but there was a Gauss, a Faraday, and a Darwin. Today there are few scholars who can call themselves mathematicians or physicists or biologists without restriction.
A man may be a topologist or an acoustician or a coleopterist. He will be filled with the jargon of his field, and will know all its literature and all its ramifications, but, more frequently than not, he will regard the next subject as something belonging to his colleague three doors down the corridor, and will consider any interest in it on his own part as an unwarrantable breach of privacy.
– Norbert Wiener, Cybernetics or the Control and Communication in the Animal and the Machine. 1948.
Since the mention of Wiener has occurred, it might also be useful to consider his trenchant advice just before the start of the above passage:
For many years Dr. Rosenblueth and I had shared the conviction that the most fruitful areas for the growth of sciences were those which had been neglected as a no-man’s land between the various established fields […]
_________________________
Hey thanks Shubhendu for the Post. Had known that Leibniz was great contributor to Maths & Science who had been overshadowed by other scientists of his era (mostly Newton) either by achievement or by politics.
But his contributions to formal systems & symbolic logic are quite note worthy of which I had no idea.
Hi Jitin,
Thanks for reading and (I’m assuming ;-) liking it.
I had very often heard phrases like Leibniz was the greatest logician from the time of Aristotle to 1857 etc etc. However, over the past year I have been trying to teach myself Kolmogorov Complexity little by little. Only now do I realize how fundamental his contributions to these other areas were.
Hi,
I enjoyed reading this. Never knew about Leibniz’s these contributions & even so at his time :)
Just for interest what exactly is “Kolmogorov Complexity”, u know wiki blacked out. (preferred in 140 characters :) )
Jitin,
It is a sort of a descriptive complexity. i.e How do you can measure how complex an object is? One way of doing this is to measure the computational resources completely necessary to specify the object (implicit assumption – any object can be specified in binary for maintaining generality).
As an example if I were to repeat ‘A’ 100 times, The string does have 100 characters, But I could specify it in much lesser (say A-100 needs 4). At the same time a string generated completely at random would have a description of the same size (it can not be less).
It has a fundamental relationship with areas such as machine learning or theoretical CS. (Infact the basic idea of what is now called Kolmogorov complexity orignated in what is recognized as the first paper in machine learning by Ray Solomonoff. The article linked to has some description on how it is fundamental to machine learning.)
Thanks Shubhendu,
Now I get it, at least the context is clear.
Reblogged this on wernerschwartz.