Feeds:
Posts

## Some Interesting Courses: Neural Nets, Visualization, ML and Astrophysical Chemistry

Here are a number of interesting courses, two of which I am looking at for the past two weeks and that i would hopefully finish by the end of August-September.

## Introduction to Neural Networks (MIT):

These days, amongst the other things that I have at hand including a project on content based image retrieval. I have been making it a point to look at a MIT course on Neural Networks. And needless to say, I am getting to learn loads.

I would like to emphasize that though I have implemented a signature verification system using Neural Nets, I am by no means good with them. I can be classified a beginner. The tool that I am more comfortable with are Support Vector Machines.

I have been wanting to know more about them for some years now, but I never really got the time or you can say the opportunity. Now that I can invest some time, I am glad I came across this course. So far I have been able to look at 7 lectures and I should say that I am MORE than very happy with the course. I think it is very detailed and extremely well suited for the beginner as well as the expert.

The instructor is H. Sebastian Seung who is the professor of computational neuroscience at the MIT.

The course has 25 lectures each one packed with a great amount of information. Meaning, the lectures might work slow for those who are not very familiar with this stuff.

The video lectures can be accessed over here. I must admit that i am a little disappointed that these lectures are not available on you-tube. That’s because the downloads are rather large in size. But I found them worth it any way.

The lectures cover the following:

Lecture 1: Classical neurodynamics
Lecture 2: Linear threshold neuron
Lecture 3: Multilayer perceptrons
Lecture 4: Convolutional networks and vision
Lecture 5: Amplification and attenuation
Lecture 6: Lateral inhibition in the retina
Lecture 7: Linear recurrent networks
Lecture 8: Nonlinear global inhibition
Lecture 9: Permitted and forbidden sets
Lecture 10: Lateral excitation and inhibition
Lecture 11: Objectives and optimization
Lecture 12: Excitatory-inhibitory networks
Lecture 13: Associative memory I
Lecture 14: Associative memory II
Lecture 15: Vector quantization and competitive learning
Lecture 16: Principal component analysis
Lecture 17: Models of neural development
Lecture 18: Independent component analysis
Lecture 19: Nonnegative matrix factorization. Delta rule.
Lecture 20: Backpropagation I
Lecture 21: Backpropagation II
Lecture 22: Contrastive Hebbian learning
Lecture 23: Reinforcement Learning I
Lecture 24: Reinforcement Learning II
Lecture 25: Review session

The good thing is that I have formally studied most of the stuff after lecture 13 , but going by the quality of lectures so far (first 7), I would not mind seeing them again.

Course Video Lectures.

Prof H. Sebastian Seung’s Homepage.

_____

## Visualization:

This is a Harvard course. I don’t know when I’ll get the time to have a look at this course, but it sure looks extremely interesting. And I am sure a number of people would be interested in having a look at it. It looks like a course that be covered up pretty quickly actually.

The course description says the following:

The amount and complexity of information produced in science, engineering, business, and everyday human activity is increasing at staggering rates. The goal of this course is to expose you to visual representation methods and techniques that increase the understanding of complex data. Good visualizations not only present a visual interpretation of data, but do so by improving comprehension, communication, and decision making.

In this course you will learn how the human visual system processes and perceives images, good design practices for visualization, tools for visualization of data from a variety of fields, collecting data from web sites with Python, and programming of interactive visualization applications using Processing.

The topics covered are:

• Data and Image Models
• Visual Perception & Cognitive Principles
• Color Encoding
• Design Principles of Effective Visualizations
• Interaction
• Graphs & Charts
• Trees and Networks
• Higher-dimensional Data
• Unstructured Text and Document Collections
• Images and Video
• Scientific Visualization
• Medical Visualization
• Social Visualization
• Visualization & The Arts

Course Syllabus.

Lectures, Slides and other materials.

Video Lectures

_____

This is one course that I would  be looking at some parts of after I have covered the course on Neural Nets.  I am yet to glance at the first lecture or the materials, so i can not say how they would be like. But I sure am expecting a lot from them going by the topics they are covering.

The topics covered in a broad sense are:

• Bayesian Networks
• Statistical NLP
• Reinforcement Learning
• Bayes Filtering
• Distributed AI and Multi-Agent systems
• An Introduction to Game Theory

Course Home.

_____

## Astrophysical Chemistry:

I don’t know if I would be able to squeeze in time for these. But because of my amateurish interest in chemistry (If I were not an electrical engineer, I would have been into Chemistry), and because I have very high regard for Dr Harry Kroto (who is delivering them) I would try and make it a point to have a look at them. I think I’ll skip gym for some days to have a look at them. ;-)

[Nobel Laureate Harry Kroto with a Bucky-Ball model – Image Source : richarddawkins.net]

Dr Harold Kroto’s Homepage.

Astrophysical Chemistry Lectures

_____

Onionesque Reality Home >>

I started writing this post on 18 June with the singular aim of posting it by 22 June. The objective of this post was to celebrate the life and ideas of Tommy Gold (May 22, 1920 – June 22, 2004) on his fourth death anniversary. But after that I did not have much access to the Internet for reasons I had posted about earlier, and so sadly I missed that date. After that I did not edit and post it as I thought there would be little point. Now I think it is okay to  post it instead of deleting it all together. A tribute to Thomas Gold would still be the aim though I regret I could not post in time.

[Image Source]

Quoting Thomas Gold (Source):

New ideas in science are not always right just because they are new. Nor are the old ideas always wrong just because they are old. A critical attitude is clearly required of every scientist. But what is required is to be equally critical to the old ideas as to the new. Whenever the established ideas are accepted uncritically, but conflicting new evidence is brushed aside and not reported because it does not fit, then that particular science is in deep trouble – and it has happened quite often in the historical past. If we look over the history of science, there are very long periods when the uncritical acceptance of the established ideas was a real hindrance to the pursuit of the new. Our period is not going to be all that different in that respect, I regret to say.

This paragraph reminds me of a post on Gaping Void, a blog that I just discovered two days back on the fantastic Reasonable Deviations. The post, titled Good Ideas Have Lonely Childhoods is highly recommended to read, as a vast majority of good ideas are heretical and this post is on a heretic. Infact this post on Gaping Void prompted me to publish this forgotten draft!

Thomas Gold was a true renaissance man, a brilliant polymath and a controversial figure who Freeman Dyson has described as a modern heretic. Gold was born as an Austrian and was educated in Switzerland and the UK, Initially he worked with Hermann Bondi and Fred Hoyle and then later accepted an appointment with the prestigious Cornell University and remained there till his death.

Gold portrays the typical rebel scientist, with a penchant for controversy and working against general and strongly held theories. Gold worked across a large number of fields- Cosmology, Biophysics, Astrophysics, Geophysics, Space Engineering etc. Throughout his career Gold never cared about being wrong or of the opposition. He had this knack of turning out to be right. He however was not afraid to be wrong, infact he has been very famously wrong two times and he took both times in good humor. Such was his intellect that he never cared of any opposition and his ideas have always been very interesting. I hope to chronicle some of his major ideas here.

Coming back, as I said he has been famously wrong two times:

1. First was the steady state theory. Gold along with Fred Hoyle and Hermann Bondi developed and published the steady state theory of the universe in 1948. The three thought that it was impossible to think that all of matter could be created out of an initial singularity. The theory proposed that new matter is created continuously and this accounts for the constant density of the expanding universe. Though this seems to have violated the first law of thermodynamics the steady state had a number of supporters in the 50s and the 60s but the discovery of the cosmic background radiation which basically is a remnant of the big bang or explosion was the first major blow to it and over time its wide acceptance declined to only a very few cosmologists like Jayant V. Narlikar, who very recently have proposed alternatives and modifications to the original idea of steady state like the quasi steady state. However whatever said and done, the competition between the Big Bang and the Steady State spurred a lot of research which ultimately has helped us understand the cosmos better as good competition always does.

2. His second major incorrect idea was proposed in 1955, when he said that moon’s surface was covered with a fine rock powder that is electro-statically supported. He later said that astronauts would sink as soon as they landed on the moon. His theory influenced the design of the American Surveyor lunar landing probes to a very large extent. But their precautions were excessive and most of the fears were unfounded, though when the Apollo 11 crew bought back soil samples from the moon, it was indeed powdery though nowhere close to the extent Gold had proposed it to be. However a lot of astronomers credit a lot of development in planetology in subsequent years to Gold’s initial work and ideas on the lunar regolith.

[The famous photo of the footprint on the Lunar Surface: The Lunar soil was powdery as predicted by Gold but nowhere to the extent he had thought so. Image Source : Wikipedia Commons]

On both the occasions Gold took “defeat” in good humor, the trademark of a good scientist is that he is never afraid to be wrong. He once remarked:

Science is no fun, if you are never wrong!

In choosing a hypothesis there is no virtue in timidity and no shame in sometimes being wrong.

The second quote is not supposed to be humorous by the way.

On most occasions however, Thomas Gold had this knack of turning out to be right inspite of facing intense criticism initially. Some of his heretical ideas that turned out right were:

1. Pitch Discriminative Ability of the Ear: One of the first of Tommy Gold’s ideas that was received with much hostility and was summarily rejected by the experts of the time was his theory and experiments on hearing and pitch discrimination. In 1946 immediately after the great war, Gold got interested in the ability of the human ear to discriminate the pitch of musical sounds. It was a question that was perplexing the auditory physiologists of the time, and Gold fresh from working with the royal navy on radars and communications thought of the physiology of hearing in those terms. The human ear can tell the difference when a pure tone changes by as little as one percent. Gold thought that the ear contained a set of resonators finely tuned, whereas the prevailing view of the time was that the internal structure of the ear was too weak and flabby to resonate and all the interpretation of the sounds and tones happened in the brain, with the information being communicated by neural signals.

Gold designed a very simple and elegant experiment to prove the experts, the professional auditory physiologists wrong. The experiment has been described by Freeman Dyson in his book, The Scientist as Rebel as he himself was a part of the experiment. Prof Freeman writes:

He (Gold) fed into the headphones a signal consisting of short pulses of a pure tone, separated by intervals of silence. The silent intervals were atleast ten times as long as the period of the pure tone. The pulses were all of the same shape, but they had phases that could be reversed independently….Sometimes Gold gave all the pulses the same phase and some times he alternated the phases so that the even pulses had one phase and the odd pulses had the opposite phase. All I had to do was to sit with the headphones on my ears and listen while Gold put in the signals with either constant or alternating phases. I had to tell him from the sound whether the phase was constant or alternating. When the silent intervals between pulses was ten times the period of the pure tone, it was easy to tell the difference. I heard a noise like a mosquito, a hum and a buzz sounding together, and the quality of the hum changed noticeably when the phases were changed from constant to alternating. We repeated the trials with longer silent intervals. I could still tell the difference, when the silent interval was as long as thiry periods.

This elegant experiment showed that the human ear could remember the phase of a signal after it has stopped for thirty times the period of the signal and proved that pitch discrimination was done not in the brain but in the ear. To be able to remember the phase, the ear should have finely tuned resonators that continue to vibrate during the period of silence.

Now armed with experimental evidence for his theory that pitch discrimination was done in the ear, Gold also had a theory on how there could be very finely tuned resonators made up of the weak and flabby material in the ear. He proposed that the ear involved an active – not a passive – receiver, one in which positive feedback, not just passive detection is involved. He said that the ear had an electrical feedback system, the mechanical resonators are coupled to the electrically powered sensors so that the overall system works like an active tuned amplifier. The positive feedback would counteract the dissipation taking place in the flabby internal structure of the ear.

Gold’s findings and ideas were rejected by the experts of the field, who said Gold was an ignorant outsider with absolutely no knowledge or training in physiology. Gold however always maintained he was right. Thirty years later, auditory physiologists armed with more sophisticated tools discovered that Gold was indeed correct. The electrical sensors and the feedback system in the ear were identified.

Gold’s two papers on hearing published in 1948 remain highly cited to this day.

2. Pulsars: One of his ideas that was rather quickly accepted was his idea on what a Pulsar was. After being discovered by radio astronomers Gold proposed that they were rotation neutron stars.

[A schematic of a Pulsar. Image Source: Wikipedia Commons]

After some initial disapproval this idea was accepted almost immediately by the “experts”. Gold himself has written this on this matter in an article authored by him titled The Inertia of Scientific Thought:

Shortly after the discovery of pulsars I wished to present an interpretation of what pulsars were, at this first pulsar conference: namely that they were rotating neutron stars. The chief organiser of this conference said to me, “Tommy, if I allow for that crazy an interpretation, there is no limit to what I would have to allow”. I was not allowed five minutes floor time, although I in fact spoke from the floor. A few months later, this same organiser started a paper with the sentence, “It is now generally considered that pulsars are rotating neutron stars”.

3. The Arrow of Time: In the 60s Gold wrote extensively on The Arrow of Time, and held the view that the universe will re collapse someday and that the arrow of time will reverse. His views remain controversial till today and a vast majority of cosmologists don’t even take it seriously. It remains to be seen if Gold’s hypothesis would be respected.

4. Polar Wandering: In the 1950s while at the royal observatory, Gold became interested in the instability of Earth’s axis of rotation or the wandering pole. He wrote a number of papers on plasmas and magentic fields in the solar system and also coined the term “The Earth’s Magnetosphere”. In 1955 he published yet another revolutionary paper “Instability of the Earth’s Axis of Rotation“. Gold made the view that large scale polar wandering could be expected to occur in relatively short geological time spans. That is, he expressed the possibility that the Earth’s axis of rotation could migrate by 90 degrees in a time of under a million years. This effectively means that in such a case, points at the equator would come to the poles and points at the poles would come at the equator. Gold argued that this 90 degree migration would be triggered by movements of mass that would cause the old axis of rotation to become unstable. A large accumulation of ice at the poles for example might be one reason why such a flip could occur. His paper was ignored largely for over 40-45 years, largely because at that time the research was focused on plate tectonics and continental drift.

In 1997 a Caltech professor Joseph Kirschvink, who is an expert in these areas published a paper that suggested that such a 90 degree flip indeed happened at least once in the past in the early Cambrian era. This holds much significance given the fact that this large scale migration of the poles coincides with the so called “Cambrian Explosion“. Gold’s work was finally confirmed after being ignored for decades.

5. Abiogenic Origin of Petroleum: When I first read about the theory of abiogenic origin of petroleum promoted by Tommy Gold and many Soviet and Ukrainian Geologists, I was immediately reminded of my old organic chemistry texts that spoke of the abiogenic origin theory given by Mendeleev almost 150 years ago. This was called Mendeleev’s Carbide Theory and it died after the biological theory of petroleum origin was widely accepted.

Speaking as a layman who has little knowledge of geology, petroleum etc, I would say any theory of petroleum origin must broadly explain the following points:

1. Its association with Brine.

2. Presence of $N$ and $S$ compounds.

3. Presence of biomarkers, chlorophyll and haemin in it.

4. It’s optically active nature.

According to Mendeleev’s Carbide theory:

1. The molten metals in the Earth’s interior combined with carbon from coal deposits to form the corresponding carbides.

• $Ca + 2C ---> Ca C_2$
• $Mg + 2C---> Mg C_2$
• $4Al + 3C---> Al_4 C_3$

2. The carbides reacted with steam or water under high temperature and pressure to form a mixture of saturated and unsaturated hydrocarbons.

• $Ca C_2 + 2H_2 O---> Ca(OH)_2 + C_@ H_2$
• $Al_4 C_3 +12H_2 O---> 4Al(OH)_3 +3C H_4$

3. The unsaturated hydrocarbons underwent a series of reactions such as hydrogenation, isomerisation, polymerisation and alkylation to form a number of hydrocarbons.

• $C_2 H_2 ---> C_2 H_4 ---> C_2 H_6$
• $3[C_2 H_2]---> C_6 H_6$

etc.

This theory got the support by the work of Moissan and Sabatier and Senderen. Moissan obtained a petroleum like liquid by the hydrogenation of Uranium Carbide, Sabatier and Senderen obtained a petroleum type substance by the hydrogenation of Acetylene.

However the theory was in time replaced by the theory of biological origin as it failed to account for:

1. The presence of Nitrogen and Sulphur compounds.

2. Presence of Haemin and Chlorophyll.

3. Optically active nature.

After almost hundred years, the abiogenic theory was resurrected by the great Russian geologist Nikolai Alexandrovitch Kudryavtse in 1951. This was worked on extensively by a number of Russians in the coming two decades.

In the west Thomas Gold was the only major proponent of it. And this is his most controversial theory, not only because it was opposed by powerful oil industry lobbyists but also because Gold faced much flak for plagiarism, something that Gold refused to acknowledge, in his later works he cited the works of the Russian scientists in the field. He maintained that he was simply not aware of the work done by the Soviet Geologists and that he cited their work once he became aware of it. Gold proposed that the natural gas and the oil came from reservoirs from deep within the Earth and are simply relics of the formation of the Earth. And that the biological molecules found in them did not show they had a biological origin but rather that they were contaminated by living creatures. He remained critical of the proponents of the theory of biological origin as then it could not be explained why there were hydrocarbon reserves on other planets when there had been no life on them. This theory remains controversial, Gold could not live to defend it. However an elegant experiment performed provides some evidence that Gold could indeed again be right.

Dyson wrote the following on an EDGE essay in this regard:

Just a few weeks before he died, some chemists at the Carnegie Institution in Washington did a beautiful experiment in a diamond anvil cell, [Scott et al., 2004]. They mixed together tiny quantities of three things that we know exist in the mantle of the earth, and observed them at the pressure and temperature appropriate to the mantle about two hundred kilometers down. The three things were calcium carbonate which is sedimentary rock, iron oxide which is a component of igneous rock, and water. These three things are certainly present when a slab of subducted ocean floor descends from a deep ocean trench into the mantle. The experiment showed that they react quickly to produce lots of methane, which is natural gas. Knowing the result of the experiment, we can be sure that big quantities of natural gas exist in the mantle two hundred kilometers down. We do not know how much of this natural gas pushes its way up through cracks and channels in the overlying rock to form the shallow reservoirs of natural gas that we are now burning. If the gas moves up rapidly enough, it will arrive intact in the cooler regions where the reservoirs are found. If it moves too slowly through the hot region, the methane may be reconverted to carbonate rock and water. The Carnegie Institute experiment shows that there is at least a possibility that Tommy Gold was right and the natural gas reservoirs are fed from deep below. The chemists sent an E-mail to Tommy Gold to tell him their result, and got back a message that he had died three days earlier.

6. The Deep Hot Biosphere: I am yet to read this book, though I have been thinking of reading it for almost a year now.

[The Deep Hot Biosphere, Image Source : Amazon]

In this controversial but famous theory Gold proposes that the entire crust of the Earth uptill a depth of a few miles is populated by living creatures. The biosphere that we see is only a very small part of it. The most ancient part of it is much larger and is much warmer. In 1992 Gold referred to ocean vents that pump bacteria from the depth of the Earth in support of his views. A number of such hydrothermal vents have since then been discovered. There is increasing evidence that his yet another controversial theory might just be right. Even if it is not, the evidence collected will help us understand our planet much better.

[A Black Smoker Hydrothermal Vent]

Finally Quoting Prof Freeman Dyson on him again:

Gold’s theories are always original, always important, usually controversial, and usually right.

1. The Scientist as Rebel : Chapter 3 – Freeman Dyson (Amazon)

2. The Inertia of Scientific Thought – Thomas Gold

3. The Deep Hot Biosphere – Thomas Gold

4. Heretical Thoughts about Science and Society – Freeman Dyson

Onionesque Reality Home >>