Plain Simple Stupid Language

People don't implement plain simple stupid language. It is an understandable but grave mistake. To implementers it may seem pointless and to users unattractive. But to students of language implementation plainness is a virtue, simplicity a must, and even stupidity can be helpful.

Modern programming languages are complex beasts. They are like cheetahs, tigers, lions, and even dragons. Some have wings like pegasus. When we speak of beasts we remark on their speed, their sharpness of teeth and claws, their unusual ability of breathing fire and winged flight. We rarely remark that they have eyes, mouth, and legs; hearts, lungs, brains, and kidneys. Above all we don't talk of the fact that they are animals -- and that they can do things we expect animals to be able to do. Those are implicitly understood.

Imagine that you are to explain what is a dog in words to an alien who never saw any animals. To people of Earth a particular animal is often described as an animal plus any difference to what I would call a plain simple stupid animal. But when you really try to describe it, you soon realize that the most of complexity is in that it is an animal, not that it is a dog. A dog may be simple, certainly simpler than dragons or pegasuses (pegasi?), but it still is an animal which is insanely complex in itself.

Programming a board game AI

This semester, the final project of the problem solving class, which belongs to the computer science department, was to program an AI for a board game called Halma, whose variant is often known by the name Chinese Checkers.

Typical board game AI consists of following parts: move generator, evaluation function, and tree search. As the name implies, move generator generates possible moves; it encodes the rule of the game, judging which moves are legal and which moves are not. If move generator encodes the rule, evalution function encodes the "intelligence"; it is a function from the state of the game to some number. Usually, larger number means that the game is favorable to the player. But evaluation function doesn't need to be perfect; that's where tree search comes in. Tree search can "amplify" the minimal intelligence of evaluation function.

Because of such amplification effects, programming a board game AI is an eerie experience. When you program an unfamiliar game, it is a routine experience for your program to outperform you. It is more surprising since even the dumbest evaluation function can beat you. Often evalution function as simple as "substract the number of moves I can make from the number of moves the opponent can make" works surprisingly well. Notice that this function is not even game-specific.

In tree search, the program considers all moves I and the opponent can make to certain "depth", and passes resulting positions to evaluation function. The program assumes that the opponent tries to minimize the value of evaluation function, and tries to maximize the value of evaluation function after its move. The secret is that the program can examine a lot of positions quickly. In my case, the program searched to depth of 5 moves and examined millions of positions for a single move. This sheer quantity of positions compensate for poor quality of evalution function.

There are rich literatures concerning the programming of a board game AI. After all, people programmed computers to play chess as early as 1956. At this dawn of the programming, valiant programmers of Los Alamos laboratory programmed their huge cranking machine, which was actually less capable than cheap graphing calculators of today, to play a simplified version of chess on 6 by 6 board (see Wikipedia, Los Alamos chess). It won a game against a human who just had learned the game. We came a long way since then.

Geology

Shen Kuo(沈括) was a Chinese scientist of the 11th century. His work, Meng Xi Bi Tan(夢溪筆談), contains his scientific writings.

In one particularly stunning passage, he describes a theory of land formation by sedimentation, and an evidence for coastline movement. Following is the Chinese text and my translation. I got the Chinese text of Meng Xi Bi Tan from the Project Gutenberg.

余奉使河北, 邊太行而北, 山崖之間, 往往銜螺蚌殼及石子如鳥卵者, 橫亙石壁如帶. 此乃昔之海濱, 今東距海已近千里. 所謂大陸者, 皆濁泥所湮耳. (卷二十四)

When I was in Hebei, I went north, Taihang Mountains on one side, and between cliffs, often seashells and stones like bird's eggs crossed stone walls like a belt. Once this was the seashore, now a thousand li east of the sea. So-called continents are all submerged mud. (Volume 24)

This insight is comparable to that of James Hutton, a Scottish geologist of the 18th century, who is considered the father of modern geology. In Concerning the System of the Earth, its Duration and Stability, printed in 1785, he wrote:

Hence we are led to conclude, that the greater part of our land, if not the whole had been produced by operations natural to this globe; but that in order to make this land a permanent body, resisting the operations of the waters, two things had been required; 1st, The consolidation of masses formed by collections of loose or incoherent materials; 2dly, The elevation of those consolidated masses from the bottom of the sea, the place where they were collected, to the stations in which they now remain above the level of the ocean.

James Hutton is often unappreciated, but he must be considered as an equal of Isaac Newton and Charles Darwin. He published one of the most profound idea in the history of science, that the Earth is old. Like Darwin's theory, his theory challenged Christian doctrine, since it was incompatible with the account in Genesis. And it was the idea that the Earth is old, a half century later, that let Darwin think there was enough time for all life to evolve.

Witch on a journey

Some think that it is futile to translate poems, or lyrics for that matter. Many people try anyway. Here is my take on translating one of my favorite Korean pop songs.

Witch on a journey
included in "Welcome To My Beach", Kona (1997).

Now listen to my words
at least once a day
clean your crystal balls
get up early in the morning
drink a cup of white milk
and three lemon candies a day
don't forget to brush your teeth
before you sleep

A new wind is blowing for your journey
now raise your little broomstick
and fly to the blue sky
don't worry
and remember

Our beginning more mysterious than magic
the night and the shining promise
I will believe and wait
for the day you will come back to me
and we will start again
new days
in the wonderful world

Black cat's name's Pippi
and black dress is ironed
have you forgotten anything?
check again!

(Repeat)

Risk assessment

I know I am behind schedule in writing blog posts. Desperate, here is a quick translation of a short piece I wrote in Korean.

Risk assessment

In assessing risks, it is meaningless to assert that the risk of A is high because some B has the high probability of causing A. The high probability of causing A should be considered together with the probability of B itself.

Here is an example of the fallacy:

Mad cow disease has the high probability of causing death. Therefore, it is highly probable to die from mad cow disease.

Compare with:

Mad cow disease has the high probability of causing death. However, mad cow disease is rare. Therefore, it is unlikely to die from mad cow disease.

Newcomb's paradox

Newcomb's paradox is an interesting thought experiment. Some philosophers think it has to do with the problem of free will and causality. Some think it is silly. There are many slightly differing formulations, and some think the details of the formulation matter a lot. I don't take the paradox too seriously and I don't think the details matter, nevertheless it is very interesting to observe people's reaction when they hear this paradox for the first time.

So here is my version of the paradox.

You are presented with two boxes, A and B. The box A is transparent, and inside it is $1,000. The box B is opaque. You are presented with two choices. Either you can take both boxes, or you can take the box B only. And then there is the predictor. With high accuracy the predictor predicts your choice. If he/she/it predicts you will take both boxes, it puts nothing in the box B. If he/she/it predicts you will take the box B only, it puts $1,000,000 in the box B. What is your choice?

Argument 1. The box B has either $1,000,000 or nothing. The predictor already predicted my choice and decided the content of the box B, so my choice doesn't affect the content of the box B in any way. Taking both boxes is always superior to taking the box B only, by the amount of $1,000. Therefore I take both boxes.

Argument 2. Since the predictor is highly accurate, if I choose both boxes it is highly likely that I will find the box B to be empty, and that my income will be $1,000. Since the predictor is highly accurate, if I choose the box B only it is highly likely that I will find the box B to have $1,000,000, and that my income will be $1,000,000. Therefore I take the box B only.

Note that the first argument is independent of the predictor's accuracy. Also note that the mechanism of the predictor's prediction is completely unspecified, and you have no way to assess its accuracy other than that it is "highly accurate".

Let's change that. Suppose that you are an observer, and you observed 10 (100, 1000, whatever) people choosing either both boxes or one box. Suppose that you observed the predictor's prediction to be always correct. Would it change your choice? Does the number of observations matter? Note that if you support the first argument, you have no reason to change your choice.

Also note that unless the mechanism of the prediction is specified, you can argue that the past performance is no proof of the future performance, and to think otherwise succumbs to the gambler's fallacy. The gambler may think that if coin toss turned head ten times in a row, the next toss is likely to turn head (or tail). Of course if coin is fair, each toss is independent and head and tail are equally likely, no matter what.

Suppose that you observed the predictor's prediction to be 90% correct. It is easy to cacluate that the expected value for both boxes is $101,000 and for one box is $900,000. Would the fact that the predictor is not perfect change your choice? Does the exact probability matter? Does the ratio of the amount of money matter? If your choice is based on the expected value, they probably do.

Some argues that the predictor as presented in the paradox can not exist. Suppose that people in the paradox are replaced with AI programs, and the predictor has much faster computer to run programs. Then it is obvious to me that the predictor can indeed predict the behavior. Since I accept the Turing thesis (which states, in short, human-like AI is possible), I also accept that such predictor can exist.

Also note that the predictor in the paradox does not need to be perfect. Public opinion polls routinely make predictions such as that females are more likely to vote for the candidate X, that people who live in Y are more likely to vote for the candidate Z, etc. How can you argue that it is impossible to predict the human behavior?

(P.S. So what is my choice? I choose the box B. My reason shall remain private though.)

Pinball

There is an increasing danger of practicing tetrapyloctomy, as discussion gets longer. This post is another attempt at discussing learning, research, and related matters. Readers should judge whether I succeeded in avoiding the danger.

Let me first cite my sources. The first part of the argument closely follows the article The Two Cultures of Mathematics by Timothy Gowers. The link contains the complete article. The second part of the argument was influenced by the chapter titled "Pinball" from the book The Soul of a New Machine by Tracy Kidder. I can't link to the full text, but the link contains a good summary of pinball analogy.

Now, here is a short recap of the discussion so far:

1. You are scientists so you should love learning.
2. Scientists learn just enough so that they can do research.
3. This is just a semantic matter; scientists must indeed love learning, else why would they bother to do the research?
4. Consider an analogy to writers. Reading is not what they are paid for; writing is. Similarly, scientists do not learn science; scientists do science.
5. I wrote that scientists must be motivated by a love of learning. What motivates scientists to do research? Certainly not money. I think it is outcome. Perhaps you are imagining a sort of performance-art scientist, for whom the very process of research provides its own reward, regardless of outcome?

Certainly, I am not making ridiculous claims like that scientists love research but not learning, or that writers love writing but not reading. I am talking about the priority. Consider the following two statements:

1. The point of scientific research is to understand the nature better.
2. The point of understanding the nature is to do the better scientific research.

At first, the second statement may seem odd. Some variations should help understand it better: 1. The point of understanding the nature is to apply that understanding; one application is as a basis for an effort to achieve deeper understanding. 2. The point of learning existing knowledge about the nature is to do research (as opposed to intrinsic value of such knowledge).

There is truth in both statements, but one does not necessarily agree with both statements to the same degree. I claim that average scientists are inclined to the second rather than the first. You could argue that I am imagining "performance-art scientist"; actually I am claiming that it is not an imagination, but probably a majority. I will argue two points, that outcome is often not the most important, and that research is often primarily seen as a vehicle for further research.

It seems to me that scientists are often not too conscious about outcome of their research. The case of physicists participated in the Manhattan Project is a classic one.

As a student in the computer science department, I want to give another example. Computers are often said to be general purpose, which means, unlike calculators, televisons, or MP3 players, it can do many tasks (like calculation, playing video, and playing audio) given appropriate softwares and peripherals. It is also a common knowledge that general purpose computer chips are used heavily in military applications. If one wants to avoid any association with tanks or missiles because one is against wars, there is no way other than ceasing to work on electronic chip design at all. However, in reality, it won't be too much of exaggeration to assert that most electronic chip designers are utterly uninterested in most applications of their work -- applications that range from executing financial strategies in derivative market to producing virtual child pornography.

It also seems that research are often judged in terms of potential for further research, rather than in isolation. One often talks about "profitable line of research". Naive interpretation is that this refers to research that brings profit. It actually means that the research is likely to lead to further research.

So is the goal of the game of pinball. You win one game, you get to play another. For writers, you finish one novel, you get to write another. For mathematicians, you solve one difficult problem, you get to solve another. For engineers, be they civil engineers, computer hardware designers, or software developers, you complete one engineering project, you get to join another. For scientists, you research one subject, you get to research it further.

I am not saying that one particular round of pinball, or one theorem, or one bridge, or one paper is not important; but as or more important is the process of creation, and the hope of better future. Whatever you learn from others, by definition, is a thing of the past. Whatever you learned yourself, becomes a thing of the past when you learned it. Whatever you are yet to learn, especially if nobody has ever learned it, is the most interesting.

Speaking of my experience, I often longed for the beginning of a new software project when one project was almost at the end. The endgame is not as interesting. However, the endgame, or the polish, absolutely matters in commercial software development. I often thought that this is why most research software, as opposed to commercial software, lacks in polish -- because there is less pressure for the polished product.

If Rowling does not want to write another Harry Potter book, I can understand.