The Loebner Contest is version of the Turing Test which is held annually, and is open to anyone who cares to write a conversation simulator. The rules of the Loebner Contest can be found on the Loebner Prize Home Page.
Rather suprisingly, my HeX program has won the 1996 Loebner contest, beating four other computer entries. The results are as below (you can read the transcripts of each program by clicking on the name of their author):
The standard of Loebner Contest entries rose in 1997. The SEPO program didn't win, and since SEPO is a big improvement over HeX, I take that as a good sign for the future of the competition.
On second thoughts, having now taken a look at the transcripts I can see that the winning program was very clever because it's opening gambit was to ask the judges about something which had been in the news recently (the night before the contest, in fact). I personally feel that SEPO's conversation was more realistic, and that the judges were swayed to choosing David's entry because it could handle conversation about a recent news item rather well. I'm not disparaging his work at all, I just feel that this was the deciding factor, and I find that rather interesting.
The results of the 1997 contest are:
In 1998, the Loebner Contest was held outside of the USA for the first time in its history. In fact, it was held in Australia, so I was able to attend in person. Four human beings and six computer programs made up the confederates. Ten judges had twelve minutes to chat to each of the confederates and rank them from most human to least human.
Rather surprisingly, MegaHAL was ranked as the second most human computer program. I entered the contest expecting it to come last, as it uses a technique quite different from normal conversation simulators.
The results of the 1998 contest are:
The Loebner Contest was once again held in Australia in 1999, this time in the "sleepy town" of Adelaide. Once again I attended in person. NonI, the program I entered, came a disappointing fourth, which I put down to a lack of training on my part. The program needed to be trained from scratch: it learnt everything during this period, including how to simulate typing. I just didn't put in enough of an effort.
The results of the 1999 contest are:
I was livid when I saw the results for the 2000 Loebner Contest, which was held at Dartmouth College, the site of the famous summer workshop which initiated the field of Artificial Intelligence. My submission, eBrain (an improved version of SEPO) came second. However, the contest organisers deemed to use data files which I'd submitted several months before the contest, instead of the ones I'd submitted just before the contest, on the grounds that they hadn't had time to test them! The result is that my 2000 entry rather embarassingly resembles my 1997 entry, although that was not intended to be the case. Rather, I'd submitted the 1997 data files so the new executable could be tested by the organisers while new data files, which make use of the new eBrain features, were being developed.
The results of the 2000 contest are:
In 2001, the Loebner Contest will be held at the London Science Museum on Saturday October 13.