IlliGAL Blogging

Last day in GECCO. Wednesday is always as intense as any other day in the conference. Last minute discussions, hugs with friends that we won’t see till next year if we are lucky, business cards exchanging at an incredible pace. Today my post is going to be short. I spent most of the time talking to people and attending a few talks on EDAs. To celebrate another thrilling GECCO some of us went for dinner to a nice Indian restaurant in Georgetown---Pier Luca’s fault again---and then, we went to watch the opening of the War of the Worlds. I am back to the hotel now. I need to rush now to pack and get some sleep before we ride back to Urbana. Yes! We are leaving at 6am ;)

This morning in the business meeting of SIGEVO, Riccardo Poli announced the winners of the 2005 Human-Competitive Results Competition (the "Humies"). A previous post discussed the contest, and over 20 applicants vied for $10,000 in prize money. The judges selected two Gold Medalists, Randy Bartels for using GAs to shape laser pulses to effect quantum dynamics control (see here) and Preble, Lipson, and Lipson for using genetic algorithms to design photonic crystals (see here). Other medalists used GAs to evolve KOTHs in Core Wars, top backgammon, chess, and robocode players, optical lenses that infringe on existing patents, fingerprint wavelets that beat the FBI wavelet, and a scalable quantum Fourier Transform.

The Humies were a highlight of the conference, and the competition will heat up as the word spreads.

I am sitting on some of the EDA sessions and I can help thinking about a discussion I had with Tian-Li Yu when I was preparing the papers about the compact classifier system. The discussion was about the main differences of DSMGA when compared to eCGA or BOA. DSMGA, unlike eCGA and BOA, provides a crisp and clear presentation of the identified building blocks at the end of a run—just to mention the reason of my wondering. Once the population has converged, eCGA and BOA models just indicate that all the variables are independent. From a substructure identification point of view, the final DSM gives a clear picture of how the problem looks like. Such a property keeps getting my attention every time I sit in an EDA session.

Mental not to self: I need to talk to Tian-Li again. Genetics-based machine learning systems that can provide problem decomposition at the end of the run is one of the challenges I would like to see more solutions on.

At long last I arrived at GECCO yesterday afternoon. Saturday was the lil penguin's Bar Mitzvah, and he did a wonderful job. On Sunday, we drove the older penguin to Northwestern University for the NHSI program on journalism (the famed "cherub" program). On Monday, an exhausted father, scraped himself up, drove to Bloomington and arrived at the GECCO venue at L'Enfant Plaza Hotel at about 5 pm.

The conference was in full swing with nearly 600 attendees. Both keynote speakers were a hit, and I helped judge the Humies (or Human Competitive Competition) in the morning. Excellent finalist presentations, and I'll have more to report on that score later.

Tonight is the poster session, and the the lineup appears to be very strong. This conference marks the 20th anniversary of regular genetic algorithms conferences with the first ICGA being held in Pittsburgh at CMU in 1985.

In one of today’s GECCO sessions, I run into a talk by Doug Newell founder of Genalytics. First I wasn’t sure what he would cover, but in the end it was a very interesting talk. He covered several topics. He started talking about direct marketing. The bottom line, direct marketing is still using mail as a main stream--yes, regular mail. Corporations are working to learn how to improve their targeting capabilities. Sophisticated analytics can improve the targeting up to a 25%. However, statisticians, usually using SAS, and taking around 3 weeks synthesize such models for improvement. Some of the methods use regression, logistics egression, neural networks, and decision trees. Doug show some of their work were GAs can beat the traditional approaches. Another friendly marketing area for GA applications—in Doug’s words--is fraud detention.

Next talk was about Eric Bonabeau presented some of the work of Icosystems on interactive evolution. They path--different from our current work on iGAs--empowers users with all sorts of freedom over the evolutionary process. One application: tile design. They are interested in the navigation across the search space, which is why they are more interested on the evolutionary path, instead of getting a final solution.

Mental note to self: Lack of fit of the adjustment of the model (LOF) by Flor Castillo (Dow Chemichal). Any of these statistics could be used in genetic-based machine learning?

GECCO, day 2. Still blogging offline. The conference has not started yet, but today I am chasing tutorials. I always like the chance to attend different workshops and tutorials for free two days before the conference. As mentioned in an early post, yesterday we had the eighth International Workshop on Learning Classifier Systems. Today, there are two related tutorials.

The first tutorial was Martin Butz’s and “The XCS Learning Classifier Systems: From Theory to Applications”. The second related tutorial was by Tim Kovacs with a general introduction to “Learning Classifier Systems”. Both gave a nice overview on the Michigan style classifiers---should we start a no Pittsburgh left behind tutorial? ;)

Mental note to self: could we rearrange the schedule next year so we can have the tutorial before the workshop? It may help bring more people in to the LCS world.

Another mental note to self: there is no talk about creativity and innovation support, should we have a talk about the DISCUS effort next year?

A final note for Nosophorus. On my way from Martin’s tutorial to the OBUPM 2005 workshop I saw Ingo Rechenberg getting ready for his tutorial. Sorry, I could not stop by, but I can tell you that the room was getting crowded. Then, I came back because Kumara Sastry was giving his in the same room. I just so people still talking about his presentation.

Finally, it happened. Today, from 8:50am to 6:30pm we have been involved in IWLCS—you may find the schedule here. In my humble opinion, this has been a great and lively workshop—I saw business card changing hands, envelope calculations, and hallway discussions. We had 17 papers, 3 on going works, and a big discussion session.

The morning stared with the “Methods and Scenarios” session. From the problems with class imbalance to policy transfer a big deal of interesting ideas were proposed. The same stream of interesting ideas happened with the second, and last, morning session “Representations”. There were too many ideas to choose a small subset, but to give you and idea, the “Continuous Actions” talk by Stewart S. Wilson start breaching the dam toward such kind of actions.

Lunch break. We tried to go for lunch near the hotel. The part of D.C. where GECCO is being held does not seem to be friendly during the weekends. Eventually, we had lunch at the food court of the Smithsonian Museum. It was the only place that our exploration mode was able to find before we turned into exploitation mode.

And talking about exploration/exploitation modes, that one was one of the hot discussion topics in the first afternoon session “Reinforcement Learning”. “Multiagent Systems”, “Applications”, and “Environments”, were the other three afternoon sessions. Some ideas also expanded sentinel class predictor reminded me similar efforts on chance discovery. Some other work showed clear interactions with reinforcement learning theoretical work. Interesting remarks were also done into the realm of multiple rewards are available.

Oh, I almost forgot. Yes, as usual, we all got for a big and relaxing dinner together.

My absence from the airwaves was due to my impromptu college visits with Max Tuesday and Wednesday. We visited Earlham College, Denison University, and Kenyon College. I would like to blog at length about the visits, but suffice it to say right now, that visiting small colleges as opposed to hotshot Ivies is the difference between night and day. Incredible personal attention, lovely, well-designed facilities for small numbers of students, just a different planet. Makes me wish I had my own college education to do over. Max's ranking of the three was (1) Earlham, (2) Kenyon, and (3) Denison. More later, but my other penguin is having his Bar Mitzvah Saturday. To read or review parts I-III, click here.

Paul Wiegand passes along the following website as containing useful information for GECCO attendees. GECCO-2005 starts on Saturday and goes until Wednesday. IlliGAL Blogging bloggers intend to blog live from the conference for those of you who can't attend (and for those of you who can).

Some of the IB members went today to the US Grand Prix of F1 at Indianapolis. It has been a remarkable unique Grand Prix. You may be already aware that no pilot or car has been the rising star of this race. The work of Michelin has put them in the hall of fame. They better adapt quickly, reliably, and accurately. A mistake like this can ruin a year of good work as fast as you snap your fingers.

Below you can find a fragment of one of the several letters that FIA and Michelin have been exchanging this weekend. You may find the letters here and here.

[...] We (FIA) are very surprised that this difficulty has arisen. As you (Michelin) know, each team is allowed to bring two different types of tyre to an event so as to ensure that a back-up (usually of lower performance) is available should problems occur. It is hard to understand why you (Michelin) have not supplied your teams with such a tyre given your years of experience at Indianapolis [....]

For some reason, blog search engine Technorati has not been crawling IlliGAL Blogging for the last month or two. The trouble is apparently cleaned up, and you can once again search for IB results using Technorati.

Working on genetic algorithms is a lonely pursuit, so the following article should be of interest to IlliGAL Blogging readers, as should the following social networking site.

Hans-Georg Beyer, editor-in-chief of the GECCO conference proceedings this year, just sent me an email saying that the GECCO-2005 proceedings table of contents is available at this link. Instructions for full citations are available here.

The 2005 conference program is quite impressive (here), and HGB and his team of track editors deserve a round of virtual applause from the blogosphere (what sound do bits make when clapped together?). Great job, a job made more challenging by the transition to ACM status under SIGEVO.

GECCO-2005 is ready to go, with what looks like record attendance, a terrific program, and live coverage by IlliGAL Blogging bloggers. Wireless internet willing, our band of bloggers will cover happenings at GECCO from Saturday to Wednesday, 25-29 June 2005. Read IlliGAL Blogging for live GECCO coverage, or better yet, visit us at the conference.

I came back from my recent trip to Shanghai with advice to my sons (see here), but Mark Steyn's column has me wondering. Read the whole thing especially the last paragraph. via Instapundit

I'm 2/3rds of the way through a terrific short lecture series on the Ethics of Aristotle given by Father Joseph Koterski at Fordham University. The series is a blow-by-blow description of the Nicomachean Ethics, and in 12 sessions Father Koterski highlights the key ideas in a way that makes reading the Ethics so much easier. The course is part of the Teaching Company's stable of courses that I have discussed previously (here).

In my previous post I included computer scientists in my discussion of specialization and the postmodern engineer. Doing so reminded me that one of my pet peeves over many years has been that the field of computer science chose to call itself "computer science" and not use a term that included the term "engineering."

There is no proper argument that computer science is one. Science is largely descriptive: it works primarily on describing that that is. Engineering is primarily prescriptive: it says what should be built and how things should be done. Without the invention of an engineered object, the computer, so-called computer science would have nothing to describe, and computer science's past has been a glorious history of continual improvement, engineering, and invention of some of the most powerful technological artifacts known to man.

None of this says, of course, that science is not used in computer science toward its primary mission of creating better computer systems, and none of this says that computer scientists don't contribute to science through their descriptive efforts, but other disciplines of engineering also use and contribute to science in a similar ways. So why did computer science "defect" from the other engineering disciplines and choose to ally itself nominally with the sciences?

History and prestige give much of the answer. Computer science arose in the aftermath of World War II and the success of the glory projects, the atomic bomb and radar, were largely attributed to the science involved. A more careful reading of history (see this) suggests that the success of the glory project required a good bit of formal and informal engineeirng acumen. Moreover, a better understanding of the Allied victory, generally, would credit engineering as part of a larger production dynamo that produced materiel for the war effort more rapidly and in larger quantities than had ever before been seen. Such nuance is largely lost to the larger culture, however, and the postwar zeitgeist raised the prestige of science. That CS chose to call itself a science can be understood in that milieu even if the name is inaccurate.

Unfortunately, the harm caused by this choice is not restricted to sloppy speech. What a field chooses to call itself matters. For example, the training of CS graduates teaches them "theory" as some sort of pristine exercise in proper mathematics as opposed to teaching model building as something that is integral to proper engineering. As a result, CS majors tend to treat theory as something divorced from the engineering of systems in a manner not seen in disciplines that choose to call their practitioners "engineers." My campaign for little models is largely an effort to counteract the imbalance in the theory education of computer scientists.

Card carrying computer scientists have recognized these problems. For example, Peter Denning's famous article Educating a New Engineer (here, ACM access required) chose the term "engineer" in making a case for change in educating the CS major of the future, and he created what he called the Center for the New Engineer to promulgate that vision. The selection of the term "engineer" was not an accident, and part of the point was to correct the errors of a CS past in which science was overvalued above the primary prescriptive mission of the discipline.

CS is the most recent discipline of engineering to be created, but it probably won't be the last. Let us hope that the next one that comes along has the wisdom and courage to choose a more accurate name.

Self SEO has an interesting post about the art of business blogging. One of the key ideas of the post is that bloggers are unusual birds in this age of specialization in that the good ones tend to be generalists:

Bloggers, from what I have seen in my years of blogging, tend to be more generalist than specialist, except for a few who are both...which is to say they have wide knowledge and interests, but also have one or two things that they know a great deal about.

Business bloggers, ideally, fall into this last category. They know a great deal about their company, service, products and so on, but also have a wide-band tuning which allows them to connect with people of all stripes, types and interests. After all, one should not expect that a customer or client know all the fine points of what they are seeking to buy, yet the seller should know those fine points, but also a good deal about the customer as well. This is where a wide-band understanding comes in handy.

This sounds about right; however, I've been wondering lately whether these postmodern times are as friendly to the notion of a specialist as was the Cold War was. In engineering, at any rate, it seems that companies more and more value those engineers who can communicate with marketers, finance people, manufacturing types, and customers. No longer is it acceptable to sit at a drafting table or CAD tube and think only grand technical thoughts alone. Postmodern engineers (and computer scientists) need to be more broadly educated and broadly interested to be successful, and certain changes are coming about to an engineer's education, albeit slowly, in recognition of the workplace shift.

None of this suggests that specialists won't continue. Division of labor and the efficiency it brings about will continue to demand specialization, but the specialization will be more and more be like the second category of business blogger: "which is to say they have wide knowledge and interests, but also have one or two things that they know a great deal about."

A group of IlliGAL researchers is on the road doing an important study of .... the blues at the Chicago Blues Fest this weekend. By day, mild-mannered Kumara Sastry bangs out difficult solutions to impossible difference equations with the best of them, but set him down in front of a Buddy Guy recording (You Got a Hole in Your Sole clip here) and he becomes a wild and crazy blues guy.

Regular reader Nosophorus found my learning to sail post a bit tame and suggests surfing an Amazonian pororoca or tidal bore in the Amazon River delta. In the famous words of the Brian Wilson, "Let's go surfing now, everybody's learning how, come on a safari with me."

Deepak at AI-complete has a nice post about a paper by Vitanyi and Cilibrasi that is causing a buzz. Read the post and read the paper.

OptionsScalper has a post about genetic algorithms here. The newly started blog covers a variety of topics from software engineering, to finance, to emergent computation. Why not take a look?

Josep Maria Terricabras is one of the most well-know contemporary philosophers of Catalonia. In his blog he has a post reflecting about everyday “naïve induction”. The ultimate point is a reference of how people ignore “the irrelevance of warrant” thesis by David Hume (1711-1776). A rudimentary translation of an excerpt of his post elaborates like this:

“…Support the hypothesis that things that happened in the past in a certain way will keep repeating in the future in the same manner shows a great naivety…”

As I mentioned in an early post, I am revisiting some of the initial works about genetics-based machine learning. Reading such an elaborated argument by Terricabras, besides sending me back to my high school philosophy classes, it reminded the hypothesis that we almost always take for granted in learning process. Once you have properly learning from experience and history, such knowledge will be useful for understanding the new inexperienced situations. In another words, given a training data set (or environment), once I learnt a certain model, such a model will describe the new data I collect. Even in dynamic environments, people assume that a learned dynamic model may explain future dynamics.

Unfortunately, Hume’s irrelevance of warrant thesis reminds us how careful and humble our claims should be about the generalization/inducting capabilities of any machine-based learning system.

One of our readers enjoyed the sailing blogging (here and here), so we continue our voyage with a discussion of sailing lessons. In the States, two organizations dominate keelboat sailing lessons, the American Sailing Association and US Sailing. Both offer curricula and certified instructors to teach budding sailors, and the resulting certification is akin to a driver's license that can be used as a credential to charter a bareboat.

My wife and I have taken ASA courses, starting with 101 and 103 at Torresen Marine in Muskegon, Michigan. I continued on with Bareboat Chartering 104 at Fairwind Sailing in Chicago with Ben Sells. The courses were a combination of hands-on sailing and coursework in the rules of the road, boat terminology and workings, and navigation, and the courses did result in my being able to lease a boat. Once you've got your ticket, it is possible to charter boats all over the world from organizations such as Sunsail and others.

So, if you're looking for a break from the rigors of genetic algorithms, consider learning how to sail today.

jwz has a nice post about a Japanese robot that can ballroom dance with a partner by sensing necessary spatial movements from partner feedback. Where was she when I was taking dance classes at David Lin's Regent Ballroom?

The trip to Mexico was interesting for several reasons. The CONCEV 2005 conference was a pool of ideas. I was sitting at Francisco Herrera talk on memetic algorithms and I could not stop thinking how fun was we both were there, and neither of us were giving a talk about genetics-based machine learning. But, that was not the reason for feeling uncomfortable sitting in a very comfortable chair in the auditorium; Francisco’s talk was sharp and inspiring. Suddenly, I just realized that there was something else buzzing in my mind: how easy we may be trying to reinvent the wheel.

In another words, sometimes we realize that after inverting our efforts on a particular endeavor, someone had already gone through the same problem, pain, and thought process. Sometimes trying to face the problem straight without looking around first is a good guarantee for reinventing the wheel. In a previous post, Dave Goldberg has coined the term WORN (write once, read never) for such a phenomenon of postmodern times. Thinking about it, it is hard not to be trap in it with the explosion of available information and the time required to successfully dive in it looking for the clue we need.

Among other reasons, the WORN wheel issue is the tip of the iceberg of why I am currently revisiting four genetics-based machine learning papers. Since I am a Pittsburgh style guy, let me suggest first:

Neri F. and Giordana A. (1995). "A Distributed Genetic Algorithm for Concept Learning", Proceedngs of the International Conference on Genetic Algorithms (Pittsburgh, PA), Morgan Kaufmann, pp. 436-443.


Konstam, A. (1994) “N-group classification using genetic algorithms”, Proceedings of the 1994 ACM symposium on Applied computing (Phoenix, AZ), ACM Press, pp. 212-216.

The other pillow papers I would like to suggest are Stewart Wilson’s ones. These two papers are greatly responsible for changing how people approached the so-called Michigan style classifier systems, setting the basis of the current efforts in the field.

Wilson, S.W. (1991) "The Animat Path to AI". In From Animals to Animats: Proceedings of The First International Conference on Simulation of Adaptive Behavior, J.-A. Meyer and S.W. Wilson, eds., Cambridge, MA: The MIT Press/Bradford Books, pp. 15-21.


Wilson, S.W. (1995), "Classifier fitness based on accuracy", Evolutionary Computation, 3(2), 149-175.

It's the weekend in the summer and a GA guy's fancy turns to...sailing. The family and I had our first sail on the Urdragon last weekend (see previous post here) and had a total blast. Although it's hard to get lost going up and down the Chicago coastline, as the good obsessive compulsive that I am, I'm working on building up my collection of navigational aids and toys.

Just found an Explorist 100 GPS receiver on sale for $79.99 at that famous marine supply store Meijers. The 100 is a basic model, but the technology that is now crammed into a little box for a little price is truly amazing. Also got a VHF 100 two-way handheld marine radio from West Marine (on the way back from China the week before last, I had 8 hours to kill in LA, went to Marina del Rey, went to West Marine, got a VHF radio, rented a bicycle, rode to Santa Monica and back, had coffee, and went back to the airport). Again, the VHF 100 is a basic model, but it is submersible, has digital squelch, and is a pretty good value on sale for $129.95.

Next, we need some good old fashioned charts and charting tools, and I'm looking at Maptech, their Lake Michigan chartbook, and some single charts along with parallel rules and dividers. Maptech is one of a number of companies licensed to reproduce NOAA marine and USGS togographical charts in paper and electronic form.

Throw in a hand bearing compass, and these toys ought to help me train up my crew in the basics of coastal navigation the old fashioned way and in the GPS age. Some time later this summer, the good lab IlliGAL will be transported onto the good ship Urdragon, and we will see if the landlubbing labbies are as good cranking the winches as they are at typing up papers. Genetic algorithms are one thing, general quarters are another.

The Slurrier has some interesting thoughts on the changing role of academic citation here:

It struck me the other day that there has been a shift in the social role of citation in academia. From what little I understand of it, bibliography arose as credentialing for the citing author, not as a chit paid to the cited. More recently, however — perhaps just since libraries and search have become more automated? — citation has come to be something owed to the cited. You write a paper in hopes that, or perhaps expecting that others will cite you, if only in a review article, somewhere down the line.

I'm not sure I buy the argument. After all, hasn't citation always played a dual function of an author showing familiarity with and acknowledging prior art? Nonetheless, the question whether online search changes citation in important ways is an interesting one. Online search cheapens research, but the lower cost also lowers the bar on what constitutes research. Moreover, online search does little to eliminate the WORN (write once, read never) phenomenon of postmodern times.

Thinking along these lines begs us to ask whether we can eliminate the plethora of citation standards that now exist and the lack of reference work standards. For example, wouldn't it be nice to have a UCC (universal citation code) like a UPC (universal product code). That way we could label citations once in a single database, and have them called up correctly whenever needed. The UCC would also make it easier to track citation usage, as the placement of the UCC in a document could be reported to a tracking website that would keep tabs on citation statistics.

The complete schedule for GECCO-2005 (the biggest, best GA/EC conference on the planet) is available here. Wow, what a lineup. There is still time to register, and even though student and conference housing is full, there are plenty of reasonable hotels available online. I'm particular heartened by the increase in papers on linkage, EDAs, and building blocks (BBs), even papers submitted by former card-carrying members of the BB boobirds. It's too bad it took them two decades to see the light, but better late than never.

Rob & I appear to be in a game riff, so here's some more. Project Nero is a collaboration of Digital Media Collaboratory and the Neuroevolution Group at the University of Texas at Austin in which genetic algorithms and neural nets are used to evolve agents that adapt to different scenarios in a game of training and battle. Some battle screenshots are available here, and a variety of movies are available here. The NERO team is headed by Texas faculty member Risto Miikkulainen. Hat tip to lemons in space.

Will Wright creator of Sim City, the Sims, and other popular simulation-based games previews his new game Spore in a BBC article. The ambitious project simulates the evolution of organisms in a large multi-player controlled ecosystem. I met Will Wright some number of years ago at a Sante Fe Institute workshop organized by John Holland at the University of Michigan, and in that sense, it is not surprising that he has tackled evolution and genetics as the basis for a computer game. It is not clear from the BBC article or a more detailed account at Gamespy whether Spore uses genetic algorithms or not. If so, the idea is similar to Dawkins Blind Watchmaker code of many years ago, but the organisms in Spore are 3D, dynamic, and generated from 22,000 different animations.

Do any readers know of more GA/EC detail on Spore? The project is breathtaking in its ambition and scope. I'm not much of a computer game fan (ask the 2 penguins), but even I might have to own a copy of this one. Hat tip to Step into the future.