IlliGAL Blogging

See here.

At the AFOSR meeting Marek Druzdzel from Pitt is talking about various ways to go beyond standard ICI and Noisy OR techniques to create Bayes nets that have smaller numbers of parameters. Probablistic ICI, Noisy OR+/OR-, and Demorgan gates are the techniques suggested. Code and papers are cited at his Decision Systems Lab site and his GeNIe and SMILE system downloads are available here.

Oak Ridge National Laboratory is using genetic algorithms in design of nuclear power plants (see here). Oak Ridge researchers Mike Hilliard and the late Gunar Liepins were probably the first researchers at ORNL to embrace GAs. I remember meeting them and working with them shortly after joining the faculty of the University of Alabama in 1984.

See here.

See some short comments on Nozick's book Examined Life at TEE.

I'm blogging from the Ambassador Room of the Ramada Inn at Fort Walton Beach, Florida. I'm attending the AFOSR Discrete Math and Optimization Program review meeting. Darrell Whitley is the only other GA guy here. Sitting in a seminar room when there are swimming pools, bikinis, beaches to be seen is tough, but no one said being a college professor was going to be easy.

Regular reader and commentator, Nosophorus, has taken me to task for patenting the ideas behind hBOA (see post and his comments here). I gave a quicky answer in the comments section, but he has persisted in his critique. I'd like to answer all of his comments more completely and formally in this post. Specifically, I will summarize and answer his argument. Then I will argue why patenting is not only acceptable, but why it is sometimes right.

His argument, if I understand it, is as follows:

1. Patenting inhibits further research in the field.
2. If GAs had been patented originally, the field would be worse off.
3. Patenting creates an incentive to only tell the good things about an idea.
4. Patenting is unethical and a bad attitude.

These ideas are simply wrong, as some simple reasoning reveals.

First, the idea that patenting inhibits further research goes against the very idea why patents are granted. Governments issue patents (a property right in an idea) to encourage inventors to disclose their secrets so that others may improve upon them. The property right is granted for a limited time, and during that time the inventor has a monopoly on the idea represented in the claims of the patent. After the limited time (a rather short time when viewed in historical terms), the idea passes into the public domain. At the time the idea is revealed, others may improve upon the idea and seek patents on their marginal contributions. In this way, patents encourage, not discourage, invention by permitting inventors to achieve monopoly rents on their idea for a limited time. Of course, these profits are not without costs and risks (not the least of which are the legal costs of obtaining and defending a patent), and the granting of a patent is not a guarantee of a life of riches. Many patents never make a nickel, a Euro, a Yen, or a Cruzeiro.

The idea that the field would not exist had genetic algorithms or other forms of EC been patented originally has been proven wrong by another example from the field, genetic programming. John Koza's original work in GP was the subject of numerous patents very early in the game, but genetic programming has been one of the hottest areas in the field over the last decade. Our seeking and obtaining a patent on hBOA has not slowed estimation of distribution algorithm research in the least, nor do we believe that it will slow it going forward. Inventors generally seek only to license their work to commercial applications of their ideas and academic research generally progresses unimpeded. Moreover, because commercially viable ideas are protected by patents, other academics also have an incentive to improve on the original idea and seek patents of their own.

The idea that patents create an incentive to tell only the good stuff about an idea has it exactly backward. Academic papers have almost no incentives to tell the bad stuff. The academic literature is full of comparisons that put method X in the a good light on problem Y while ignoring problems Z that show the method in a bad light. This continues and only gets corrected when other authors happen to come along, but seeking the truth in some abstract sense is a very weak corrective. On the other hand, when ideas are commercialized they are scrutinized vigorously by potential investors who want to know the strengths and weaknesses of an idea before placing hard cash on the line. If you have every had your ideas scrutizined by venture capitalists you know that VCs as a group have some of the best BS detectors on the planet. Having a PhD and a good reputation does not prevent you from being ripped to shreds by a potential investor who does not understand or accept your weak answer to his good question.

Finally, my critic has said the following:

What I want to say is that, as you have the full permission to modify and research/investigate your patent, you also have the right to show us, the audience, only what you want! Yes, it is a very unethical and bad atitude, but that can happen and would be very nice to sell your patent for other persons, making money ($$$) with it!!

I would prefer that my critic use namecalling somewhat less and make reasoned arguments somewhat more, but like the other parts of his series of comments, his words are baseless and wrong. Although I cannot know the inner workings of the mind that writes these words, I think I can speculate on the wellspring of my critic's confusion. The heart of his criticism is that my critic believes that academics work for the public at large and that the product of academic minds is a public good to be shared freely by all. Academics do perform much of their work publicly and freely, but the traditions of the academy have always encouraged outside consultation and publication of for-profit books (Should I not receive royalties for my books or consulting fees when I do consulting?). Therefore, the implicit agreement between academics and their universities has always recognized that academics can pursue outside interests that receive funding from private parties. Patenting by universities and faculty has a somewhat more recent history, but the principle is the same. Academics do not forego the right to profit from the products of their minds.

Moreover, it seems to me that my critic's words are also based on a bias against free markets, business, and commerce generally. I do believe in life, liberty, and the pursuit of GAs and free exchange among free people is an important way to guarantee those things. The record of the 20th century shows unquivocally the superiority of markets over government planning when it comes to advancing and spreading advanced technology. My critic believes that patenting the ideas behind hBOA will somehow slow the spread of the idea, but in fact the opposite is happening. Because Martin Pelikan and I have patented hBOA companies are interested in using and licensing the technology in commercial applications because they can obtain a license to the patent, protect themselves (temporarily) from their competition, and gain a competitive advantage in their niche. And this is exactly the point of intellectual property in general. If you give people a property right, they can exploit it, gain a foothold, and the idea will get traction in the marketplace. By this market reasoning, as good academics interested in the spread of our ideas, Martin and I would have been wrong NOT to patent hBOA and seek licensees for it. We would have committed malpractice against maximizing the influence of our thinking.

In short, none of the arguments made by this critic stand up to any scrutiny at all. A more reasoned understanding of our times, I believe, is leading and will lead others in our field to transfer their ideas into the real world where they can make important differences in everyday life. The logic of that technology transfer is leading universities and academics closer to commerce, and thus inexorably to patents. Choosing what to patent and what merely to publish is a tricky decision, but over time more of us will be led to conclude that patenting is increasingly right if we wish to maximize the influence and practice of our ideas.

is talking at NIGEL about using Pitt style LCSs for protein structure prediction. See here for pdf of presentation close to one he is presenting today.

Pelikan and Goldberg's patent, "A Method for Optimizing a Solution Set" issued today as US Patent No. 7,047,169. See here.

PLL is talking about computed prediction.

Ester is talking about using XCS in unbalanced datasets.

Jorge Casillas is talking about the group from Granada SCI2S and his working on scalable fuzzy XCS. He mentions the KEEL group in Spain, Knowledge Extraction Using Evolutionary Learning (see site here).

Alwyn Barry described some pretty work in placing a formal framework around all of classifier systems in the (a) dynamic programming formalism, (b) value function approximation, and (c) rule replacement.

Xavier Llora is talking about using chi-ary ECGA to learn rules.

Lashon is talking about the early days in the 70s and 80s of irrational exuberance about artificial intelligence and how Holland couldn't understand how symbolic AIers were going to avoid brittleness.

Needs then:

• exploit regularities in the environments
• generalizations selective, pragmatic, and subject to exceptions
• learning must be incremental and coupled to performance
• rules as tentative hypotheses, not consistent assertions

Classifiers evolved from those early days with a strong emphasis on cognition. Reinforcement learning came along (with roots out of UofM) and gave important formalism.

Lashon wonders if we are solving problems that take advantage of what classifier systems do well and how to describe the architecture of classifier systems that will. He contrasted LCSs with the success of SOAR in solving problems and becoming more widely accepted.

After break Martin Butz is up next.

I'm sitting listening to Dispankar Dasgupta talk about artificial immune systems at Xavier Llora's NIGEL workshop. This morning the talks are

• Stewart W. Wilson: "Can We Do Captchas?"
• David E. Goldberg: "Searle, Intentionality, and the Future of Classifier Systems"
• Dipankar Dasgupta: "Artificial Immune Systems in Anomaly Detection"
• Lashon Booker: "A Retrospective Look at Classifier System Research"

Stewart challenged us to go beyond "place/value functions" and work on problems that were more open ended. I challenged the field to work on creating "intentional machines" using some of Searle's ideas contained in Mind, Language & Society. Dipankar presented an overview of his work on artificial immune systems. Lashon is up next.

See my short commentary on Pink's A Whole New Mind at TEE here.

See here.

Classes ended Wednesday, and I am just getting my head screwed on straight again. They say you can't teach an old dog new tricks, but I just taught GE 331/IE 300 for the first time. The course is 1/3 probability and 2/3 statistics for engineers. Enjoyed the material, but doing a first-time pass at the material in front of 130+ students was a bit like doing a high wire act without a net after 2 weeks of practice.

In going through a fairly standard engineering probability and statistics curriculum, I have three comments. First, I agree (to some extent) with some critics of our field that we could as a group do a better job in using fairly standard statistical methods to present our findings. Fair enough. Second, it seems to me that the similarities and differences between GA/EC practice and the practice of design of experiments, ANOVA, and model building deserve closer scrutiny and articulation. The practice of estimation of distribution algorithms (EDAs) might benefit from paying attention to modern statistics and vice versa. Third, it seems to me that statistics (like most fields derived from mathematics) is concerned with rigor to the exclusion of economy. An engineering discipline of statistics would explicitly account for and permit relaxation of inferential rigor on economic grounds. Regular readers of this blog will recognize this as the economy of thought (see here) argument made in The Design of Innovation.

The list of papers to be presented at the Ninth International Workshop on Learning Classifier Systems (IWLCS 2006) can be found here.