AGS TechBlog
A source and journal of ideas

AFRL Logo

The Synthetic Teammate project is conducting research on synthetic agents that can be integrated into team training simulations. Key challenges in advancing this technology are designing and implementing the agents so that the agents closely match human behavior across several cognitive capacities. These capacities include situation assessment, task behavior, language comprehension, and language generation. The initial application for the synthetic teammate research is an agent functioning as the pilot of an Unmanned Aerial Vehicle (UAV) within a synthetic task environment. Another challenge is achieving a high enough level of performance from the agent that does not detract from team training for human particpants,
More information on the Synthetic Teammate project is available at The Synthetic Teammate Project.
The language comprehension component of the synthetic teammate research is based on Double R Theory. Double R Theory is a linguistic theory of the grammatical encoding and integration of referential and relational meaning in English. The associated grammar is fundamentally a Cognitive Linguistic theory and the use of the term “grammar” encompasses meaning, function and structure as it does in Cognitive Grammar (also known as Cognitive Semantics). More information as well as scientific publications can be found at the Double R Theory Site.

I have begun a look into openNLP, the Java library of natural language processing tools.
Those posts will be on the AI Forum (rather than posting them in two places).
You can find them at this link =>http://agsforum.agstechnet.com/index.php?board=2.0

-Stu

Last week I visited University of Georgia. While there, an old friend from the Defense Department asked me about network performance assessment. He wanted to know how to assess and measure whether the “network” is “up” or “down” for a large enterprise?

I would reword this question a bit to mirror the factors that are most important to the company’s management or customers. The reworded question could take several forms:

• To what extent is your company’s network functional for important services?
• Is the network processing all of the requested work or are parts of the network overloaded?
• Has the network been attacked or hacked? Are unauthorized persons accessing the network for services or information?

These are good questions for any company that runs a network or networks. Answering these types of questions can pose some problems. In isolation, the applications that provide most network services have no way of knowing the broader intent of network use and therefore can’t report “inappropriate” requests and actions. The shear number of services and methods (for example, at the application layer, BGP, DHCP, DNS, FTP, GTP, HTTP, IMAP, IRC, Megaco, MGCP, NNTP, NTP, POP, RIP, RPC, RTP, RTSP, SDP, SIP, SMTP, SNMP, SOAP, SSH, Telnet, TLS/SSL, XMPP, and at the internet Layer, IP (IPv4, IPv6), ICMP, ICMPv6, IGMP, IPsec, and many others) available on a network add to the difficulty. In overcoming this difficulty, artificial intelligence approaches can be highly effective in assisting network managers in understanding activities on the network.

Whether you work for the U.S. Department of Defense or a smaller enterprise, there are a large number of tools and applications available to assist the IT professional with monitoring the network, attached hardware, services and usage. Network tools and testing schemes can and do generate large amounts (megabytes+) of data concerning low-level network performance measures such as response times, packet losses, logged entries, etc. If you don’t have an Army of technicians and analysts to study all this information, then artificial intelligence methods can help analyze and reduce the quantity of data that humans must review and can point toward the most important issues that a human analyst should attend to.

artificial intelligence

So what might be an example of how to use artificial intelligence tools in the context of network monitoring, diagnosis, and remediation? Naturally, there is an almost unlimited variety of designs and techniques that could be applied in a hypothetical case. So what follows is one example of what could be done. Of course, the specifics of an actual network, its services, its hardware, and the company’s objectives would dictate the most appropriate techniques that should be applied in an actual case.

Very simple agent components can serve as the basis for network instrumentation. The agents reside on network nodes such as mail, web, or database servers, or other types of devices. These installed agents poll their assigned service or protocol, do simple metric collection, and send reports to each other for cooperative reasons and report back to a repository agent to add to the collective information about the network.

A machine learning system or classifier operates on the main data repositories looking for regularities or clusters within the data. These classified patterns can form the basis of summary reports to IT professionals for further action.

The summary reports can also be used by an expert system or rule-based production system to suggest or even autonomously take action to remediate problems within the network. The summary reports feed into a “dashboard” status system. Many human managers will prefer to review a “dashboard” type of status report which shows top level indicators of network health and possibly contains colored (red, yellow, green) status summaries for network operational functions which are of concern to the network owners.

So, the artificial intelligence tools applied (specifically, agents, machine learning/classifiers, expert systems, and rule-based production systems) work together to collect desired measures and information, analyze and summarize key events and information, report appropriate information to managers and even take action to remediate issues that are detected on the network.

I hope this generates some thinking on how to apply artificial intelligence techniques on real world applications.

-Stu

Day before yesterday, a colleague asked me if I had any tools that would be “good for managing a large set of airborne radars, electronic jammers, and related systems”.  We had a nice discussion about what artificial intelligence tools were good for and how to apply them on a large project.

What am I good for?

During our conversation, it occurred to me that there are probably a large number of “laymen” that are not part of the artificial intelligence research or engineering community that are in the same situation as my colleague.  She has heard the term “artificial intelligence” or AI, but doesn’t know whether AI tools and approaches would help her with her specific program.

So without a background in AI, how do managers make decisions about whether to pursue AI solutions in their endeavors? The sine qua non is a good layman’s outline of example occasions with a bit of explanation.

This needs to be something more than the typical description of how to apply a given technique to the traveling salesman problem or another classical artificial intelligence problem.

Given the need, I will be doing a series of short posts to scratch this itch.

Stay tuned….

The main site has had a face lift. You can see it at
AGS TechNet

Check it out.
Cheers!

So what do we mean when we say that a software system can be “adaptive”, “reactive” and “autonomous”?

For our purposes, we stick pretty close to the Webster definitions of the terms. But here is the gist of the terms as we use them:

“adaptive” - The software system contains a complex set of possible actions and affecters. It chooses from among these possible actions in combinations and orders that are not predetermined but rather are chosen by the software system to achieve the system’s goals as it deems best. So, the system “adapts” its behavior to its situation.

“reactive” - The software system maintains a perception of its “environment”. This environment may be a pure information environment or it may contain perceptions of the physical world. Both the system’s actions and the system’s goals will be modified in response to these perceptions. In other words, the system “reacts” to the world around it.

“autonomous” - The software system’s goals, actions and perceptions (within the scope of its design purpose) are sufficiently complex and robust such that the system can perceive, decide, react, act, and achieve its goals with very little interaction with human operators. The system largely acts on its own for extended periods of time. The distinction of “autonomous” and “semi-autonomous” are matters of the length of time without interaction and the degree of any interaction.

Hopefully, that helps clarify what we mean by “adaptive, reactive and autonomous software systems”.

-Stu

Loebner Prize Gold Medal

Since today is October 12th, 2008, this must be the weekend of the Loebner Prize Competition. I think I’ll stroll over to their website and see what is on tap today …..

Robert Medeksza (1st place in 2007, 2nd in 2006) with Ultra Hal and Rollo Carpenter (3rd place runner up from 2007, 1st place in 2006) with Jabberwacky are again finalists this year. However, I see that Noah Duncan (2nd place in 2007 and 3rd place in 2006) did not appear in the finals this year–not sure if he applied or not.

I don’t see anything that has yet been posted concerning results…. I’ll check back tomorrow.

In the meantime, if you want to check out the site for yourself, you can find it at: Loebner Prize. If you are not familiar with the Loebner Prize or contest, the site offers a good amount of detail. Essentially, the Loebner Competition is a contest based on the Turing Test–whether a judge can tell the difference between a human and a computer in a typed (”online”) conversation.

-Stu

Today, I noticed the article, “Linking Brains, Computers” by Lee Gomes in the Wall Street Journal, Online Edition.

http://online.wsj.com/public/article/SB121554314155636505-sVIxBrTvKgZgQUFUCQbN_pBncKw_20080808.html

Because I am a sharing kind-o-guy, here is the note I sent to Lee:

Lee,

            I just saw your WSJ article from July 9^th. I appreciate you taking the time to point out the fallacy of comparing synapses to transistors. I hope that was your main point in reporting yours and Dr Koch’s view on the subject.

            I may agree that a statement like the following is rubbish: “We are close to producing human level cognitive performance in a computer because of the transistor count.”

            However, I would totally disagree if the statement was more like this: “We don’t understand the brain well enough to produce superhuman cognitive performance from a computer.”

            We already have many machines that outperform humans on many, many tasks. Examples are databases for fact recall, data mining systems for analyzing large data sets, straightforward number crunching for problem solving speed, etc.  Just like a robotic backhoe can dig ditches faster than a human with a shovel; computers are much better suited to some tasks than humans.

            It is not clear at all that we need to understand every detail of how the human brain works in order to make machines that can do better than humans on important tasks.

            And of course, I could care less whether or not a computer can be made to interact and dialog with me on the full breadth of interesting topics spanning from good wines to music, poetry and other arts.  For that activity, we already have humans…..

Sincerely,

Stu Rodgers
AGS TechNet
“Artificial Intelligence, Cognitive Modeling and Engineering”
Dayton, OH
www.agstechnet.com

This morning we signed up as a sponsor for the International Conference on Cognitive Modeling 2009 in Manchester, UK.

The conference is an outstanding global forum on the computational modeling of human cognition. You can find additional information at the ICCM 2009 web site.

I hope to see you there!

-Stu

Interesting article the other day at PhysOrg concerning artificial intelligence research and playing poker at the University of Alberta.

I wrote a post on it in the AI Forum. Rather than repeating the whole thing here, I’ll just point you to the post over there: http://agsforum.agstechnet.com/index.php?topic=22.0

-Stu

We were fortunate to have AFRL research scientist, Dr. Kevin Gluck, visit our new location last Thursday. During the meeting, we discussed our current project: math modeling of human performance in complex environments. Following our weekly teleconference with the research team and demonstrations of the Predictive Performance Optimizer software tool, we enjoyed some informal discussions over dinner.

Dr. Gluck’s visit was an excellent kickoff for our new location!

-Stu

As you might have noticed from the new address that appears below the logo on the main site, we have recently changed our address.

It is possible that this will be the last in a very long string of relocations (moves)–hopefully!

At least this move comes with a (slightly) bigger office.

-Stu

As we continue to build our application development capability and processes, we are very happy that we have been accepted as a NetBeans’ Community Partner. You can find out more about the NetBeans Community Partner program at: http://www.netbeans.org/community/partners/list-community.html

The MindModeling@Home site ( http://mindmodeling.org ) is up and running.  At AGS TechNet, we are volunteering some of our computer time to assist them in their goals. 

Volunteer computing is a way to use your computer’s “idle” time to contribute some number crunching power to assist in computationally intensive research work.

If you are interested, it is fairly simple to get set up and all the steps are explained on MindModeling’s site.  Essentially, you just download and install an application called “BOINC” from UC Berkeley’s site. It allows you to set how much and under what conditions your own computer will get and execute jobs for the research effort.  Next, using BOINC you join the volunteer computing effort(s) that you want to support.  Then, when you are not using your computer, it will download jobs, execute them, and upload the results.  There are many Universities, researchers, companies, and other interested folks that have been participating since 1995 in volunteer computing efforts.  Today, there are over a million BOINC users and the scale of the overall computing effort puts the volunteer computing community easily in the league of the most powerful supercomputers.

If you haven’t heard of it before, you might check it out. Very cool!

-Stu

As I mentioned in some earlier posts, I have yet to find a good site that reviews, rates, lists pros and cons, and similar things for the many artificial intelligence tools, approaches and systems you can find on the web.

Today, we are kicking off our new forum that will hopefully scratch this itch. The new forum can be found at http://agsforum.agstechnet.com

The reviews I intend to do on the forum are not meant as a criticism to anybody’s work, but I feel that many people will be able to use the information to more quickly find the right tools to help them with whatever application they are pursuing.

Its brand new, so there is no meat there yet. I will be writing about systems I know about as much as time permits. Please chip in if you are so inclined.

Enjoy!

-Stu

Check out the beginning of the new surveys page: http://www.agstechnet.com/surveys.html

I will be using the surveys to best focus on the AI tools, systems, and techniques that you think matter most.

This is intended to follow up on my previous post: http://www.agstechnet.com/blog/2007/11/29/the-really-big-list-of-ai-tools/

If you have thoughts or questions concerning a community feedback site for artificial intelligence tools and techniques, please send me a note.

-Stu 

A client recently asked me what I meant when I refered to “heuristic search.”  Heuristic search is performed when “brute force search” (or all-possible-paths-search) is computationally intractable.  Essentially, you can think of a heuristic as a “rule of thumb” or an approach to calculate an approximate answer.  Here’s a more detailed description…

A heuristic is defined by Meriam Webster’s on line dictionary as follows:

Main Entry: heu·ris·tic
Pronunciation: hyu-’ris-tik
Function: adjective
Etymology: German heuristisch, from New Latin heuristicus, from Greek heuriskein to discover; akin to Old Irish fo-fúair he found
: involving or serving as an aid to learning, discovery, or problem-solving by experimental and especially trial-and-error methods heuristic assumption>; also
: of or relating to exploratory problem-solving techniques that utilize self-educating techniques (as the evaluation of feedback) to improve performance heuristic computer program>

Heuristics are used in the context of high complexity to find good solutions to problems that might be impossible to solve for an exact or the globally optimum solution.

Examples of the use of heuristic search are found in many common optimization problems.  In many optimization efforts, computations can take billions of billions of years to find the exact solution even on the fastest computers. This is a case where using heuristics may allow you to find a good solution even if the optimal solution is unknown.

-Stu

Researchers at the National Institute of Standards and Technology (NIST) and the University of Texas, Arlington have released a variety of briefings, reports, and results concerning the automated combinatorial testing for software.   Many of the major news feeds were carrying the stories yesterday.

After reviewing their reports and findings, it sounds like good news for automated testing systems.

There never has been enough time or money to do all the testing that could or should be done for any non-trivial project.  But these results at least provide some support to the idea that a substantial majority of software defects can be effectively captured by automated testing with something less than full-exhaustive testing.  See the following link for the full article, the reports and the results: http://csrc.nist.gov/groups/SNS/acts/index.html

It is normal in the software industry to adopt approaches to testing that cover the most commonly used features and cases.  In my experience, you normally get the most bang for the buck by focusing on expected use cases. 

Still, these results were encouraging enough to motivate me to look into the new tool to be released to see how I like it.  I sent Rick Kuhn a note today to get some info.  I will post the results of my assessment of the tool when I am finished.

-Stu

My search has not been in vain.  The biggest (I am not suggesting that size is the most important metric) list found so far is on none other than AAAI’s site.  You can see it at:

 http://www.aaai.org/AITopics/html/soft.html

 It is a general list of the full spectrum of AI tools–and it is more comprehensive than anything else I’ve found.  The contents relating to natural language processing seems to be good.

Unfortunately, there still is no sense of which tools are best for certain applications, what is highly supported, etc.  I think this would be enormously useful–especially for small teams looking for components that already fill a need. 

So last night I sent Carol Hamilton, AAAI’s Executive Director, a note asking about possibly making a change to the site.  I hope that together we will be able to gin up something useful.

Two other good lists found seem worth mentioning (in no particular order) that are specific to NLP:

• http://opennlp.sourceforge.net/projects.html
• http://aclweb.org/aclwiki/index.php?title=State_of_the_art

More to follow.

- Stu

While at the AAAI Fall Symposium this week, it occurred to me that there may not be a really good, reasonably complete, business oriented open source site for tools that businesses, especially small businesses, can use in natural language processing applications.

To be useful to small businesses, the tools would need to be well written, well documented, and constructed in a way that enabled straightforward interface connections between the various parts.  I started making a list of possible component tools that might be included in such a list.  The list is a draft list of course, but hopefully some of you readers out there will help out and give me some additional, maybe even better ideas.

Draft List of Possible NLP Components in an Open Source Portal for Business

• Parser
• Grammar - context free or context sensitive
• Grammar editing tool
• Training text corpus
• Front end user interface
• Synonym dictionary and editor
• Ontology and editor
• Tutorials and documentation for integrating into applications

So far, I have looked around at several sites including NLUC, OpenNLP, and several wikis and university sites.  Several offer some very useful material, but none seemed to offer the “one stop shop” that I had in mind.

If you have some ideas, please leave a comment or send me a note.  If I can’t find something within a few weeks, then I will start ginning one up.

-Stu