AGS TechBlog
A source and journal of ideas

Last week I visited University of Georgia. While there, an old friend from the Defense Department asked me about network performance assessment. He wanted to know how to assess and measure whether the “network” is “up” or “down” for a large enterprise?

I would reword this question a bit to mirror the factors that are most important to the company’s management or customers. The reworded question could take several forms:

• To what extent is your company’s network functional for important services?
• Is the network processing all of the requested work or are parts of the network overloaded?
• Has the network been attacked or hacked? Are unauthorized persons accessing the network for services or information?

These are good questions for any company that runs a network or networks. Answering these types of questions can pose some problems. In isolation, the applications that provide most network services have no way of knowing the broader intent of network use and therefore can’t report “inappropriate” requests and actions. The shear number of services and methods (for example, at the application layer, BGP, DHCP, DNS, FTP, GTP, HTTP, IMAP, IRC, Megaco, MGCP, NNTP, NTP, POP, RIP, RPC, RTP, RTSP, SDP, SIP, SMTP, SNMP, SOAP, SSH, Telnet, TLS/SSL, XMPP, and at the internet Layer, IP (IPv4, IPv6), ICMP, ICMPv6, IGMP, IPsec, and many others) available on a network add to the difficulty. In overcoming this difficulty, artificial intelligence approaches can be highly effective in assisting network managers in understanding activities on the network.

Whether you work for the U.S. Department of Defense or a smaller enterprise, there are a large number of tools and applications available to assist the IT professional with monitoring the network, attached hardware, services and usage. Network tools and testing schemes can and do generate large amounts (megabytes+) of data concerning low-level network performance measures such as response times, packet losses, logged entries, etc. If you don’t have an Army of technicians and analysts to study all this information, then artificial intelligence methods can help analyze and reduce the quantity of data that humans must review and can point toward the most important issues that a human analyst should attend to.

artificial intelligence

So what might be an example of how to use artificial intelligence tools in the context of network monitoring, diagnosis, and remediation? Naturally, there is an almost unlimited variety of designs and techniques that could be applied in a hypothetical case. So what follows is one example of what could be done. Of course, the specifics of an actual network, its services, its hardware, and the company’s objectives would dictate the most appropriate techniques that should be applied in an actual case.

Very simple agent components can serve as the basis for network instrumentation. The agents reside on network nodes such as mail, web, or database servers, or other types of devices. These installed agents poll their assigned service or protocol, do simple metric collection, and send reports to each other for cooperative reasons and report back to a repository agent to add to the collective information about the network.

A machine learning system or classifier operates on the main data repositories looking for regularities or clusters within the data. These classified patterns can form the basis of summary reports to IT professionals for further action.

The summary reports can also be used by an expert system or rule-based production system to suggest or even autonomously take action to remediate problems within the network. The summary reports feed into a “dashboard” status system. Many human managers will prefer to review a “dashboard” type of status report which shows top level indicators of network health and possibly contains colored (red, yellow, green) status summaries for network operational functions which are of concern to the network owners.

So, the artificial intelligence tools applied (specifically, agents, machine learning/classifiers, expert systems, and rule-based production systems) work together to collect desired measures and information, analyze and summarize key events and information, report appropriate information to managers and even take action to remediate issues that are detected on the network.

I hope this generates some thinking on how to apply artificial intelligence techniques on real world applications.

-Stu

So what do we mean when we say that a software system can be “adaptive”, “reactive” and “autonomous”?

For our purposes, we stick pretty close to the Webster definitions of the terms. But here is the gist of the terms as we use them:

“adaptive” - The software system contains a complex set of possible actions and affecters. It chooses from among these possible actions in combinations and orders that are not predetermined but rather are chosen by the software system to achieve the system’s goals as it deems best. So, the system “adapts” its behavior to its situation.

“reactive” - The software system maintains a perception of its “environment”. This environment may be a pure information environment or it may contain perceptions of the physical world. Both the system’s actions and the system’s goals will be modified in response to these perceptions. In other words, the system “reacts” to the world around it.

“autonomous” - The software system’s goals, actions and perceptions (within the scope of its design purpose) are sufficiently complex and robust such that the system can perceive, decide, react, act, and achieve its goals with very little interaction with human operators. The system largely acts on its own for extended periods of time. The distinction of “autonomous” and “semi-autonomous” are matters of the length of time without interaction and the degree of any interaction.

Hopefully, that helps clarify what we mean by “adaptive, reactive and autonomous software systems”.

-Stu

This morning we signed up as a sponsor for the International Conference on Cognitive Modeling 2009 in Manchester, UK.

The conference is an outstanding global forum on the computational modeling of human cognition. You can find additional information at the ICCM 2009 web site.

I hope to see you there!

-Stu

Interesting article the other day at PhysOrg concerning artificial intelligence research and playing poker at the University of Alberta.

I wrote a post on it in the AI Forum. Rather than repeating the whole thing here, I’ll just point you to the post over there: http://agsforum.agstechnet.com/index.php?topic=22.0

-Stu

We were fortunate to have AFRL research scientist, Dr. Kevin Gluck, visit our new location last Thursday. During the meeting, we discussed our current project: math modeling of human performance in complex environments. Following our weekly teleconference with the research team and demonstrations of the Predictive Performance Optimizer software tool, we enjoyed some informal discussions over dinner.

Dr. Gluck’s visit was an excellent kickoff for our new location!

-Stu

As we continue to build our application development capability and processes, we are very happy that we have been accepted as a NetBeans’ Community Partner. You can find out more about the NetBeans Community Partner program at: http://www.netbeans.org/community/partners/list-community.html

The MindModeling@Home site ( http://mindmodeling.org ) is up and running.  At AGS TechNet, we are volunteering some of our computer time to assist them in their goals. 

Volunteer computing is a way to use your computer’s “idle” time to contribute some number crunching power to assist in computationally intensive research work.

If you are interested, it is fairly simple to get set up and all the steps are explained on MindModeling’s site.  Essentially, you just download and install an application called “BOINC” from UC Berkeley’s site. It allows you to set how much and under what conditions your own computer will get and execute jobs for the research effort.  Next, using BOINC you join the volunteer computing effort(s) that you want to support.  Then, when you are not using your computer, it will download jobs, execute them, and upload the results.  There are many Universities, researchers, companies, and other interested folks that have been participating since 1995 in volunteer computing efforts.  Today, there are over a million BOINC users and the scale of the overall computing effort puts the volunteer computing community easily in the league of the most powerful supercomputers.

If you haven’t heard of it before, you might check it out. Very cool!

-Stu

As I mentioned in some earlier posts, I have yet to find a good site that reviews, rates, lists pros and cons, and similar things for the many artificial intelligence tools, approaches and systems you can find on the web.

Today, we are kicking off our new forum that will hopefully scratch this itch. The new forum can be found at http://agsforum.agstechnet.com

The reviews I intend to do on the forum are not meant as a criticism to anybody’s work, but I feel that many people will be able to use the information to more quickly find the right tools to help them with whatever application they are pursuing.

Its brand new, so there is no meat there yet. I will be writing about systems I know about as much as time permits. Please chip in if you are so inclined.

Enjoy!

-Stu

Researchers at the National Institute of Standards and Technology (NIST) and the University of Texas, Arlington have released a variety of briefings, reports, and results concerning the automated combinatorial testing for software.   Many of the major news feeds were carrying the stories yesterday.

After reviewing their reports and findings, it sounds like good news for automated testing systems.

There never has been enough time or money to do all the testing that could or should be done for any non-trivial project.  But these results at least provide some support to the idea that a substantial majority of software defects can be effectively captured by automated testing with something less than full-exhaustive testing.  See the following link for the full article, the reports and the results: http://csrc.nist.gov/groups/SNS/acts/index.html

It is normal in the software industry to adopt approaches to testing that cover the most commonly used features and cases.  In my experience, you normally get the most bang for the buck by focusing on expected use cases. 

Still, these results were encouraging enough to motivate me to look into the new tool to be released to see how I like it.  I sent Rick Kuhn a note today to get some info.  I will post the results of my assessment of the tool when I am finished.

-Stu

While at the AAAI Fall Symposium this week, it occurred to me that there may not be a really good, reasonably complete, business oriented open source site for tools that businesses, especially small businesses, can use in natural language processing applications.

To be useful to small businesses, the tools would need to be well written, well documented, and constructed in a way that enabled straightforward interface connections between the various parts.  I started making a list of possible component tools that might be included in such a list.  The list is a draft list of course, but hopefully some of you readers out there will help out and give me some additional, maybe even better ideas.

Draft List of Possible NLP Components in an Open Source Portal for Business

• Parser
• Grammar - context free or context sensitive
• Grammar editing tool
• Training text corpus
• Front end user interface
• Synonym dictionary and editor
• Ontology and editor
• Tutorials and documentation for integrating into applications

So far, I have looked around at several sites including NLUC, OpenNLP, and several wikis and university sites.  Several offer some very useful material, but none seemed to offer the “one stop shop” that I had in mind.

If you have some ideas, please leave a comment or send me a note.  If I can’t find something within a few weeks, then I will start ginning one up.

-Stu