Publications

1996
M. Tambe, W. L. Johnson, and W. Shen. 1996. “ Adaptive agent tracking in real-world multiagent domains: A preliminary report.” In AAAI Spring Symposium on Adaptation, Coevolution and Learning in Multiagent Systems.Abstract
In multi-agent environments, the task of agent track-ing (i.e., tracking other agents’ mental states) in-creases in difficulty when a tracker (tracking agent)only has an imperfect model of the trackee (trackedagent). Such model imperfections arise in many real-world situations, where a tracker faces resource con-straints and imperfect information, and the trackeesthemselves modify their behaviors dynamically. Whilesuch model imperfections are unavoidable, a trackermust nonetheless attempt to be adaptive in its agenttracking. In this paper, we analyze some key issuesin adaptive agent tracking, and describe an initial ap-proach based on discrimination-based learning. Themain idea is to identify the deficiency of a model basedon prediction failures, and revise the model by usingfeatures that are critical in discriminating successfuland failed episodes. Our preliminary experiments insimulated air-to-air combat environments have shownsome promising results but many problems remainopen for future research.
1996_8_teamcore_ss96_01_018.pdf
Milind Tambe and P. S. Rosenbloom. 1996. “Event tracking in a dynamic multi-agent environment.” Computational Intelligence 12.Abstract
This paper focuses on event tracking in one complex and dynamic multi-agent environment: the air-combat simulation environment. It analyzes the challenges that an automated pilot agent must face when tracking events in this environment. This analysis reveals three new issues that have not been addressed in previous work in this area: (i) tracking events generated by agents' flexible and reactive behaviors, (ii) tracking events in the context of continuous agent interactions, and (iii) tracking events in real-time. The paper proposes one solution to address these issues. One key idea in this solution is that the (architectural) mechanisms that an agent employs in generating its own flexible and reactive behaviors can be used to track other agents' flexible and reactive behaviors in real-time. A second key idea is the use of a world-centered representation for modeling agent interactions. The solution is demonstrated using an implementation of an automated pilot agent.
1996_4_teamcore_final3.pdf
Milind Tambe. 1996. “ Executing Team Plans in Dynamic Multi-agent environments.” In AAAI Fall Symposium on Plan Execution.Abstract
This paper focuses on flexible teamwork in dynamic and real-world multi-agent domains. Such teamwork is not simply a union of agents’ simultaneous execution of individual plans,even if such execution pre-coordinated. Indeed, uncertain-ties in complex, dynamic domains often obstruct pre-plannedcoordination, with a resultant breakdown in teamwork. Thecentral hypothesis in this paper is that for durable teamwork,agents should be provided explicit team plans, which directlyexpress a team’s joint activities. When agents execute such team plans, they abide by certain "commonsense" conventions of teamwork. Essentially, such conventions providea deeper model of teamwork, facilitating flexible reasoningabout coordination activities. Such a framework also frees the planner or the knowledge engineer from specifying very detailed low-level coordination plans. This framework has been implemented in the context of a real-world synthetic environment for helicopter-combat simulation.
1996_2_teamcore_team_plan2_final.pdf
1996. Intelligent Agents: Vol II, Workshop on Agents, theories, architectures and languages (ATAL-95). Publisher's VersionAbstract
This book is based on the second International Workshop on Agent Theories, Architectures, and Languages, held in conjunction with the International Joint Conference on Artificial Intelligence, IJCAI'95 in Montreal, Canada in August 1995.
The 26 papers are revised final versions of the workshop presentations selected from a total of 54 submissions; also included is a comprehensive introduction, a detailed bibliography listing 355 relevant publications, and a subject index. The book is structured into seven sections, reflecting the most current major directions in agent-related research. Together with its predecessor, Intelligent Agents, published as volume 890 in the LNAI series, this book provides a timely and comprehensive state-of-the-art report.
Milind Tambe. 1996. “ Teamwork in real-world, dynamic environments.” In International conference on multi-agent systems (ICMAS96).Abstract
Flexibleteamworkinreal-worldmulti-agentdomainsismorethana unionofagents’simultaneousexecutionofindividualplans,evenif suchexecutionpre-coordinated.Indeed,uncertaintiesincomplex,dynamicdomainsoftenobstructpre-plannedcoordination,witha resultantbreakdowninteamwork.Thecentralhypothesisinthispaperis thatfordurableteamwork,agentsshouldbeprovidedexplicitteamplansandanunderlyingmodelof teamworkthatexplicitlyoutlinestheircommitmentsandresponsibilitiesasparticipantsin teamactivities.Sucha modelenablesteammemberstoflexiblyreasonaboutcoordinationactivities.Theunderlyingmodelwehaveprovidedisbasedonthejointintentionsframework;althoughwepresentsomekeymodificationstoreflectthepracticalconstraintsin(some)real-worlddomains.Thisframeworkhasbeenimplementedinthecontextofa real-worldsyntheticenvironmentforhelicopter-combatsimulation;someempiricalresultsarepresented.
1996_1_teamcore_tambe_teamwork_1996.pdf
Milind Tambe. 1996. “ Tracking dynamic team activity.” In National Conference on Artificial Intelligence (AAAI96).Abstract
AI researchers are striving to build complex multi-agent worlds with intended applications ranging from the RoboCup robotic soccer tournaments, to interactive virtual theatre, to large-scale real-world battlefield simulations. Agent tracking --- monitoring other agent's actions and inferring their higher-level goals and intentions --- is a central requirement in such worlds. While previous work has mostly focused on tracking individual agents, this paper goes beyond by focusing on agent teams. Team tracking poses the challenge of tracking a team's joint goals and plans. Dynamic, real-time environments add to the challenge, as ambiguities have to be resolved in real-time.
1996_6_teamcore_aaai96team_final.pdf
Milind Tambe. 1996. “ Tracking dynamic team activity: An extended report.” USC ISI Technical report RR, Pp. 96-435.Abstract
AI researchers are striving to build complex multi-agent worlds with intended applications ranging from the RoboCup robotic soccer tournaments, to interactive virtual theatre, to large-scale real-world battlefield simulations. Agent tracking --- monitoring other agent's actions and inferring their higher-level goals and intentions --- is a central requirement in such worlds. While previous work has mostly focused on tracking individual agents, this paper goes beyond by focusing on agent teams. Team tracking poses the challenge of tracking a team's joint goals and plans. Dynamic, real-time environments add to the challenge, as ambiguities have to be resolved in real-time.
1996_7_teamcore_rr_96_435.pdf
Milind Tambe and P. S. Rosenbloom. 1996. “Architectures for agents that track other agents in multi-agent worlds.” In Intelligent Agents, Vol II Springer Verlag Lecture Notes in Artificial Intelligence (LNAI 1037).Abstract
In multi-agent environments, an intelligent agent often needs to interact with other individuals or groups of agents to achieve its goals. Agent tracking is one key capability required for intelligent interaction. It involves monitoring the observable actions of other agents and inferring their unobserved actions, plans, goals and behaviors. This article examines the implications of such an agent tracking capability for agent architectures. It specifically focuses on real-time and dynamic environments, where an intelligent agent is faced with the challenge of tracking the highly flexible mix of goal-driven and reactive behaviors of other agents, in real-time. The key implication is that an agent architecture needs to provide direct support for flexible and efficient reasoning about other agents' models. In this article, such support takes the form of an architectural capability to execute the other agent's models, enabling mental simulation of their behaviors. Other architectural requirements that follow include the capabilities for (pseudo-) simultaneous execution of multiple agent models, dynamic sharing and unsharing of multiple agent models and high bandwidth inter-model communication. We have implemented an agent architecture, an experimental variant of the Soar integrated architecture, that conforms to all of these requirements. Agents based on this architecture have been implemented to execute two different tasks in a real-time, dynamic, multi-agent domain. The article presents experimental results illustrating the agents' dynamic behavior.
1996_3_teamcore_atal2.pdf
1995
Milind Tambe, K. Schwamb, and P. S. Rosenbloom. 1995. “ Constraints and design choice's in building intelligent pilots for simulated aircraft: Extended Abstract.” In AAAI Spring Symposium on 'Lessons Learned from Implemented Software Architectures for Physical Agents'.Abstract
This paper focuses on our recent research effort aimedat developing human-like, intelligent agents (virtualhumans) for large-scale, interactive simulationenvironments (virtual reality). These simulatedenvironments have sufficiently high fidelity andrealism[l 1,23] that constructing intelligent agentsrequires us to face many of the hard research challengesfaced by physical agents in the real world -- inparticular, the integration of a variety of intelligentcapabilities, including goal-driven behavior, reactivity,real-time performance, planning, learning, spatial andtemporal reasoning, and natural languagecommunication. However, since this is a syntheticenvironment, these intelligent agents do not have to dealwith issues of low-level perception and robotic control.Important applications of this agent technology can be found in areas such as education [14],manufacturing [11],entertainment [2, 12]and training.
1995_5_teamcore_writeup4.pdf
Milind Tambe. 1995. “ Recursive agent and agent-group tracking in a real-time dynamic environment.” In International conference on multi-agent systems (ICMAS).Abstract
Agent tracking is an important capability an intelligent agent requires for interacting with other agents. It involves monitoring the observable actions of other agents as well as inferring their unobserved actions or high-level goals and behaviors. This paper focuses on a key challenge for agent tracking: recursive tracking of individuals or groups of agents. The paper first introduces aa approach for tracking recursive agent models. To tame the resultant growth in the tracking effort and aid real-time performance, the paper then presents model sharing, an optimization that involves sharing the effort of tracking multiple models. Such shared models are dynamically unshared as needed -- in effect, a model is selectively tracked if it is dissimilar enough to require unsharing. The paper also discusses the application of recursive modeling in service of deception, and the impact of sensor imperfections. This investigation is based on our on-going effort to build intelligent pilot agents for a real-world synthetic air-combat environment.
1995_4_teamcore_tambe95recursive.pdf
Milind Tambe and P. S. Rosenbloom. 1995. “ RESC: An approach for dynamic, real-time agent tracking.” In International joint conference on Artificial Intelligence (IJCAI), 3rd ed., 12: Pp. 499-522.Abstract
Agent tracking involves monitoring the observ­able actions of other agents as well as infer­ring their unobserved actions, plans, goals and behaviors. In a dynamic, real-time environ­ment, an intelligent agent faces the challenge of tracking other agents' flexible mix of goal-driven and reactive behaviors, and doing so in real-time, despite ambiguities. This paper presents RESC (REal-time Situated Commit­ments), an approach that enables an intelligent agent to meet this challenge. RESC's situat-edness derives from its constant uninterrupted attention to the current world situation — it always tracks other agents' on-going actions in the context of this situation. Despite ambigu­ities, RESC quickly commits to a single inter­pretation of the on-going actions (without an extensive examination of the alternatives), and uses that in service of interpretation of future actions. However, should its commitments lead to inconsistencies in tracking, it uses single-state backtracking to undo some of the commit­ments and repair the inconsistencies. Together, RESC's situatedness, immediate commitment, and single-state backtracking conspire in pro­viding RESC its real-time character. RESC is implemented in the context of intelli­gent pilot agents participating in a real-world synthetic air-combat environment. Experimen­tal results illustrating RESC's effectiveness are presented.
1995_3_teamcore_resc.pdf
W. Lewis Johnson and Milind Tambe. 1995. “ Using Machine Learning To Extend Autonomous Agent Capabilities.” In Summer Computer Simulation Conference.Abstract
What kinds of knowledge can Soar/IFOR agents learn in the combat simulation environment? In our investigations so far, we have found a number of learning opportunities in our systems, which yield several types of learned rules. For example, some rules speed up the agents' decision making, while other rules reorganize the agent's tactical knowledge for the purpose of on-line explanation generation. Yet, it is also important to ask a second question: Can machine learning make a significant difference in Soar/IFOR agent performance? The main issue here is that battlefield simulations are a real-world application of AI technology. The threshold which machine learning must surpass in order to be useful in this environment is therefore quite high. It is not sufficient to show that machine learning is applicable "in principle" via small-scale demonstrations; we must also demonstrate that learning provides significant benefits that outweigh any hidden costs. Thus, the overall objective of this work is to determine how machine learning can provide practical benefits to real-world applications of artificial intelligence. Our results so far have identified instances where machine learning succeeds in meeting these various requirements, and therefore can be an important resource in agent development. We have conducted extensive learning experiments in the laboratory, and have conducted demonstrations employing agents that learn; to date, however, learning has not yet been employed in large-scale exercises. The role of machine learning in Soar/IFOR is expected to broaden as practical impediments to learning are resolved, and the capabilities that agents are expected to exhibit are broadened.
1995_6_teamcore_johnson_tambe95.pdf
Milind Tambe, K. Schwamb, and P. S. Rosenbloom. 1995. “Building intelligent pilots for simulated rotary wing aircraft.” In Conference on computer generated forces and behavioral representation.Abstract
Abstract There are two RWA in the scenario, just behind the The Soar/IFOR project has been developing ridge, indicated by the contour lines. The other intelligent pilot agents (henceforth IPs) for vehicles in the figure are a convoy of "enemy" participation in simulated battlefield environments. ground vehicles  tanks and anti-aircraft vehicles  While previously the project was mainly focused on controlled by ModSAF. The RWA are IPs for fixed-wing aircraft (FWA), more recently, the approximately 2.5 miles from the convoy. The IPs project has also started developing IPs for rotaryhave hidden their helicopters behind the ridge (their wing aircraft (RWA). This paper presents a approximate hiding area is specified to them in preliminary report on the development of IPs for advance). They unmask these helicopters by popping RWA. It focuses on two important issues that arise in out from behind the ridge to launch missiles at the this development. The first is a requirement for enemy vehicles, and quickly remask (hide) by reasoning about the terrain  when compared to an dipping behind the ridge to survive retaliatory FWA IP, an RWA IP needs to fly much closer to the attacks. They subsequently change their hiding terrain and in general take advantage of the terrain for position to avoid predictability when they pop out cover and concealment. The second issue relates to later. code and concept sharing between the FWA and RWA IPs. While sharing promises to cut down the development time for RWA IPs by taking advantage of our previous work for the FWA, it is not straightforward. The paper discusses the two issues in some detail and presents our initial resolutions of these issues.
1995_2_teamcore_rwa_final.pdf
Milind Tambe, W. L. Johnson, R. M. Jones, F. Koss, J. E. Laird, P. S. Rosenbloom, and K Schwamb. 1995. “Intelligent Agents for Interactive Simulation Environments.” AI Magazine 16 (1), Pp. 15-39 .Abstract
Interactive simulation environments constitute one of today’s promising emerging technologies, withapplications in areas such as education, manufacturing, entertainment and training. These environmentsare also rich domains for building and investigating intelligent automated agents, with requirements forthe integration of a variety of agent capabilities, but without the costs and demands of low-levelperceptual processing or robotic control.Our project is aimed at developing human-like, intelligent agents that can interact with each other, as wellas with humans in such virtual environments. Our current target is intelligent automated pilots forbattlefield simulation environments. These are dynamic, interactive, multi-agent environments that poseinteresting challenges for research on specialized agent capabilities as well as on the integration of thesecapabilities in the development of "complete" pilot agents. We are addressing these challenges throughdevelopment of a pilot agent, called TacAir-Soar, within the Soar architecture.The purpose of this article is to provide an overview of this domain and project by analyzing thechallenges that automated pilots face in battlefield simulations, describing how TacAir-Soar issuccessfully able to address many of themTacAir-Soar pilots have already successfully participated inconstrained air-combat simulations against expert human pilotsand discussing the issues involved inresolving the remaining research challenges
1995_1_teamcore_tambe95intelligent.pdf
1994
Milind Tambe, R. M. Jones, J. E. Laird, P. S. Rosenbloom, and K. Schwamb. 1994. “ Building believable agents for simulation environments: Extended Abstract.” In AAAI Spring Symposium on 'Believable Agents'.Abstract
The goal of our research effort is to develop generic technology for intelligent automated agents in simulation environments. These agents are to behave believably like humans in these environments. In this context, believability refers to the indistinguishability of these agents from humans, given the task being performed, its scope, and the allowable mode(s) of interaction during task performance. For instance, for a given simulation task, one allowable mode of interaction with an agent may be typewritten questions and answers on a limited subject matter. Alternatively, a different allowable mode of interaction for the same (or different) task may be speech rather than typewritten words. In all these cases, believability implies that the agent must be indistinguishable from a human, given the particular mode of interaction.
1994_2_teamcore_symp_paper.pdf
Milind Tambe and Paul S. Rosenbloom. 1994. “ Event Tracking for an Intelligent Automated Agent.” In Time94: An international workshop on temporal representation and reasoning. .Abstract
In a dynamic, multi-agent environment, an automated intelligent agent is often faced with the possibility that other agents may instigate events that actually hinder or help the achievement of its own goals. To act intelligently in such an environment, an automated agent needs an event tracking capability to continually monitor the occurrence of such events and the temporal relationships among them.  This capability enables an agent to infer the occurrence of important unobserved events as well as obtain a better understanding of interaction among events. This paper focuses on event tracking in one complex and dynamic multi-agent environment: the air-combat simulation environment. It analyzes the challenges that an automated pilot agent must face when tracking events in this environment. This analysis reveals some novel constraints on event tracking that arise from complex multi-agent interactions. The paper proposes one solution to address these constraints, and demonstrates it using a simple re-implementation of an existing automated pilot agent.
1994_5_teamcore_time_final1994.pdf
Milind Tambe and P. S. Rosenbloom. 1994. “ Event tracking in complex multiagent environments.” In Conference on computer generated forces and behavioral representation .Abstract
This paper focuses on event tracking in one complex and dynamic multi-agent environment: the air-combat simulation environment. It analyzes the challenges that an automated pilot agent must face when tracking events in this environment. This analysis reveals three new issues that have not been addressed in previous work in this area: (i) tracking events generated by agents' flexible and reactive behaviors, (ii) tracking events in the context of continuous agent interactions, and (iii) tracking events in real-time. The paper proposes one solution to address these issues. One key idea in this solution is that the (architectural) mechanisms that an agent employs in generating its own flexible and reactive behaviors can be used to track other agents' flexible and reactive behaviors in real-time. A second key idea is the use of a world-centered representation for modeling agent interactions. The solution is demonstrated using an implementation of an automated pilot agent.
1994_1_teamcore_pr_final.pdf
Paul S. Rosenbloom, W. Lewis Johnson, Randolph M. Jones, Frank Koss, and John E. Laird. 1994. “ Intelligent Automated Agents for Tactical Air Simulation: A Progress Report.” In Fourth Conference on Computer Generated Forces and Behavioral Representation. Orlando, FL.Abstract

time, flexibly use a small amount of tactical This article reports on recent progress in the development of TacAir-Soar, an intelligent automated agent for tactical air simulation. This includes progress in expanding the agent’s coverage of the tactical air domain, progress in enhancing the quality of the agent’s behavior, and progress in building an infrastructure for research and development in this area. knowledge about two classes of one-versusone (1-v-1) Beyond Visual Range (BVR) tactical air scenarios. In the non-jinking bogey scenarios, one plane (the non-jinking bogey) is unarmed and maintains a straight-and-level flight path. The other plane is armed with long-range radar-guided, medium-range radar-guided, and short-range infrared-guided missiles. Its task is to set up for a sequence of missile shots, at

1994_4_teamcore_tacairsoar.pdf
M. Tambe and P. S. Rosenbloom. 1994. “Investigating production system representations for non-combinatorial match.” Artificial Intelligence (AIJ), 68, 1, Pp. 155-199.Abstract

Eliminating combinatorics from the match in production systems (or rule-based systems) is important for expert systems, real-time performance, machine learning (particularly with respect to the utility issue), parallel implementations and cognitive modeling. In [74], the unique-attribute representation was introduced to eliminate combinatorics from the match. However, in so doing, unique-attributes engender a sufficiently negative set of trade-offs, so that investigating whether there are alternative representations that yield better trade-offs becomes of critical importance.

This article identifies two promising spaces of such alternatives, and explores a number of the alternatives within these spaces. The first space is generated from local syntactic restrictions on working memory. Within this space, unique-attributes is shown to be the best alternative possible. The second space comes from restrictions on the search performed during the match of individual productions (match-search). In particular, this space is derived from the combination of a new, more relaxed, match formulation (instantiationless match) and a set of restrictions derived from the constraint-satisfaction literature. Within this space, new alternatives are found that outperform unique-attributes in some, but not yet all, domains.

1994_3_teamcore_aij94.pdf
1993
A. Acharya and Milind Tambe. 1993. “ Collection oriented match.” In Conference on Information and Knowledge Management .Abstract
Match algorithms capable of handling large amounts of dat% without giving up expressiveness are a key requirement for successful integration of relational database systems and powerful rule-based systems. Algorithms that have been used for database rule systems have usually been unable to support large and complex rule sets, while the algorithms that have been used for rule-based expert systems do not scale welt with data. Furthermore% these algorithms do not ~ovide support for collection (or set) oriented production languages. This paper ~oposes a basic shift in the nature of match algorithms: from tuple-oriented to collectwn-oriented. A collection-oriented match algorithm matches each condition in a production with a collection of tuples and generatea collection-oriented instarrtiutwns, i.e., instantiation that have collection of tuples corresponding to each condition. This approach shows great promise for efllciently matching expressive productions against large amounts of data. In addition, it provides direct support for collection-oriented production languages. We have found that many existing tuple-oriented match algorithms can be easily rmnsfonned to their collection-oriented analogues. This paper presents the transformation of Rete to Collection Rete as an example and compares the two based on a set of benchmarks. Results presented in this paper show tha~ for large amounts of data, a relatively underoptitnized implementation of Collection Rete achieves orders of magnitude improvement in time and space over an optimized version of Rete. The results establish the feasibility of collection-oriented match for integrated database-production systems.
1993_1_teamcore_cikm_final.pdf

Pages