call for papers, previous message

From:     David Hart 
Subject:  CFP: AIJ Special Issue Devoted to Empirical AI
Date:     Mon, 25 Jul 94 16:22:31 -0500

Reply-to: dhart@cs.umass.edu



                             Call for Papers

        Special Issue of the Artificial Intelligence Journal 
           Devoted to Empirical Artificial Intelligence

              Editors:  Paul Cohen and Bruce Porter

We are looking for papers that characterize and explain the
behaviors of systems in task environments.  Papers should report
results of studies of AI systems, or new techniques for studying
systems.  The studies should be empirical, by which we mean "based
on observation" (not exclusively "experimental," and certainly not
exclusively statistical hypothesis testing).  Examples (some of which
are already in the AI literature) include:

    A report of performance comparisons of message-understanding 
      systems, explaining why some systems perform better than
      others in some task environments

    A study of commonly-used benchmarks or test sets, explaining why 
      a simple algorithm performs well on many of them

    A study of the empirical time and space complexity of an 
      important algorithm or sample of algorithms

    Results of corpus-based machine-translation projects

    A paper that introduces a feature of a task that suggests why 
      some task instances are easy and others difficult, and tests 
      this claim

    Theoretical explanations (with appropriate empirical backing) 
      of unexpected empirical results, such as constant-time 
      performance on the million-queens problem 

    A statistical procedure for comparing performance profiles 
      such as learning curves

    A resampling method for confidence intervals for statistics 
      computed from censored data (e.g., due to cutoffs on run times)

    A paper that postulates (on empirical or theoretical grounds) 
      an equivalence class of systems that appeared superficially 
      different, providing empirical evidence that, on some 
      important measures, members of the class are more similar 
      to each other than they are to nonmembers. 

The empirical orientation will not preclude theoretical articles; it
is often difficult to explain and generalize results without a
theoretical framework.  However, the overriding criterion for papers
will be whether they attempt to characterize, compare, predict,
explain and generalize what we observe when we run AI systems.

This is an atypical special issue because many of us think there is
nothing special about empirical AI.  It isn't a subfield or a
particular topic, but rather a methodology that applies to many
subfields and topics.  We are concerned, however, that despite the
scope of empirical AI, it might be underrepresented in the pages of
the Artificial Intelligence Journal.  This special issue is an
experiment to find out: if the number of submitted, publishable papers
is high, then we may conclude that the Journal could publish a higher
proportion of such papers in the future, and this issue might be
inaugural rather than special. 

Three principles will guide reviewers: Papers should be interesting,
they should be convincing, and in most cases they should pose a
question or make a claim.  A paper might be unassailable from a
methodological standpoint, but if it is an unmotivated empirical
exercise (e.g., "I wonder, for no particular reason, which of these
two algorithms is faster"), it won't be accepted.  In the other
corner, we can envision fascinating papers devoid of convincing
evidence.  Different interpretations of "convincing" are appropriate
at different stages of projects and for different kinds of projects;
for example, the standards for hypothesis testing are stricter than
those for exploratory studies, and the standards for new empirical
methods are of a different kind, pertaining to power and validity.
If, however, the focus of a paper is a claim, then convincing evidence
must be provided.

                      Deadline: Jan. 10, 1995.  

Please contact either of the editors as soon as possible to tell us
whether you intend to submit a paper, and include a few lines
describing the paper, so we can gauge the level of interest and the
sorts of work we'll be receiving.

Request:  Due to the broad nature of this call, it will be difficult
to reach all potential contributors.  So, please tell a friend...



            The Editorial Board for this issue includes:

B. Chandrasekaran, Eugene Charniak, Mark Drummond, John Fox, Steve
Hanks, Lynette Hirschman, Adele Howe, Rob Holte, Steve Minton, Jack
Mostow, Martha Pollack, Ross Quinlan, David Waltz, Charles Weems.


***

Dave Hart
UMass, Amherst
dhart@cs.umass.edu