Benchmark driver for incremental modular experiments

This is a driver for performing tests of intermod driver.

The test performed consists of analyzing different states of the sets of modules of a give directory. This is done by adding a number of random clauses each iteration of the test and performing the analysis afterwards.

Results of the analysis are left in the directory from which the test was runned.

Important!: Each time a benchmark is going to be rerunned the previous analysis directory has to be removed.

This module is thought to be used as an executable:

 ./exec <bench_id> <domain> [Opts]

The seed is used to be able to repeat the same 'random' sequence in several tests.

For example:

./exec qsort gr --edit_type add --n_edits 5 incremental modular --rand 1

will perform a test of adding 5 random clauses in the files of benchmark qsort with an incremental modular analysis.

The benchmarks that can be currently performed are present in test_dirs.pl.

Options available

config_opt(incremental).
config_opt(modular).
config_opt(bottom_up).
config_opt(bottom_up_cls).
config_opt(top_down).
config_opt(monolithic).
config_opt(plai).
config_opt(dd).
config_opt(di).
config_opt(no_pers).
config_opt(basic).
config_opt(trace).
config_opt(fact_info).
config_opt(under_all).
config_opt(over_all).
config_opt(assertions).
config_opt(verbose).
config_opt(debug_cls).
config_opt('--show-gat').
config_opt('--show-lat').
config_opt('--show-cls').
config_opt('--show-raw-output').
config_opt('--config-exp').
config_opt('--config-bndl').
config_opt('--debug-steps').
config_opt('--start').
config_opt('--steps').
config_opt('--seq_sz').
config_opt('--user_tag').
config_opt('--edit_type').

Template for bundle configuration

% Experiment analysis configuration for bundles.

% all paths are relative to the bundle location.

priority_modules([]).  % prioritized *module names* in that order. Typically the
                       % entry point should be prioritized

% If missing, taken from test_dirs.pl
%% entry_module('cmds/main1.pl').
%% entry_module('cmds/main2.pl').

bundle_location('.'). % relative to ciao-devel/bndls

edit_dir(lib).    % directory of the sources to be tested
edit_dir(cmds).   % the dir that contains the entry point modules needs to be specified
% TODO: read src_dir from manifest

% bundle_op(Prio,Pos,Op). % bundle-specific operator (for naive reader)
% by default assertions, fsyntax, dcg, and regtypes package operators are included

Template for experiment configuration

% Experiment configuration

% paths are relative to the location of this file, a subdirectory will be
% created in this directory
results_dir('.').      % (statistics, intermediate analyses, ...)

edition_mode(predicate).
% add/delete a full clause or predicate (not mixed for now)

% TODO: not implemented using absolute number of clauses or % of the program

module_permanence(0.5). % probability of staying in the same module after a change
predicate_permanence(0.5). % probability of editing the same predicate after a change

% probability of performing an addition/deletition action in a step
addition_probability(0.6). % must be greater than deletion if we want to finish the bundle
deletion_probability(0.6). % must be greater than addition if we want to remove the bundle

% TODO: code mutations are not considered here (change atoms or reorder literals)