Because the artifacts are automatically packed when you download the benchmarks, it might take some time for the download dialog to appear.
| DrJava_ICSE |
| | | Author: | Zebao Gao, Yalan Liang | | Last Updated: | November 3, 2016, 22:08:28, CDT | | Language: | Java | | Size: | 305 Events, 92813 LOC | | Versions: | 20130901-r5756 |
|
| | DrJava is a lightweight development environment for writing Java programs. (http://www.drjava.org).
This benchmark is based on DrJava version 20130901-r5756, the lastest version as of Sep 1st, 2013.
We downloaded this version of DrJava, obtained its GUI structured using the GUIRipper, generated its Event-Flow Graph (EFG), and used the EFG
to generate all possible test cases of length 2. We executed all the test cases on an instrumented version of Rachota and randomly pick 200 executable test cases for the further evaluation. For these 200 test cases, we collected code coverage reports (xml and html), GUI state files (called oracle files), and test logs. All these artifacts are bundled in this benchmark. |
| Application |
| | Downloaded from: https://sourceforge.net/projects/drjava/files/1.%20DrJava%20Stable%20Releases/drjava-stable-20130901-r5756/ |
| | Root directory: . |
| | Source code: drjava_src_class/src |
| | Binary: drjava_src_class/class |
| | Instrumented binary: drjava_src_class/inst_class |
| Models |
| | Event-flow graph (EFG): output_ubuntu/DrJava-Guitar.EFG |
| | GUI structure: output_ubuntu/DrJava-Guitar.GUI |
| Environment |
| | Operating system: Ubuntu 12.04Mac OSX 10.8Redhat Scientific Linux 6.4 |
| | JDK (with JRE): Oracle JDK 1.6Oracle JDK 1.7Open JDK 1.6 |
| Configuration |
| | Best: means the best configuration (our gold standard for the experiments). For this we use the same configuration setup and use the same initial input files for the applications so that its starting state is the same. We also control the time (when possible) and fix the Java version to Oracle 6. To obtain the best configuration, we first tried to control as many factors as possible, and heuristically selected the best delay value for each different platform (where best shows the smallest variation based on a visual inspection of a sample of the test cases). We then fixed this configuration as our best configuration and created variants of these for study. |
| | Unctrl: means uncontrolled. This is expected to be our worst configuration. We do not control any of the factors mentioned. We just run our test cases with the default tool delay value (0ms), and do not reset the starting configuration files or provide a fixed input file. We use a random date (on the two platforms where we can control this) |
| | No init: Starting from our best configuration, we removed the initial configuration/starting state files. |
| | D-0ms: Starting from our best configuration, we varied the step delay value to 0ms. |
| | D-50ms: Starting from our best configuration, we varied the step delay value to 50ms. |
| | D-100ms: Starting from our best configuration, we varied the step delay value to 100ms. |
| | D-200ms: Starting from our best configuration, we varied the step delay value to 200ms. |
| | Opn-6: Starting from our best configuration, we use the open JDK 6 instead of Oracle JDK 6. |
| | Orc-7: Starting from our best configuration, we use the Oracle JDK 7 instead of Oracle JDK 6. |
| Scripts |
| | Best: scripts/replay_report_drjava_best.sh |
| | Unctrl: scripts/replay_report_drjava_unctrl.sh |
| | No init: scripts/replay_report_drjava_no_init.sh |
| | D-0ms: scripts/replay_report_drjava_delay_0ms.sh |
| | D-50ms: scripts/replay_report_drjava_delay_50ms.sh |
| | D-100ms: scripts/replay_report_drjava_delay_100ms.sh |
| | D-200ms: scripts/replay_report_drjava_delay_200ms.sh |
| Tools |
| | Guitar and Cobertura: ../GuitarAndCobertura |
| Test Suites |
| | test suite: output_ubuntu/testcases |
| Experiment data |
| | Best: statistic_results/data_Best.pdf |
| | Unctrl: statistic_results/data_Unctrl.pdf |
| | No init: statistic_results/data_No-Init.pdf |
| | D-0ms: statistic_results/data_D-0ms.pdf |
| | D-50ms: statistic_results/data_D-50ms.pdf |
| | D-100ms: statistic_results/data_D-100ms.pdf |
| | D-200ms: statistic_results/data_D-200ms.pdf |
| | Open JDK 6: statistic_results/data_Opn6.pdf |
| | Oracle JDK 7: statistic_results/data_Orc7.pdf |
| BibTeX for Reference |
| | BibTeX file: bibtex/ICSE.2015.bib |
