evaluation

Evaluation.md

@@ -0,0 +1,25 @@
+# Evaluation Concept
+
+Approximately 15-25 participants are expected for the evaluation. The evaluation is split in two phases:
+
+## Phase 1 - Learning
+
+In the first phase, the participants play every puzzle once. This phase helps the engine get a basic understanding of the difficulty of the puzzles. Each room will contain only a single puzzle. This way we can accurately tell how long one puzzle took to solve.
+
+This phase is done with approximately 8-10 persons.
+
+## Phase 2 - Target Time
+
+In the second phase, the engine is given a target time of 15 minutes and tries to generate rooms that long. The puzzles here are still split up in single rooms to allow the engine to further improve its time estimates.
+
+This phase is done with the remaining participants.
+
+## Evaluation Survey
+
+After the evaluation, each participant will also fill out a short survey with which we hope to gather feedback about the following topics:
+
+* How enjoyable the experience was
+* The individual puzzles
+* Puzzle type preferences
+* The room generation
+* Ideas for improvement