DTU Informatics

ERMI Summer School 2015 at DTU

Main Campus, Kongens Lyngby, Denmark

Empirical Research Methods in Informatics

Today, more and more papers get published that contain an element of empirical research methods in them.
In fact, empirical methods are so prominent now, that in the call for papers for the VL/HCC conference, we are reminded that there are also other ways to provide evidence for claims besides human subject studies.
However, empirical research is not yet widely taught to our graduate students.
Therefore, the course „Empirical Research Methods in Informatics“ was introduced at the Technical University of Denmark (DTU), which is the basis for this tutorial.


The course focuses on the practical aspects of empirical research, that is, there will be only short introductory lectures on the various topics. After that, participants shall work hands-on with real case studies, some of which derive from my own work. That also means that, while there are no formally prerequisites to the course, understanding Software Engineering as a discipline will greatly improve your understanding of the course material.

For instance, participants will

  • study existing experimental set-ups with a view to finding shortcomings and threats to validity and how they impact the claims put forward in a given article;
  • develop a small experimental set-up for a given case study in small groups and present their design in a plenary discussion; and
  • define and refine a research question, and discuss alternative approaches to providing evidence for or against it.

Of course, there is more to empirical research than what can be taught in a tutorial. Thus, the objective of this course is only to provide its attendees with a starting point, possibly removing any inhibitions that might be there, and equip aspiring researches with some skills and first practical exercises in leading methodologically sound research.

After attending this course, participants

  • will be aware of the potential and limitations of empirical research methods;
  • are capable of choosing an appropriate research paradigm for a given problem; and
  • can assess the quality of the empirical research reported in an article, such as for a review.

Given the time restriction, we will focus mostly on controlled experiments and generic study design as far as the practical exercises are concerned. Other paradigms will only be covered by lecture-format introductions.

Intended Audience

The course is intended for people that have little or no background in empirical research methods, in particular people with a technical or formal methods background. We will assume a good understanding of current research topics in Software Engineering for the case studies. So, the ideal participant would be someone who has just started his or her PhD in Computer Science / Software Engineering (or will start it soon), and requires overview knowledge about empirical research methods to inform his or her research project.



The course generally follows a schema where there is some introductory lecture in the morning (typically, 9:00-10:00 or 10:30), then a practical exercises in groups until Lunch. After Lunch, feedback on the results and a follow-up exercise last till 16:00, when we gather together to look at, and critique, each others work, give and receive feedback, and answer any left-over questions that have accumulated over the day. Breaks are taken at your own discretion. I will wander around to discuss with the groups to help them get on with their work. On the last day, we expand on one of the exercises from the first day (study design), but this time, we prepare proper presentations and devote greater space to presentation, discussion, and feedback.

Monday Tuesday Wednesday Thursday Friday
Philosophical Perspective
Controlled Experiments
Threats to Validity
Field Work
Qualitative Methods:
Secondary Research
Mapping Studies
Study Design
Group Project on own research topics
Observations and Measurements Physiological Measurements
Statistical Techniques
Observational Techniques
Grounded Theory
Structured Literature Reviews
Tertiary Studies
Project Presentations
Wrap-Up and Feedback
Evening - Social Event - - -
Breaks will taken as needed. Plenary feedback is given after each session, individual/group feedback is given continuously.

A more detailed program will be announced in the course. Number of attendants permitting, we will try to tailor it somewhat to the audience. After the great success last years, we will even have a little social program, too: either an outing to the nearby deer park and a social dinner in a hunting lodge, or a little sailing cruise to an island in the Oresund. Either should be fun!

Teaching material

The course is not based on a single textbook, but uses excerpts from different books, and some research articles. All participants will receive a paper copy of the lecture material. PDF files will be available on request.

Practical Details

Place and Date

The ERMI Summer School 2015 will be held 17.-21.8.2015 on the main campus of DTU in Kongens Lyngby (see campus map). More detailed information about the venue and how to get there will be announced in due course. We will be either in building 101 or in building 324, depending on the booking situation. More details will be distributed in good time. It is a good idea to come a little early on the first day for socialising.

Sign Up

If you are interested in taking this course, please get in contact with Harald Störrle. Start by sending me an email with your name, affiliation, status/program. It might also be a good idea if you let me know your expectations, and level of expertise.

If we both agree that it is a good idea you take this course, register with DTU's central administration.

  • If you are a PhD student from DTU, this means following the usual path (i.e., sending an email to Pia Christoffersen, or the PhD-Administration if she is unavailable) including your s-number/email, department, and the course number.
  • If you are not a DTU student and register in good time, go to DTU's Signup Page to participate. Mark the field "signed up with teacher". Attendance is free of charge, but you will need to take care of travel and accommodation yourself.
  • If you are not a DTU student and register late, just send me an email. If I ok it, make the wire transfer, and you're in.
Attendance is limited to twenty (20), preference will be given to PhD students from DTU, but first come, first serve! The crucial date will be when we receive your fee (details see below).


Since the number of participants is ever growing, we have to ask for a small contribution of 100 Euro per participant this year. Please pay by bank transfer according to the following instructions.

Travel arrangements and Accommodation

The course includes Lunches, and refreshments during the breaks (coffee etc.). The social dinner is also included, but all other meals and accommodation is not.

If you are coming from outside DTU, and possibly from outside Denmark, we might be able to provide budget lodging on Campus - but you have to let us know before the end of April. Another option is the newly opened budget hotel Wake Up Copenhagen, which seems to be a decent (if remote) place, and offers very reasonable rates, for Danish standards. Be warned, that Denmark is an expensive place to stay! Here is a list of a few affordable places to stay. We do not have on-campus accommodation available on short notice.

last update: June 8th, 2015; HSTO